Recent events, summarized by Tim O'Reilly here, have prompted me to spell out a policy for comments on this blog. I encourage people to write critically about our community, industry, and media, but I reserve the right to remove pointless flamebait or personal attacks as well as spam. If an email address accompanies extremely poor commentary, I'll contact the author personally about making sensible edits or retractions. If not, I'll simply delete it. Fair enough?

Update: changed the wording a bit so as not to point at anyone. The comments I get here provide much, if not all, of my blog's value. I won't delete non-spam comments lightly, but I have no obligation to give jerks a voice here.

Update (2007-04-10): a related demo.

Update: added examples of query results.

My previous post about WxS, RPC, and REST raised a few questions about whether queries fit into a RESTful GIS. The answer is: yes, queries remain indispensible. Indexes are a valuable part of your GIS, and a query API provides web agents access to the indexes.

Consider a very minimal municipal GIS which tracks properties, or parcels. Each parcel has many attributes, and the GIS indexes, at the very least, the following: a unique parcel id, the name of the parcel's owner, and the geospatial footprint of the parcel. These indexes allow a user to efficiently find all properties owned by an individual, or find all properties potentially impacted by construction along a particular path.

In a RESTful GIS, each parcel is a resource, and has a URL like:

http://example.org/parcels/[id]

Dereferencing the URL http://example.org/parcels/1 returns a representation, JSON in this case:

{ "id":       "1",
  "owner":    "Homer Simpson",
  "geometry": {
    "type":        "Polygon",
    "coordinates": [[...], ...] },
  ...
}

The "parcels" feature type is itself a resource. A useful representation of this resource would be a collection that includes URIs for, and data about, individual parcel resources -- all marshalled directly out of the GIS's indexes:

{ "parcels": [
  { "id": 1, "uri": "http://example.org/parcels/1",
    "owner": "Homer Simpson",
    "bbox": [1000.0, 1000.0, 1001.0, 1001.0] },
  { "id": 2, "uri": "http://example.org/parcels/2",
    "owner": "Ned Flanders",
    "bbox": [1001.0, 1001.0, 1002.0, 1002.0] },
  ...
  ]
}

That collection might easily include the precise footprints of properties, but we'll simply consider bounding boxes here.

A query API should return criteria-based subsets of that collection, leveraging the system's indexes. Which properties are going to be condemned to make way for the new monorail?:

GET /parcels/?bbox=0,0,2000,2000

The answer is: the parcels with URIs http://example.org/parcels/42 and http://example.org/parcels/83:

{ "parcels": [
  { "id": 42, "uri": "http://example.org/parcels/42",
    "owner": "Moe Szyslak",
    "bbox": [...] },
  { "id": 83, "uri": "http://example.org/parcels/83",
    "owner": "Kwik-E-Mart Corporation",
    "bbox": [...] }
  ]
}

Which properties are suffering catastrophic loss in value?:

GET /parcels/?adjacent(owner(Simpson))

The answer is: the parcel with URI http://example.org/parcels/2:

{ "parcels": [
  { "id": 2, "uri": "http://example.org/parcels/2",
    "owner": "Ned Flanders",
    "bbox": [...] }
  ]
}

The specific query parameters or URL templates to use are an implementation detail that I won't get into here (OpenSearch seems promising).

The gist of all this is that a RESTful feature query returns key, indexed data about feature resources along with a URI to the feature resources themselves in the same way that a Google Search returns data from its index, with links, instead of dumping the entire Web into your browser.

Earlier this month I made some hand-waving arguments for separating the concerns of MapServer's web application and cartographic engine on general principles. There is now a new MapServer development proposal that allows me to make a specific example.

It's been proposed that the MapServer web app could generate vanilla XML responses to GetFeature, GetFeatureInfo, and other WxS requests, and user-configured XSLT would provide a finishing touch to the responses. It's a nice idea. The early implementation plan was to load up existing configuration parameters with additional meaning. Stylesheet as metadata or something like that. Actually, specification of a XSLT stylesheet for transforming WxS responses has nothing to do with cartography, and nothing to do with map or layer metadata. It is solely a concern of mapserv, the web application. Therefore, let's configure it separately from the Map section of a mapfile. Transformation of a WFS GetFeature response might be done like this:

# The traditional cartographic config section
Map
  # All the usual map layers, etc.
  ...
End

# *NEW* application config section
Application
WFS
  GetFeature
    XSLT       On
    Stylesheet "/mapserver/xslt/example.xsl"
  End
End

That's declarative, orthogonal, and crystal clear.

I met some of the GeoDjango folks this weekend, and am looking forward to collaborating with them. The Python Cartographic Library is not just for Zope3, after all.

GeoRSS evangelism is part of what I do for Pleiades. The new support for GeoRSS in Google Maps is the spark that will set it off in the digital humanities. For example, check out this feed of Celtic coin finds from the British Museum's Celtic Coin Index:

http://www.finds.org.uk/CCI/functions/atom-export-all.php?denomination=Stater

Feed on the map

Previously, I asserted that the Python Cartographic Library feature API was superior to anything generated trivially from C++ code (even excellent C++) by SWIG. Of course, even PCL's API can be improved. I've been inspired by Django's database abstraction API to experiment with something even easier to use. Friday night I hacked on PCL's GeoRSS module, and tied up loose ends this afternoon. See branches/PCL-newfeatureapi/PCL-GeoRSS.

Feature sources are absent from the new API. A feature type class has a query manager attribute, features, and methods of this object provide iterators over features. For example, let's find items of a GeoRSS in a region of interest using a bounding box filter:

>>> url = 'http://pleiades.stoa.org/places/settlement.atom'
>>> store = GeoRSSFeatureStore(url)
>>> Entry = store.featuretype('entry')
>>> Entry.features.count
230

# Filter for intersection with a
# (29dE, 36dN, 30dE, 37dN) bounding box

>>> features = [f for f in Entry.features.filter(
...                        bbox=(29.0, 36.0, 30.0, 37.0)
...                        )
...            ]
>>> len(features)
62

# Inspect the first feature

>>> f = features[0]
>>> f.id
'http://pleiades.stoa.org/places/638749'
>>> f.properties.title
u'Antiphellos/Habesos'
>>> f.properties.the_geom.toWKT()
'POINT (29.6370000000000005 36.1931999999999974)'
>>> type(f.context)
<class 'feedparser.FeedParserDict'>

A GeoRSS feature's context is a reference to that item's parsed data structure. Everything feedparser can glean about the item (and that's nearly everything) is thereby available to a programmer.

Here's an example, using a better feed, of using a Python filter expression to obtain an iterator over only certain items tagged "web":

>>> url = 'http://sgillies.net/blog/feeds/entries/'
>>> store = GeoRSSFeatureStore(url)
>>> Entry = store.featuretype('entry')
>>> Entry.features.count
31

>>> features = [f for f in Entry.features.filter(
...                        properties="'web' in f.tags"
...                        )
...            ]
>>> len(features)
12

>>> f = features[0]
>>> f.id
'http://sgillies.net/blog/entries/412'
>>> f.properties.title
u'GeoRSS and Validation'
>>> f.properties.tags
[u'web']
>>> f.properties.the_geom.toWKT()
'POINT (-105.0958300000000065 40.5869900000000001)'

# More detail about the tags, via the feature context

>>> tag = f.context.tags[0]
>>> tag.term
u'web'
>>> tag.scheme
u'http://sgillies.net/blog/categories/'
>>> tag.label
u'Web'

That's dirt simple. Following the Django lead, creating and saving new features ought to be as straightforward as:

>>> new_georss_entry = Entry(title=u'GeoRSS Everywhere', ...)
>>> new_georss_entry.save()
>>> Entries.features.count
32

Sam Ruby finds GeoRSS and finds it a bit confusing. My blog entries feed validates perfectly, unlike the slashgeo feed Google references.

Update: Simon Willison finds GeoRSS too. When Google adopts anything, the effect is huge.