Sean Gillies Blog

The industry mainstream has now heard of REST, but not everyone gets it yet. Example: this Directions article.

When writing about REST, the most important things to communicate are the central tenets. Tim Bray explains them like this:

• You have a lot of things in the system, identified by URIs.
• The protocol (probably HTTP) only knows one MEP [message exchange pattern]: single-request, single-response, with the request directed at a URI.
• An important subset of the requests, what HTTP calls GET, are read-only, safe, idempotent.
• You expect to ship a lot of URIs around in the bodies of requests and responses, and use them in operations that feel like link following.

Andrews doesn't address the tenets and, instead, muddles around with "GET URLs" and "POST URLs" in an article that misinforms readers. Most of what's wrong in the article is summarized in the first sentance of the closing paragraph (emphasis mine):

Until then, I don't have much of an opinion about whether the industry should standardize on POST or GET request types. Both types of request use predictable, standardized formats that can be executed elegantly or poorly. The REST discussion reveals that the industry should consider that predictability is not the only goal of standards. Intuitiveness and accessibility may be equally important when establishing information exchange standards.

In reality, there's no such controversy. A RESTful GIS web service uses both POST and GET: the former to create new web resources, the latter to fetch representations of resources in a safe (no side-effects) way.

It is spreading. And spreading.

That's what Raj Singh seems to be saying here:

For example, most of the REST material I've read treats images as indivisible, binary resources. This won't work in the geo field, where images are in the gigabyte and terabyte range and you've got to design services that provide access to portions of those images. However, aside from imagery I don't see much of a problem with using a resource-oriented approach to information access. In fact, in the sensor web arena I think a REST approach makes a ton of sense.

Raj, you're reading the wrong material. No REST architect accepts that the image "file" is the only possible resource. If I had a global sea-surface temperature coverage resource at /sst/current/, I could easily treat individual pixels as resources of their own: /sst/current/{pixel},{line} is just one possible URL form. I could even interpolate to fractional positions within the coverage, or do lat/long: /sst/current/{latitude},{longitude}. Modes of an empirical orthogonal function decomposition (hot topic when I was in school) could be modeled as resources: /sst/current/eof/{n}. Simple temperature slices are another possible set of algorithmic resources: /sst/current/?tmin=28 would be the warm waters that spawn tropical storms. All of the above are RESTful and no client needs to know whether I have a single file or multiple databases backing the resources. Furthermore, a resource-oriented approach scales better than an endpoint-based protocol like WCS. As the number of resources grows, you simply spread them over more servers like Google has done with its Earth and Maps tiles.

There may be geospatial problems that REST can't tackle, but access to arbitrary regions of a coverage is not one of them.

I'm also a bit disappointed to read this:

After all this [research], I think I know what's going on, but I don't think there's any one clear explanation (despite some nice pieces of the puzzle here [1] and here [2]) available, and there has certainly been little effort to analyze the REST architecture in relation to geographic information systems theory, so that's what I'll try to do now.

Of all the great posts and email threads which break GIS web services down in RESTful terms, he picks that muddled Directions article and another that only quotes one of my emails? This trivializes the geospatial community's conversation about REST, one that has been enlightening and amusing us for about a year now. The fact that there will be three REST related presentations at FOSS4G hints at the maturity of the topic.

Charlie is on a roll. The difficulty with promoting REST is that we can't yet say "Products X, Y, and Z support it", or "Companies A, B, and C are standardizing on it, and so should you". The case has to be made for technical superiority (in many situations) of REST over the entrenched OGC architecture, and that means some jargon and details that seem esoteric to a lot of GIS practitioners. Charlie's latest are good posts that don't require continual reference to the HTTP/1.1 or WxS specs.

Here's something I've read a few times, including in my comments:

I'm still trying to get my head around the idea of specifying arbitrary BBOXes for image requests in REST format: seems utterly uncacheable, and thus against one of the REST goals.

If you seek to build a high performance, highly scalable map image or coverage service (such as the one Google needs to support Earth and Maps), exposing an infinite number of view resources is a design anti-pattern that you must avoid. (Note: a WMS doesn't expose a truly infinite number of resources, but a number N approximately equal to 10 to the power of 4W, the precision width in digits of the bounding box coordinates. N approaches infinity fairly quickly: MapServer exposes something like 10⁶⁶ unique views of a world map.)

If you want to cache effectively, you must switch your design over to a finite number of view resources. Tiles, in another word. There are other advantages to the REST architecture besides cacheability, so don't get the notion that REST fails for large values of resource number N.

A bunch of us are going to try to meet up on 9/5 to test the interoperability of our Geo-Atompub implementations and hash out the problems that crop up. Geometry representation is a certain issue: the multiple choices of GeoRSS are not good for services standardizing on Atom. We need to apply the Highlander Principle here.

Feel free to join us; it's not a closed-doors thing. See the wiki page for details.

In preparation, I've run Tim Bray's APE against my service and it is (mostly) conforming. It looks like someone was testing another client against it last night.

GData is AtomPub, more or less. The GML and GeoRSS namespaces appear in the examples. Have I ever mentioned that I think it's extremely interesting and significant that Google isn't using WFS-T as a gateway for geo-tagged Picasa items?

AtomPub again. Jason points out that OpenSocial is also using {http://www.georss.org/georss}where.

That's a great term. Bill de hÓra:

Ascribing REST to the SimpleDb API might be down to lack of knowledge or marketecture; hopefully they'll get the API* sorted out soon, before it gets used heavily.

The GIS industry leader is soon to release new REST stuff. We'll see whether it abuses the architectural style like Amazon does or if it's the real deal.

Update (2007-12-17): from Assaf Arkin:

1999 called, they want their CGI scripts back.

That's more or less how I feel about the OGC WxS service architecture.

Here's the take-away from the past year's discussion about REST and geospatial. My own take, that is. There are differences of opinion among the participants of the Geo-Web-REST group.

REST is an architectural style, not an API. When you hear or read "... REST API ...", that's a smell, a signal that something might not be quite right inside.

REST is the architectural style of the World Wide Web. "Web API" is a more meaningful term, although not all Web APIs are RESTful. Some are GETSful.

The OGC service architecture is not RESTful. The style of the OGC architecture is non-RESTful CGI/RPC. Feature, map, and coverage resources are kept hidden behind service endpoints. We're still partying like it's 1999.

GML is compatible with REST, but XML Schema is still a wretched beast and there are no XLink-traversing GeoWeb browsers of note.

Look to Google Earth and KML for REST inspiration. It's about the links. Also look to Google's AtomPub-ish APIs for YouTube, Picasa, and OpenSocial. There's no transactional WFS involved there, and that's something to think about. Google chose AtomPub for wider interoperability.

REST is not a silver bullet. Distributed applications are hard.

These are the tenets of REST:

• Give every “thing” an ID
• Link things together
• Use standard methods
• Resources with multiple representations
• Communicate statelessly

Do read and recommend Stefan Tilkov's Brief Introduction to REST, from which the tenets above are taken. It's excellent.

Disregard the Emerging Technology: Geospatial Web Services and REST article in Directions. I don't care who linked you to it; they probably didn't read it very closely. This sounds mean, but that article will make you dumber about REST.

Joe Gregorio's atompubbase looks promising. To try it out, I ran the apexer program against my Hammock site:

sean@lenny:~$ apexer service http://zcologia.com/hammock/index.atom
sean@lenny:~$ apexer lc
0   Places
sean@lenny:~$ apexer collection 0
sean@lenny:~$ apexer ls
0   Theater at Hierapolis
1   Springfield/Ninoe
2   Springs at Hierapolis
3   Big Hendy Grove
4   Little Hendy Grove
5   Navarro Vineyards
6   Wehlener Sonnenuhr
7   Pic St-Loup
8   Vineyards Domaine de l'Hortus
9   Vineyards Domaine de l'Hortus
sean@lenny:~$ apexer entry 8
sean@lenny:~$ apexer get
sean@lenny:~$ cat entry
<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom"
  xmlns:georss="http://www.georss.org/georss">
  <title>Vineyards Domaine de l'Hortus</title>
  <link rel="alternate" type="text/html"
    href="http://zcologia.com/hammock/places/10.html"/>
  <link rel="edit" type="application/atom+xml"
    href="http://zcologia.com/hammock/places/10"/>
  <link rel="edit-media" type="application/vnd.google-earth.kml+xml"
    href="http://zcologia.com/hammock/places/10.kml"/>
  <summary>
    Mourvedre, according to http://www.wineanorak.com/hortus.htm
  </summary>
  <author><name>anonymous</name></author>
  <published>2007-08-06T16:57:40-06:00</published>
  <updated>2007-08-06T16:57:40-06:00</updated>
  <georss:where>
    <gml:Point xmlns:gml="http://www.opengis.net/gml">
      <gml:pos>43.7896297407 3.83361116447</gml:pos>
    </gml:Point>
  </georss:where>
  <content/>
</entry>

I edited that entry and posted it back to the collection:

sean@lenny:~$ apexer create entry --content-type="application/atom+xml"
sean@lenny:~$ apexer ls --all
0   Theater at Hierapolis
1   Springfield/Ninoe
...
60   Duplicate Vineyards Domaine de l'Hortus

See the result. Publishing geodata can be as simple as that. I don't think AtomPub is going to take over the geospatial world in 2008 -- few of our architects have even heard of it yet let alone begun to dabble outside WxS -- but we'll see a few more high profile implementations. Google Earth as an AtomPub client maybe? The services (Picasa, YouTube, etc) are certainly there already.

An SOA governance platform based on AtomPub. Why not? Via James Snell.

It's great to see more people in the geospatial community thinking outside the SOAP box. Still, there's more to REST than HTTP and plain old XML (POX) or JSON. Fuzzy's service architecture, typified by requests like:

GET v1/ws_geo_getextent.php?geotable=places&srid=4269&parameters=id=10 HTTP/1.1
Host: webservices.example.com

is the venerable REST-RPC hybrid explained in Chapter 1 of RESTful Web Services. This architecture is okay for read-only services, but doesn't easily accommodate creation, update, or deletion of resources (it doesn't actually expose any resources at all), and sets you up for troubles like those of SimpleDB. Even if you dodge the pitfall of:

GET v1/ws_geo_deleterow?geotable=places&parameters=id=10 HTTP/1.1
Host: webservices.example.com

D'oh!

by properly using HTTP DELETE, you've lost uniform interface to your resources and no longer have a RESTful architecture.

Seems like a cool project. A little restraint in hyping REST is all I'm suggesting.

This week I finally got around to working on a task that's been haunting me: OpenLayers as an AtomPub client. Chris Schmidt has done much of the work already, but I'm going to see it all the way through over the next few months.

A quick review of AtomPub and how it relates to geospatial applications is probably needed here. There are 3 different types of resources in the Atom Publishing Protocol (omitting for now workspaces): services, collections, and entries. the analogous components in a GIS architecture are: services, featuretypes (or coverages), and features. An AtomPub service document is in some ways like an OWS capabilities document, but simpler because AtomPub doesn't fool around with non-HTTP semantics and bakes more metadata into its single protocol. An Atom feed document is rather a lot like a WFS feature collection document, and an Atom entry is also much like a GML feature, but much more standardized. OpenLayers does WFS already, and needs little modification to be a good AtomPub client.

Here's the little modification that I have done: a new OpenLayers.Format.Atom lets me ignore the jumble of namespaces and elements that is RSS 1 and 2. This Atom.js is simpler than GeoRSS.js. The Atom entry atom:content element is the prime avenue for extending Atom, and I've also added support for that. Best practice for georeferencing Atom entries is to use a GML simple features geometry (see OGC 06-049r1) within a georss:where element and OpenLayers.Format.GMLSF supports simple features GML within my new Atom format. The GMLSF format writes gml:pos and gml:posList only, no gml:coordinates, and uses "exterior" and "interior" as ring element tags. On the other hand, it is just as forgiving as OpenLayers.Format.GML when reading. So now an OpenLayers app can write well calibrated feature entries to be POSTed to an AtomPub-style collection like the one at http://zcologia.com/kw/++rest++knowhere/demo. (I'll explain the odd URL in a future post).

That collection is live. I've copied over a few entries from my Hammock app like so:

sean@lenny:~$ curl -X GET http://zcologia.com/hammock/places/6.atom > 6.atom
sean@lenny:~$ curl -X POST -v \
-H "Content-Type: application/atom+xml" \
-H "Slug: Navarro Vineyards" \
--data @6.atom \
http://zcologia.com/kw/++rest++knowhere/demo
* About to connect() to zcologia.com port 80
*   Trying ...... connected
* Connected to zcologia.com (...) port 80
> POST /kw/++rest++knowhere/demo HTTP/1.1
> User-Agent: curl/7.15.5 (i486-pc-linux-gnu) ...
> Host: zcologia.com
> Accept: */*
> Content-Type: application/atom+xml
> Slug: Wehlener Sonnenuhr
> Content-Length: 1301
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
HTTP/1.1 201 Created
< Date: Sun, 17 Feb 2008 20:46:21 GMT
< Server: Twisted/2.5.0 ...
< Content-Length: 73
< Accept-Ranges: bytes
< X-Content-Type-Warning: guessed from content
< Location: http://zcologia.com/kw/++rest++knowhere/demo/wehlener-sonnenuhr
< Content-Type: text/plain;charset=utf-8
Connection #0 to host zcologia.com left intact
* Closing connection #0
Location: http://zcologia.com/kw/++rest++knowhere/demo/wehlener-sonnenuhr

(The ability to copy resources is built into HTTP.) If you browse to http://zcologia.com/kw/demo/map (and wait a few seconds for a pile of javascript to download) you'll see this placemark displayed among others as a vector layer in an OpenLayers map of the demo collection. Toward implementing AtomPub, I have a second vector to serve as a buffer for to-be-posted placemarks. Draw a new geometry in the map using the OpenLayers tools and you'll see it in green. Then set a title, summary, text, a placemark name slug, and click "post". If you've picked a slug that collides with an existing placemark name, you'll see an error message. Otherwise, you'll see the location of the newly created placemark.

Still yet to do are placemark edit and deletion as Chris's AtomPub demo already does. I think that getting this right will require a new OpenLayers.Layer.AtomPub, one that behaves much like the existing WFS layer, but interacts with the server using GET/PUT/DELETE/POST. The protocol implementation, such as it is, currently exists in this page template. The only real novelty in it is the use of the HTTP Slug request header to make suggestions for resource names.

function postPlacemark() {
  var feature = buffer.features[0];
  feature.attributes.title = $("pm-title").value;
  feature.attributes.description = $("pm-summary").value;
  feature.attributes["content"] = $("pm-content").value;
  var atom = format.write(feature);
  var options = {
    method: "post",
    contentType: "application/atom+xml",
    postBody: atom,
    onSuccess: updatePage,
    onFailure: function(xhr) {
      update_status("Failed post (status code "+xhr.status+"). Check your URL.");
    },
  };
  var slug = $("pm-slug").value;
  if (slug.length > 0) {
    options.requestHeaders = ["Slug", slug];
  }
  new OpenLayers.Ajax.Request(collection_URL, options);
}

My server has the final say, and currently just lowercases the slugs and replaces whitespace with a dash.

I submitted the new Atom and GMLSF formats to OpenLayers (#1366) and look forward to working with the developers to get them into an upcoming release.

Umibot writes:

After some hard work/late nights and prodding from the geo-techno elite, we’ve humbly completed and are pleased to offer RESTful access to our neighborhood API.

In their "REST API", you can use either GET or POST (trampling the uniform interface) to invoke a remote method and get a JSON or XML response. It's better than SOAP, but I'm dismayed that this is hyped as REST.