Look what I found in KML while looking for hyperlinks: a RESTful service description format. A neat find because we need more working examples of service descriptions.

KML's "NetworkLink" isn't the most well-named element. It's not so much of a link, really, but a slot for feature data that is filled in during a user session. The data is provided by a read-only web service described by the content of the network link's "Link" element. There are 3 service flavors. A link with nothing other than a "href" element tells the client to fetch the resource identified by link[@href]. A link with an "onStop" view refresh mode tells the client to append a bounding box parameter to the service's resource identifier. Lastly, a link may give the client a URI template of sorts via the "viewFormat" element.

This is the shape of RESTful service description. The server tells the client how to make requests for feature data in the context of a particular KML document. It might use a slightly or radically different description in a different context. It might evolve the description over time within the same KML context to meet changing client needs or engineering constraints.

KML was on the cutting edge here, but has diverged from the web. Using standard links with the same semantics of those in HTML and Atom instead of an "href" would be good. Standardizing on URI templates might be a nice feature for the future, as would better separation of concerns in "Link". Specification of on what event and how often the "network link" is refreshed is mixed up with the service description. Some of the elements seem to me to belong in the network link rather than the link.

I often gripe about the KML format ("Snippet"!), but there's much good in it – the RESTful service description pattern in particular.

Update (2009-05-13): all aboard the snippet train.

I stumbled onto Solving GIS Problems today. It has an ESRI bent, but it's Python, and there's code – a rare thing in the geoblogosphere. And we agree on a bunch of things such as how logging wins and arcgisscripting cursors fail. Subscribed.

Jeff Harrison tries to flip the alchemist label back onto the web wonks:

Maybe what we should be thinking about is how to advance GeoWeb platforms NOT as proprietary implementations using an alchemy of REST, JSON, RSS, KML - but as Community GeoWeb Platforms that combine international standard geospatial web services with Web 2.0 technologies.

Nice try, but no. We agree about the proprietary flavor of some of the "GeoWebs" out there. Google, in particular, promotes a "GeoWeb" that is dominated by its proprietary browser and its proprietary search. Our role in this "GeoWeb" is to make KML files and expose them through sitemaps, increasing the value of Google's proprietary index in exchange for some return traffic through Google's proprietary browser. Sharecropping, in other words. Still, the OGC's SOA isn't appropriate for a "GeoWeb". There's no web there. Principles of web architecture, standard protocols like HTTP/1.1, and standard formats like JSON, Atom – and even KML – remain the way to geographically enrich the existing web.

The Generic Geometry Library (via Mateusz Loskot) looks interesting. Algorithms operating on coordinate arrays. Optional geometry classes. Shapely's new plugin framework was motivated by my interest in Python (or Java or .NET) implementations, but a plugin using a C or Cython interface to ggl could be neat.

Uploaded to http://pypi.python.org/pypi/Shapely/1.0.12. This release fixes a reference counting bug that caused your Python process to grow when executing code like:

for p in polygons:
    u = u.union(p)

As Randy George finds out, the notion of a "GeoRSS format" is confusing:

There seems to be some confusion about GeoRSS mime type - application/xml, or text/xml, or application/rss+xml, or even application/georss+xml show up in a brief google search? In the end I used a Virtual Earth api viewer to consume the GeoRSS results, which isn’t exactly known for caring about header content anyway. I worked for awhile trying to get the GeoRSS acceptable to OpenLayers.Layer.GeoRSS but never succeeded. It easily accepted static .xml end points, but I never was able to get a dynamic servlet endpoint to work. I probably didn’t find the correct mime type.

There is no GeoRSS media type. Atom's is application/atom+xml. RSS 2.0 uses application/rss+xml, and RSS 1.0 might use application/rdf+xml. You should use one of those depending on whether your GeoRSS elements appear in an Atom, RSS 2.0, or RSS 1.0 feed. Why? Atom has its own processing model. When you request a feed, the server writes application/atom+xml in the response's content-type header to inform you that you'd better use the Atom content processing rules if you want to make sense of the data.

Is there anything to be gained by having a special GeoRSS media type that overrode the Atom or RSS media types?

Yesterday, I saw a lot of links to David Chappell's ESRI Developers Summit keynote. I tried and failed to stick with the video, but I did read through the slides. They're largely good, but there are some errors. Before his presentation becomes canon in the ESRI user community, I'd like folks to consider:

The watering down of REST constraints on slide 14 into optional principles that you apply or not "whenever possible", and disregard for the hypertext constraint. In fact, there's no mention of the hypertext constraint at all in the slides. Does Chappell not understand it? Not believe in it? It's impossible to say from the slides; maybe he went into it more, live. At any rate, substitute "when you want certain derived properties" for "whenever possible" and you get closer to the essence of REST. If you want the property of cacheability, then you add the uniform interface and stateless communication constraints. If you want the property of loose coupling, you add the hypertext constraint.

On slide 18, Chappell fails to mention an option that's familiar to all of us even if we're not entirely aware of it: code on demand. The client library provided by the service can be downloaded by clients at the time the service is accessed. On demand. You're probably already using OpenLayers (or something like it) in a way that's close to an on-demand library for negotiating with a service. Chappell's second option on that slide, while useful enough, has very little to do with REST.

On slide 20, Chappell confuses REST with the half-baked "REST APIs" that don't use service descriptions or any sort of hypertext representations. If you stick to standardized representation formats, there's no more dependence on written documentation than in the SOAP case. Developers of SOAP tools had to read a pile of specs. Developers of RESTful HTTP tools will have to read a pile of specs too.

Finally, there's a lot of slides spent trying to explain how to choose between SOAP and REST. Ultimately, it comes down to this: if you want your system to have the properties that derive from REST constraints, you choose the REST style. If you want to integrate in the Web, you choose REST. If you want a system that tries to abstract away the network, letting your developers program with remote services as though they were local objects, you choose SOAP. Pick your architectural style. I don't think it has to be any more complicated than that.

Update (2009-04-09): Pete (below) has shamed me into watching the video all the way through. I'm glad I did. I knew this stuff already, but it's a very good presentation for the ESRI user/developer. Unfortunately, despite Pete's claims, my quibbles with his slides stand. Chappell didn't cover the hypertext constraint (though he did an excellent overview of the wins from uniform interface and universal identifiers). An adequate treatment would have consumed too much time, granted, but some mention would be nice. The hypertext constraint has indeed been a matter of debate, but the wins are becoming clear. Without links in content, well-defined formats, and code on demand, developers are indeed dependent on service provider libraries, but this state of affairs isn't a fault of the REST style. Still, to use SOAP in your enterprise behind the firewall, to use the REST style outside on the Web is the obvious conclusion. Good answers to questions, too, including comments about AtomPub and the issue of snowflake APIs.

Keytree provides some utilities for manipulating KML using the ElementTree API. I've added factories for KML placemark and geometry elements, input being geojson or Shapely objects:

>>> from geojson import Feature
>>> f = Feature('1',
...             geometry={
...                 'type': 'Point',
...                 'coordinates': (-122.364383, 37.824663999999999)
...                 },
...             title='Feature 1',
...             summary='The first feature',
...             content='Blah, blah, blah.'
...             )

Any object that provides the Python geo-feature interface will do. Next, you need a KML document as context for a new placemark:

>>> data = """
... <kml xmlns="http://www.opengis.net/kml/2.2">
...   <Document>
...   </Document>
... </kml>
... """
>>> from xml.etree import ElementTree
>>> kml = ElementTree.fromstring(data)

Make a placemark element using the element factory:

>>> elem = keytree.element(kml, f)
>>> import pprint
>>> pprint.pprint((elem.tag, elem.text, list(elem)))
('{http://www.opengis.net/kml/2.2}Placemark',
 None,
 [<Element {http://www.opengis.net/kml/2.2}name at ...>,
  <Element {http://www.opengis.net/kml/2.2}Snippet at ...>,
  <Element {http://www.opengis.net/kml/2.2}description at ...>,
  <Element {http://www.opengis.net/kml/2.2}Point at ...>])
>>> pprint.pprint(list((e.tag, e.text, list(e)) for e in elem))
[('{http://www.opengis.net/kml/2.2}name', 'Feature 1', []),
 ('{http://www.opengis.net/kml/2.2}Snippet', 'The first feature', []),
 ('{http://www.opengis.net/kml/2.2}description', 'Blah, blah, blah.', []),
 ('{http://www.opengis.net/kml/2.2}Point',
  None,
  [<{http://www.opengis.net/kml/2.2}Element coordinates at ...>])]

This element could be appended to the Document element, or you could use the subelement factory:

>>> elem = keytree.subelement(kml[0], f)

More at http://pypi.python.org/pypi/keytree/.

Now, I'm trying to decide if something similar would be useful for Atom with GeoRSS.

Another good question from the OGC REST discussion:

In essence, within REST, how do I ask for a collection of measurements between any time1 and time2? This is simple with SOA, but seems to require predefined granularization by the service in ROA or perhaps uses an adaptation of SOA-like specifications as Pat has suggested in the past.

SOA didn't say how to spatially or temporally slice data. SOA said "have services". Services with well-defined interfaces. It's up to communities to define those interfaces. It's the same for RESTful architectures. REST just says use universal identifiers for entities, use uniform methods, and use hypertext as the engine of application state. The entities and the hypertext formats are up to the community.

How would you ask a sensible observation service for measurements between two times? This device is blogging away, adding observations to the head of a feed every N microseconds. If you're interested in historical data – and everything becomes historical quickly if it's sampling very frequently – the service should allow you to slice temporally, returning feeds that begin and end at or near specified times. The Atom format spec doesn't specify how to do this. It was developed in the tradition of layering specifications and recommendations. Search is to be layered on top of Atom like feed paging (RFC 5005) is layered on top of Atom. One possible way to do this is with XHTML forms.

Consider a sensible observation service identified by the URI http://example.com/sosa. Dereferencing that gets you an Atom feed of N entries, ideally containing links to the next "page" and a link to a search form:

<feed xmlns="http://www.w3.org/2005/Atom">
  <link rel="search"
    type="application/xhtml+xml"
    href="http://example.com/sosa/search"
    />
    ...
  </feed>

The "search" relation isn't standardized yet, but something like it will be. The exact URI of the search form isn't important at all, only that it can be found via a link. Its representation would be something like this:

<html xmlns="http://www.w3.org/1999/xhtml"
  xmlns:sos-search="http://example.com/namespaces/sos-search"
  >
  <body>
    <form id="sos-search:form"
      method="GET"
      action="http://example.com/sosa/search"
      >
      <input id="sos-search:start" type="text" name="start"/>
      <label for="sos-search:start">Minimum bounding timestamp, RFC 3339</label>
      <input id="sos-search:stop" type="text" name="stop"/>
      <label for="sos-search:stop">Maximum bounding timestamp, RFC 3339</label>
      <input type="submit" name="search"/>
    </form>
  </body>
</html>

The "start" and "stop" inputs are borrowed from Python's sequence slicing syntax. A community that standardized on form element ids could have a self-describing search interface, one that was also helpful to human users (at least) outside that community. Any "sos-search" aware client could recognize from this form, by parsing elements with "sos-search" ids, that URIs like http://example.com/sosa/search?start=2003-12-13T18:30:02Z&stop=2003-12-13T19:30:02Z identify Atom feeds of temporally sliced data.

I'm not sure XHTML forms are the best solution, but it's illustrative of one way to do RESTful search interfaces.

How to RESTfully change the state of power lines and poles? From a thread on the OGC's mass-market-geo list (archives available only to subscribers, for shame):

Actually, I am really talking about poles and wires. Specifically power poles and lines. My example is directly derived from real life. A local electric system has tons of old wood poles that it wants to replace with concrete poles. It is doing this by installing the concrete pole 1m behind the existing pole and then restringing the power lines.

So, the client has a feature type POLES (Oracle table) with all the poles and a feature type LINES (Oracle table) with all the power lines. Not the actual named but you get the idea.

For each pole that is replaced, they update the existing pole record since the new pole will have the same pole id as the pole it replaces but the new pole will be in a slight different position.

Using the current WFS specification, I can make these changes in a single request. I simply POST the following the the URL of the server:

<wfs:Transaction>...

The question of how to do this in a single transaction is a big juicy red herring that has list subscribers rather distracted. The process of disconnecting and taking down wires takes some time, and presumably has to be done before you can take down the pole they connect to. Certainly the new pole can't be erected before the old one is torn down, and only after that happens can new lines be brought in to connect to the new pole. This is not an atomic operation. Bad weather, equipment failure, injury, or any mishap might interrupt the process and leave it an unfinished state for some time. Treating it as atomic in your information system seems more hopeful than realistic, and perhaps even harmful. Once the lines and poles come down, you can't be giving anyone the false impression that they're still up.

Let's say you've modeled your power system RESTfully. You've got pole resources like http://example.com/power/poles/1. You've line resources like http://example.com/power/lines/1 and http://example.com/power/lines/2. These resources aren't poles and lines by virtue of the strings in the URI, of course, that's just a convenient design. Lines 1 and 2 connect at pole 1. A utility crew keeps your information system up to date as they work like this:

• DELETE lines http://example.com/power/lines/1 and http://example.com/power/lines/2 as they are physically dismantled.
• DELETE pole http://example.com/power/poles/1 as it's cut down and dragged away.
• PUT new state (new location, new material attributes, etc) to http://example.com/power/poles/1 when the new pole is up.
• PUT new state (new lengths, etc) to lines http://example.com/power/lines/1 and http://example.com/power/lines/2 after they've been reconnected.

This isn't finance. Sometimes non-transactional is more honest.