Grading REST

Earlier this week, Raj Singh graded the OGC's services against REST and then graded the REST architectural style itself:

The fundamental problem with REST and GIS in my opinion, is that REST optimizes for data access, and OGC services are optimized for data processing. I consider GIS a type of OLAP system, and we as an industry will continue to resist REST because it would be counter-productive to expose data via atomic resources that all had their own URL (and metadata!?!) because everything we really want to do with the data would get harder.

In comments, Stu Charlton points out that REST is in fact optimized for data processing. Myself, I think it's simply in a different style, one that looks more goal oriented or agent oriented; I've linked to a past post by Charlton on hypermedia agents that has many good comments of its own.

Is it true to that the GIS industry resists REST? Some companies and consortia do, others don't. It seems like there's a press release about a new Geospatial REST API every other day. We've just seen a "REST API standard" document released by the 800 lb gorilla itself. A lot of this is just marketing, but there is also a growing, honest interest in resources instead of endpoints and HTTP as an application protocol.

I'm not sure grading OGC services benefits anybody, but if you absolutely have to grade your services against a REST maturity model it's probably best to use a standard model.

Related blogs and feeds

My announcement of the Shapely 1.2.4 release was the last such regular release announcement on this blog. Every Shapely, Rtree, spatialindex, Geojson, and OWSLib release will now be announced at [feed] in collaboration with other developers. I'll continue to write about especially notable ones here.

The same goes for my work on Pleiades: my news items will appear along with those of other users and administrators at [feed]. Some portions or summaries of them will be featured here from time to time.

Visualizing Pleiades imports

I've just completed the first round of data imports into Pleiades. We've passed the 25,000 milestone and are moving on to improving the spatial indexing and visualization of the places and locations.

I'm experimenting with using Polymaps to see what we've imported, clustering by the grid cells of our map directory sources, and appreciating the simplicity of the Polymaps approach. I can see using it in the small maps that accompany each Pleiades place. A good "web-scale" basemap for the ancient world would be more appropriate than the Flickr tiles, but I'm enjoying the boundary-free geography of the Flickr layer. Like Pleiades and the R-Tree server I'm writing, Polymaps speaks GeoJSON, the little format that could.

GDAL Web driver

Network file systems or web services got you down? Sidestep them with a lightweight and high performance GDAL filesystem implemented on HTTP:

People sometimes talk about “RESTful” services on the web, and I’ll admit that there’s a lot to that that I don’t really understand. I’ll admit that the tiff format is not designed to have HTTP ‘links’ to each pixel — but I think the fact that by fetching a small set of header information, GDAL is then able to find out where the metadata is, and request only that data, saving (in this case) more than a gigabyte of network bandwidth… that’s pretty frickin’ cool.

Part of what makes this work is the Web's built-in support for byte range requests and partial representations. See RFC 2616 section 14.35.1 and HTTPbis part 5:

Since all HTTP entities are represented in HTTP messages as sequences of bytes, the concept of a byte range is meaningful for any HTTP entity.

Make a small byte range request for TIFF metadata so you can map pixel and row (and maybe pyramid depth) to an entity byte range and then request just the right subset of the image.

For what it's worth, there's a W3C Media Fragments Working Group with a mission of making image subsets more addressable.


Re: GDAL Web driver

Author: Christopher Schmidt

Hey, I recognize some of that text.

I'm not really sure how your second quote addresses my actual "I don't understand" section (which is terribly worded, bad on me for writing under the influence of sudafed). The HTTP spec surely describes byte ranges as meaningful, but does that mean that accessing things via byte ranges is RESTful?

Re: GDAL Web driver

Author: Sean

I should edit my post to make it clear that the second quote isn't meant to be a rebuttal or counterpoint to your first one. Partial representations aren't part of "textbook" REST, but aren't inconsistent with it.

In Rtree news

This morning Howard Butler announced an upcoming Spatialindex 1.6.0 release:, to be followed by an Rtree release. I'm becoming more interested in seeing libspatialindex currently packaged for Linux distributions, particularly Ubuntu. Lucid has 1.4.0. I wonder if I can parlay co-working with a Launchpad developer into a fast track for 1.6.

After a long time away I'm using Rtree (0.6) in anger this week. I love what Howard and Brent Pederson have done with the code and docs. Try out

>>> from rtree import Rtree
>>> help(Rtree)

and you'll see what I mean. I've also uploaded new HTML docs (by Howard, using Sphinx). Let us know if you find any errors or unclear directions.

My own Rtree-using project is a simple, batching index for GeoJSON feature-like objects. Deserialized GeoJSON goes in, deserialized GeoJSON comes out. The interface is shown by this base class [indexing/]:

class BaseIndex(object):
    """Base class for indexes

    Deals in GeoJSON-like dicts we will call ``items``, like:

      {'id': 'db441f41-ec34-40fb-9f62-28b5a9f2f0e5',    # required
       'bbox': (0.0, 0.0, 1.0, 1.0),                    # recommended
       'geometry':                                      # required
         {'type': 'LineString', 'coordinates': ((0.0, 0.0), (1.0, 1.0))},
       'properties': { ... }                            # optional
       ... }

    def intid(self, item):
        """Return a unique integer id for the item"""
        raise NotImplementedError
    def bbox(self, item):
        """Return a (minx, miny, maxx, maxy) tuple for the item"""
        return bbox(item)
    def intersection(self, bbox):
        """Return an iterator over items that intersect with the bbox"""
        raise NotImplementedError
    def nearest(self, bbox, limit=0):
        """Return an iterator over the nearest N=limit items to the bbox"""
        raise NotImplementedError
    def index_item(self, itemid, bbox, item):
        """Index item using unique integer itemid and bbox as key"""
        raise NotImplementedError
    def unindex_item(self, itemid, bbox):
        """Unindex the item with (itemid, bbox) as key"""
        raise NotImplementedError

Some of you (2 people, tops) will recognize this from zope.catalog. My first implementation is an index that uses an instance of Rtree as a forward mapping and an instance of OOBTree.OOBTree (from the ZODB package) as a backward mapping (item values and id/bbox keys) [vrtree/].

class VRtreeIndex(BaseIndex):
    """An index with an R-tree as the forward mapping and a B-tree as the
    backward mapping

    Modeled after the example at

    def clear(self):
        self.fwd = Rtree()
        self.bwd = OOBTree.OOBTree()
    def __init__(self):
    def intid(self, item):
        """Rtree requires unsigned long ids. Python's hash() yields signed
        ints, so we add 2**32 to hashed values for the former."""
        return hash(item['id']) + OFFSET
    def intersection(self, bbox):
        """Return an iterator over Items that intersect with the bbox"""
        for hit in self.fwd.intersection(bbox, True):
            yield self.bwd[(, tuple(hit.bbox))]
    def nearest(self, bbox, limit=1):
        """Return an iterator over the nearest N=limit Items to the bbox"""
        for hit in self.fwd.nearest(bbox, limit, True):
            yield self.bwd[(, tuple(hit.bbox))]
    def index_item(self, itemid, bbox, item):
        """Add an Item to the index"""
        if (itemid, bbox) in self.bwd:
            self.unindex_item(itemid, bbox)
        self.fwd.add(itemid, bbox)
        self.bwd[(itemid, bbox)] = item
    def unindex_item(self, itemid, bbox):
        """Remove an Item from the index"""
        value = self.bwd.get((itemid, bbox))
        if value is None:
        del self.bwd[(itemid, bbox)]
        self.fwd.delete(itemid, bbox)

I think this is the first time I've ever written code using a tuple as a mapping key, but it's a perfect fit for an R-tree that takes a tuple of integer id and bounding box as a key. This index will be given a persistence wrapper and fronted by a Tornado based service that accepts JSON data and dispatches to index methods. A "Spatial Solr", if you will, which I attempt to describe in the README.

That's not the only possible implementation. One that needed to run on App Engine (for example), where there's no C-dependent Rtree, might use the App Engine datastore for backward mappings, and load an instance of pyrtree.RTree from the former on startup; the mapping serving as persistence for an R-Tree that doesn't have its own persistence mechanism.

Naming projects gets harder and harder. "Vaytrou" is a verlan play on the verb trouver (to find). "Vétrou" would be more authentic, but GitHub refused.

Update (2010-09-17): Here's the Vaytrou server. Simple. Handlers to accept batches of items and perform queries are in the works.

Update (2010-09-17): Spatialindex 1.6.0 is now available from links at


Re: In Rtree news

Author: René-Luc D'HONT

Now, I understand the 'Vaytrou' name!!! I initially think about 'way true'!!! I'd like to work with Tornado, if it's possible!

Re: In Rtree news

Author: Sean

Have you forked Vaytrou yet? It would be fun to work with you on it.

Re: In Rtree news

Author: whit

Nice to see the standalone http rtree index resurface! and yeah, I did recognize it as zope.catalog. Might take a stab at running it with gevent.

Shapely 1.2.4

Shapely 1.2.4 has been tagged (see Sdists are available from and This releases fixes bugs involved with loading the libgeos_c DLL/SO and generally improves the user experience in the case of not being able to load them, if you can believe that. If you try to import from Shapely without GEOS on your system, you get:

>>> from shapely.geometry import Point
Traceback (most recent call last):
OSError: Could not find library geos_c or load any of its variants

Also in 1.2.4, AttributeError is raised when you call a GEOS-backed method that isn't supported by your OS's geos_c version. In 1.2.3, you'd see IndexError if you called either of the linear interpolations methods with GEOS < 3.1. Exceptions are part of the API, too, and these changes make the API a bit more user-friendly.


No available geos

Author: Reinout van Rees

Not having geos available isn't so strange. I normally work on linux, but had to install a (geo) django website on windows a few weeks ago. In combination with oracle.

Problem: I couldn't find a geos .exe installer for windows. In the end I had to install postgres+postgis to get my hands on a geos dll ;-)

Re: Shapely 1.2.4

Author: Sean

I saw that you'd blogged about that here and I forwarded it along to people on Twitter. Shapely's Win32 installers include the GEOS DLL, so Windows users don't have this problem, but it's possible to install Shapely on *nix systems without the library. In the previous version, it would fail in that case, but with cryptic errors.

At the market

While the year-round market on the Boulevard des Arceaux in Montpellier is the best I've ever frequented, the Larimer County Farmers' Market is a pretty good, if short, substitute. Here are some sights you don't see at a market in the Languedoc:

French people who've tasted sweet corn love it, but it's just not grown in the south of France like it is in the USA's Midwest. This man here had corn, only corn. At Arceaux it was common to see a vendor of oysters only or exclusively honey, but vegetable sellers almost always had some range. An exception was aspargus: in season, you could bring nothing but boxes of asparagus to market and rake in the euros.

One of the sounds of the Fort Collins market in September is the roar of propane jets:

The smell of freshly roasted green Anaheims, Poblanos, and Big Jims pervades the market atmosphere. I grab a couple bags every Saturday, peel them immediately, and toss them in the deep freeze for winter stews.

We like our melons big in the USA. Like the chiles, these are typical of Southeastern Colorado.

Colorado watermelon tops French pasteque, but I prefer the smaller orange Lunel melons to the bloated Rocky Ford cantaloupes. These ripen later around here, which meant we left France at the end of melon season and arrived in Colorado at the beginning of melon season. It's been the same situation for peaches:

Peaches from Colorado's Western Slope are unrivaled.


Re: At the market

Author: Tyler

An interesting website for farmers markets is


which allows venders to post lists and pictures of what they are selling. It also show connections between restaurants serving local foods and the farms that produce it. It was started in Ann Arbor and most of the current content is for the greater Detroit area, but the founder has told me that Colorado may be one of the next regions to expand to.

Why not GeoJSON?

I've seen people question ESRI's decision not to use GeoJSON in their server API spec (note: sometime after I wrote "If it's a #GIS spec, you can be sure there's a click-through agreement to read it" the click-through was thankfully eliminated). Since we in the GeoJSON working group haven't gotten around to actually standardizing the format through the IETF (for example, the body that published RFC 4627 on JSON), it's completely forgivable. The CC-BY license on the GeoJSON spec is clear on rights to use the work itself, but perhaps not clear enough to some about the rights to "GeoJSON technology". Myself, I don't consider GeoJSON a technology, but it's courts that decide, not me. One must also remember that ESRI's spec is the documentation of a web API that, while maybe not written before the GeoJSON working group started, shipped before we had a published GeoJSON spec and well before it got serious traction.

I do like a couple aspects of ESRI's JSON geometry representations. Geometry objects aren't primarily determined by their "type", but by their properties: an object with an "x" and a "y" is a point, an object with "paths" is a polyline, an object with "rings" is a polygon. We considered this "duck typing" approach when drafting the GeoJSON spec, but explicit typing won out instead. In addition, with the exception of point objects, the coordinates are represented in a way that's entirely compatible with a GeoJSON reader. It's a shame about the points, but otherwise there's a lot of common ground. ESRI JSON features look a lot like GeoJSON features, too.

I'm not sure what to say about the rest of the spec. It's awfully large and I get a flash of OOXML deja vu that may yet pass. It's certainly different. I'm looking forward to other reviews and analysis.

Browsing spatially referenced Djatoka images with OpenLayers

A while back my colleague Hugh Cayless wrote a Djatoka layer for use with OpenLayers. It is deployed at UNC's library, allowing you to view over 3000 North Carolina Maps. View the source on any map and you'll quickly get a sense of how it's rigged up. You pass a URL encoding requests for image regions and a URL encoding requests for image metadata to the OpenURL layer constructor, it figures out the grid and zoom levels for you, and you add the layer to a map that's configured using the layer's resolution, maximum extent, and tile size parameters. It works much like a TMS layer, but in pixel units.

I began with this code for a different project and modified it to satisfy some of my project's special needs:

  • The imagery of a physical object and its annotations will be shown in their natural units – centimeters.

  • Square tiles will be used instead of rectangular tiles with the same aspect ratio as the physical object (approximately 13:1).

  • Browser and image server are in different domains, complicating discovery of image metadata.

I decided to go with Djatoka instead of an open source GIS map server like GeoServer or MapServer because the project, while spatial in nature, doesn't require cartographic projections or spatial indexing; I was interested in trying something new; and NYU's already got one running (Thanks, Hugh!). I ended up modifying Hugh and Cliff's OpenURL.js considerably. Now, it's more like a hybrid between OpenLayers.Layer.Image and OpenLayers.Layer.WMS. The code is at

Unlike a GIS map server, Djatoka doesn't pad requests: it fails if you ask for regions outside the image. To have square pixels, I've had to pad my imagery a bit. It's now 43008 pixels wide by 3072 pixels high, exactly 14:1. This makes the right tile size 192 x 192 pixels. As things stand, my new OpenURL layer can't be used in the same map as Hugh's, so I haven't bothered to rename it yet. It's used like this:

var RES = 1.0/60.3; // Centimeters per pixel
var size = new OpenLayers.Size(43008, 3072);
var origin = new OpenLayers.LonLat(-16.6222, -1.2174);
var extent = new OpenLayers.Bounds(
    origin.lon,, origin.lon + size.w*RES, + size.h*RES);

// Zoom factors of 1/8 to 4
var resolutions = [8.0*RES, 4.0*RES, 2.0*RES, 1.0*RES, 0.5*RES, 0.25*RES];

// At zoom level 0, the image is covered by 2 x 28 tiles
var tileSize = new OpenLayers.Size(192, 192);

var mosaic = new OpenLayers.Layer.OpenURL(
  'All section, mosaicked',
  '', // for example, 404 in reality
  { format: 'image/jpeg',
    units: 'cm',
    tileSize: tileSize,
    resolutions: resolutions }

var map = new OpenLayers.Map(
  { units: 'cm',
    tileSize: tileSize,
    resolutions: resolutions }

A very simple application of this layer can be seen at

Shapely 1.2.3

Shapely's had a shape() function that makes geometric objects from GeoJSON-like mappings for some time. Now, by popular request, is the inverse: mapping():

>>> from shapely.geometry import Point, mapping
>>> m = mapping(Point(0, 0))
>>> m['type']
>>> m['coordinates']
(0.0, 0.0)

The new function operates on any object that provides __geo_interface__ [link]. This release also fixes a major bug involving GEOS versions < 3.1 introduced in 1.2.2. You can download an sdist from Windows installers should be coming soon.

As I like to remind people every few releases, major portions of this work were supported by a grant (for Pleiades) from the U.S. National Endowment for the Humanities (

Update (2010-08-22): Win32 and Win-AMD64 installers have been uploaded to the locations above.