Feature Demo

Update (2007-09-20): Hammock is evolving.

Update (2007-04-13): coordinates attribute has been reformatted from [x, y] to [[x, y]] to keep up with the evolving GeoJSON RFC. If you're curious about the source code, see the following post.

This evening I cobbled together a demo of a feature service inspired by Joe Gregorio's Robaccia. It's a toy model of something not unlike a RESTful WFS-T, based on the Python wsgiref, selector, elementtree, and simplejson packages.

http://sgillies.net/images/map-before.jpg

The root resource is at http://sgillies.net:8081/features, and the default view is a JSON array of links to objects. (Note: Douglas Crockford has proposed the application/json content type, and I've only used text/plain here to make it easier for people to follow the links in their browsers.) The URLs http://sgillies.net:8081/features.atom and http://sgillies.net:8081/features.kml return GeoRSS (Atom) and KML representations respectively.

Over the course of several images, I'm going to walk through the process of getting representations of feature resources, and posting new resources to the system.

The first image shows all the features in the system as seen using KML in Google Maps [link]. More features may have been posted to the system by the time you click on that link, so you may see more points. It's also possible that these particular features here may have popped off, as I've capped the number of features in the system at 30. Like I said, it's just a toy. Only 288 lines of Python.

Now, fire up your RESTful feature service client, and follow my instructions to add a new feature to the system. You've probably got several such clients on your system already. My favorite is good ole vim. Seriously.

http://sgillies.net/images/getting-template.jpg

You can obtain a representation of any of the system's features by reading the output of a curl command into an empty buffer:

:r ! curl -s --url http://sgillies.net:8081/features/1

The -s option prevents capture of download rate stats. Use any feature URI that you find in http://sgillies.net:8081/features. The resulting JSON text will be a template for a new feature.

http://sgillies.net/images/got-template.jpg

Edit the coordinates (this toy service is currently restricted to points), title, and summary. Any value you provide for id will be overridden, as will any timestamps.

http://sgillies.net/images/posting-data.jpg

Now, post this data to the features resource by piping the buffer contents back through curl:

:% ! curl -s --url http://sgillies.net:8081/features -d @-

If you typed carefully, you'll get a result like:

http://sgillies.net/images/post-results.jpg

Go ahead and browse to the resultant URL and admire the shiny new feature.

http://sgillies.net/images/map-after.jpg

Reload the Google Maps page, and you'll see be able to see your new feature. If you're a careful typist, do play around with this as much as you want. I'll keep the service running as much as I can, though the features may come and go. If need be, I'll apply my blog comment policy to features posted by visitors and delete any obnoxious crap.

With luck, I'll add the capability to PUT and DELETE features next week.

I'll continue to tweak this toy and use it to explore Pleiades use cases: posting KML files created with Google Earth (would be nice to do it directly), and creating new features via GE and WebDAV are a few of the things we're looking at. Vim as a feature service client is fun for a while, but gets old quickly, and is a harrowing experiencing for a non-hacker. Still, the fact that you can use vim shows the enormous potential flexibility of this kind of system architecture.

Comments

Re: Feature Demo

Author: Rob De Almeida

Very nice! I also liked the idea of using vim as a REST client. :)

Re: Feature Demo

Author: Bryan

Nice stuff. Couldn't make google maps do it's business, but did engage with the server just fine ... I much appreciate you putting these sorts of ideas up.

Re: Feature Demo

Author: Sean

Pasting http://zcologia.com:8081/features.kml into the http://maps.google.com search field doesn't work for you? It is a fun idea. Now I need a better client. I'd like to see Google Earth support POST (better yet, PUT) of KML to arbitrary URLs. In my mind, that's more generally useful than implementing the client side of WFS-T.

Re: Feature Demo

Author: Guillaume

Nice toy actually. pasting http://zcologia.com:8081/features.kml into googleMaps is ok, but not with the http://zcologia.com:8081/features.atom. Any idea why ? I'm greatly interested in the REST approach, but I have some misunderstanding about it. As a web service giving access to ressources, I think I get it. But how can a full application be build around that ? imagine a map, with a legend, and a query/info tool. If I click into a legend checkbox to hide a layer on the map, must I "GET" both map and legend services ? how to synchronize them ? When I click with info tool on the map, I need as a result both a new map with the selected object highlighted on it, and a text stream giving information about it. Same question : how can I build that in a restfull way ? Thanks

Re: Feature Demo

Author: Kristian

Guillaume, "When I click with info tool on the map, I need as a result both a new map with the selected object highlighted on it, and a text stream giving information about it." For instance by having https://example.com/getInfo/${id} return { info: 'Info text here', map: 'http://example.com/getMap/${bbox}/${id}' } or somesuch.

Re: Feature Demo

Author: Sean

I'm not sure why the Atom feed isn't rendered. It is valid. Make it easier, Guillaume. Your info tool could query a resource like GET /map/featureinfo?x=-100.0&y=40.0 which could return information to be displayed in a table, and a shape to be rendered in a transparent overlay. Keep the current map (tiles) unmodified.

Re: Feature Demo

Author: Guillaume

ok, it's getting clearer. Thanks for these clues.

Game Over for GeoDRM?

GeoDRM proponents have argued the necessity of geospatial digital rights management partly by analogy to the entertainment industry. So, it seems that Apple, Microsoft, and their digital music partners pulling out of DRM also pulls the rug out from under GeoDRM. Geodata users don't want digital rights management any more than do users of digital music. Does this mean the end for GeoDRM? Maybe yes, and maybe no (Mark Shuttleworth):

There are some ideas that are broken, but attractive enough to some people that they are doomed to be tried again and again.

On the other hand, work on rights expression is useful to all of us, and even more so in the light of recent revelations that Creative Commons doesn't want to get into the geodata business.

Geometries for Python Update

Update (2008-01-21): See http://pypi.python.org/pypi/Shapely/1.0.

An older, slightly outdated, post of mine about Python geometry packages is getting read enough to warrant an update.

Geometries from mapscript.py and ogr.py haven't changed in the past year. The GeoDjango developers are finding that the geometry API in geos.py isn't so bad -- perhaps because they're doing the bulk of their operations and transforms within the database -- but the fact remains that the GEOS project is trying to deprecate use of the C++ API, and that this puts geos.py in some peril.

Since the Python Cartographic Library's 0.11 release, geometries are now a part of the PCL-Core component's cartography.geometry package. Howard Butler added pickle support, and we've fixed a number of bugs since the start of 2006. We've also reduced the dependencies of PCL-Core, as you can see in INSTALL.txt. Remember, "zope" is now a namespace package, and you can install zope.interface [Cheeseshop] separately from any other piece of Zope 3.

I have nothing to report about FDO's brand new geometries for Python. My understanding is that they are also based on GEOS.

Comments

Re: Geometries for Python Update

Author: Allan

The thing that stopped me from going much further with GeoDjango was the need for GEOS. As far as I could tell with the version I had, GEOS was just being used to pack/unpack WKBs. PostGIS can do that already so I wonder if it would be possible to use WKT on the GeoDjango side instead of WKB. Unfortunately, trying to install GEOS with Python bindings on Mac OS X was just too much for my meager abilities. However, I realize that it's early days and I'm not complaining. I think it's great that you guys are working on this.

Re: Geometries for Python Update

Author: Rob

GeoDjango will always need to depend on something (eg. GEOS) otherwise it forces the DB server to do a lot of work. GeoTypes can be a python-only parser for WKB, but then you still are without a solution for geometric operations (area(), length(), x(), y(), transform(), ...) so you end up back at GEOS. My research has also turned up OGR's python bindings which are both maintained, and have no dependencies except OGR itself. Experiences? Comments?

Re: Geometries for Python Update

Author: Jason Birch

Heh. Well, FDO's Python wrappers build on Linux now, so that's slightly new... I could be wrong, but to my knowledge GEOS is not used by FDO at all (it is by MapGuide). It might would be a good idea for FDO to use GEOS to provide some more advanced spatial processing for providers that don't offer it natively. For instance, the MySQL provider is limited to envelope operations. But maybe people using MySQL are OK with this :) Jason

Re: Geometries for Python Update

Author: Sean

Like I said, Jason, I'm ignorant about FDO geometries and had nothing to report. Rob, if you go back through my previous post, you'll find out everything I know about ogr.py's geometry API.

Re: Geometries for Python Update

Author: Matt

I was also having problems with GeoDjango and the geos dependency on my intel mac but I managed to get it working with the r5008 update and manually setting my lib_name = 'libgeos_c.dylib' in django/contrib/gis/geos/GEOSGeometry.py. The original value in that file points to a library that doesn't exist on mac when you build geos 3.0.0RC4 but the older version of libegeos seems to work.

Re: Geometries for Python Update

Author: Sean

The ctypes-based geometry module in r5008 looks kickass. I'm checking it out now.

GeoRSS: Worse is Better

Sam Ruby is using GeoRSS, and the emergence of new feed based applications from Google and Microsoft, as an example of what non-elite web programmers can accomplish without XML schemas (or SOAP). The uptake of GeoRSS (and RSS more generally) demonstrates once again how worse is often better. People are typically generating GeoRSS feeds using tools that are completely ignorant of XML Schema, validating them via web sites, and processing fetched feeds using the DOM or XPath capabilities of their browser's JavaScript engine (GeoRSS.js). That the Geo-Web is forming out of GeoRSS and KML -- initially released without a schema -- rather than GML is yet another indictment against XSD.

The James Clark post to which Ruby replies is also a good read. Something like TEDI would come in handy at times, maybe even for the Python feature protocol I'm working on.

Comments

Re: GeoRSS: Worse is Better

Author: Allan

The link to Sam Ruby's blog should be http://www.intertwingly.net/blog/2007/04/06/Targeting-Non-Elite-Programmers I definitely count myself in the worse is better school...

Re: GeoRSS: Worse is Better

Author: Sean

Thanks, Allan. Link fixed.

Re: GeoRSS: Worse is Better

Author: Chris Andrews

As someone who has created a hobby content management service, integrated it with Google maps, and was recently able to plot my RSS feeds on someone else's mashup, I'd have to say that I'm all for 'worse is better.' At the end of the day, I want something that works... now! I once had so many clients sending me XML with no DTD or Schema for a longterm integration project that I created a tool to marshal/unmarshal XML directly into Java classes. (Yes, XML, not XSD or DTD.) While I'm a standards guy, the reality is that XML is too easy to abuse outside of the standard definition tools. Nice blog.

Conference Wireless How-to

For the FOSS4G 2007 organizers: wireless lessons from PyCons 2006 and 2007.

Comments

Re: Conference Wireless How-to

Author: Paul Ramsey

Great reference. I'll make sure the conference center technical folks go over it. They seemed quite open to trying to understand how our conference differed from the usual bill-of-fare.

Re: Conference Wireless How-to

Author: Paul Ramsey

More from the conference centre:
We are currently working on a major upgrade to the system that should address all of the issues in that article. I hope to be able to provide you with the technical specs and an access point diagram before the end of the month. We are also adding a 100Mbps fiber connection.

Re: Conference Wireless How-to

Author: Sean

I'm glad the link was useful, Paul. I've run events in my time, but none of them had logistical requirements far beyond lined fields, cones, drinking water, and kegs of beer. Satisfying the network needs of hundreds of virtual globe, WxS (uncacheable!), and IRC users is going to be a big job.

Silence on Spatial Search

Update (2007-06-04): More search help, but no details, from Google.

Update: Paul Ramsey reports that the game-changing impact of spatial search was brought up in the CalGIS conference keynote.

Ed Parsons, a few weeks ago, was befuddled by the GIS industry's almost-imperceptible response to Google's spatial search news. There hasn't been any new tide of interest since, and there are 3 primary reasons.

First and foremost, there's no public API for the spatial search, and no access other than via Google Earth or Maps (example here). What's a geospatial developer to do? Navigate GMaps to a location, search, and then scrape the page for results? Ugh.

Next, there's very little information from Google about the steps geo-webmasters should take to be spatially indexed. One scant blog post about KML in sitemaps and a handful of quotes in Directions don't add up to much. I'd like to see some simple spatial tools integrated with Google's Webmaster Central. Maybe this will be a part of Ed's new job?

I'm not going to trot out the famous Upton Sinclair quote here, but it can not be denied that geospatial search offerings from Google (and soon, I'm sure, from other search engines) are disruptive to the mainstream GIS industry. Many of us are catalog programmers or catalog salespersons, and there's a bit of ducking and covering going on. Hoping that the disruption goes away without altering the business isn't going to work. Over the next few years, the relevance of sites like the Geospatial One-Stop, or projects like OGC Web Catalog Service, will evaporate just like the relevance of Yahoo's web catalog did in the late 1990s.

Comments

More REST and GIS Blogging

REST gets a mention from Sebastian Good, though his post is more about the current REST-less state of enterprise GIS.

Update: And this from Keyur Shah.

Me, again: enterprisey, business-to-business services will continue to use SOAP, but if there is going to be a Geo-Web, it will be based on REST.

Dumbing Down REST

In a comment on my last post, I pointed out that content negotiation is low on my priorities because there aren't enough smart user agents yet. In a related (and better) post, Dave Thomas explains how dumb the browser is, and how you might dumb down your applications. His idea is immediately relevant to anyone developing a RESTful GIS; when we're constrained to the same small handful of verbs (GET, POST, PUT, etc), it's easy to transfer and share architectural concepts.

Via Simon Willison.

Comments

Re: Dumbing Down REST

Author: Dave Smith

I certainly agree with the KISS principle, but I'd couch the "dumbing down" a little differently - services shouldn't necessarily be "dumbed down", but they SHOULD be clean and simple, and accessible "lights-out" without the need for a lot of RTFM and developer noggin time. How this dumbing-down/KISS is reconciled with the other SOA paradigm of "chunky vs. chatty" is yet another matter... The other thing that everyone seems in a hurry to overlook in the rush to pursue SOA (not specifically with REST but definitely JSON and other efforts) is to consider that it's not just client-server communication but also server-server communication that SOA solutions should support. Just a few of the little notions rattling around in my head, am still trying to make sense of it all.

Re: Dumbing Down REST

Author: Sean

Dumbing down, in this case, refers to making your sophisticated, 4-verb (GET, POST, PUT, DELETE) application responsive to user agents that can only GET and POST. Dave Thomas, and large segments of the Rails (or PHP, or Python, or what have you) community, might be more concerned about web browser applications, but server-server interactions have not been forgotten at all. Servers interact with other servers in the same way that clients interact with servers, using the same small, constraining set of verbs. Everything is just another user agent.

Re: Dumbing Down REST

Author: Mark Fredrickson

Ha. I had exactly the same thought when I read DT's article (picked up from another source). Lately, I've pondering the ability to use ATOM+GeoRSS as a quick foray into RESTful geo-services. Any thoughts from the web zeitgeist?

Google JSON and Geo

Interesting, but Google's JSON, literally transcribed from Atom, is almost exactly as cumbersome as XML (example here). Apparently, it's all about evading the browser's same origin policy. Google should throw its weight behind Douglas Crockford's JSONRequest proposal.

My Comment Policy

Recent events, summarized by Tim O'Reilly here, have prompted me to spell out a policy for comments on this blog. I encourage people to write critically about our community, industry, and media, but I reserve the right to remove pointless flamebait or personal attacks as well as spam. If an email address accompanies extremely poor commentary, I'll contact the author personally about making sensible edits or retractions. If not, I'll simply delete it. Fair enough?

Update: changed the wording a bit so as not to point at anyone. The comments I get here provide much, if not all, of my blog's value. I won't delete non-spam comments lightly, but I have no obligation to give jerks a voice here.

Comments

Re: My Comment Policy

Author: Jason Birch

No comment.

Re: My Comment Policy

Author: Andy

It's your blog you can do whatever you want with it. I'm not half as nice as you are. I delete comments that are even mildly annoying. Blogs are a way for us to get our thoughts and our views out where others can see them. They aren't in most cases a running dialog with the readers. Comments are a privilege.