2008 (old posts, page 14)

Get your Grand Tour on

My interest in pro cycling bottomed out last summer, but Get Your Grand Tour On has completely revived it. Inspired by David Rees's brilliant Get Your War On -- brace yourself for sarcasm and obscenity.

Speaking of cycling: last weekend I took my vintage 1987 Hardrock on a trail for the first time in 15 years. The 40 pound frame and classic 80's geometry (slightly downsloping top tube) make pulling heavy trailer loads a joy. In the background is the 10th Mountain Division Hut Association's newest hut, just west of Tennessee Pass (3117 meters).


REST on the conference circuit 2008

Anything changed since 2007? Yes: REST, the architectural style of the Web, has finally broken through into the "GeoWeb" conference program. The rumored REST track didn't make it, and the agenda remains skewed toward the usual OGC SOA architecture, but 4 out of 66 workshops and papers (of which, to be fair, only about a third cover web services at all) is a nice first step. I assume REST will be a more popular topic at the upcoming ESRI User Conference, but the agenda is so big and the search function too limited to say for sure.

It looks like FOSS4G 2008 has less REST on its agenda than did the 2007 version. That conference website doesn't provide any abstract search at all, so I'd have to scrape it, but only after first obtaining a login. Login to see abstracts ... are you kidding me?

Update (2008-07-09): Josh Livni tipped me that abstracts can now be viewed without login on the FOSS4G conference site, and abstract search seems to have been enabled as well. Using it confirmed my initial impressions: there's no REST on the agenda outside of a geocoding service with a REST API and mention of GeoServer's new REST style administration interface. Other than that, it's back to all W*S, all the time. It looks like the REST session at FOSS4G 2007 was a fluke.


Re: REST on the conference circuit 2008

Author: Andrew Larcombe

Follow the money. Have you ever had a client request a REST API instead of W*S for an application?

Re: REST on the conference circuit 2008

Author: Sean

I understand where you're coming from, and that I'm in a different position with a bit more freedom to choose. Still, the FOSS4G community should be paying closer attention. Google gets it; Microsoft gets it; ESRI gets it; don't be left behind.


All REST Monday continues. Except for very large values of "inconsistent" -- like "one or the other doesn't use HTTP" -- I think a little more explanation is needed to support this:

REST-based architectures are not inconsistent with the WFS specification.

Again, the five key principles of REST (use of HTTP is implicit), from Stefan Tilkov again, are:

  • Give every "thing" an ID
  • Link things together
  • Use standard methods
  • Resources with multiple representations
  • Communicate statelessly

The last of these is the only one to which WFS 1.0 strictly adheres. WFS does not specify permanent URIs for individual features; does not specify hypertext links among features, types, and services; uses GET or POST, yes, but interchangeably and in ignorance of PUT and DELETE; and does not specify use of HTTP content negotiation to select data formats. All of Tilkov's REST anti-patterns save number 5 (cookies) are written into the WFS specification, and so it still seems to me that WFS is mostly inconsistent with REST.


Re: Explain?

Author: Christopher Schmidt

Yes. Thank you. Saves me having to write my own post saying the same thing. :) At some point, I should write about why WFS is so poorly supported with OpenLayers -- while things like FeatureServer take less than a dozen lines of code to write support for. (Well, that and the fact that implementations are scare, examples outside of implementations are scarcer.)

Geojson 1.0 beta 1

We've renamed our "GeoJSON" package to "geojson", uploaded the first beta release, and plan to finalize it within a couple weeks.

Geojson provides geometry, feature, and collection classes, and supports pickle-style dump and load of objects that provide the lab's Python geo interface. Here's an example of a round-trip through the GeoJSON format:

>>> import geojson
>>> p = geojson.Point([0.0, 0.0])
>>> p
Point(coordinates=[0.0, 0.0])
>>> data = geojson.dumps(p)
>>> data
'{"type": "Point", "coordinates": [0.0, 0.0]}'
>>> q = geojson.loads(data, object_hook=geojson.GeoJSON.to_instance)
>>> q
Point(coordinates=[0.0, 0.0])

The geometry classes interoperate with Shapely via the geo interface:

>>> from shapely.geometry import asShape
>>> x = asShape(p)
>>> x.wkt
'POINT (0.0000000000000000 0.0000000000000000)'

A manual is forthcoming; for now there are many doctest examples. We would appreciate your giving it a try, which is as easy as:

$ easy_install geojson

Fava time


We had our first favas this evening, sautéed with garlic in olive oil. Pink Côtes de Provence, not Chianti, and ravioli, not liver. These were young enough that we didn't bother to peel the individual beans after shelling; the skins have a slightly bitter, artichoke quality that complements the intensely sweet pea interior (if you like hoppy beer, you'll like favas). Our toddler found them tolerable enough, though she by far prefers the snow peas that are also just starting to arrive -- raw, never ever cooked, and best straight off the vine. It's great to have peas and beans now as we were getting a bit burned out on spinach and turnips.


Re: Fava time

Author: Abu Rashid

mmmm foole. bon apetit.


Andy Powell's presentation on Web 2.0 and repositories is just as relevant a read for GIS designers as it is for archivists. These communities have much in common, including:

  • A mission to archive and curate comprehensive, unique datasets.
  • Information architectures, formats, and protocols that pre-date the Web.
  • Entrenched agendas that overshadow architectural concerns.

In the institutional repository space, open access voices don't want their agenda derailed by debate about the re-architecture of the systems to be opened. Likewise in the open source GIS, or FOSS4G, space: businesses that are barely succeeding in winning government and industry clients over to open source solutions based on "OpenGIS" standards aren't keen on having these standards undermined for any reason.

Powell's assessment is that the way forward for repositories is either simpler (Atom, for example) or more complex (RDF, which I'm told is to the OGC what the XML mapfile is to the MapServer community) than the current approach. Or both. I think this is true for geospatial as well.

GeoJSON ships

Congratulations, everyone. From the official announcement:

The GeoJSON Authors are proud to announce the finalization of the GeoJSON 1.0 Specification.

Representing more than a year's worth of community discussion and development, the GeoJSON specification describes an easy to use, extensible format for transferring geographic data over the web. With support in more than 20 different applications, GeoJSON is already quickly becoming a de facto standard for transferring geographic data in a JSON format. The finalization of the spec represents the final step in formalizing the GeoJSON format for encoding this data.

More information on GeoJSON can be found at http://geojson.org/, or from the GeoJSON mailing list at http://lists.geojson.org/listinfo.cgi/geojson-geojson.org.

More from Martin and Christopher.

Guerrilla SOA

If you are interested in applying Agile software development methodology to geographic or GIS Web services, you may also be interested in this talk by a couple of ThoughtWorks gurus: Does my bus look big in this?. In the second half, after examining the faults of conventional enterprise SOA, Martin Fowler and Jim Webber make analogies from Agile development practices and benefits to using the Web/REST for agile deployment and integration.

Feature query languages

Earlier this year I had a need to do some vector wrangling, an itch for a better Python API, and curiosity about Pyrex and Cython; egged on by Matt Perry, I wrote a package I called WorldMill. It's getting used out there, I'm pleased to say, and I'm getting feature requests and patches for feature attribute filters. The patch I'm holding right now enables filters using OGR's attribute filter language, a subset of SQL. For example, the following filter expressions passes all features with a numerical "value1" attribute greater than zero:

value1 > 0

It's a limited subset of SQL. You can't compare the values of different feature attributes, for one thing:

value1 > value2  # fail

Nevermind that. What concerns me right now as I face this WorldMill patch is that SQL may not be the right model at all for this sort of domain specific language. Certainly not for WorldMill, maybe not right for any modern abstract interface. Yes, GIS data still tends to be flat, tabular stuff -- shapefiles are still common and most organizations that have moved beyond them have advanced to a RDBMS (the ultimate state of being for data if you believe Martin and Paul). However, exceptional data is increasing. GML permits, even promotes, complex, nested data structures; GeoJSON does as well. KML and Atom are not tabular formats, and "live" on the Web, not in a RDBMS. The mere existence of SPARQL says to me that SQL doesn't cut it for non-tabular data. I'm not sure I want WorldMill to span the universe of non-RDBMS storage, but a fraction of it for sure, hence my lack of enthusiasm about SQL-based feature filters.

What are the alternatives? MapServer expressions? Too quirky. GQL? Not much point since App Engine already has an API. OGC XML filters or a Python embodiment thereof? Good grief, no. Python expressions like I used in PCL? These days I prefer something more generic. Could Javascript be it? Fits GeoJSON like a glove, obviously; works in the browser; it's the view/query language for CouchDB, a popular non-relational data store (seriously, check out CouchDB views and tell me that isn't a neat paradigm), so it's available on the server or in the cloud as well. The Python-Spidermonkey project may have reawakened just in time for me. At any rate, I'm very curious how other folks evaluate filter and query languages.


Re: Feature query languages

Author: Alex Willmer

Perhaps css selectors (as used in jquery) or xquery expressions. Whatever you choose, better to copy an existing syntax than create something new.


Author: Stefano Costa

Just yesterday I was trying to understand whether CouchDB could be used to store geothings and their non-geo attributes. Its non relational and distributed model looks very attractive for a number of reasons. I found this general introduction http://vmx.cx/cgi-bin/blog/index.cgi/couchdb-and-geodata%3A2008-05-03%3Aen%2Cgeo%2CCouchDB but it's not that interesting because the author doesn't give any reference to actual implementations of CouchDB as geodatabase. If I had more experience both with CouchDb and geodatabases probably I could investigate by myself...

Re: Feature query languages

Author: Martin Davis

Great minds think alike (but not always in the same language). I've been working on a SQL-based query language which can leverage all the great Java geospatial APIs out there. It's called JEQL - web site is http://tsusiatsoftware.net/jeql/main.html I chose the follow the SQL paradigm for several reasons: - familiar, mature, proven utility - good harmony with important projects such as PostGIS - I think the SQL paradigm has a lot of juice in it even when processing hierarchical data. See for instance Oracle's handling of XML data - maybe not quite as pretty as you'd like, but quite powerful. Also see the ability to express queries over relational hierarchies. I'm hopeful that this paradigm could be extended to allow querying over full graphs - which is required for some kinds of geometric processing. - SQL has some ugly warts which people always stumble over, but I was curious to see whether these could be resolved with better language functionality. So far I'm happy that they can... However, like you I do have an eye out for a better query paradigm which would provide cleaner handling of hierachical data. So far the best one I've seen is XQuery. However, IMO it is too bogged down in XML syntactic details. I want something which is representation-independent. XSLT is even worse - no-one should be forced to code in XML! As far as I can see SPARQL is both too limited and too general - it operates over RDF triples, but can process arbitrary graph structures. It's not clear to me that this would improve the ability to handle tabular data, let alone hierarchical data. The problem with very general languages is that their expressiveness is necessarily limited. Also, if I understand correctly SPARQL itself has fairly limited power - it needs an associated inference engine to provide more smarts. Fascinating subject, though. It seems suspicious to me that after alll these years no more powerful query language than SQL (or at least the relational paradigm) has gained widespread acceptance. Of course, this is not a proof that no such thing exists - but it makes me very cautious about spending a lot of energy trying to invent such a thing myself.

Re: Feature query languages

Author: Martin Davis

Another comment, on using Javascript (or any procedural language) as a filter or query language. Seems to me this has a few potential issues: - you are tied to the semantics of the underlying language for your data model - the expressions are not declarative (and thus hard to optimize) - you end up with a mixture of language syntax db40 has used a similar approach in Java. I think they do some clever tricks to do some optimization, but you're never going to get the kinds of deep optimization that any good SQL engine does.

Re: Feature query languages

Author: Sean

Martin, JEQL looks interesting. I read the CSV to KML example. It occurs to me that a DSL based on Ruby (JRuby in your case) might be more appreciated for all kinds of reason these days. I agree with you that SQL hits the sweet spot for relational data, but it's less useful for more loosely structured data, and it seems like there will be more and more of that in the future. I'd like to be able to manipulate loosely structured data in a more natural (even elegant) way than loading it into a RDBMS for processing and exporting the results.