G(eo)nomes of Atlanta

Martin Daly:

The OGC Geospatial Search Summit was held here at the Atlanta TC yesterday. It was an invitation only, behind closed doors event - grumble, grumble, moan, moan - so I don’t know any details about what went on.

Inside the "Open" Geospatial Consortium there are yet more inner circles?

Python packaging and builds

I don't follow the django-developers list so I might have missed Kevin Teague's great overview and analysis of Python packaging, dependencies, and builds (via Simon Willison). I've referred to setuptools, easy_install, and zc.buildout a number times, but never explained them quite as thoroughly or as well.

Setuptools is the not-quite-standard replacement for Python's distutils module. It has a big set of features, the one of which that helps me the most is versioning of dependencies. I can specify that zgeo.plone.kml 0.2 requires version 0.1 or higher of zgeo.plone.geographer, which in turn requires version 0.3 or higher of zgeo.geographer, and easy_install of the first one of those packages gets the proper dependencies in a cascade. I'm finding this to be better for my sanity than use of the svn:externals property to pull in tagged revisions as we used to do.

Despite what Teague says (update: I misinterpreted, see clarification in comments), I'm finding zc.buildout to be a big help with managing the C/C++ components in Pleiades. Because the GEOS project is -- how shall I say it? -- cautious about bug fix releases, and version 3.0 absent from Linux distros, I'm using Kai Lautaportti's hexagon.recipe.cmmi buildout recipe to download the GEOS 3.0.0 tarball, patch it, compile, make, and make install into an isolated environment for our web application with no more configuration than this:

[libgeos]
recipe = hexagonit.recipe.cmmi
url = http://geos.refractions.net/downloads/geos-3.0.0.tar.bz2
patch-options = -p1
patches =
  ${buildout:directory}/patches/geos-3.0.0-conditional.patch

Works like a charm.

To avoid polluting the Python package index with software of no use outside Pleiades, I've made our own specialized index. Easy_install of pleiades.policy (my obvious cargo cult of Martin Aspeli's example) pulls in pleiades.kml, and then the appropriate versions of zgeo.plone.kml and zgeo.kml. I'm happy to explain how to make such an index to anybody who's interested.

Do read Teague's post to the django-developers list if you're new to the Python software ecosystem, and if you develop any curiosity about how Zope works, read his excellent article about its component architecture.

Comments

Re: Python packaging and builds

Author: Kevin Teague

Thanks :) To clarify, I meant that buildout isn't useful for the problem of source code recompilation. It works great for source code compilation. The distinction between compilation and recompilation is the later is designed to help someone developing in a compiled language where you have an edit-compile-test cycle, a recompilation tool will do fancy things like look at timestamps to decide if an individual file needs to be recompiled, or manage compilation on multiprocessers or machines. If you're assembling parts of an application from source though, a one time 5 minute compilation step is no big deal. But if you have to go through that compilation every time you want to test a change in your code it's a pain. That being said, I guess it'd be possible to develop a buildout recipe which is useful for this type of workflow.

Re: Python packaging and builds

Author: Sean

Ah, yes I agree with you. Developing that GEOS patch within the buildout would have been painful.

C-List and proud of it

Technorati is totally busted for doing this kind of analysis. As I wrote on Gorman's blog, almost all of my blog's "authority" comes from automatic linking in the sidebars of several other blogs, blogs which never ever engage mine in any other way. In sum: Technorati's authority is a worthless measure of my blog's interestingness. Look closely at the authority of other blogs and I'm certain you'll see the same pattern.

Update (2008-09-10): It was all for nothing after all, which is good because the last thing we need is more gatekeepers to the blogosphere.

Comments

Re: C-List and proud of it

Author: SeanG

Some of those blogs haven't been updated in months and others are of dubious relation to mapping.

Re: C-List and proud of it

Author: Sean

Uh huh. Don't get me started about the content of these blogs. Any list of top geo-blogs without Paul Ramsey's or Kirk Kuykendall's is pretty sad. But, you see: I'm just about the only one that links to those blogs, and so they have negligible "authority" compared to other more trivial blogs. Technorati can't even deal with my blog move ... my old URL has an authority of ~50 and my current one ~10. It's the same blog (every request redirected), so do I get a total of 60? It's a crappy measure.

POST(a) and POST(p)

It's great to see others like Vish continuing to write about REST in geospatial (I continue to stalk the term). With a couple exceptions, he's right on. The principles of REST to which he refers are:

  • Give every “thing” an ID

  • Link things together

  • Use standard methods

  • Resources with multiple representations

  • Communicate statelessly

Do read and recommend Stefan Tilkov's Brief Introduction to REST, from which the tenets above are taken. It's excellent. Vish focuses on the third of these -- standard methods and their use -- and highlights the risks of unsafe GET. Unfortunately he's crossed up the methods a bit. The actual uses of the different standard methods are:

  • DELETE: delete a resource

  • GET: get a representation of a resource

  • POST: submit data to a resource

  • PUT: update the state of a resource

There is little lost by thinking of DELETE, GET, and PUT in CRUD terms. One well known use of POST is in resource factories, creating new placemarks or features in a collection, for example:

# Request
POST /features/new HTTP/1.1
Host: gis.example.com
...

[New entity]

# Response
HTTP/1.1 201 Created
Location: http://gis.example.com/features/1
...

Sometimes people refer to this as POST(a) (as in "append"), but POST is more than CRUD's "Create". It can also be used to process data and return it without necessarily creating any new server-side resources. Call this POST(p) (for "process"). Vish is considering POST(p) as a larger volume GET, but that's not quite right because GET requests should be safe and POST requests should be treated as unsafe. Rather than try to switch between GET and POST requests to a single resource (like /features in the example above) depending on the size of the request data, I'd add a new resource to handle POST(p) requests and make it distinct from the ones that are taking POST(a) and GET requests.

Call it a pet peeve, but it's "resource", not "REST endpoint". It's pretty hard to fully grok REST while clutching to the web services endpoint concept.

Update (2008-09-08): I knew Vish had simply transposed entries in his list, and he's fixed up the methods.

Comments

Re: POST(a) and POST(p)

Author: Keyur

Good discussion. ~ Rather than try to switch between GET and POST requests to a single resource depending on the size of the request data, I'd add a new resource to handle POST(p) requests and make it distinct from the ones that are taking POST(a) and GET requests. I don't see the benefit of one against the other. Allowing GET as well as POST to a /query resource v. using /query only for GET and /query/post (?) for POST is probably a subjective call. In both cases you are bending the REST principle of POST-only-unsafe-requests to work around a real world limitation (number of GET characters).

Re: POST(a) and POST(p)

Author: Brian Hamlin

sounds good.. however I have to say that I have just built a RESTful server based on a FeatureServer model, but found that the way Python's CGI handled DELETE prevented me from actually using it.. I ended up using a GET with a do_delete param in it.. so much for the perfect world...

Re: POST(a) and POST(p)

Author: Sean

Python web frameworks generally have incomplete or even poor support for REST. The cgi module is probably the worst of all. Grok's is okay, but makes nice clean URLs too hard. Selector is a neat URL dispatcher for RESTful apps.

Re: POST(a) and POST(p)

Author: Sean

I agree about the subjectivity, Keyur. I've blogged a number of times about query resources as a way to design around the query string limit -- created via a factory (POST(a)) and then accessed safely via GET, but the idea remains unpopular.

The other GeoWeb

I forgot to mention that there is yet a third "GeoWeb": the web of data that links to GeoNames, U.S. Census, etc. While running yesterday I started to sketch out a talk tentatively titled "GeoWeb: fact or fiction?" or "GeoWeb: I want to believe" that could be fun to deliver next year.

On the road

I'm going on a trip next month to help my wife with her meetings at Neobiota, scout for next year's sabbatical, and workshop with the Concordia principals. I'd enjoy meeting up with open source GIS, archaeology, or classics programmers along my route:

  • 24-25 September: Prague, CZ

  • 27-29 September: Colmar, FR

  • 01-08 October: Arles and Montpellier, FR

  • 12-15 October: London, UK

In other words: Praha, Argentorate, Argentovaria, Lugdunum, Arelate, and Londinium, with a side trip to Aquae Sulis. Looks like I could even bump into Andrew Turner.

Comments

Re: On the road

Author: Alban

Hi, Lugdunum was Lyon in France. Not Montpellier. Have a nice journey.

Re: On the road

Author: Sean

Right. I added Lugdunum because we have a layover in Lyon while changing trains, and might do a little bit of sightseeing. Montpellier was not, to my knowledge, itself a Roman or Greek settlement.

Re: On the road

Author: Andy P

Sean, Don't forget the visit to 'Suindune' - no, not Roman or Greek, but its Saxon name. We are at the junction of 2 Roman roads, however...

Re: On the road

Author: Sean

And Suindune! Any traces of the roads, Andy? I'd like to see them.

Will the real "GeoWeb" please stand up?

On the one hand we have a "GeoWeb" that is a kind of social network, one linked not by hypertext (as are the FOAF and XFN networks) but by Tobler's Law, in which geographically adjacent web resources are your best friends (or mortal enemies). I'm sympathetic to this one, although I often disagree with the style of its evangelists. On the other hand is the "GeoWeb" of spatial data infrastructures, geographic entity registries, W*S, and other elements of an information architecture that is older than and still largely disconnected from the Web. Is one of these the real "GeoWeb", or are they just two parts of the same elephant? Can anyone describe how such a beast works?

GIS-Python Lab website update

I've tuned up the http://gispython.org page; adding links to similar enterprises, highlighting the email list, and exposing project news items. Note that I'm using the Python Package Index's Topic :: Scientific/Engineering :: GIS as the canonical list of open source geospatial software for Python. I hope the update is helpful.

Comments

Python Package Index

Author: Kurt

I'll have that GIS tag in my next release of noaadata (vessel tracking utilities etc).

Re: GIS-Python Lab website update

Author: Sean

Kurt, ever consider making it installable from PyPI?
$ easy_install noaadata
Nearly trivial to do if its pure Python, and you can pull in dependencies too.

Friends don't let friends use Endnote

Kurt, I have in fact seen mapping integrated with Zotero. Shekhar Krishnan used the GeoNames database to locate items by their place of publication in a demo at THATCamp.

Comments

Re: Friends don't let friends use Endnote

Author: steven Citron-Pousty

Alright Sean this time you have stepped over the line - dissing Endnote to people who have gone through the PhD process is sacrilege. I swear, with the early days of Word, Wordperfect, Excel, and other random crashing programs Endnote was rock solid and it did just what you needed. I have about 600 entries in Endnote and still love it's capabilities. I agree that it would be nicer if the datastore was not a locked-up format but still... I may have challenge you to a noodle duel when we finally meet in person.

Re: Friends don't let friends use Endnote

Author: Sean

Steve, my point is that now there is an excellent non-proprietary and open source alternative. It imports your existing Endnote files. I wouldn't be surprised if it integrated nicely with Ubiquity.