More fun With Curl and AtomPub

GData is AtomPub, more or less. The GML and GeoRSS namespaces appear in the examples. Have I ever mentioned that I think it's extremely interesting and significant that Google isn't using WFS-T as a gateway for geo-tagged Picasa items?

Plone R-Tree Spatial Index

At the 2006 Plone Conference sprint, Shaun Walbridge and I wrote a Quadtree-based spatial index for Plone. Unlike the portal catalog, it was a localized index, turning a Plone folder of georeferenced content into a shapefile of sorts. It was a nice proof of concept, but was limited by lack of persistence. A few months ago Howard Butler made it possible to persist Rtree indexes on the filesystem, and last week I finally made the time to rewrite the original Plone product into a persistent R-tree index for Zope/Plone data.

SpatialIndex keeps its original design. Adapting a Plone folder to Products.SpatialIndex.interfaces.ISpatialIndex creates an index on disk alongside the ZODB data, with a name that corresponds to the folder's physical path. Content objects can then be adapted to Products.PleiadesGeocoder.interfaces.IGeoItemSimple and added to the index. Ultimately, the index may be queried for the records of items that intersect with a bounding box. The capabilities are summarized in the session below, using a parks folder that contains a lee-martinez document:

>>> parks = app['plone']['parks']
>>> document = parks['lee-martinez']
>>> from Products.SpatialIndex.interfaces import ISpatialIndex
>>> index = ISpatialIndex(parks)
>>> from Products.PleiadesGeocoder.interfaces import IGeoItemSimple
>>> geoitem = IGeoItemSimple(document)
>>> geoitem.setGeoInterface('Point', (-105.08442, 40.59512))
>>> index.add(geoitem)
>>> hits = index.intersects((-106, 40, -105, 41))
>>> [h for h in hits]
[('lee-martinez', (-105.08442, 40.59512, -105.08442, 40.59512)]

SpatialIndex depends on

You should get PleiadesGeocoder 1.0a1 and SpatialIndex 1.0a1 from the repositories:

$ svn co http://icon.stoa.org/svn/pleiades/PleiadesGeocoder/tags/rel-1.0a1\
 PleiadesGeocoder
$ svn co http://svn.gispython.org/svn/primagis/SpatialIndex/tags/rel-1.0a1\
 SpatialIndex

Additionally, SpatialIndex provides a yet-under-construction index management view through which you count the indexed items and reindex folders, and another public view (@@spatialindex) that can be used in various custom forms and pages.

http://sgillies.net/images/manage-index.jpg

Comments

Re: Plone R-Tree Spatial Index

Author: Yves Moisan

This morning on slashgeo an article points at http://www.directionsmag.com/press.releases/?duty=Show&id=19542&trv=1 : "... CartaLens, an innovative geospatial digital asset management ... Unlike other digital asset management solutions that only manage structured content, CartaLens is able to search and retrieve location-based information from both structured content and a broad base of digital content such as photos, video, audio and documents, ..." You mean one can map progress reports like little Word of PDF icons on a map ;-). I don't think it is emphasized enough that the stack you and Kai are building allows all sorts of structured/semi-structured/unstructured content to be viewed on maps now (and without the ArcGIS requirement). It will be interesting to see what data CartaLens sites typically provide and how close we can be with the Zope/Plone/Sean*/Kai* platform. Cheers, Yves

Re: Plone R-Tree Spatial Index

Author: Sean

I've always assumed that one turned the MetaCarta engine loose on networks or storage systems to find information, and that it was rather different from the software I'm developing.

Re: Plone R-Tree Spatial Index

Author: Yves Moisan

I must admit I know nothing about MetaCarta. It just striked me as not particularly groundbreaking news it was to be able to view unstructured content on maps, as I've been doing that (I think ??) with Plone and its various "location products" for many months now. But I may well be underestimating MetaCarta's engine and overestimating Plone.

Re: Plone R-Tree Spatial Index

Author: brentp

sean, somewhat related: is it possible to add an rtree index to a group of shapely objects? it could sidestep a lot of conversions back and forth from python to postGIS if one could have a persistent (pickle/sqlite) group of shapely object with an also persistent index.

Re: Plone R-Tree Spatial Index

Author: Sean

Yes, that can be done. Shapely geometries are pickleable. Make a dict, for example, and store geometries with unique keys. Then index all the stored geometries, hashing the keys to get integer ids for the R-Tree:
  >>> import pickle
  >>> import rtree
  >>> import shapely
  >>> store['geom0'] = shapely.geometry.Point(0, 0)
  >>> index = rtree.Rtree('the_points')
  >>> for key, geom in store.items():
  ...     index.add(hash(key), geom.bounds)
  >>> f = open('the_points.pik', 'wb')
  >>> pickle.dump(store, f)
  >>> f.close()
That would produce a Python shapefile of sorts: the_points.pik, the_points.idx, and the_points.dat.

Re: Plone R-Tree Spatial Index

Author: brentp

ah, too easy! thanks. i'll try it out.

Lines and Polygons in Plone

Update (2007-10-17): the main Pleiades site has the update. See the Barrington Atlas map feed and KML. If you're interested in trying this out, be sure to get revision 1185.

I finally implemented lines and polygons in our PleiadesGeocoder software not so much for display of ancient roads as for making georeferenced feeds that show where we will have new data coming online. Here's an example feed and map, and a snapshot of the KML in Google Earth below:

http://sgillies.net/images/pgc_polys.jpg

Also new is a form for setting the location of any Plone content. Location is still stored in GeoRSS (Simple) form in PleiadesGeocoder (mostly to delay content migration), but the form takes GeoJSON. The new code I'm working on for Plone will be storing locations as GeoJSON geometries.

http://sgillies.net/images/where_tab.jpg

After we get a few bugs out, this will be released as PleiadesGeocoder 0.7 for Plone 2.5, and I'll get back to work on the next generation for Plone 3.

Comments

Re: Lines and Polygons in Plone

Author: Yves Moisan

Can't wait for the Plone 3 work :-). I just got myself an OpenID account to start playing with on a Plone 3 site. Maps next. Cheers,

Rocket City Baloney

Am I ever glad I took a pass (sorry, Tom) on coming down to Huntsville for this. I suppose this is the Bizarro version of Damian Conway's preach-to-the-choir FOSS4G keynote: telling the overdogs of the military-industrial complex (who, when you think about it, have a financial stake in global trouble) exactly what they want to hear.

OpenStreetMap Fort Collins

I didn't know that OSM covered the Fort. The city gives away street data, and that appears to be the primary source, but somebody has added the Spring Creek and Mason bike paths, the bike extension of Centre through the CSU campus, the bike path from the library to the Meridian/Plum intersection (along which I tow my kid to daycare), and the path around City Park Lake. One of these days, I'm going to get off my butt and help out with OSM.

Comments

Re: OpenStreetMap Fort Collins

Author: glenn

very cool... I'm in Fort Collins so its nice to see that effort expand! Glenn (gisuser.com)