In Rtree news
This morning Howard Butler announced an upcoming Spatialindex 1.6.0 release: http://lists.gispython.org/pipermail/spatialindex/2010-September/000232.html, to be followed by an Rtree release. I'm becoming more interested in seeing libspatialindex currently packaged for Linux distributions, particularly Ubuntu. Lucid has 1.4.0. I wonder if I can parlay co-working with a Launchpad developer into a fast track for 1.6.
After a long time away I'm using Rtree (0.6) in anger this week. I love what Howard and Brent Pederson have done with the code and docs. Try out
and you'll see what I mean. I've also uploaded new HTML docs (by Howard, using Sphinx). Let us know if you find any errors or unclear directions.
My own Rtree-using project is a simple, batching index for GeoJSON feature-like objects. Deserialized GeoJSON goes in, deserialized GeoJSON comes out. The interface is shown by this base class [indexing/__init__.py#L45]:
class BaseIndex(object): """Base class for indexes Deals in GeoJSON-like dicts we will call ``items``, like: {'id': 'db441f41-ec34-40fb-9f62-28b5a9f2f0e5', # required 'bbox': (0.0, 0.0, 1.0, 1.0), # recommended 'geometry': # required {'type': 'LineString', 'coordinates': ((0.0, 0.0), (1.0, 1.0))}, 'properties': { ... } # optional ... } """ def intid(self, item): """Return a unique integer id for the item""" raise NotImplementedError def bbox(self, item): """Return a (minx, miny, maxx, maxy) tuple for the item""" return bbox(item) def intersection(self, bbox): """Return an iterator over items that intersect with the bbox""" raise NotImplementedError def nearest(self, bbox, limit=0): """Return an iterator over the nearest N=limit items to the bbox""" raise NotImplementedError def index_item(self, itemid, bbox, item): """Index item using unique integer itemid and bbox as key""" raise NotImplementedError def unindex_item(self, itemid, bbox): """Unindex the item with (itemid, bbox) as key""" raise NotImplementedError
Some of you (2 people, tops) will recognize this from zope.catalog. My first implementation is an index that uses an instance of Rtree as a forward mapping and an instance of OOBTree.OOBTree (from the ZODB package) as a backward mapping (item values and id/bbox keys) [vrtree/__init__.py#L10].
class VRtreeIndex(BaseIndex): """An index with an R-tree as the forward mapping and a B-tree as the backward mapping Modeled after the example at http://docs.zope.org/zope.catalog/index.html. """ def clear(self): self.fwd = Rtree() self.bwd = OOBTree.OOBTree() def __init__(self): self.clear() def intid(self, item): """Rtree requires unsigned long ids. Python's hash() yields signed ints, so we add 2**32 to hashed values for the former.""" return hash(item['id']) + OFFSET def intersection(self, bbox): """Return an iterator over Items that intersect with the bbox""" for hit in self.fwd.intersection(bbox, True): yield self.bwd[(hit.id, tuple(hit.bbox))] def nearest(self, bbox, limit=1): """Return an iterator over the nearest N=limit Items to the bbox""" for hit in self.fwd.nearest(bbox, limit, True): yield self.bwd[(hit.id, tuple(hit.bbox))] def index_item(self, itemid, bbox, item): """Add an Item to the index""" if (itemid, bbox) in self.bwd: self.unindex_item(itemid, bbox) self.fwd.add(itemid, bbox) self.bwd[(itemid, bbox)] = item def unindex_item(self, itemid, bbox): """Remove an Item from the index""" value = self.bwd.get((itemid, bbox)) if value is None: return del self.bwd[(itemid, bbox)] self.fwd.delete(itemid, bbox)
I think this is the first time I've ever written code using a tuple as a mapping key, but it's a perfect fit for an R-tree that takes a tuple of integer id and bounding box as a key. This index will be given a persistence wrapper and fronted by a Tornado based service that accepts JSON data and dispatches to index methods. A "Spatial Solr", if you will, which I attempt to describe in the README.
That's not the only possible implementation. One that needed to run on App Engine (for example), where there's no C-dependent Rtree, might use the App Engine datastore for backward mappings, and load an instance of pyrtree.RTree from the former on startup; the mapping serving as persistence for an R-Tree that doesn't have its own persistence mechanism.
Naming projects gets harder and harder. "Vaytrou" is a verlan play on the verb trouver (to find). "Vétrou" would be more authentic, but GitHub refused.
Update (2010-09-17): Here's the Vaytrou server. Simple. Handlers to accept batches of items and perform queries are in the works.
Update (2010-09-17): Spatialindex 1.6.0 is now available from links at http://trac.gispython.org/spatialindex/wiki/Releases.
Comments
Re: In Rtree news
Author: René-Luc D'HONT
Now, I understand the 'Vaytrou' name!!! I initially think about 'way true'!!! I'd like to work with Tornado, if it's possible!
Re: In Rtree news
Author: Sean
Have you forked Vaytrou yet? It would be fun to work with you on it.
Re: In Rtree news
Author: whit
Nice to see the standalone http rtree index resurface! and yeah, I did recognize it as zope.catalog. Might take a stab at running it with gevent.