2009 (old posts, page 5)

OpenLayers Atom

I mentioned a Format/Atom.js file in the previous post. I've overhauled some older code so that it conforms almost completely to RFC 4287 and submitted a patch for OpenLayers: #1366. Download it into a buildout and you can patch it into the latest and greatest OpenLayers like this:

parts = openlayers-format-atom

recipe = zgeo.recipe.openlayers
url = http://openlayers.org/download/OpenLayers-2.8-rc5.tar.gz
patches =
include =

The format's tests give a pretty good overview of its capabilities:

// write entry 6: Atom links
function test_Format_Atom_writeentry6(t) {

    // Feature attributes in Atom namespace
    var atomAttrib = {
          title: "Atom test",
          summary: "An Atom testing feature",
          updated: "2009-06-02T10:00:00Z",
          links: [
            { href: "http://example.com/blog/1", rel: "alternate" }

    var fx = new OpenLayers.Feature.Vector(null, {atom: atomAttrib});
    fx.fid = 'urn:uuid:82ede847-b31a-4e3d-b773-7471bad154ed';

    var writer = new OpenLayers.Format.Atom();
    var data = writer.write(fx);

      // begin document
      '<entry xmlns="http://www.w3.org/2005/Atom">'               +
      '<id>urn:uuid:82ede847-b31a-4e3d-b773-7471bad154ed</id>'    +
      '<link href="http://example.com/blog/1" rel="alternate"/>'  +
      '<summary>An Atom testing feature</summary>'                +
      '<title>Atom test</title>'                                  +
      '<updated>2009-06-02T10:00:00Z</updated>'                   +
      // end document
      'Writes an entry doc with Atom constructs and links'

// write out point -- just enough to see that we're getting the
// georss:where element with a Point. We'll trust GML.v3 to get the
// details right.
function test_Format_Atom_writepoint(t) {

    var point = new OpenLayers.Geometry.Point(-111.04, 45.68);
    var fx = new OpenLayers.Feature.Vector(point, {});
    fx.fid = 'urn:uuid:82ede847-b31a-4e3d-b773-7471bad154ed';

    var writer = new OpenLayers.Format.Atom();
    var data = writer.write(fx);

      // begin document
      '<entry xmlns="http://www.w3.org/2005/Atom">'                 +
      '<id>urn:uuid:82ede847-b31a-4e3d-b773-7471bad154ed</id>'      +
      '<title>undefined</title>'                                    +
      '<georss:where xmlns:georss="http://www.georss.org/georss">'  +
      '  <gml:Point xmlns:gml="http://www.opengis.net/gml">'        +
      '    <gml:pos>45.68 -111.04</gml:pos>'                        +
      '  </gml:Point>'                                              +
      '</georss:where>'                                             +
      // end document
      'Writes an entry doc with a point location'

The patch brings a few new constructs to OpenLayers: Atom categories, persons, and links as javascript objects. The latter have been expressed before in Subbu's blog (http://www.subbu.org/blog/2008/10/generalized-linking), and the former follow. An atom namespace is introduced into a features attributes object to minimize collisions or confusion with other formats (link in GeoRSS.js).

Writing a feature to Atom format is the hard part. Posting to an AtomPub service is easy. With jQuery:

var entry = new OpenLayers.Feature.Vector(geom, data);
var writer = new OpenLayers.Format.Atom();

    async: false,
    type: "POST",
    url: $("link[rel=service]").attr("href"),
    data: writer.write(entry),
    processData: false,
    success: function(msg) {
        alert( "Data Saved: " + msg );
    beforeSend: function(req) {
        req.setRequestHeader("Slug", slug);

OpenLayers buildout recipe

I've just uploaded my first zc.buildout recipe to PyPI: http://pypi.python.org/pypi/zgeo.recipe.openlayers/. It's a recipe for creating custom builds of OpenLayers.js. Based on Kai Lautaportti's hexagonit.recipe.cmmi, it does some of the same useful tricks: working on either OpenLayers distributions (by url option) or local checkouts (by path option), and code patching (by patch-options and patches). Put a list of files in the include option, following the OpenLayers documentation for custom build profiles, and crank out the custom compressed files.

I've put it to use in a project that implements nothing other than the Atom publishing protocol. No layers, no WxS, no strategies. My custom profile buildout configuration is very simple:

parts = openlayers-bingo

recipe = zgeo.recipe.openlayers
path = ${buildout:directory}/src/openlayers
include =

Running it generates the output you may have seen before:

$ bin/buildout -c buildout.cfg
Uninstalling openlayers-bingo.
Installing openlayers-bingo.
openlayers-bingo: Using local source directory: /Users/seang/code/ol-recipes/src/openlayers
Merging libraries.
Importing: OpenLayers.js
Importing: OpenLayers/BaseTypes.js
Importing: OpenLayers/SingleFile.js
Importing: OpenLayers/Util.js
Importing: OpenLayers/BaseTypes/Class.js
Importing: OpenLayers/Format/Atom.js
Importing: OpenLayers/Format/GeoJSON.js

Resolution pass 1...
Importing: OpenLayers/BaseTypes/Bounds.js
Importing: OpenLayers/BaseTypes/Element.js
Importing: OpenLayers/BaseTypes/LonLat.js
Importing: OpenLayers/BaseTypes/Pixel.js
Importing: OpenLayers/BaseTypes/Size.js
Importing: OpenLayers/Console.js
Importing: OpenLayers/Feature/Vector.js
Importing: OpenLayers/Format/GML/v3.js
Importing: OpenLayers/Format/JSON.js
Importing: OpenLayers/Format/XML.js
Importing: OpenLayers/Geometry/LineString.js
Importing: OpenLayers/Geometry/MultiLineString.js
Importing: OpenLayers/Geometry/MultiPoint.js
Importing: OpenLayers/Geometry/MultiPolygon.js
Importing: OpenLayers/Geometry/Point.js
Importing: OpenLayers/Geometry/Polygon.js
Importing: OpenLayers/Lang/en.js

Resolution pass 2...
Importing: OpenLayers/Feature.js
Importing: OpenLayers/Format.js
Importing: OpenLayers/Format/GML/Base.js
Importing: OpenLayers/Geometry.js
Importing: OpenLayers/Geometry/Collection.js
Importing: OpenLayers/Geometry/Curve.js
Importing: OpenLayers/Geometry/LinearRing.js
Importing: OpenLayers/Lang.js

Resolution pass 3...
Importing: OpenLayers/Format/GML.js
Importing: OpenLayers/Format/WKT.js
Importing: OpenLayers/Marker.js
Importing: OpenLayers/Popup/AnchoredBubble.js

Resolution pass 4...
Importing: OpenLayers/Events.js
Importing: OpenLayers/Icon.js
Importing: OpenLayers/Popup/Anchored.js

Resolution pass 5...
Importing: OpenLayers/Popup.js

Re-ordering files...

Exporting:  OpenLayers/SingleFile.js
Exporting:  OpenLayers.js
Exporting:  OpenLayers/BaseTypes.js
Exporting:  OpenLayers/BaseTypes/Class.js
Exporting:  OpenLayers/Util.js
Exporting:  OpenLayers/BaseTypes/Element.js
Exporting:  OpenLayers/BaseTypes/Size.js
Exporting:  OpenLayers/Console.js
Exporting:  OpenLayers/Icon.js
Exporting:  OpenLayers/Popup.js
Exporting:  OpenLayers/BaseTypes/Bounds.js
Exporting:  OpenLayers/BaseTypes/LonLat.js
Exporting:  OpenLayers/BaseTypes/Pixel.js
Exporting:  OpenLayers/Lang.js
Exporting:  OpenLayers/Popup/Anchored.js
Exporting:  OpenLayers/Events.js
Exporting:  OpenLayers/Format.js
Exporting:  OpenLayers/Lang/en.js
Exporting:  OpenLayers/Popup/AnchoredBubble.js
Exporting:  OpenLayers/Format/JSON.js
Exporting:  OpenLayers/Format/XML.js
Exporting:  OpenLayers/Marker.js
Exporting:  OpenLayers/Feature.js
Exporting:  OpenLayers/Feature/Vector.js
Exporting:  OpenLayers/Format/WKT.js
Exporting:  OpenLayers/Geometry.js
Exporting:  OpenLayers/Geometry/Collection.js
Exporting:  OpenLayers/Geometry/Point.js
Exporting:  OpenLayers/Geometry/MultiPoint.js
Exporting:  OpenLayers/Geometry/Curve.js
Exporting:  OpenLayers/Geometry/LineString.js
Exporting:  OpenLayers/Geometry/LinearRing.js
Exporting:  OpenLayers/Geometry/MultiLineString.js
Exporting:  OpenLayers/Geometry/Polygon.js
Exporting:  OpenLayers/Geometry/MultiPolygon.js
Exporting:  OpenLayers/Format/GML.js
Exporting:  OpenLayers/Format/GeoJSON.js
Exporting:  OpenLayers/Format/GML/Base.js
Exporting:  OpenLayers/Format/GML/v3.js
Exporting:  OpenLayers/Format/Atom.js

Total files merged: 40
Compressing using jsmin.
Adding license file.
Writing to OpenLayers.js.

And yields these files, ready to be copied or linked to a location for production:

$ ls -l parts/openlayers-bingo/
total 360
-rw-r--r--   1 seang  staff  180958 Jun  3 15:35 OpenLayers.js
drwxr-xr-x  25 seang  staff     850 Jun  3 15:23 img
drwxr-xr-x   3 seang  staff     102 Jun  3 15:23 theme

Oh, and collective.checkdocs is great for finding bugs in your package's long descriptions.

Jack Camel

Apparently, my oldest daughter will be given the ESRI koolaid in 6 years. Owning the GIS curriculum in US colleges isn't locking in future revenue streams enough already? At least her home-schooling in Python won't be totally wasted. Perhaps that's even the free and open-source technology the author is talking about: Python is certainly the most accessible language supported by Google and ESRI.

Via Adena Schutzberg, who reads the Coloradoan so I don't have to. Title reference.


Re: Jack Camel

Author: so sad

i'm so sorry ... everyday math .. and now this.

Geo + AtomPub

Google Maps Data API. We have been telling you AtomPub would be a pretty big deal for a couple years now. Have you been working on your HTTP chops in that time?


Re: Geo + AtomPub

Author: Jason Birch

Yeah, it's cool. I like the OpenSearch integration.

I'm not sure I like that public feature feeds need to be wrapped in Atom and accessed via a Google Auth. Maybe I'm missing something and there are other public ways to get straight KML (and other?) representations.

I'd have liked to see a JSON-based alternate--working with XML makes me itchy--but I guess I can't really expect that with the strong investment in GDATA.

Re: Geo + AtomPub

Author: Sean

Well, there are the http://maps.google.com/maps/ms?... URIs for KML via the MyMaps permalinks. You'd think they'd be linked to from the map feeds unless they've something else in the works.


Some reactions on the web to the Google Maps Data API announcement ...

Me (paraphrasing):

Dammit, I told you to pay attention to this stuff.

Raj Singh:

Google announced a data API for Maps this morning in San Jose. This is basically a CRUD service for storing geodata in “the cloud” that leverages Atom in lots of ways. That didn’t sound very world-shaking to me at first since there aren’t even any basic spatial query functions, but there are some ways in which this could be a game-changing service — if you trust Google to be your data custodian.

Simon Willison:

I’m disappointed by this one—it’s really just a CRUD store for the KML files used in Google MyMaps. It would be a lot more useful if it let you perform geospatial calculations against your stored map data using some kind of query API—a cloud service alternative to tools like PostGIS.

Brady Forrest:

Geo data can get very large very quickly. Serving it can get expensive. This Data API will help NGOs, non-profits and developers make their data available without breaking the bank. Google's goals for doing this are obvious. If the data is on their servers they can index it easier and make it readily available to their users. There will be concern that Google will have too much of their data, but as long as Google does not block other search engines and allows developers to remove their data I think that this will be a non-issue.

Fake Ed Parsons:

Come and learn how lat49 and geocommons no longer have business models at the Google Booth

Diving into Shapely 1.1

Update (2009-09-16)

We're changing the future release version for this work from 1.1 to 2.0 to make space for smaller scope changes in the interim. See http://sgillies.net/blog/934/updates-to-the-shapely-roadmap.

The code that will become Shapely 1.1 is mostly in working (but not by any means production ready) order and ready for testing by folks that are interested in prepared geometries or new plugins. Documentation and doctests are not up to date, though the unit tests are largely passing, and with good coverage of most modules. It's not been tested on Windows at all, but might work with some coaxing.

Coordinates now are stored primarily as Python arrays of floats. GEOS geometries are created as needed and cached, with some positive and negative effects on performance. Python access to coordinates is much more efficient than it has been. Chaining of operations is less efficient because we're unnecessarily creating Python arrays for anonymous geometries. For example, the following code

p = Point(0.0, 0.0)
b = p.buffer(10.0)
x = list(b.exterior.coords)

benchmarks like this (in microseconds):

1.1a1:   873 usec/pass
1.0.12:  957 usec/pass

but this code, with operation chaining

p = Point(0.0, 0.0)
a = p.buffer(10.0).exterior.convex_hull.area


1.1a1:  1931 usec/pass
1.0.12:  271 usec/pass

I have some ideas about how not to load Python arrays in the case of anonymous geometries that should address this issue.

Shapely 1.1 has a new architecture. Most users won't even notice, but methods of geometries now call on entry points of plugin packages. The default plugin package is shapely.geos, but you can switch via a function in shapely.implementation as you need.

There are a few known bugs and changes to look out for:

  • Adaptation of the geo interface is broken (no asShape()).
  • Heterogeneous geometry collections are broken.
  • Prepared geometry module shapely.prepared needs to be moved to shapely.geos.prepared (only available if you have GEOS 3.1 anyhow).
  • We're now expecting coordinates to come from Numpy as arrays of x's, y's, and z's which means that Shapely integrates with Numpy in the same manner as matplotlib.

To dive in and try it out (with GEOS 3.0+), I suggest you make a fresh Python 2.5 virtualenv and easy_install the shapely.geos package

$ virtualenv --no-site-packages try-shapely
$ cd try-shapely
$ source bin/activate
(try-shapely)$ easy_install -i http://gispython.org/dist/index shapely.geos

Being a dependency of shapely.geos, Shapely is automatically fetched. If you'd like to hack on them (patches welcome!), check the code out and install them into the virtualenv in development mode

(try-shapely)$ mkdir src; cd src
(try-shapely)$ svn co http://svn.gispython.org/svn/gispy/Shapely/trunk Shapely
(try-shapely)$ cd Shapely
(try-shapely)$ python setup.py develop
(try-shapely)$ cd ..
(try-shapely)$ svn co http://svn.gispython.org/svn/gispy/shapely.geos/trunk shapely-geos
(try-shapely)$ cd shapely-geos
(try-shapely)$ python setup.py develop
(try-shapely)$ cd ../..

Despite all the changes inside, the interface (with a few exceptions) is the same as 1.0:

(try-shapely)$ python
Python 2.5.2 (r252:60911, Dec 23 2008, 09:29:43)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from shapely.geometry import Point
>>> p = Point(0.0, 0.0)
>>> p.wkt
'POINT (0.0000000000000000 0.0000000000000000)'

Future for geospatial on App Engine?

Having finally made the time and excuse to develop (virtualenv + Pylons) and deploy (with appengine-monkey) a very simple application on Google's App Engine, I'm thinking a little more about the support GAE could offer for geospatial apps. My keytree package works just fine on GAE (with xml.etree), so there's handy KML reading and writing. Simplejson, minus speed-ups, is good to go with GAE, and therefore so is geojson. Neither contain many files and so there's no harm in including them in each app (Pylons and its dependencies, on the other hand, eat up 80% of your file quota all by themselves). What's lacking is support for geometry operations and predicates, whether in the datastore API (ala GeoDjango/PostGIS), or in Python ala Shapely.

I can't believe that GEOS will ever be a part of GAE, but support for Java opens the door for JTS. I hope Martin Davis or somebody at OpenGeo grabbed one of the offered chances to try Java and will report on it. Would you have to use Jython to get at the JTS operations and predicates, or could you communicate via protocol buffers within your application? I do not know, being new to the platform, but would like to find out. Ideally, this kind of geospatial applications platform would allow you to create your own services for analyzing spatial features and share them with other applications if it wasn't going to provide them itself. Going over HTTP to a JTS-based geometry/topology service hosted in the same cloud would suck.


Re: Future for geospatial on App Engine?

Author: Brian Flood

you can upload both python and java to the same GAE application, so yes they can share the same datastore. the specific access models may differ, but the underlying entities are shared.

fwiw, calling intra-cloud over http is not as bad as you would expect.



Re: Future for geospatial on App Engine?

Author: Martin Davis

This sounds very exciting! I was wondering when Google would add Java support to AppEngine - given their support for Java (eg GWT) it seemed a natural fit.

Not sure when I'll have the cycles to play around with JTS on AppEngine - but I'd be very interested in supporting/collorating/interacting with anyone trying it out.