Shapely 1.0.4

One bug fix and a simple GEOS geometry cache to improve performance when wrapping coordinate data stored outside of Shapely. http://pypi.python.org/pypi/Shapely/1.0.4.

In comments on a post about Javascript geometries, Paul Ramsey and Martin Davis wondered about a pure Python version of JTS. It might be an interesting project, and maybe you could coax reasonable performance out of it with psyco or Cython, but it wouldn't perform as well as something based on C (for C Python) or Java (for Jython) or C# (for IronPython). Consider Ian Bicking's HTML parsing benchmarks: lxml, based on libxml2, is far faster than anything else. The need to calculate spatial operations in C rather than Python is greater yet.

If Google would provide some basic spatial indexing (for App Engine), you could try to go the rest of the distance with the help of an external web geoprocessing service. You might not need a PyTS after all.

Comments

Re: Shapely 1.0.4

Author: Martin Davis

Sean, you must have been reading my mind. I was going to email you and ask if you thought there was any need for a pure Python port of JTS. I guess I'm not surprised to hear that a PyTS wouldn't have the performance of JTS or GEOS. On the other hand, there are lots of times where what is needed is functionality, and performance is a secondary concern. Perhaps the input data is small or well-bounded, or there aren't tight requirements on response time. (Human-facing spatial apps often fit this profile). So I'm not convinced that there wouldn't be an audience for PyTS. Any interest in exploring what it would take and how it might happen?

Re: Shapely 1.0.4

Author: Sean

Send me that email, Martin.

Keytree

Keytree (also known as pleiades.keytree in our repository) is a simple package of ElementTree helpers for parsing KML. I needed it, didn't see anything else like it out there, and put in a little extra work to make it more generally useful. If you're already using ElementTree and are curious, give it a try. It plays well with Shapely, of course:

>>> from xml.etree import ElementTree
>>> tree = ElementTree.parse(open('archaic.kml', 'rb'))
>>> kmlns = tree.getroot().tag.split('}')[0][1:]
>>> placemarks = tree.findall('*/{%s}Placemark' % kmlns)
>>> p0 = placemarks[0]
>>> import keytree
>>> f = keytree.feature(p0)
>>> from shapely.geometry import asShape
>>> shape = asShape(f.geometry)
>>> shape.wkt
'POINT (21.9725000000000001 32.8962999999999965)'

Keytree parses out points, line strings, and polygons, but not yet geometry collections. I'll make a real release of it after I get a chance to work on compatibility with lxml.

Comments

Re: Keytree

Author: Matt Priour

>>> from xml.etree import ElementTree >>> tree = ElementTree.parse(open('archaic.kml', 'rb')) >>> kmlns = tree.getroot().tag.split('}')[0][1:] >>> placemarks = tree.findall('*/{%s}Placemark' % kmlns) >>> p0 = placemarks[0]
Is it possible to have keytree just automatically do the kmlns & placemarks operations you have done here? It seems that is keytree is for KML then it should automatically know those 2 things and they could be implemented as helper/convience(sic) functions rather than always having to manually do that everytime you want to start working with KML feature extraction/import.

Re: Keytree

Author: Sean

Indeed. If KML docs were constrained to a Document/Folder/Placemark structure, I would use the WorldMill model. But KML is more loosely structured than that -- you can have folders of folders, and placemarks and folders as siblings -- and for now I'm leaving the how of getting Placemark elements up to the user (me, for all I know).

Blacklisted

Aha!

Several FRII accounts have been compromised over the past few weeks due to a spoof email sent out to obtain their login info. These compromised accounts, unfortunately, led to the FRII ip being blacklisted on several ISP's. We our hopefully on the tail end of the matter but it sounds like Akismet might be an additional ISP we need to contact. If you haven't attempted to send a message to an Akismet user today, go ahead and attempt to again. If you get a bounce back, please forward the message to customercare@frii.com for our evaluation.

I got confirmation yesterday that Akismet was indeed sending my comments to WordPress blog spam bins. I think this is the first time I've ever been blacklisted.

And now I just received the spoof email:

Dear Frii Email Account Owner,

This message is from Frii messaging center to all Frii email account owners.

We are currently upgrading our data base and e-mail account center.

We are deleting all unused Frii email account to create morespace for new accounts.

To prevent your account from closing you will have to update it below sothat we will know that it's a present used account.

CONFIRM YOUR EMAIL IDENTITY BELOW Email Username : ......... ..... EMAIL Password :................

Warning!!! Account owner that refuses to update his or her account within Seven days of receiving this warning will lose his or her account permanently.

Thank you for using Frii!

I can't believe people still fall for this con.

Comments

Re: Blacklisted

Author: Mark

Sorry - your comments should now be good but any problems get in touch at http://akismet.com

Re: Blacklisted

Author: Allan Doyle

This must be a new standard phishing attack. MIT went through a couple of waves of it a few weeks ago. I don't think we came off it any better, I'm sorry to say.

SpatialIndex 1.3.1

Howard and Marios have released SpatialIndex 1.3.1. This fixes a few memory leaks and gets us a couple steps closer to being ready for production on Windows. From the release notes:

  • Fixed gcc/vc compatibility issues with TemporaryFile.cc
  • Fixed memory leaks in LineSegment.cc
  • Fixed memory leaks in MovingRegion.cc
  • Fixed memory leaks in mvrtree/Node.cc
  • Renamed rand48.hpp to rand48.h

Python Geo Frameworks 2

Here, continuing on from my previous post, are some of the forces arrayed against an all-encompassing OSGeo Python framework:

Scarcity: few develop GIS applications using Python, fewer applications that would benefit from a global library, and fewer yet would contribute to the development of the global library. From my experience, you can't expect more than 3 people at the top of this community pyramid. That's probably not enough. And you can't really count on this number increasing very much in the future. Users want apps and data, not libraries and APIs. See Google Earth. Yes, there are exceptions, but Java, not Python, is their home.

Scale: GDAL, OGR, PROJ4, GRASS, QGIS, MapServer, ... data, analysis, projections, stats, graphical user interfaces, printed and plotted maps, service oriented architecture, ... that's a lot to keep in your head. Designing such a thing is a big job for a small team, not something you knock out in a week unless you join the GeoTools/GeoAPI cargo cult like I did when I started PCL (after which you'll realize that Python programmers don't want GeoTools). Programming in Python is more productive than programming in Java, or .NET, but designing Python libraries isn't any easier. Without a driving business need or lots of itch scratching (which won't happen, as I explained above), the task of designing a global OSGeo Python library is doomed by the size of its scope.

Physics: a Python library that depends on

  • GDAL/OGR (and Xerces, etc)
  • MapServer (and Agg, etc)
  • GRASS
  • QGIS (and Qt, etc)

is like a giant cinema spider, awesome on the drawing board, but in reality something that can't but collapse under its own weight. There are too many dependencies, too many of which have no current binaries, with incompatible Python bindings written by hand, or generated by SWIG or SIP. There's also a rather large amount of functional overlap among these projects, governance of which is something that GeoTools (which I'm continuing to hold up as a successful holistic library) doesn't have to be concerned about. I suspect the social and software engineering details haven't fully dawned on OSGeo developers yet. These details are not an itch I need to scratch, they are a patch of poison oak I need to avoid falling into.

Well, that's the verbose edition. The shorter history of PCL: it's brutally hard and there is precious little reward for the designer. I feel like there's more to write on Python and geospatial or GIS frameworks, especially with reference to the Python web framework debates, but that's for another day.

Comments

Re: Python Geo Frameworks 2

Author: brentp

sean, thoughts on how this fits in with the buffer protocol or geo-interface? as someone who fits in your categorization of a python programmer who doesnt want a framework, those seem like reasonable alternatives--to allow passing data between various distinct components.

Re: Python Geo Frameworks 2

Author: Timmie

Hi Sean, I really like this post. Good addition to what was discussed on the list. I'd be interested to see more posts on this topic. Integration/interfacing of the diefferent pythonic geo tools. Regards!

GeoJSON 1.0a4

GeoJSON, the Python project, is a reference implementation of GeoJSON, the format specification, and also of the Python geo interface. Matthew Russell and I have brought it up to date with the current draft version of the spec and uploaded 1.0a4 to PyPI.

Before the final 1.0, I think we need to rename this. The package name, "geojson", is fine, but a new project name would invite less confusion with the specification project.

WordPress Commenting Trouble

My comments on WordPress blogs this week are vanishing without any confirmation or trace. Dan Cohen's Digital Campus, Stephen Ramsay, and now Andrew Turner. Nothing. I'll bundle them together here as a last resort:

  • Digital Campus episode 25: good stuff! Anybody who is wondering whether they'd better start porting their PHP apps to Python should read Ian Bicking's recent posts about PHP and Python deployment.
  • Stephen Ramsay: I have the same reaction to "My" sites. That word can be almost like a code smell.
  • Andrew Turner: better yet curl -X PUT ... ;) I'm not a Twitter user, but still, there's an HTTP method dedicated to updating the state of a web resource.

Comments

Re: WordPress Commenting Trouble

Author: James Fee

What am I doing wrong that allows you to continue commenting on my blog?

Re: WordPress Commenting Trouble

Author: Sean

I don't know. Are you an update behind?

Re: WordPress Commenting Trouble

Author: James Fee

I'm on the latest version, you probably are just getting in the SPAM folder.

Python Geo Frameworks

Got an email today with a link to an OSGeo Python Library proposal:

Several OSGeo software projects support Python. However, a global abstraction layer is lacking which would help to do "OSGeo Python programming". We think of well documented bindings to the various software projects which are handled as plugins (or whatever appropriate).

I'm going to write more on this topic in the near future, but let me begin simply by warning that those who do not learn from the history of PCL are doomed to repeat it.

Comments

Re: Python Geo Frameworks

Author: Paul Ramsey

And the lessons are?...