PyCon conference day ~0

It's the last day of PyCon for me. This means I'll miss the 4 days of sprints, which I regret and intend never to miss again – this event is just getting started for many attendees – but I need to get back and resume sprinting on house remodeling and preparing to move.

I'm very happy that I decided to volunteer to be a session runner. I got to help David Beazley, the session chair, a programmer and teacher whom I've referenced many times on my blog. I've been using his software (including SWIG) for many years. It's neat to be able to tell someone, in person, that you admire and have profited from their work, right? I got to meet speakers: Jason Scheirer, Zain Memmon, Ricardo Kirkner. I got to meet Barry Warsaw, the GNU Mailman lead, in the Green Room. Mailman! And Alex Martelli. And Paul Smith. And conference staff, volunteers all, like Doug Napoleone and Anna Martelli Ravenscroft. Want to meet people serendipitously? Volunteer to chair or run a PyCon session.

I met a bunch of other great folks that I've only met online, you know who you are: thanks for tracking me down or allowing me to track you down.

Permission or Forgiveness

A few minutes ago Alex Martelli gave a talk on "Permission or Forgiveness?"

Grace Murray Hopper's famous motto, "It's easier to ask forgiveness than permission", has many useful applications -- in Python, in concurrency, in networking, as well of course as in real life. However, it's not universally valid. This talk explores both useful and damaging applications of this principle.

Howard Butler clued me in to this motto years ago. I'd like to read more about Grace Hopper. She invented a compiler in 1952 and here's the cover of a book about its language.

http://farm8.staticflickr.com/7070/6822097856_a18f15602f_z_d.jpg

Martelli pointed out that just because it's easier to ask forgiveness doesn't make it always ethical or right to not ask permission instead. Use of the imagery above falls into the category of cases in which you ask for permission. The Computer History Museum readily grants permission, as long as I mention the following:

Image courtesy of Computer History Museum.

The cover is masterpiece of midcentury (mid-20th, that is) graphic design: the waves, the typography, and most of all, our friend the atom.

Comments

Re: Permission or Forgiveness

Author: Martin Davis

Nice post, Sean.

Gotta love that postwar starry-eyed optimism about science - "if it has atoms it must be great!".

Also gotta love the title - "Automatic Programming". All we have do is to sit back and watch the teletype churn away! Ironic that Grace Hopper was apparently also the originator of the expression "bug".

Zen of Python vector data processing

Just for fun. You're tried import this in the Python interpreter, right?

>>> from fiona import this_bis
The Zen of Python Vector Data Processing, by Sean Gillies

Files are just files.
A record that's read stays read.
Objects are good, but data is better.
Record types aren't special enough to break the rules.
Functional programming is a great idea -- let's do more of that!

Source: https://github.com/sgillies/Fiona/blob/981b78ed22677efbb6c918c62574a832b1b50f85/src/fiona/this_bis.py

Feedparser and GeoRSS/GML

Remember GeoRSS? It's no longer the new hotness, but it's still around. There's a question about parsing the format with Python on GIS Stack Exchange which got me to dust off an old project.

In 2007, I submitted a patch for feedparser. It parsed both the simple and GML flavors of GeoRSS into GeoJSON [1] style dicts. It had a usage example. It had tests! Nevertheless, the maintainer of feedparser dismissed it because (I paraphrase) "what does JSON have to do with RSS?"

This is the only WONTFIX that has ever really gotten under my skin. To scratch this itch for good, I've patched feedparser 4.1 and, until I come up with a better idea, am distributing it from my public Dropbox folder: http://dl.dropbox.com/u/10325831/feedparser-4.1-georss.tar.gz. To install:

$ pip install http://dl.dropbox.com/u/10325831/feedparser-4.1-georss.tar.gz

or download it any way you want and install using the setup.py script in the tarball.

It's dead simple to use.

>>> import feedparser
>>> feed = feedparser.parse("http://earthquake.usgs.gov/earthquakes/catalogs/1hour-M1.xml")
>>> feed.entries[0]['where']
{'type': 'Point', 'coordinates': (-122.8282, 38.844700000000003)}

[1] The little format that could. http://geojson.org.

Ongoing blog series

I've been writing some posts under the rubric of "geoprocessing for humans". These are generally about keeping software simple, predictable, symmetrical, safe, readable, and well-documented. And simple. Most of all: simple.

A couple of my recent posts are a little different, being about functional programming, partial functions, processing entire files of records without using any for loops, etc – stuff that a GIS programmer/analyst might not recognize as scripting or as even practical for all I know. I think I'll categorize these as "geoprocessing for hipsters". I don't usually begin coding in this way, but often arrive here after a few iterations.

If you find these rubrics fun and/or pedagogical, jump on in.

Geoprocessing for humans: close() and with

Fiona 0.8 has a bunch of important new features: file bounds/extents, sync of bounds and record count with writes, protection against iteration restarts, and more:

0.8 (2012-02-21)
----------------
- Replaced .opened attribute with .closed (product of collection() is always
  opened). Also a __del__() which will close a Collection,  but still not to be
  depended upon.
- Added writerecords method.
- Added a record buffer and better counting of records in a collection.
- Manage one iterator per collection/session.
- Added a read-only bounds property.

I've also expanded the manual (still under construction), documenting among other things what Fiona does to keep users from stumbling over external resources: http://toblerity.github.com/fiona/manual.html#closing-files. Don't del your collections or set them to None and then sacrifice a chicken while crossing your fingers and hoping for the best – just call close() or use with and enjoy some peace of mind.

Update (2012-02-22): "layers" (as in OGR) in the last sentence changed to "collections" (as in Fiona).

Connecting many places

Things you find in Pleiades that you don't find in a typical geographic information system include relationships between places that are expressed in the data itself. I blogged about this last summer (accompanying figure reprised below) and talked about it at the 2011 Digital Humanities conference (our poster here).

http://farm6.static.flickr.com/5312/5885119283_fe08cf3758_z_d.jpg

Until today, connections between places have been a little sparse. Loading 200+ milecastles and turrets with connection to Hadrian's Wall changes the situation at least in Britannia. The representation of Hadrian's Wall in Pleiades doesn't have a published spatial extent of its own, but gets one by virtue of its connections to these other small places. Here is a screenshot from Google Maps.

http://farm8.staticflickr.com/7169/6838202849_73308ba375_b_d.jpg

Here's a closeup near Brampton. The lone placemark at the bottom represents an old quarry that supplied nearby milecastles with rock. For the moment at least, we're asserting that the quarry was connected to the fortifications.

http://farm8.staticflickr.com/7172/6838202859_b19995f36a_b_d.jpg

The remains of Hadrian's Wall are a popular hiking itinerary today. The connected places in the maps above don't quite describe the itinerary because they aren't chained to each other, but I can't stop thinking that we should be making it possible to represent ancient itineraries like the Antonine using places from Pleiades.