Some say the "Geo-Gopher" is surging, and it may well be. Any numbers?

Update (2008-12-11): all the discussion is over here.

Python 3.0 is final and I've pushed my port of Shapely to GitHub: http://github.com/sgillies/shapely-3k/tree/master.

The Shapely tests depend quite heavily on Numpy, and are somewhat broken in the new project, but it seems to work:

sean@lenny:~/code/shapely-3k$ python3.0
Python 3.0 (r30:67503, Dec  4 2008, 12:17:44)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> from shapely.geometry import Point
>>> r = Point(0.0, 0.0).buffer(10.0).exterior
>>> r.geom_type
'LinearRing'
>>> r.length
62.806623139095088
>>> import binascii
>>> binascii.hexlify(r.wkb)
b'01020000004200000000...'

It's strings vs bytes now in Python 3.0, which is better, but demands the change in Shapely idiom shown above.

Two posts from the Zope world caught my eye recently. In one, Kapil Thangavelu writes off OpenLayers too quickly. In another, Chris McDonough cautions people like me against giving up on the ZODB.

OpenLayers is large by default, but can be made more trim. Prune away the vector and raster formats you won't need, and you can get it to a more reasonable size. It's also not true that OpenLayers must pull image tiles from a WMS or TileCache instance. It's my understanding that the EveryBlock project has written a custom image layer that fetches tiles from an HTTP server or proxy, which is what Kapil is looking for. The big imagery projects on my radar are going to involve collaborative annotation, requiring some easy digitization and drawing tools and RESTful protocols for publishing captured features. OpenLayers provides these tools.

Update (2008-12-02): Revisiting Kapil's post I see that Christopher Schmidt has already made my points. Jordan Anderson's comment has more backup for OpenLayers.

A few years ago, a colleague asked me how to lay out a Python project. He was asking about web projects specifically, coming from a PHP background with an established tradition of file and directory structure. I didn't have a recommendation at the time other than to follow the practices of other successful projects. As Python use increases in GIS, and shops move beyond one-off scripts to reusable packages of standard processing code designed to be distributed around their systems, more GIS developers will be asking the same question. My answer is the same, but now I recommend a particular tool for arriving at the answer: paster and Paste Script. In its author's words:

paster is a two-level command, where the second level (e.g., paster help, paster create, etc) is pluggable.

Commands are attached to Python Eggs, i.e., to the package you distribute and someone installs. The commands are identified using entry points.

Paste Script has many more features, but is worth getting for the project creation feature alone. Get it from the Python Package Index: http://pypi.python.org/pypi/PasteScript/1.7.2, or by using easy_install:

$ easy_install PasteScript

The paster program comes with one template for a basic Python package. When executed, it prompts you for package name and metadata. A package of code that put a nice wrapper around arcgisscripting could be created like this:

sean@lenny:/tmp$ paster create -t basic_package
Selected and implied templates:
  PasteScript#basic_package  A basic setuptools-enabled package

Enter project name: biarcgis
Variables:
  egg:      biarcgis
  package:  biarcgis
  project:  biarcgis
Enter version (Version (like 0.1)) ['']:
Enter description (One-line description of the package) ['']:
...
Creating template basic_package
Creating directory ./biarcgis
  Recursing into +package+
    Creating ./biarcgis/biarcgis/
    Copying __init__.py to ./biarcgis/biarcgis/__init__.py
  Copying setup.cfg to ./biarcgis/setup.cfg
  Copying setup.py_tmpl to ./biarcgis/setup.py
Running /usr/bin/python setup.py egg_info

The result:

biarcgis
|-- biarcgis
|   `-- __init__.py
|-- biarcgis.egg-info
|   |-- PKG-INFO
|   |-- SOURCES.txt
|   |-- dependency_links.txt
|   |-- entry_points.txt
|   |-- not-zip-safe
|   |-- paster_plugins.txt
|   `-- top_level.txt
|-- setup.cfg
`-- setup.py

2 directories, 10 files

Let's say your shop has a standard license and authorship policy, wants more package metadata, wants to make sure all packages have a standard test suite, or just prefers a different layout. You can write your own project templates to meet these needs. I like the Pylons wiki example.

Update (2008-12-02): Kurt Schwehr wants to know where is the ChangeLog. The basic package template doesn't provide one, perhaps because the author doesn't want to presume what you want to name your log of changes. The basic Zope and Plone templates from ZopeSkel do provide a docs directory and a HISTORY.txt file for changes. The former even provides a test suite. ZopeSkel is an example of a paster template package that a shop (or community, like Zope's) might make to standardize its Python software.

I've tried with varying degrees of success to express a certain thesis; Aristotle Pagaltzis nails it on rest-discuss:

... Resources derive from the solution domain, not part of the problem domain. Creating resources for concepts that the solution requires is how modeling works in REST terms; they don’t have to derive from any aspect of the problem in order to be justified.

Vish gets it. The problem is that you've got a legacy database and network access to it (SOAP/W*S) sucks (no uniform interface, no caching, etc). The solution: create new web resources that interface easily and cleanly with client-side javascript. These resources don't have to map one-to-one to any particular object or method in your database or framework.

Interestingly, the quote occurs in a thread about "query resources", a notion I've blogged and which remains difficult to swallow for GIS folks.

Do you want better gravy than you had last year? It's not too late to begin tonight.

The first step is to make dark turkey stock from scratch instead of using canned chicken broth. Yes, from scratch. You expected otherwise from this blog? Go to your favorite market or butcher and acquire turkey necks and backs, preferably from natural, free-ranging birds. One of each, minimum, for every 3 cups of the final stock. Then get a big bottle of interesting beer – something like Dogfish Head Theobroma or The Reverend from Left Hand Brewing. It's for you, not the stock. Let's presume you have carrots, celery, onion, white wine, sea salt, whole fresh peppercorns, bay leaves, thyme, parsley.

Open the beer and pour it into a large clean glass. Toss the turkey parts and chopped vegetables with a bit of canola oil and roast, in a pan in the oven, or in a skillet on the stove-top, turning several times, until everything is nicely browned and sticking to the metal. This sticky goo is called the "fond" and you want it like Gollum wants his precious. Go slow if you're new to this, you have a beer for refreshment. Don't burn the precious!

Transfer the browned chunks into a pot. Pour off oil and fat rendered from the bird parts. Don't lose the priceless fond, you want to deglaze it from the pan or skillet using half a cup of white wine, heat, and a wooden spoon or spatula. Transfer the wine and fond to the stock pot, add water to cover, and bring to a simmer. Do not boil the stock. Over the next 30 minutes, skim the scum that rises to the top as you savor your beer. Afterwards, add ground pepper, a bit of salt (mainly to assist the chemistry, you'll salt to taste on T-Day), thyme and parsely sprigs, and leave it simmering (not boiling) alone for an hour and a half while you tidy up the kitchen. Hopefully, you've paced yourself or had the foresight to get 2 beers.

Since you didn't do this on the weekend and are, like me, short of time in the evenings, do a quick wrap-up tonight and finish in the morning. Strain the stock into another pot or ceramic bowl, something you can cover, and set it to chill it overnight. Tomorrow morning, disturbing the settled solids as little as possible, spoon congealed fat off the top (at least 90% of it), and ladle it into a container suitable for freezing. Don't sweat a little cloudiness, you won't notice when it's made into gravy.

To seal the deal on T-Day itself, it's important to have a degreasing cup because the only thing worse than vendor lock-in is greasy gravy. Go get one over the next few days. The larger the better. It's like a measuring cup with a spout coming off the bottom. Pour in a greasy liquid mix, the fat rises to the top, and you carefully pour fat-free liquid from the bottom. Better gravy requires just the right amount of fat and no more. Your grandmother knew this. (Note: if your grandmother is making the gravy this year, disregard this post and just give her a hug and make sure that her beer glass stays full.)

After roasting your turkey, you'll have in the pan several cups of fatty juices and another marvellous layer of precious, precious fond. Spoon off a tablespoon of clear fat for each cup of dark turkey stock. Then add white wine to the pan, heat, and scrape up the fond. It is these resulting juices that you will carefully degrease using your new cup. Bring the turkey stock to a simmer in a saucepan and refill your beer glass.

While the juices are separating in the degreasing cup, make a roux with 4 tablespoons of flour to each 3 tablespoons of clear turkey fat. Stir together over medium heat in a pan. This frying step, without any water, lets the flour absorb fat and will help prevent lumps in the final product. It should bubble, and turn a nice light brown color. Roux is an interesting form of matter; the Cajuns are supposed to have as many kinds of roux as the Inuit do kinds of snow. Let it cool a bit so that it doesn't splatter when you add the stock and whisk the stock in, briskly, a bit at a time. Now added the degreased juices and bring it to a simmer, adjusting the salt and pepper. What ever you do, don't add more flour: thicken it via evaporation if you must. I've seen people try to thicken thin gravy by addition of "Freedom Roux" (flour and water), but you can avoid that by getting the roux right from the start, French style.

Bring it to the table hot, in a classy gravy boat, and indulge.