I've tagged 0.12 and uploaded an sdist to PyPI: https://pypi.python.org/pypi/Fiona/0.12. Changes:

• Fix broken installation of extension modules (#35).

• Log CPL errors at their matching Python log levels.

• Use upper case for encoding names within OGR, lower case in Python.

The 0.11 release was broken and I yanked it, but it had some nice new features and bug fixes, too:

• Cythonize .pyx files (#34).

• Work with or around OGR's internal recoding of record data (#35).

• Fix bug in serialization of int/float PROJ.4 params.

A couple of OS X users who are satisfying Fiona's GDAL requirements with the frameworks from kyngchaos.com have contacted me about this import error:

>>> import fiona
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/kilroy/Desktop/fiona_env/lib/python2.7/site-packages/fiona/__init__.py", line 70, in <module>
    from fiona.collection import Collection, supported_drivers
  File "/Users/kilroy/Desktop/fiona_env/lib/python2.7/site-packages/fiona/collection.py", line 7, in <module>
    from fiona.ogrext import Iterator, Session, WritingSession
ImportError: dlopen(/Users/kilroy/Desktop/fiona_env/lib/python2.7/site-packages/fiona/ogrext.so, 2): Symbol not found: _CPLSetThreadLocalConfigOption
  Referenced from: /Users/kilroy/Desktop/fiona_env/lib/python2.7/site-packages/fiona/ogrext.so
  Expected in: flat namespace
 in /Users/kilroy/Desktop/fiona_env/lib/python2.7/site-packages/fiona/ogrext.so

This is a sign that you need

DYLD_LIBRARY_PATH=/Library/Frameworks/GDAL.framework/GDAL

in your environment, whether that's a virtualenv or set in your .profile.

Not only was I out of town during the days the vultures usually return to Fort Collins, I no longer live around the corner from the big roost on Mountain Ave. Last May, my family and I left Old Town for the Sheely Addition and will no longer be able to keep such close tabs on the Harbingers of Spring. I think this may be the last post in this series.

I did see Turkey Vultures during my family trip to Zion National Park last week, but I did not see what I was hoping to see: a California Condor. Rangers told us that Condors have been sighted in the park already this Spring, but I saw not a one despite being outdoors looking up every moment from sunrise to sunset. Until next time I'll have to make do with photos like this one by ChrisWegg:

Previously: 2012, 2011, 2010, 2009, 2008.

I've tagged 0.10 and uploaded an sdist to PyPI: https://pypi.python.org/pypi/Fiona/0.10.

Changes:

• Add function to get the width of str type properties.

• Handle validation and schema representation of 3D geometry types (#29).

• Return {'geometry': None} in the case of a NULL geometry (#31).

I've also updated the documentation with an explanation in the manual of how "3D" geometries are read and written.

It's tagged and uploaded to PyPI.

Here's a quick example that demonstrates a couple new features: support for variable fixed text field width and mixing of single and multipart one and two-dimensional geometric objects (linestrings and multilinestrings; polygons and multipolygons) in Shapefiles.

>>> c = fiona.open('ne_110m_geography_regions_polys.shp', 'r')

The fiona.open function is the new way to get an open collection, but the old fiona.collection function still works. Keep in mind that the first parameter of fiona.open (the path) is the only positional parameter. All others are keyword parameters, but it's fine to treat the mode, the 2nd parameter in the function definition, as a positional parameter as I did above.

>>> c.name
'ne_110m_geography_regions_polys'
>>> c.mode
'r'
>>> c.closed
False

A collection has a few file-like properties.

>>> c.crs
{'no_defs': True, 'ellps': 'WGS84', 'datum': 'WGS84', 'proj': 'longlat'}
>>> c.driver
'ESRI Shapefile'
>>> import pprint
>>> pprint.pprint(r.schema)
{'geometry': 'Polygon',
 'properties': {u'featurecla': 'str:32',
                u'name': 'str:254',
                u'namealt': 'str:254',
                u'region': 'str:50',
                u'scalerank': 'float',
                u'subregion': 'str:50'}}

The collection describes itself as containing records with polygon type geometries and six properties. Five of those properties are str (text) type, with maximum lengths of values being 32 ('str:32') to 254 ('str:254') characters.

>>> records = list(r)
>>> len([r for r in records if r['geometry']['type'] == 'Polygon'])
52
>>> len([r for r in records if r['geometry']['type'] == 'MultiPolygon'])
8

There are 52 records with polygon type geometries in this file and 8 with multipolygons.

All the Fiona examples, including the one only for functional programming geeks, are updated for version 0.9. So is the manual.

I've been remiss in pointing out a couple good blog posts about using Fiona. Tom MacWright wrote GIS with Python, Shapely, and Fiona last fall (widely linked, Tom's reach is huge), and Steve Citron-Pousty wrote Using Open Source GIS tools for spatial data - QGIS, GDAL and Python just the other day. I'm excited to see creative and super productive programmers putting Fiona to work and writing about it.

Last night, I saw that Tom MacWright had posted a script for combining Shapefile fields. I thought I'd try to adapt it to Fiona 0.9 and a slightly more functional (FP, I mean) style, and I'm glad I did because I in the process I discovered that my slightly old version of GDAL and OGR isn't detecting the Windows-1252 encoding of the Natural Earth dataset. Version 0.9.1 of Fiona is now tagged and uploaded to PyPI and it lets a user specify the proper encoding of files if needed.

My version of Tom's script tries to be smart about the width of the new text field and warns if the new values will be truncated. The combining is also done within a function, which allows all output to be written in one statement. Our versions of the script run equally fast.

import fiona
import itertools
import logging
import sys

logging.basicConfig(stream=sys.stderr, level=logging.WARN)

def text_width(val):
    return int((val.split(":")[1:] or ["80"])[0])

with fiona.open(
        '/Users/seang/Downloads/ne_50m_admin_0_countries/'
        'ne_50m_admin_0_countries.shp',
        'r',
        encoding='Windows-1252') as inp:

    output_schema = inp.schema.copy()

    # Declare that the output shapefile will have a new text field called
    # a3_sov, with an appropriately sized field width. 254 is the maximum
    # for this format.
    width = min(254,
        text_width(inp.schema['properties']['sov_a3']) +
        text_width(inp.schema['properties']['sovereignt']))
    output_schema['properties']['a3_sov'] = 'str:%d' % width

    # Define a function that combines properties to produce a value
    # for the 'a3_sov' property. It warns if the value will be truncated
    # at 254 characters.
    def combine_fields(rec):
        val = rec['properties']['sov_a3'] + rec['properties']['sovereignt']
        if len(val) > 254:
            log.warn("Value %s will be truncated to 254 chars", val)
        rec['properties']['a3_sov'] = val
        return rec

    with fiona.open(
            'ne_50m_admin_0_countries_a3_sov.shp', 'w',
            crs=input.crs,
            driver=input.driver,
            schema=output_schema) as out:

        out.writerecords(
            itertools.imap(combine_fields, inp))

I'm working on submissions for the upcoming FOSS4G-NA conference (aka MapServer User Meeting XI) and am experimenting with opening the abstracts up to comments and forks as I write them at https://gist.github.com/sgillies/4745903. If you like the direction I'm going with the abstracts, feel free to comment. If it looks like I'm overlooking something important, please comment. If you're interested in co-presenting please fork the Gist and we'll talk. The results will go into the FOSS4G-NA submission spreadsheet on the 14th.

I've tagged 1.2.7 and uploaded an sdist to PyPI. There's an important bug fix for Windows users in this one. Please heed the warning I've attached to the README, dear folks.

The highlight of 1.2.17 is the module of affine transformation functions and utilities written by Mike Toews: shapely.affinity. The manual isn't up to date yet, but the module is itself very well documented. Here's a brief example of the rotate function, which calls the module's more general affine_transformation function.

# Default origin is the center of the geometry's bounding box.
# Default unit of angle is the decimal degree, positive is counter-clockwise.
# a) 90°, default origin
from shapely.affinity import rotate
rotated = rotate(line, 90.0)

# b) 90°, origin: (0, 0)
rotated = rotate(line, 90.0, origin=(0.0, 0.0))

In the figures below, the original line is shown in low alpha grey, the rotated line in blue, and the origin of rotation as a point.

I'm pleased to be able to say that 6 different programmers contributed to this release. I think this Shapely package just might catch on after all.

Update (2013-01-28): Windows binaries by Allan Adair: http://pypi.python.org/pypi/Shapely#downloads.

It's been a warm week here in the Fort. The snow is gone and the ground is a bit muddy. Wednesday, I rode my bike to the Pineridge Natural Area trailhead at Cottonwood Glen Park with the intent of running, but found the trail closed. I appreciate that the City is looking out for its trails, but was nonetheless a bit disappointed. Is it open today? I'd probably have to call to find out. If only there was something like a live StreetView so I could check if the gate is open or closed...

Which reminds me that last June I saw a Google StreetView car coming out of the parking lot at the south end of Overland Drive as I came down the Pineridge trail and through the gate. I reported at the time:

The sweaty guy in black with the blurry face running down from Pineridge in a future Street View update? Me.

So I checked and last summer's imagery is now live (I'm not sure when it was updated – my neighborhood's imagery is from August, and probably wasn't the last to be driven, so maybe only quite recently). The sweaty closeups of me are mercifully omitted (Overland dead ends here and there must have been double coverage as the car drove in and back out) but I am clearly visible walking back to my bike as the StreetView car leaves.

Oh, and I have my Twitter archive, too, which is how I tracked down that old post.

Why didn't anybody tell me about Krampus earlier?!

If you ask me, Dad is enjoying the scene a little too much.

Have you heard of Flickr, the photo sharing site with Creative Commons licensing options and a great API for access to photos and metadata? I ask because our collective attention span is so short that I wonder whether our memory hasn't been similarly impaired. Flickr has weathered harsh criticism from former employees and speculation about the number of days until Yahoo kills it, and has regained for now the fickle favor of the web. A year ago, I wrote about Flickr's extra love for Pleiades machine tags: http://sgillies.net/blog/1103/flickr-support-for-ancient-world-places. I'm going to take the opportunity of Flickr's comeback to write about what's new in tagging photos with Pleiades places.

Flickr reports almost 7000 photos with pleiades:*= tags: http://www.flickr.com/photos/tags/pleiades:*=/. Of these, over 4800 use pleiades:depicts= to indicate that they depict the site(s) of an ancient place; the ancient place, or what remains of it, is the subject of the photo. Over 1500 photos use pleiades:findspot= to indicate that the subject of the photo was found in the context of the ancient place. The pleiades:origin= tag is another interesting one and Flickr has almost 200 of these.

My favorite is the pleiades:atteststo= tag for photo subjects that bear the name of or otherwise attest to the existence of an ancient place. The queen of these (in my opinion) is Dan Diffendale's photo of an inscribed decree concerning the founding of the colony of Brea.

Making the Pleiades machine tags easy to use is a challenge we haven't yet overcome. It's in the nature of these tags to be specific to a relatively small number of photos. The opaqueness of our numerical identifiers further hinders reuse; when you type "pleiades:" in the tag field on Flickr (or Aperture or whatever) the resulting list of suggestions like:

pleiades:depicts=42
pleiades:depicts=43
pleiades:depicts=44
...

isn't super helpful to mere mortals (which is why things like rdfs:label exist in more robust semantic web systems, right?). Despite the limitations, I've known that we could do better at showing to users what to do and yesterday I checked off something that has been on the todo list for a while: cut-and-paste machine tags for the Pleiades place pages.

On the right side of pages (like Palmyra's, http://pleiades.stoa.org/places/668331) you'll see section labeled "Photos" that may contain a portrait of the place and links to related Flickr photos. Below this you will now see two suggested tags:

Use this tag in Flickr to mark depictions of this place's site(s):

  pleiades:depicts=668331

or this one to mark objects found here:

  pleiades:findspot=668331

If you use these, I'd appreciate a note. The approach is very low tech (select, ctrl-c, ctrl-v) and user tracking in such detail is not in our budget. I'm looking forward to finding a bump in tagged photos soon.