Fiona makes reading and writing data boring

One of my goals for Fiona is to make it the most boring and predictable package for reading and writing geo data ever. I want it to refute the notion that "spatial is special" and have no gotchas, no surprises. As a benchmark for predictability and dullness consider fiona.tool, a minimal and limited replacement for ogrinfo and ogr2ogr inspired by Python's json.tool. You give it a data file name and write a description to a file or stdout

$ python -mfiona.tool -d docs/data/test_uk.shp

Description of source: <open Collection 'docs/data/test_uk.shp:test_uk', mode
'r' at 0x2038b10>

Coordinate reference system (source.crs):
{'datum': 'WGS84', 'no_defs': True, 'proj': 'longlat'}

Format driver (source.driver):
'ESRI Shapefile'

Data description (source.schema):
{'geometry': 'Polygon',
 'properties': {u'AREA': 'float',
                u'CAT': 'float',
                u'CNTRY_NAME': 'str',
                u'FIPS_CNTRY': 'str',
                u'POP_CNTRY': 'float'}}

or convert the file's data to GeoJSON.

$ python -mfiona.tool --indent 2 docs/data/test_uk.shp /tmp/test_uk.json
$ head /tmp/test_uk.json
{
  "type": "FeatureCollection",
  "features": [
    {
      "geometry": null,
      "id": "0",
      "properties": {
        "POP_CNTRY": 60270708.0,
        "CNTRY_NAME": "United Kingdom",
        "AREA": 244820.0,

Most of the source is devoted to parsing arguments and formatting the descriptions. The core of it is this:

# The Fiona data tool.

if __name__ == '__main__':

    # ...
    args = parser.parse_args()

    with sys.stdout as sink:

        with fiona.open(args.infile, 'r') as source:

            if args.description:
                meta = source.meta.copy()
                meta.update(name=args.infile)
                if args.json:
                    sink.write(json.dumps(meta, indent=args.indent))
                else:
                    sink.write("\nDescription of source: %r" % source)
                    print("\nCoordinate reference system (source.crs):")
                    pprint.pprint(meta['crs'], stream=sink)
                    # ...
            else:
                collection = {'type': 'FeatureCollection'}
                collection['features'] = list(source)
                sink.write(json.dumps(collection, indent=args.indent))

Does that underwhelm you? Are you asking "is that all there is?" Good. It underwhelms me, too. All there really is to Fiona is an open function that returns an iterable object that you can treat as a limited file or io stream and plain old Python dicts. No layers, no features, no shadow classes and reference counting. Just concepts that Python programmers already understand. Files, iterators, and dicts. Nothing special.

By the way, how about that use of sys.stdout as a context manager? I expected it wouldn't be a problem, and it isn't. The Python language keeps getting better and better. I'm developing Fiona for Python 2.7 right now and experimenting with a full switch to 3.3, with backported releases for 2.7 and 2.6.

Comments

stdout as a context manager

Author: Jonathan Hartley

Hey,

Lovely post as always, but I'm puzzled by the use of stdout as a context manager, and my google-fu apparently isn't up to it this early in the morning (lots of results for using context managers to redirect stdout)

I can see from here (http://docs.python.org/2/library/stdtypes.html#file.close) that File objects used as context managers get closed when exiting the block, so I expect that you're closing (and hence presumably flushing) stdout when you leave the outermost 'with'. But this is the end of the program, so this would happen anyway.

So why are you doing it? Thanks.

Re: Fiona makes reading and writing data boring

Author: Sean

What I didn't point out is that in the actual code, the sink (output stream) could be either stdout or a Fiona Collection object depending on the parsed script args. That they both work as context managers lets me write less code. The reasons I'm using ``with`` instead of letting the file close at garbage collection time are that it future-proofs my script a bit against code I may add after the features are read and written and that I'm planning to hang some OGR state on the Fiona Collection soon and want to use __enter__ and __exit__ to manage it reliably.