Geoprocessing for humans: date and time

Fiona 0.7 roughly supports OGR date/time fields. Date, time, and datetime field values are turned into strings conforming to RFC 3339 "Date and Time on the Internet: Timestamps". Fiona is ignoring time zones in this version, but then OGR itself doesn't have much support for time zones, and neither do common vector data formats.

There's an example of adding a date type field to a shapefile in test_collection.py.

with collection("docs/data/test_uk.shp", "r") as source:
    schema = source.schema.copy()
    schema['geometry'] = 'Point'
    schema['properties']['date'] = 'date'
    with collection(
            "test_write_date.shp", "w", "ESRI Shapefile", schema
            ) as sink:
        for f in source.filter(bbox=(-5.0, 55.0, 0.0, 60.0)):
            f['geometry'] = {
                'type': 'Point',
                'coordinates': f['geometry']['coordinates'][0][0] }
            f['properties']['date'] = "2012-01-29"
            sink.write(f)

A look at the shapefile's feature table in QGIS shows that I'm getting writing of dates right.

http://farm8.staticflickr.com/7141/6787537911_1312a73981_b_d.jpg

Reading that shapefile back in Fiona confirms that dates are read properly.

>>> from fiona import collection
>>> c = collection("test_write_date.shp", "r")
>>> from pprint import pprint
>>> pprint(c.schema)
{'geometry': 'Point',
 'properties': {'AREA': 'float',
                'CAT': 'float',
                'CNTRY_NAME': 'str',
                'FIPS_CNTRY': 'str',
                'POP_CNTRY': 'float',
                'date': 'date'}}
>>> for f in c:
...     print f['properties']['date']
...
2012-01-29
2012-01-29
2012-01-29
2012-01-29
2012-01-29
2012-01-29
2012-01-29

Be careful with this feature. Unless your data is destined for a legacy system, I think you're better off keeping track of time as RFC 3339 strings in a text field. Among other advantages, you'd gain millisecond precision and precise expression of UTC time offset.

Comments

Re: Geoprocessing for humans: date and time

Author: Michael Weisman

Good to see more OGR field types supported.

Any reason for using a string representation of dates within Fiona rather than using Python's native datetime objects and converting them to the string representation OGR is expecting at write time?

Re: Geoprocessing for humans: date and time

Author: Sean

One of my goals for Fiona is to go light on classes, stick to built in Python types, and make sure that features can be trivially serialized to JSON. Datetime objects fail that last test. Try json.dumps(datetime.time(...)) and you'll get a TypeError. I did consider (year, month, day, hour, minute, second, millisecond) tuples, but RFC 3339 strings suit my needs better. They're ready to show to humans, sort them lexically and you get cheap temporal sorting, and they're easy to parse. Fiona has fiona.rfc3339.parse_date and friends which take strings and return tuples you could pass to the datetime constructors.