How to decorate Python GIS code
Last month I blogged about Python logging and how to avoid using print statements in geoprocessing code. But your crufty old code isn't going to rewrite itself, and you're overworked already. An efficient fix would be optimal, and I've got one that only requires a little time to learn how to use Python decorators.
Say you have a module and function that does some geoprocessing work and prints various messages along the way. Something like this:
def work(): print "Starting some work." print "Doing some work ..." print "Finished the work ..." if __name__ == "__main__": work()
which, when run, produces output in your terminal.
$ python work.py Starting some work. Doing some work ... Finished the work ...
Now, your function is much more gnarly than work(), and rewriting it will only sap your goodwill toward its author. You'd think it would be possible to somehow wrap the work() function, catching those print statements and redirecting them to a logger – while not breaking code that calls work() – all in a reusable fashion. And it is possible, using a decorator like the 'logprints' class in the code below:
import logging from StringIO import StringIO import sys logging.basicConfig( level=logging.DEBUG, format='%(asctime)s %(levelname)s %(message)s', filename='work.log', filemode='w' ) class logprints(object): def __init__(self, func): # Called when function is decorated self.func = func def __call__(self, *args, **kwargs): # Called when decorated function is called # save reference to stdout saved = sys.stdout # make a string buffer and redirect stdout net = StringIO() sys.stdout = net # call original function retval = self.func(*args, **kwargs) # restore stdout sys.stdout = saved # read captured lines and log them net.seek(0) for line in net.readlines(): logging.info(line.rstrip()) # return original function's return value(s) return retval @logprints def work(): print "Starting some work." print "Doing some work ..." print "Finished the work ..." if __name__ == "__main__": work()
The statement "@logprints" is interpreted as "decorate the immediately following function with the 'logprints' class." On import of this module, the method logprints.__init__() is called with 'work' as the sole argument. Afterwards, when work() is interpreted, logprints.__call__() is called. That method acts as a proxy for the original, now decorated, function. Here is the print capturing and logging decorator in action:
$ python work2.py $ cat work.log 2008-12-30 12:14:44,044 INFO Starting some work. 2008-12-30 12:14:44,044 INFO Doing some work ... 2008-12-30 12:14:44,044 INFO Finished the work ...
Yes, you could have redirected the output of the original script in the terminal, but remember that Python's logging module sets you up to do much more.
I've recently learned how to use parameterized decorators by following the examples in Bruce Eckel's article. I'm using one to deprecate functions in Shapely:
import warnings class deprecated(object): """Mark a function deprecated. """ def __init__(self, version="'unknown'"): self.version = version self.msg_tmpl = "Call to deprecated function '%s', to be removed in version %s" def __call__(self, func): def wrapping(*args, **kwargs): warnings.warn(self.msg_tmpl % (func.__name__, self.version), DeprecationWarning, stacklevel=2 ) return func(*args, **kwargs) wrapping.__name__ = func.__name__ wrapping.__doc__ = func.__doc__ wrapping.__dict__.update(func.__dict__) return wrapping
Marking a function deprecated like:
>>> from shapely.deprecation import deprecated >>> @deprecated(version="1.1") ... def foo(): ... return None ...
causes a warning to be emitted when the function is called:
>>> foo() /Users/seang/code/gispy-lab/bin/labpy:1: DeprecationWarning: Call to deprecated function 'foo', to be removed in version 1.1
Deprecation-marking decorators are a great solution (which I first saw used, in a different form, in Zope 3). Why would you want to rewrite a function that's going away in the next software version?
Decorators can also be chained. In Shapely I've factored the check for non-nullness of GEOS geometries into a decorator and chain it with the built-in property decorator:
@property @exceptNull def geoms(self): return GeometrySequence(self, LineString)
To this effect:
>>> from shapely.geometry import MultiPoint >>> m = MultiPoint() >>> m.geoms Traceback (most recent call last): ... ValueError: Null geometry supports no operations
The exception is raised by the 'exceptNull' decorator.
Not much specifically about GIS here, I'll admit, but GIS programming in Python is, or should be, just Python programming. Feel free to comment if you see any interesting applications of decorators.
Comments
Re: How to decorate Python GIS code
Author: brentp
nice logging decorator. i believe it's also good form to use the decorator module: http://pypi.python.org/pypi/decorator to reduce boiler-plate. and it gives you decorator.decorator to decorate your decorators.