Emailing Python Script Errors

After my workshop, one of the attendees asked me how he would go about mailing the results or errors from a geoprocessing script. I did some hand waving about smtplib and traceback, two of Python's standard libraries, and will follow up now with some concrete, tested code.

In the example below we import three modules that will be used to access Python exceptions and stack tracebacks, format them as strings, and interface with an email server:

import smtplib
import traceback
import StringIO

def process():
    """This function performs an invalid list access, raising an error"""
    return [1, 2][3]

def mail_error(msg):
    """Mail an error message"""
    server = smtplib.SMTP('mail.sgillies.net')
    server.sendmail('overnight@sgillies.net', ['overnight-admin@sgillies.net'],
                    msg)

The process() function is a stand-in for whatever function is doing your geoprocessing. In this example, all it does is make an invalid list access, and thereby cause an error (an IndexError) to be raised. The mail_error() function makes a connection to a local mail server and sends the specified message from overnight to overnight-admin. The from address does not necessarily have to be a valid address unless your system has extremely tight security. Make sure to use the proper mail host for your own system. See the smtplib docs for more details.

Next, define a mail message template, and the main script body:

ERROR_TEMPLATE = """
Subject: Processing Error

Processing has failed. Traceback and error are shown below:

%s
"""

try:
    result = process()
except:
    exc_string = StringIO.StringIO()
    traceback.print_exc(file=exc_string)
    message = ERROR_TEMPLATE % (exc_string.getvalue())
    mail_error(message)

The error template has one string "slot", into which will be interpolated a system traceback. In the body of the script, we call the process function. All exceptions are caught, we use the print_exc() function of the traceback module to get the current stack traceback, and then pass this traceback as a string to the mail_error() function.

print_exc() requires a Python file-like object as an argument. This example uses Python's handy StringIO class to make an in-memory file that is named exc_string. The traceback is written to this file, we interpolate the contents of the file (getvalue) into the template, and then send the interpolated template via email.

Email is only one option, of course. Doing geoprocessing with Python rather than VB means that you can mix in all kinds of great standard and non-standard functionality. You might syndicate processing errors using RSS, generate stats and plots using matplotlib, or ping your enterprise's paging system.

Update: Howard reminds to make a stronger point about the error template. The templates drives the email, and Reply-To:, CC: and such can be added along with the subject. He also points out that many SMTP servers use pop-before-smtp to authenticate email senders. In this case, you'd want to look into poplib.

GIS in the Rockies Workshop Materials

I brought too few resource sheets to the workshop. The original PDF is at resources.pdf. The talk and programming exercises are here:

I plan to follow up on the several good technical questions in the next week or so, do check back with this site.

The emerging standards track was held in a sky box high above the Denver Broncos' Invesco Field, a novel venue for sure. Dale Laushman of The Uptime Group introduced open source and open standards, and with a question revealed that only a fraction of attendees knew for sure that open source was being used in their enterprise. We've still got a long way to go, or an untapped market, depending on how you look at it.

Brian Timoney of The Timoney Group is using Google Maps and Google Earth as the core of his services, and demonstrated these to a full house.

Oracle's Daniel Geringer gave a great overview of Oracle Locator and Spatial, and indulged me in questions afterwards. I've been skeptical about storing raster data in an RDBMS, but he's convinced me to think again.

Geoff Anderson of Cloudshadow Group wrapped up the track with a workshop on building applications using geospatial web services. I'm envious of his ability to write code live and pull it off. There are a lot of PostGIS users who would love to see his take on a TIGER-based geocoder.

GDAL, MapServer, and USACE

I've been forwarded a postcard of sorts from a GIS user in the U.S. Army Corps of Engineers which gives big props to MapServer and GDAL for helping to power their Katrina response. While analysts and engineers are using ArcGIS on their desktops and notebooks, our favorite open source tools are efficiently and reliably doing much of the grunt work. Moving earth and filling sandbags, if you will. I'm expecting Tyler Mitchell to have more details soon.

Geo-Web Blog

Ron Lake wrote to clue me in on his new Geo-Web blog. He's already written a book -- which I've checked out of Colorado State's library -- numerous specifications, white papers, articles, but I think some regular postings that examine applications familiar to the blogosphere could really help people learn about GML. I'm particularly interested in the new simpler profiles, and seeing what can be done to bridge the divide between the live web (RSS, Atom) and the geo web (GML).

Catching up with previous posts, it's interesting to see how sincerely Google (Keyhole) Earth's KML has flattered GML 1.0.

Mapnik Update

Artem Pavlenko's Mapnik continues to advance, and I'm still trying to keep an eye on this as a another potential rendering engine for PCL and PrimaGIS. Agg, Boost, thread-safety ... I welcome this, but since he's starting mostly from scratch, there is a bit of catching up yet to do. While simple, Artem's Hello World demonstrates again how good Agg can be for producing clean looking maps. Would be great to see some other layers, and examples of the feature-level transparency that MapServer lacks.

Ancient Mapping Technologies

Via The Map Room I see that the oldest surviving road map of Great Britain is browseable through an almost equally dated and obsolete ArcIMS viewer. One of the greatest achievements of Google's disruptive map application has been to bury the notion that this pre-CSS, pre-AJAX web app is an acceptable way to present maps online.

Comments

Re: Ancient Mapping Technologies

Author: John Kavanagh

ArcIMS is an absolute bear to develop in. Google Maps's API beats the ArcIMS aimsadmin and params.js.