Sean Gillies Blog

Last week GIS Monitor published some fantastic flamebait by Manifold's product manager, Dimitri Rotow. Exploiting a flamboyant personality and the natural tensions between commercial and open source projects seems a bit irresponsible to me. Is this the new tone of the magazine? If so, I suggest a change of name to GIS Foodfight.

Howard Butler has a thoughtful and humorous rebuttal on his blog.

The initial release of Google Maps was news worthy. But this? Satellite or aerial imagery layers in web mapping are commonplace. Privacy concerns voiced in the media are overblown, unless you have been trying to hide an illegal Olympic-sized pool on your property. The imagery doesn't zoom in enough to reveal that you are stealing cable or violating your neighborhood's covenant against pets or air-drying laundry. There are real concerns for privacy today, but this is not one.

Three books related to MapServer and/or Open Source GIS are due to be published this summer. It's fascinating how a community with no paper books can suddenly produce a trio. Almost like how Hollywood can burp up a pair of asteroid/volcano/virus disaster movies simultaneously -- except that these books won't suck.

Tyler Mitchell's Web Mapping Illustrated (oreilly) was the first that I noticed. My understanding is that Tyler is covering everything from data processing and desktop GIS with OpenEV to web applications using MapServer. I proof-read his chapter on mapscript and found it very good. He's not re-publishing the tired old community scripts; these are fresh and clearly coded Python, Perl, PHP, and Ruby examples. I'm definitely getting this one. Should be available in time for OSG05.

Next, i was clued-in to Mapping Hacks (oreilly) by Schuyler Erle, Rich Gibson, and Jo Walsh. This book addresses not the mainstream analyst/developer, but hackers, inventors, artists, and hobbyists. Their interest in scripting MapServer seemed to grow with time and I'm very impressed with the sample code that I found on their companion site: http://mappinghacks.com. Has also been fun watching Schuyler latch onto Python. Or is it Python latching onto Schuyler? Being a fairly conventional GIS type, I think I have more to learn from this book than from Web Mapping Illustrated and am eager to pick it up. Should also be available in time for OSG05.

Lastly, there is Bill Kropla's MapServer: Open Source GIS Development (apress). I know very little about this book or its author, but do know the technical reviewer who assures me that it will be a comprehensive guide to MapServer installation, configuration, and templating. No publication date announced, but probably this summer post-OSG05.

According to Tim O'Reilly, sales of PHP books are up 16% in the last year. C# sales are also up slightly. While sales of other language books are down, Python continues to gain on Perl. My sense is that trends in the MapServer community roughly parallel the trends in computer language book sales.

Perl was the original mapscript language, followed soon after by Tcl and PHP (3). Use of Tcl mapscript has, as far as I can tell, ceased. O'Reilly makes no mention of Tcl in his article. Over the last year use of Perl mapscript has eroded (measured by user and developer list traffic), with users moving to PHP (4), Python, and Ruby. Use of Java mapscript continues to grow very slowly. MapServer's C# interface is attracting much more interest in the past year. This appear to be an entirely new set of users that are not crossing over from another mapscript-ing language.

Interest in PHP mapscript, already huge, continues to grow despite the lack of clear breakthroughs for PHP 5 or DSO support. It's easy to understand; PHP is pervasive, well documented and promoted, and (my bias may be showing) how is a PHP shop going to migrate its applications anyway? To Python? How would they pick from the many frameworks? To Java? They've chosen PHP because it is easy and flexible, and would struggle to shoe-horn their apps into a Java framework. There are new PHP applications and there are legacy PHP applications, but I can't imagine any moving away from PHP.

In the next year, I expect the Perl -> PHP trend to continue for MapServer. Java mapscript users will probably be switching to packages from the maturing GeoTools/GeoServer/uDig nexus. Python users will probably be switching from mapscript to PCL. C# developers, who now seem to be entirely in the Microsoft rather than Mono camp, may be picking the Carbon project over MapServer. I'm looking forward to the meeting this summer and the chance to hear other takes on the linguistic landscape of the future.

Nice article yesterday in the NY Times about sensor webs. We need more environmental applications of location technology, and fewer silly authoritarian applications.

By coincidence, I've just finished reading the 21st annual Year's Best Science Fiction anthology. The last story in this collection is Terry Bisson's "Dear Abbey", featuring a planetary environmental sensor web grown omniscient and omnipotent. A fantastic story. Charles Stross's "Rogue Farm" was the only one I enjoyed more.

The April-June 2005 issue of ArcUser contains an article by Hobu on pages 34-36. It's an introduction to Python and GIS, and vector, grid, and raster processing with ogr.py and gdal.py for users of ArcGIS 9 -- which includes Python and makes a small subset of its functionality scriptable. Good writing, too bad he seems to have retired from blogging ;)

I ventured further into the magaine and found myself comparing the agility of ArcGIS and the GDAL/OGR Python modules while reading the tutorial on pages 46-49 ("Creating Accurate Footprints of Registered Aerial Images"). It's a rather involved and round-about manual process requiring an expensive ArcGIS add-on (Spatial Analyst). I'm sure that one could also manage to open a beer bottle using dentistry tools, but not as readily as with a bottle opener. Consider the following Python script:

# Create footprint shapefile
import ogr

driver = ogr.GetDriverByName('ESRI Shapefile')
footprints_shp = driver.CreateDataSource('footprints.shp')
footprints = footprints_shp.CreateLayer('footprints',
                 geom_type=ogr.wkbPolygon)
fd = ogr.FieldDefn('FILENAME', ogr.OFTString)
fd.SetWidth(30)
footprints.CreateField(fd)

# Loop over a number of georeferenced images
import glob
import gdal

files = glob.glob('*.jpg')
for file in files:
    # Get georeferencing and size of imagery
    dataset = gdal.Open(file)
    g = dataset.GetGeoTransform()
    pixels = dataset.RasterXSize
    lines = dataset.RasterYSize
    minx = g[0]
    maxx = minx + pixels * g[1]
    maxy = g[3]
    miny = maxy + lines * g[5]

    # append to the 'footprints' layer
    wkt = 'POLYGON ((%f %f, %f %f, %f %f, %f %f, %f %f))' \
        % (minx, miny, minx, maxy, maxx, maxy, maxx, miny, minx, miny)
    g = ogr.CreateGeometryFromWkt(wkt)
    f = ogr.Feature(feature_def=footprints.GetLayerDefn())
    f.SetField(0, file)
    f.SetGeometryDirectly(g)
    footprints.CreateFeature(f)
    f.Destroy()

# destroy footprints_shp to flush and close
footprints_shp.Destroy()

No fooling about -- directly to the imagery bounds and write a polygon. Simple and straight-forward, the right tool for the job.

Yesterday afternoon, post workshop, I went with my brother across the river to visit the Walker Art Center. Among the major works -- the Chuck Close potraits, the Warhols -- was a fun interface to information about the museum's works: the dialog table. This interface consists of museum objects, and avatars for various "Locate", "Describe", etc operations on the objects. These elements are dragged to each other to execute operations.

How are elements selected and translated? This is the fun part. The user stands holding her hand over the dialog table, with illumination from above. The shadow of her hand on the dialog table screen, and its gestures, are captured and processed. A pinching motion creates a hotspot for object selection, and the selected object can be translated with a dragging gesture. Once you get the hang of it, it's quite natural and fun.

I really disagree with Adena Schutzberg on this one where she applauds Autodesk's RealDWG countering move against OpenDWG:

"Real" sounds well, authentic, true, and it being from Autodesk, who owns the format, is a pretty good name. The term "open," I believe is the main source for "fear uncertainty and doubt" in IT. No offence to Open Design Alliance, but "open" is just not a strong word these days.

Autodesk certainly can't name it ClosedDWG; it would take off like a jet with a closed throttle. Is RealDWG any better? I've been watching the meaning of the word real erode for years. The obvious recent example, Reality Television, is actually the opposite of reality. Face it, "real" is not what it used to be.

The assertion that the word open itself is the source of FUD is nonsense. FUD is the flop sweat of no longer innovative companies. Open is one of the richest words in the English language, spanning (with its derivatives) 12 pages in my OED. Not every meaning is positive in every context, but it is an overwhelmingly positive word. Ask family, friends, and colleagues whether they consider themselves open-minded. I'd be extremely surprised if a majority say no.

Industry was better represented and more involved at this year's conference. A new Directions article lists 10 ideas and themes, and examines the significance for both the open source community and the geospatial industry's mainstream.

Tyler Mitchell's Web Mapping Illustrated, published by O'Reilly, could have spared me about a week of hair-pulling back in 2000 when I was just beginning to discover open source GIS software. Our projects are reasonably well documented, and there is an enormous amount of knowledge within the community, but there has never before been a broad and coherent synthesis of that information. Finally, new users can see the entire domain of open source mapping from data creation, to data processing, to digital map. We've needed this book for a long time.

We expect O'Reilly books to be excellent productions, and indeed it is. In appearance, style, and scale, it is like the Web Services Essentials and SVG Essentials -- also edited by Simon St. Laurent -- but has a larger format, better paper, heavier binding, and copious color illustrations. The book takes on a big workflow and diverse tools, but Tyler manages to keep it together with a consistent mapping narrative, clear writing style, and concrete, replicable examples.

Highlights of Web Mapping include:

• 17 pages on installation of MapServer from binary or source distributions.
• Thorough overview of inspecting vector and raster data with ogrinfo and gdalinfo. I was reminded that OGR is geared towards working with directories of shapefiles, something I hadn't fully appreciated.
• Two chapters on MapServer configuration and templating. I'm a stickler for mapfile style, and Tyler's is worth emulating.
• Excellent chapter on configuring MapServer to provide and consume OGC web map and feature services.
• The best existing overview of PostGIS, at least until somebody writes PostGIS in a Nutshell.
• Nice introduction to programming with MapScript. Tyler even chose the right language for all his primary examples :)

I would have liked to have seen Tyler take on some of MapServer's anti-patterns, or help users avoid these 300 layer map config files we keep hearing about on the mapserver-users list. I also disagreed with his use of PostGIS geometries as unique keys for spatial sub-selects in Chapter 13. Spatial tables should always be provided with a lightweight unique identifier for this purpose. The book also suffers a bit from too many color screenshots of OpenEV. Powerful as it may be, OpenEV's primitive interface isn't going to sell any copies of this book.

I also wonder whether or not Google, which launched their web map just as this book was going into print, hasn't permanently altered the meaning of "web mapping" and thus moved the book's target. Now, to most of the IT world, Google's map is web mapping; there is one web map, and it's all a matter of annotating it with your own points. A distinction between Tyler's workflow and tools, and what the masses call "web mapping" is important.

In my estimation, users of ArcIMS/ArcSDE who are looking to broaden their horizons will find Web Mapping slightly too introductory, but a still valuable aid in getting over the first humps. Definitely worth buying. IT and Web professionals looking to break into geospatial and mapping work will find this book to be the ideal starting point, as will those who are graduating from Google map hacks to more unique and data-dependent applications.

Dead at 85. Chief Engineer Montgomery Scott was my favorite of Star Trek's supporting characters, and a patron saint of all IT people who've been called upon to do the impossible again and again. Oddly, I watched the episode The Deadly Years just last night, and saw Scotty come within minutes of succumbing to radiation poisoning and extremely accelerated aging.

How does a programmer get on the GML train? I was asked this question yesterday, and didn't have a satisfactory answer. The specification is not the place to start, a book takes a few days to arrive in your hands, and articles on the web are mainly aimed at managers. Today I was clued in to Ron Lake's brand new Wikipedia GML pages. This is where to begin.

Via The Map Room I see that the oldest surviving road map of Great Britain is browseable through an almost equally dated and obsolete ArcIMS viewer. One of the greatest achievements of Google's disruptive map application has been to bury the notion that this pre-CSS, pre-AJAX web app is an acceptable way to present maps online.

I regularly scan James Fee's great Spatially Adjusted blog for news from the ESRI side of the blogosphere. One of his favorite ideas is that ArcWeb services are equally good for web mashups as Google Maps. Besides being untrue -- SOAP is an impediment to mashups -- it's wrong-headed.

Yes, one can use the ArcWeb SOAP API to push point data to the service to be rendered onto a map and display the resulting map image in a web page, but this is not what we're talking about when we say mashup. A web map mashup, in the chicagocrime.org sense, combines data (locations, images, etc) and maps in your browser. The browser is where the mashing occurs, increasingly with the use of tools like Greasemonkey, not a GIS server.

Furthermore, a mashup begins as an unsanctioned operation or workaround for some web service that has no useful API. ArcWeb services, on the other hand, are strictly defined in order to appeal not to mashers and hackers, but to the Enterprise (managers, I mean, not the USS Enterprise). Calling the normal use of this business-suited SOAP API a "mashup" seems a bit phony, an attempt to co-opt buzz and cool.

Great stuff: Hobu interviews James Fee. This is not, thankfully, a discussion of blogging, but a wide-ranging dialogue about our profession and industry.