2008 (old posts, page 13)


This is P. Strictus, perhaps the most beautiful perennial wildflower of the Mountain West, just beginning to bloom today. We've got several of these around the yard, grown from seed I collected near Granby, CO in 2006. The neighborhood bees are also big fans, and last year inspired our little toddler to yell, "the bees are going in the tunnels"! Indeed. I'm rather pleased at how this shot turned out.


In the background you can see a crimson eruption of P. eatonii, a native of the Colorado Plateau.

In other news, my wonderful gig at UNC is up. I've got a break before my next one starts (continuing Pleiades and getting more into digital humanities), and plan to spend some of it on vacation, some of it in the garden, some of it getting back into home brewing (beer for sure, electronics maybe a little), and some of it on cool Web projects.


Re: Penstamonium

Author: mpd

Don't mix electronics and beer!

Re: Penstemonium

Author: Sean

I got an introduction to Arduino at THATCamp and I'm hooked. I can think of a number of ways to use it in a nano-brewery.

Re: Penstemonium

Author: mpd

Electronics first, beer second is OK. The other way round is not so good.

Atom as service oriented architecture

Andrew Turner's act of data liberation reminded me that I'd made a similar point at THATCamp. Web applications are often coupled directly to a database as shown on the left of the diagram below, and other applications on the Web that can't access the database must scrape data from the primary app (illustrated by a dashed scrape-scrape-scape line). A better architectural design pattern is shown on the right of the diagram: use Atom or KML (especially for geographic apps) as a general purpose service layer to which many apps (including the cool apps of the future 2 and 3) can connect. The New York Public Library is one institution using this design, and my sense from attending the mashup session at THATCamp is that there will soon be others.


This is not entirely new to GIS architects. If you subscribe exclusively to the OGC service architecture, you would of course use a WFS instead, but Atom has the advantages of being more generic and more attuned to the architecture of the Web itself.


Re: Atom as service oriented architecture

Author: Bryan

... which is of course why I'm trying to wrap my GML (and most everything else) in an atom service layer ... Spot on.

I'm sabotaging the fight for a sustainable climate?

Following up on the OSGeo Python API discussion (nothing new there), I stumbled onto this warning that Atom and JSON may be sealing our doom:

Some of the issues that are are attracting a lot of effort are about simplifying spatial data (GeoRSS, GeoJSON, BXFS etc). These appear to be about catering to the 'pretty picture' use of spatial information.

GeoRSS and GeoJSON are about disseminating data effectively on the Web and designing for serendipitous reuse. GeoRSS is a hypertext format for building distributed applications. GeoJSON is a wire format for passing geometries between clients and servers of "Web 2.0" applications. Both of these features will be increasingly important to future research applications, and GML is well suited to neither of them.

I'm regularly seeing serious efforts to address the analysis use of spatial data (e.g. GML 3 and complex features) ridiculed.

I have seen the complexity of GML ridiculed, yes, but I have never seen scientific analysis itself ridiculed by the same parties.

Meanwhile 2050 is fast approaching, if we are to believe the climate change predictions.

No, GML skepticism does not equate to ambivalence about science or the sustainability of our planet.


Re: I'm sabotaging the fight for a sustainable climate?

Author: Chester Latham

The only problem I've had with global climate data is researchers and governments not making it available. You'd think they'd want the sun shining on their important information.

Re: I'm sabotaging the fight for a sustainable climate?

Author: Bryan

Chester: Exactly what global climate data can't you get? Try http://www.ipcc-data.org/ As usual this particular issue manages to get my attention. I'm depressed but having my cake and eating it too!

Re: I'm sabotaging the fight for a sustainable climate?

Author: Sean

Bryan, you are exactly right in that post. Atom and GML aren't exclusive.

Everyone's a historian now

All of my non-humanities readers know that there is historical content on Wikipedia cribbed from previously published information, but may not be aware of other "croud-sourcing" scholarship that is producing entirely new historical information. Stephen Mihm's article in the Boston Globe highlights several different projects using the Internet and its communities to examine American History in new ways. It's too bad there aren't more crowds of ancient Greeks and Romans around today to source like this.

Via Dan Cohen, who I missed at the NEH/CNR last October, but get to meet this weekend at THATCamp.

AtomPub will drink WFS's milkshake

Sooner or later. If you don't believe me, read Dare Obasanjo:

... Back in the RSS vs. Atom days I used to get frustrated that people were spending so much time reinventing the wheel with an RSS clone when the real gaping hole in the infrastructure was a standard editing protocol. It took a little longer than I expected (Sam Ruby started talking about in 2003) but the effort has succeeded way beyond my wildest dreams. All I wanted was a standard editing protocol for blogs and content management systems and we've gotten so much more.

Structurally, GIS data is pretty mundane, tabular stuff, and a good fit for AtomPub. Unless the tooling coming out of Microsoft turns out to be vapor and Google's push to become the ITCZ of cloud computing bonks, AtomPub will be ubiquitous. You'll be using it for sound, photos, video, contacts, social data -- and you'll be using it for spatial data too.


Re: AtomPub will drink WFS's milkshake

Author: Roberto

Unless (...) Google's push to become the ITCZ of cloud computing bonks...
Made my day! :)


Looking at the xISBN service docs tonight while running Pleiades data import tests, I see that there is support for "REST-ful" short URLs in version 1 and cool URIs in version 2. That seems pretty cool to me, so let's try one out:

sean@lenny:~$ curl -v http://xisbn.worldcat.org/webservices/xid/isbn/0596002815
> GET /webservices/xid/isbn/0596002815 HTTP/1.1
> Host: xisbn.worldcat.org
> Accept: */*
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: text/xml;charset=UTF-8
< Content-Length: 305
< Date: Wed, 28 May 2008 06:04:12 GMT
<?xml version="1.0" encoding="UTF-8"?>
<rsp xmlns="http://worldcat.org/xid/isbn/" stat="ok">
<isbn    >0596002815</isbn>
<isbn    >1565928938</isbn>
<isbn    >1565924649</isbn>
<isbn    >0596513984</isbn>
<isbn    >2841770893</isbn>
<isbn    >1600330215</isbn>
<isbn    >8371975961</isbn>

Looks good (links would be even better), but that "stat" attribute on the rsp element smells a little fishy ... let's try a bogus ISBN:

sean@lenny:~$ curl -v http://xisbn.worldcat.org/webservices/xid/isbn/05960bogus
> GET /webservices/xid/isbn/05960bogus HTTP/1.1
> Host: xisbn.worldcat.org
> Accept: */*
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: text/xml;charset=UTF-8
< Content-Length: 103
< Date: Wed, 28 May 2008 06:13:38 GMT
<?xml version="1.0" encoding="UTF-8" ?>
<rsp xmlns="http://worldcat.org/xid/isbn/" stat="invalidId"/>

The response has a status code 200, which should mean success, but there's an error code specific to xISBN in the representation: "invalidID". The underlying RPC nature of xISBN leaks through the cool URI abstraction in this situation. That request has to result in a 404 ("Not Found") if xISBN is going to be RESTful, right?

Speaking of curl, Mark Nottingham points out a potential pitfall and collects a bunch of useful comments here.


Re: xISBN and REST

Author: erich

xISBN successfully found a result for the bogus isbn, and the result was that it was bogus. You have a problem with that???

Planet Geospatial

Did James Fee really reboot? I'm not seeing a difference.

It's weird what an influence his site has on the informal discourse of the GIS business. It's hugely popular with the GIS community. I've heard from James that the traffic is big. It's my greatest referrer by far, about 3x what I get from Google searches. I'm not sure how a new blogger would get found in the past few years without emailing James. I've depended on Planet Geospatial to find new bloggers (Dan Shoutis and Regina Obe, to name a couple -- thanks, James!), but finding this signal in all the noise is increasingly frustrating. Only a very small volume of Planet Geospatial interests me, so little that my Greasemonkey script has become useless -- it often wipes the page clean.

(Note: I've probably benefited from Planet Geospatial as much as anyone: it was Planet Me-o-spatial for a while after the last reboot because he, Andrew Turner, and Howard Butler, IIRC, weren't posting nearly as much as I was at the time. A lot of those readers have stuck around (subscribed, even), if only for the spectacle of it -- watching me rip on OSGeo, tear into the OGC's service architecture, bite off chicken heads, or whatever.)

James has said he doesn't want to be a "gatekeeper", and maybe we can spread the load around a bit more. New referring planets have appeared in my logs: Planet OSGeo (funny, because I'm possibly the only unaffiliated open source person in the world), Planet PerryGeo, the Spatial Galaxy planet. I'd like to see new bloggers email the admins of those sites as well and give folks like me new and different places to discover new blogs.


Re: Planet Geospatial

Author: Matt Perry

I didn't know anyone had actually found that planetperrygeo page.. honestly I had kind of forgotten about it. Good to see the cron job is still working! Maybe now i'll have some motivation to fix that css.

Re: Planet Geospatial

Author: James Fee

The first rule of Planet Geospatial is - you do not talk about Planet Geospatial.

Re: Planet Geospatial

Author: Sean

You could remove me ... I'm sorta curious about the impact that would have. I suspect most of the audience would applaud.

Re: Planet Geospatial

Author: James Fee

And what, be left with Google Maps satellite photo updates?

Re: Planet Geospatial

Author: Sean

And yes, blogging about Planet Geospatial is lame. I won't make a habit of it.

Re: Planet Geospatial

Author: Regina

Sean, Glad to see someone reads my blog. Makes me feel inspired to update it. I have to admit I was feeling kind of depressed after being taken off of James' list.

Public Service, Public Data, and the Web

Next time I speak to a non-technical audience about the "GeoWeb", I'm going to lean heavily on Paul Ramsey's clever and informative talk (click through for the PDF).

Glossing over the different architectures of the Internet is the right thing to do for that audience, so long as you don't get them hooked with lots of Web examples (like pages 84-93 of the PDF) and then sell them "Web Services" (like WFS on page 94) that aren't actually of the Web and haven't really been a success at getting data into the public's hands. Don't bait and switch, in other words.

Barrington Atlas Feature IDs and Unicode Normalization

Last week I was NYU's Institute for the Study of the Ancient World to plan our Concordia project, an effort to interlink projects like Pleiades, IRCyr, the ANS database, and APIS, and build a traversable, searchable network of data based on Web architecture. I'll be blogging more about this all through the year -- expect my blog to intersect more with the Mapufacture and FortiusOne blogs if they continue on course. Some may even be interesting to mainstream GIS designers and developers. ESRI's implicit embrace of Web architecture (along with the explicit embrace of Google) was one of the big stories out of Where 2.0, after all.

One thing that cropped up in our sessions was the need of other Ancient World projects to be able to refer to Barrington Atlas features in Pleiades by URIs derived from their atlas labels. Tom came up with a template for these URIs:


and simple rules for normalization that just so happen to be already implemented in plone.i18n. We've forked (in a friendly way: forking is now cool thanks to git, right?) plone.i18n and removed the Zope utilities and all dependency on the Zope component architecture. The result is pleiades.normalizer, and it reduces Barrington Atlas labels which may contain annotation and non-ASCII characters to ASCII strings suitable for use in the URI template:

>>> from pleiades.normalizer import normalizer

>>> list(normalizer.normalizeN(u'Tetrapyrgia'))

>>> list(normalizer.normalizeN(u'Timeles fl. '))

>>> list(normalizer.normalizeN(u'*Tyinda'))

>>> list(normalizer.normalizeN(u'[Agrai]'))

>>> list(normalizer.normalizeN(u'Kalaba(n)tia'))

>>> list(normalizer.normalizeN(u'Tripolis ad Maeandrum/Apollonia ad Maeandrum/Antoniopolis'))
['tripolis-ad-maeandrum', 'apollonia-ad-maeandrum', 'antoniopolis']

>>> list(normalizer.normalizeN(unicode('Ağva', 'utf-8')))

>>> list(normalizer.normalizeN(unicode('Çaykenarı', 'utf-8')))

The algorithm normalizes non-ASCII characters (normal form KD) and discards elements which are not letters or digits:

U+011F -> (g, U+0306) -> g

U+0131, the last character in Çaykenarı, is a bit of a troublemaker. Our ASCII "i" has the diacritical mark relative to its dotless latin cousin, the inverse of the usual situation, and we have to make a special exception for it in the code.

If you're getting started in web development with Python you might find pleiades.normalizer handy, or at least a starting point for your own normalization code. We won't be publishing it to PyPI, but you can get an egg via:

$ easy_install http://atlantides.org/eggcarton/pleiades.normalizer-0.1.tar.gz