There and back again

We're back in Fort Fun. After an initial snafu the travel gods smiled on us and the flights and connections went more smoothly than we had expected. There was a bit of airsickness and our poussette went missing, but we got some sleep on the long CDG to ATL leg, Customs didn't balk at the number of wine bottles we brought in our luggage, and we arrived not too burned out to celebrate with take-out pizza and a St. Lupulin. The kids even slept until five o'clock this morning. I'm declaring it a win.

The help and hospitality of many people made our séjour an amazing experience. Thank you, everyone, and particularly our nounou, Célia Mourrut; our hosts and friends at CBGP, Denis Bourguet, Arnaud Estoup, Benoît Facon, Renaud Vitalis, and Miguel Navascues; our friend across the street at the USDA, René Sforza; in Heidelberg the Epigraphische Datenbank Heidelberg team: Francisca Feraudi-Gruénais, Brigitte Graef, and James Cowey; in Montpellier, René-Luc D'Hont (3LIZ) and Gerald Fenoy (Zoo Project), and Vincent Heurteaux, Adrian Custer, and Martin Deruisseaux from Geomatys; in Torino, Giorgio Borelli and Silvio Tomatis (collective.geo and more), and Stefano Costa (Open Archaeology); in Isère (the Department of), Eric Lemoine. I'm looking forward to returning the favor some day.

Next steps are making some fresh salsa (I could never find ripe tomatoes, onion, cilantro, and chiles simultaneously at Montpellier's Arceaux market), unpacking, and getting some fall season vegetables planted.

Bye bye

Waiting at the Louis Blanc tram stop after probably our last family dinner in the centre ville – at Le Roule ma Poule in the relatively sleepy north side of the Ecusson – we saw a guy using the tramway as bike lane.

http://farm5.static.flickr.com/4084/4831955732_c82c97d439_z_d.jpg

If you keep an eye on the board at every stop and time it right, you can roll along unimpeded.

http://farm5.static.flickr.com/4109/4831955746_74941e032a_z_d.jpg

Into the sunset.

http://farm5.static.flickr.com/4084/4831955764_6aee696e5f_z_d.jpg

Au revoir, Montpellier.

Comments

Re: Bye bye

Author: ReLuc

Bye! Good trip to country home and next time in USA ;-)

Shapely recipes

I'm committed to keeping the Shapely API small. If it were a Swiss Army knife, it would be a slim one that fits comfortably in your hand and doesn't require you to wear a belt to keep your pants up when you pocket it. This means saying no to features. I recently read a person's opinion that Shapely should support SRIDs – like PostGIS does, I assume the reasoning to be. The GEOS library does in fact implement spatial reference system identification numbers for geometric objects, but I'm leaving this out of Shapely. Such identification numbers are relevant only to OGC standard (SFSQL) databases. On the web we identify coordinate systems by URIs. In a Python application, a spatial reference system should be a Python object not an integer pointing to some object outside the environment. A premise of Shapely is that if your application really needs to deal with transformations between coordinate systems, you'll know best how to add back this missing complexity. What's actually missing from Shapely is not the SRID concept, but a recipe or two for how to add in that concept.

Another requested feature left out is the splitting of lines. Inclusion of GEOS's linear referencing in the Shapely core barely made the cut (I think it could have been a good example of an extension), but makes it possible to implement line cutting algorithms in not very many lines of code. A naive one is shown below.

from shapely.geometry import LineString, Point

def cut(line, distance):
    # Cuts a line in two at a distance from its starting point
    if distance <= 0.0 or distance >= line.length:
        return [LineString(line)]
    coords = list(line.coords)
    for i, p in enumerate(coords):
        pd = line.project(Point(p))
        if pd == distance:
            return [
                LineString(coords[:i+1]),
                LineString(coords[i:])]
        if pd > distance:
            cp = line.interpolate(distance)
            return [
                LineString(coords[:i] + [(cp.x, cp.y)]),
                LineString([(cp.x, cp.y)] + coords[i:])]

The result (see cut.py):

http://sgillies.net/images/cut.png

This figure was made with matplotlib, of course. Control points of the line are shown as circles, and the boundary points are displayed in black.

I can imagine that different users and different applications will want to chop up lines in grossly or subtly different ways. The above returns a list of LineString, but another application might want a MultiLineString or an iterator that nibbles at the input while reading from another iterator of bite sizes. Who knows? Cutting up a line simply isn't a primitive operation. A recipe for doing this type of operation using Shapely seems better to me than including one implementation in the core.

Geometric object constructors that validate their input have been proposed from time to time. Validation is particular and expensive and better (in my opinion) left to a developer. Maybe your input has already been validated; maybe objects deemed "invalid" by JTS or GEOS are useful to you in some way. Shapely's constructors don't validate and aren't cluttered with validate=True arguments or the consequent compound if/else statement. Disappointed? The good news is that it's easy to add back the complexity of validation. The Shapely manual includes a recipe for a validating decorator.

from functools import wraps

from shapely.geos import TopologicalError

def validate(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        ob = func(*args, **kwargs)
        if not ob.is_valid:
            raise TopologicalError(
                "Given arguments do not determine a valid geometric object")
        return ob
    return wrapper

The decorator could be used like so:

>>> @validate
... def ring(coordinates):
...     return LinearRing(coordinates)
...
>>> coords = [(0, 0), (1, 1), (1, -1), (0, 1)]
>>> ring(coords)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 7, in wrapper
shapely.geos.TopologicalError: Given arguments do not determine a valid geometric object

Not everyone is fond of decorators and might want to do this differently. Subclass the geometry types, write some Zope-style adapters, or monkey-patch asShape – it's your call. If you will need validation, you will know (or discover) best how to exploit Shapely to implement it.

By keeping Shapely light and simple I expect we can keep it easy to use and extend. Indulge me in a food analogy, if you will. Shapely isn't ketchup, it's the ripe tomato: raw, but more useful in a wider range of recipes.

HTTP FTW

Waouh (scroll down, no internal anchors), this is a big deal to me:

  • OGC TC directs the OGC-NA that all new OGC identifiers issued for persistent public OGC resources shall be http URIs, instead of URNs
  • New standards and new major versions of existing standards shall use http URIs for persistent public OGC resources to replace OGC URN identifiers defined in previous standards and versions, unless OGC-NA approves an exception

Use of HTTP URIs is axiomatic for "linked data". This begins to make OGC stuff relevant to the web.

Update (2010-07-08): More on Allan Doyle's blog.

Pleiades and DARMC

Here is ISAW's Associate Director for Digital Programs (my boss), Tom Elliot, on collaboration between Pleiades and DARMC [Ramping up Pleiades 2]:

Meanwhile, we've been in dialog with Michael McCormick, Guoping Huang and Kelly Gibson at Harvard. They're the driving force behind the Digital Atlas of Roman and Medieval Civilization, with whom we're collaborating under the new grant. Our aim is to collate and share the datasets assembled by both projects and to cross-link our web applications. This will bring more accurate coordinates for many features into Pleiades, as well as a number of new features that will expand our time horizon into the middle ages. You'll get a choice of display and map interaction modes and, eventually, the ability to move back and forth between both resources. We'll keep you posted as the timeline for this portion of the work is refined.

Plotting geometries on C and Java Python platforms

GeoScript now has something like descartes [plotting geometries]:

The plotting functionality makes use of the JFreeChart library, a popular open source Java framework for creating diagrams and charts. Another example of one of the benefits of the marriage of Java and Python that is Jython.

It reminds me of the run of such posts I went on this past spring. JFreeChart might be great, but Java platform developers shouldn't underestimate what matplotlib brings to the C Python platform: loads of tools for earth science visualization and excellent integration with tried and true numerical libraries. That said, I'll freely admit to some JTS envy.

BTW, Shapely 1.2.1 is now accepted to Debian unstable.

Rough KML locations for geo URIs

I've combined my interests in the brand new RFC 5870 and representing rough locations in KML at http://ruffgeo.appspot.com/. Comments are very welcome, except about the name of the app.

Update (2010-06-18): Fixed a content disposition issue with Firefox. Apologies.

Comments

Re: Rough KML locations for geo URIs

Author: Tim Schaub

This could make a nice bookmarklet for browsers that aren't configured to follow geo URI:

<a href="javascript:(function() {var a, aa=document.getElementsByTagName('a'); for (var i=0,ii=aa.length;i<ii;++i) {a=aa[i]; if (a.href && a.href.indexOf('geo:')===0) {a.onclick = function() {window.location = 'http://ruffgeo.appspot.com/kml?q=' + encodeURIComponent(this.href); return false;}}}})();">geo bookmarklet</a>

Representing rough locations in KML (3)

After some reading of Bittner and Stell [1], I've decided that my most recent approach to representing fuzzy locations was too sloppy: spatial objects and the regions in which they are located were conflated. By leveraging the container-like nature of kml:MultiGeometry, I've come up with a new one in which my spatial objects (like an ancient city's footprint) are always represented as KML Features and the regions they occupy are always represented as KML Geometries. So far, it seems like it will let us represent vague objects (like "Narbonensis", the Roman province which covered the Mediterranean coast of what is now France and had no absolute boundaries) as well as crisp objects (like "Lattara", or your mobile device's GPS location) which are located imprecisely in our information systems.

In this approach, a MultiGeometry would represent the rough location of a spatial object and collect statements about relations to other regions which would also be represented as MultiGeometries, one per relationship predicate. Predicates would be represented as text nodes of relation elements from a future namespace. The predicates themselves would come from 9IM or RCC models. In the example below, I show collections of relations that form a 3-piece concentric partition of space (like an "egg-yolk" [2]).

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2"
  xmlns:x="http://example.com/x">
<Document>
  <Placemark>
    <name>Spatial object located at an indeterminate region</name>

    <MultiGeometry>
      <!-- representation of indeterminate region -->

      <MultiGeometry>
        <!-- core/yolk -->
        <x:relation>Contains/hasProperPart</x:partition>
        <Polygon>...</Polygon>
      </MultiGeometry>

      <MultiGeometry>
        <!-- wide-boundary/white -->
        <x:relation>Within/properPartOf</x:relation>
        <Polygon>...</Polygon>
      </MultiGeometry>

      <MultiGeometry>
        <!-- exterior -->
        <x:relation>Disjoint</x:relation>
      </MultiGeometry>

    </MultiGeometry>

  </Placemark>
</Document>
</kml>

Test drive this if you like: http://sgillies.net/files/rough.kml.

[1] T. Bittner and J.G. Stell, “Vagueness and Rough Location,” GeoInformatica, vol. 6, Jun. 2002, pp. 99-121.
[2] B. Porter, Handbook of knowledge representation, Amsterdam ;Boston: Elsevier, 2007.

Shape of things to come?

Can you identify the spatial object having this footprint?

http://farm5.static.flickr.com/4018/4710849357_8c4e212f69_d.jpg

Update (2010-06-18): Welcome James Fee blog readers. I've updated the picture. Code to produce it is at http://bitbucket.org/sgillies/plot-pac-n/src. I've made use of a handy shapefile wrapper for OGR and modified it to provide JSON objects.

Comments

Re: Shape of things to come?

Author: tia

aside from Washinton,Oregon,California,Arizona,Utah and Colorado?

Re: Shape of things to come?

Author: Mark

Would it be the PAC-10/11/12/14 if a Utah school joins?

Re: Shape of things to come?

Author: Brian Timoney

A lazy geographer using a lat/long "projection" ?

Re: Shape of things to come?

Author: Dennis

a polygon

Re: Shape of things to come?

Author: Brooks

red oil spill?

Re: Shape of things to come?

Author: Sean

"A" Utah school, Mark?! Regard the color of that polygon. But, yes, that would be the new Pac-N.

Brian, my (anonymous) desktop GIS doesn't do antialiasing or floating point pixel positions like matplotlib or mapnik (AGG) and so the census polygons looked like crap in UTM 12. I'll see if I can't make something more purty at halftime.

Re: Shape of things to come?

Author: Kirk

I see a red person with a small nose pointing a large gun eastward.

Or maybe I just see a rorschach.

Re: Shape of things to come?

Author: Sean

Kirk, I see the old Pac-10 opening a Utah (2nd best Pac-N team since 2003) shaped can of whoop-ass on the SEC and Big 10.