Closing down the Lab

If you're a suscriber to the GIS-Python projects mailing list, you've seen the discussion about migrating projects to other hosting services. If you're not a subscriber your first notice might have been seeing this at the Trac site:

http://sgillies.net/images/closing-lab.jpg

OWSLib is going to Sourceforge, the zgeo.* projects are being superceded by the collective.geo suite at CoActivate, and most everything else has been moved to GitHub.

When Kai and I started the GIS-Python effort in 2005, running our own development site looked like the best option. Today, it's nowhere near the best option and feels increasingly like a hinderance. I'm looking forward to spending less time on operations and more time on coding. I hope you'll stick with the projects through the move and beyond.

Ancient Toponym of the Week: Horeb Mons

Reconciling objects from DARMC with Pleiades today, I came across the late antique monastery at Horeb Mons. I don't have a copy of the Tabula Imperii Romani with me this morning and won't be able to check for a while (WorldCat says there is one at the Colorado State library), but this looks to be Saint Catherine's Monastery (a UNESCO World Heritage Site). The Pleiades entry for this place is surprisingly skeletal.

The first occurrence of "Horeb", חֹרֵב in Hebrew, χωρηβ in Greek, is in the Old Testament's Book of Exodus. The Barrington Atlas, on which Pleiades is largely based, does not attest to a location for the biblical Mount Horeb itself.

I've never been to the Sinai Peninsula, but visited Mount Horeb, Wisconsin a few times in the 1990's. Its Mustard Museum may never be recognized by UNESCO, but is an important stop for a completely different kind of pilgrimage.

Update (2011-08-18): I've checked out the TIR and brought it back to ISAW West; Saint Catherine's is at Tou Batou, not Horeb Mons.

Plus Me

I never signed up for Facebook and I don't anticipate using Google+ as a not-Facebook, but I'm plenty curious about the UI and design and so have signed up to help train Google's social advertising algorithms. Google+ seems to have a lot of rules – use your real name and gender, no companies allowed, etc – but the experience of rules being lifted over time (assuming they will be) is better than the one of rules being added over time (I'm looking at you, Twitter).

Shapely on PyPy

Last night I began experimenting with PyPy following Bob Ippolito's notes. PyPy's JIT may open the door to fast pure Python computation geometry algorithms, but in the meanwhile I'd like to see if my various libraries will work with PyPy. Thanks to work by Oliver Tonhofer, Shapely does.

(pypy-2d192eaab45f-osx64)s3:pypy-env seang$ pypy
Python 2.7.1 (2d192eaab45f, Jul 07 2011, 22:10:02)
[PyPy 1.5.0-alpha0 with GCC 4.0.1] on darwin
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``not your fault, more like it's a moving target''
>>>> from shapely.geometry import Point
>>>> p = Point(0.0, 0.0)
>>>> p.buffer(10.0).area
313.65484905459385

Shapely 1.2.11 coming soon.

Update (2011-07-09): In case anyone else, like me, is in the position of having to bootstrap PyPy from source without a binary (running OS X 10.5.x here), I'll provide some extra notes. Where Ippolito writes:

# Translate PyPy (expect this a while, at least an hour for me)
(cd pypy/translator/goal; pypy translate.py -Ojit)

I used my system Python 2.5 ("python" instead of "pypy") and GCC 4.0.1 from XCode 3.1.3 to translate successfully. Time elapsed was about 2.5 hours on my MacBook Pro (2.8GHz Intel Core 2 Duo with 4 GB RAM). Compiling CPython from source on this same computer takes only several minutes.

Connecting places

The poster I made for DH 2011 shows "connects with" type relationships between several places, a new and distinctly "un-GIS" feature of Pleiades. It's meant to relate fuzzy places to each other with just enough semantics to afford researchers some useful and thought provoking networks, while not ruling out more specific future relationships.

Consider the figure below, a graph in which every node is a Pleiades place and every edge is a "connects with" relationship taken from the NeoGeo Spatial Ontology. This ontology is based on the Region connection calculus or RCC.

http://farm6.static.flickr.com/5312/5885119283_fe08cf3758_z_d.jpg

The Internum and Gallicum Mares are respectively the Mediterranean Sea and Gulf of Lion. Have you ever seen either of these bodies of water represented as a GIS feature? I have not. Typically, they manifest (or don't) on a map as negative space, labeled but border-less lacunae. Even when there's bathymetry on a map, the label is disconnected. If there's a feature dataset that allows one to answer the question "is the Mediterranean connected with the Gulf of Lion", I haven't seen it. Frontal boundaries do exist in the ocean, but they are dynamic features. Any dataset that modeled the boundary between the two as anything like national borders or real property lines would be more "truthy" than truthful. Still, geographers can agree that certain features define these bodies of water and that they are connected. The physical state of one affects the other on human time scales. People, goods, and information can travel by boat between these bodies of water, passing from one to the other, even if the exact moment and location of the transition is not determinable.

The Volcarum Stagna are the coastal lagoons (étangs) of the Languedoc. They are separated from the Gulf of Lion by narrow sand beaches. GIS analysis would say that the shorelines of the lagoons and gulf do not touch, but whether by canals or portage, the Volcarum Stagna and the Gulf of Lion are certainly connected from a transport perspective. Physically, too, as water is exchanged through the porous strips of intervening sand, themselves pushed around by the tides and waves of the sea.

The Ledus (now Lez) River heads in the foothills of the Cebenna Mons (Cévennes Mountains) and flows into the Volcarum Stagna at the ancient trading port of Lattara. It is connected with all three of those places. Likewise, the Rhodanus (Rhône) is connected with the Alps and the Gallicum Mare even though we're not able to say exactly what are the boundaries of the Alps, what is the exact course (and width) of the river, or what was the exact position of the mouth of the Rhône in ancient times.

GIS analysis can be quite sensitive to scale. The lengths of coastlines from 1:25000 and 1:1000000 (for example) scale datasets will differ unless the later is created by very careful generalization of the former. Now add computational geometry to the mix: connections found between features at a small scale may be different in number from those found between features at a large scale. Rather than twist lines and polygons beyond the limits of certainty and relevance to make them snap and connect and then require users to master complex GIS tools to derive relationships between places than might turn out to vary with scale, we can simply encode the most immediate relationships – at human scale – directly in the structured data about the places. The evidence for the relationships could very well be from GIS analysis, but it may also be from a reading of ancient or medieval maps, geographies, milestones, itineraries, manifests, or diaries.

The Ordnance Survey is a leader in publishing this kind of linked geographic data. See for example the resource about the City of Southampton, in which contained, containing, and touching cities, wards and boroughs are plainly enumerated. Some of the relationships don't follow from mathematical treatment; places might be part of other administrative regions from which they are geometrically disjoint. This is more or less what we're trying to do with Pleiades and it's been comforting to follow in the footsteps of OS researchers like John Goodwin. A difference in the Pleiades approach is that we're going to see how far we can get with no semantics other than "connects with", glossing over the difference between overlapping, touching, and the other RCC relations. Users tell me that our approach will be good enough for qualitative studies of trade and exchange in the ancient world.

Without a "disjoint from" relation, the Pleiades approach is akin to Facebook's. Absence of "dislike" semantics doesn't seem to be hurting the Facebook social graph, and I'm optimistic about the mileage Pleiades can get without implementing a "disjoint from" relation.

Comments

Semantic Relationships and Geographic Networks

Author: Elijah Meeks

I'd be particularly interested in procedural methods for deriving this kind of content from existing maps so that well-described geographic systems could exist alongside the curated and not-map-amenable systems you're dealing with above.

Do you "pack" the features at different scales? Also, has anyone come up with theoretical methods to analyze this kind of network? We're getting ready to build a transportation network of the Roman Empire over here and up until now I'd only thought to build a typical geographic network with historic travel costs and conditions, but to represent regions, geographic/environmental systems and so on in this manner might provide interesting opportunities for analysis.

Re: Connecting places

Author: Sean

John Goodwin, who I mentioned above, is working on reasoning over RCC relations. What I'm talking about for Pleiades will only support very basic stuff like "What are the settlements connected to the Aegean Sea with degree less than N?" We'll have all the Barrington's roads in Pleiades soon and I figure that we'll connect them to settlements and stations using what semi-structured data we have – most of it text like "Luteva → Villevieille → Nemausus" – even before we have any digitized lines (from Harvard's DARMC, I trust that you've seen their data). It would be great to have your measures of length and cost!

Packing? I assume you mean something like a proper part relation [1]. No, though I'd like to as soon as there's a customer for it.

[1] http://geovocab.org/spatial.html#PP

Re: Connecting places

Author: Elijah Meeks

That's what I'm thinking, coupled with some sense of the scale of the feature. If that was in place and the scales were somehow matched then the N-degree connections would be meaningful. Otherwise, a system like this would be too heavily influenced by the author, don't you think?

Pleiades "un-GIS" poster

This is the graphic from the imagemap version of the poster Tom Elliott and I put together for the Digital Humanities conference at Stanford University. It's a Frankenstein's monster of a diagram, showing relationships on different planes between Pleiades resources and code and other resources and communities.

http://pleiades.stoa.org/docs/graph-poster-2.jpg

By "un-GIS", we mean that our geographic information system compares to industry standard GIS as an unconference like a ThatCamp or WhereCamp compares to AAG or ESRIUC. We're not rejecting principles of geography and cartography at all, but we are reducing formality and complexity and trading other properties of conventional GIS for scalability and flexibility. It went over well at DH11.

Inside the Pleiades maps

Representing roughly located places in a map is one of the more interesting technical challenges of Pleiades. I've been through a few iterations and have finally settled on aggregating roughly located places by their common bounding boxes and portraying them as a cloud in the map. There are two of these in the neighborhood of ancient Cardiff: the one to the east representing some large Roman administrative regions, the one to the west representing a few rivers and other imprecisely located places of Roman-era Wales.

http://farm4.static.flickr.com/3489/5833140692_898da9a7ea_d.jpg

What makes this different from the common point clustering approaches is that these cloud markers, at smallish scales, hover around the edges of the map pane to suggest that they are "somewhere" on the horizon of the neighborhood.

http://farm6.static.flickr.com/5104/5833140696_8b9efea189_d.jpg

This effect is produced by sliding the markers along a line through the context and aggregate box centroids. Zoom in and the cloud approaches the context centroid.

http://farm6.static.flickr.com/5236/5833140704_1750c8f6b1_d.jpg

Zoom out and it approaches the the far side of a circle circumscribing the aggregate bounding box. On a mouseover event the aggregate bounding box itself is shown, and the members of the aggregation are listed in the cloud marker's popup bubble on a click event.

http://farm6.static.flickr.com/5144/5833303518_8cba5e5d30_d.jpg

I've wanted to accomplish this in a way that's as transparent and declarative as possible. The aggregate data is given to the client (Google Maps API in my case) in a JSON format. The line along which the markers will slide is represented as a GeoJSON LineString, but the feature itself is given a special "pleiades.stoa.org.BoxBoundedRoughFeature" type to distinguish this particular hovering-around-the-horizon kind of feature from a default GeoJSON feature. The javascript client running in the Pleiades page interprets this feature's data appropriately and displays it not as a line, but as a lurking object anchored to a variable point.

{
  "bbox": [
    -5.0,
    50.0,
    0.0,
    55.0
  ],
  "features": [
    {
      "bbox": [
        -5.0,
        50.0,
        0.0,
        55.0
      ],
      "geometry": {
        "coordinates": [
          [
            -3.1805009999999996,
            51.509679984152292
          ],
          [
            1.992620138265649,
            53.60762000668263
          ]
        ],
        "type": "LineString"
      },
      "id": "(-5.0, 50.0, 0.0, 55.0)",
      "properties": {
        "description": "<div>...</div>",
        "link": "http://pleiades.stoa.org/search?...",
        "snippet": "5.0 degree by 5.0 degree cell",
        "title": "Aggregation of roughly located objects"
      },
      "type": "pleiades.stoa.org.BoxBoundedRoughFeature"
    },
    {
      "bbox": [
        -4.0,
        51.0,
        -3.0,
        52.0
      ],
      "geometry": {
        "coordinates": [
          [
            -3.1805009999999996,
            51.481298999999993
          ],
          [
            -4.4406943342012717,
            51.565821152330514
          ]
        ],
        "type": "LineString"
      },
      "id": "(-4.0, 51.0, -3.0, 52.0)",
      "properties": {
        "description": "<div>...</div>",
        "link": "http://pleiades.stoa.org/search?...",
        "snippet": "1.0 degree by 1.0 degree cell",
        "title": "Aggregation of roughly located objects"
      },
      "type": "pleiades.stoa.org.BoxBoundedRoughFeature"
    }
  ],
  "id": "79706",
  "type": "FeatureCollection"
}

In the interest of transparency, this data is encoded in the web page using a JSON Data URI. There's no web service endpoint or data tucked away inside javascript; any client code can get the data from the <link> element with rel="r-where" in exactly the same way the Pleiades javascript does. I'm not saying this is the one true style of web mapping, but it's a style that I find very intriguing right now.

Comments

Re: Inside the Pleiades maps

Author: Peter Rushforth

Sean,

I dont' see rel="r-where" in your listing? What does r-where mean? Roughly where I suppose. If the r-where relation were defined in a

public link relations

repository, I would tend to agree that it is a good way to convey map semantics.

Other good link relations could be "east", "west","north-east" etc for common map operations like panning. Zooming could be "zoom-in", "zoom-out".

Re: Inside the Pleiades maps

Author: Sean

Yeah, this rel type is nowhere (pun intended) near standardized. I'm still experimenting with this pattern. Speaking of map panning and zooming types, You might be interested in Erik Wilde's work on tiled feeds: http://dret.net/netdret/docs/oracle11/.

Historical tweeting experiment

Before embarking on a family road trip last week I grabbed my copy of The Exploration of the Colorado River and Its Canyons, John Wesley Powell's account of his 1869 trip down the Green and Colorado Rivers. It has been a few years since I've read it, and it wasn't until a picnic lunch at Green River, Wyoming's Expedition Island Park that I realized his trip, shifted by 142 years, is happening now. For fun, inspired by @samuelpepys and @iHerodotus, I'm going to post a few 140 characters excerpts daily. If you're interested, follow @majjwpowell on Twitter.

Open source soil

This morning I moved the compost pile from the old New Zealand box on the left into the new box on the right. Despite the cool weather, it's been cooking well and there's multiple wheelbarrow loads of mature humus for the garden.

http://farm4.static.flickr.com/3377/5768710299_75903cc02f_z_d.jpg

I'd be surprised if we're not reducing the amount of trash that goes to the curb by half. I've visited cities that have curbside composting pickup; Seattle has had it for several years and Montpellier was trying it out during our séjour last year. It was super convenient, but it's extra satisfying to be rendering our own stems, peels, cores, and grounds into our own soil.

Comments

Re: Open source soil

Author: Tim Michael

Nice work. Between composting and recycling each of the seven items our city accepts, my wife and I average about a bag of actual trash each week. Very satisfying when I see my neighbors lug a full dumpster to the curb each week.

Ancient world base map tiles

Pleiades isn't a mapping site, but we have a few maps. There is one small scale map showing recent activity on the site currently centered on Asia Minor, and one larger scale map showing the immediate neighborhood of every positively located ancient place. The maps are extremely simple, just three kinds of overlays and a base map layer, the Google Maps terrain base map. The positives qualities of the Google base map are: global coverage, good cartography, high availability, excellent detail (full resolution to better than 1:10,000, all we need for Pleiades), and generally favorable (for us) terms of use. On the other hand: modern transportation ways and borders and labels are burned in with no options to style them away, and the terms of use are changing to a less favorable state. Google has announced that advertisements will appear in new applications. Existing applications like Pleiades are exempt for now, but I assume the terms will change again in the future (Google reserves the right to change terms) and that we will be subjected to Gmail-style contextual advertising or prominent commercial place markers somewhere down the road. Ads are entirely inappropriate for Pleiades. I'd rather show a plain white background than an ad-smeared terrain base map. Eventually, we'll need a base map with more favorable terms of use and fewer modern artefacts. Other projects will too, and so I'd like to kick off a discussion of how we might collaborate on creating one.

I'm impressed by MapBox's Natural Earth Hypsometric and Bathymetry base layer. The data behind it is public domain and the software stack is open source. There's even some open source support for running the Mapnik tile renderer on Amazon EC2. One might extend it using more detailed elevation data to generate the extra zoom levels that MapBox doesn't host. Pleiades doesn't need to zoom in to street level, but needs zoom levels equivalent to Google Map's level 12 at least. I'm curious whether global tiles at these zoom levels are possible with the service's SQLite-based architecture. The company's $500 per month tile hosting seems pretty reasonable, especially if it includes hosting the highest resolution tiles. Hosting global high resolution tiles at 12-15 zoom levels isn't going to be trivial or cheap, that's for sure.

Mixing of modern and ancient tiles could be a good interim approach. We might use the MapBox hypsometry and bathymetry (for example, or another equivalent tile set) as a base map layer and then overlay custom tiles generated in a matching style wherever we have ancient GIS layers which differ from modern layers. This requires more analysis to identify the tiles at all levels that would be impacted by feature differences – mainly the removal of modern canals and reservoirs, but also some shoreline changes – but in the end requires many fewer tiles tailored exclusively for ancient world map users and less duplicated effort.