2008 (old posts, page 6)

Rethinking GSDI Architecture

Ed Parsons and Chris Holmes were at the GSDI-10 conference and have each written posts about "GeoWebs" and "Global Spatial Data Infrastructures" that touch one of my favorite topics: architecture. Ed says that it is time for grand data sharing designs to yield to an evolutionary approach:

So we need to move away from the "Grand Design" approach and build SDI's organically and simply, perhaps making use of the new Global infrastructures that companies like Google and Microsoft have made available to bootstrap the technology, and deliver faster benefits and to make the case for more in depth infrastructures at a later date.

It's great that the GSDI community is hearing this. Yes, Ed is pitching Google's services as he pitches the Web, but I think it's still an honest argument for the Web. The Web works: not only for Google, but for thousands of other business, governmental, and non-profit enterprises, and for billions of users.

I only need to make one little correction or addendum to these paragraphs in Ed's post:

The Grand Design approach does introduce an additional issue which is technology related, many of the current SDI projects are planned to deliver over decades, with technological developments continuing to move rapidly, it is difficult to plan to implement using a technology which will be obsolete years before the infrastructure goes live.., as it is today the best available standards as drafted by OGC are moving from basic http interfaces to the more web services friendly SOAP based interfaces, while the leading edge is looking to REST based interfaces.

Mostly right. When we say "REST", we mean protocols based on the HTTP application protocol, and the architecture that makes the Web a success. REST is not a radical prescription at all. When you consider that REST won the distributed systems architecture war, to take the losing approach forward seems much more radical.

Chris Holmes's presentation is focused on social change, top-down yielding to bottom-up, but when he writes (on slide 19) that GSDI should

build on what's out there, on successful architectures of participation.

there are to me clear technical implications: The Web has an "architecture of participation", and its architectural style is named Representational State Transfer, or REST. I don't fault Chris at all for not directly pointing out that the OGC architecture is misaligned with the architecture of the Web: it's a distraction from his social message and possibly demoralizing to GIS data managers. But I don't think you can for very long dodge the issue that a "GeoWeb" really demands a RESTful Web-like architecture, and something like HTML. Hypertext, not GetCapabilities() and GetData().

Comments

Re: Rethinking GSDI Architecture

Author: Ed Parsons

Sean, I think we are in agreement I did not mean to suggest that REST was radical, it is just some way away from where we are now with the earlier http based OGC interfaces. In terms of searching for Geospatial content a RESTful approach makes life much easier.. ed

Re: Rethinking GSDI Architecture

Author: Sean

And I'm glad we're in agreement. It's just that I didn't think it was clear enough that the leading edge (REST) uses good old reliable HTTP and is arguably the more conservative approach to distributed systems.

Re: Rethinking GSDI Architecture

Author: Allan Doyle

Conservative architecture is something that evolves from what people are familiar with, and that is not necessarily the same as something that is elegant, simple, and robust. Computing architectures "matured" in the days of IDLs (interface definition languages) and compiled stubs that when instantiated provided systems that were distributed but still tightly coupled. Sun RPC begat the Distributed Computing Environment which begat CORBA. Microsoft COM begat DCOM which begat OLE COM (or was it COM OLE, and which came first anyway? it's all a blur). Then while the radicals were looking at REST, the conservatives began to build WS-*. From a business and management control perspective, REST is radical. It removes all the ivory-tower layers that have been accreting for what must be close to two decades. So no matter how conservative it may seem to those who understand REST, it remains something mysterious to a large segment of the computing industry. Similarly, SDIs have grown up around the notion of control, this time in terms of metadata, framework data, data accuracy standards, and on and on. The radical thought here is that maybe you don't have to have all of that at once to start doing something useful. The good qualities of SDI will eventually remain, but largely hidden from the view of most of the beneficiaries of all the geospatial functionality that has been able to push its way into the forefront of late.

Re: Rethinking GSDI Architecture

Author: Allan Doyle

Or, to be more succinct: REST is the radical thought is that you can have your apps be (part of) the web instead of just having them use the web. And therein lies the "architecture of participation".

Re: Rethinking GSDI Architecture

Author: Sean

Allan, I appreciate the reminder that the Web still seems new and maybe a little immature to the old guard. I admit that I take it for granted.

G(eo)nomes of Vancouver

Anybody else ever play Steve Jackson's Illuminati card game? The "Gnomes of Zurich" jumped instantly to mind when I read that the "GeoWeb" conference is the "Davos of Geo" on Ron Lake's (written by Roisin Clarke?) blog.

In emails I've seen mention of a possible REST agenda at the conference. That would be a good thing.

Frugosapalooza in Fort Collins

I'm so glad Dave Bouwman made it out because the question of the night was: coming from an all-ESRI environment, how hard is it to learn and get up to speed on an entirely new software stack? It's hard, and I don't deny that, but Dave reminded people that you don't need to take on everything at once. I think it's easier to start using open source on the very front or back ends. Maybe just try OpenLayers as the map interface for a new project, or try gdal_translate or ogr2ogr for ploughing through data offline before it is loaded into your database.

Turnout was surprisingly good: 10 of us, which is even better, per capita, than the huge (25-ish) Denver meetup. The MapWrecker was sorely missed, but the up side of that is nothing was broken.

By coincidence, the local Hacking Society was also meeting at the Bean Cycle tonight. Conversation with Sean and Evelyn reminded me that I probably should have submitted a presentation on Shapely and other Python GIS software to the PyCon committee.

Comments

Re: Frugosapalooza in Fort Collins

Author: Howard Butler

I submitted a generic GIS-in-Python overview talk giving the 50k view of libraries and utilities you can use in Python to PyCon, and it was rejected. There were some positive comments on my submission, but I think there just wasn't enough room in the conference or interest.

Re: Frugosapalooza in Fort Collins

Author: Sean

Now I am feeling thankful for the time I didn't spend on a submission. Still, I should attend PyCon some day.

Not-Quite-Web Processing Service

I had a longer post about the new WPS specification that I scrapped after I realized that it reduces to this: the OGC WPS working group gets the formats and parameters more or less right as usual (good), but still can't find any respect for HTTP as an application protocol (bad).

Web geo-processing came up at tonight's meetup in the form of this question: how do I compute zonal statistics on a raster over the web using open source? I said that GRASS would be part of the answer. It wasn't until the meeting broke up that I realized I'd neglected to mention PyWPS, which has a tantalizing kernel of Pythonic goodness.

Meanwhile, it looks like ESRI's ArcGIS Server might be the prime mover for RESTful geo-processing this year.

Update: or maybe it won't. This is not exactly an endorsement by Vish.

Comments

Re: Not-Quite-Web Processing Service

Author: Stefano Costa

Sean, it's worth noting that PyWPS is heavily based on GRASS (or other programs like R and GDAL/OGR) for all its analytical functions. See http://pywps.ominiverdi.org/ for some live examples. That said, I'll wait for another post from you about RESTful geoweb services to learn something about it.

Re: Not-Quite-Web Processing Service

Author: Sean

Yes, that was my point about PyWPS: that it has solved the problem of calling GRASS routines in the context of a web request, and is therefore worth looking at despite my concerns about the WPS protocol.

InscriptOL Source

By popular demand (okay, 2 people asked), you can now browse or clone the inscriptOL repo. I need to get back to the GIS-y aspects of Pleiades and won't be actively developing inscriptOL for a while, but patches are very welcome. SVG.js needs to be extended so that the format supports points and polygons as well as polylines. OpenLayers.Format.GML was my starting point and the difference between the polyline methods in the files should be illustrative. Code to browse and select source imagery, with setting of bounds and dimensions to appropriate values should also not be much of a task. Sound support for editing traced features is the most complex of the first things that need to be done to turn this into a serious tool.

Really, There is More to REST Than HTTP + POX

Umibot writes:

After some hard work/late nights and prodding from the geo-techno elite, we’ve humbly completed and are pleased to offer RESTful access to our neighborhood API.

In their "REST API", you can use either GET or POST (trampling the uniform interface) to invoke a remote method and get a JSON or XML response. It's better than SOAP, but I'm dismayed that this is hyped as REST.

Python, MapServer, and WSGI

To enable this is why I added an OWSRequest class and image byte access to mapscript. I'm glad they are getting used.

I was pleased to stumble onto Brent's blog a week ago: I spent my undergrad afternoons working in a molecular biology lab and still dabble in genomics. His genome browser inspired me to see OpenLayers as more than a web GIS tool.

Comments

Re: Python, MapServer, and WSGI

Author: Tim Maddle

I've found the MapScript OWSRequest capability invaluable for overlaying layers in Google Earth, so don't doubt that it's a valuable feature.

Re: Python, MapServer, and WSGI

Author: brent

hi sean, good to hear i've inspired you somehow. i'm an avid reader (and user) of your code and blog. i'm really having fun with mapscript lately.

Digitizing Ancient Inscriptions with OpenLayers

Update: For some background on this particular inscription see Tom's post on the demarcation between Thraces and Moesi.

The subject of digitally annotating texts and art works came up at last fall's NEH/CNR conference, and I thought: we could do that with OpenLayers. Recently, I read this post at the Stoa Consortium and thought: we could really do this with OpenLayers. And that's what we did today at the Pleiades skunk works: put a photo of an inscribed stone in an OpenLayers image layer, drew polylines over it in a vector layer, and then wrote the layer out to SVG: http://sgillies.net/images/MXITRIB.svg.

http://sgillies.net/images/MXITRIB.jpg

Feel free to try it out: http://atlantides.org/inscriptol/. Once we add the capability to edit or delete polylines, add text annotations/metadata, and save the results this could be the beginning of a useful tool. Check out that nifty SVG transform group: maybe I haven't lost all my linear algebra yet.

OpenLayers simply rocks.

Comments

Re: Digitizing Ancient Inscriptions with OpenLayers

Author: Paul Ramsey

What's the use case for digitized inscriptions? I don't comprehend.

Re: Digitizing Ancient Inscriptions with OpenLayers

Author: Yves Moisan

Same question here, but I do find it neat to use OpenLayers in images other than geospatial in nature. Annotations seem to me like the low hanging fruit here. An example : being able to share e.g. an air photo amongst a group of people that could trace out where they think the terminal moraine is and discuss through annotations the relative merits of delineation A vs B would be one heck of an interesting tool for photointerpreters. I have to look seriously at OpenLayers.

Re: Digitizing Ancient Inscriptions with OpenLayers

Author: Sean

Collaborative interpretation, for sure. And last night I received an email from a researcher who is using traces like these to train character or inscription extraction software. The humanities are more quantitative than you'd guess.

Re: Digitizing Ancient Inscriptions with OpenLayers

Author: Tom Elliott

I've started a thread aimed at answering Paul's question at Current Epigraphy. There's already one helpful response ...

Re: Digitizing Ancient Inscriptions with OpenLayers

Author: KoS

Great work. Neat use. I was thinking. From the stand point of capturing the text. Couldn't a laser scanner, running a edge detection on an image or something else similar, do almost the same thing? If you want vectors too, run a raster to vector conversion afterwards. KoS

Re: Digitizing Ancient Inscriptions with OpenLayers

Author: Tom Elliott

KoS: Certainly, and there are some folks working with automated techniques (see: M. Terras, Image to Interpretation: An Intelligent System to Aid Historians in Reading the Vindolanda Texts, Oxford, 2006, ISBN: 9780199204557 - publisher's blurb). Tools for manual work can still be valuable ... for example, for expert cleanup of automated vector creation (i.e., supervision and selection), or just for singling out arbitrary portions of an image for subsequent annotation.

Re: Digitizing Ancient Inscriptions with OpenLayers

Author: Stefano Costa

Really nice, for an archaeologist like me. This makes me think about 2 things: 1) it would be great to have an OGRGeometry method (and a Shapely one, too) like ExportToSVG(). This would allow easy vector image generation from geometries. 2) when using typical GIS tools like OL, OGR, for something that is geometric but not geographic, it's not always clear how to state that your XY(Z) coordinates are not in a specific CRS, nor lat/lon. See for example http://iosa.sharesource.org/ where we use OGR for the analysis of an ancient wall.

Re: Digitizing Ancient Inscriptions with OpenLayers

Author: Sean

Stefano, I know almost nothing about archaeological photogrammetry, so I'm very interested in learning from your projects.

No WMS in Google Static Maps API?

I think this says something: Google Maps Without the Scripting.

Update (2008-07-21): Everyblock WMS requests are gone, replaced by requests for tile resources at URIs such as http://tile.a.everyblock.com/1.0/main/4/77,502.png.

Comments

Re: No WMS in Google Static Maps API?

Author: Ian Turton

That Google Maps can't hope to support the flexibility of the full WMS interface so they have to provide this cut down wanabe service with a nasty API key hacked in?

Re: No WMS in Google Static Maps API?

Author: Sean

Either that (that Google can't design or engineer maps for the Web) or that when it comes to street maps WMS == YAGNI.

Re: No WMS in Google Static Maps API?

Author: Ian Turton

And yet in a blog post you referred to only the other day we see people pointing out just this deficiency of Google Maps - http://blog.everyblock.com/2008/feb/18/maps/. If all you want is to server up static images of your maps the Google maps is the way to do it, but as soon as you want something more complex you are going to wish you'd stuffed all that data into a nice standards compliant WMS server from the start so that all your client development work isn't wasted.

Re: No WMS in Google Static Maps API?

Author: Sean

No, according to that post the EveryBlock developers break from Google over cartography, not over map service design. If you check out their maps you'll see fixed zoom levels and tiles: http://nyc.everyblock.com/streets/brooklyn/kent-ave/473-493/map/ They are using TileCache, which – while it uses the WMS parameterization of what you want in an image (something WMS gets right) – is not exactly a ringing endorsement of WMS, the protocol. Notice that they are using named map styles? That's more or less the Google Maps approach too: SLD also == YAGNI for applications of this sort. I know this must feel like bashing, but the OGC has impressed a distorted (and now obsolete) view of web services on the GIS community. I'm trying to educate and perhaps even clear up a bit of the distortion.

Re: No WMS in Google Static Maps API?

Author: Ian Turton

When I look at the actual map requests it looks like a straight WMS request to me, right down to the annoying OGC specific error format. So if I want to view their maps in my own client (e.g. uDig) at any scale I like I can, if I really wanted to I could probably even work out how to add my own SLD to it but that clearly isn't required for this application and may well not be supported by their stack (I've never used mapnik so I can't say). While I understand where you are coming from Sean, and I agree with a lot of what you say distorting applications like this which are OGC based to support your argument does nothing to convince those of us who'd prefer to see neogeographers work with the OGC to fix things instead of sniping from the sidelines.

Re: No WMS in Google Static Maps API?

Author: Sean

Ian, I'm not trying to convince you. I'm writing to dispel the hype around OGC web services. Hype like this (not particularly egregious, just fresh), in which there is not even a hint that there is anything wrong with the WxS architecture. On the other hand, I will give the OGC due credit for any effort to develop standards more fit for the Web. Would be easier to recognize the efforts if they were more in the open, I must say.