2007 (old posts, page 18)

Web Geo-Processing, Pull Style

The growing consensus is that map image and feature services can be (and should be) done RESTfully. Is there any aspect of web GIS that cannot? Geo-processing, perhaps?

Last week I started to roughly sketch out how one might RESTify an OGC Web Processing Service (WPS). Post data to a process resource, get back a 202 ("Accepted") response with a "job" URL to be checked for status ... more or less what Richardson and Ruby propose in Chapter 8 of "RESTful Web Services". Then I remembered that there is a different approach, at least for data that is already on The Web: pull your data through a processing resource with HTTP GET. Like Yahoo Pipes.

I've got a feed at http://pleiades.stoa.org/places/settlement/classical.atom. It contains the locations of human settlements in ancient Lycia and Pisidia attested to be occupied during the Classical period, and I want to find the intersections of their spheres of influence. For that matter, I'd like anyone to be able to find those intersections, using whatever radius they wish. Furthermore, I'd like to let people process practically any GeoRSS feed in this way.

To this end, I've made a process resource at http://sgillies.net/mush/self-intersection. You pull another feed through it via a URL like:

http://sgillies.net/mush/self-intersection/
?url=http%3A%2F%2Fpleiades.stoa.org%2Fplaces%2Fsettlement%2Fclassical.atom&d=10.0

The source data's URL is provided as the url parameter, and a radius of 10.0 kilometers is specified using the d (distance) parameter. Go ahead and get the result feed and display it using Google Maps.

http://sgillies.net/images/overlaps.png

Here are a couple more examples:

OpenContext projects feed, 200 km intersections

Geo-Annotated Reuters News feed, 100 km intersections

Interested? Feel free to try the user interface to the services at http://sgillies.net/mush.html.

Keep in mind that this service is just an educational toy. There's little support for asynchronous processing, and attempts to pull big feeds will certainly time out. Feeds with less than 50 or so intersections will be fairly responsive. YMMV. I'll try to keep it running as much as I can.

Intersection of 2 different feeds is obviously the next step.

Comments

By ref + by value

Author: Keyur

Sean, Taking url as input works great for someone who has the web infrastructure (a public web server among other things) at their disposal - I call this the "parameter by reference" approach... But for casual users without this infrastructure, it'd be handy to also allow the value directly in the parameter - "parameter by value". Of course, parameter by value does not work for large datasets - you have to do parameter by ref for such cases or do a POST... But for users who are trying to do simple things (ex. buffer a single point) it works very well. Cheers.

Re: Web Geo-Processing, Pull Style

Author: Sean

You can say "parameter by reference"; I'll say "pull".

Re: Web Geo-Processing, Pull Style

Author: Kyle

Wow... a very cool gp tool you have here. This gets me so excited for what's to come on this new frontier!

Sound Advice for INSPIRE

It's been reported that the architects of the INSPIRE enterprise are likely to choose WS-*. Before the decision is made, they should read Benjamin Carlyle's lessons of The Web. The point Carlyle makes often, and better than almost anyone else, is the ability of Web-like architectures to evolve. Should INSPIRE's enterprise be a distributed object system, upgrading to new versions across the board simultaneously (and this certainly means rarely)? Or should it be more like The Web, allowing clients, servers, and data to be upgraded independently and as needed? Carlyle's blog is bursting with this kind of sound advice.

Comments

Re: Sound Advice for INSPIRE

Author: Yves Moisan

Hi Sean, You've got a typo in one hyperlink : *z*http://www.soundadvice.id.au/blog/2007/06/10/#lessonsOfTheWeb On a related note, I just came across a news on slashdot today which has this interesting excerpt : "Seventy-three percent of [IT] projects with labor cost of less than $750,000 succeed," Jim Johnson says. "But only 3 percent of projects a with labor cost of over $10 million succeed". http://www.cio.com/article/124309 Small is beautiful. I know that ;-). I'll take a small REST now. Cheers,

Re: Sound Advice for INSPIRE

Author: Sean

Thanks, Yves.

REST follow on

Author: Yves Moisan

Sean, In case you didn't know, there seems to be folks interested in REST at the Multimedia Sprint (Plone 4 Artists) : http://www.openplans.org/projects/plone4artists-sprint/topics Excerpt : " RESTful web services for AT content types and vocabularies We have many use cases that involved the sharing/integrating content and vocabularies between systems. We think that REST is an appropriate technology for such goals. Our work will likely take the form of: * Service providers that adapt AT content objects to be exposed via REST * Tools to consume RESTful content/vocabularplone.org/products/ies into our systems." "AT content objects exposed via REST" may be interesting for Plone/PrimaGIS. Could Atom feeds be wrapped in AT objects to be exposed via REST also ? Cheers

Re: Sound Advice for INSPIRE

Author: Sean

Yes: /news/494/designing-simple-gis-services-for-zope/

Why There is no REST in WxS

OGC web services have no uniform interface, and therefore are not well suited for RESTful architectures. That's the why not. A uniform interface is the core tenet of REST. Ron Lake has a comment here.

Let me remind readers that the Geo-Web-REST discussion group is open to the public.

Comments

Re: Why There is no REST in WxS

Author: Bill Thorp

Where is the "No REST for the Wicked (WxS)" title for this?

Re: Why There is no REST in WxS

Author: Sean

Too obvious.

Software and Camping Gear

Tom Kralidis asked the following question about OWS common metadata parsing, but it could be asked about almost any aspect of web GIS:

But then I thought why not just implement this in MapServer, and have the functionality exposed via mapscript?

Why write new web GIS software? Why not add to MapServer? I'm going to explain by analogy: it's like picking camping gear.

Call me soft, but I like filtered water, hot meals and drinks, shelter from the rain, and a warm bed when I go camping. For the sake of argument, let's call these the basic requirements for alpine camping equipment:

  • water filtration and storage;
  • heat source for cooking food;
  • shelter from rain and snow;
  • warm bedding;
  • means of transporting the above.

There are 2 distinctly different philosophical approaches to meeting these requirements. The first involves acquiring:

  • a hand-pumped sub-micron water filter, and several liter bottles;
  • a compact single-burner kerosene stove, with pot, kettle, cup, and spoon;
  • a weatherproof, folding tent with light, collapsable graphite framework;
  • a sleeping bag and stuff-sack;
  • a backpack.

The REI approach, if you will (disclosure: I am an REI member). The second approach is to get a recreational vehicle with everything (including the kitchen sink) built in. See where I'm going with this analogy? The RV is certainly convenient, but gives you no flexibility. The configuration of its components is fixed. None of them can be removed and used away from the vehicle. The REI approach, on the other hand, provides maximum flexibility. Want to camp kilometers from a road? Pack your gear in the backpack and walk. Water sources are far from your campsite? Take your filter and bottles to the source and carry treated water back. Maybe you're just "carpet camping" at a friend's ski cabin? Bring only your sleeping bag.

My projects require similarly modular, agile software tools. There are places that MapServer can't readily go, and that's why I'm working on OWSLib, Quadtree, Rtree, GeoJSON, and Shapely.

Comments

Re: Software and Camping Gear

Author: Tom Kralidis

In some instances, it's alot easier to build code on top of existing, approved packages so as not to introduce anything 'new' into the mix. For example, it would be easier to build GeoJSON support in OGR, so then one can utilize them from within an already-built-and-deployed GDAL/OGR scenario. P.S. I'm a sucker for analogies -- they are really helpful!

Re: Software and Camping Gear

Author: hobu

Tom, I recently committed GeoJSON output and input of geometries to the next-gen Python bindings in OGR. It has allowed me to do things like this (a service that can reproject GeoJSON from URLs): http://geoservices.hobu.biz/project/?url=%22http://geoservices.hobu.biz/political/json/johnson%22&inref=EPSG:26915&outref=EPSG:4326&callback=reproject Howard

Re: Software and Camping Gear

Author: Sean

How about this: build reprojection of GeoJSON data on the web into ogr2ogr? ;)

Re: Software and Camping Gear

Author: hobu

ogr2ogr wouldn't be very webby, would it? As a side note, I will not be implementing any specification that is described on a wiki page in a rigid, statically-typed language like C++. When the dust settles on the spec, we can look at making it part of OGR proper. Howard

Re: Software and Camping Gear

Author: Tom Kralidis

hobu: to clarify, OGR proper has GeoJSON support, right? So I can get to them via Python or Perl, etc.? As for reprojection, I'm a big fan of the ogr/wcts in the GDAL dist, and use it alot. Great web service!!

Re: Software and Camping Gear

Author: hobu

Don't know that there is such a thing as 'proper' GeoJSON support because there isn't a hard spec yet.... Anyway, OGR has Geometry-only GeoJSON i/o in the Python next-gen bindings in subversion. So, no, it isn't in OGR proper as a real driver, just a couple of Python methods in the Python bindings. When the spec settles down, maybe I'll make the effort to write a C++ driver for GeoJSON. It's likely to be messy because OGR doesn't really have a concept of a free-standing Feature, however, and GeoJSON almost exclusively does.

The "GeoWeb Ecosystem"?

I'm stalking the "GeoWeb" buzzword via Technorati and found this prime specimen, which takes it to the next level with "GeoWeb Ecosystem". Most of my offline friends are ecologists, and they talk about little else, so I'm steeped in ecology and sensitive to the use of ecosystem in marketing.

If Google is the primary source of the "GeoWeb Ecosystem", then it's a pretty limited and fragile ecosystem. Like the organisms living off hydrogen sulfide leaking from a deep-sea vent, we must huddle closely around an API, pray to Neptune that it doesn't shut off, and dream of brighter, richer realms.

Comments

Re: The "GeoWeb Ecosystem"?

Author: Matthew Giger

Ha! I love it. You forgot to mention being completely in the dark.

Re: The "GeoWeb Ecosystem"?

Author: Brian Timoneyu

I only want to huddle with GIS Certified Specialists. And maybe a cute sales rep. BT

Chocolate and Peanut Butter

Completely untrue:

"It's like combining chocolate and peanut butter. They're good by themselves, but the combination is much more valuable than when they are served in isolation."

(John Hanke, Google's director of maps, in Business Week)

The combination of chocolate and peanut butter is in no way better than pure chocolate. To my dismay, Hanke appears to be either a barbarian, or one who cynically perpetuates the fraudulent premise of Reese's Peanut-Butter Cups.

Comments

Re: Chocolate and Peanut Butter

Author: me

Lies! Chocolate with PB chunks has always been the best Baskin Robbins flavor!

North and South

Steve C.-P. pointed me to this bad analogy by David Chappell

To anybody who's paying attention and who's not a hopeless partisan, the war between REST and WS-* is over. The war ended in a truce rather than crushing victory for one side -- it's Korea, not World War II. The now-obvious truth is that both technologies have value, and both will be used going forward.

that Elliotte Rusty Harold picks off and runs the other way for a touchdown:

That's a nice analogy. Take it one step further though. WS-* is North Korea and REST is South Korea. While REST will go on to become an economic powerhouse with steadily increasing standards of living for all its citizens, WS-* is doomed to sixty+ years of starvation, poverty, tyranny, and defections until it eventually collapses from its own fundamental inadequacies and is absorbed into the more sensible policies of its neighbor to the South.

Real Life Object Databases

I enjoyed Martin Davis's love letter to the relational model, but would like to remind readers that object databases aren't entirely academic. The Zope Object Database (ZODB) is employed in Zope and Plone, and supports sites including the Nature Conservancy, the FGDC, the CIA, KCRW, the Brazilian Government, and many other regional or communal government sites. I appreciate relational databases (PostgreSQL in particular) and love SQL, but also appreciate being able to model data and manipulate it purely in Python.

Comments

Perspective from a Zope Refugee

Author: Justin Bronn

I appreciate relational databases (PostgreSQL in particular) and love SQL, but also appreciate being able to model data and manipulate it purely in Python.
How is using a strange XML dialect to define your data (ZCML) modeling in "pure python"? I'll stick with my relational database before I let Zope devour every last megabyte of my RAM again.

Re: Real Life Object Databases

Author: Sean

ZCML is for configuring Zope components and has nothing to do with ZODB.

Re: Real Life Object Databases

Author: Justin Bronn

Point taken -- but how many of the sites you listed use ZODB independently, i.e., without Zope? For use in web applications, as you implied, the two are nearly inseparable.