Behind the curtain

James Fee asks:

Curious, does it matter to a RESTful zealot that ESRI’s REST API is a cover for SOAP?

Not terribly. From a client's perspective, as long as there are bookmarkable entry points, solid formats, and links to other application states, everything is fine. A web client shouldn't be aware of what's going on behind the resource representations. When you work with Wikipedia, for example, you're operating on web pages about topics, not chunks of text in database records. That's an implementation detail, and something's broken if it leaks through to the client.

Typically there are other protocols involved in a RESTful service. Your web framework, at the very least, is connecting to an RDBMS using some specialized RPC. ESRI's ArcGIS Server just has one extra layer in there. This design seems inside-out to me, and probably sucks for other reasons, but it doesn't necessarily make things un-RESTful.

Unbearable liteness

If there's anything "GeoWeb" folks understand about REST, it's that REST is "lightweight". While it's possible to achieve goals simply in a RESTful system, lightness is your agenda, not REST's. Check out Roy Fielding's re-enumeration of the constraints you have to follow to get all the good properties of REST in "REST APIs must be hypertext driven". That's about as "lite" as an apprenticeship at Shaolin Temple.

On the other hand, I'm appreciating Steve Vinoski's view on RESTful HTTP as a disruptive technology in IEEE Internet Computing:

RESTful HTTP, on the other hand, has all the makings of a disruptive technology to the RPC market. As RPC systems [ed: Sun RPC and Apollo NCS, DCE, Corba, RMI, J2EE, SOAP, and WS-*] moved up-market and gained capabilities and features over time to continue to satisfy the most demanding customers, they overshot more and more potential users who shunned the complexity and cost of such systems. In RESTful HTTP, which was born in the adjacent market of the World Wide Web and is a sustaining technology there, these overshot users are finding an approach that helps them build solutions that are less expensive, simpler to build, and easier to extend and maintain than what RPC approaches can offer. It's precisely these qualities that make RESTful HTTP a disruptive technology in this context.

It's possible to do things simply using HTTP, without much REST constraint. This is the "lightweight" approach understood by the geospatial community as the "GeoWeb". The reason RESTful HTTP is disruptive is because it addresses the needs of the high end market as well. Systems applying the REST constraints can scale massively and can evolve to meet changing requirements, and that's why I'm uncomfortable with REST being pitched as "lite". Lets you achieve simple goals simply, but not itself a simple thing – is this too wonky a notion to get across?

Transliterating from Greek and Latin

Update (2010-07-08): The latest and greatest is uploaded to PyPI: http://pypi.python.org/pypi/pleiades.transliteration

The code we use to transliterate names from Greek and Latin writing systems in the style of the Barrington Atlas is now available from our package index.

$ easy_install -i http://atlantides.org/eggcarton/index/ pleiades.transliteration

For example, the name Ἀφροδεισιάδος from an edition of a letter from Trajan to Smyrna [IAph 2007 8.33] is transliterated to Aphrodeisiados:

>>> from pleiades.transliteration import transliterate_name
>>> s = u'\u1f08\u03c6\u03c1\u03bf\u03b4\u03b5\u03b9\u03c3\u03b\u03ac\u03b4\u03bf\u03c2'
>>> transliterate_name('grc', s)
'Aphrodeisiados'

My first vimperator script

I wrote my first vimperator plugin script last night. While browsing any of my blog posts I can now type :manage and Firefox opens the "edit" URI found in the page's head (a concept ripped off from AtomPub).

/*
 * Opens the URI of the first link in the document with rel="edit" in the same
 * tab.
 *
 * Usage:
 *   :manage
 *
 * @author Sean Gillies (sean.gillies@gmail.com)
 * @version 0.1
 */

commands.add(["manage"],
  "Open document's edit URI",
  function(args) {
    var documentNode = window.content.document;
    var editNode = documentNode.evaluate(
                    '//link[@rel="edit"]',
                    documentNode,
                    null,
                    XPathResult.FIRST_ORDERED_NODE_TYPE,
                    null
                    ).singleNodeValue;
    var editURI = editNode.getAttribute("href");
    liberator.open(editURI, null, true);
  }
);

Comments

Re: My first vimperator script

Author: jachym

Yeah, vimerator rulez! Imho it is much better, than this strange ubiquity thing...

Give it a REST

Tobin Bradley writes:

We found out a few days before the conference that we had won the G. Herbert Stout Award for Visionary Use of GIS for our REST Web Services Framework. Mrs. Stout was there to present the award, and it was a very humbling and happy experience for us. Congratulations to the Mecklenburg County GIS staff, and congratulations to Asheville for winning in the city category - it was definitely well deserved!

On the one hand, it's neat that anything "REST" wins an award in geospatial. On the other hand, REST can't take any credit because there actually isn't any of it in these services. And that's fine (except for the misleading label) because Mecklenburg County GIS doesn't need REST for this. REST is a style for huge, distributed information architectures that need to last for decades, like the Web (or even a national spatial data infrastructure).

I'd like to see more GIS developers follow the lead of CloudMade and tout HTTP APIs. Not only would it be in almost all cases more truthful, there's the advantage to being able to point users to the HTTP specification, HTTP libraries and tools, and not having to explain why there's no "REST specification".

Update (2009-02-23): Where are my manners? Congratulations on the award. Making an HTTP API that users like is no small thing, and neither is open sourcing a well-documented implementation to a GIS community that still tends to be locked up by proprietary vendors.

Comments

Re: Give it a REST

Author: Tobin

Hi Sean,

Thanks for the congratulations! We're using the term REST loosely to describe the transport method (as opposed to, say, SOAP) rather than the more correct Fielding definition of the term. I think that's fairly common use of the term and I think you can differentiate between REST and a RESTful application, but I can see how one might object. Terminology is a common problem for me. To this day my wife claims my geranium isn't a geranium.

Cheers, and great job with your blog!

Re: Give it a REST

Author: Sean

Common misuse of the term would be more accurate. We have to take our terms seriously.

Plugins for Shapely

I haven't started working on any pure Python (for App Engine) geometry predicates or operators, or any other alternatives to GEOS, but I've begun to make it possible. All GEOS dependency is being moved to its own shapely.geos module, which will be the default plugin provider for Shapely's new plugin framework.

The framework defines a few interfaces (geometry compiling or checking provider, area and length provider, bounds provider, etc) and entry points. When a geometry is asked to compute its bounds, for example, it delegates to a provider that has been wired up to the geometry factory on import. Here's a peek at how the default providers are registered in Shapely's setup.py:

...
setup(name          = 'Shapely',
      version       = '1.1.0',
      ...
      entry_points = """
          [shapely.geometry]
          geometryChecker=shapely.geos.geometry:geometryChecker
          metricProperties=shapely.geos.metrics:metricProperties
          geometryProperties=shapely.geos.topology:geometryProperties
          """,
)

And in shapely.geometry, the providers are activated with a function:

...
use('Shapely>=1.1.0')

from geo import shape, asShape
from point import Point asPoint
from linestring import LineString, asLineString
from polygon import Polygon, asPolygon
from multipoint import MultiPoint, asMultiPoint
from multilinestring import MultiLineString, asMultiLineString
from multipolygon import MultiPolygon, asMultiPolygon
from collection import GeometryCollection

Users can override the default GEOS providers by writing and installing (with setuptools) new packages that provide the same named end points, and "using" those at run time:

from shapely import geometry
geometry.use('MyGeometry')
...

point = geometry.Point(0.0, 0.0)
x = point.buffer(10.0) # calls on provider defined by MyGeometry package

In theory, this makes it possible to a write an application using Shapely that can run on either C Python, Jython, or IronPython using the appropriate backend for the platform, as long as setuptools and pkg_resources work. I've read that they will in the next Jython release. IronPython seems to be a little farther behind. In practice, the plugin framework is helping to improve the testability and quality of Shapely's code.

Comments

Re: Plugins for Shapely

Author: brent

thanks sean. very cool.

for those who didnt know about setuptools entry points, this is a good intro:

http://lucumr.pocoo.org/blogarchive/setuptools-plugins

Critique of WxS, en Français

I think this is the first time I've been translated. Thank you, René-luc D'Hont. I read French just well enough to verify that it's a faithful translation. I disagree with the comment at the end about GML's fitness: GML is XML, which is right for the Web; GML has a unique content type (not just text/xml), which is also right for the Web. The problems with GML are XML Schema and that it's not RDF.

Comments

Re: Critique of WxS, en Français.

Author: ReLuc

Well, GML is right for Web, I apologize. I just wanted to say that GML is hard to use in a browser environment. GeoJSON and GeoRSS are best designing for Web. And I'm agree with you about XML SChema in GML!

Making data more citable

Shorter Kurt Schwehr: your data needs a cool URI.

Bryan Lawrence is another scientist that blogs about this issue, including the messy details of citing data in traditional hypertext-less paper journals. Of course, getting credit for publishing digital data is a completely different, social issue, which probably has to wait for a generational change in the sciences.

Anglo-European Open Source Archaeo/Geo/GIS events?

So, I'm going to be in Montpellier, France, for one year starting 1 June. I'm completely new to the neighborhood. I've got the First Open Source GIS UK Conference, EuroPython 2009, and OGRS 2009 on my calendar. What else is going on?

Comments

Re: Anglo-European Open Source Archaeo/Geo/GIS events?

Author: Vincent HEURTEAUX

Hello Sean,

Welcome in Montpellier !

If you want to meet people working in Opensource Geospatial project Geomatys folk's are based here and will attend to OGRS in July, then just e-mail us if you want to travel with us or just talk about geospatial stuff.

Cheers,

Vincent

Re: Anglo-European Open Source Archaeo/Geo/GIS events?

Author: Mateusz Loskot

Sean,

Looks like there is a chance to have a beer together if you will be visiting UK :-)

Mat

Re: Anglo-European Open Source Archaeo/Geo/GIS events?

Author: Olivier

Hi Sean,

There's also PgDay Europe in Paris, this year, not yet scheduled, but something like 2 days in October.

Olivier

Re: Anglo-European Open Source Archaeo/Geo/GIS events?

Author: Stefano Costa

Sean, that's great news. You'll miss our late-april ArcheOpenSource event in Rome, but you should be still here (in Europe) for next year's workshop.

Looking forward to meet you then.

Stefano

Re: Anglo-European Open Source Archaeo/Geo/GIS events?

Author: Sean

Merci, grazie, and thanks. I'm looking forward to this trip. Perhaps I'll even be able to host an ISAW sponsored event after we get established in the Old World :)

Re: Anglo-European Open Source Archaeo/Geo/GIS events?

Author: Schuyler Erle

Don't forget the OSM conference -- State of the Map 2009 in Amsterdam, second week of July:

http://www.stateofthemap.org/

What's the beef?

The answer to:

@sgillies What's the beef with OGC WMS and WFS?

requires a few more than 140 characters.

First, let me review the good about the OGC service architecture and its W*S specs. The OGC has made interoperability a top priority in GIS. Everybody recognizes this is a huge accomplishment. I do too. My favorite byproduct is the increasing priority of open access. It's no accident. The OGC intended that interoperability would lead to more open access to data, and it has. It's a wonderful thing. My other favorite, and perhaps more accidental, byproduct is that thinking of GIS services as interchangeable commodity components leads rather quickly to considering open source implementations. I also think the OGC has done a fine job identifying and standardizing the parameters of our common processes, and a generally good job on message formats. So much good, I must be in heaven, right?

My beef with W*S is that its architects didn't do their Web homework. Despite the "Web" in the name, service design isn't informed by Web architecture and the understanding of HTTP (the Hypertext Transfer Protocol) begins and ends with CGI (the Common Gateway Interface). W*S understands and uses the Web as an alchemist understood and used the elements. We bear the cost of needless reinvention: "Update sequence" instead of HTTP Expiration and Validation, "Web Geolinking Service" instead of standard HTTP interaction, "GeoDDS" instead of Atom. Despite the idea that W*S are designed to be transport-neutral, HTTP is the only significant "distributed computing type" (what the architects call "transport"). The USGS Framework WFS uses no other transport than HTTP. GeoBase uses no other transport than HTTP. Still, our "Web" services remain things that are not really of the Web.

Another minor beef is that in our interoperability fervor we have made standardization holy. The GIS community largely believes that standards should come before implementation, should be built in clean rooms by an elite group of standards scientists, and this stifles innovation. I depend on standards as much as anyone, but I feel we should be standardizing on best practices more than we currently do.

Comments

Re: What's the beef?

Author: Jachym

Further more, I'm missing propper support of SOAP in W*S specifications, which would make OGC "Webservice" to W3C "Webservice" (if I understand this well).This particular thing makes OGC OWS incompatible with Inspire, which is essential for european GISers.

Re: What's the beef?

Author: Sean

The only thing I'll say about SOAP as a transport, is that it isn't any more of a transport than HTTP is. Look at the mess that is a SOAP DCPType for WxS: WxS (transport independent) over SOAP (not a transport) over HTTP (not a transport) over TCP (ah, there's the actual transport).

Re: What's the beef?

Author: C. Reed

One minor disagreement - your last statement is incorrect. The vast majority of new OGC candidate standards are "birthed" in the hot bed of implementation and not in a "clean room". These birthing areas could be in the wild or in an OGC test bed. WMS and WFS were birthed around the same time as some other web standards (SOAP, WSDL, etc) but back then no one had even thought of implementing SOAP in 1998 and 1999. So, we are dealing with some legacy here.

Re: What's the beef?

Author: Sean

Carl (yes?), I agree that there seems to be a positive trend (GeoRSS, KML, GeoPDF), but is it really a sea change in how the OGC makes standards?