2008 (old posts, page 5)

Pleistocene World WMS?

Yo, Lazyweb: is there a publicly available WMS providing maps of the Pleistocene world, or some layers thereof?

http://sgillies.net/images/pleistmap.jpg

(Image from USGS pub http://pubs.usgs.gov/gip/continents/.)

Comments

Re: Pleistocene World WMS?

Author: Ian Turton

If you've got the data we've got the WMS.

Re: Pleistocene World WMS?

Author: Sean

If I had the data I would probably render tiles for my application with TileCache.

Shapely Mention

It feels good to read about Shapely being put to use in EveryBlock alongside OpenLayers, Mapnik, PostGIS, OGR, and TileCache.

Comments

Re: Shapely Mention

Author: Allan Doyle

That EveryBlock blog post sure shows the good side of the open source geo stack. A refreshing change from this and this.

Re: Shapely Mention

Author: Matt Priour

I have a question about both Shapely & Worldmill. Do either of them depend on the OGR library or can either of them be used without having that library available?

Re: Shapely Mention

Author: Sean

The Windows installer for Shapely includes a GEOS DLL and has no other dependencies. My intention is to one day include the GDAL DLL in WorldMill.

Re: Shapely Mention

Author: Yves Moisan

I had the same feeling as Allan about the page you point us to : it's a refreshing post. GE is becoming so entrenched in certain sectors of activities that people tend to forget they just can't simply copy/paste their images in a report without infringing satellite photo license terms and I'm sure a whole list of other legal terms. One nasty side effect of GE's ubiquitousness is that it further subdues the meaning of free as in freedom. Why bother with freedom when we have it all in GE? Still, GE definitely is a useful tool and kml output is so easy to generate these days ...

Nit of the Day

RSS 1.0 is a format, RSS 2.0 is a format, Atom is a format. GeoRSS is not a document format as such: it is a trio (one deprecated) of ways to format geospatial metadata within documents.

Comments

Re: Nit of the Day

Author: Yves Moisan

To dot the i's and cross the t's for noobs like me, the "trio" refers to RSS 1, 2 and Atom ? And the deprecated one is RSS 1.0 ?

Re: Nit of the Day

Author: Jason Birch

Lol... I certainly understand this, but there's no way that I'm going to say "RSS with GeoRSS-formatted spatial metadata" every time I refer to an RSS feed that contains GeoRSS.

Re: Nit of the Day

Author: Jason Birch

Yves, I would expect that Sean is referring to the three ways that GeoRSS allows you to specify the geometries: Simple, GML, and W3C Geo. The last is the deprecated one. http://www.georss.org/

Re: Nit of the Day

Author: Yves Moisan

@Jason. I just saw it on the GeoRSS site. In fact, I was just coming back to apologize for the noise, but you got there before me. When I say I'm a noob, I mean it ;-). Cheers,

Re: Nit of the Day

Author: Christopher Schmidt

I'd actually argue that it isn't even that: It's simply a syntax for adding geographically-centric tags to *any* element in XML. (I think I even got that confirmed on a GeoRSS conference call.)

Re: Nit of the Day

Author: Artem Pavlenko

Chris, I think you're right here - it is just a syntax. Anyways, this is such trivial stuff (GeoRSS). Is it time to move on? ;)

Re: Nit of the Day

Author: Sean

What do you mean by "move on", Artem? Do you mean move on from GeoRSS to something else? Do you mean move on to doing interesting things with Atom and GML? I think we already are doing that.

Re: Nit of the Day

Author: Artem Pavlenko

Sean, is it me or has GeoRSS been 'over-blogged'? Don't get me wrong, it is a useful micro format (or just a syntax) for mixing in spatial data. It serves its purpose well. But it's only a tiny fraction of the interesting things around :D

Re: Nit of the Day

Author: Sean

I agree that GeoRSS is blogged out. The perception that it is a format like the shapefile is one consequence. On the other hand, I don't think that building geospatial applications using Atom and Atompub type architecture is being blogged nearly enough.

Atom and GML Simple for OpenLayers

Update (2008-02-21)

Protocol discussion ongoing at: http://trac.openlayers.org/wiki/Proposal/Protocols

This week I finally got around to working on a task that's been haunting me: OpenLayers as an AtomPub client. Chris Schmidt has done much of the work already, but I'm going to see it all the way through over the next few months.

A quick review of AtomPub and how it relates to geospatial applications is probably needed here. There are 3 different types of resources in the Atom Publishing Protocol (omitting for now workspaces): services, collections, and entries. the analogous components in a GIS architecture are: services, featuretypes (or coverages), and features. An AtomPub service document is in some ways like an OWS capabilities document, but simpler because AtomPub doesn't fool around with non-HTTP semantics and bakes more metadata into its single protocol. An Atom feed document is rather a lot like a WFS feature collection document, and an Atom entry is also much like a GML feature, but much more standardized. OpenLayers does WFS already, and needs little modification to be a good AtomPub client.

Here's the little modification that I have done: a new OpenLayers.Format.Atom lets me ignore the jumble of namespaces and elements that is RSS 1 and 2. This Atom.js is simpler than GeoRSS.js. The Atom entry atom:content element is the prime avenue for extending Atom, and I've also added support for that. Best practice for georeferencing Atom entries is to use a GML simple features geometry (see OGC 06-049r1) within a georss:where element and OpenLayers.Format.GMLSF supports simple features GML within my new Atom format. The GMLSF format writes gml:pos and gml:posList only, no gml:coordinates, and uses "exterior" and "interior" as ring element tags. On the other hand, it is just as forgiving as OpenLayers.Format.GML when reading. So now an OpenLayers app can write well calibrated feature entries to be POSTed to an AtomPub-style collection like the one at http://sgillies.net/kw/++rest++knowhere/demo. (I'll explain the odd URL in a future post).

That collection is live. I've copied over a few entries from my Hammock app like so:

sean@lenny:~$ curl -X GET http://sgillies.net/hammock/places/6.atom > 6.atom
sean@lenny:~$ curl -X POST -v \
-H "Content-Type: application/atom+xml" \
-H "Slug: Navarro Vineyards" \
--data @6.atom \
http://sgillies.net/kw/++rest++knowhere/demo
* About to connect() to sgillies.net port 80
*   Trying ...... connected
* Connected to sgillies.net (...) port 80
> POST /kw/++rest++knowhere/demo HTTP/1.1
> User-Agent: curl/7.15.5 (i486-pc-linux-gnu) ...
> Host: sgillies.net
> Accept: */*
> Content-Type: application/atom+xml
> Slug: Wehlener Sonnenuhr
> Content-Length: 1301
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
HTTP/1.1 201 Created
< Date: Sun, 17 Feb 2008 20:46:21 GMT
< Server: Twisted/2.5.0 ...
< Content-Length: 73
< Accept-Ranges: bytes
< X-Content-Type-Warning: guessed from content
< Location: http://sgillies.net/kw/++rest++knowhere/demo/wehlener-sonnenuhr
< Content-Type: text/plain;charset=utf-8
Connection #0 to host sgillies.net left intact
* Closing connection #0
Location: http://sgillies.net/kw/++rest++knowhere/demo/wehlener-sonnenuhr

(The ability to copy resources is built into HTTP.) If you browse to http://sgillies.net/kw/demo/map (and wait a few seconds for a pile of javascript to download) you'll see this placemark displayed among others as a vector layer in an OpenLayers map of the demo collection. Toward implementing AtomPub, I have a second vector to serve as a buffer for to-be-posted placemarks. Draw a new geometry in the map using the OpenLayers tools and you'll see it in green. Then set a title, summary, text, a placemark name slug, and click "post". If you've picked a slug that collides with an existing placemark name, you'll see an error message. Otherwise, you'll see the location of the newly created placemark.

Still yet to do are placemark edit and deletion as Chris's AtomPub demo already does. I think that getting this right will require a new OpenLayers.Layer.AtomPub, one that behaves much like the existing WFS layer, but interacts with the server using GET/PUT/DELETE/POST. The protocol implementation, such as it is, currently exists in this page template. The only real novelty in it is the use of the HTTP Slug request header to make suggestions for resource names.

function postPlacemark() {
  var feature = buffer.features[0];
  feature.attributes.title = $("pm-title").value;
  feature.attributes.description = $("pm-summary").value;
  feature.attributes["content"] = $("pm-content").value;
  var atom = format.write(feature);
  var options = {
    method: "post",
    contentType: "application/atom+xml",
    postBody: atom,
    onSuccess: updatePage,
    onFailure: function(xhr) {
      update_status("Failed post (status code "+xhr.status+"). Check your URL.");
    },
  };
  var slug = $("pm-slug").value;
  if (slug.length > 0) {
    options.requestHeaders = ["Slug", slug];
  }
  new OpenLayers.Ajax.Request(collection_URL, options);
}

My server has the final say, and currently just lowercases the slugs and replaces whitespace with a dash.

I submitted the new Atom and GMLSF formats to OpenLayers (#1366) and look forward to working with the developers to get them into an upcoming release.

Comments

Re: Atom and GML Simple for OpenLayers

Author: Yves Moisan

Nice! Eventually being able to delete a vertex while editing (or after) would be useful. I'm a dyslexic typer :-)

Re: Atom and GML Simple for OpenLayers

Author: Tim Schaub

Sounds good Sean. The scattered protocol logic in OpenLayers has been bugging me for a while. I've been tossing around ideas for a protocol class (related to vector CRUD protocols). Before you go following the WFS mess (in OpenLayers) and create an AtomPub layer, please get in touch. I'd like to work on something together that would benefit us all.

Re: Atom and GML Simple for OpenLayers

Author: Brian Flood

hi sean can you keep me in the loop as well? I'd be happy to do whatever I can with the OpenLayers support but I'm also interested in a .net client to AtomPub (for use in a non web editor like ArcMap). It would be nice to syncronize with what you all have done so far. also, of interest is MS's recent decision to make AtomPub the publishing basis for their future ADO.NET Data Services [1]. Their Url schema, while open, is slightly different then what has been discussed for geo-enabled atompub feed (filtering, querying etc). any thoughts? cheers brian [1] AtomPub in Astoria Lots more info in this document: Astoria link

Re: Atom and GML Simple for OpenLayers

Author: Sean

Brian, we're going to take the discussion here: http://trac.openlayers.org/wiki/Proposal/Protocols (initial version by Chris Schmidt). I don't follow Astoria at all, but its convergence with AtomPub seems to be big and good news.

Still Not Getting it

Some guy who either can't give up the straw man argument or still doesn't get it writes:

Just like a biker ends up carrying more or less the same weight -- whether as a bike or as a lock, the GIS system users end up paying, one way or another. They pay the software vendor if they choose a commercial platform, or they pay the implementer (in staff time or consultant fees) to glue together the pieces of the “free” solution.

And who exactly is arguing otherwise? Open source is about freedom, not about cost-free operation. How many times does this have to be said?

Comments

Re: Still Not Getting it

Author: Gary

It's a good argument when you believe that the only thing open source has going for it is that it's free.

Re: Still Not Getting it

Author: atanas entchev

I am the author of the blog you are referencing. Just like a diamond, this issue has many sides (you can tell that I like analogies). The developer community sees one side, and for that side, represented by your opinion, I am restating the obvious. There are other sides. My blog is targeted to my readers, who are also my clients or prospects. They are municipal engineers, planners, administrators. And a lot of them "don't get it" if I have to use your terminology. A lot of municipal GIS implementers, with little or no IT experience, think that open source *does* equal free. For them is the bicycle analogy.

Re: Still Not Getting it

Author: Sean

So why don't you educate them about the freedom aspect? Why tell them only that between open and proprietary it's all a wash?

Re: Still Not Getting it

Author: James Fee

It just amazes me how hard it is for folks to get beyond the cost aspect of FOSS. I never bring up costs in my marketing, but the freedom FOSS offers. It seems to me folks don't even listen to what I'm saying. My hats off to those that have been dealing with this problem for years.

Re: Still Not Getting it

Author: alexamici

Around here for GIS commercial software the customer pays the licenses and the implementer, expecially in the server space. I might just be lucky with my customers, but apparently most of them get it quite right: no license + full price service = 50% off.

Re: Still Not Getting it

Author: Dave Lowther

@alexamici: ABSOLUTELY. I work with both flavors and my fees usually do not vary. Licensing does...

Re: Still Not Getting it

Author: guillaume sueur

How many time does it have to be said ? Many, maybe written on the moon, not to forget... In April, the french GeoEvenement will begin an "Open Source conferences session" (http://www.ortech.fr/geo-evenement/jeudi_5_avril.php) by this conference (harldy translated) : "Open-Source software, true opportunity or cost-free illusion ?".

Re: Still Not Getting it

Author: atanas entchev

First off, for the record, I am not affiliated with any software vendor. Secondly, I work in an environment totally dominated by ESRI. In New Jersey, the state and all county governments are heavily invested in ESRI technology. So, to a would-be local NJ municipal implementer, the question is not 'what?' but 'how much?'. Typically, the first words out of a NJ municipal official's mouth would be that their town wants to implement ArcView (*not* GIS), and would want to know the cost. The 'freedom' that the developer community values in open source is very different from the 'freedom' a municipal official wants -- the freedom from worry that there will be no support tomorrow for the solution they implement today. On a very rare occasion (I have a handful of examples) will a municipality see the perceived zero cost of implementing FOSS as a sufficiently attractive offset for the perceived uncertainty of implementing FOSS. So, without taking sides, my post points out that the implementation cost of FOSS is not 'zero'. I hope I was clear.

Re: Still Not Getting it

Author: Yves Moisan

atanas entchev says "In New Jersey, the state and all county governments are heavily invested in ESRI technology". Same here, north of the border. Even NGO's are in that situation ! And why is that ? Part of the answer lies in the fact that all our universities have almost solely proprietary geomatics software labs. We should start a domain : edu4edu.org, as in education for educational institutions.

Re: Still Not Getting it

Author: atanas entchev

Yves Moisan says: "Part of the answer lies in the fact that all our universities have almost solely proprietary geomatics software labs." How true this is! I was indoctrinated into GIS while doing graduate studies at Rutgers University in New Brunswick, NJ in 1991. We used PC ARC/INFO. As I was graduating, ESRI released ArcView 1, which promptly found its way into the Rutgers computer lab. The rest is history, as they say.

Re: Still Not Getting it

Author: Sean

Atanas, your post was half flamebait, half cute analogy that doesn't hold up. Linux a wasted effort? Open source GIS a wasted effort by implication? Ridiculous.

Re: Still Not Getting it

Author: atanas entchev

Posting 'flamebait' wasn't my intention, but I should have known better, entering a minefield... :)

Re: Still Not Getting it

Author: Chris

Atanas: Maybe your NJ municipal purchasers need to be gently encouraged to think outside the big box labelled "ESRI"? Talk to other similar organisations who are already exploring alternatives that they think might save them time/money? Try Jason Birch at Nanaimo, BC, for example: http://www.jasonbirch.com/nodes/category/nanaimo/ James Fee is right about the nature of "freedom" in this context, and here in the UK and the wider EU, we are seeing a growing trend of public sector organisations looking at "free" alternatives to the big lock-in contracts with the likes of Microsoft etc for various application areas, even if they are not yet ready to let go of MS-Mommy's hand completely. It's very early days yet, but it looks to me like the GIS sector may be at the beginning of a similar process. And even in the paid-for GIS market, ESRI is not the only option, it's just the biggest (in all kinds of ways :0)).

Growth of the "GeoWeb"

Raj Singh writes this about growth of OGC services and standards:

I was over at Harvard yesterday talking to people from the GSD, the Herbarium, the Library, the new Center for Geographic Analysis, and MassGIS. It was great to see some old friends and make some new ones. One topic that came up was why you don’t see OGC standards in widespread usage. I argued that the geospatial Web is where the regular Web was in 1996, when the tech industry thought that HTML was so easy and powerful that everyone would build their own Web sites – not by writing HTML but by using tools like Dreamweaver. As it turns out, that was still way too high a barrier to entry. People didn’t want the hassle of designing a site from scratch. They wanted to post a blog entry or a MySpace page. That’s when the Web saw a real quantum leap in content.

In 1995, the number of web sites increased from about 10,000 to about 100,000. By the end of 1996, there were about 600,000 web sites. If there are this many WxS servers out there, and the number is growing at such a rate, I'm just not seeing it. My own take is that the WxS-based "GeoWeb" is more like Gopher circa 1993, before the advent of the Web: useful, technically adequate for many purposes, but ultimately just an experiment in web-building. There are design defects in OGC protocols (which use HTTP while keeping it at arm's length) that Raj does not acknowledge here: it's not just a matter of making better geo-editors. WxS was a worthy experiment, but ultimately not the way forward. The explosive growth that Raj is looking forward to is going to happen instead on the new axis of KML (with GML) and HTTP.

More Thinking Beyond the Spatial RDBMS

Update (2008-02-05): Is it a meme yet? Gary Sherman is thinking outside the RDBMS box too.

Interesting question from Kirk Kuykendall:

What I’m proposing here sure sounds a lot like what Hadoop does. I wonder if we will ever see the day when we can use something like Hadoop for geoprocessing with ArcObjects on .NET?

The answer is yes, right now. Maybe on windows, for sure on Linux. The why you'd want to put proprietary per-server-licensed software in the mix – when the point of Hadoop is to leverage the combination of commodity hardware and open source – escapes me. Streaming mappers using JTS or GEOS (or Shapely) seem like the way to go. I'd almost bet that somebody somewhere is already doing this, perhaps with their own proprietary geoprocessing software instead of open source. Now what we really need is spatial indexes for HDFS ... maybe Paul Ramsey is interested in tackling this at his new gig?

Entity Tag

OGC update sequence? Why bother when HTTP/1.1 already specifies ETag?

Comments

Re: Entity Tag

Author: Christopher Schmidt

I'm glad I wasn't the only one who had that thought.

Re: Entity Tag

Author: Allan Doyle

History. Inertia. The OGC started in the era of CORBA/COM/OLE/etc. and back then it was considered a virtue to remain completely independent of the underlying transport. So the specs tended to accrete ideas from various places and that accretion was not perfect (ha! there's an understatement, right?). Update sequence actually mimics the Serial member of the DNS SOA record (section 3.3.13. of RFC 1035 A number of other DNS ideas were also batted around, mainly TTL. So much for history, now for inertia. The question becomes, how to best morph from the hodgepodge that is WxS to actual web services? Can it happen inside OGC? Or does it have to happen outside? Can there be a gradual shift, or should there be a clean break? The Geo REST community would be seen as the natural place to start this kind of a thing. However, given my recent experiences with GeoRSS and GeoJSON, I'm not sure whether it can be done by a purely volunteer group. But maybe a week in a monastery could focus enough minds to do it.

Re: Entity Tag

Author: Sean

It think it has to be an outside effort. I care about this stuff and my employer is not an OGC member.

Re: Entity Tag

Author: Homme Zwaagstra

Why indeed! If you're using mapscript I guess you *could* use the MAP/WEB/METADATA/ows_updatesequence to set your own Etag/Last-Modified header, but then if you're scripting you'd probably have far more suitable places to get that data from anyway.

FDO and OSGeo Incubation

Geoff Zeiss writes:

Graduating OSGEO Incubation includes requirements for open community operation, a responsible project governance model, code provenance and license verification and general good project operation. Graduating incubation is the OSGeo seal of approval for a project and gives potential users of the project added confidence in the viability and safety of the project.

It looks like it is up to cranky old me to point out that of FDO's 17 committers, "haris" is the only one who is not an employee of or contractor to Autodesk. I'm not counting "timstrang", who has 1 commit for 0 lines of code. 13 Autodesk employees, 2 Autodesk contractors, and 1 independent committer.

Don't get me wrong, FDO looks like a technically excellent project, and they've done their best to do project business in the open and recruit new people. But it seems to me that it's failed to attract a diverse group of contributors. It probably would not have been graduated yet from an incubator like Apache's. Do non-diverse open source projects suck? No. I'm almost sole committer on several projects that kick ass, but I'll be the first to admit that they wouldn't be ready to graduate from the Apache incubator (even if they were accepted).

Maybe the real issue is whether the OSGeo brand is to represent standards as high as Apache's. It might represent lower standards, and that would be fine. Many entities happily used GDAL and GRASS before they were OSGeo branded. However, if the brand is supposed to be the equal of Apache's, OSGeo's "A for effort" incubation policy needs to be reconsidered.

Update: a reader says I was wrong to count mloskot as an Autodesk contractor. That would make 2 independent committers.

Comments

Re: FDO and OSGeo Incubation

Author: Bill Thorp

Code with Punditry? Say it ain't so, Manifold! Are they lacking users? Or just lacking bug fixes? All of the OS code patches I've contributed were on horribly, obviously flawed projects. Come to think of it, I need to check out this FDO software!!

Geo Web Frameworks

Howard Butler gives me something to pin on the Pleiades bulletin board (nothing motivates quite like adversity):

[...] I do have a lot of experience with Zope/Plone, and GeoDjango fills some big holes in the Zope stack when it comes to geospatial stuff that I don’t see ever being filled.

and then goes on to write all kinds of sensible stuff about GeoDjango. I'm not a GeoDjango user, but I've benefited from interaction with the developers. Justin Bronn inspired me to apply ctypes to GEOS (Shapely is in some sense a rip-off of GeoDjango, which was itself inspired partly by PCL's geometry package), and meeting him was one of my FOSS4G 2007 highlights. It's a worthy project.

The biggest reason to go with Django over Zope is, without a doubt, PostGIS: as a open source GIS developer/consultant you've already invested huge amounts of data and brain cells in Paul Ramsey's killer app while developing MapServer maps, and every single bit of this investment carries over into Django application development. You could say the same for Rails. None of your MapServer skills translate into Zope skills.

Of course, PostGIS can't help you at organizational boundaries, and Django doesn't provide "GeoWeb" services any more than Zope does. It's up to GeoDjango to develop WxS (or alternative) implementations.

Here's a disturbing thought: if only MapServer (the CGI program) implemented the Model-View-Controller pattern ... you wouldn't need GeoDjango at all. Or Django, for that matter. Use OGR as your connector to data (geospatial or not), and make entire dynamic "GeoWeb" sites entirely through mapfile configuration and CGI query string parameters.