2008 (old posts, page 7)

Fire Eagle and Shapely

Fire Eagle GeoJSON is fixed. Shapely is great for doing things with your data.

>>> import simplejson
>>> data = simplejson.loads(json)
>>> from shapely.geometry import asShape
>>> for location in data['user']['location_hierarchy']:
...     print location['name']
...     print asShape(location['geometry']).wkt
...
600 N Sherwood St, Fort Collins, CO
POINT (-105.0842514037999962 40.5944633483999979)
Fort Collins, CO
POLYGON ((-105.1527862548999934 40.4800186156999970, -104.9821014404000010 40.4
800186156999970, -104.9821014404000010 40.6392784119000012, -105.152786254899993
4 40.6392784119000012, -105.1527862548999934 40.4800186156999970))
Colorado
POLYGON ((-109.0602493285999941 36.9923210143999981, -102.0415573119999948 36.9
923210143999981, -102.0415573119999948 41.0033493042000003, -109.060249328599994
1 41.0033493042000003, -109.0602493285999941 36.9923210143999981))
United States
POLYGON ((-167.2764129638999862 18.9108390807999989, -66.6879425048999934 18.91
08390807999989, -66.6879425048999934 72.8960571288999972, -167.2764129638999862
72.8960571288999972, -167.2764129638999862 18.9108390807999989))

The Fire Eagle tutorial app is also a neat introduction to OAuth.

Fire Eagle GeoJSON Oops

Update (2008-03-13)

Fixed.

Update (2008-03-12)

Problem to be solved this week, I hear.

Sharper eyes than mine noticed that the Fire Eagle "geojson" format is a bit out of date relative to the spec. I checked my location using the walkthrough app:

>>> import simplejson
>>> data = simplejson.loads(json)
>>> from pprint import pprint
>>> pprint(data['user']['location_hierarchy'][0]['geojson'])
{u'coordinates': [-105.0842514038, 40.594463348399998], u'type': u'Point'}
>>> from shapely.geometry import asShape
>>> x = asShape(data['user']['location_hierarchy'][0]['geojson'])
>>> x
<shapely.geometry.point.PointAdapter object at 0x834ce6c>
>>> x.wkt
'POINT (-105.0842514037999962 40.5944633483999979)'

Fire Eagle point representations are okay, and can be adapted into Shapely geometries. How about the polygon in the location hierarchy?

>>> pprint(data['user']['location_hierarchy'][1]['geojson'])
{u'bbox': [[-105.15278625489999, 40.480018615699997],
           [-104.9821014404, 40.639278411900001]],
 u'coordinates': [[-105.15278625489999, 40.480018615699997],
                  [-104.9821014404, 40.480018615699997],
                  [-104.9821014404, 40.639278411900001],
                  [-105.15278625489999, 40.639278411900001],
                  [-105.15278625489999, 40.480018615699997]],
 u'type': u'Polygon'}
>>> x = asShape(data['user']['location_hierarchy'][1]['geojson'])
>>> x.exterior
Traceback (most recent call last):
...
TypeError: object of type 'float' has no len()

The Fire Eagle polygon representation is wrong: it should be a list of rings, not a ring. (Memo to myself: clearly, Shapely needs to handle this situation better, providing a less ambiguous exception at the very least.)

If we insert that ring into a list, it would be fine:

>>> fe_geom = data['user']['location_hierarchy'][1]['geojson']
>>> fixed_geom = {'type': fe_geom['type'], 'coordinates': [fe_geom['coordinates']]}
>>> x = asShape(fixed_geom)
>>> x
<shapely.geometry.polygon.PolygonAdapter object at 0xb7d0746c>
>>> x.exterior.length
0.65988922139999318
>>> x.area
0.027183228771704648

We've got a problem.

Sunset on Nebuka

The last traces of snow will be gone from my yard today and these evergreen bunching onions have been enjoying the warmth and sun. Being buried in snow all winter didn't bother them at all.

http://sgillies.net/images/nebuka.jpg

As I've said before, one of the luxuries of small-scale gardening is the freedom to roll the dice against bad weather. Lettuce, spinach, radishes, turnips, and peas are going in this weekend; worst case is that I lose an afternoon and a few packets of seeds. According to the CPC March-April-May seasonal outlook, it is likely to be another warm and dry spring in Northern Colorado on account of the strong La Nina conditions:

THE TEMPERATURE OUTLOOK FOR MARCH FAVORS ABOVE AVERAGE TEMPERATURES ACROSS MOST OF THE SOUTH FROM NEW MEXICO EASTWARD... ACROSS THE SOUTHERN AND CENTRAL PLAINS... AND ALONG THE EASTERN SEABOARD FROM FLORIDA NORTHWARD THROUGH THE NORTHERN MID-ATLANTIC. BELOW-AVERAGE TEMPERATURES ARE MOST LIKELY IN THE PACIFIC NORTHWEST AND IN SOUTHERN CALIFORNIA AND ALSO IN SOUTHERN ALASKA AND THE ALASKAN PANHANDLE. THE OBSERVED BELOW-AVERAGE SSTS ALONG THE CALIFORNIA COAST FAVOR A CONTINUATION OF BELOW AVERAGE TEMPERATURES IN SOUTHERN CALIFORNIA. THE TEMPERATURE FORECAST OVER THE REST OF THE CONUS AND ALASKA IS CONSISTENT WITH OBSERVATIONS FROM PREVIOUS STRONG LA NINA EPISODES. THE TEMPERATURE FORECAST IN THESE REGIONS IS ALSO SUPPORTED BY THE SMLR AND THE CCA, AS WELL AS UPDATED FORECASTS FROM THE CFS.

THE PRECIPITATION FORECAST FOR MARCH USED FEBRUARY - APRIL LA NINA COMPOSITES AS THE STARTING POINT, WHICH INDICATES MEAN DRY CONDITIONS ACROSS THE ENTIRE SOUTHERN U.S. AND WETTER THAN AVERAGE CONDITIONS IN THE PACIFIC NORTHWEST AND IN THE OHIO AND TENNESSEE VALLEYS. INDICATIONS IN THE SHORT TERM FAVORED EXPANDING THE REGION IN THE OHIO AND TENNESSEE VALLEYS INTO PARTS OF THE NORTHEAST... AND SCALING BACK SOME OF THE DRY IN THE SOUTHEAST. DRIER THAN AVERAGE CONDITIONS ARE ALSO FAVORED IN SOUTHERN ALASKA... AS INDICATED BY THE CFS AND THE CCA.

Comments

Re: Sunset on Nebuka

Author: James Fee

Finally, the sooner the snow melts in Colorado, the sooner it can drain into the Colorado River and into Arizona's water allotment. We really do appreciate all the hard work you folks up there do for us and our water needs. Not too happy about the dry spring up there, but at least the snow pack will get us through another year of filling our pools with Colorado River water.

Re: Sunset on Nebuka

Author: Sean

My friend Eric in Durango has your snow right here.

Re: Sunset on Nebuka

Author: James Fee

Sweet, I can't wait to wash down the sidewalk in front of my house with that stuff.

Re: Sunset on Nebuka

Author: Sean

I'm pretty sure Colorado River water is going to farmers and golf courses. Aren't you draining your own aquifer dry for pools and sidewalk washing? Note to readers outside the Southwest: some of James's neighbors in the desert actually do hose down the paved areas of their properties using treated drinking water instead of using a broom. It's an amazing mindset here.

Re: Sunset on Nebuka

Author: James Fee

The Central Arizona Project pumps Colorado River water into our aquifer. I guess one could say we don't "directly" use the Colorado River water for drinking, but we take all our allotment and pump it directly into the ground. When Tempe,AZ decided to "fill" the Salt River to make Tempe Town Lake, they used CAP water because letting water out of the Salt River dams was too expensive. I will say Tempe finally is going to use reclaimed water to "top off" the lake from evaporation, rather than taking more CAP water. It is true that CAP is supposed to give water to agriculture uses, but the realities has meant that it is being pumped into aquifers that are used by both agriculture and municipalities. Plus as Phoenix has grown, land that once was used for farming has been replaced with houses which use CAP water. Arizona has more water than it can use thanks to how much of the Colorado River fronts our state. This will change in time, but keep the water flowing down from Colorado.

ArcDeveloper REST

ArcGIS developers are embracing Web architecture as enthusiastically as we on the open source side. Maybe even a bit more. Via Dave Bouwman, and James Fee.

The community that would become OSGeo has long criticized ESRI and its users for tepid support of WxS protocols and formats. I suspect (cynic, me) that lesser enthusiasm had more to do with business competition than to foresight of the changing tide, but it seems to me that ESRI and the ArcGIS server users, having drunk less of the kool-aid, are now in a better position to innovate with REST than the free and open source GIS community. The Windows tools don't hurt either.

More WxS Hinting From Feeds

Looking back, I realize I forgot to mention an easy way to help ordinary web clients find and tap into WMS from a feed: HTML forms. Here's a feed entry representing a form:

<entry>
  <title>GetMap Form</title>
  <summary>Easy entry point to a web map service</summary>
  <link
    rel="alternate"
    type="text/html"
    href="http://example.com/getmap-form.html"
    />
  ...
</entry>

where getmap-form.html is:

<html>
  ...
  <form method="GET" action="http://example.com/wms">
    <input type="hidden" name="service" value="WMS"/>
    <input type="hidden" name="request" value="GetMap"/>
    <input type="text" name="layers" value="layer1,layer2"/>
    <input type="text" name="bbox" value="-180.0,-90.0,180.0,90.0"/>
  ...
  </form>
</html>

For a more concrete (though not exactly WMS) example, see this form.

Comments

Re: More WxS Hinting From Feeds

Author: Jeroen Ticheler

Hi Sean, Thanks for some great input! I removed my URI Template stuff from the feed again and now included a full GetMap request producing a PNG file and added a link with only the basic information and a (custom) hint. That should work for people that want to consume the feed information and exploit the W*S services. The above is also a great idea, although it is tied to a specific client and that's something I hope not to do to make it easy for other clients to use the resources in a clean way. I may still include it though :-) Ciao, Jeroen

Re: More WxS Hinting From Feeds

Author: Sean

Tied to a specific client? Not really: there's a very wide range and huge number of web clients that handle HTML forms. There's nothing out there that handles the hints in your links. Specifically, I did a GET on one of your alternate links and it gave me a service exception document (along with a completely misleading HTTP "200 OK" status). Web clients expect dereferenceable, not bogus, URIs in a link.

Frugosapalooza Wrap Up

Helping Brian realize his idea was fun. Here's his summary. What's next?

There was considerable discussion as to what's next. We hope to offer "getting started" workshops in the coming months to help new users over the learning curve hump. In addition, getting experienced users together for more "deep dive" knowledge sharing is a priority.

I suspect that there are a lot of folks who can't make the trip to Capetown later this year. This might open a small market for open source GIS workshops and face-to-face meetings in this region, like the early MapServer/GDAL/PostGIS meetings. I'd like to be a part of that.

Representing OGC Services in Atom Feeds

After reading Jeroen's latest post I have a better understanding of what he's trying to accomplish. I too have a use case for representing OGC services in feeds. I'm a critic of the OGC service architecture but can't deny that they are being used and referenced in academic studies, and that there is therefore a need to model and represent them in an ORE resource map feed. I'm going to propose a way to do that and then work back toward Jeroen's use case.

Consider this feed, an ORE-like aggregation that Tom is using to demonstrate how we can form a federation of digital classics projects using a syndication architecture:

http://homepages.nyu.edu/%7Ete20/examples/nearby/cyrene.atom

How does one represent an OGC web map service (WMS) in this feed? As an entry:

<feed>
  ...

  <entry>
    <title>Cyrene WMS</title>
    <id>urn:uuid:56b8421c-5150-458d-ac92-ee9e698593b5</id>
    ...
    <link rel="alternate" href="???"/>
  </entry>

</feed>

What's the link href to be? My interpretation of the alternate link relation is that it should be the URI to the closest thing a WMS has to a "home page". Ideally that would be its capabilities document. (Aside: making an OWS capabilities document a plain old web resource instead of an RPC call would greatly improve the OGC service discovery story. I'm not the first to make this suggestion.)

The entry then becomes something like:

<entry>
  <title>Cyrene WMS</title>
  <id>urn:uuid:56b8421c-5150-458d-ac92-ee9e698593b5</id>
  ...
  <link
    rel="alternate"
    href="http://example.com/?service=WMS&request=GetCapabilities..."
    type="application/vnd.ogc.wms_xml"
    />
</entry>

It would also be very useful to aggregate into an ORE resource map specific images from a WMS, particularly if they are used as illustrations in publication:

<entry>
  <title>Figure 8: Cyrene Overview</title>
  <id>urn:uuid:408cdb6c-185d-4c45-862b-2809158e0c4f</id>
  ...
  <link
    rel="alternate"
    href="http://example.com/?service=WMS&request=GetMap&format=image/png&bbox=..."
    type="image/png"
    />
</entry>

There's less need for extra metadata in this case: specifying image/png says it all. It's implicit that clients will address the link href with a GET request, and that you can get a map image directly like this is one of the redeeming features of WMS.

So, that's how I see OGC services represented in an ORE-like feed. Jeroen's use case is different: he'd like to use a feed as a lite capabilities document. I'm not sure this is a great idea. Bypassing the service capabilities seems a bit dangerous. If it's only that capabilities documents are too big (and I've run into some that are 4 megabytes, which is ridiculous), providers should be encouraged to break these down into smaller collections of logically related feature types, coverages, and layers. Still too big? Take advantage of HTTP's built-in caching mechanisms. Provide ETags and Last-Modified times to help clients limit fetches and parsing. Implement caching proxies.

If you still need to represent GetMap (or other functions) in a feed, there are some possibilities, and some limitations. URI templates don't belong in atom:link. Instead, Joe Gregorio has proposed a link_template element. It's not official by any means, but in combination with a new relation (allowed by Atom), it could let you make an entry like so:

<entry>
  <title>GetMap Endpoint</title>
  <id>urn:uuid:09e7a48a-428a-4418-aef4-ea157806d9db</id>
  ...
  <t:link_template
    rel="http://opengis.net/relations/getmap-endpoint"
    href="http://example.com/?service=WMS&request=GetMap&format={format}&bbox={bbox}"
    />
</entry>

I think that a custom link relation could be the hint Jeroen is looking for. The OGC's content type problem still needs to be fixed.

Comments

Re: Representing OGC Services in Atom Feeds

Author: Doug Nebert

I have been considering the same problem lately, wishing to expose a synopsis of deeper metadata records in catalogs to commercial search engines. It appears that the GeoRSS/Atom track is the appropriate one, especially with the ability to represent the geo:box or gml:envelope, and the fact that Atom is extensible. While the majority of our synopsized resources are describing data, some number will also be describing services to access or present the data. In fact, multiple links could be described with the resource. For describing resources in the GEOSS Service Registry, where they exist, entries include fields for a URN to the service type (from ISO or OGC or other), a ServiceInfo field (to link to capabilities or WSDL or narrative), and a ServiceURL (aka the endpoint). The latter two may not always be different but not distinguishing them could lead to breaks in usability. See: http://geossregistries.info . I agree with the proposal for having a link property or two but would rather generalize the problem to flag 1) that it is a service resource, 2) that it is of a certain type, 3) where service information is located, and 4) the location of the service entrypoint stub. Obviously, this link could be repeated for various flavors of access -- data source, WMS access, WFS access, DVD order.

Re: Representing OGC Services in Atom Feeds

Author: Sean

The ORE-Atom authors suggest using rdf:type elements within an entry to indicate the type of the linked resource. So, if there was a URN for "OGC WMS capabilities request" (and it looks like there is), you might just add <rdf:type>urn:ogc:service:wms:...</rdf:type> to your entry. I don't know if there is a good solution to your #4: it's implicit that link hrefs should be requested using GET, and WxS implementors don't generally have any resource to be fetched at the online resource URL. The capabilities response is the closest thing you have to a service home page.

The Fear and Insecurity Industry

Speaking of breaking our addictions, how about our society's addiction to scaring itself silly with over-hyped movie-plot threats and bioterrorism risk maps? According to the map, I live in a medium risk zone ... should I be moving to Montana soon? How do these risk levels compare to the level of risk my family faces from reckless and/or drunk drivers on I-25? Or to the risk that an otherwise solidly patriotic food company sends botulism-tainted green beans or E. coli-infected ground beef to the pantry of my kid's daycare? Or to the risk that a uranium mine in my neighborhood turns into another Love Canal? Or to the risk that my low-income neighbors face because they can't afford health care and preventative diagnosis and treatment? If the risk of bioterrorism is actually greater than, or even equal to, the risk of these more mundane, everyday, and reducible threats, I'll eat my hat.

Fun With Shapely

Very cool. Is this neogeogenomics? I hope the JTS/GEOS architects are equally pleased. You could do this with the OGR Python bindings of course, but since there are no projections involved, and no GIS data to read – only the comparably baroque genetic software formats – there's little point in doing so. I designed Shapely to provide simple spatial functions and otherwise get out from under the feet of your imagination.

Comments

Re: Fun With Shapely

Author: brentp

but with ogr, it'd be more code to translate to/from numpy arrays for plotting -- unless ogr bindings provide the array interface and i just missed that. any chance for 1D spatial operators in geos, like: >>> LineString1D(astart, astop).overlaps(LineString1D(bstart, bstop)) for that kind of thing i use: >>> LineString([[astart, 0], [astop, 0]]).overlaps(...) maybe then "neogeogenomics" would catch on.

Re: Fun With Shapely

Author: Sean

I'm not sure that I understand you. The 1D calculations would be trivial, yes? Is the distance between their center points less than or equal to half the sum of their lengths?

Re: Fun With Shapely

Author: brentp

yes, they are trivial--and your method is simpler than i'd been using. But then for a Multi1D, you do that n times. and Multi1D to Multi1D, n*n times... maybe i'm too lazy.