2008 (old posts, page 4)

Watching the Telethon

I'm with Paul:

Summary: No matter whether you're building on ESRI or open source, if you are building something complex your staff will have to learn a few new skills. Their prior experience with core concepts like programming and IT will serve them well in both domains, and the learning curve will be no worse either way.

If your tech company doesn't have, or can't learn, the fundamental Javascript, SQL, and Java skills needed to deploy and run OpenLayers, PostGIS, and GeoServer (substitute C# and SharpMap for Java and GeoServer if you must), you'll be passed sooner or later by the companies that do.

Comments

Re: Watching the Telethon

Author: Dave

Yep. Me thinks there may be a shakeout of the traditional GIS consulting industry coming up...

Re: Watching the Telethon

Author: Paul Ramsey

Rather *another* shake up coming... I remember when I started the consulting gig here, all the big GIS consultancies in town were built around staffs of mad AML gurus, Arc/INFO data capture gnomes, and cartographic production. All the AML is gone, and the only cartographic shop still standing is adding an application development sideline.

Frugosapalooza Schedule

4 nights, 4 Front Range cities, smells like Otto's jacket [1]:

  • 2/19, Denver: Forest Room 5, 6PM-8PM
  • 2/26, Fort Collins: the Bean Cycle, 6PM-8PM
  • 2/27, Colorado Springs: Jack Quinn's Pub, 6PM-8PM
  • 3/4, Boulder: Boulder Beer Company, 6PM-8PM

Keep an eye on the wiki for details.

[1] Obligatory Simpsons reference, not actually true.

Iterators. Iterators. Iterators.

Update (2008-02-11): Kurt Schwehr is thinking about Python code quality too.

Stalking "Python GIS" with Google's blog search led me today to this:

import arcgisscripting
gp = arcgisscripting.create()
gp.toolbox = "analysis"

# Set the workspace
gp.workspace = "c:\\Houston\\CrimeData"

# Call ListFeatureClasses
fcs = gp.ListFeatureClasses( )

#loop through the feature classes and buffer
fcs.reset()
fc = fcs.next()

while fc:
     outName = "buffered_" + fc
     gp.buffer(fc, outName, "500 Feet")
     fc = fcs.next()

What a joyless way to work with Python. The requirement that programmers call reset() and next() is also a sure recipe for bugs. Ideally, the "enumeration objects" would implement the iterator protocol, and you'd just do this:

>>> gp.workspace = r"c:\Houston\CrimeData"
>>> for fc in gp.ListFeatureClasses():
...     outName = "buffered_" + fc
...     gp.buffer(fc, outName, "500 Feet")

Better yet, you're not going to need to pass an argument into the enumeration factory, so replace it with a property:

>>> gp.workspace = r"c:\Houston\CrimeData"
>>> for fc in gp.featureClasses:
...     outName = "buffered_" + fc
...     gp.buffer(fc, outName, "500 Feet")

Tight. Of course, I've got plans to do it even better in a future release of WorldMill. Something like:

>>> inws = mill.workspace(r"c:\Houston\CrimeData", "r")
>>> outws = mill.workspace(r"c:\temp", "w")
>>> for cname, collection in inws.items():
...     outws.add(buffer(collection), "buffered_" + cname)

Windows users: save on slashes by using Python's raw (r) strings as I did above.

Comments

Re: Iterators. Iterators. Iterators.

Author: Sean

There's a lot of code in my blog, something that distinguishes me from the purely punditrous. I've finally added syntax highlighting for the code. Go ahead, bring on the bike shed painting comments.

Re: Iterators. Iterators. Iterators.

Author: James Fee

Hmm, code over pictures of thongs. You might be on to something there.

Re: raw strings

Author: max

NTish versions of Windows(and so python running on those versions) understand forward slashes, like c:/blah, so no need for raw strings, unless you need to support win9x.

Re: Iterators. Iterators. Iterators.

Author: Sean

Python 2.5, at least, still favors "\\" (see os.path.sep) on Windows, as you can see when you do os.path.abspath("c:/temp/x/y/z"). Therefore, I think users will encounter fewer problems if they use the Windows convention in a raw string.

Shapely 1.0.1

A bug fix release: Shapely 1.0.1. Now you can safely use highly condensed operation-chaining expressions like this one:

Python 2.5.1 (r251:54863, May  2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> from shapely.geometry import Point
>>> numpy.asarray(Point(0.0, 0.0).buffer(1.0).boundary.coords)
array([[  1.00000000e+00,   0.00000000e+00],
       ...
     [  1.00000000e+00,   0.00000000e+00]])

ESRI's RESTful API

Back to business now. One of the surest ways to tell if there is any Web in a "GeoWeb" site is hit Ctrl-U. I just did that for http://serverx.esri.com/ArcGISJavaScriptAPI/SuperTuesday.html and I like what I see so far.

As I told Matt Priour, I know for a fact that there are developers at ESRI who get REST. Several are posters to the Geo-Web-REST group. It's clear from the demo that there is resource-oriented thinking going on at ESRI. Consider this map layer home page that you can find through the demo source: SuperTuesdaySample/MapServer/1. There you find metadata and links to subordinate resources such as a query. The HTML representation of the query is a form which doubles as a query description. I am 100 percent in favor of this approach; it's exactly what I've done with Mush. Traverse upward from the layer to the map server resource and you'll also find WSDL and a JSON service description. I'm not against service descriptions at all, but I wonder why not WADL? Or for that matter, why not OWS Capabilities?

This is good stuff. I'm curious to see how the PUT, DELETE, and POST story goes.

Postscript: Looks like James Fee beat me to "view source".

Comments

Re: ESRI's RESTful API

Author: James Fee

Jinx!

Feeding Birds Again

Bam! Something just hit the window here. Looking out, I saw this Sharp-shinned Hawk nail a Starling.

http://sgillies.net/images/lunchtime.jpg

I can't decide which feels more liberating: posting amateur bird photos, or writing about electoral politics.

Comments

Re: Feeding Birds Again

Author: Matt Perry

A couple weeks ago I watched a red-tail hawk nab a pigeon in mid-flight. The badly-injured pigeon somehow managed to escape, only to meet it's maker when it slammed into my office window. Apparently witnessing bird slaughter is just part of working in the GIS industry.

Re: Feeding Birds Again

Author: steven Citron-Pousty

Down with starlings - death to invasives. Just as an FYI - it is really hard to tell the difference between a Coopers and Sharp-Shinned hawk. They are really close in size especially a female sharp-shinned with a male cooper's. http://www.birds.cornell.edu/pfw/AboutBirdsandFeeding/accipiterIDtable.htm

Re: Feeding Birds Again

Author: Sean

My feelings exactly, Steve. I wonder if a Starling's flavor is as bad as its grooming and manners? I've seen this male (you can just barely make out the orange streaks on the belly in my photo) Sharp-shin around the house fairly often. It's a bit more than half the size of the Coopers. I saw a Goshawk while backpacking in Utah's Henry Mountains in 1987, but not one since.

Getting Political

Democrats caucus in Colorado. I've been a slacker about attending these in the past because the nomination has already been decided before the caucus date since I moved here in 1993. It looks like it's going to be an interesting process. At the very least I'll get to hang out with my activist neighbors for the evening and visit the Junior High School that my daughter would attend should we stay in this end of Fort Collins.

I would have liked to see Republicans pick a kook like Huckabee or a patent phony like Romney, but it seems like they are going to select the strongest of their options after all. McCain doesn't demonize minorities and immigrants, he's not anti-science, he's sensible about climate change and the environment, he's taken a stand against state use of torture (including the shameful and criminal torture practiced by agents of the US), and the media fluffs him like there's no tomorrow. He can run hard to the center in the general election without losing credibility with the right-wing base -- he's got the magical "Maverick" card -- and that will make him extremely hard to beat.

While McCain may be less terrible than his peers, he's still far from me on important issues, and I'll be voting against him. For me, the choice of who to vote for comes down to "electability". I admire both Clinton and Obama, but I think Obama is the better Democratic candidate this year, this campaign. His youth contrasts well against McCain's age, and I can't help but feel that the Republican base won't fire up against him quite as hotly as they would against Clinton. It's going to be a close election, and every single-issue Clinton-hating voter that stays on the bench counts.

Ha! The Denver Post feels differently about McCain. Probably a Colorado River rivalry thing.

Comments

Re: Getting Political

Author: Matt Ball

I'm in your camp Sean. I just attended a jam-packed Caucus in my Denver precinct where parking was hard to find. The turnout was tremendous, which is encouraging for both the state of mind, and voter empowerment. I'm happy to report that in my precinct Obama received 100 votes to Hillary's 37. I'm pleased with the quality of the candidates this year, but can't stand the thought of another dynasty win. We need some fresh ideas, enthusiasm, optimism and passion. Can't wait until the convention here in August.

Re: Getting Political

Author: Justin Bornn

he's taken a stand against state use of torture (including the shameful and criminal torture practiced by agents of the US)
Sorry Sean, I'm gonna have to call you out on this one. McCain was one of the U.S. Senators (along with Lindsey Graham and John Warner) primarily responsible for one of the worst pieces of legislation ever passed in U.S. history -- the Military Commissions Act of 2006 (MCA). Not only did the MCA legalize torture it also abolished habeus corpus, a foundation of Anglo-American legal tradition since the 13th century and a right guaranteed by the constitution (making the MCA prima facie unconstitutional to anyone with a shred of legal knowledge). Specifically, McCain's so-called "stand" was no more than:
capitulat[ing] almost in full to the White House, "winning" only the most meaninglessly symbolic linguistic changes to the bill while acquiescing to its most Draconian provisions
McCain, a POW who endured years of torture in North Vietnam, is an enabler of tyranny who crafted a law allowing the President to torture and detain American citizens indefinitely. In essence, McCain sold his soul for a bid at the presidency -- I bristle with disgust at the prospect of him taking office.

Re: Getting Political

Author: Bryan

... go the democrats! (Isn't McCain supposed to believe in Intelligent Design to the point where he could be a problem?)

Re: Getting Political

Author: Sean

The number of people at the caucus last night was nearly overwhelming. Obama won my precinct (and the state) handily. I wasn't listing the things I like about McCain, I was listing the qualities that will let him run to the center with some success. There's no doubt in my mind that if elected, he would continue to trade away our civil liberties in the name of security.

Re: Getting Political

Author: Justin Bronn

Sean, I knew you weren't promoting McCain -- it's clear from your post. I just wanted to address the misconception (perpetuated by the mainstream media) that he took a stand on torture, when the opposite is true. Needless to say, it's an issue I'm passionate about. I need to start my own blog rather than polluting your comments w/political links.

Re: Getting Political

Author: TimB

The choice of who to vote for comes down to "electability"? Sounds like the prom queen popularity elections in high school. I try to maintain faith that society will eventually back and vote for the candidate that has the best problem solving skills, ability to work with others, and views that best mesh with the voter. Yet, I am always saddened that candidates are elected based on popularity. I guess we get the government we deserve in the end, as incompetent as it may be :-(

Re: Getting Political

Author: Sean

Justin, do start your blog: I'd be a reader. Don't sweat the links. My blog has always been about politics, open source vs proprietary in particular. I'll use James Snell (who I read regularly for insights into AtomPub) as an example of a person who might stay at home on an Obama-McCain election night, but who will more likely come out to vote against Clinton. I'm not saying that this kind of political calculus should be the main factor in choosing, or that it necessarily selects the best presidents, but you can't ignore Clinton-fever any more than you can ignore gravity. Read George Packer's New Yorker article yet? That's a good one.

Re: Getting Political

Author: James Fee

Caucus seems like a hassle. I voted in our primary months ago (heck I can't even remember who I voted for). AZ automatically sends out absentee voter forms. I vote before they can start calling me asking me to vote. :)

Re: Getting Political

Author: KoS

Primaries are a waste of time and money. Put all the individuals who want to run for president on the ballot in November. Vote once and be done with it. One comment thou. You admiration for Clinton is way misplaced. But not surprising. KoS

Re: Getting Political

Author: Sean

Caucusing is time consuming indeed, but does put some humanity back into the political process.

Re: Getting Political

Author: KoS

One other comment. Might want to brush up the political commentary. "He can run hard to the center in the general election without losing credibility with the right-wing base" You do realize he already lost credibility with the righ-wing base? Guess you have missed all the caterwauling against McCain from the right. And it didn't start recently, been going on now for years. I can't vote for McCain, he has a few screws lose. But more importantly, the way he treated his first wife, sorta did a Newt. Plus some of his policy stands. I would love to vote for Obama, but can't due to some of his policies and his past voting stances while in IL. Otherwise seems like a good guy, granted he has baggage like the rest of them. So far, it would seem, his bags are much smaller than the rest. KoS

Long Live Chicago Crime

The transition of chicagocrime.org has been noted by geo-bloggers, but they are missing a key part of the story: the resources that had been hosted on chicagocrime.org are not dead at all. They have been moved to a new host. Their original URLs have been retired, but the resources themselves carry on. I'm not speaking figuratively. Stability is a virtue, but Web architecture makes it possible for resources to move when necessary. The chicagocrime.org move is a perfectly ordinary transition of resources on the Web.

How is this accomplished? Requests for http://chicagocrime.org are redirected to http://everyblock.com. I don't mean that you get a web page that says, "click here to go to the new web site". I don't mean that http://chicagocrime.org is maintained as a proxy server for content from http://everyblock.com. I mean that the Chicago Crime host responds to your request with a 301 Moved Permanently response:

sean@lenny:~$ curl -v http://chicagocrime.org/
* About to connect() to chicagocrime.org port 80
*   Trying 64.207.128.91... connected
* Connected to chicagocrime.org (64.207.128.91) port 80
> GET / HTTP/1.1
> Host: chicagocrime.org
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: nginx/0.5.35
< Date: Tue, 05 Feb 2008 16:57:32 GMT
< Content-Type: text/html
< Content-Length: 185
< Connection: keep-alive
< Location: http://chicago.everyblock.com/crime/
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/0.5.35</center>
</body>
</html>
* Connection #0 to host chicagocrime.org left intact
* Closing connection #0

That response tells user agents that the resource formerly found at http://chicagocrime.org/ has been moved to http://chicago.everyblock.com/crime/, and that user agents may consider it to be the same resource. Its representation ("format" to you GIS folks) may have changed a little bit, and the less the better for non-human users, but it should be substantially the same. Somewhat smart user agents, like your web browser, will follow up on this response with a request to the new resource location, making the redirect more or less invisible to you. Smarter user agents might update their bookmarks accordingly.

Look a little more deeply and you'll notice that not only has the root resource been permanently moved, the descendent resources have been moved as well:

sean@lenny:~$ curl -v http://www.chicagocrime.org/zipcodes/60615/
* About to connect() to www.chicagocrime.org port 80
*   Trying 64.207.128.91... connected
* Connected to www.chicagocrime.org (64.207.128.91) port 80
> GET /zipcodes/60615/ HTTP/1.1
> Host: www.chicagocrime.org
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: nginx/0.5.35
< Date: Tue, 05 Feb 2008 17:11:30 GMT
< Content-Type: text/html
< Content-Length: 185
< Connection: keep-alive
< Location: http://chicago.everyblock.com/locations/zipcodes/60615/
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/0.5.35</center>
</body>
</html>
* Connection #0 to host www.chicagocrime.org left intact
* Closing connection #0

Browse to http://www.chicagocrime.org/zipcodes/60615/ to see this in action. Starting with a sane URL scheme as Chicago Crime did certainly makes the transition a lot easier.

I can't say for sure that every single resource from chicagocrime.org has been moved in this way, but it wouldn't surprise me if they have, and if it took no more than 20 lines of configuration on the old server. By the way, HTTP specifies a status code that can be used to tell user agents that a resource really is no more: 410 Gone. I wouldn't be surprised if that was employed somewhere on what's left of chicagocrime.org.

Chicago Crime resources are moved, the Web as an application goes on without missing a beat. At least the pieces of it that were only loosely coupled to http://chicagocrime.org; any script that was downloading resources from the old site without following redirection is well and surely (and perhaps rightly) broken. Geographic sites and services can migrate or evolve using the same mechanism, a mechanism that is built into the Web. Does your GIS "web" service client understand redirects, or is it crazy-glued to those service endpoints?

All of the above is elementary web knowledge, but the Chicago Crime move is a nice lesson in applying resource-oriented thinking to a "GeoWeb" site.

Comments

Re: Long Live Chicago Crime

Author: Paul Smith

Adrian accomplished the move by setting up a virtual host for chicagocrime.org on the front-end HTTP server at EveryBlock (nginx, as seen in the HTTP headers in your curl examples). The virtual host stanza is comprised almost entirely of lines like the following:
rewrite ^/zipcodes/(\d+)/?$ http://chicago.everyblock.com/locations/zipcodes/$1/ permanent;
It gets a little verbose for the date-based URLs, mainly because of the slight impedance mismatch between chicagocrime.org date-based URLs and their new locations at chicago.everyblock.com. There's a "catch-all" directive for any resource that doesn't match the explicit rewrites. As you suspected, it was a simple matter of configuration, and also of not forgetting that your HTTP server is part of your application, not just a pass-through.

Re: Long Live Chicago Crime

Author: Sean

Thanks for comment, Paul. Everything I wrote is old hat to web folks, I know, but I thought it was a great opportunity to explain to GIS folks how things can evolve on the web.

Taking my own Advice

I had broken links on a previous blog post, but 2 mod_rewrite rules have moved resources and put all the services in working order for HTTP-savvy clients such as Google Maps. Mush, remember, is my collection of GeoRSS feed geo-processing resources. A proof of concepts: pull-style processing and Shapely.

Is it too much to expect that the winner of this contest should be cooler than Mush?

The Planet of Digital Antiquities

Update (2008-02-04)

See the new ancient world blog feed also at http://planet.atlantides.org/.

My boss, Tom Elliot, is aggregating blogs and news from the digital antiquities community at http://planet.atlantides.org/electra/. Have a peek into the world of ancient texts, historical sims, and virtual archaeology.

Comments

Re: The Planet of Digital Antiquities

Author: Tom elliott

Sean, thanks for the plug. Early visitors will have gotten 404 when they tried to subscribe to the aggregator feed. That's fixed now.