Map Servers

Map servers -- the CGI programs and servlets that render infinitely customizable map images of any scale, size, and format -- remain useful, even increasingly useful, in this tiled world. The MapServer project, for example, is climbing steadily up out of its own "trough of disillusionment" [see hype cycle] towards a 5.0 release next month. Two use cases are driving it forward. Some data are so dynamic that caching is unfeasible or even undesirable. Aircraft traffic is an obvious case. The other growing role for MapServer is that of a tile engine for applications like TileCache. Its already-good cartographic options are getting dramatically better, and longer rendering times become acceptable if your data changes on a time scale of several days or more.

Plone 3.0

Plone 3.0 is released today. Built-in versioning, slick ajax interface, and more. Pleiades will be upgrading from 2.5 in October after I get back from FOSS4G and an NEH meeting. I've already been testing our code against the Plone trunk since the start of this year and things are working pretty well. Remember, Kai has already brought PrimaGIS up to speed with the new version Plone.

GIS Certification

I'm skeptical about gatekeepers and skeptical about the value of GIS certification. It seems so medieval. Hopefully somebody will watch and make sure they don't violate their own code of ethics by lobbying city, county, and state governments to ordain preference for, or requirement of, certification.

I've paid $40 to the Red Cross for education and certification in child CPR, but for that money I received quality training from an experienced nurse. I exchanged money for skills, plus a piece of paper attesting to those skills.

Comments

Re: GIS Certification

Author: Brian Timoney

Yeah, well, with KML being our Wycliffe bible (really, ponder how much a GISer's power came from knowing a shapefile was actually more than one file) and legions of .NET developers poised to perform geoprocessing solely through SQL commands with SQL Server 2008, this imprimatur would seem fated to a sort of mustiness sooner rather than later. That said, I'm always impressed by contemporaries skilled in the art of latin declensions... Brian

Re: GIS Certification

Author: Sean

I love the Wycliffe reference. Speaking of Latin: ever seen Damian Conway's (FOSS4G keynote presenter) Perligata?.

Re: GIS Certification

Author: Anonymous

IMHO, GIS Professional Certification is simply another step in the progression of the industry. Even though I'm certified, I'm not holding my breath that any state or government will mandate new employees hold a GISP title. Endorsements don't really hold any weight per se, and that's all that NaCO has done here. Personally, the certification matters to me in terms of recognition that I've completed the requisite education, experience, and contributions to the profession. I've been working with GIS for over thirteen years, so I have no qualms about filling out the forms (or taking an exam if warranted) to add GISP after my name.

Re: GIS Certification

Author: Matt Perry

Well the Guilds were meant to bring young apprentices up to speed (in secret) wheras certification basically says "go forth and learn and come to us when you're done". Certification just means you've demonstrated your skills to a satisfactory level to some third party. I really don't know how much weight that holds .. I guess it depends on who you're trying to convince.

Re: GIS Certification

Author: Anonymous

If you're attempting to seek employment in a geospatial industry, I'd say that "demonstrating your skills to a satisfactory level" is paramount. Potential employers should examine the criteria for becoming a GISP before making any decisions based on an applicant's certification or lack thereof. It's their prerogative to classify how much "weight" the certification holds for them.

Re: GIS Certification

Author: Matt Perry

Anonymous, The best way to demonstrate competency is with a complete portfolio of your work and a conversational knowledge of the subject. That's not to say all degrees/certifications are useless (if they are then I've wasted many of my best years!). It's just that passing tests is a lot different than being a competent professional. Potential employers know this and, like you said, will weight importance of certification appropriately.

Re: GIS Certification

Author: Guillaume Sueur

Wouldn't there be an interest in a quality of service certification for online cartographic ressources ? I mean when you really use WMS or WFS data for your work , you may need to have garanty of : quality of data, quality of WMS implementation and garanty of hardware quality (will the server still exist when I'll need these data to finish my report, does it have enough bandwith etc). Most of geographic people are interested in web mapping services, but are a little afraid by making their activity depending on some remote data they can't manage directly.

Re: GIS Certification

Author: Sean

Guillaume, I see even less value in web service certification. Eventually, once the "GeoWeb" rejoins the Web, we'll see properties emerge that indicate the authority and value of services -- like a geographic PageRank. This is the first of my posts to ever get an Anonymous comment. I'm not sure if that's a meta-comment on certification or not.

Tufte and Cartography

Good stuff. My favorite Tufte tip is outlining regions with a slightly darker hue of the fill color. Because our visual system is non-linear, a difference as small as 5-6% can sharpen up your map dramatically.

Why WPS?

There are no less than five presentations on web processing services (OGC WPS) at the upcoming FOSS4G conference. I'm looking forward to asking these people what exactly WPS does that can't be done with straight HTTP POST and the 202 (Accepted) status code. What exactly does one gain here by abstracting away the Web?

Related: Sebastian Good has more about "a sink" here.

Tiles

Here's something I've read a few times, including in my comments:

I'm still trying to get my head around the idea of specifying arbitrary BBOXes for image requests in REST format: seems utterly uncacheable, and thus against one of the REST goals.

If you seek to build a high performance, highly scalable map image or coverage service (such as the one Google needs to support Earth and Maps), exposing an infinite number of view resources is a design anti-pattern that you must avoid. (Note: a WMS doesn't expose a truly infinite number of resources, but a number N approximately equal to 10 to the power of 4W, the precision width in digits of the bounding box coordinates. N approaches infinity fairly quickly: MapServer exposes something like 1066 unique views of a world map.)

If you want to cache effectively, you must switch your design over to a finite number of view resources. Tiles, in another word. There are other advantages to the REST architecture besides cacheability, so don't get the notion that REST fails for large values of resource number N.

Comments

Re: Tiles

Author: Andrew Larcombe

"If you want to cache effectively, you must switch your design over to a finite number of view resources. Tiles, in another word." Could you clarify what you mean here. Do you mean that a coverage service should not implement a response to something like http://my.mapserver.com/coverage/bbox/-90.2,20.142,-84.2,28.142 but rather force the user to request a specific tile even though part of it may be outside of their required target area, eg http://my.mapserver.com/coverage/tile/148

Re: Tiles

Author: Sean

The latter. Force? I disagree with the implication. Rather, it's an offer to clients: faster, more reliable performance in exchange for accepting the responsibility for mosaicking tiles. In Chapter 5 of "RESTful Web Services", you'll see an example of infinitely many maps, but I think the authors were trying to make a point about hypermedia and connectedness at the expense of cacheability.

Re: Tiles

Author: Frank Steggink

Andrew, The "old fashioned" bounding box gives you too fine-grained control over the extent of the image / coverage. Suppose if you move your coverage 0.1 unit, then the request is already different, and the previous request in the cache cannot be reused. But, nobody is prescribing you that the step should always be 0.1. It can also be 0.01, 0.001, 0.0001, etc., so this gives the nearly endless amount of possibilities of requests, which is not cacheable. When using tiles, your control as the client, is forced to be much coarser grained. The steps are much larger (but they depend on the level of detail), and since the size of the tile is fixed, there is no need for the upper bound coordinate, further reducing the number of possibilities. However, you will be bound to a number of fixed zoom scales, but for many cases this is not a problem. Take for example the Google Maps tiles. Suppose there are 21 levels (0 to 20). This means that at level 0 there is only one tile, at level 1 there are 4 (2x2), at level 2 there are 16 (4x4), up to level 20, where there are 2^40 (about 1.1 * 10^12) tiles. The total amount of tiles is about 1.46 * 10^12 tiles. Although this is a huge amount (only reached when the entire world is available at the maximum resolution), it is still many magnitudes smaller than the amount of unique views with MapServer. (Yes, the projections are different, etc., but that doesn't narrow the chasm.) The real number of available tiles in Google Maps is much smaller (one to ten billion?), so this amount, and especially the tiles used by a typical user or for a typical application, will be cacheable. A coverage server can still include a request like your example, but a RESTful coverage server can't. It doesn't matter much if a substantial amount of the image data will be outside the target area, because the server has to do much less processing, if it has a good caching algorithm (if the server itself doesn't use a tile cache). This will usally more than compensate the time it takes to stream the extra content to the client. And for "popular" tiles, the actual request might even never hit the server. Of course the client application should only request the tiles it actually needs. It is unnecessary to download tiles from Madagascar, if you're only interested in some data from Alaska. Please have a look at this article of Charlie Savage in which he clearly explains drawbacks of WMS; one of which is the dynamic bounding box.

Re: Tiles

Author: Andrew Larcombe

Sean, Frank, Thanks for the replies. Having written and implemented both tiled map coverage servers and WMS servers I've got a handle on the pros and cons of each one. WMS is great in theory, but only scales well under certain conditions. Conditions when it tends towards acting as a tiled map server. And don't even get me started on SLDs! :) OTOH, the argument to move some of the burden from the server (big, powerful, lots of bandwith) to the client (small, puny, dial up connection) just to be militantly RESTful seems to go against the grain somewhat. I wonder if a server offering only tiles isn't really geographic in any unique sense? The actual resources could be anything from a map, to a mosaiced gigapixel image, but unless you can specify the crs and a bbox what is really geographic about it? Cheers, Andrew

Re: Tiles

Author: Sean

"Militantly"? Nobody is forcing you to do anything. I'm only pointing out the trade-offs you have to make to cache. If your clients aren't going to overwhelm your server (and if they're on dialup, how could they?), you can keep your untiled resources.

Re: Tiles

Author: Andrew Larcombe

Hi Sean, I totally agree that if I were allowed to implement BBOX requests, chances are that cacheing would likely be largely uneffective, possibly even detrimental to performance. The 'militant' tag referred to Frank's statement that "A coverage server can still include a request like your example, but a RESTful coverage server can't." ie I don't have a choice, in order to be RESTful I must not implement BBOXs. Cheers, A

Re: Tiles

Author: Frank Steggink

Hi Andrew, As Sean mentioned, nobody is forcing anything. If you want to implement a geographic image service, you're free to choose if you adhere to the REST style or not. My remark wasn't meant to be militant, it's a simple observation, nothing more. As Sean explained in the article, BBOX requests can not be cached well (since they're most likely unique every time), so they are not RESTful. Keep in mind that REST is not a standard, but an architectural style. It takes a couple of things into consideration, of which one is the cacheability of reponses to specific requests. How much you would like to build a RESTful server, is entirely up to you. And about "being geographic": a tile is not less geographic than a BBOX request. The only difference is that you don't request it with geographic coordinates directly (but it is possible to do that, if you wish), but with an index which is mapped to geographical coordinates under water. You can still map all pixels to geographical coordinates, if you know how the tile indexes are mapped. To me, that is still "being geographic" (but your definition is likely different). This is despite the fact that the client isn't able to specify the BBOX precisely (instead the client should pick the right tiles), or to specify another CRS. (Although, in the case of multiple CRS's, it is also possible to make tile sets in different CRS's.)

Re: Tiles

Author: Charlie

Thanks for the plug Frank. WMS is fairly RESTful. The two bits of ugliness are the GetRequest/GetCapabilities weirdness (should be totally eliminated) and the format parameter (sadly necessary due to browser limitations). The problem with WMS is that its not scaleable. Tiling is more scaleable because it lets you pre-render maps, taking them out of the main code path when fulfilling requests. As far as cacheable, they are certainly cacheable on your hard-drive. Are they cacheable using the Web infrastructure? In theory, but think for a second about how many tiles something like Google maps uses. According to my calculations, at zoom level 20 its 1,099,511,627,776 tiles. And Google has since added two more zoom levels I believe. Is the web infrastructure really going to help you out here? I have absolutely no idea. Charlie

Re: Tiles

Author: Sean

According to the math on the back of this envelope here, the GMaps tiles are up to 10**54 times more cacheable than the output of a WMS ;)

Re: Tiles

Author: Charlie

Which doesn't prove anything at all, does it? If its too many tiles to cache in either case, you're just comparing uncacheable versus super uncacheable. Anyway, I'd be surprised if the speed gain here has to do with web caching (minus Google distributing the tiles around the world using their own datacenters). I think instead its all about the pre-rendering. But it would be interesting to see the performance numbers to see for sure.

Re: Tiles

Author: Allan

Charlie - there's not really a difference between "pre-rendering" and "web caching", is there? Note that it's not the BBOX itself that makes things uncacheable. It's the BBOX without constraints. This is shown by the success of Chris Schmidt's tilecache software that works by limiting the BBOX to specific values (effectively creating tiles). So it comes down to whether you want to be able to use geographic coordinates (BBOX but with constraints) or names (tiles in the more traditional sense. Both can be RESTful.

Re: Tiles

Author: Charlie

Allan, Sure there is. Let's say you rely only on web caching. And let's say your users tend to zoom into wherever they live - and your users live all around the world. You'll end up generating millions of tiles on the fly (the first time the tile is requested you have to render it of course) and that will kill your performance for all the reasons I described in my blog post. Pre-rendering instead creates all the tiles ahead of time so you can serve them directly from your hard-disk. That is a gigantic performance win - whether or not the tiles later get cached by proxies on the web. Last, set BBOXes are just tiles like you say so I see no difference (a set BBOX is a name more or less). But remember that WMS also lets you vary the visible layers and styles on a map. If you want to use pre-rendered tiles, you have constrain the problem by dissallowing that (or perhaps just a few styles). Anyway, I agree this isn't a REST vs. non-REST debate. WMS is RESTful. Its just no performant. Charlie

Explaining REST

Charlie is on a roll. The difficulty with promoting REST is that we can't yet say "Products X, Y, and Z support it", or "Companies A, B, and C are standardizing on it, and so should you". The case has to be made for technical superiority (in many situations) of REST over the entrenched OGC architecture, and that means some jargon and details that seem esoteric to a lot of GIS practitioners. Charlie's latest are good posts that don't require continual reference to the HTTP/1.1 or WxS specs.

That's not Agile Geography

Looks like the standards architects are not going for the KML + Atompub idea after all.

Comments

Re: That's not Agile Geography

Author: Chris Holmes

On the contrary, if Atompub makes it in I don't think that it'd need to be relegated to the 'services' module. I think that's the beauty of using it, you'd just need an atom link that lets you edit, as you pointed out. And Andrew's blog posts are just now caught up to where we got at the kick off meeting, which was before your suggestion of atompub+kml. Doing KML editing is outside the scope of work of the testbed, but a number of us are excited to think about it and to experiment. It should also be noted that the notion of services in Andrew's post are not limited to the ones listed there, and indeed we're actively looking for more use cases. The initial use cases were focused around replacing Context documents, which have no notion of editing at all.

Camping With Google Earth

Last week I used Google Earth to scout sites for my 21 month-old daughter's first ever camping trip. The imagery of the Never Summer Range is excellent, and I quickly found a couple good candidates. We used one of them Saturday night: it's marked along with a hiking trailhead and the spot where our little mountaineer announced she was too tired and hungry to continue in camping-debut.kml.

One thing the imagery (2004? 2005?) doesn't show is how bad the mountain pine beetle epidemic is in that part of the Routt National Forest. The slope facing our campsite was up to 10% dead. We saw large stands that might have been 20% dead.