Tiles

Here's something I've read a few times, including in my comments:

I'm still trying to get my head around the idea of specifying arbitrary BBOXes for image requests in REST format: seems utterly uncacheable, and thus against one of the REST goals.

If you seek to build a high performance, highly scalable map image or coverage service (such as the one Google needs to support Earth and Maps), exposing an infinite number of view resources is a design anti-pattern that you must avoid. (Note: a WMS doesn't expose a truly infinite number of resources, but a number N approximately equal to 10 to the power of 4W, the precision width in digits of the bounding box coordinates. N approaches infinity fairly quickly: MapServer exposes something like 1066 unique views of a world map.)

If you want to cache effectively, you must switch your design over to a finite number of view resources. Tiles, in another word. There are other advantages to the REST architecture besides cacheability, so don't get the notion that REST fails for large values of resource number N.

Comments

Re: Tiles

Author: Andrew Larcombe

"If you want to cache effectively, you must switch your design over to a finite number of view resources. Tiles, in another word." Could you clarify what you mean here. Do you mean that a coverage service should not implement a response to something like http://my.mapserver.com/coverage/bbox/-90.2,20.142,-84.2,28.142 but rather force the user to request a specific tile even though part of it may be outside of their required target area, eg http://my.mapserver.com/coverage/tile/148

Re: Tiles

Author: Sean

The latter. Force? I disagree with the implication. Rather, it's an offer to clients: faster, more reliable performance in exchange for accepting the responsibility for mosaicking tiles. In Chapter 5 of "RESTful Web Services", you'll see an example of infinitely many maps, but I think the authors were trying to make a point about hypermedia and connectedness at the expense of cacheability.

Re: Tiles

Author: Frank Steggink

Andrew, The "old fashioned" bounding box gives you too fine-grained control over the extent of the image / coverage. Suppose if you move your coverage 0.1 unit, then the request is already different, and the previous request in the cache cannot be reused. But, nobody is prescribing you that the step should always be 0.1. It can also be 0.01, 0.001, 0.0001, etc., so this gives the nearly endless amount of possibilities of requests, which is not cacheable. When using tiles, your control as the client, is forced to be much coarser grained. The steps are much larger (but they depend on the level of detail), and since the size of the tile is fixed, there is no need for the upper bound coordinate, further reducing the number of possibilities. However, you will be bound to a number of fixed zoom scales, but for many cases this is not a problem. Take for example the Google Maps tiles. Suppose there are 21 levels (0 to 20). This means that at level 0 there is only one tile, at level 1 there are 4 (2x2), at level 2 there are 16 (4x4), up to level 20, where there are 2^40 (about 1.1 * 10^12) tiles. The total amount of tiles is about 1.46 * 10^12 tiles. Although this is a huge amount (only reached when the entire world is available at the maximum resolution), it is still many magnitudes smaller than the amount of unique views with MapServer. (Yes, the projections are different, etc., but that doesn't narrow the chasm.) The real number of available tiles in Google Maps is much smaller (one to ten billion?), so this amount, and especially the tiles used by a typical user or for a typical application, will be cacheable. A coverage server can still include a request like your example, but a RESTful coverage server can't. It doesn't matter much if a substantial amount of the image data will be outside the target area, because the server has to do much less processing, if it has a good caching algorithm (if the server itself doesn't use a tile cache). This will usally more than compensate the time it takes to stream the extra content to the client. And for "popular" tiles, the actual request might even never hit the server. Of course the client application should only request the tiles it actually needs. It is unnecessary to download tiles from Madagascar, if you're only interested in some data from Alaska. Please have a look at this article of Charlie Savage in which he clearly explains drawbacks of WMS; one of which is the dynamic bounding box.

Re: Tiles

Author: Andrew Larcombe

Sean, Frank, Thanks for the replies. Having written and implemented both tiled map coverage servers and WMS servers I've got a handle on the pros and cons of each one. WMS is great in theory, but only scales well under certain conditions. Conditions when it tends towards acting as a tiled map server. And don't even get me started on SLDs! :) OTOH, the argument to move some of the burden from the server (big, powerful, lots of bandwith) to the client (small, puny, dial up connection) just to be militantly RESTful seems to go against the grain somewhat. I wonder if a server offering only tiles isn't really geographic in any unique sense? The actual resources could be anything from a map, to a mosaiced gigapixel image, but unless you can specify the crs and a bbox what is really geographic about it? Cheers, Andrew

Re: Tiles

Author: Sean

"Militantly"? Nobody is forcing you to do anything. I'm only pointing out the trade-offs you have to make to cache. If your clients aren't going to overwhelm your server (and if they're on dialup, how could they?), you can keep your untiled resources.

Re: Tiles

Author: Andrew Larcombe

Hi Sean, I totally agree that if I were allowed to implement BBOX requests, chances are that cacheing would likely be largely uneffective, possibly even detrimental to performance. The 'militant' tag referred to Frank's statement that "A coverage server can still include a request like your example, but a RESTful coverage server can't." ie I don't have a choice, in order to be RESTful I must not implement BBOXs. Cheers, A

Re: Tiles

Author: Frank Steggink

Hi Andrew, As Sean mentioned, nobody is forcing anything. If you want to implement a geographic image service, you're free to choose if you adhere to the REST style or not. My remark wasn't meant to be militant, it's a simple observation, nothing more. As Sean explained in the article, BBOX requests can not be cached well (since they're most likely unique every time), so they are not RESTful. Keep in mind that REST is not a standard, but an architectural style. It takes a couple of things into consideration, of which one is the cacheability of reponses to specific requests. How much you would like to build a RESTful server, is entirely up to you. And about "being geographic": a tile is not less geographic than a BBOX request. The only difference is that you don't request it with geographic coordinates directly (but it is possible to do that, if you wish), but with an index which is mapped to geographical coordinates under water. You can still map all pixels to geographical coordinates, if you know how the tile indexes are mapped. To me, that is still "being geographic" (but your definition is likely different). This is despite the fact that the client isn't able to specify the BBOX precisely (instead the client should pick the right tiles), or to specify another CRS. (Although, in the case of multiple CRS's, it is also possible to make tile sets in different CRS's.)

Re: Tiles

Author: Charlie

Thanks for the plug Frank. WMS is fairly RESTful. The two bits of ugliness are the GetRequest/GetCapabilities weirdness (should be totally eliminated) and the format parameter (sadly necessary due to browser limitations). The problem with WMS is that its not scaleable. Tiling is more scaleable because it lets you pre-render maps, taking them out of the main code path when fulfilling requests. As far as cacheable, they are certainly cacheable on your hard-drive. Are they cacheable using the Web infrastructure? In theory, but think for a second about how many tiles something like Google maps uses. According to my calculations, at zoom level 20 its 1,099,511,627,776 tiles. And Google has since added two more zoom levels I believe. Is the web infrastructure really going to help you out here? I have absolutely no idea. Charlie

Re: Tiles

Author: Sean

According to the math on the back of this envelope here, the GMaps tiles are up to 10**54 times more cacheable than the output of a WMS ;)

Re: Tiles

Author: Charlie

Which doesn't prove anything at all, does it? If its too many tiles to cache in either case, you're just comparing uncacheable versus super uncacheable. Anyway, I'd be surprised if the speed gain here has to do with web caching (minus Google distributing the tiles around the world using their own datacenters). I think instead its all about the pre-rendering. But it would be interesting to see the performance numbers to see for sure.

Re: Tiles

Author: Allan

Charlie - there's not really a difference between "pre-rendering" and "web caching", is there? Note that it's not the BBOX itself that makes things uncacheable. It's the BBOX without constraints. This is shown by the success of Chris Schmidt's tilecache software that works by limiting the BBOX to specific values (effectively creating tiles). So it comes down to whether you want to be able to use geographic coordinates (BBOX but with constraints) or names (tiles in the more traditional sense. Both can be RESTful.

Re: Tiles

Author: Charlie

Allan, Sure there is. Let's say you rely only on web caching. And let's say your users tend to zoom into wherever they live - and your users live all around the world. You'll end up generating millions of tiles on the fly (the first time the tile is requested you have to render it of course) and that will kill your performance for all the reasons I described in my blog post. Pre-rendering instead creates all the tiles ahead of time so you can serve them directly from your hard-disk. That is a gigantic performance win - whether or not the tiles later get cached by proxies on the web. Last, set BBOXes are just tiles like you say so I see no difference (a set BBOX is a name more or less). But remember that WMS also lets you vary the visible layers and styles on a map. If you want to use pre-rendered tiles, you have constrain the problem by dissallowing that (or perhaps just a few styles). Anyway, I agree this isn't a REST vs. non-REST debate. WMS is RESTful. Its just no performant. Charlie