Sean Gillies (Posts about rest)https://sgillies.net/tags/rest.atom2023-12-31T01:26:24ZSean GilliesNikolaRasterio, GDAL, GeoTIFF, and REST on the Mapbox bloghttps://sgillies.net/2017/12/18/rasterio-and-rest-on-the-mapbox-blog.html2017-12-18T22:44:33-07:002017-12-18T22:44:33-07:00Sean Gillies<p>Last Friday, I finished a post for the Mapbox blog:
<a class="reference external" href="https://blog.mapbox.com/build-for-the-cloud-with-rasterio-3254d5d60289">https://blog.mapbox.com/build-for-the-cloud-with-rasterio-3254d5d60289</a>. It was
a pleasure to write about a new industry best practice, give some props, and
toot my favorite open source project's horn a bit. I'm pleased that it's been
well received.</p>
<figure>
<img alt="https://c1.staticflickr.com/1/149/421186578_fbb062368e_o.jpg" src="https://c1.staticflickr.com/1/149/421186578_fbb062368e_o.jpg">
<figcaption>
<p>Roy T. Fielding Has a Posse, by Paul Downey.
<a class="reference external" href="https://www.flickr.com/photos/psd/421186578/">https://www.flickr.com/photos/psd/421186578/</a></p>
</figcaption>
</figure>
<p>I had fun coming up with the thesis that the cloud-optimized GeoTIFF format
developed by Even Rouault (at
<a class="reference external" href="https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF">https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF</a>) and evangelized on
a new web site by Chris Holmes (<a class="reference external" href="http://www.cogeo.org/">http://www.cogeo.org/</a>) is not just an image
format, but an <a class="reference external" href="https://tools.ietf.org/html/rfc6838#section-4.2.5">internet application media type</a> and that GDAL's
curl-based virtual file system is practically a web browser. I especially
enjoyed pointing out that I think that systems using cloud-optimized GeoTIFFs
can be much more in the classic REST style than most of the "REST APIs" we use
today. In most geospatial REST APIs, the data format is of minor importance,
relegated to a <code class="docutils literal">f=format</code> parameter in a query string. In the architecture
emerging around GDAL and cloud-optimized GeoTIFFs, the GeoTIFF representation
of a raster plays a major role in directing the GDAL browser. It's not just one
of many roughly equivalent flavors of imagery.</p>
<p>On Twitter today, I floated the idea of registering a media type for the
cloud-optimized GeoTIFF format that would distinguish it from ordinary TIFFs.
It could be interesting to collaborate on this with other folks in the
business.</p>
<p>Finally, I don't know when Charlie Loyd discovered an old man yelling at the
cloud in Landsat imagery near Darwin, Australia, or why he saved it until my
blog post, but I'm grateful he did.</p>
<figure>
<img alt="https://c1.staticflickr.com/5/4599/39151424991_7463ee8514_b.jpg" src="https://c1.staticflickr.com/5/4599/39151424991_7463ee8514_b.jpg">
</figure>Simple in theoryhttps://sgillies.net/2011/11/18/simple-in-theory.html2011-11-18T00:00:00-07:002011-11-18T00:00:00-07:00Sean Gillies<p>Rich Hickey's "Simple Made Easy" <a class="reference external" href="http://www.infoq.com/presentations/Simple-Made-Easy">presentation</a> at Strange Loop, recommended to
me by my Clojure programming co-worker <a class="reference external" href="http://philomousos.blogspot.com/">Hugh Cayless</a>, is flat out awesome.
"Guardrail Programming" and "Knitted Castle" are my new favorite metaphors. Hickey has a compelling theory about complexity
and after watching the presentation, I feel like I can be a better advocate for
simplicity. Advocate to those who like theory, at least. For others, the proof
remains in the pudding, whether simple means better software.</p>
<p>REST, the architectural style, didn't factor into Hickey's talk at all, but is a
great example of an approach that chooses simplicity over ease. REST is hard.
It is. You're wrong if you've been thinking that REST is easier than SOAP or
COM. Look at almost any (there are exceptions, yes) so called "REST API" and
you'll see something produced by web programmers that tried to apply the REST
style and either couldn't get their heads around it or gave up on it under
pressure to deliver. REST is hard to understand and it can be difficult to
explain its benefits to managers and customers that prioritize ease over
simplicity. REST is hard, but REST is simple. It is predictable and you can
reason about what you can or cannot do with it.</p>
<p>There's a notion in the humanities that DH (digital humanities) is
<a class="reference external" href="http://digitalhumanitiesnow.org/2011/11/digital-humanities-and-theory-round-up-part-2/">undertheorized</a>. I'm not a humanist, really, just a programmer, but
I strongly disagree. Programmers in the humanities are doing a great amount of
theoretical work. As well as reading Hugh's <a class="reference external" href="http://philomousos.blogspot.com/2011/11/tei-in-other-formats-part-first-html.html">recent</a> <a class="reference external" href="http://philomousos.blogspot.com/2011/11/tei-in-other-formats-part-second-theory.html">posts</a>, digital humanities
theorists owe themselves a look at Hickey's theory of complexity and Roy
Fielding's theory of representational state transfer. The world of programming
and the field of humanities programming and computer are more theorized than
they appears to non-programmers.</p>REST in six lines?https://sgillies.net/2011/05/09/rest-in-six-lines.html2011-05-09T00:00:00-06:002011-05-09T00:00:00-06:00Sean Gillies<p>Duncan Cragg makes REST <a class="reference external" href="http://duncan-cragg.org/blog/post/mature-rest-easy/">easy</a>:</p>
<blockquote>
<p>Here's how easy it is to be RESTful - "Level 3" - in just six lines:</p>
<pre class="literal-block">GET /resource/1 HTTP/1.1
Host: restful.com
HTTP/1.1 200 OK
Cache-Control: max-age=100
Content-Type: application/json
{ .., "next": "/resource/2", .. }</pre>
<p>Pretty simple, huh? If you're surprised by my use of JSON, hold on a minute, I'll
get to that.</p>
</blockquote>
<p>I still think that get our heads around REST is the most difficult (and largely
unaccomplished) part. The other obstacle is our frameworks: ones with REST baked in
are rare and unpolished, mature ones often make REST difficult.</p>Grading RESThttps://sgillies.net/2010/10/22/grading-rest.html2010-10-22T00:00:00-06:002010-10-22T00:00:00-06:00Sean Gillies<p>Earlier this week, Raj Singh graded the OGC's services against REST and then
graded the REST architectural style <a class="reference external" href="http://www.rajsingh.org/blog/2010/10/19/ogc-services-and-rest/">itself</a>:</p>
<blockquote>
<p>The fundamental problem with REST and GIS in my opinion, is that REST
optimizes for data <em>access</em>, and OGC services are optimized for data
<em>processing</em>. I consider GIS a type of OLAP system, and we as an industry
will continue to resist REST because it would be counter-productive to expose
data via atomic resources that all had their own URL (and metadata!?!)
because everything we really want to do with the data would get harder.</p>
</blockquote>
<p>In comments, Stu Charlton points out that REST is in fact optimized for data
processing. Myself, I think it's simply in a different style, one that looks more goal
oriented or agent oriented; I've <a class="reference external" href="http://sgillies.net/blog/1010/restful-hypermedia-agents">linked</a> to a past post by Charlton on
<a class="reference external" href="http://www.stucharlton.com/blog/archives/2010/03/building-a-restful-hypermedia.html">hypermedia agents</a> that has many good comments of its own.</p>
<p>Is it true to that the GIS industry resists REST? Some companies and consortia
do, others don't. It seems like there's a press release about a new Geospatial
REST API every other day. We've just seen a "REST API standard" document
released by the 800 lb gorilla itself. A lot of this is just marketing, but
there is also a growing, honest interest in resources instead of endpoints and
HTTP as an application protocol.</p>
<p>I'm not sure grading OGC services benefits anybody, but if you absolutely have
to grade your services against a REST maturity model it's probably best to use
a <a class="reference external" href="http://martinfowler.com/articles/richardsonMaturityModel.html">standard model</a>.</p>No end-pointshttps://sgillies.net/2010/04/22/no-end-points.html2010-04-22T00:00:00-06:002010-04-22T00:00:00-06:00Sean Gillies<p>Subbu Allamaraju on the Facebook Graph API and <a class="reference external" href="http://www.subbu.org/blog/2010/04/uncomplicated-hypermedia-facebooks-graph-api">links</a></p>
<blockquote>
<p>It is interconnected as every representation is linked to related resources. For instance, an album representation is linked to the profile that posted the album. Each user representation is linked to a collection of groups that the user is member of. This is hypermedia in action. Hypermedia does not have to be complicated – and Facebook just proves it.</p>
</blockquote>RESTful hypermedia agentshttps://sgillies.net/2010/03/30/restful-hypermedia-agents.html2010-03-30T00:00:00-06:002010-03-30T00:00:00-06:00Sean Gillies<p>Stuart Charlton says stuff about <a class="reference external" href="http://www.stucharlton.com/blog/archives/2010/03/building-a-restful-hypermedia.html">Building a RESTful Hypermedia Agent, Part 1</a>:</p>
<blockquote>
<p>Building a hypermedia-aware client is rather different from building a
typical client in a client/server system. It may not be immediately
intuitive. But, I believe the notions are rooted in (quite literally) decades
of experience in other computing domains that are agent-oriented. Game
behaviour engines, control systems, reactive or event-driven systems all have
been developed with this programming approach in mind.</p>
</blockquote>
<p>He points to a <a class="reference external" href="http://books.google.com/books?id=2LEDbFXvGbgC&lpg=PA27&dq=russell%20norvig%20agent&pg=PA52#v=onepage&q=russell%20norvig%20agent&f=false">diagram</a> from <a class="reference external" href="http://aima.cs.berkeley.edu/">Artifical Intelligence: A Modern Approach</a> <a class="footnote-reference brackets" href="https://sgillies.net/2010/03/30/restful-hypermedia-agents.html#footnote-1" id="footnote-reference-1" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> and adapts it to the RESTful web.
Agent sensors become HTTP's "safe" methods (GET), effectors the "unsafe"
methods (POST, PUT, DELETE). The HTTP protocol and content type definitions
make up the agent's model of the evolving, mutable state of its environment
(the web). I'm enjoying thinking about RESTful client-server interactions on
the web in these well-reasoned terms. The REST style, to me, is all about
enabling software agents. Not just web browsers or search index crawlers, but
agents that might mine your texts, geocode your news, or ETL your spatial data (and wash your socks).
I'm looking forward to the next installment.</p>
<aside class="footnote-list brackets">
<aside class="footnote brackets" id="footnote-1" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="https://sgillies.net/2010/03/30/restful-hypermedia-agents.html#footnote-reference-1">1</a><span class="fn-bracket">]</span></span>
<p>S.J. Russell and P. Norvig, Artificial Intelligence, Prentice Hall, 2009.</p>
</aside>
</aside>Sensors, things, and the Webhttps://sgillies.net/2010/03/15/sensors-things-and-the-web.html2010-03-15T00:00:00-06:002010-03-15T00:00:00-06:00Sean Gillies<p>My readers are probably aware of the OGC's Sensor Web <a class="reference external" href="http://www.opengeospatial.org/projects/groups/sensorweb">initiatives</a>, but there's another, different vision of a "Web of Things" using the architecture and infrastructure of the actual web we have now (URIs, HTTP, Atom, JSON, HTML, Javascript) that's well articulated in this <a class="reference external" href="http://www.slideshare.net/misterdom/web-of-things-connecting-people-and-objects-on-the-web-3425316#">SXSW presentation</a> by Vlad Trifa and Dominique Guinard (and also in their <a class="reference external" href="http://www.webofthings.com/">blog</a>, via <a class="reference external" href="http://thisweekinrest.wordpress.com/2010/03/15/this-week-in-rest-volume-7-mar-8-2010-mar-14-2010/">This week in REST</a>) and in an associated <a class="reference external" href="https://www.inf.ethz.ch/research/disstechreps/techreports/show?serial=663&lang=en">technical report</a> (<a class="reference external" href="ftp://ftp.inf.ethz.ch/pub/publications/tech-reports/6xx/663.pdf">PDF</a>).</p>
<section id="comments">
<h2>Comments</h2>
<section id="re-sensors-things-and-the-web">
<h3>Re: Sensors, things, and the Web</h3>
<p>Author: Miguel Montesinos</p>
<p>I deeply agree with the "Web of Things" vision. I don't think that's the "different vision", but one of the most likely visions to happen.</p><p></p><p>I'm involved in several similar European R&D projects and initiatives, and none of them cares about OGC's SWE, apart from what we are proposing.</p><p></p><p>SWE is quite powerful, but really complicated for becoming a backbone of the Internet of Things.</p><p></p><p>Miguel</p></section>
</section>In which we go into the weeds for some RESThttps://sgillies.net/2010/01/29/in-which-we-go-into-the-weeds-for-some-rest.html2010-01-29T00:00:00-07:002010-01-29T00:00:00-07:00Sean Gillies<p>On the descending portion of the hype cycle now it seems that, like a guy
in a "Rock Star" t-shirt, a "REST API" most likely isn't. It might be using
HTTP as a uniform interface and identifying things with URIs, but then you find
it provides text/xml or application/json responses with no links and
out-of-band rules for teleporting (you can't call it traversing) to other
parts of the API. Tight coupling like that is <a class="reference external" href="http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven">not</a> what REST is about.</p>
<p>One that's getting very close is GeoServer's <a class="reference external" href="http://docs.geoserver.org/2.0.x/en/user/extensions/rest/rest-config-api.html">Configuration API</a>. It has links
from workspaces to datastores to layers, and a non-HTML client should in theory
be able to follow them, changing the configuration state of the service in a
step-by-step manner, led by the service itself, much in the same way you would
through a web browser. All from one bookmarkable URI. This is what REST is
about.</p>
<p>I say "in theory" because the GeoServer API doesn't hold water for formats
other than HTML. Here's the problem: given a bookmarked URI ending in
"workspaces" like <a class="reference external" href="http://example.com/workspaces">http://example.com/workspaces</a>, how does a client determine
that this URI identifies a resource to which you can POST a new workspace and
begin the configuration process using in-band information <em>only</em>? If you're
working with a text/html representation of the resource, you'll be shown a
form, and away you go, RESTfully. The semantics of forms, and specifically that
submitting one sends data to certain URI, are defined in the text/html media
type standard. A client doesn't need any out-of-band information: the form is
in the representation, the semantics are specified by the standard "text/html"
value of Content-Type header, both in-band. Now, if the server sends you back a
text/xml response, there's no way for a client to know only from in-band
information how it is to act on the response. That it's a certain type of
resource (a GeoServer Workspace) because the URI ends in "workspaces" and the
representation has a root <workspaces> element? That's out of band. That the
bookmarked URI is a "GeoServer workspace bookmark"? That's out of band too.</p>
<p>AtomPub, on the other hand, holds water because the POST-ability of service
resources (for creating new collections) is standardized under the media type
"application/atomsvc+xml". If a client GETs a URI and that format comes back,
the POST-ability is communicated, in-band. The "application/atom+xml" media
type does the same for collections and entries, especially in its specification that an "edit" link tells the client via which resource it modifies entry and collection state. Standardizing on Atom and AtomPub, if you
can, is therefore a good bet.</p>
<p>The interesting thing about REST that distinguishes itself from other styles is
that interaction is driven by in-band information. Loose coupling,
evolvability, and longevity are properties of a system that has the hypertext
constraint. To get these properties, GeoServer and other APIs need to eliminate
the out-of-band communication. Standardize on media types like HTML or Atom,
mint their own media types (application/vnd.geoserver+xml or some such), or use
links with standard relations in HTTP headers (aka <a class="reference external" href="http://sgillies.net/blog/985/proposed-standard-for-web-linking">Web Linking</a>) and push for
client support of those.</p>
<section id="comments">
<h2>Comments</h2>
<section id="re-in-which-we-go-into-the-weeds-for-some-rest">
<h3>Re: In which we go into the weeds for some REST</h3>
<p>Author: <a class="reference external" href="http://opengeo.org">Chris Holmes</a></p>
<p>Thanks for the review Sean, our goal is to make GeoServer as RESTful as possible (indeed when we have the time we'd like to do REST feature access alternative to WFS).</p><p></p><p>Practically I'm still not sure of the best way forward. Atom does seem better than text/xml, but even if we did that wouldn't we still want to have text/xml representations of resources? Or are you advocating replacing all the text/xml responses with Atom/AtomPub?</p><p></p><p>As for application/vnd.geoserver+xml - isn't that out of band in its own way? Like developing a client against it you'd still need to know something about that format? Or you're saying it'd just be a better self-documenting one? I'd be interested in your ideas of what exactly that looks like. And again, would it replace text/xml responses?</p><p></p><p>As for Web Linking, it looks great but is it even accepted yet? Not that we're opposed to implementing a developing standard and encouraging its adoption, but I think things like feature access through REST are higher priority for us. And the idea with that is we'd just add http headers to our text/xml responses? If you want to help us you could sketch out exactly what headers we should add - I think it's pretty easy to add in extra http headers, so if it's not much of an effort we might be able to do it soon.</p></section>
<section id="re-in-which-we-go-into-the-weeds-for-some-rest-1">
<h3>Re: In which we go into the weeds for some REST</h3>
<p>Author: Allan Doyle</p>
<p>Chris asks "As for application/vnd.geoserver+xml - isn't that out of band in its own way? " -- That was my question, too. I was going to ask about "application/atomsvc+xml" instead.</p><p></p><p>Sean said</p><p></p><blockquote>AtomPub, on the other hand, holds water because the POST-ability of service resources (for creating new collections) is standardized under the media type "application/atomsvc+xml". If a client GETs a URI and that format comes back, the POST-ability is communicated, in-band.</blockquote><p></p><p></p><p>Isn't that only because RFC 5023 says it's POST-able? Then RFC 5023 is the out-of-band knowledge.</p></section>
<section id="re-in-which-we-go-into-the-weeds-for-some-rest-2">
<h3>Re: In which we go into the weeds for some REST</h3>
<p>Author: Sean</p>
<p>RFC 5023 isn't out-of-band: it and application/atomsvc+xml and application/atom+xml are part of the fabric of the web.</p><p></p><p></p><p>For a different take on the subject you should check out <a href="http://www.subbu.org/blog/2009/12/media-types-and-plumbing"></a><a href="http://www.subbu.org/blog/2009/12/media-types-and-plumbing">http://www.subbu.org/blog/2009/12/media-types-and-plumbing</a>.</p></section>
<section id="governance-of-out-of-band-semantics">
<h3>Governance of out of band semantics</h3>
<p>Author: Rob Atkinson</p>
<p>There is a significant implication in using a special MIME type that is "part of the fabric" of the web to indicate how a client is supposed to interpret content, and the actions it may take as a result. This basically implies that conformant RESTful semantics are only possible within the governance framework of the web "fabric" - its not open to application domains to define semantics or behaviour of APIs.</p><p></p><p>Perhaps application API semantics have to be considered as an out of band (from REST point of view) on top of REST. I.e. REST semantics is an out-of-band part of any application API. This perhaps makes sense, because strong governance (fabric of the web) is useful in an out-of-band context, whereas private application semantics are much more problematic as out-of-band information, because they are hard to discover, formalise consistently, create and interpret.</p></section>
<section id="re-in-which-we-go-into-the-weeds-for-some-rest-3">
<h3>Re: In which we go into the weeds for some REST</h3>
<p>Author: Sean</p>
<p>I really must rethink what I've said about the GeoServer API not holding water after another read of Subbu's post. I've agreed with those who say "application/xml is not the media type you're looking for" (Mark Baker, Jim Webber) if you want to use web links, not XLinks. That link semantics aren't conveyed by the Atom namespace, but by the media type. While that's still my take, I should give Subbu's a try. If you stick to the minimum HTTP protocol (Atom invents a somewhat specialized one) and straight XML processing (Atom has distinctly different processing rules, like "must ignore"), application/xml might be fine. At any rate, I think you'd at least need an actual namespace for workspace and data store elements to make this hold water using application/xml.</p><p>No mistake: a configuration API is a great place to start getting into the REST style, and GeoServer is getting close to nailing it.</p></section>
<section id="re-in-which-we-go-into-the-weeds-for-some-rest-4">
<h3>Re: In which we go into the weeds for some REST</h3>
<p>Author: Sean</p>
<p>There's another issue in the GeoServer docs, which document a practice of interaction driven through a fixed URI hierarchy (see Fielding's 4 bullet at <a href="http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven">http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven</a>). There's a protocol implied there that application/xml doesn't begin to hint at. Better: drive interaction through the links that are already present in GeoServer's workspace (and friends) representations. GeoServer is ready for REST in a lot of ways, but documents a contrary usage that will result in unnecessary coupling.</p></section>
</section>Linking UK datahttps://sgillies.net/2010/01/28/linking-uk-data.html2010-01-28T00:00:00-07:002010-01-28T00:00:00-07:00Sean Gillies<p>I haven't seen any links from geospatial or GIS blogs to Jeni Tennison's excellent piece about the motivation for choosing the web's architecture as the architecture for the UK's open data initiative and for choosing linked data instead of the usual "services" that make up US data initiatives.</p>
<blockquote>
<p>Why?</p>
<p>Because linked data is just a term for how to publish data on the web while working with the web. And the web is the best architecture we know for publishing information in a hugely diverse and distributed environment, in a gradual and sustainable way.</p>
</blockquote>
<p><a class="reference external" href="http://www.jenitennison.com/blog/node/140">Read it</a> and check out the links to tutorials about creating linked data.</p>
<section id="comments">
<h2>Comments</h2>
<section id="re-linking-uk-data">
<h3>Re: Linking UK data</h3>
<p>Author: Ian Turton</p>
<p>I think that is because all the UK data is just text and excel tables. The OS will give up their data when it's pried from their cold dead fingers, and don't even think about geocoding via a postcode Royal Mail are even worse!</p><p></p><p>Ian</p></section>
<section id="re-linking-uk-data-1">
<h3>Re: Linking UK data</h3>
<p>Author: Sean</p>
<p>GIS data, rasters excepted, is also largely tabular, wouldn't you say? What's a shapefile if not a table? GML allows different structures, but is less commonly used in that way, and the RDF model is equally suited for those special complex features cases.</p><p></p><p></p><p>Speaking of the OS, it may not giving away coordinates yet, but has interesting and possibly useful linked data at <a href="http://data.ordnancesurvey.co.uk/"></a><a href="http://data.ordnancesurvey.co.uk/">http://data.ordnancesurvey.co.uk/</a>.</p></section>
</section>WS-REST 2010https://sgillies.net/2010/01/05/ws-rest-2010.html2010-01-05T00:00:00-07:002010-01-05T00:00:00-07:00Sean Gillies<p>To be held at WWW 2010 in Raleigh, North Carolina next 26 April 2010 [<a class="reference external" href="http://ws-rest.org/">site</a>]:</p>
<blockquote>
<p>This first edition of WS-REST, co-located with the WWW2010 conference, aims at providing an academic forum for discussing current emerging research topics centered around the application of REST, as well as advanced application scenarios for building large scale distributed systems.</p>
</blockquote>
<p>It would be neat to see some papers on geospatial applications of the REST architectural style on the program.</p>