URI Opacity Revisited

Posted in Web by AST on Monday, April 28th, 2008

Amoxil Generic Buy Clarinex Online Neurontin Without Prescription Topamax No Prescription Soma For Sale Glucotrol Generic Buy Aricept Online Stromectol Without Prescription Lotrisone No Prescription Celexa For Sale

Lipstick on a pigThe question regarding “meaningful” or “human friendly” URIs comes up a lot if you’re trying to design a Website or Web-accessible information architecture (including a Web service). This subject came up again recently on the SOA mailing list (I didn’t see the beginning of the thread, but this is where I started paying attention).

There are some key points about URIs and opacity that anyone who’s really going to the trouble to design them should keep in mind (originally presented by Dan Connolly at a W3C event back in 2001):

DO

  • …make URIs mnemonic when it’s convenient

DON’T

  • …depend on finding metadata in URIs
  • …depend on search engine heuristics regarding query parameters
  • …depend on file extensions
  • …depend on URIs for user interface

Basically, don’t peek inside the URI. Notice there’s a lot more “don’ts” than “dos”. Also notice if you’re tempted to provide mnemonic names like so (based on the example from the discussion thread):

  http://www.example.com/parts/sku1234/orders/order56789

the natural extension is to start writing clients based on “constructing” URIs something like this:

  def to_s
    "#{baseuri}/parts/sku#{sku}/orders/order#{order_id}"
  end

Taken a little further and you’ve a “REST API” (from flikr’s API documentation):

Photo Source URLs

You can construct the source URL to a photo once you know its ID, server ID, farm ID and secret, as returned by many API methods.

The URL takes the following format:

http://farm{farm-id}.static.flickr.com/{server-id}/{id}_{secret}.jpg
	or
http://farm{farm-id}.static.flickr.com/{server-id}/{id}_{secret}_[mstb].jpg
	or
http://farm{farm-id}.static.flickr.com/{server-id}/{id}_{o-secret}_o.(jpg|gif|png)

Now, if you think this is a “hypermedia application”, you need some serious help. The above represents an application programming interface with an explicit contract between the client and the server about what parameters it will accept and where they can be placed. Notice as well where this explicit contract is defined: in another resource—the API specification. There’s no other source of information on how to retrieve a particular photo other than this documentation. You, as the service consumer, also need to know a whole lot about flickr’s infrastructure layout (retrieved from somewhere), or you won’t be able to retrieve the photo.

Once you know the API, making the specified requests to the specified URIs – constructed by the client – will result in a predictable response from the server. Functionally, it’s equivalent to:

  server_ref = naming.get_remote_interface_ref("server-id")
  result = server_ref.get_photo(photo_id)

The only thing that’s different is the transport and the invocation mechanism.

Hypermedia Applications

Consider instead that a hypermedia application is “an application which uses associative relationships amongst information contained within multiple media data for the purpose of facilitating access to, and manipulation of, the information encapsulated by the data.” (Lowe & Hall, 1998). They go on to say:

“The second aspect of the definition relates to the associative relationships amongst the information. What makes hypermedia different from other information management systems is that it utilises the complex associative inter-relationships amongst the information which underlies hypermedia applications - hence the common focus on non-linearity and networks.”

The only “complex associative inter-relationships” present in an API are its method name and parameters. Even the preconditions and postconditions have to come from somewhere else. Also, each function has no relationship to any other function in the API (unless it represents methods in a class). Even if it’s a class and you determine the methods and signatures via reflection, there’s still no description of the order in which those methods should be called or how they relate to each other obtainable based on the API itself.

To me, the big thing about hypermedia is that the service provider “owns” all of the URIs and the resources, and the client only has a limited, uniform interface that it can use to talk to the server. The interesting thing about this uniform interface isn’t the transfer protocol (HTTP in this case), it’s the hypermedia resource format exchanged between the client and the server that defines what operations are available and allows the client (which only knows how to render the content, follow links and submit forms, for example) to navigate the hypermedia application. This format might be XHTML, it might be RDF, it might be XTM or it might be something totally different, but the point is that the client doesn’t go making any assumptions about how to access the information architecture of the service.

This characteristic is why hypermedia applications scale, and this is also why hypermedia applications don’t generally break if you start at the front door rather than trying to climb in the window by starting in the “middle” of the information space. The service may choose to allow you multiple entry points (”front doors”) that it’s willing to guarantee (at least for a while), e.g. permalinks for blogs or other dynamic services, but that choice is entirely up to the service provider and the client can’t do anything to influence it. The operations it performs should be totally bounded by the operations provided by the last hypermedia document received from the service, e.g. the service’s current application state. It’s totally dependent on the service to tell it what it can do when and to what.

The Big Problem

While all the above discussion about hypermedia applications is great stuff, the big problem with interesting hypermedia applications is that you need to understand what all the links do. With people, it’s easier, but with automated service consumers, it’s much harder.

Semantic technology is making progress in this area, and you can describe the available operations using RDF or XTM, but the “dark side” of the API is just sooo much simpler to develop, test and deploy. Unfortunately, there’s no easy solution for this problem now, so there’s a lot of people building a lot of HTTP-based APIs, and there’s eventually going to be a lot of brittle, hard to gracefully evolve services out there that miss the real benefits of the uniform hypermedia interface.

Final Thoughts

The two most important things you need to remember about URIs are that:

  1. Whoever creates the URI (and I don’t mean the template), really owns the URI, and anyone’s interpretation of it other than its creator can’t be guaranteed to have the same semantics.
  2. If you build “friendly”, non-opaque URIs, people will write clients that attempt to construct URIs based on either explicit documentation from the service provider, or based on implicit assumptions about how it works gained from observation. If you don’t want this to happen because you’re not prepared to support it, then you’d better use really opaque URIs for your service’s information architecture.

Ultimately, the service is still the final authority on what the URI means, because it is the one that does something interesting with it. Once the client gives the URI to the service, it’s lost any control over what happens. Sure, with HTTP-based APIs, there’s documentation, and certain things are supposed to happen, but there’s an awful lot of room for things to get out of sync.

4 Comments »

  1. Nick Gall said,

    April 29, 2008 at 9:57 pm

    Andrew,

    While I agree that there is a tradeoff between the ease of use/understanding gained by placing human-understandable metadata in URIs and the ease of interface change (enabled by loose coupling), I think you are leaning a bit too far on the ease of change side.

    In particular, the advice in the Dan Connolly quote is out of date and out of touch with the TAG recommendation concerning this issue: http://www.w3.org/2001/tag/doc/metaDataInURI-31 . One of key its conclusions is:

    “Assignment authorities may publish specifications detailing the structure and semantics of the URIs they assign. Other users of those URIs may use such specifications to infer information about resources identified by URI assigned by that authority.”

    Both Tim Berners-Lee (whose “Cool URIs” is also superceded by the finding) and Dan Connolly approved this finding.

    So your statement that “if you think this [Flickr’s URI template] is a “hypermedia application”, you need some serious help” seems a bit over the top, given that the TAG finding explicitly allows such templating — both in HTML FORMs and out-of-band interface documentation.

    I’d be curious about your thoughts on the finding, especially what parts you disagree with. For me, the key takeaway is the following:

    “Good Practice: Resource metadata that will change SHOULD NOT be encoded in a URI.”

    I would generalize that to:

    “Resource metadata that will change should not be encoded in the client in either hardcoded URI metadata elements or hardcoded format metadata elements.”

    It doesn’t matter whether a metadata element is in a documented URI template or in a documented microformat template, or in a documented resource representation schema — if the element documented and then encoded (eg “hardcoded”) into the client, then you have tight coupling.

    But there is no escaping some amount of tight coupling. All loosely coupled aspects of an interface depend directly or indirectly on tight coupling somewhere else. In other words loose coupling must be bootstrapped via tight coupling.

    Thinking through what parts of an interface must be tight so that the other parts can be loose is the essence of good interface design. Choose wisely and you get stable interfaces that are productive for decades (eg TCP/IP, SCSI, HTTP, HTML, URI). Choose poorly and you get either unstable interfaces are arthritic ones.

  2. AST said,

    May 1, 2008 at 9:10 am

    For those playing along at home, the discussion continues over on rest-discuss:

  3. Jacek said,

    June 10, 2009 at 7:56 pm

    Andrew,
    your blog entry has been in my to-read queue for some time, and finally I’ve gone through it. It’s generally very good (and agreeable to me), but there’s one point I’d like to raise:

    > The operations it performs should be totally bounded by the operations provided by the last hypermedia document received from the service, e.g. the service’s current application state.

    Since there’s no guarantee, after whatever time the client needs to process the last document received, that it reflects the service’s *current* application state.

    Therefore I’d relax this - the operations the client performs should be bounded by the operations provided by the hypermedia documents received earlier from the service.

    I.e., the client can go back() in its history. Sure, the more back it goes, the higher the likelihood that the documents contain obsolete operations, but otherwise you would be suggesting that all hypermedia documents should contain explicit links back or up in the service resource hierarchy (back is hard because the service may not know where the client came from, modulo referer header), for example every item would have to link to (most) every container in which it is.

    I think such design isn’t really necessary - the back button works quite well, after all, and there’s no reason it shouldn’t work in non-browser clients as well.

  4. AST said,

    June 11, 2009 at 12:03 pm

    Jacek,

    Fair point, but I wasn’t intentionally trying to restrict it. “Last” in this case can be interpreted as relative, so if you click “Back”, that’s now the “last” document as far as the hypermedia exchange is concerned.

    The idea I was trying to get across was that you could fall over a hypermedia document lying on the street, pick it up and following the links should be all you need to do. I realize that this glosses over a lot about actually *understanding* the semantics of the markup, but I hope you get the idea.

    One thing for sure that I DID NOT imply, was that you should embed “back” links of any kind in the document. You might have links that effectively go “back”, but only if it’s a valid state transition from the current representation. Only the app creating the representation can know this.

    Thanks for reading the post! :)

    ast

Leave a Comment