Previously, I’d figured out a way to convince ROS to work under Linux. What I didn’t realize at the time, was that it only works on 32-bit Linux and not 64-bit. There are probably a couple of reasons for this sorry state of affairs. First, Sun isn’t going to provide a 64-bit Java plug-in until the release of Java 7 (or 1.7 if you’re one of us old-timers that’s worked with it since the beginning). Second, I’d say that the core “kcrypto” part of the way ROS works hasn’t been revisited in a while (since it’s built on Baltimore’s crypto stuff, and they haven’t been in this space for about 5 years).

Therefore, if you must have access to the Irish Revenue’s ROS business services from Linux, you’d better stick with either a 32-bit OS, or at least the 32-bit version of Firefox. I’ve tried every 64-bit Java plug-in that I can get my hands on, but to no avail. I’m harboring secret hopes that maybe when Sun releases Java 7, it’ll break ROS and they’ll have to revisit it in a way that makes it more accessible to non-Windows hosts, but I don’t think I’ll be holding my breath.

The closest I’ve gotten is with the gcj/icedtea version, but I get the following error:

jvmcheckerurl /java/javachecker.jsp
readBrowserDetails:: JVM Vendor: Sun Microsystems Inc., JVM Version: 1.6.0Kcrypto Installed? trueKCrypto Version: 8,0,1,0
Crypto release KeyTools Crypto v5.2
Crypto version 5.2
KCrypto present
getBrowserDetails:: JVM Vendor: Sun Microsystems Inc., JVM Version: 1.6.0Kcrypto Installed? trueKCrypto Version: null
getJVMVendor().trim().equals(browserDetail.getJVMVendor().trim()) true
getJVMVersion().trim().equals(browserDetail.getJVMVersion().trim() true
this.isKcryptoInstalled()==browserDetail.isKcryptoInstalled() true
 readBrowserDetails().equals(getBrowserDetails()) true
readBrowserDetails:: JVM Vendor: Sun Microsystems Inc., JVM Version: 1.6.0Kcrypto Installed? trueKCrypto Version: 8,0,1,0
Crypto release KeyTools Crypto v5.2
Crypto version 5.2
KCrypto present
getBrowserDetails:: JVM Vendor: Sun Microsystems Inc., JVM Version: 1.6.0Kcrypto Installed? trueKCrypto Version: null
getJVMVendor().trim().equals(browserDetail.getJVMVendor().trim()) true
getJVMVersion().trim().equals(browserDetail.getJVMVersion().trim() true
this.isKcryptoInstalled()==browserDetail.isKcryptoInstalled() true
matchBrowserDetails==true
GCJ PLUGIN: thread 0x622920: plugin_in_pipe_callback
GCJ PLUGIN: thread 0x622920: plugin_in_pipe_callback: setting status exception: java.security.AccessControlException: access denied (java.security.SecurityPermission putProviderProperty.JCRYPTO).
  PIPE: plugin read: status exception: java.security.AccessControlException: access denied (java.security.SecurityPermission putProviderProperty.JCRYPTO).
GCJ PLUGIN: thread 0x622920: plugin_in_pipe_callback return
  PIPE: appletviewer wrote: status exception: java.security.AccessControlException: access denied (java.security.SecurityPermission putProviderProperty.JCRYPTO).
java.security.AccessControlException: access denied (java.security.SecurityPermission putProviderProperty.JCRYPTO)
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:342)
	at java.security.AccessController.checkPermission(AccessController.java:553)
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
	at java.lang.SecurityManager.checkSecurityAccess(SecurityManager.java:1715)
	at java.security.Provider.check(Provider.java:403)
	at java.security.Provider.put(Provider.java:326)
	at com.baltimore.jcrypto.provider.JCRYPTO.<init>([DashoPro-V1.3-013000])
	at com.baltimore.psd.revenue.applet.AuthenticateApplet.init(Unknown Source)
	at sun.applet.AppletPanel.run(AppletPanel.java:436)
	at java.lang.Thread.run(Thread.java:636)

Doing a bit of digging on Google is actually kinda difficult, but eventually just searching for the putProviderProperty.JCRYPTO part yields the following gem (on the Java Forums from 1998!):

If you take the cert that either KeyTools or BouncyCastle is signed with and look at the netscape cert type extension you can see that it has the following fields set:

SSL Client Authentication,SSL CA,SMIME CA,Signature CA (87)

You will note that ObjectSigning is not there which is really odd for a “codesigning” certificate.

A bit of a google search brings up this link (I can’t find this doc on sun’s site)

http://www.archive.cc.yamaguchi-u.ac.jp/apl/jce1.2.1/BUGS.html

The JCE Code Signing CA uses Netscape CMS 4.1 to import Certificate Signing Requests (CSRs) from users and generate code-signing certificates that the users can utilize to sign their providers or exempt applications. The CSRs generated by keytool are in the PKCS#10 format. A bug in Netscape CMS 4.1 causes it to be unable to import a PKCS#10 request if it is directed to generate an object (code) signing certificate. But it can import a PKCS#10 request if it is directed to generate an SSL server certificate. This problem is expected to be fixed in Netscape CMS 4.2.

Workaround: The JCE Code Signing CA will issue SSL server certificates for code signing for now. It will be able to issue object signing certificates once we upgrade to Netscape CMS 4.2 after it becomes generally available

I presume that both we (baltimore) and BouncyCastle would need to get our codesigning certificates re-issued by Netscape CMS 4.2 before they would be accepted as individually signed jars by the java plugin.

Note:I emailed the java-security list about this almost 2 weeks ago with no response to date.

regards

kevin

Kevin O’Regan
Team Lead
KeyTools Pro Java, Baltimore Technologies.

Not wanting to take the time to figure out how to turn on the security debugging for the plugin (mentioned by Kevin earlier in his reply), I don’t know if this is the same issue or not, but my guess is that someone’s decided that Java 7 should actually get rid of some of the workarounds included over the years (at least for the pre-releases). This choice could very well cause really old code (presumably like ROS’s kcrypto.jar) to break in new (or old) and exciting ways.

Even if I did take it further and figured out what was actually wrong, I’m not sure I could fix it this time. In the amount of time I’ve spent arsing around with trying to get it working, I could’ve bought VMWare Workstation (twice!), so it isn’t really a very good use of my time.

It is a shame to see how poor 64-bit client operating system support is at the moment. Even Vista doesn’t come pre-installed on new machines in 64-bit mode. If I wanted a 32-bit machine, I’d have kept my old ones. I don’t understand the mentality of why vendors think this is a good thing because 32-bit code works in backwards-compatible mode. That’s like buying a V8 and then unhooking the fuel injectors to half the cylinders. It just doesn’t make any sense.

Maybe after 64-bit x86 processors have been out for 5-6 years—oh wait…

There are some key points about URIs and opacity that anyone who’s really going to the trouble to design them should keep in mind (originally presented by Dan Connolly at a W3C event back in 2001):

DO

• …make URIs mnemonic when it’s convenient

DON’T

• …depend on finding metadata in URIs
• …depend on search engine heuristics regarding query parameters
• …depend on file extensions
• …depend on URIs for user interface

Basically, don’t peek inside the URI. Notice there’s a lot more “don’ts” than “dos”. Also notice if you’re tempted to provide mnemonic names like so (based on the example from the discussion thread):

  http://www.example.com/parts/sku1234/orders/order56789

the natural extension is to start writing clients based on “constructing” URIs something like this:

  def to_s
    "#{baseuri}/parts/sku#{sku}/orders/order#{order_id}"
  end

Taken a little further and you’ve a “REST API” (from flikr’s API documentation):

Photo Source URLs

You can construct the source URL to a photo once you know its ID, server ID, farm ID and secret, as returned by many API methods.

The URL takes the following format:

http://farm{farm-id}.static.flickr.com/{server-id}/{id}_{secret}.jpg
	or
http://farm{farm-id}.static.flickr.com/{server-id}/{id}_{secret}_[mstb].jpg
	or
http://farm{farm-id}.static.flickr.com/{server-id}/{id}_{o-secret}_o.(jpg|gif|png)

Now, if you think this is a “hypermedia application”, you need some serious help. The above represents an application programming interface with an explicit contract between the client and the server about what parameters it will accept and where they can be placed. Notice as well where this explicit contract is defined: in another resource—the API specification. There’s no other source of information on how to retrieve a particular photo other than this documentation. You, as the service consumer, also need to know a whole lot about flickr’s infrastructure layout (retrieved from somewhere), or you won’t be able to retrieve the photo.

Once you know the API, making the specified requests to the specified URIs – constructed by the client – will result in a predictable response from the server. Functionally, it’s equivalent to:

  server_ref = naming.get_remote_interface_ref("server-id")
  result = server_ref.get_photo(photo_id)

The only thing that’s different is the transport and the invocation mechanism.

Hypermedia Applications

Consider instead that a hypermedia application is “an application which uses associative relationships amongst information contained within multiple media data for the purpose of facilitating access to, and manipulation of, the information encapsulated by the data.” (Lowe & Hall, 1998). They go on to say:

“The second aspect of the definition relates to the associative relationships amongst the information. What makes hypermedia different from other information management systems is that it utilises the complex associative inter-relationships amongst the information which underlies hypermedia applications - hence the common focus on non-linearity and networks.”

The only “complex associative inter-relationships” present in an API are its method name and parameters. Even the preconditions and postconditions have to come from somewhere else. Also, each function has no relationship to any other function in the API (unless it represents methods in a class). Even if it’s a class and you determine the methods and signatures via reflection, there’s still no description of the order in which those methods should be called or how they relate to each other obtainable based on the API itself.

To me, the big thing about hypermedia is that the service provider “owns” all of the URIs and the resources, and the client only has a limited, uniform interface that it can use to talk to the server. The interesting thing about this uniform interface isn’t the transfer protocol (HTTP in this case), it’s the hypermedia resource format exchanged between the client and the server that defines what operations are available and allows the client (which only knows how to render the content, follow links and submit forms, for example) to navigate the hypermedia application. This format might be XHTML, it might be RDF, it might be XTM or it might be something totally different, but the point is that the client doesn’t go making any assumptions about how to access the information architecture of the service.

This characteristic is why hypermedia applications scale, and this is also why hypermedia applications don’t generally break if you start at the front door rather than trying to climb in the window by starting in the “middle” of the information space. The service may choose to allow you multiple entry points (”front doors”) that it’s willing to guarantee (at least for a while), e.g. permalinks for blogs or other dynamic services, but that choice is entirely up to the service provider and the client can’t do anything to influence it. The operations it performs should be totally bounded by the operations provided by the last hypermedia document received from the service, e.g. the service’s current application state. It’s totally dependent on the service to tell it what it can do when and to what.

The Big Problem

While all the above discussion about hypermedia applications is great stuff, the big problem with interesting hypermedia applications is that you need to understand what all the links do. With people, it’s easier, but with automated service consumers, it’s much harder.

Semantic technology is making progress in this area, and you can describe the available operations using RDF or XTM, but the “dark side” of the API is just sooo much simpler to develop, test and deploy. Unfortunately, there’s no easy solution for this problem now, so there’s a lot of people building a lot of HTTP-based APIs, and there’s eventually going to be a lot of brittle, hard to gracefully evolve services out there that miss the real benefits of the uniform hypermedia interface.

Final Thoughts

The two most important things you need to remember about URIs are that:

1. Whoever creates the URI (and I don’t mean the template), really owns the URI, and anyone’s interpretation of it other than its creator can’t be guaranteed to have the same semantics.
2. If you build “friendly”, non-opaque URIs, people will write clients that attempt to construct URIs based on either explicit documentation from the service provider, or based on implicit assumptions about how it works gained from observation. If you don’t want this to happen because you’re not prepared to support it, then you’d better use really opaque URIs for your service’s information architecture.

Ultimately, the service is still the final authority on what the URI means, because it is the one that does something interesting with it. Once the client gives the URI to the service, it’s lost any control over what happens. Sure, with HTTP-based APIs, there’s documentation, and certain things are supposed to happen, but there’s an awful lot of room for things to get out of sync.

Hard to believe that I haven’t posted anything since Feb ‘07. I’ve a few draft posts that I’d written last year that I may or may not finish and post at some stage, but there are a few things I want to talk about again, so I’m making the time.

First off, the facelift. One of the things I’ve been playing around with for some time is getting more into visual design. Having spent some time doing this and being frustrated by not being able to get what was in my head out on to some kind of medium, things have recently started to get a bit better. I’ve had some ideas for a new blog theme for a while now, but I just hadn’t gotten to it.

Overall, I’m pretty happy with this one. The CSS is a lot cleaner before and so’s the layout, but I will say that PHP and I are not friends. There’s still a few minor tweaks to do where I have to dig inside some wordpress functions since it insists on outputting embedded markup. I have to say that I’m not too impressed with the way it generates content, but I’m sure it was done to make it as easy as possible. Unfortunately, those things often turn out to be difficult to change once you change your context.

There’s more changes to come, but they’re more like minor surgery than a facelift and they’ll all be under the covers (hopefully). I expect things to stabilize pretty quickly though. In getting a bit more serious about managing the blog, I’ve come up with some wordpress tools that I’ll probably share once I’ve tested them a bit more. I found that it was a RPITA to move wordpress through a normal release cycle from a dev to a staging and then production environment without interactively tweaking a bunch of things, so I’ve written some scripts to make this process a bit easier.

Hope you like the new look.

Secondly, I’m very happy to say that OASIS has accepted my proposal for a presentation at the OASIS Symposium in April. I’ll be expanding on some of the ideas that I discussed in my “Is ‘REST API’ an Oxymoron?” post and putting them more solidly in a business context. The presentation, “Interactive SOA: Towards Content-Centric Services”, will be part of the session on the challenges ahead for e-Business and e-Government. The presentation represents some on-going research into the challenges around enabling information services within an SOA using hypermedia. Hopefully, sometime in 2007 I’ll be able to publish some more technical information relating to the research, but doing so without understanding how it will deliver measurable value to organizations isn’t in line with what we’re about.

Finally, I’ve updated the Event Calendar to include links to the presentation slides that are publicly available. I’ve added my slides from both the first US Government’s SOA for E-Government conference as well as the OASIS Adoption Forum in London. I need to check with the InfoSeCon organizers to see if I can publish my presentations from last year or not, because the proceedings were only made available to conference attendees.

If you have any questions/comments/problems with accessing any of the information, please drop me an email at any of the addresses on the About page.

I finally completed the convoluted identity proofing process required by the Irish Revenue Commissioners’ Revenue Online System (ROS) so that I could view/file all of the necessary tax forms for Archistry myself. The registration process/identity proofing for business customers theoretically involves the following 3 steps:

1. Application for a ROS Access Number
2. Apply for your digital certificate
3. Retrieve your digital certificate

Each one of the first two steps involves submission of details to the ROS site and waiting for a letter from Revenue with a code or password that must be used to get to the next step, so there can be a considerable amount of time involved in completing the whole process before you can actually use the service.

“For your own security”, ROS requires 2-factor authentication to enter the site as well as to submit any particular tax form. The authentication is based on the digital certificate you retrieve from ROS as well as a password you create for the certificate. This password can be subsequently changed after you initially provide it. Since you need to store this certificate locally, ROS decided that the best way to implement the authentication mechanism was using both a Java Applet and a custom package, KCrypto (not to be confused with the KDE package by the same name), that includes software from Baltimore to handle the certificate operations.

Problems arise when you aren’t running Windows or MacOS, and, according to the system requirements page even though you can execute the pre-requisite applet, Unix or Linux isn’t yet a supported platform—even though it is promised “soon”. The root of the issue is really two-fold: first, on an operating system where you just can’t throw files higgledy-piggledy anywhere you like, the applet installer doesn’t have the permissions to write the files, and the running applet doesn’t appear to make any attempt to put the certificates anywhere other than “C:\ROS”. Even if MacOS X is a supported platform, from the comments on a blog post from Tom Raftery in 2004, people are still having trouble using ROS under MacOS X in September 2006. Other people have tried using Wine to get the ROS applet to work under Linux, but they didn’t have much luck either.

What’s that? “Java is platform independent; you know, ‘write once, run anywhere’,” you say? “But, it’s a Web application,” you say?

Not quite.

Even with all those obstacles, it is possible to get it working natively under Linux if you have a little patience and can tolerate a few display glitches.

Making ROS work under Linux

NOTE: ROS does not officially support Linux, so these instructions are provided “as is” and do not imply any sort of warranty or liability by me if you have problems with your taxes if you try and use Linux to access ROS. Use at your own risk.

I’m using Firefox 2 under Linux with the JRE 1.4 plug-in. Everything works up until you attempt to install the applet as part of Step #3 for Retrieval of your Certificate. I managed to make everything work, but I had to re-trace my steps back to retrieval of the certificate because once the required libraries are installed, it kept asking me to log in. I hadn’t gotten the certificate yet, so the log in drop-down for certificates was empty.

When attempting to download and install the supporting applet, you will first encounter an error. Using information from the Java Console, you can identify exactly what the issue is. Because I wasn’t clever enough to save them, I can’t tell you exactly what they were, but you will see lots of “permission denied” errors at various points during the install. To display the Java Plug-in Console, run $JAVA_HOME/jre/bin/ControlPanel and set it to “always display” before you start trying to install the ROS applet.

At some point, you will get a message about downloading kcrypto-sun-1.4.jar. You will need to manually download this file from https://www.ros.ie/applets/kcrypto-sun-1.4.jar and copy it to: $JAVA_HOME/jre/lib/ext/.

This is the key step. Once you have done this, the applet installation will work and you should be able to start at the ROS “Home Page” and start the process of retrieving your certificate again.

Issues

Once the installation issue is solved, you will have to deal with a set of applets built using absolute positioning with the Windows control metrics. This means that there are overlaps between the controls and the height of some of the text fields will prevent you from seeing what you’re typing. Most of these aren’t anything but annoying.

As previously mentioned, your certificates will be stored in $HOME/C:/ROS/, which, while annoying, seems to work.

If I notice anything else as I use the service more, I’ll update this article with the new information. Otherwise, if you’re determined to use Linux for running your business in Ireland, I hope this article helps you get around some lazy coding on the part of the ROS team.

It is quite clear that ROS was intended first and foremost as a Windows service. Having written real cross-platform Java applications using JFC in the past, it isn’t generally too hard to do it, if that’s what you’re starting out to do and you have some understanding of the target platforms. Bolting it on later nearly always results in a number of places where people make assumptions about where things live (having a drive C: on something which isn’t Windows, for example) that could’ve been avoided with a bit more planning at the beginning.

All of us are tired, but doing very well. We would like to thank Geraldine and all of the other midwives and staff who helped bring little Alex into the world.

For family and friends who want to keep track of how the little fella is doing, we’ll be putting up a separate site for the family soon.

Anyone who’s been in IT for a while and who has had to try and help a family member do something “obvious and simple” under Windows knows the frustration of trying to do remote support the “old way”. You can talk to the person, but you’ve no idea what they’re doing, and you’ve no idea what they really see in front of them if they’re not an experienced computer user and know hot to accurately describe the problem they’re having (and if they were, you probably wouldn’t be talking to them about computer problems anyway).

Enter WebEx. I have to say that I think this is a brilliant idea. For those of you who haven’t used WebEx, it provides you with a Java client that allows two or more parties to hold virtual meetings. The meeting moderator can share their desktop with the rest of the meeting, but they can also pass control for desktop sharing to other members of the group, so that they can give presentations or whatever. The installation of the WebEx client is relatively easy based on following a URL that can be sent in an email, and then all the other people need to do is connect to the virtual meeting and dial into the conference call number or any other number that you choose to set up. Imagine then if you, as a meeting moderator, could give control of desktop sharing to your mother who couldn’t figure out where the “www bar” went in her Web browser. In about 5 seconds you would realize that somehow it had been either full-screened or that it had been disabled from the View menu. Problem solved.

Now, if you’re a professional support organization, compare the actual cost of 10,000 5-10 minute support calls vs. the cost of 3500 20-30 minute support calls—not only in terms of time, but in terms of the effectiveness of your customer support function. The benefits and cost savings would be fairly significant any way you cut it.

Of course, WebEx isn’t without its problems as well. A few people at BearingPoint had trouble getting it installed/upgraded or just using it—and these were well-paid, experienced IT professionals, so maybe the reality isn’t quite in line with the vision yet. However, I don’t think it would take much effort on the part of WebEx to address these issues to make the installation and usability much better for the non-technical user. Also, since broadband in the home is becoming more ubiquitous, it means that you potentially can offer this level of support to more than just your corporate customers with a high-speed Internet connection.

The down side is that WebEx isn’t exactly cheap, but then you need to evaluate that cost against the benefits your organization can achieve in improving the overall efficiency and quality of its customer support. For some organizations, this will not be an option, but I think it is something that any company offering any kind of IT product support to end users should really consider carefully. It isn’t perfect, but it is a whole lot better than the situation painted by the very old computer support joke about the computer not working and the user eventually needing a flashlight to see if it was plugged in or not. Why? Well, you see, the power had been cut to the building… Many times seeing the situation for yourself is worth more than 1,000 words; it can actually be worth cold, hard cash.

One of the light bulbs that went off in my head during the aforementioned discussion was it finally clicked as to what the uniform interface constraint “hypermedia as the engine of application state” (Fielding 2000, §5.1.5) actually means (to me, anyway). Now that I think I understand it, I also understand why it is so hard for people to really get what it means: most of the people trying to figure this out are experienced, traditional programmers. What do traditional programmers do? If they’re any good and they’re dealing with large-scale distributed systems, they spend an awful lot of time on the design of the “perfect” API for their remote components. Perfect in this context means that it is the optimal trade-off between all of the architectural constraints and system requirements to deliver an efficient distributed system.

API Brain Damage

I’m going to go out on a limb here, but I think this “API brain damage” is part of why we haven’t been able to significantly advance the state of the art of software system design over the last 30 years. We all have it, because from the moment you learn what a compiler is and get started with programming they’re all around you. As you develop your own skills and experience, you’re exposed to both good and bad API design, and this all gets mixed in together in our little brains so that each of us develops their own perspective on what is a “good” vs. a “bad” API. Most of us can, by this stage, pass such a judgment in 30 minutes or less.

However, I think the effects of this brain damage are to create some pretty fundamental assumptions about how computer systems work. If you need programmatic access (here it is again, reinforced by the terms we use to describe our requirements) to a set of functionality (for kicks, lets call it a service), then the first thing we as programmers want to know is “where’s the API?”. It’s ingrained in our very being because of years and years of positive reinforcement. We all have it…and I think it’s a tragic mistake.

Even people who have a pretty good understanding of what REST is get pulled back into the primordial slime just as they’re about to sprout legs and walk upright. Joe Gregorio’s really good article on building a RESTful system, Constructing or Traversing URIs?, is a prime example of this type of thinking. I’m not criticizing Joe here, I’m criticizing the way we, as software architects and developers, have been trained to think.

The point of Joe’s article is that hypermedia is about link traversal, but that because the possible URI space is nearly infinite, it’s reasonable to publish recipes for link construction under the argument that this is what HTML Forms using the GET method do anyway (emphasis added). This “optimization” undermines what to me is the whole point of “hypermedia as the engine of application state”: link traversal, and this is recognized by the rest of the article Joe wrote. As soon as you start down this slippery slope, you’ve lost all of the advantages I see in even bothering with hypermedia at all. It no longer becomes part of the application, it’s just data that’s shipped around via some transfer mechanism, in this case HTTP.

Hypermedia Applications

Many things have contributed to my opinion about this topic, including this statement by Eric Newcomer recently on the mailing list:

I should just note that comparing the Web to WS-* is an apples-to-oranges comparison (one being an application and the other being a collection of specifications).

With apologies to Eric, I initially sorta dismissed this statement because I was focused on my conversation with Steve Jones—but I shouldn’t have. Eric is exactly right here. The World Wide Web is a hypermedia application, not just a collection of specifications. Again, our software development training doesn’t help us here very much. Good ol’ functional decomposition makes us (well, me at least) want to see the Web as HTTP (RFC 2616), URIs (RFC 3986), MIME types (Wikipedia entry to set of related RFCs), HTML/XHTML (W3C markup specifications) and HTML forms (W3C recommendation and RFC 2388).

In actual fact, it’s all of those working together to provide, as Eric said, the Web as a distributed hypermedia application (which, if I remember correctly is a point also made by Roy Fielding himself in the dissertation). The Web works because both user agents (browsers) and the server applications use all of these specifications together to expose a set of functionality to the interactive user. There is no a priori agreement between the browser and the Web server as to how the information service built on these specifications used, but because of agreement on how these specifications are used together, as an application, it doesn’t matter if today, CNN.com is built using Microsoft ASP, and tomorrow it’s built on PHP. Apart from the application of the “Cool URIs Don’t Change” principle, if a user starts from http://www.cnn.com, they will always be able to utilize the CNN news service via their browser’s implementation of those specifications and the implicit agreement of CNN to publish its service in accordance with them too.

The moment that CNN or any other service provider publishes a recipe, guideline or specification of how to access specific parts of that service, e.g. the latest headlines may be found at http://www.cnn.com/headlines/ or http://headlines.cnn.com/ or whatever, as Joe points out in the article, they’ve made an implicit commitment to support that API (because that’s what it essentially is) for a period of time. When someone comes along an implements a specialized headline grabber that follows that API, and CNN decides to change it due to idle whim or genuine business need, that headline grabber client is now broken. If it’s just a single user, maybe this isn’t a big deal, but if it is every 3rd-party trading partner of your organization, the impact is a bit more significant.

A hypermedia application such as Atom or RSS and content negotiation via MIME types, HTTP accept headers and embedded <link/> tags mean that this sort of evolution could happen without breaking the client—provided there is agreement on the application semantics of how those things should be both used and interpreted between clients and servers. Anything else means you’re back to brittle, API based systems that can no longer evolve independently of each other.

Closing Thoughts

I don’t have all of the answers here, but I think that the notion of a “REST API” is an oxymoron because REST is about dynamic evolvability of clients and servers based on codified understandings of previously agreed application semantics. This means that HTML browsers and Web servers agree to provide the Web application in a way that both can understand and use, but it also means that RSS/Atom feed readers and server feeds agree on the way they interact to both access and provide the syndication application.

What I’m saying is that the nature and specification of the hypermedia application is as key to REST as how you use the HTTP verbs. However, since the HTTP verbs are essentially an API that programmers can get their heads around, that’s where everyone’s focus is at the moment. I think this is a diversion from where people should be thinking about REST. As long as you agree the semantics of the hypermedia application (HTML+Forms+MIME+HTTP or Atom+XHTML+MIME+HTTP), the way that application is implemented on the server should be an implementation detail and not something exposed to clients in terms of an API.

If the hypermedia constructs being used to describe the interaction between the clients and servers are not rich enough to abstract these things so the client needs to know that it’s supposed to POST data to URI x rather than being able to simply traverse hypermedia provided by the server (meaning the operation, data and location are provided to the client in a way it can understand rather than having any of this hard-coded as client implementation logic), then the hypermedia application being used (and not the information service) has not been sufficiently defined. Using the existing and emerging specifications for describing content and interaction, it should be possible to specify the application. If it isn’t, then we need to be spending our efforts on a way to do that rather than arguing about the RESTfulness of so-and-so’s latest HTTP-based API.

To me, this is the real problem to be solved in implementing RESTful systems. I think there are people who are starting to realize this need implicitly, but I think it’s time we made that need an explicit requirement of systems implemented in the REST style. If you can’t describe the interaction via hypermedia and link traversal semantics only, then I don’t think the system truly meets the requirements of REST as I understand them today. The uniform interface is hypermedia, not HTTP. Focusing on HTTP is not seeing the forest for the trees.

Comments, flames and discussion are more than welcome.

I have tremendous respect for Steve and everyone else on the list [the service-oriented-architecture list] that I’ve interacted with, so this isn’t personal in any way. I think it is important to understand a bit of industry history in light of lots of smart people and vendors trying to figure out how to field an SOA that works.

A lot of us on this list have been doing distributed computing for a long time. Most of us have done a lot of one or more of CORBA and DCOM before RMI/EJB came on the scene and certainly before XML-RPC and SOAP came on the scene (some people have been doing it earlier than that).

The thing about a programming paradigm is that to get any good at actually doing something with it, it takes a lot of time and effort to learn how to think and design in a way that takes advantage of it. CORBA, DCOM and EJB and the like are about extending the local programming model to remote systems in a more-or-less coherent way.

All of them are object-oriented in that you create a service with a defined set of capabilities and a given interface. This interface is normally designed in similar ways to local interfaces in that it exposes a fairly rich and domain-specific API for interaction between clients and servers. Most of the early mistakes people make in developing CORBA, DCOM and EJB projects are in the granularity of those interfaces because they forget or don’t consider the effect of the cost and overhead of communicating over the network vs the costs within the same address space, e.g. “normal” objects.

Learning how to optimize the tradeoff between a rich, domain-specific interface and one that is efficient is one of the key things in learning how to design and develop successful distributed object systems.

If you take a look at the history involved in developing these systems, formalization of CORBA started in 1990 at the OMG, DCOM surfaced around 1993 and RMI and EJB emerged in 1997. Getting all of these technologies implemented took a lot of work because most of them are naturally fairly complex. It isn’t easy trying to make a remote system look like it is a local one. Lots of vendors produced a lot of products, and some companies were founded around some of these technologies.

While each of these technologies is good (to varying degrees) at providing a distributed object computing platform within a local physical environment, they didn’t scale very well over long distances or between enterprises. Most of them required a large number of proprietary ports to be opened in company networks, which has security implications not to mention just the operational issues of making it happen.

On the other hand, HTTP and Web pages nicely sailed through port 80 which, in most cases, was already open. Both vendors and customers said, “Wouldn’t it be great if we could do things like CORBA, but using HTTP?” Enter XML, XML-RPC and SOAP in 1999-2000.

Now, if you were a vendor that had spent millions in R&D in getting distributed objects computing working in CORBA, DCOM and EJB but had come up against limiting factors such as complexity of deployment (all those ports), lack of interoperability between CORBA, DCOM and EJB and the way the Web was influencing the development of applications, what would you do?

I bet you’d figure out how to take all those things you’d been doing and make them work over ubiquitous Web protocols. I’m not saying this is necessarily bad and doesn’t have its place, but there are two other big reasons why you might think it would be reasonable to do this:

• it is the way major software vendors had been developing systems since as early as 1990, meaning
• there was a legion of software developers who understood how to develop distributed systems using those concepts and mechanisms

Vendors are protecting their investment because they need to stay in business and keep their shareholders happy, but somehow make their distributed computing technologies work together as more and more people are running heterogeneous environments not only internally, but across trading partners.

The Web is different, however.

In the same way that messaging-oriented middleware (MOM) isn’t the same way of thinking about solving distributed computing problems as using distributed objects, building successful distributed hypermedia applications using REST for either human/computer or computer/computer interaction requires a shift in the way you think about the problem.

If you can’t suspend your assumptions about how things ought to work to understand how they do work in a different environment, e.g. MOM or REST, you’ll forever be frustrated and not understand the advantages and disadvantages of this approach over any other. From my perspective on the recent ROA/SOA thread, this is where we are and why reaching any sort of common understanding is and will continue to be so difficult.