Posted in SOA by AST on Saturday, March 25th, 2006
I first read about self-directed messages in David A. Chappell’s book Enterprise Service Bus. In it, these types of messages are illustrated using itinerary based routing where the message sender knows what steps the message needs to take to accomplish a complete business process (an instance of the Routing Slip pattern described by Hohpe et al.). At the same time, I was doing some security research on PKI which led me to grid computing. Since then, I’ve been convinced that the key to scaling applications lies in the Internet model of highly-distributed computing. By combining grid computing and the Internet model of scalability with a generalization of itinerary-based routing, you end up with self-directed messages. This concept is the basis of the W3C’s WS Choreography Model.
What’s the Problem?
In real life when you try to do too many things at the same time, a bottleneck develops. We’ve all been there. You have a list of things you’re supposed to do each week that seems to always get longer rather than shorter. The net effect is that the things you’re supposed to be doing on which other people are depending just don’t get turned around in a timely fashion. Therefore, you’re the one everyone is waiting on before they can continue with their own ever-increasing list of tasks.
This same problem is common in software systems. In fact, well-proven design patterns like Model-View-Controller and even the nature of the main() function in most programming languages can tend to encourage this behavior. The control of what your program is trying to accomplish needs to go somewhere, so it’s only natural to manage that control in a central place. This is why programs start at main() and why the Controller of MVC exists.
However, the history of software and hardware development has shown that eventually, you’re going to need to do more than one thing at a time. To do that, you need to split what you’re trying to do into parts that can be replicated. This is the reason for multi-CPU hardware, time-sharing operating systems and multi-threading. However, the problem still remains one of control: something needs to direct what happens when.
Who’s in Charge Here?
The traditional answer to handling this control is that something, either the human, a process or a block of code directs all of the other parts of the program. This type of control is called orchestration, because, like a symphony orchestra, the musicians are supposed to follow the conductor. They aren’t supposed to do anything on their own. Orchestration requires that there is central control somewhere, but the activities can then be spread across multiple CPUs, processes or threads.
While this approach helps, there is still a lot of time spent in trying to coordinate the different pieces. The more pieces there are to control, the more effort it takes to control them–picture trying to keep five kittens in the same box. The easy answer to the problem is vertical scaling by adding more resources to allow the controller to manage these additional tasks. This can obviously only be a short-term solution, because it is addressing the symptom of the problem rather than the core problem: how do I get more things accomplished more efficiently.
In software systems, this question is often asked, but no one likes the best answer. The best answer is to re-think about how the solution works, but by this time, there’s normally a lot of time and money invested in building what’s already there. Ripping it apart to make it more efficient is generally not an option, so we’re back to growing more arms so we can keep more kittens in the box.
Lessons from Life
As most of us can’t grow more arms, one of two things generally happens:
- You change the rules of the game (put a lid on the box)
- You ask for help
Most of the time you can’t change the rules, so you end up asking for help. In the “real world” this is called delegation, and it’s a hard thing for people to do–especially for things where they’re very concerned about the outcome. Delegation only works if who or what you delegate to has the autonomy to work independently from you without excessive interruptions or micro-management. They can only do this if you provide them enough information for them to do what you want them to do.
In software systems, the core enabler of this type of delegation building in support for asynchronous interactions. However, the use of asynchronous communication between components also implies event-driven behavior–an example of inversion of control (as explained by Martin Fowler). If you don’t have an event handling protocol, you don’t have an effective way to communicate with those you’re delegating to.
Unfortunately, lots of systems are still implemented based on a synchronous model. While sometimes it is desirable and necessary to use the Request-Reply pattern, it is also an easy way to kill your system performance by trying to apply synchronous thinking to an asynchronous environment. If all you are doing is RPC-style distributed computing, then you aren’t really delegating control anywhere else. You’re still directing the orchestra.
Let’s Dance!
If you can combine delegation and event handling, you can spread the work around and only act when you need to. The controller in your system is no longer the bottleneck, because everyone participating in the system has an assigned task and a protocol or way of communicating that allows them to communicate their results in an efficient manner.
Task description + asynchronous communication + event handling = choreography
In dance, the choreographer is the person who has the big picture in mind. They achieve a desired response from the audience by coordinating the movements of the dancers on the stage. Unlike the orchestra conductor, they don’t participate in the show. They have already done their work by planning it out and holding the rehearsals so that everyone knows their steps. When the curtain goes up, they’re watching from the wings.
Once the choreography is set, it can be executed many times without any interaction from the choreographer. This feature enables horizontal scaling because each instance of the choreography can be executed in parallel without putting any extra demands on the choreographer. By applying the lessons of the Internet and grid computing on how to build agents or processes that can execute these choreographies, and encoding the choreographies into the message, you get massive scalability.
Showtime!
It is important to note that the choreographies embedded within the message are not the process to be executed by the agent. Instead, they simply say what happens based on the outcome of the process. They are really state machines which need only relate a result code from a source to a destination. For example, it could be as simple as the following table specifying the URL for an agent, the response code and next destination:
- http://foobar.com/task1 : HTTP 200 ⇒ http://foobar.com/task2
- http://foobar.com/task1 : HTTP 500 ⇒ http://foobar.com/start
Of course, there needs to be a common processing model of what these things mean by all participants and what is supposed to be sent, etc., but this is the basic idea.
While this approach provides massive scalability because there is no point of central control, it also has several security concerns:
- How do you ensure the integrity of the choreography
- How do you manage the accountability of the choreography
- How do you track the progress of the choreography
These are tough questions, but they need to be sufficiently considered before choreography style services can be deployed. Item #1 can be addressed through the use of digital signatures. Item #2 is a people and organizational issue that needs to be solved outside of any particular technology.
Item #3 has a number of solutions, potentially related to tracking requests for a remote choreography signature, but it is important to not sacrifice the efficiencies gained from using choreographies by sending status messages equivalent to “I’m now going to the next step; Ok, I’m here”. This approach will indirectly introduce all the overhead of an orchestrated service that you were trying to leave behind.
Curtain Call
I fully believe that the only way to achieve Internet-scale SOA is to follow the lessons available from implementing the Internet and World-Wide Web. The model is that there is no central point of control, there are agreed communications protocols and data formats, and lots and lots of redundancy exists to provide both reliability and efficiency. Most of these end up being either completely or nearly transparent to the end user, regardless if they’re using ssh to access a server in Germany or using a Web browser to access Google.
The work being done by the W3C with WS-Choreography is a step in the right direction, but there are a number of hurdles before this model is going to materialize. The first and foremost is the mind shift that needs to take place so that architects and developers start thinking asynchronously. Until that happens, the way services are designed will produce more symphonies than ballets, and the conductors are going to need lots more arms to keep everything on track.
Permalink
Posted in SOA by AST on Thursday, November 10th, 2005
Interoperability is a pretty hot topic these days as more and more people are either implementing or talking about implementing Web services and especially SOA environments. However, if something isn’t done about it, the current initiatives in the WS-* space may unwittingly hurl us back into the days of worrying about how we were going to port applications across Windows, Motif and the Mac.
It is a common misconception in the Java world among people who don’t have a chance to read the fine print that JMS provides an interoperable solution for asynchronous, reliable messaging. However, what JMS actually provides is a consistent programming interface across a wide variety of Message Oriented Middleware (MOM) implementations. According to the JMS Specification, version 1.1, the objective of JMS is to:
“[define] a common set of enterprise messaging concepts and facilities. It attempts to minimize the set of concepts a Java language programmer must learn to use messaging products. It strives to maximize portability [emphasis added] of messaging applications.”
Note that there is nothing said in the above objectives about the interoperability of JMS implementations. The topic of portability is discussed further in the specification:
“The primary portability objective is that new, JMS only, applications are portable across products within the same messaging domain“ [emphasis added].
Finally, under the section What JMS Does Not Include is the crucial point about JMS and interoperability:
“Wire Protocol - JMS does not define a wire protocol for messaging.”
This statement is perfectly within the goals of JMS as a portable API for enterprise messaging. It is much the same approach as the JDBC API. JDBC means that you can write an application to talk to any database supplying a compliant JDBC driver. JMS means that you can write an application to talk to any messaging system supplying a compliant JMS implementation—but it doesn’t mean that you can use JMS to bridge messaging implementations. It is strange that so many people inherently understand this issue with JDBC, but miss this crucial point about JMS. Most people know that Java runs on lots of different platforms, but they typically don’t have a lot of exposure to MOM implementations, therefore they don’t see the distinction. I think Sun could have chosen a bit better naming for what they called the JMS provider to equate it a bit more closely with the JDBC driver concept, but that ship already sailed on April 12, 2002 (the date on the 1.1 specification).
So, to stress the point (there will be a quiz later): JMS provides a compatible API for enterprise messaging, but it does not provide an interoperable message transport protocol.
Enter Web Services
Historically, people solved the organizational interchange problem using the ubiquitous comma-delimited file (CSV), which is ironic itself in that there is no official format specification for CSV. Faced with the challenge of exposing or bridging MOM implementations, it makes sense to do it in a way which uses an interoperable wire protocol rather than relying on yet another proprietary solution (YAPS). The architecture specified by the W3C’s Web Services Architecture (WSA) attempts to address these issues through the use of WSDL, SOAP and additional protocols for reliable messaging. The current players are WS-Reliability from Fujitsu, Hitachi, NEC, Oracle, Sonic and Sun, and WS-ReliableMessaging from BEA, IBM, Microsoft and TIBCO.
In theory, both of these specifications do the same thing, however the detail of each shows a few subtle differences. These differences aren’t terribly important to this discussion, however. They focus mainly on whether there is requirement for other WS-* specifications and how batches of messages can be transmitted.
If you’ve already subscribed to the WS-* approach to Web services, these are pretty much the choices. The ebXML Message Service Specification (a.k.a. ebMS) is related closely to WS-Reliability, but it includes some things which are specific to its role in ebXML. According to the XML Cover Pages description, ebMS will probably be updated to position it more closely with WS-Reliability in the next version.
Leaving aside the argument of how complex it is to implement these specifications, they have been proven to varying degrees to be interoperable between vendor implementations.
“Great, so we’ve solved our problems. All we have to do is provide a Web services endpoint connected to our JMS vendor and we’re in business, right?”
Ah, Grasshopper… You have much yet to learn.
Inverting the Problem
The real issue with the WS-R* specifications is the same as for JMS: the devil is in the detail. While WS-R* may standardize the wire protocol into sending XML documents over HTTP, which should be interoperable as long as the XML is conformant to the specification, we have a problem at the next layer of the application: the API.
Anyone who has attempted to write cross-platform C or C++ code (and sometimes, even Java code, but to a much lesser degree) that did anything very complex has run into the problems that while both of these languages may be formal specifications, the libraries for doing useful things on a given platform aren’t always formally specified. With the latest POSIX specification, this has gotten a lot easier in the UNIX world than it used to be, but something as critical as a Graphical User Interface still has many proprietary (or at least divergent) variants, depending on your environment. Some of these are now open source like GTK+ and Qt, but there is still a very active demand for a cross-platform UI toolkit (e.g. FOX and wxWidgets (formerly wxWindows).
The reason is that pragmatic programmers really only want to write things once and be able to support the widest possible platforms. Even today, the user interface is still one of the biggest challenges to actually accomplishing this goal. While it’s true that Java provides JFC/Swing, that isn’t always the right solution for all applications. There are still advantages to writing UI applications in C++ over Java, depending on the type of application you’re developing.
Many of the patterns in the venerable Gang of Four book are framed with examples describing how they may be employed to minimize the dependence of an application on a particular UI toolkit. As illustrated in the book, this isn’t done just to cause more work and write more code; it is done to encapsulate the parts of your application that may change often (like the UI) from the parts of it that won’t (or shouldn’t, like the business logic).
This lesson, which many of today’s Java programmers have never been exposed to in the way that people who programmed before Java and JFC were, is still a critical aspect of successful software design. Part of the problem with today’s sophisticated developer tools is that it is very easy to let the tools do things for you. This approach can seem tempting for many types of tasks: refactoring simple name changes and automating the creation of boilerplate Java Bean attribute accessors, for example. However, today’s developer needs to be very wary of what’s going on behind the “magic curtain” of the sophisticated IDE. These are the things that will make a difference between your web service being deployable across various application servers and one which is stuck in the tendrils of YAPS.
It doesn’t need to be this way. While BEA’s implementation of WS-Reliability makes extensive use of the JDK 1.5 annotation feature, the open source implementation of WS-Reliability from Fujitsu, Hitachi and NEC, RM4GS, provides very neat integration into a J2EE container environment as a JCA component. Yes, the implementation of the specification is proprietary (even if it is open source), but the semantics of using it are exactly the same as any other JCA component. That’s the point of the specification.
The important thing about this to a developer is that their implementation code is relatively clean:
main(String[]args) {
InitialContext ctx = new InitialContext();
// Obtain a Connection.
// Specify the JNDI name of the connection factory.
ConnectionFactory cf = (ConnectionFactory)ctx.lookup("eis/rm4gs");
Connection conn = cf.getConnection(true, 0);
// Obtain a P2Pdestination. Specify the name of the queue.
P2PDestination dest = (P2PDestination)ctx.lookup("eis/SimpleQueue");
// Create a sending session.
Policy[] policies = new Policy[] { Policy.EXACTLY_ONCE, };
SendingSession session = conn.createSendingSession(dest, policies);
// Start the RM4GS transaction.
conn.begin();
// Create a message.
TextMessage msg1 = (TextMessage)conn.createMessage(MessageType.TEXT);
msg1.setMessage(...);
// Send the message.
session.send(msg1);
// Complete the RM4GS transaction.
conn.commit();
// Release the message.
msg1.release();
// Release the session.
session.close();
// Release the connection.
conn.close();
}
You still have the proprietary implementation imports, but at least the model is relatively straightforward. Also, these imports are actually interfaces, so if you ever really needed to, you could just bridge another implementation’s classes with the Adapter pattern.
The Apache Sandesha implementation of WS-ReliableMessaging is based on Axis and works a bit closer to the metal. From the User Guide, using it isn’t terribly complex:
public static void main(String[] args) {
try {
Service service = new Service();
Call call = (Call) service.createCall();
SandeshaContext ctx = new SandeshaContext();
ctx.initCall(call,targetUrl, "urn:wsrm:Ping", Constants.ClientProperties.IN_ONLY);
call.setOperationName(new QName("http://tempuri.org/", "ping"));
call.addParameter("arg1", XMLType.XSD_STRING, ParameterMode.IN);
call.invoke(new Object[]{"Sandesha Ping 1"});
call.invoke(new Object[]{"Sandesha Ping 2"});
ctx.setLastMessage(call);
call.invoke(new Object[]{"Sandesha Ping 3"});
ctx.endSequence();} catch (Exception e) {
e.printStackTrace();
}
}
The BEA implementation present in WebLogic 9, looks a bit different:
package examples.webservices.reliable;
import javax.jws.WebMethod;
import javax.jws.WebService;
import javax.jws.Oneway;
import weblogic.jws.WLHttpTransport;
import weblogic.jws.ReliabilityBuffer;
import weblogic.jws.BufferQueue;
import weblogic.jws.Policy;
/**
* Simple reliable Web Service.
*/
@WebService(name="ReliableHelloWorldPortType",
serviceName="ReliableHelloWorldService")
@WLHttpTransport(contextPath="ReliableHelloWorld",
serviceUri="ReliableHelloWorld",
portName="ReliableHelloWorldServicePort")
@Policy(uri="ReliableHelloWorldPolicy.xml",
direction=Policy.Direction.both,
attachToWsdl=true)
@BufferQueue(name="webservices.reliable.queue")
public class ReliableHelloWorldImpl {
@WebMethod()
@Oneway()
@ReliabilityBuffer(retryCount=10, retryDelay="10 seconds")
public void helloWorld(String input) {
System.out.println(" Hello World " + input);
}
}
I personally don’t like the use of the JDK annotations—especially when there are more annotations than Java code, but that isn’t the point either. The above 3 examples are supposed to all accomplish the same thing: reliable delivery of a message from point A to B, or in WSA-speak: between a requester agent and a provider agent. However, if you were the one implementing the service, or in our case, a simple Messaging Bridge between JMS and something else (maybe another JMS implementation), your code is intrinsically tied to the vendor implementation. Change vendors, change your code. You’ve just inverted the JMS interoperability problem and have interoperability without compatibility rather than compatible interoperability.
Learning from the Past
To solve this problem requires a look at some historical technologies: Berkeley Sockets and CORBA.
The Berkeley socket API was in a similar position to JMS, it provided an API on top of a message transport protocol. The key difference here is that the transport protocol in question was TCP/IP. It had already been standardized from an interoperability perspective with RFC 793, so what Berkeley sockets provided was a widely implemented interface for interacting with TCP/IP to send messages (at least, it was eventually widely implemented). Sun was not in a similar position to do this with JMS because that would’ve required them to get all of the MOM vendors on the planet to agree on a particular message exchange transport protocol. While that wouldn’t necessarily be a bad thing (and make our problem evaporate nicely), it was something that wasn’t likely to happen.
CORBA is also relevant here because it took the position of defining its own, end-to-end world: an interoperability-focused transport protocol in IIOP, interoperable operation interface specification in IDL and interoperable programming API specifications in the various language bindings. In all, it covered a lot of ground and was quite ambitious. However, I think the people behind CORBA knew that they wouldn’t really have portable distributed objects without specifying all of these things. At the time, none of these existed on which to build, so while what Berkeley sockets did was put a popular, simple API on top of TCP/IP, CORBA drew a box around a problem and said: “This is the box. These are the toys. Now, go have fun.”
Unfortunately, it didn’t work as well as was hoped. While it could be argued that by using XML to specify the semantics of reliable message delivery in the WS-R* specifications provides interoperability, there are very few programmers who are going to generate XML directly—especially with all the composable layers and namespaces required by those specifications. Therefore, the first thing people do with some complex, error-prone task is to automate it—by putting an API around it. This is exactly what the 3 examples above demonstrated, but by only specifying the wire and “close to the wire” aspects, it invites vendors to fill in the gaps. And, being motivated to make money so they can survive, these gaps will be filled with proprietary APIs.
The problem with proprietary APIs is slightly different than it was in the past. With today’s market volatility, the number of mergers and acquisitions in the software industry is somewhat alarming. While your vendor may provide an implementation of feature X today, tomorrow they may have bought or be OEM-ing that feature from someone else—with a different API. You’re caught in the middle. If you don’t change your application, your underlying tools won’t be supported for long (unless they have a truly massive installed base—ask IBM about how many Informix database servers are still out there). If you change your application, it costs you money: both from the additional licensing costs (the “upgrade” is unlikely to be free) and for the time it takes to change the code and completely regression test it against the new tools. Also, it probably introduced new bugs because it is a natural impulse to “enhance” software when you’re porting it, causing more time and money to be spent.
I believe there is a very real danger with all of the hype and speed of adoption of Web services that little thought is being given to the longer-term (even medium-term) compatibility and interoperability of the solution implementations. The motivations of the WS-* camp is similar, but opposite to Sun with JMS. Given the trouble they are having in agreeing core specifications, it is extremely unlikely that the vendors will settle on a common API to implement these specifications. They wouldn’t want to actually be seen as cooperating with the competition too much, or there wouldn’t be enough potential ROI in custom tools to offset the costs of hammering out the interoperability standards in the first place.
Unfortunately, it is the adopters who suffer the consequences. Regardless of what people actually think about WSA and the other specifications, it’s trying to do something good; so was CORBA. The problem is that if enough people blindly follow the easy path to “speedy deployment” of Web services through a reliance on incompatible vendor tools (not the underlying specifications), Web services risks some of the same bad press that CORBA received when people eventually wanted to migrate from one implementation or vendor to another. The lesson here is that the parts that were specified and agreed actually worked and were interoperable, however the parts that were left to “implementation details” are always the parts of a system that end up causing pain and cost to the customer.
I believe that the JCA approach taken by Fujitsu and company in implementing the RM4GS is probably about the best we can hope for in the Java world for achieving both compatibility and interoperability for reliable messaging. Wrapping these specifications in a standardized JCA interface is not overly complicated (see the JavaWorld article Connect the enterprise with the JCA, Part 2). Now that I’ve seen the RM4GS implementation, I’m somewhat surprised no one else has thought to put these two things together yet. Even an article on IBM’s Developer Works, Choosing among JCA, JMS and Web services for EAI fails to consider the power of putting JCA and Web services together. As customers responsible for building systems based on these technologies, we should insist that it is not only desirable, but also possible, to provide portable interoperability when implementing or interacting with Web services. They won’t do it unless we ask for it, but I suppose there aren’t really many people who are looking at Web services for asynchronous messaging yet. Most people see it as a way to invoke remote objects over HTTP, rather than as a way to embrace asynchronous messaging on the scale of the Internet, so that hurdle needs to be jumped first.
Remember, as Yoda says: “Once you start down the dark path, forever will it dominate your destiny.” Don’t let your vendors steer you onto that path. Demand portable interoperability in your software solutions. If you don’t invest in it today, you’ll end up paying for it tomorrow.
Permalink
Posted in SOA by AST on Sunday, August 14th, 2005
I started this originally as a comment to Dave Orchard’s blog entry about leaks in protocol abstractions, but it got kinda long. So, I moved it here as a proper entry.
As I don’t claim to be an expert on WSDL (1.1 or 2.0), I’m slightly confused by the fact that you can apparently declare an operation style in WSDL 2.0. I haven’t looked at the specification since around December, but in some of the work that I’m doing at the moment, I ended up bumping into it trying to determine if the message exchange patterns specified by the JBI specification implicitly assumed synchronous or asynchronous behavior.
Somehow in all of this and a few google searches later, I discovered David’s blog entry. I’m wondering if the specification of the optional operation type at the interface level isn’t another example of this type of protocol leaking.
The reason that I say this is that my understanding of the old 1.1 portType/operation was to specify an abstract description of the messages supported by the interface. The 1.1 specification seems to provide a derived or implied MEP based on the child elements (input/output/fault), however it stresses that the mechanisms for obtaining those messages were left to the binding:
Note that a request-response operation is an abstract notion; a particular binding must be consulted to determine how the messages are actually sent: within a single communication (such as HTTP request/response), or as two independent communications (such as two HTTP requests) [WSDL 1.1].
I realize this is only tangentially related to his post (and that David and the rest of the working group have probably thought about this quite a bit), but I don’t understand why it is important to restrict the interface operation semantics globally. From a service interface perspective, isn’t it equally valid to use asynchronous messaging or synchronous RPC (potentially performed over two asynchronous calls)? You would get the same messages for input/output and fault in both cases. You may actually want to specify different QoS at the binding layer to make your application code simpler if you’re consuming the service by connecting to a different endpoint (for example, using a request/response HTTP call).
With what we have at the moment, the service and the consumer just need to know what messages they’re going to get, but that communication can happen over asynchronous or synchronous channels. I was planning on taking advantage of the fact that the abstract operation didn’t care about the calling semantics of obtaining the messages and let that part of the picture be filled in by the binding (I’m not using SOAP or regular HTTP). The way it was, I was going to have to come up with some clever custom binding stuff anyway, but the 2.0 spec may blow that plan out of the water because I’ll need to at least support 1.1 for a while in addition to planning for supporting 2.0 when it is final.
Any clarifications/rationale would be most welcome from David or anyone else. I’m sure I’m missing something obvious here…
Permalink
Posted in SOA by AST on Sunday, March 6th, 2005
In my inbox tonight was a link to this post on TheServerSide.COM: Opinion: SOAP is comatose, long live REST. Since October, I’ve been involved in a pretty good sized SOA project which is not based around SOAP, but isn’t quite based around REST either (although the original design was heavily influenced by someone who is certainly an advocate of REST).
REST vs. SOAP is currently enjoying a similar polarization to vi vs. Emacs. There is a large body of people who believe in Keep It Simple, Stupid (KISS) principles for Web Services, and most of them are in the REST camp. A lot of these same people seem to see the SOAP camp as being a conspiratorial movement by big software vendors like IBM and Microsoft. Personally, I believe the reality is somewhere in between.
The thing that I can’t quite understand (after much reading on the WS-* and REST philosophies) is: what the hell difference does it make?
Both of them are tools. To me, it is up to us architects to figure out which tool best suits the job at hand. Lots of places seem to offer both. This would seem to indicate that you can, in fact, accomplish the same things with a degenerate application of SOAP that is based on document-based message exchanges.
Where it really gets interesting is how hard do you want it to be to integrate with an application or system. Of course, this is where you can get locked in to proprietary vendor implementations, but, at least, if the vendors are working together, there’s a chance that more than one vendor will be able to talk to each other. If you “roll your own”, you’ve got to not only provide a reliable implementation of said custom mechanism, but you’ve also got to 1) tell people how to use it and 2) support it when it breaks.
I agree with one commenter on one of the blog comments (can’t remember which one now, sorry) who said something along the lines of: the KISS principles of REST work great when talking to Amazon or Yahoo, but if that’s not what you want to do (or, more importantly, need to do), then you have a bit of a problem. From my own personal experience, most of these problems arise from the security requirements of SOA architectures. Most people are totally behind the asynchronous messaging capabilities of such a system, but they want the reliability and security they’ve come to expect from being exposed to years of deployments of MQSeries and other proven messaging systems. When they ask about security, they say things like, “We had that years ago. Ok, sure, it didn’t talk to everything, but it worked.”
Before everyone gets too upset and thinks that I’m on the side of the vendors, I’m not. I would much prefer to have an open source implementation of some tool than one that I couldn’t fix when it broke, however, in the “real world” where a lot of these systems get deployed, that isn’t an option. You don’t have that many people capable (or willing) to bet their career on an open source solution. Fortunately, there are more than there used to be, but the people behind a lot of big projects want support contracts, not support teams. This is just a fact of life.
Where the REST vs. SOAP debate gets really interesting is after you get the message. Now what?
While I agree that the WS-* specifications are vast, sometimes confusing and not widely implemented, I’d rather try and find one that worked than try and write one myself. If I write it, I’m not working on the problem I’m trying to solve (believe me, this has been one of my great historical problems). Again, specifically with regards to security, SSL only goes so far—even using two-way client authentication. If you want to encrypt part of the data in your XML message so that it can be audited, do you come up with something yourself, or do you try and find a solution that other people have already proven works? In my mind, this is where the complexity comes from. The only thing is that the W3C and others have just tried to standardize some of it. That makes it public. That also makes it a target.
Is SOAP+WSDL+UDDI+WS-* overkill for looking up a book on Amazon? Of course it is, but that’s only one context. In another context, you either rely on parts of the infrastructure to solve your problems for you, or you have to push all that complexity on the client. In a lot of the cases for my project, the client doesn’t want that complexity because they don’t have the capability to handle it themselves. They have to pay a Systems Integrator (SI) to sort all that out for them. That’s cost, which limits their willingness to “join the club”. While I really don’t like the “point-n-click your way to Hell” tools, they’re a reality we have to deal with (and no, I’m not going off on that tangent in the middle of this post).
Personally, I don’t think the client application should necessarily have to care about message correlation, security, or reliability. We have to build the system so that it’s easy to join in, but not just on the surface. You have to make it possible to automate that processing without a lot of burnt sawdust. This is what makes the ESB approach to SOA such a draw for me. Looking at it from the client’s point of view, if I have to pay $30K for a software license so that it’s easy for me to “join the club” vs. several million to an SI, I know what I’d choose. It’s up to us architects to keep the dependencies on that $30K piece of software to a minimum through “integrating at the edge” and other, sensible design idioms and patterns.
The bottom line is that you pay either way. If you’re providing the SOA that pushes the complexity on the client, they pay the money, but you loose customers because doing something once they’re connected is so damn hard. If you provide a sophisticated infrastructure, you pay up-front to put it in place, but you also take away a lot of the pain involved with sending the messages from point A to point B. Which is more important to your customer?
System’s design is about putting the complexity in the right place. Simple systems can do very powerful things, but, sometimes, you need complex systems to solve simple problems. Our job is to know the difference, identify the right solution and do the best for our customers, because, if we don’t have customers, we don’t have jobs.
Permalink
· Next entries »
