Yossi Dahan [BizTalk]


Thursday, December 27, 2007

The message box as a service boundary

For the last 18 months or so I've been working on a very exciting, and quite large, BizTalk implementation here in the UK, I'll leave the full details of it for now, but I can tell you that it involves all the nice buzzwords we keep hearing about SOA, SaaS, S+S, ESB at least to some degree (and with various level of quality, if we're honest)

Anyway, as you can imagine we're using web services quite extensively - we expose a lot of them, and consume even more; some are internal to the company (but cross teams, although not so much platforms) and many are external (which do cross platform as well)

The reasons to use service oriented architecture should be very clear to everyone by now, as are the famous four tenants of SOA.

In out implementation we've abstracted the calls to all the internal web services through utility orchestrations which would take a message in our canonical format , convert it to the service's format, call the web service and transform the response to the canonical format before returning it to the calling process; this way we can re-use those transformations, and have a central place to deal with each request, apply error handling, etc.

From the parent process we then use call orchestration to initiate these utility orchestrations passing in the request cannonical message and receiving the response canonical message as an out param, which is quite efficient (when initiating orchestrations through the call orchestration shape the request does not go through the message box)

As we're doing the transformations in these utility processes, we consider them to be in the boundary of our process, and not, obviously, within the boundary of the called service, for this reason we call the web service from the process rather than the actual assembly behind the web service.

When we, within the utility process call the web service, what actually happens is that the request message (now in the WS' format) gets published to the message box, being picked up by the send port which would pass it to the SOAP adpater which, in turn will serialises it and transmits it over the wire to the service; the service then deseralises the message on the other end before executing whatever code needs to be executed and the entire process now repeats in the opposite direction.

In this case the service boundary is the web service endpoint.

A few weeks ago I had what I thought was a brilliant idea - why not treat the message box as the service boundary!?

If I had a process that takes in the service's format of the message using a directly bound receive shpae and a filter, execute the code internally (as we're now inside the service boundary we can use the service code directly from expression shape, no need to go through a web service) and when finished publish the response back to the message box (in it's own format), I could have simply published a request message for that service, and get the response published back for me; correlation should be used, but this can be handled using self-correlating ports or a correlation set.

The client process would do pretty much the same - it would use a utility process to transform the canonical format to the service's format and publish the request. it would then use correlation to receive the response and transform it back to the canonical format before retunrning it to the calling process(synchronously).

What would we save? - following this approach for at least some of our internal services can save us the need to serialise the messages over the network; in the web service case we have to go through the message box from the process to the send port anyway, so going through the message box from one process to aonther would not make a difference, but all the network traffic and the work by the SOAP adapter (which is far from being efficient) can be saved.

This was a good idea (I thought anyway), but I suspect it won't work, as it has two main flaws (and I will be extremely happy to get some ideas around those) -

Firsly - both subsystems will need to exist on the same BizTalk group so that they share the same message box and so we could use pub/sub to exchange messages between them (on it's own this is not necessarily a problem, but it is the main cause for the next one, which is the big one)

Secondly - the schemas will have to be shared -

When you're adding a web reference to a web service from a standard .net project a proxy gets generated for you; that proxy will include a local version of all the classes used by the web service (these will be in YOUR code namespace rather then ther service's but will serialize to the same XML).

Equally - when you add a web reference in a BizTalk project, you get schemas generated so you can create messages to send and receive to/from the web service; these will be in the service's XML namespace as they have to represent the XML supported by it, and here lies the problem.

If both the service implementation and the client implementation are on the same BizTalk group, the schemas will have to be shared as there's no way to deploy two schemas using the same root node and namespace and we all know that sharing schemas is a bad idea as it strongly couples the implementation together and that pretty much renders the idea useless (this, confusingly I suspect, means we're sharing a class and not a contract).

Of course one could play around with the idea of having two BizTalk groups and communicating between them, and although you can choose better transports than SOAP for that internal communication I suspects that brings us closer to simply calling the web service and so I'd rather stay with that standrad approach.

Labels: , ,


Post a Comment

<< Home