Yossi Dahan [BizTalk]

Google
 

Tuesday, June 21, 2011

BizTalk and the MOS protocol

In a meeting a few weeks ago the question of how to support the MOS protocol with BizTalk came up.

The MOS protocol, used in the media industry, has two flavours -

  • Versions 3.x are implemented as ‘proper’ web services
  • Versions 2.x are implemented as ‘xml over TCP’

As the former is a no-brainer for BizTalk, I wanted to look at what it would take to support the later-

The protocol (as pretty much any protocol) defines 4 elements –

  1. Message Format
  2. Message exchange patterns
  3. Profiles/Capabilities
  4. Communication protocol

Message Format

The message format for MOS is XML, and the schema is available from the MOS protocol web site.
I have downloaded the schema and successfully used it from BizTalk with no issues – so creating and parsing MOS messages is pretty much effortless.

One observation I have made, though, is that the authors decided to define one schema with a <mos> root element, under which one can use one of a number of supported elements; this means that in essence there’s only one MOS message, and it is the child element that defines what the request or response actually are, which in turn means a little bit more effort is required from participants, such as BizTalk server, to identify received messages.

In BizTalk that would probably mean a custom disassembler to wrap the out-of-the-box Xml Disassembler and overwrite the message type with the correct one (based on the child element rather than root element);this is a fairly simple component to write and so supporting the MOS format is pretty simple.

Message Exchange Patterns

My understanding of the MOS protocol is that the message exchange pattern is a quite simple request-reply pattern over the same connection (which, I believe, gets terminated after each request), and as such is very easy to support with BizTalk.

The only complicating matter here is that a request needs to include a contiguously incremented message id.
This can be be done using ordered delivery and/or a convoy pattern with a component, either in an orchestration or a pipeline component, back by a database that would inject the correct message id to each request, so technically this should not pose a problem and is not difficult to implement (certainly not more difficult than it would be in any other MOS client or server implementation).

The problem I have with that is more one of a principal – requiring ordered delivery, which is the implication of such a requirement, really affects the throughput that can be achieved as requests have to be serialised, and I wonder whether there’s a good enough justification for that in this case.

Rant over, this is the protocol requirement and the good news is that it shouldn’t be difficult to achieve with BizTalk.

Profiles/Capabilities

The MOS protocol defines sets of messages that need to be supported for each device profile, but that doesn’t affect the core design of the solution on top of BizTalk but rather which messages should be expected and obviously coded against.

If the handling of the messages is to be done in BizTalk then typically I would expect an orchestration for each message type in the supported profiles; if BizTalk only relays the messages, than ports with schemas transformations might be all that’s needed.

Communication Protocol

The communication protocol is a simple TCP/IP socket, with the expectations that a server would listen on a couple of ports and, presumably, be able to handle requests from multiple clients, but only one at a time on a single connection, with the conversation ending with every response.

This really is an over simplified approach, but that also mean that it is not challenging to implement and, whilst there is no TCP/IP adapter for BizTalk out of the box, there are several approaches one can take -

Community BizTalk TCP/IP Adapter

The Open source adapter on codeplex can receive and send messages to and from BizTalk over TCP sockets and supports one-way, request-response and even duplex communications, with extensive configurable options.

The adapter was even designed to support ordered delivery which may come in very handy if the message id requirement described above is to be adhered to.

The adapter was initially developed for BizTalk 2006 and then adapted for BizTalk 2009; to make it work with BizTalk 2010 I downloaded the source code and compiled it locally (I had to remove the post build events that registered the assemblies in the GAC, and so I had to register the assemblies in the GAC myself later on when updating the adapter’s code.

I also had to update the check in the installer that was introduced to confirm that the correct version of BizTalk is installed in the setup project -

image

Once that was done I was able to compile and run the installer, which allowed me to add the adapter in the BizTalk Admin Console -

image

I then followed the instructions on testing the adapter in the user guide supplied to verify it is working as expected, which it did, and so – using the TCP adapter with BizTalk 2010 is pretty straight forward.

There is one issue with the adapter with respect to the MOS protocol, though, and that is that the adapter has been designed to expect, quite rightly, some framing around the messages exchanged indicating the beginning and end of each message, but the MOS protocol does not, it simply assumes that over a single connection there will be one request and up-to one response.

Trying to configure a receive location without framing characters reported an error when I tried to enable it – “The Messaging Engine failed to add a receive location "Receive Location1" with URL “…." to the adapter "TCP". Reason: "20: The property '/Config/frameStartDelimiter' is missing from the configuration.".”

Looking to solve this I found I need to make a few changes - the adapter provides an XSD schema with it’s configuration options, used by the admin console to ‘render’ the settings pages, and this schema needed to be updated to indicate the framing elements are optional.
Following from that there were a few places in the code that referred to those values where support for the potential of a null value needed to be added to avoid an ‘object reference not set to an instance of an object’ error, which is fair enough.

The biggest effort had to be made in the piece of code that actually reads the message from the buffer as this loops through the bytes received and looking for the framing bytes, but a ‘quick and dirty’ version of these changes took a few cycles of code and test and within a couple of hours I’ve had a working adapter with support for messages with no framing which was able to support sending and receiving MOS messages over TCP sockets.

Custom WCF Channel

As an alternative to using a full blown BizTalk adapter, and needing to understand the adapter framework APIs, one could write a custom WCF channel to support the simple TCP socket communication required by MOS.

This, I suspect, is not the typical code one writes (is there such a thing), but someone who’s comfortable with socket programming and familiar with the WCF programming model shouldn’t find it too difficult and – the good thing is – there’s no BizTalk knowledge required and the custom channel can be used from any .net application and not just BizTalk.

In my view – if the TCP adapter did not exist already, this would probably have been the best way forward as it requires less investment, less BizTalk specific knowledge, and a solution that is usable both within and outside BizTalk.

However, as the adapter does exist, and could do the job with minimal effort in adaptation, writing a custom channel is probably not necessary.

Custom LOB adapter SDK implementation

In between the two options mentioned above exist the ability to develop a custom adapter based on the LOB adapter SDK,
Developing such an adapter does require some knowledge of the framework and BizTalk, but this is significantly less than it would have been required to build a full blown adapter from scratch; in addition, such an adapter is re-usable from any .net client, and the programming is focused a lot more on the WCF programming model than on anything BizTalk specific.

The main benefit an adapter provides, in my view, on top of a custom WCF channel,  when the communication is fairly simple, is the developer experience – the browsing of the ‘target system’, the generation of code artefacts, etc. and when there’s a lot to provide through the adapter this can be easily worth while.

However – in the case of the MOS protocol - unless one plans to encapsulate a lot of the protocol knowledge in the adapter, which I don’t think makes sense in this context, I don’t think there’s a strong enough justification for developing a customer adapter, as there’s nothing really to chose from or generate in Visual Studio.

Taking all of the above into consideration I would say that in order to support the basic TCP socket communication required for the MOS protocol I would customise and use the community TCP adapter, but as an alternative, if that is not desired, I would develop a custom WCF channel.

Conclusion

The MOS protocol is quite a simple protocol to implement (in fact – it might just be a bit too simple) and doing so on top of BizTalk, as expected, is not difficult.

Some investment in adopting the TCP adapter to suite this protocol is required, but this would not be significant, and could be done generically, Everything else is pretty much a day in a BizTalk developer’s life..

Smile

Labels:

1 Comments:

  • Hi,

    i have a problem with my custom WCF LOB adapter. i want to share the TCP connection between receive and sent ports as i only have one connection to TCP server.

    i receive message from TCP server then convert it and publish it to message box then the send port must use the same TCP connection to send reply back to server.

    how can i do this?

    thanks,

    By Anonymous Anonymous, at 10/12/2012, 14:41  

Post a Comment

<< Home