Yossi Dahan [BizTalk]

Google
 

Sunday, June 04, 2006

Sync to Async conversion

The scenario explained

There's a scenario I bumped into for the second time last week and it always poses some dillema - that is a scenario in which you want to expose a synchronous interface to the external world but do the work asynchronously in the backend.

Here are two examples of when such need can be useful -

1. Pure messaging - you are exposing a synchronous web service, when a request comes in you need to forward it to another service, possibly of a third party, and return the response, maybe do a couple of transformations in the middle.
That other service is implemented for asynchronous communication.

2. Orchestraction - you are exposing a synchronus web service in the front end because your client cannot implement a call back for async.
As a request comes in a single process needs to be executed on it, but the decision which one is made at runtime and not design time (according to data in the message)
This means you cannot simply bind the orchestration two-way port to the two way SOAP receive port.

While the two scenarios are different, they share the same problem - how to route the message coming back (from the third party async callback or from the dynamically selected orcehstartion) - to the instance of the receive port that accepted the initial request.

The problem

When BizTalk publishes a message from a two way receive port into the message box it creates an instance subscription, so when the response message is published back to the message box it will get routed back to that instance of the receive port to be returned to the original sender.
This subscribtion is based on a correlation token which is generated and written to the EpmRRCorrelationToken system context property (as well as a "true" value in the RouteDirectToToTP)

In a classic scenario where a two way send port subscribesto a two way receive port, when the send port publishes the response tothe message box BizTalk will ensure that the EpmRRCorrelationToken has the correct value and is promoted.

Looking to implement the second scenario described above, I've created an orchestration that subscribes to the message box to pick up the request message using a directly bound two way port.

The process then does all that needs to be done and when it finished it published the message to the message box through the send side of the two way receive port it got the initial request from.

Following my understanding of the correlation logic described above I make sure to set and promote the values of both EpmRRCorrelationToken and RouteDirectToToTP (to be accurate the EpmRRCorrelationToken has already rather magically added to my newlycreated response message, but it was not promoted). I do this by initialising a correlation set with these values on the send shape.

To my surprise - the message published to the message box had the EpmRRCorrelationToken property in a not promoted state, regadless of my best efforts. It seems as if BizTalk is actually ensuring that it is not promoted

This, of course, causes a routing failure as the subscription cannot befilled without this property.

Error by design

It seems that Microsoft has decided to force such implementations to fail by design. Or so it was explained to me a while ago.

The reason being that (as far as I understand), since the orchestration in my scenario is not tightly bound to the two way receive port, it is possible, theoretically, that more the one subscribers (orchestrations or others) could get the message and return a response, in which case BizTalk will have no way of knowing which response should actually be returned to the sender.

Solution

So, taking into this into account, I need to create an orchestration that will be tightly bound to the receive port at design time.

This orchestration will then get every message from the receive port, and pretty much publish the message back to the message box using a "direct" send port, only that I will make sure I initialize a correlation set on the sand shape to help me get the response back into this instance of the orchestration.

This, of course, involves coming up with some correlation token, defining a correlation type that uses it, and then making sure it's in the message context before the message is sent.

The other orchestration (the one we really intended to execute, and selected dynamically based on the message context) will then kick in and pick up the message, and when it complets its work it will re-publish the message back to the message box ensuring the correlation token provided by the generic orchestration exists in the message and promoted to its context (this is most likely do be done, counter intuitively maybe, by initialising the correlation set on the send shape).

This will ensure the message will get routed back to the correct instance of the generic orchestration and through it back to the sender synchronously.

The main problem with this is that it creates the same sort of problem we had anyway. If more then one orchestration picks up the request a race condition is created. In this case, since it is a private implementation BizTalk will allow it and any responses after the first one will become zombies.

So - we got what we could have had anyway if Microsoft wouldn't decide to "protect" us from it, only that now we have another orchestration on the way which is of course a performance hit.

Personally I would prefer if BizTalk would behave the same - let us publish the response from whatever and return the first response coming in, but at least there is an easy way to get this to working and, from my experience, as long as you're careful with your subscription, it is working quite well.

7 Comments:

  • Hey,

    Iam trying to do something simialar.
    Do you happen to have some sample code ?
    It is very much appreciated.

    Thanks in advance,
    Dan ddia at hotmail dot com

    By Anonymous Dan, at 17/01/2007, 19:17  

  • Sorry Dan, I don't have anything I can post.

    I have gone over the post again and tried to make the solution desctiption a little bit clearer (hope I've managed to do so).

    If you have more specific questions you can write to my email and I'll try to answer as soon as I can, just write to yossi.dahan[at]sabratech[co][uk].

    By Blogger Yossi Dahan, at 17/01/2007, 22:41  

  • I'm trying do something similar, but I'm confused at how you prevent the original orchestration from picking up the message immediately with the correlation set?

    By Anonymous Alastair, at 01/06/2007, 11:10  

  • In our case, as I believe in many, the request and the response are of different message types and so this is not a problem.

    In the couple of cases where we needed to use the same schema we resorted to creating a multi-part type to indicate when it is a response and use that as the type of the received message.

    Hope this makes sense.

    Yossi

    By Blogger Yossi Dahan, at 01/06/2007, 12:00  

  • Hi Yossi. Do you know if the bahaviour was changed in R2? We appear to have run into this problem using R1, but I have just created some test code in R2, and everything works fine. I have a 2-way Receive Port (using the WCF adapter) which I published from schemas. I have a simple orchestration with a single two-way direct bound port. I subscribe to messages from the receive port using a filter on the activation Receive shape. Within the orchestration, I am mapping my request message to a different message type for the response. I am not copying any context from the request to the response. The response is happily returned to the calling test harness code without any problems at all.

    I can't find any suggestion of a hotfix for this problem, or any statement saying it is a new feature in R2, but I do wonder if MS has quietly changed the behaviour without telling us.

    By Blogger Charles Young, at 26/06/2008, 18:06  

  • Hi Charles - no I don't, but I have never tried so can't confirm or deny :-)

    I certainly did not hear/read anything about that changin.

    To be honest I would be very happy to find out that was the case, but also a little bit surprised as I remember very clearly the long discussions I've had with guys from MS on this at the time when I question that design decision...

    If anything that'll be all credits to them though! :-)

    By Blogger Yossi Dahan, at 26/06/2008, 22:24  

  • I need to research this properly to establish the truth. However, a few weeks ago, someone suggested to me that other rather similar restrictions (specifically the inability to control the MessageType property on messages sent from an orchestration) may have disappeared. I am therefore currently adopting a working hypothesis that Microsoft has changed the implementation of orchestration ports and removed these restrictions in R2. If this is true, it is very welcome news. I have been spitting blood for years over these unnecessary restrictions. They seriously limit the raw power of the subscription model in BizTalk, making it very hard in many scenarios to implement good design patterns. The argument you heard from MS is spurious, as your workaround demonstrates. If you direct bind, you always have to take greater responsibility for your message context and subscriptions. If my working hypothesis proves correct, it would have been nice for Microsoft to have told us about the changes.

    Incidentally, after further testing, I can confirm that the behaviour we are seeing in R1 is exactly as you describe.

    By Blogger Charles Young, at 27/06/2008, 10:51  

Post a Comment

<< Home