Yossi Dahan [BizTalk]

Google
 

Sunday, July 24, 2005

Merging two messages in BizTalk 2004

Ok, so here goes the first "serious" posting (and turned out to be a bit long...sorry...)

Yesterday afternoon I was facing a challenge I was surprised I did not stumble into earlier-

We needed to merge two messages, each containing a different subset of the information we need, into a single message. Both messages have a key field we can use to match the entries.

consider the following two files:

<?xml version="1.0" encoding="utf-8" ?>

<ListA>

<Items>

<Item>

<code>1</code>

<description>one</description>

</Item>

<Item>

<code>2</code>

<description>two</description>

</Item>

</Items>

</ListA>


and

<?xml version="1.0" encoding="utf-8" ?>

<ListB>

<Items>

<Item>

<code>1</code>

<price>1.11</price>

</Item>

<Item>

<code>2</code>

<price>2.22</price>

</Item>

</Items>

</ListB>


(see if you can guess what then expected result is...)

Facing this challenge three ideas came to mind:

  • Using the mapper
  • Developing a custom component
  • Going through a database table

  • Using the mapper
    Of course my first bet was the mapper.
    Initially I was thinking this was a great opportunity to take the looping functoid to the extreme. So I set up a parallel convoy accepting both messages followed by a transform shape (yes - inside a construct, of course) having both messages as the input and the result as the output.
    I was thinking I'd drop the looping functoid on the map surface, add a logical functoid or two and set up something like "for each item in list A loop through the items in list B until you find...". The looping functoid, however, thought differently.
    Now I believe I was simply giving it much more credit than I should have (and someone please correct me if I'm wrong)

    Custom coding
    frustrated from the mapper I resorted to what we do best - write custom code. Replacing the transform with a message assignment calling a .net method passing both messages as parameters and expecting the result as the return value;
    I have to say developing the code to accepts two messages and merges them using .net, DOM and XPath is easy enough and quickly done. Running it with a couple of 6MB messages might be easy but nothing even close to quick.
    But at least that's done. So now....Back to the map option.

    Using the mapper with custom xslt
    I've decided to get rid of the looping functoid and add custom xslt script. First I traditionally mapped each element from of the first part of the source (list A) to the output. Then I added a scripting functoid to the palette and set it to "Inline XSLT Call Template".
    the script itself is quite simple - receiving the value of the code element from the current item node in list A and selecting the price value from the corresponding item in list B. Had to play a bit with the XPath due to the different namespaces, but got it nicely done.

    While at it I've also gathered a full custom xslt to achieve the same result, used it instead of the map altogether.
    These two approaches more or less covered the first option.

    I did not yet test any of them with large files, this will have to wait for Monday, but I sure hope to get somewhat better results then with the custom code.

    The database option
    now - I have to admit the database option is the one I like less. Seems like too much work and hassle for something that simple. Plus - there isn't really a good way to read and write large amount of data to a database from BizTalk. The idea, however is quite simple - write all the records from message A to a database table, update records with data from list B (with just a little bit of effort the order of the messages should not be important) and then read the combined record.
    very simple really, and even avoids the need for an orchestration and convoys (required for both the previous options) if I could find a way to safely figure out when it's safe to read the data which shouldn't be a problem.
    yet it seems too much, so I would leave it for a last resort.

    If anyone has other ideas on how this could be achieved I would love to hear them, as I've said Monday is performance test day and then we'll see.

    25th Nov. 2005 - Note - I've just realised I never posted how I eventually approached this, so I did. you can real all about it here

    0 Comments:

    Post a Comment

    << Home