Yossi Dahan [BizTalk]


Tuesday, June 26, 2007

processing xml in two phases in xsl

Over the last few months I've been involved in the development of quite a few xsl scripts; one requirement that kept coming up and, as I'm not an xsl expert I always thought is impossible, was to perform what I would describe as - two phase parsing -where we run one bit of xsl to create an interim xml only to run another bit of xsl on it to get the results we want.

Here's an example of this requirement, I hope I can describe it in a way that makes sense:

Imagine you have two xml messages that are linked - one has a list of items, and the other their prices, something like this:

<?xml version="1.0" encoding="utf-8"?>
<ns0:Root xmlns:ns0="http://schemas.microsoft.com/BizTalk/2003/aggschema">
<item type="book" name="book1" barcode="12"/>
<item type="book" name="book2" barcode="34"/>
<item type="cd" name="cd1" barcode="56"/>
<item type="cd" name="cd2" barcode="78"/>
<price price="10.5">
<price price="24.70">
<price price="56.20">
<price price="90.14">

(This might not make much sense as an example, but trust me - it represents a real world scenario that does, also - note the structure we've used to get both messages is the one BizTalk uses to get multiple message parts into a map)

Now - what we want to get in the end is this:


As far as I can tell, getting from #1 to #2 in one go is not possible (but I'd love to hear otherwise), so our only conclusion was that we need to go through two stages in processing - in the first one we would de-normalise the two messages to one flattened xml, and in the second we will get the distinct types and sums.

So, to tackle the first stage we wrote a simple xsl that creates the interm xml we wanted - the output looks like this:

<item type="book" total="10.5" />
<item type="book" total="24.70" />
<item type="cd" total="56.20" />
<item type="cd" total="90.14" />

than, we put this xsl script inside a variable declaration; our xsl now looks like this:

<xsl:template match="/">
<xsl:variable name="items">
<xsl:for-each select="/*[local-name()='Root' and namespace-uri()='http://schemas.microsoft.com/BizTalk/2003/aggschema']/*[local-name()='InputMessagePart_0' and namespace-uri()='']/*[local-name()='items' and namespace-uri()='']/*[local-name()='item' and namespace-uri()='']">
<xsl:element name="item">
<xsl:attribute name="type">
<xsl:value-of select="@type"/>
<xsl:variable name="barcode" select="@barcode"/>
<xsl:attribute name="total">
<xsl:value-of select="/*[local-name()='Root' and namespace-uri()='http://schemas.microsoft.com/BizTalk/2003/aggschema']/*[local-name()='InputMessagePart_1' and namespace-uri()='']/*[local-name()='prices' and namespace-uri()='']/*[local-name()='price' and namespace-uri()='' and child::*/child::*=$barcode]/@price"/>

(sorry for the long xpaths, we've had to make them "BizTalk friendly"...)

That gives us the interm xml we need to work on and is one of two "magic" bits to get this working - we did not realise up until now that we could put whole chunk of xml into a variable, and that we could use xsl to create that xml in a variable.

Having the xml in a variable, we hoped, would allow us to run another set of xsl on it before outputing it as the script's result. but there was one additional maginc point missing - as the variable was not retrieved in the normal <variable name="myVar" select="someXPath"/> it's contents is not considered a node-set but just a literal that looks like a node-set and as such we could not use this variable in further xpaths in the script.

Thankfully, Microsoft has provided a function to convert one to the other, so we had the next line of xsl to our script:

<xsl:variable name="itemsNodeSet" select="msxsl:node-set($items)"/>

we had to add the namespace declaratino for msxsl in the stylesheeet declaraion -


Now that we have the interm xml in a variable, and is considered a node-set we could simply run the last bit of xsl we need to get the totals:

(the for each uses another nice technique we use in xsl to get a distinct list of items in a list)

<xsl:for-each select="$itemsNodeSet/*[not(@type=preceding-sibling::*/@type)]">
<xsl:value-of select="@type"/>
<xsl:variable name="type" select="@type"/>
<xsl:value-of select="sum($itemsNodeSet/item[@type=$type]/@total)"/>

and voila! - the output is exactly what we wanted!

Labels: ,

Monday, June 18, 2007

When is a subscription created for a correlation set passed as a parameter?

Ben Cops has asked a great question in a newsgroup last week in a thread I was involved with -

If you initialise a correlation set and then pass it as a parameter to an
async started orchestration where it is followed, when does the subscription
come into scope? When the correlation is initialised, or when the
orchestration containing the "follow" starts?

Initially, my instinct told me the subscription will be created when the correlation set is initialised; my instinct was wrong. (only that now that I know the answer I find it difficult to trace back my initial line of thought)

I think it was generally around the fact that I was confused to think (let's say - due to a momentary lapse of reason) that the subscription will be created as soon as the correlation set is initialised, and that received messages will get queued to be processed by the yet to be started orchestration.

The fact I've missed at that point, as Ben has kindly pointed out, is that the instance subscription that gets created is built of the correlation set properties and values (no problem there), but also with the message type defined in the receive shape following the correlation.

In this case, as the orchestration that will follow the initialised correlation set has not been loaded yet, BizTalk does not know what message type will be used, not to mention it cannot possibly know any details about the instance that is supposed to receive such message when it arrives, and so the subscription cannot be created at this point.

Only when the started orchestration gets called, the recevied message type is known and the subscription gets created (even before the receive shape is execute).

To prove this I've created a small sample (and before you ask - no - I did not upload it) built around two small orchestrations:

orchestration #1 starts with a received message (to trigger it), it then sends it out while initalising a correlation set, and then carries on to start orchestration #2 using a start shape passing the intialised correlation set as a parameter.

orchestration #2 has an expression shape followed by a receive shape configured to follow the correlation set received as an orchestration parameter.

I've put breakpoints in orchestration #1 on the send shape and start orchestration shape and in orchestration #2 on the expression shape.

I've executed the process and at the first breakpoint (the initalising send shape), I checked in the admin console to see that, as expected, the subscription was not (yet) created (as the correlation set was not yet initialised).

At the second breakpoint (the start orchestration shape) the correlation set was already initialised, but I still could not see the subscription in the admin console.

Only after the start shape executed when I was parked in the expression shape of orchestration #2 (before the receive shape), I could see the subscription in the query.

So - all I can say is - apologies for misleading you Ben and thanks for a great question.


Monday, June 04, 2007

Unexpected use of keyword 'request'

We've had to consume a web service today with a method that took one paremeter called "request"; We've added the reference to the WS, created a message of the type we've needed; created a map to create an instance of it and assigned it to the web service as such -

wsRequest.request = requestMessage;

(where requestMessage is the result of a map we're usign to build the request message)

The orchestration designer highlighted the word "request" with the error: "Unexpected use of the keyword 'request'" (or something similiar, this is off the top of my head)

Apparently, and not very surprisingly one might say, the word 'request' cannot be used as a part name; in fact - if you try to create a multi-part message and set the name of one of the parts to the word 'request' you get a nice messagebox explaining this cannot be done; it is only because in our case the multi-part message was created automatically when we added the web reference that we ended up with such a part name to begin with.

This brings up the question - can't the designer be smart enough to warn us at the time of adding the web refernce? as we clearly can't use a web service with a parameter of that name a warning would have been nice (but not an error, as we might need to use other methods in that web service which are ok)

In our case, as the web service was developed in ternally, we could (relatively) easily change the name of the parameter to something more accurate, had this not be the case....