Another dive into the SOAP adapter's behaviour
Being keen to fine tune our solution's performance we turned to look into the message exchanges between the SOAP adapter and one of the web services we're usign quite extensively.
Two things became obvious very quickly (well, to be fair - they may have not been without some very useful tips from our colleagues working for the 3rd party providing that service) -
I'm no network expert; I hold only the basic understanding of HTTP you'd expect from someone dealing with messaging and integration; the way I understand it then is that if a requester sets the "Expect:100-continue" HTTP header on a request, the client would sends the HTTP headers only to the server and then wait to receive a response with "100-continue" status.
It is only when the "100-continue" status is returned from the server that the client sends the actual request.
I also understand that, in order to support older implementations of HTTP (or those that do not support 100-continue), if the client does not receive 100-continue it will send the request anyway, but only after the delay introduced while waiting for the server to respond.
What all that means is about 200ms delay to every message sent from BizTalk.
I'm sure Microsoft have given a lot of consideration to this before deciding this would be the default behaviour of web services in .net but to us, communicating with a long-term partner over a good leased line, it would have been nice to be able to turn it off and skip the extra step.
Unfortunately, although spending some time on this we could not, as of now, find a way to do so.
Our best bet was (and still is) to set the relevant setting in the ServicePointManager in our custom web service proxy, but this does not seem to have the desired effect.
From a bit of reading we've done it seems there's some element of re-use here and that the ServicePointManager may use an existing service point, in which case our newly set property will not be used.
It could be that this is created somewhere in the adapter before our custom proxy or something like that, we truly don't know at this point.
Fortunately this is an area we’ve had better success in, and it's good because it has a significantly bigger impact on performance than the 100-continue problem.
The web service I've mentioned before uses basic authentication on the wire and then further elements of security in the message, so on the BizTalk port we've set the credentials the SOAP adapter should use when calling this web service, and, as you'd expect, this is working just fine.
However, analysing the traffic going out of BizTalk on the wire we've noticed that for each message we send to the 3rd party, the SOAP adapter would transmit the message out without the security HTTP headers first, receive an HTTP 401 (Unauthorised) error and only then send the request with the HTTP headers baring the credentials we've set in the send port.
Obviously, and especially when relatively large messages are involved, this has a significant impact on the performance of the server (and the farm as a whole as more bandwidth is used for the message exchange).
Again, doing some reading and experimenting, we've learnt that this is pretty much down to the choices made around the DEFAULT behaviour of .net, and again, we thought, we had no control over it.
But, while we could not do anything if we were using the standard configuration of the SOAP adapter, the fact that were using a custom web proxy to call the web service (for other reasons) meant we could try and tweak the way messages were transmitted, and tweaking we did.
It didn't help us in the 100-continue case, but it definitely helped with the pre-authentication requirement - all we had to do is add a line setting the PreAuthenticate property of the HttpWebRequest to true before using it and BizTalk (well, the .net framework web service stack it uses) would quite happily provide the credentials up front avoiding the need to duplicate the round trip to the 3rd party server.
Much better now!