Yossi Dahan [BizTalk]


Tuesday, November 24, 2009

What not to do – projects

So – what not to do?

Here’s one – don’t take 15 different interfaces, from 15 different providers, and the canonical format, and your internal process and bundle it into the same set of projects.

My predecessor (MyP in short) in this project had followed the best practice and ensured (for the most part) that schemas, maps, pipelines and processes each have their own project.

This is good – to start with, in the old days, mixing them used to cause all sorts of build issues, but – although I haven’t tried in several years now – I assume that’s all in the past, but – more importantly – this is bad practice because different artifacts have different ‘resilience to change’ – if you need to add a small shape to a small process that gets instantiated as a result of a receive shape (and not being ‘started’ or ‘called’) you can usually un-deploy it fairly easily.

Change a schema, on the other hand, and usually there’s a whole raft of artifacts you now have to un-deploy with it.

For that reason – mixing two artifact types in the same assembly, whilst not technically problematic, usually suggests you will have maintenance nightmare in the near future (unless you don’t mind down time, and regression tests, that is).

Anyway, as I was saying, MyP did separate the different artifact types to different assemblies, but equally MyP only had one assembly of each; so – when we are processing a message arriving from partner A, for which we have a schema, a pipeline, a map or two and an orchestration – these were split across four assemblies; equally - when we are processing a message arriving from partner B, for which we also have a schema, a pipeline, a map or two and an orchestration, all different – these were also split across THE SAME four assemblies.

On the surface not much wrong with this, the problem is what happens when you want to change a small thing in, say the schema for partner B, used from within a map, which in turn gets used within an orchestration – you now have to un-deploy the orchestration assembly, so that you can un-deploy the maps assembly, so that you can un-deploy the schema assembly, so that you can redeploy the new version of the schema assembly followed by the rest. (ok – so I’m assuming versioning and side-by-side deployment are not being used – which I have to – for my story);

So – all that is the general pain that is part of the BizTalk developer’s day, why am I bringing this here? well because you’ve taken a BizTalk ‘challenge’ and doubled it - for one – you took process A down due to a change in process B; why? had they existed in separate assembly sets you wouldn’t need to… and two – as you’ve released new code – you have to test new code; only that now you have to test two sets of code, including one you haven’t intended to change (but may have, by mistake or otherwise); again – had there been two sets of assemblies, you could probably get away with testing just the scenarios related to the ones you’ve changed.

Now multiply this by about 15 partners, and you see how it can be quite wrong. I hope.

Labels: ,


  • Seriously, I think you are a bit whiny. YourP seems to made things like automated deploy/undeploy and testing extremely easy for you.

    You just have to do it. Grit you teeth and get on with it.

    By Anonymous Mikael Sand, at 25/11/2009, 07:22  

  • Hi Mikael

    Thank you for your comment, but I'm not sure I understand it.

    Me 'whining' is not about the fact that I have to work more to deploy the solution.

    MyP had already created a deploy script for the solution (which is good work), and I would have done the same.

    This is not about laziness, but about the impact of the need to deploy more than you have to.

    Deployment always mean some downtime, and some re-test; the one affects users and the other costs money (to the client) and so surely minimising what you re-deploy is a good thing?

    By Blogger Yossi Dahan, at 25/11/2009, 07:40  

  • I agree with you Yossi, really important splitting the artefacts into different assemblies in a meaningful way(keeping dependencies in order). Think about responsibilities e.g. "systems", "processes" or "services".

    By Blogger Unknown, at 25/11/2009, 08:33  

  • Well i don't mind if everything is in the same assembly as long as the dependencies are ok.

    Let's say we have a Common Data model.

    The CDM is in a seperate assembly for sure. (a schema is in it)

    But now we have to receive something from SQL, do some orchestration processing on it and finally map that to the CDM and Send it off to the messagebox. This Receiving project could possibly have:

    - pipelines
    - Schemas
    - Orchestrations
    - Ports
    - Bindings
    - Business Rules.

    I see no harm in putting those artefacts that exist solely to perform some logic to map an unsupported format to a CDM ( Known format) into one DLL.

    Cause if the schema changes a bit, the maps and possibly the orchestration will change as well... So then you only have to deploy this one dll.

    It all depends on getting the dependencies right....

    By Anonymous Anonymous, at 25/11/2009, 14:16  

  • I agree that schema, maps and ports could be placed in a Receive-assembly, thats what i mean with an application for "system". But does orchestrations really fit into the Recive-assembly? Often you try to design your orchestration in a loosely couple way, e.g. using MsgBox direct binding(no logical ports) for better reusability. Wouldn't be great to put these orchestrations in a separate assembly containing orchestrations for a specific process/use case/function?

    By Blogger Unknown, at 25/11/2009, 20:52  

  • I agree with the sentiment totally. What we do in our current gig is to maintain the concept of a "connector" per target system which is in turn comprised of specific schemas, maps (to and from the canonical) and orchestrations. Changes to any partner schema is localized to the connector and there's no trouble anywhere else. In my previous project, the backend systems /connectors were treated as full fledged Applications in their own right and only referenced a common "Platform" application. That model also worked very well.


    By Blogger Unknown, at 30/11/2009, 13:34  

  • @benji

    I think we have the same way of working....

    What i call a RIS (Receiveing integration service) is a connector for you.

    I also have a SIS ( Sending integration service) and probably that's a connector for you to0...

    Do you have On-Ramp / Off-Ramp Connectors ? or do you call them differently

    By Anonymous Patrick Wellink, at 30/11/2009, 15:07  

  • Hi Robert

    I'm generally with you on this, although I can see how sometimes you may have an orchestration that is only serving a 'receive function' and it may make sense to put it with other receive artifacts.

    I would rarely do that, though, and would always opt to have a separate assembly - after all - the cost is very low (assuming you already have a good deployment framework in place - which everybody should!)

    By Blogger Yossi Dahan, at 01/12/2009, 08:27  

  • Good point Yossi. Orchestrations that only serves receive/send-functions should of course belong to those assemblies.

    By Blogger Unknown, at 01/12/2009, 08:35  

  • @Patrick,
    yes, a similar way of working. As I am not using the ESB toolkit i dont have any ramps etc. In the past I used to differentiate between "send connectors" and "receive connectors" but now i just keep a single connector per backend system and that has orchs, maps, schemas, custom components etc that all pertain to that system alone. But the connector itself is always split up into multiple assemblies so that i can support long running orchs if necessary (and version the schema independently) although to be honest I've never had any orchs running that long that they couldnt be completely stopped and undeployed.


    By Blogger Unknown, at 08/12/2009, 08:27  

Post a Comment

<< Home