Yossi Dahan [BizTalk]

Google
 

Wednesday, March 04, 2009

Oslo based solution for deploying BizTalk applications

Since PDC I’ve been working on and off on an “Oslo” based solution for deploying a BizTalk application; unfortunately I couldn’t get a good chunk of time to play with this, so it’s been dragging a bit, but I’m getting close, so here are some details -

I’m a big advocate of automated builds; it’s a topic that probably deserves a post of its own, so I won’t get started on this here, but the idea is that one must have a way to be confident that, when its time to [re-]deploy the app, it will get deployed successfully time after time, without a hitch;

From my experience, deploying a BizTalk application of a decent size is often not trivial, and often it does not go smoothly if done manually (things are being missed, done out of order, wrong versions being picked up, etc), which does not help to boost the confidence in the solution (or BizTalk as a whole) in the organisation; automating the process can save a lot of time (can be done unattended, while in a meeting, out for lunch or overnight), save a lot of head scratching, boost the confidence in the solution and set the ground for proper automated builds, auto testing etc.

To that extent, I have previously built an MSBuild based framework for deploying BizTalk applications.

While I’m sure it does not suit all purposes as it was developed to support those scenarios I had, I think it’s quite comprehensive and had served me well over the past couple of years.

It allows one to provide an MSBuild-formatted file with some parameters and lists and, using the pre-created framework, would do things like remove the existing application (after terminating current instances and un-enlisting services), build the new solution(s), deploy assemblies to BizTalk, add/import binding files, start ports etc.

This is working great for us, and we’ve been using it extensively for quite a while now, but there’s one major downside – it requires one to maintain those MSBuild scripts.

Now -  MSBuild is cool, I do like it very much, but as it’s generic – it does not speak in BizTalk terms, and as it’s XML – it’s quite verbose, and I wanted an easier format for all of us to work with.

So – inspired after seeing Oslo and even more driven after visiting PDC - I’ve decided to come up with an MGrammar based language for describing a BizTalk application to be deployed.

So far I’ve come up with “version 1.0” of my grammar that allows to describe the major artifacts in a BizTalk application and a basic runtime to process source files written in this language, the language allows you to describe things like -

  • References to other applications
  • Solutions and projects to build
  • BizTalk assemblies to deploy
  • Orchestrations contained in those (these can then be terminated before attempting to remove the application prior to deployment)
  • .net assemblies to deploy
  • Binding file to import
  • Binding files to add (and the name of the environment to attach to those)

As Oslo hasn’t RTM-ed yet, I can’t quite rely on it yet, and so I cannot use it in production in any shape or form, or can I?
I found a good middle ground for us which allows us to gain from the benefits I’m hoping to get by using my language, while not exposing ourselves too much to the risks of using early technology –

At the moment the runtime I’ve built for this is used to generate the MSBuild scripts we’re already using out of the source files; in this way - if Oslo disappears tomorrow, or significantly changes (not that I think that’s going to be the case), we’re safe – we still have the MSBuild scripts as “checked-in artifacts” and so we have lost nothing.

So – how does an instance of my language looks like? here’s an example:

Application MyApp
{
//add references to other applications
add reference SomeOtherApp;
add reference AndAnother,andAThird;

//build the solution
build "SomeFolder\SomeOtherFolder\SomeSolution.sln";
build "SomeFolder\SomeOtherFolder\SomeProject.btproj";

//add required biztalk assemblies
add biztalk assembly "SomeFolder\SomeOtherFolder\bin\deployment\Schemas.dll"
version=1.0.0.0
culture=neutral
publicKeyToken=314bd7037656ea65;

add biztalk assembly "SomeFolder\SomeOtherFolder\bin\deployment\Maps.dll"
version=1.0.0.0
culture=neutral
publicKeyToken=314bd7037656ea65;

add biztalk assembly "SomeFolder\SomeOtherFolder\bin\deployment\Orchestrations.dll"
with orchestrations
{
MyApp.SomeOrchestration,
MyApp.AnottherOrchestration,
MyApp.AndAnotherOrchestration
}
version=1.0.0.0
culture=neutral
publicKeyToken=314bd7037656ea65;

//and .net helpers assembly
add assembly "SomeFolder\SomeOtherFolder\bin\release\HRG.FareWatch.Hotel.Helpers.dll"
version=1.0.0.0
culture=neutral
publicKeyToken=314bd7037656ea65;
//import dev bindings
import binding "SomeFolder\SomeOtherFolder\Bindings.xml";

//add various environment's bindings.
add binding "SomeFolder\SomeOtherFolder\Bindings.Dev.xml" "development";
add binding "SomeFolder\SomeOtherFolder\Bindings.Stage.xml" "stage";
add binding "SomeFolder\SomeOtherFolder\Bindings.Prod.xml" "production";
}


as I’ve said – I already have a runtime that generates the MSBuild scripts required to deploy these - the runtime outputs several files to the temp folder - 




  • The “framework” MSBuild script that contains all the targets I’m using


  • The Microsoft.Sdc.Tasks.BizTalk.dll and the Microsoft.Sdc.Tasks.dll which contains many useful custom tasks.


  • The relevant import file for the SDC tasks


  • The generated MSBuild file that contains all the various artifacts based on the source code



That latter file, which is the equivalent of my source code, and what we had to maintain so far, looks like -



<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="DeployAll">
<Import Project="Deploy.Framework" />
<ItemGroup>
<MyApp Include="SomeOtherApp" />
<MyApp Include="AndAnother" />
<MyApp Include="andAThird" />
</ItemGroup>
<ItemGroup>
<Bindings Include="General">
<path>SomeFolder\SomeOtherFolder\Bindings.xml</path>
</Bindings>
</ItemGroup>
<ItemGroup>
<EnvironmentBindings Include="General">
<path>"SomeFolder\SomeOtherFolder\Bindings.Dev.xml"</path>
<environment>development</environment>
</EnvironmentBindings>
<EnvironmentBindings Include="General">
<path>"SomeFolder\SomeOtherFolder\Bindings.Stage.xml"</path>
<environment>stage</environment>
</EnvironmentBindings>
<EnvironmentBindings Include="General">
<path>"SomeFolder\SomeOtherFolder\Bindings.Prod.xml"</path>
<environment>production</environment>
</EnvironmentBindings>
</ItemGroup>
<ItemGroup>
<ExternNetAssembly Include="HRG.FareWatch.Hotel.Helpers">
<Version>1.0.0.0</Version>
<Culture>neutral</Culture>
<PublicKeyToken>314bd7037656ea65</PublicKeyToken>
<ProcessorArchitecture>MSIL</ProcessorArchitecture>
<path>SomeFolder\SomeOtherFolder\bin\release</path>
</ExternNetAssembly>
</ItemGroup>
<ItemGroup>
<Solution Include="SomeSolution">
<path>SomeFolder\SomeOtherFolder</path>
</Solution>
<Solution Include="SomeProject">
<path>SomeFolder\SomeOtherFolder</path>
</Solution>
</ItemGroup>
<ItemGroup>
<BTSProject Include="Schemas">
<Version>1.0.0.0</Version>
<Culture>neutral</Culture>
<PublicKeyToken>314bd7037656ea65</PublicKeyToken>
<ProcessorArchitecture>MSIL</ProcessorArchitecture>
<path>SomeFolder\SomeOtherFolder\bin\deployment</path>
</BTSProject>
<BTSProject Include="Maps">
<Version>1.0.0.0</Version>
<Culture>neutral</Culture>
<PublicKeyToken>314bd7037656ea65</PublicKeyToken>
<ProcessorArchitecture>MSIL</ProcessorArchitecture>
<path>SomeFolder\SomeOtherFolder\bin\deployment</path>
</BTSProject>
<BTSProject Include="Orchestrations">
<Version>1.0.0.0</Version>
<Culture>neutral</Culture>
<PublicKeyToken>314bd7037656ea65</PublicKeyToken>
<ProcessorArchitecture>MSIL</ProcessorArchitecture>
<path>SomeFolder\SomeOtherFolder\bin\deployment</path>
</BTSProject>
</ItemGroup>
<PropertyGroup>
<BTApplicationName>MyApp</BTApplicationName>
<!-- Set for a remote deployment -->
<!-- Deploying BizTalk Server name - leave blank if local-->
<!-- not currently supported by runtime-->
<BTServerName>
</BTServerName>
<!-- Deploying BizTalk Server database - leave blank if BizTalkMsgBoxDb-->
<BTServerDatabase>
</BTServerDatabase>
<!-- Deploying BizTalk Server SQL user name - leave blank if local-->
<BTServerUserName>
</BTServerUserName>
<!-- Deploying BizTalk Server SQL password - leave blank if local-->
<BTServerPassword>
</BTServerPassword>
</PropertyGroup>
</Project>


Would you agree that the former source code is easier to maintain?



 



note: one thing you would notice is that none of the paths contains a root; I assume that this would be used by different developers/IT pros which may have the code in a different path; however, as the assumption is that the code will come from source control, my framework script expects you to provide the root path to your source code and assumes the specified paths start there.





So – what’s next?



From a coding perspective - I plan to add support in my runtime to perform the actual deployment; this is I would like it to do once Oslo is in production so I’ll add it as an option through a command line switch.



This option would tell the runtime to deploy the artifacts to a specified BizTalk server using BTSTask or the Object model instead of generated the MSBuild script.



I also want to make some modification to my language definition to make the MGraph produced cleaner (and better geared towards using the repository and Quadrant at a later stage) as well as, obviously, add support for more features, after which I plan to publish both my language and the runtime somewhere (here, codeplex, DevCente..we’ll see)



I’ll post some more notes on all of these here in the near future hopefully…

Labels: , , , ,

6 Comments:

  • Dude,
    This totally rocks. Put it on CodePlex!! Please!! :-)

    cheers
    benjy

    By Blogger Benjy, at 04/03/2009, 09:46  

  • Hey Yossi, that's really super cool. A few more practical examples like this will bring people closer to Oslo and this whole new world of DSL.

    Regards,
    Saravana

    By Blogger Saravana Kumar, at 04/03/2009, 12:28  

  • Hello Yossi, looking great.
    We are using our own model driven deployment tool (SQL based) at our BizTalk ustomers, and are currently migrating it to an Oslo repository. If you like we can share some experience as it appears we're doing exactly the same :>)

    Regards,

    Peter Van Kerkhove
    peter@cnext.eu

    By Anonymous Peter Van Kerkhove, at 05/03/2009, 15:40  

  • its very good

    By Anonymous yeni oyunlar, at 04/04/2009, 08:47  

  • Have you ever looked at the Deployment Framework for BizTalk? Not only is it based on MSBuild, but we have included a number of other features for BizTalk developers.

    http://biztalkdeployment.codeplex.com/

    By Anonymous Tom Abraham, at 20/05/2009, 22:02  

  • Hi Tom

    I was aware of the deployment framework, although never used it; seems very comprehensive though.

    The exercise for me was about using Oslo, but - from building my own MSbuild based framework, I also knew I needed a more intuitive way to describe the application and MSbuild is just too generic and verbose.

    I have no doubt my work is very basic (at this stage), as I was merely trying to show an approach, although I do insist on getting it fairly complete.

    The last point worth making is that as the runtime I've built uses a plug-in model it would be very easy, and rather interesting, to combine the two - and have the runtime generate your msbuild format from my language.

    Let me know if you're interested.

    (BTW - I know I'm way late in publishing this thing, life's just a bit manic at the moment!)

    By Blogger Yossi Dahan, at 20/05/2009, 22:29  

Post a Comment

<< Home