The Arch Hacker's BizTalk Blog

Solidsoft are recruiting in the USA!

2005-08-31T17:02:00.000+01:00

I hadn't realised quite how long it's been since I've blogged, oops! Anyway, just to give you an indication as to how busy I am, Solidsoft have now become incorporated in the USA with an office in Chicago and we are currently looking at hiring top BizTalk Architects (initially, anyway!).

The advert has gone up on Monster.com. Here is the specific link to the details:

Click this link to go to job advert details

BizTalk and SOA

2004-12-13T20:58:00.000Z

So, it's been a while since I've blogged. Life has been hectic and I'm currently spreading my time between two projects so blog time has been in short supply, even though my blog-to-do list is growing!

Anyway, so what is this post all about? It's about where BizTalk fits architecturally within the new "hot" world of Service Oriented Architectures.

BizTalk Server, as we know, is a middleware product that enables integration between LOB applications and systems in an EAI scenario and trading partners in a B2B scenario. As part of this broad area of responsibility BizTalk is able to execute business processes that implement complex integration or workflow scenarios, transform data from one format into another, route messages based on message metadata or actual content, perform security duties such as encryption and signing, capture tracking information for administrators or business analysts, communicate with a variety of communication protocols and mechanisms and so on and so on and so on. In short, the platform of BizTalk Server offers a complete business process and integration suite, hence the title of Microsoft's new division that is in charge of all of this stuff.

Knowing what we know of the features of the BizTalk Server platform how does that fit into an SOA world, one in which client applications "e;patch"e; together software services that provide specific business functions and offer a cohesive whole? Ultimately, why would you want to put an integration and business process engine into the mix when effectively the client is just calling a web service or two in order to offer specific functionality to a user?

Well, my belief is that BizTalk does offer value in an SOA when it is operating within 3 distinct patterns: Service Broker, Service Aggregator and Integration Enabler.

Service Broker

A service broker sits between the client and the web service and provides brokering services on behalf of the web service to the client. By putting a broker in between, it can act as a secure boundary that allows you to provide controlled access to internal web services. The broker might be the only supported route for consumers of an organisation's web services (see figure 1).

Figure 1: Service Broker

In addition to brokering based on security boundaries, the broker might act as a routing engine capable of determining the correct web service based on either message metadata content (such as HTTP or SOAP headers) or message content itself, in a content based routing scenario (see figure 2).

Figure 2: Service Broker using CBR

The service broker, depending on caller requirements, may operate in a request-response or one-way scenario.

If the broker operates in a request-response scenario, it is likely that it will provide some form of sync-on-async operation, as in the case of BizTalk Server 2004, in order to be able to provide a synchronous response to the client.

If the broker operates in a one-way scenario, no response is required and BizTalk Server is able to process the request in an asynchronous manner.

It may well be that a response can be asynchronously delivered, in which case the client issues a one-way request to BizTalk Server that brokers the call to the appropriate web service asynchronously (to the client), while further on in the process after a response has been received from the brokered web service, BizTalk Server asynchronously delivers a notification to the client containing the response.

Either way, BizTalk Server is used as middleware to broker calls between a client and its target web services. Introducing BizTalk Server as a broker will have performance implications due to an extra hop between two endpoints, however, the benefits of imposing a routing engine and security boundary offer an attraction to naturally security sensitive organisations that would like to expose business functionality as web services.

Service Aggregator

The service aggregator I tend to term a "super service" - a service that offers an aggregated business function view of discrete separate business functions.

In this scenario, BizTalk Server is used to consume multiple web services and aggregate their results behind the facade of a web service offering a single, probably coarse grained, business function (see figure 3). The aggregator, in effect, offers the ability to create composite applications using business process rules that tie multiple finer grained web services together.

Figure 3: Service Aggregator

Creating an aggregator, or super service, in BizTalk Server would involve building an Orchestration (the composite application logic) that enable the results from multiple web service calls to be aggregated together to provide a single response.

Indeed, BizTalk Server, in this scenario, operates as both a broker and an aggregator in one - brokering all calls to internal web services but also aggregating responses.

This pattern, I expect, will be the most oft used pattern with regards to BizTalk Server and an SOA. With the power of the Orchestration Engine and Business Rules, as well as the ability to inspect, manipulate and transform messages, BizTalk Server offers a great deal as a service aggregator within a SOA.

Integration Enabler

The previous two patterns are probably the most obvious with respect to a SOA; however, there is quite often the need to integrate with LOB applications behind the web service facade in an organisation. Just because BizTalk Server may sit between the client and the web service offering brokering and aggregation services, doesn't mean to say that its role as an application integration tool disappears - far from it (see figure 4).

Figure 4: Integration Enabler

An organisation may offer a web service that integrates both ERP and warehousing systems to external trading partners. While it is probably quite easy (depending on the interfaces into each application) to write code to achieve this integration, we get back into the mindset that we have been trying to get away from. Writing point-to-point interfaces between applications is bad! It has been proven time and again that P2P interfaces are short term and never stack up, both in terms of costs and support, in the long term.

Therefore, integrating between a web service's private implementation and LOB applications should still be achieved using an integration tool such as BizTalk Server, its bread and butter purpose.

Conclusion

So, in conclusion, I think that BizTalk Server 2004 fits quite appropriately into these 3 patterns of operation within an SOA. BizTalk Server itself is trumpeted as a friend to SOA with whitepapers and patterns and practices on the subject. On the whole, it isn't just marketing fluff and Microsoft is right. However, there are caveats - the most important caveat being performance and scalability. It is possible, it's just that there are pitfalls that you need to be aware of when designing architectures around BizTalk at the heart of an SOA. That, my friend, is an article for another day...

Reserved Keywords and Distinguished Fields

2004-12-13T20:51:00.000Z

Just a quick issue I discovered the other day with one of my schemas. I added an element called timeout with some child elements to the schema (days, hours, minutes, seconds), promoted the child elements as distinguished fields and then tried to use the schema in my orchestration.

Every time I tried to use the promoted property days using myMessage.Blah.timeout.days, I kept getting a squiggly red line under timeout and days with the error: unexpected keyword: timeout, cannot find symbol ''

The issue is because the timeout name is an XLANG/s reserved keyword. Therefore, if you want to promote a field as a distinguished field to be used in an orchestration, then make sure that no elements exist that are named the same as XLANG/s reserved keywords in the path up to the root element.

When oh when will Microsoft junk XLANG/s and just use C#???

BTS Guys Understand SOA and Contract First

2004-11-15T19:20:00.000Z

Aaron Skonnard has put up a good post about why BTS developers understand SOA and contract first. This is a hot topic at Solidsoft Towers and James and Duncan certainly have championed this at work for some time.

I was talking with James the other day about the ASHX HTTP handler that bypasses all of the unnecessary XML->Object and Object->XML deserialisation and serialisation that goes on. We agreed that as a lightweight 'web service' for BTS running at the core of a SOA architecture it is essential as it reduces the evil of latency that BTS can introduce (due to its primary task of ensuring messages are durable which means trips to SQL Server that could be avoided in a request-response synchronous scenario).

Aaron Skonnard's post can be found at http://pluralsight.com/blogs/aaron/archive/2004/11/11/3440.aspx.

I've been quite busy recently and haven't been blogging as much as I'd like. I'm due to rectify this with some posts on BizTalk Server 2004 and SOA soon, certainly in the performance area as it is possible to do SOA with BizTalk with reasonable latency results provided you know what you are doing.

The Bloggers Guide to BizTalk

2004-10-05T10:03:00.000+01:00

There is a tome out called The Bloggers Guide to BizTalk put together by Alan Smith that amalgamates a lot of the BizTalk blogging content into a very handy guide in CHM format. Incredible effort and highly useful! All I can say is WOW!

Alan's blog post can be found here and the guide itself is on GotDotNet here.

Enjoy!

DateTime Serialization Part 2

2004-10-04T13:38:00.000+01:00

A colleague has just suggested that it may be because the BizTalk NT Service account may not have a profile, therefore no timezone information. So, setting a promoted property in Orchestration with the value DateTime.Now would potentially produce a time based on GMT with no timezone or daylight savings offsets in effect.

A mightily sound theory, but I haven't been able to try it yet.

Will post here when I've had a chance to do it and prove the theory once and for all...

Duncan's blog post on the subject can be found here

DateTime Serialization in BizTalk - Possible Bug?

2004-09-29T13:32:00.000+01:00

On a recent project, I came across an interesting issue with assigning the current date and time to a promoted property in a Message Assignment shape.

The problem highlighted itself in our Audit database where we were auditing process state changes. The submission date of each audited record appeared to be 1 hour ahead than the current local time (based on GMT with Daylight Savings Time in effect - meaning we are currently +1 to UTC).

Our scenario involves writing an audit record from the web site that is initiating the process, then each subsequent audit is performed at various steps within a BizTalk orchestration. Each audit record contains some meta data extracted from the envelope, plus the XML fragment of the header within the envelope and optionally the body from within the envelope.

The first audit record is written by the web site. It uses a class created by xsd.exe which is based on the envelope schema deployed in BizTalk (not a BizTalk Envelope schema though). The code populates the class, then uses the XmlSerialization class to serialise the object into XML.

Each subsequent audit within the BizTalk orchestration effectively creates a new message (based on the envelope schema), updates the messageId, processStatus and submissionDate promoted properties within a Message Assignment shape, and then calls the Audit Web Service passing the message (as a string - long story, but that's the way it currently is) as an argument.

The first audit record has the correct time in the Oracle database, but every subsequent audit record has a time that is 1 hour ahead!

In the web site I'm using the expression DateTime.Now to populate the current date/time in the submissionDate property of my object. When serialised by .NET using XmlSerialization, it converts this date into a Sortable Date Time pattern (identified as 's' in .NET) as defined by ISO 8601. In addition, the DateTime.Now value is in local time, and it takes this into account when building the ISO 8601 date string. For example:

2004-09-24T11:24:08.5177508+01:00

From this date value you can clearly see that the date/time is in UTC format with a +01:00 for the adjustment for daylight savings time. So, this is a correct date/time for British Summertime.

When I set the submissionDate promoted property in BizTalk using the expression DateTime.Now, it should output the same or similar value, taking into account that DateTime.Now returns a local time, and that the date string should encode the daylight savings offset. However, it doesn't...

The following is a date string output from BizTalk. Bearing in mind that the message is serialised into XML and then passed to the Audit Web Service. This is what is received by the web service:

2004-09-24T11:24:08.9240060Z

The date/time string format is based on the Universal Sortable Date Time pattern (identified by 'u' in .NET) and is the correct local time but does not have any daylight savings adjustment associated with it! This means that when the Audit Web Service converts the string into a DateTime object using either System.Convert.ToString() or DateTime.Parse(), it takes the daylight savings adjustment into account and adds 1 hour!. It didn't work with DateTime.ParseExact( dateTimeString, "u", System.Globalization.DateTimeInfo.InvariantInfo ) either.

So, in order to fix this, we had to use the DateTime.UtcNow in the orchestration whenever we set the submission date promoted property so that the DateTime.Parse method could take into account current regional settings when converting the string to a DateTime.

Cool XSD Validation Function for Orchestration

2004-09-18T14:54:00.000+01:00

As I promised in an earlier post, here's a cool XSD validation function that can be called from within orchestration. Just add to a library - I tend to put it into a BizTalk Utilities library full of useful little functions that complement orchestration (most of which should be provided by the orchestration engine, but that's a rant for another day!).

public static void ValidateDocument( XmlDocument businessDocument, string schemaStrongName )
{
  // Constants
  const int PARTS_IN_SCHEMA_STRONG_NAME = 2;
  const int PART_CLASS_NAME = 0;
  const int PART_QUALIFIED_ASSEMBLY_NAME = 1;

  // Parse schema strong name
  string[] assemblyNameParts = schemaStrongName.Split( new char[] { ',' }, PARTS_IN_SCHEMA_STRONG_NAME );
  string className = assemblyNameParts[PART_CLASS_NAME].Trim();
  string fullyQualifiedAssemblyName = assemblyNameParts[PART_QUALIFIED_ASSEMBLY_NAME].Trim();

  // Load assembly
  Assembly schemaAssembly = Assembly.Load( fullyQualifiedAssemblyName );

  // Create instance of the BTS schema in order to get to the actual schemas
  Type schemaType = schemaAssembly.GetType( className );
  Microsoft.XLANGs.BaseTypes.SchemaBase btsSchemaCollection = ( Microsoft.XLANGs.BaseTypes.SchemaBase )Activator.CreateInstance( schemaType );

  // Set up XML validating reader and validate document
  XmlParserContext parserContext = new XmlParserContext( null, null, "", XmlSpace.None );
  XmlValidatingReader reader = new XmlValidatingReader( businessDocument.OuterXml, XmlNodeType.Document, parserContext );
  reader.ValidationType = ValidationType.Schema;
  reader.Schemas.Add( btsSchemaCollection.SchemaCollection );
  while( reader.Read() ) {}
}

In order to call in orchestration, just create an expression shape and use the following:

ValidateDocument( myBtsMessage, myBtsMessage( BTS.SchemaStrongName ) );

Validating using Business Rules

2004-09-18T14:39:00.000+01:00

I tend to get about at work (not in the, ahem, naughty sense!) and while in Norwich a client wanted to validate their document to ensure that the end users had filled in all of the correct details. Simple standard stuff. However, when I explained that validating the document using a pipeline would suspend the message at the first error, they didn't exactly jump with joy!

They wanted to validate the document, but provide all of the errors in one go to the client if the document failed validation.

In order to do this, we proposed following the pattern used by the Swift Accelerator for BizTalk. The accelerator uses a disassembler that implements validation using business rules (run from the disassembler itself) and stores all errors in a separate document. It does not fail at the first error, but simply tries all of its rules and outputs all errors associated with the document in one go by returning the original document plus a document containing the errors.

This allows all sorts of interesting scenarios to occur, such as implementing an orchestration that handles errors allowing resubmission, notification to interested parties, etc. Quite powerful!

In the case of my client, we suggested running the business rules from within orchestration, but the concept remains the same.

Filter Subscriptions on Orchestrations

2004-09-18T14:27:00.000+01:00

I've been working for a large defence client in Portsmouth in the UK and recently come across another limitation in the filter expressions that can be applied to an activatable receive shape of an orchestration.

First some background though. When you drag an operation from a port to an activatable receive, you are effectively creating a subscription for that orchestration in the message box. When you deploy the orchestration, bind it and then enlist, the subscription will be created for that orchestration based on the message type that was defined on the operation that was bound to the receive shape.

This is important: the subscription is based on the message type of the operation defined on the port type. If you look at the subscription viewer and find your orchestration subscription (look for ServiceType of XLANG and the name will be prefixed with Activate), then you'll notice that the subscription has been automatically created with a MessageType filter.

Therein lies the problem. You see, because the subscription has been based on the message type being received, you can't set a filter in the receive shape that points to, say an envelope's promoted properties. This in my opinion is a serious flaw. If I have an envelope that contains meta data about my message, then I want to subscribe to properties on the envelope, but actually receive the message contained in the body of the envelope.

If I want to do this, then I have to define the envelope schema as a normal schema, that is, not a BizTalk Envelope schema. I then ensure that the body element is extensible, in other words, appending an <Any> element with ProcessContents set to Skip and the Namespace to ##any.

Once this has been done, receive the envelope in the orchestration (defined on the receive shape and port type) and then extract, plus optionally validate, the body inside orchestration.

This has an advantage:

The body document can be validated at a 'business' level - effectively allowing any errors from the validation to be handled within the orchestration, rather than being automatically suspended. I'll post some neat code that achieves this in a later post.

However, and this brings me all the way back to the point of my post, if I try and modularise my envelope schema by separating my header schema and then importing it into the main envelope schema, I have a major problem.

The envelope schema exists in one namespace and the header schema in another. The header schema has been imported into the envelope schema. In so far as XSD is concerned, I have a valid schema. In so far as BizTalk is concerned, it can see the header schema and can thus validate the contents (as opposed to using the Any element trick).

However, if I go to set a filter expression on the receive shape, then all of the promoted properties that are defined in the header schema do not display. Only promoted properties of the envelope schema will be displayed!

I've been bitten by filter subscriptions with BizTalk Envelopes and now with imported schemas. I so wish this will be sorted out as it limits the flexibility of the schema design that could be possible, but due to restrictions imposed by BizTalk, isn't.

I would be interested to hear some other view points on this.

BizTalk Hotfix for Business Rules

2004-09-18T13:44:00.000+01:00

It's been a while since my last update, so I'll start off gently with news about a hotfix for business rules if you have installed .NET Framework 1.1 SP1.

Click here for more information.

BAM Deployment Issue

2004-08-04T17:03:00.000+01:00

I discovered a weird problem recently when deploying a BAM definition spreadsheet using the bm.exe program.

Basically, I was getting multiple message boxes containing fn_GetString in the title, plus some other blank message boxes until finally the process ended. It looked pretty terminal!

The problem turned out to be that on my machine the locale was set to English (United Kingdom) and it looks like the BAM deployment process doesn't like it. The message boxes look like they come from Excel so I guess the BAM Excel spreadsheet macro code is to blame.

Changing the locale to English (United States) solved the issue and the bm.exe program was able to export the BAM definition XML successfully from the spreadsheet.

However, I came across another problem. I was running the bm.exe program from the folder that contained my spreadsheet. This folder is different to the BAM folder in the installation folder: \Program Files\Microsoft BizTalk Server 2004\Tracking. An error occurred which said that it couldn't find the BamConfiguration.xml file in the current folder. It seems that it created it back in the BizTalk installation folder, so no wonder!

So, I copied the spreadsheet into the Tracking folder of BizTalk, ran the BAM deployment and it worked!

So I guess the morale of this tale is don't try and be creative with the tools. As long as you live in the US and don't try to do silly things like create application folders to store custom BAM spreadsheets, then you'll be fine ;-P

<Edit>

I've just realised that the BamConfiguration.xml file is a settings file for BAM! Doh! Thanks also to Carlo for pointing that mistake out as well :)

Documentation Refreshed on July 30th

2004-08-02T13:30:00.000+01:00

Looks like the documentation has been refreshed again on the 30th July. See post below for the link.

Received Unexpected Message Type '' in Orchestration

2004-08-01T19:47:00.000+01:00

This was an error that came up recently on a project with a client in sunny Rochdale in the UK. Basically, the orchestration failed with the following error message:

Received unexpected message type '' expected message type '%s'

In effect, orchestration had received a message that had not been resolved to a message type, hence it was unable to understand it. The %s placeholder contained the name of the message type that it was expecting.

This was quite confusing at first because the message that was being returned to BizTalk by way of a solicit-response port was correct according to the schema. Everything looked as if it should match up!

After scratching our heads for a bit, we discovered that the response properties of the solicit-response messaging port was using the Passthru pipeline. This pipeline will not resolve the incoming message against a message type stored in the database! Changing the pipeline to the XMLReceive pipeline did the trick, and orchestration was able to receive the message as a response correctly.

Delivery Notification Doesn't Appear with Specify Now Binding

2004-08-01T19:46:00.000+01:00

Now, I don't pretend to understand this, so if someone can enlighten me, I would appreciate it :)

I have a web port type defined in orchestration to a OneWay web service. When I create a port based on the web port type and I select Specify Now in the port configuration, then the Delivery Notification property doesn't appear. If I go into the port properties window and change the binding to Specify Later, the Delivery Notification property appears! Change the binding back, and it disappears again! Eh?

My guess is that the delivery notification property is implicitly set when using the SOAP adapter (which is the default using a web port type and specifying the binding in the orchestration), therefore the reason why the property disappears from the property window. By setting the binding to specify later, it is unknown what adapter will be used, therefore offering the delivery notification property to be changed. From what I understand, if a SOAP fault occurs in the SOAP adapter, the SOAP fault is automatically returned to BizTalk, so would explain why the property doesn't appear if the adapter used is SOAP when specifying the binding in orchestration.

However, my dilemma is that I've specified that the web service is OneWay (which the port type understands and only offers a request message to bind to). In this instance, I have no response, but would still like to know if the adapter was able to deliver the request, hence the reason for using the delivery notification mechanism in this scenario.

Is this a bug? Answers on a postcard please...

Multiple Activate Receives and Returning Soap Faults Oddity

2004-08-01T19:45:00.000+01:00

Imagine a scenario where you have an orchestration exposed as a web service that has two public ports. Each port accepts a message of the same type, but one port is used to send new messages, and the other is used to send update messages. The port type for the port has an access modifier of public, is a request-response port type, plus has a Fault message defined in addition to the request and response messages.

In the orchestration, you create a listen shape and then add two activatable receives - one on each branch with one pointing to the request message of the new port, and one to the request message of the update port. So far so good.

At this point you're going to want to create a couple of send shapes in order to send back either a request message or a fault message. However, as I've just discovered, there's a slight catch, and it is this:

You have configured a scope shape with an exception handler block in each of the listen branches. You have it configured such that at the end of the scope you send back a response message and in the exception handler block you return a fault message for BizTalk to wrap in a SOAP exception and return to the client. This seems perfectly reasonable on the face of it. The problem occurs at compilation time.

The compiler basically barfs with must receive before sending a fault message on an implemented port. What? Well, I've certainly performed a receive before sending, so why is it complaining? If I move the send of the response after the end of the scope but still within the listen branch, then it now barfs with must receive before sending a message whose messagetype corresponds to a requestresponse operation on an implemented port.

It appears that it doesn't like having two send shapes where one may execute before the other, therefore nullifying the receive for the second send shape.

So, in order to get around this, you need to put the two send shapes (response and fault message sends) in either branch of a decision shape. So, at the end of the scope, set a flag to indicate success and in the exception handler set the same flag to indicate that a fault has occurred. Then, after the scope, create a decision shape to test the flag and either send the response or send the fault.

Just out of interest, I moved the decision down in the orchestration until it was after the end of the listen shape (that is, it wasn't directly in any of the branches of the listen shape, but just located 'after'). In this case I get both of the compiler errors detailed earlier which is quite interesting. Probably as a result of the fact that it could have been possible to exit the listen from another branch, thereby not executing the receive shape for which we're returning a response.

Returning SOAP Faults from an Orchestration Published as a Web Service

2004-08-01T19:39:00.000+01:00

I was asked the question recently: how can I return a SOAP fault from a web service call if the orchestration that has been published has failed?

Well, this is seemingly a lot easier than you would think. A request-response port type in orchestration allows multiple Fault Messages to be defined. These fault messages can be based on any message type (for example, System.String) and are returned to the web service in the same manner as returning a response to a web service, that is, via a send shape to the appropriate fault message on the operation defined on the port type.

If you were to take a look at the port type definition in the orchestration view tab in VS.NET, then you'll see the various messages defined on the operation of the port type. Each message has an associated icon that represents the message direction:

blue arrow for request
green arrow for response
red cross for fault

By simply creating the appropriate message and populating it, then sending the message to the fault message of the port, you are able to return SOAP faults to the client of the web service. Easy, eh?

Refreshed Documentation as of July 2004

2004-08-01T19:33:00.000+01:00

Follow this link to download a new update to the product documentation.

However, for some reason, we're having some problems installing it. In addition, why are the following services shut down during a documentation refresh?!

BizTalk NT Host Instances
Enterprise SSO
FTP Publishing Service
HTTP SSL
IIS Admin Service
SMTP
WMI
World Wide Web Publishing Service

Seems a bit overkill to me!! Imagine trying to get away with that in production!

Using a Map with Large Messages

2004-07-15T15:00:00.000+01:00

I was intrigued the other day about how maps were behaving in BizTalk Server 2004 with regard to potential memory problems. BizTalk Server 2004 gives us great large message support through streaming messages into the message box. However, what if we specify a map in the Receive port, or indeed anywhere in BizTalk (Orchestration or Send Port)?

Well, BizTalk implements maps as XSLT stylesheets, and the way I understand it, the code uses the XslTransform class to perform the transformation and will load the input message into the DOM. This will require a significant memory footprint if the message is large!

This is often overlooked and is worth pointing out...

Fed up of deploying BTS assemblies the 'standard' way?

2004-07-14T16:21:00.000+01:00

Well, this is a little tip about deploying a BizTalk assembly quickly and easily without having to unbind, undeploy, redeploy, rebind and all that wonderful stuff.

If you recompile a BizTalk assembly that does not affect the current bindings then all you need to do is the following:

Open a view onto the GAC by clicking Start->Run and then typing assembly
Compile BizTalk assembly
Drag and drop the compiled assembly from the bin\Development folder (we're developing, right?) into the assembly view window
Restart BizTalk NT service from BizTalk Administration snap-in, Services snap-in or the command line using net stop

It really is as simple as that!

DNJ Interview with Solidsoft CTO Andy James

2004-07-14T12:53:00.000+01:00

Ok, ok, so he's my boss ;-) Still, it's a good article about BizTalk and SOA from someone who's been about, if you know what I mean ;-)

DNJ Interview Link

Integration Patterns

2004-07-12T12:23:00.000+01:00

Microsoft has released a whole set of integration patterns here as part of their Platform Architecture Guidance (PAG) efforts.

Well worth a browse!

One-Way Web Methods

2004-07-12T10:51:00.000+01:00

I made quite a cool little discovery the other day with OneWay web methods in ASP.NET. When I hooked the web service up to my orchestration, I discovered that BizTalk had actually taken the OneWay interaction style into account and only offered a request message type, rather than both a request and response message type.

Many people probably know this already, but I found this to be really cool! How many times have I hooked up a web service with an orchestration and ignored the response because the method does not return any value?! More times than I care to remember...

KB article for installing BizTalk on WinXP SP2

2004-06-29T11:49:00.000+01:00

This knowledge base article details how to modify the registry (eek!) to get the Enterprise Single Sign-On service to work after installing Windows XP SP2.

WSE Adapter for BizTalk Server 2004 Beta Program

2004-05-27T16:55:00.000+01:00

Ok, it looks like Scott Woodgate has released details about the WSE Adapter for BizTalk beta program over on his blog here.

The WSE Adapter has a dependency on the WSE 2.0 release that has just recently been made available on the MSDN site.

WSE 2.0 Released!

2004-05-27T16:44:00.000+01:00

Although this isn't strictly related to BizTalk (watch this space though!), it's a significant release with regards to security in web services (WS-Security is implemented to the OASIS 2004 standard) and a little thing called SOAP Messaging.

SOAP Messaging allows us to move closer to the SOA goal of defining web services that can be asynchronous, loosely coupled and interact using XML based messaging; rather than RPC over HTTP (current .NET web services implementation). Ok, I know that RPC over HTTP is effectively SOAP-RPC and is implemented as an XML message, but there is an important difference there. SOAP-RPC defines a format(s) for representing a method call, its parameters and their datatypes. SOAP Messaging is more lightweight in that the body of the SOAP message is simply an XML document that can be anything - say, an invoice. There is no SOAP-RPC encoding involved. In an SOA world, the XML messages are defined using schema and the behaviour of the web service as contract, and WSE 2.0 gets us on that road.

Therefore, in my view, WSE 2.0 becomes quite a powerful platform for building web services today in anticipation of Indigo and the nirvana of an SOA future...

Persistence in Orchestration

2004-05-13T17:27:00.000+01:00

Just looking at the BizTalk help file (the latest) and was quite interested in how persistence was handled in orchestration in 2004. In 2002 a small change was made (over 2000) to make the persistence of state more efficient by allowing the persistence to be dropped at the start of a transaction. In 2004, it seems the persistence of orchestration state is better thought out with regard to resiliency of an orchestration.

It's one of those things that you know happens, but you're never quite sure exactly in what situations orchestration state is saved to the message box.

Thankfully, the latest BizTalk Server 2004 help addresses this, and can be found here. This is well worth a read if you're like me and want to know a little bit more about what happens under the covers.

Delivery Notifications in Orchestration

2004-05-13T14:49:00.000+01:00

Well, this was a little gem that was pointed out to me by a buddy from Microsoft UK who said that surely this little snippet of information was worth a drive in my new car!

Well, I'm not so sure, but it's useful nonetheless ;-)

So, onto the topic in hand. A documented feature of Send Ports in Orchestrations is that it is possible for an acknowledgement to be returned to the running orchestration when a message has been transmitted to its destination. It's often been the case (in previous versions) that it would be useful to know when a message has been transmitted, and if there is a transmission failure and the message goes into the suspend queue, then to be able to take corrective action in the orchestration. With Delivery Notifications it is possible to do that!

How does it work?

Well, when this option has been switched on, a context property called BTS.AckRequired is set that informs the adapter that a transmission acknowledgement is required. Once the message has been transmitted successfully an 'ack' is returned that the engine uses to progress the orchestration. If the message fails to be transmitted (after the retry count has been exhausted), then a 'nack' is returned instead. If a nack is received by the orchestration engine, then a DeliveryFailureException is thrown that can be caught in the orchestration using an Exception Block Handler in an enclosing non-atomic scope. The non-atomic enclosing scope is not exited until a ack/nack is received - which makes this very useful as the orchestration won't continue and will just block. The documentation states that an atomic scope is required for the send shape although I have found that it also seems to work in a non-atomic scope as well.

How do I do it?

In orchestration, perform the following steps to delivery notification heaven:

Set up a port that is used to send messages.
Look at the port properties, and set the Delivery Notification property to Transmitted.
Create a non-atomic scope shape with the transaction type set to Long Running.
Add an exception handler block to the scope to catch exceptions of type DeliveryFailureException.
Add a send shape inside the scope and hook it up to the port.

Don't forget that the non-atomic scope (or the orchestration if no scope is specified) will not complete until either an ack or a nack is received. Very useful for running some sort of corrective action in case of transmission failure. Also, I've not tried, but I wonder if, rather than putting the corrective logic in the exception block, whether the succeeded() XLANG/s expression could be used to test if the scope had failed, and then perform corrective action but in the main flow of the orchestration. Sounds feasible I guess...

Exposing Orchestrations as Web Services Update!

2004-05-04T10:52:00.000+01:00

I received a comment from Scott Colestock (thanks Scott!) on the blog entry to do with multiple schemas with the same root name and namespace, and then BizTalk failing at runtime because 2 schemas exist.

Basically, another workaround to the problem is to ensure that you use a custom pipeline that uses the XML Disassembler or Assembler (depending on which way you're going!) that specifies exactly what schema to use to resolve the message name.

Scott's article which covers this in detail can be found here.

Thanks again Scott :-)

Supported Operating Systems for FTP Adapter

2004-05-04T10:38:00.000+01:00

The following knowledge base article on the Microsoft website lists the operating systems that are supported by the FTP adapter for sending and receiving files.

Click here for the article.

BizTalk Subscription Viewer Tool

2004-04-29T10:40:00.000+01:00

Another tool that is a life-saver when it comes to working out how messages that are submitted into BizTalk are matched to 'listening' Orchestrations and Send Ports is the BizTalk Subscription Viewer utility BTSSubcriptionViewer.exe.

This tool can be found in the following folder:

\Program Files\Microsoft BizTalk Server 2004\SDK\Utilities

Once loaded, this tool is a Windows Application that has a split pane view. Initially the views are empty. Go to the Main->Load menu to load all of the subscriptions from the message box database.

Ok, this is more like it. A list of subscriptions are displayed in the top pane. If you select a subscription, then the subscription filter expression is display in the bottom pane. This effectively gives you an idea as to what properties BizTalk is looking to match in order to fire the subcriber.

There are some items in the list that are worth pointing out. Some subscriptions may be prefixed with Activate. These are Orchestration (XLANG) subscriptions where BizTalk will activate a new instance of an orchestration when the properties of a message match the subscription. There are also EPM type subscriptions. EPM stands for the Endpoint Manager and is responsible for the messaging subservice in BizTalk Server. Therefore, these subscriptions are for Send Ports that are waiting to be fired. There is also an internal subscription that has a service type of a GUID. Looking at the subscription filter expression for these (there will be one per host) suggests that this subscription manages the cache refresh of the host application (where each service of a host will periodically refresh its internal cache from the Management database). However, I'm only making an educated guess of the cache refresh subscription. Please comment here and correct me if I'm wrong :-)

So, why is the subscription viewer so important? Well, if you've ever had to implement a complex Content Based Routing project, or worked on Correlation projects, then you'll understand why. It is essential to be able to take a view of the current subscriptions in the message box especially when waiting on a message that is due to be correlated. If things go wrong (and they almost always do when initially developing the solution) then this tool can be an absolute life-saver!!

Pipeline Command Line Tools

2004-04-29T10:03:00.000+01:00

Ok, for those that don't know, there are some really handy pipeline tools in the SDK under the folder:

\Program Files\Microsoft BizTalk Server 2004\SDK\Utilities\PipelineTools

These tools allow you to test your pipeline components and, even more handily, flat file schemas without having to set up the BizTalk Server environment. They are pretty much essential when developing flat file schemas (particularly when you have a header, trailer and body schema).

Here is a list of the pipeline tools available:

DSDump.exe - this tool outputs a textual description of the schema.
Pipeline.exe - this tool executes a pipeline and produces one or more output files. Very useful for testing custom pipelines.
XMLDasm.exe - this tool executes the XML Disassembler component by simulating a receive pipeline.
XMLAsm.exe - this tool executes the XML Assembler component by simulating a send pipeline.
FFDasm.exe - this tool executes the Flat File Disassembler component by simulating a receive pipeline. Essential for flat file development!
FFAsm.exe - this tool executes the Flat File Assembler component by simulating a send pipeline. Again, essential for flat file development!

Note: The pipeline.exe tool does not access the BizTalk Server databases, so executing a pipeline that contains the BizTalk Framework Assembler/Disassembler may not work as it accesses the BizTalk Management database during runtime to match up receipts to documents.

Updated BizTalk Server 2004 Tutorial

2004-04-19T09:53:00.000+01:00

An updated BizTalk Server 2004 tutorial is now available.

Download here.

Ok, ok, so it's been out a while.....I know ;-)

Exposing Orchestrations as Web Services Gotcha!

2004-04-15T12:38:00.000+01:00

A colleague came to me today with a problem to do with duplicate schemas at runtime when running his orchestrations.

The problem turned out that during generation of the proxy (when adding a web reference to a BizTalk project) to a web service that was automatically generated by the BizTalk Web Services Publishing Wizard, then if the web service exposes a complex type such as a message, then the proxy automatically generates a schema called Reference.xsd that contains a local definition of the message.

However, this may all seem fairly benign on the surface. But, the proxy generator uses the WSDL to create the proxy orchestration (Reference.odx) and the schema (Reference.xsd), and the WSDL was created based on a web service that has been automatically generated by the BizTalk Web Services Publishing Wizard. The WSDL for the web service contains definitions for the messages that are required as parameters to the web service operations, and they are defined in the WSDL with the same root name and namespace as the real schema that they are based on!

So, when you compile the orchestration that is consuming the web service, a Reference.xsd schema containing the definitions for the messages expected as parameters to the web service, are compiled into the assembly as well.

This is a big issue, because once both assemblies are deployed (one containing the consumed orchestration that includes the original schema for the message and one containing the orchestration that is calling the web service and includes a proxy generated schema for the message) then at runtime BizTalk will fail because 2 schemas exist in the Management database with the same root element and namespace names!

So, how do we get around this? I can only think of these 2 ways:

Create a .NET class that wraps the web reference, create a static method that wraps the web service call, and then call the class method from within an Expression shape in orchestration, thereby avoiding creating the web reference inside the BizTalk project.
Pass the message as a string to the orchestration before running the wizard, but that way you'll need to convert the string back to a message reference within the consuming orchestration.

If anybody else has any other ideas, I'd be interested to hear about them here!

Working with Enumerations in a Web Service within Orchestration

2004-04-14T18:58:00.000+01:00

Well, this had me scratching my head for some time until the lightbulb lit up in my brain! How was I going to call a web service within an orchestration that used an enumeration as one of its parameters?

The answer is surprisingly simple but very consistent with the way that orchestration works in BizTalk Server 2004 with respect to web services.

When you want to call a web service from within orchestration you can either do one of 2 things:

Write a wrapper in .NET that calls the web service and call the wrapper from within an Expression shape
Add a web reference to the BizTalk project and then create a configured port based on the web port for the web service

So, what's wrong with the first option? Well, you will need to write code for starters just to call the web service, and then once created, then you need to expose the web methods you require access to in order for them to be invoked from orchestration.

Here's the rub though: while this isn't a particularly desirable route to calling a web service from within orchestration, sometimes it's absolutely necessary that it is done this way. There are 2 very good reasons for this:

You don't want to go through the message box every time you call the service (which is what happens if you call the web service through a web port in orchestration).
More importantly, the web service interface uses parameter names that are on the XLANG/s reserved keywords list. That's right! You can't name a parameter anything and just get away with it when using that web service from orchestration! Therefore if you have no control over the web service interface that you're calling, you will need to create a wrapper in order to be able to call it from orchestration.

So, in most circumstances you'll probably want to go the second route. When you add the web reference into the BizTalk project in Visual Studio.NET, instead of a proxy class in C# (or VB.NET if that's your preference) being generated behind the scenes, a BizTalk Orchestration is created as the proxy. Cool!

But, what happens when the web service method that you're calling contains a parameter that is based on an enumeration? Fortunately, when the web reference is added, any enumerations that are exposed in the web service definition (WSDL) are turned into schemas that are available in the project under the web reference itself.

So, with this information to hand, all we need to do is to create an XLANG/s message based on the enumeration schema, populate it with the right enumeration value, and then pass this message as a parameter to the web method.

The following code snippet shows how this is done and is located in the Message Assignment shape that sets up the request message for the call to the web service. If you take a look at the sample code, this particular web service method expects 4 parameters, one of which is a parameter based on an enumeration (which I've set up with a call to the XmlIfy static utility method). In addition, I made the enumeration value field in the schema a distinguished field by modifying the schema that was generated automatically when adding the web reference to the BizTalk project. Don't forget, if you update the web reference, you will need to promote the field as a distinguished field again otherwise the project won't compile! Note that you also need to construct the enumeration message as well as the web service request message in the Construct shape.

notifyRequestMessage.address = registrationMessage.escalation.sendTo;
notificationType = BizTalkUtility.XmlIfy( "<NotificationTypeEnum xmlns='http://somenamespace/notify' />" );
notificationType.NotificationTypeEnum = "SMTPEmail";
notifyRequestMessage.type = notificationType;
notifyRequestMessage.subject = "My Subject";
notifyRequestMessage.content = "My Content";

Note: Whenever I work on a BizTalk project, I always have a utility library written in C# that allows me to perform various useful little functions like turning an XML document into a string, and a string into an XML document. Highly useful!

Apologies for lack of update recently!

2004-04-14T18:13:00.000+01:00

Things are hectic here at Solidsoft Towers! Working on a big proposal that is sucking my time, so apologies for lack of recent updates. Will get back to it when the proposal is winging its way into the client's lap... ;-)

Microsoft Webcast on BizTalk Server 2004 Architecture

2004-04-08T12:11:00.000+01:00

Here's a link to a great webcast on the architecture and scalability options of BizTalk Server 2004 presented by Scott Woodgate.

Click here to access the webcast.

Visio Orchestration Designer Add-In

2004-04-05T15:22:00.000+01:00

The add-in for Visio (2002 SR-1 or 2003) to allow business analysts to design business processes that can be used by a developer to implement orchestrations is now available.

It can be downloaded here

Sharepoint Adapter

2004-04-05T10:59:00.000+01:00

Those lovely folks over at GotDotNet have available a Sharepoint adapter for BizTalk Server 2004.

You can find it here

New Information Worker documentation available

2004-04-05T10:26:00.000+01:00

There is also new documentation available for Information Workers. Download here

BizTalk Server 2004 Rollup 1 is available

2004-04-05T10:24:00.000+01:00

This is a rollup patch for BizTalk Server 2004 that should be applied to the RTM build of BizTalk Server 2004. This will be included in the upcoming service pack 1 (no release dates available yet).

Click the following link to download the rollup patch: BizTalk Server 2004 Rollup 1

At last! New documentation is available for download from Microsoft

2004-04-02T13:37:00.000+01:00

Well, we've all been waiting for it.

New documentation for BizTalk Server 2004 can be downloaded from the following locations:

Documentation
Click to download BizTalk Server 2004 documentation

SDK
Click to download BizTalk Server 2004 SDK

Installation Guide
Click to view the updated installation guide

Problem encountered installing on Windows 2000

2004-04-02T13:27:00.000+01:00

If you've installed BizTalk Server 2004 on the 'slipstream' SP4 Windows 2000 build, then when you get to the ConfigFramework wizard, you may well find that the file AUTHZ.DLL is missing from the Winnt\System32 folder.

If this is the case (and it almost certainly will be), then you'll need to re-install the full version of SP4 for Windows 2000. This will ensure that the AUTHZ.DLL file is installed, and you'll be able to run the ConfigFramework wizard.

First BizTalk Tip!

2004-04-01T14:55:00.000+01:00

Ok, first tip.

Apologies to all of those people that know this (and if you've worked with BizTalk Server 2004 for any length of time, you will!).

Any time you deploy a new BizTalk assembly, or even GACing a component that is used by orchestration or by a pipeline (except obviously components designed to run under a COM+ server application), don't forget to restart the BizTalk Server NT service BTSNTSvc.exe. You can do this via the BizTalk Server Administration MMC snap-in by finding the server and restarting it, or just simply restarting the relevant BizTalk Host in the Services MMC snap-in.

Simple eh?

PS: You have no idea how many times I've been caught by this........grrrrr!

Welcome to my blog!

2004-04-01T12:59:00.000+01:00

Well, I've been nagged and nagged to get this blog up and running, and here it is.

Now all I've got to do is to put up some worthwhile posts that will actually be of use to people....

So, with that in mind, watch this space :)