BizTalk 2006 as ESB

When implementing an SOA ESB, Microsoft BizTalk Server (BTS) could be worth considering as base platform.

The following thoughts are based on our experience with building ESB’s using BTS 2006 or BTS 2004 as a core component of an ESB. While BTS2006 is not the current version, the same thoughts can be applied to BTS 2006 R2 and BTS 2009. For BTS 2009 there might be better options though depending on the specific requirements for the ESB.

In the SOA infrastructure article we have discussed that the core functionalities of a basic ESB are:

• Publications
• Subscriptions
• Queries
• Authentication
• Authorization

So how does BTS 2006 fit into this picture?

Publications and subscriptions

The most important part of BTS 2006 and other BizTalk versions is the message box. The message box is a SQL Server database, and its main function is to receive incoming (published) messages and deliver them to the appropriate receivers (subscribers). Since BizTalk in its very nature is publish/subscribe based, it comes as no surprise that the first two bullets on the list of core functionalities, publications and subscriptions, are easily implemented using BTS 2006.

A subscriber can be represented by a BizTalk Send Port configured to deliver messages on the subscriber’s end point.  To represent a publication – a type of message/event a publishing service can send – a BizTalk Send Port Group configured with filters on relevant content or context properties of the messages would be a good choice.

This way we can make any service subscribe to any publication – and thus create a subscription – by making the send port representing the service a member of the send port group representing the publication. Thus it is possible to implement publications and subscriptions without using BizTalk Orchestrations and keeping everything fairly loosely coupled on the technical level.

Queries

Queries are a different story – especially when we are talking about synchronous queries. Because of its dependency on the publish/subscribe pattern, BizTalk is very asynchronous by nature and because of its heavy use of the message box, any operation performed on BizTalk requires several database operations in the background. BTS 2006 is not optimized for low latency scenarios – it is optimized for safety in handling the messages. As Mr. John Callaway from Quicklearn usually puts it: “BizTalk never ever loses a message”.

Because of these limitations it is advisable – especially in low latency scenarios – to consider other solutions than BizTalk for implementing the queries part of the basic ESB. It is possible to develop a “query engine” yourself, but Microsoft Managed Service Engine (MSE) might also be worth a look depending on your needs.

We often hear people say something like: “But implementing queries on BizTalk is easy – just create an orchestration…” and they are right. It is easy to create orchestrations to represent queries, but you will need a good reason for doing that. Orchestrations are expensive in terms of system resources and they are slow compared to messaging only scenarios. And if – as we often see – the exposed end point from the orchestration is identical to the end point on the service it calls, what value is added to make that performance overhead ok? The only value added here is that the calling service only has to know the BizTalk Server end point – the called service can be moved around as long as the BizTalk configuration is updated accordingly.  This functionality is easily obtained in much cheaper ways – e.g. by using MSE.

A good reason for implementing a service using a BTS 2006 Orchestration is by making it an aggregating service. An aggregating service makes calls to two or more sub services to gather the data it needs to respond to the caller. Aggregating services are a valid reason for using BTS 2006 orchestration, but it is not a part of the basic ESB. It is all about connecting services with BTS 2006 as opposed to implementing services on BTS 2006. Aggregated services – and automated processes – implemented on BizTalk can be seen as services that we can connect with the ESB just as the rest of the services in our organization.

Authentications and authorization

BTS 2006 has built-in certificate-based authentication functionality. It is really meant to be used in communication with external partners, but can with some limitations be used in the basic ESB scenario, if all users or services have their own certificates.

In most cases, though it is probably better to use a different authentication mechanism. If you are using an alternative to BTS 2006 for your queries, you will probably want to use the same authentication mechanism in both cases.  When it comes to authorization, BTS 2006 does not really give you anything out of the box. You will have to utilize an external authorization mechanism. The purpose of this article is not to discuss which external authentication or authorization mechanisms to use. We will, however, give a hint as to where these external mechanisms could be used.

BizTalk provides a pipeline architecture on both the receive side and on the send side. This means that you can “do something” with your message before it is actually processed by the message box (receive side). You can also “do something” to your message just before it is sent to the receiver (send side). These “somethings” could be checking with external authentication and authorization mechanisms if the caller is known and is allowed to perform the operation. Since you probably do not want to do unnecessary processing on any messages, the best place to put these checks is on the receive side – in BizTalk terms that means the receive pipeline.

NOTE: Very often authentication is done on a lower level than the ESB by using the operating systems validation of user credentials. This is possible in internal scenarios but makes it impractical to extend the ESB to handle requests from external partners.