The standard feature for tracking messages in a BizTalk server is the BizTalk Server 2009 Administration Console (which now includes the query capabilities of the tool formerly known as Health and Activity Tracking, or HAT). However, in large, multiserver BizTalk systems, Administration Console queries can cause performance issues under some circumstances; the large volume of data generated can have an impact on the TDDS service and backup jobs. Because of that, many BizTalk users have chosen to move away from Administration Console queries and implement a BAM solution for monitoring their BizTalk environment. You can use BAM to create an overview of the performance and behavior of a BizTalk server.
In this chapter, we describe one possible approach to tracking messages flowing through BizTalk Server. This approach works with both messaging-only applications and orchestrations, and can be implemented by using custom pipeline components or by using the Tracking Profile Editor (TPE).
Note This is a solution for tracking messages; it’s not a full health-monitoring solution for a BizTalk server farm. To monitor the health of your BizTalk servers, consider acquiring a server management product. The BizTalk Management Pack for System Center Operations Manager (SCOM) provides very detailed health monitoring for BizTalk; in our opinion, it’s the best tool out there. Other server management tools, such as HP’s OpenView, also provide management packs for BizTalk, but in our experience the SCOM tools provide much more detail on what’s happening now in your BizTalk group.
Note To disable HAT-style tracking exposed within the BizTalk Server Administration Console, use SQL Server Management Studio to open the adm_group table in BizTalkMgmtDB. This table contains only one row. Set the value of the GlobalTrackingOption field in that row to 0 to disable tracking, 1 to reenable tracking.
Observation Model
Data will be captured using two related activities; one activity is used for inbound messages, and the other is used for outbound messages.
The inbound activity contains the items shown in Table 14-1 and is shown in Figure 14-1.
Figure 14-1. The inbound activity
The outbound activity contains the items shown in Table 14-2.
Two activities are used because each inbound message can potentially generate multiple outbound messages through BizTalk’s publish-subscribe model. A one-to-many relationship exists between the inbound and outbound activities, implemented using BAM relationships. In your environment, if you are certain that each inbound message will never generate more than one outbound message, you can simplify the implementation by using a single activity that contains both inbound and outbound data. In this case, you would replace the one-to-many relationship with a continuation.
Note Instructions for creating activities and views are in Chapter 5. Instructions for implementing relationships are in Chapter 12.
Obviously, several aggregate views can be created with this data. Message size and message count are useful measures, and time, message type, and port are useful dimensions. We usually start by creating a simple aggregate view that describes the inbound data, as shown in Figure 14-2. The measures and dimensions we use are shown in Tables 14-3 and 14-4.
Figure 14-2. The aggregated monitoring data
Tracking Profile Implementation
In the simplest implementation, two tracking profiles are used. One tracking profile captures inbound information, and the other tracking profile captures outbound information.
In each tracking profile, five of the six items in the activity come from the messaging property schema, as shown in Figure 14-3. This schema can be selected by pressing the Select Event Source button in the TPE and contains the following items:
- InterchangeID
- MessageID
- PortStartTime
- PortEndTime
- Size
In the inbound activity, the ReceivePort field is captured from BTS.ReceivePortName in the BTS.bts_system_properties context property schema. In the outbound activity, the Send-Port field is captured from BTS.SPName in BTS.bts_system_properties. In both schemas, the message type field is captured from BTS.MessageType in BTS.bts_system_properties.
Note The process for creating tracking profiles is described in Chapter 6.
In addition to capturing the six fields in each activity, a relationship between the two activities is needed. This is implemented by using the interchange ID, which is available through the InterchangeID field in the messaging property schema. The interchange ID is used as the activity ID in the inbound activity. The interchange ID is also used in the inbound relationship node in the outbound activity, as shown in Figure 14-4. In the outbound activity, leave the activity ID node empty, and BAM will generate a GUID as the activity ID for each item in the activity.
Figure 14-3. The inbound activity tracking profile in TPE
Figure 14-4. The outbound activity tracking profile
Note The process for implementing relationships between activities using tracking profiles is described in Chapter 12.
InterchangeID works for creating this relationship because the interchange ID is preserved as the message is published to the message box and then picked up by its subscribers. If the application is a messaging-only solution, the interchange ID at the send port will be the same as the interchange ID at the receive port. If the application contains an orchestration, the interchange ID of a message sent by an orchestration will be the same as the interchange ID of the message that activated the orchestration.
A port mapping is needed for each of the seven items in the activity. If you want to monitor every port in the system, that’s easy to set up. In the inbound tracking profile, right-click an item in the TPE, and select Set Port Mapping. In the Select Ports dialog box, select the Show Receive Ports check box and clear the Show Send Ports check box to display a list of receive ports (see Figure 14-5). Click the >> button, and all the receive ports on your server will be monitored using the tracking profile. For the outbound tracking profile, simply select all the send ports but not the receive ports.
Figure 14-5. Setting the port mappings
Pipeline Component Implementation
As an alternative implementation, it would be possible to create custom pipeline components that used the BAM API to capture data. This would perform slightly less well than using the TPE to gather the data, as the TPE is very efficient. In most situations, the TPE is the preferred solution. Some organizations, however, may prefer to avoid administering and applying tracking profiles. In this case, it is possible to implement a similar solution using two pipeline components. One component is used in the receive pipeline, and the other component is used in the send pipeline.
Note Detailed discussion of pipeline component implementation is beyond the scope of this book. For a simple example of a custom pipeline component, refer to the book BizTalk 2006 Recipes: A Problem-Solution Approach by Mark Beckner et al. (Apress, 2006). (Most of the boilerplate code in this chapter was taken from this title.) For a more detailed discussion of the pipeline object model, see the book Pro BizTalk 2006 by George Dunphy and Ahmed Metwally (Apress, 2006).
The code in Listing 14-1 implements a pipeline component that captures the inbound activity using the BAM API.
Listing 14-1. Code for the Receive Pipeline Component
using System;
using System.Text;
using System.Resources;
using System.Reflection;
using Microsoft.BizTalk.Message.Interop;
using Microsoft.BizTalk.Component.Interop;
using Microsoft.BizTalk.Bam.EventObservation;
namespace InterchangeID
{
[ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
[ComponentCategory(CategoryTypes.CATID_Any)]
//The GUID should be unique, so run GUIDGEN and
//replace this GUID with your own
[System.Runtime.InteropServices.Guid("1081A6CA-44E5-45ef-9D1D-E144FD9C5869")]
public class InterchangeIDComponent :
Microsoft.BizTalk.Component.Interop.IBaseComponent,
Microsoft.BizTalk.Component.Interop.IComponent,
Microsoft.BizTalk.Component.Interop.IComponentUI,
Microsoft.BizTalk.Component.Interop.IPersistPropertyBag
{
#region Constructor
public InterchangeIDComponent()
{
;
}
#endregion
#region IBaseComponent Members
string IBaseComponent.Description
{
get { return "Captures Status to BAM"; }
}
string IBaseComponent.Name
{
get { return "RcvBAM"; }
}
string IBaseComponent.Version
{
get { return "1.0"; }
}
#endregion
#region IComponent Members
IBaseMessage IComponent.Execute
(IPipelineContext pContext, IBaseMessage pInMsg)
{
//Get the properties we need
string sysProps =
"http://schemas.microsoft.com/BizTalk/2003/system-properties";
string interchangeID = pInMsg.Context.Read(
"InterchangeID",
sysProps).ToString();
string messageID = pInMsg.Context.Read(
"MessageID",
sysProps).ToString();
string portStartTime = pInMsg.Context.Read(
"PortStartTime",
sysProps).ToString();
string portEndTime = pInMsg.Context.Read(
"PortEndTime",
sysProps).ToString();
string messageSize = pInMsg.Context.Read(
"Size",
sysProps).ToString();
string receivePort = pInMsg.Context.Read(
"BTS.ReceivePortName",
sysProps).ToString();
string messageType = pInMsg.Context.Read(
"BTS.MessageType",
sysProps).ToString();
//Get an instance of MessagingEventStream
EventStream eventstream = pContext.GetEventStream();
//Write to the event stream
eventstream.BeginActivity("Inbound", interchangeID);
eventstream.UpdateActivity("Inbound", interchangeID,
"InterchangeID", interchangeID,
"MessageID", messageID,
"PortStartTime", portStartTime,
"PortEndTime", portEndTime,
"Size", messageSize,
"ReceivePort", receivePort,
"MessageType", messageType);
eventstream.EndActivity("Inbound", interchangeID);
//Return message
return pInMsg;
}
#endregion
#region IComponentUI Members
IntPtr IComponentUI.Icon
{
get { return IntPtr.Zero; }
}
System.Collections.IEnumerator IComponentUI.Validate(object projectSystem)
{
return null;
}
#endregion
#region IPersistPropertyBag Members
void IPersistPropertyBag.GetClassID(out Guid classID)
{
//Should match the GUID in the
//System.Runtime.InteropServices.Guid attribute above
classID = new Guid(
"1081A6CA-44E5-45ef-9D1D-E144FD9C5869");
}
void IPersistPropertyBag.InitNew()
{
;
}
void IPersistPropertyBag.Load(IPropertyBag propertyBag, int errorLog)
{
;
}
void IPersistPropertyBag.Save(IPropertyBag propertyBag,
bool clearDirty, bool saveAllProperties)
{
;
}
#endregion
}
}
The vast majority of this pipeline component is completely standard boilerplate. The method that does the actual work is the IComponent.Execute method. This method gets the items we need from the message context, gets an instance of MessagingEventStream from the pipeline context, and then uses the BAM API to write the data to the activity. The interchange ID is used as the activity ID, just as in the tracking profile implementation.
A separate pipeline component is used for the send pipeline, as it needs to create the relationship. All the code is the same as in the receive pipeline, except for the IComponent.Execute method, which is implemented using the code shown in Listing 14-2.
Listing 14-2. The Code for the IComponent.Execute Method in the Send Pipeline Component
IBaseMessage IComponent.Execute(IPipelineContext pContext,
IBaseMessage pInMsg)
{
//Get the properties we need
string sysProps =
"http://schemas.microsoft.com/BizTalk/2003/system-properties";
string interchangeID = pInMsg.Context.Read(
"InterchangeID",
sysProps).ToString();
string messageID = pInMsg.Context.Read(
"MessageID",
sysProps).ToString();
string portStartTime = pInMsg.Context.Read(
"PortStartTime",
sysProps).ToString();
string portEndTime = pInMsg.Context.Read(
"PortEndTime",
sysProps).ToString();
string messageSize = pInMsg.Context.Read(
"Size",
sysProps).ToString();
string sendPort = pInMsg.Context.Read(
"BTS.SPName",
sysProps).ToString();
string messageType = pInMsg.Context.Read(
"BTS.MessageType",
sysProps).ToString();
//Get an instance of MessagingEventStream
EventStream eventstream = pContext.GetEventStream();
//activityGuid will be used to uniquely identify this row in the table
string activityGuid = Guid.NewGuid().ToString();
//Write to the event stream
eventstream.BeginActivity("Outbound", activityGuid);
eventstream.UpdateActivity("Outbound", activityGuid,
"InterchangeID", interchangeID,
"MessageID", messageID,
"PortStartTime", portStartTime,
"PortEndTime", portEndTime,
"Size", messageSize,
"SendPort", sendPort,
"MessageType", messageType);
//Set up the relationship
eventstream.AddRelatedActivity(
"Outbound", activityGuid,
"Inbound", interchangeID);
eventstream.EndActivity("Outbound", activityGuid);
//Return message
return pInMsg;
}
In the send pipeline component, the activity ID is a globally unique identifier (GUID) that is generated randomly. The call to UpdateActivity uses the SendPort field instead of the ReceivePort field. The additional method call used is the call to AddRelatedActivity, which establishes a relationship between the inbound and outbound activities.
Note Instructions for creating relationships between activities using the BAM API are in Chapter 12.
BAM Portal
Because of the related activities, you can use the BAM Portal to navigate from inbound messages to outbound messages and examine the outbound messages that were created by each inbound message. The Activity Search page, shown in Figure 14-6, allows the inbound and outbound activities to be searched.
Once you have located the activity of interest, clicking the activity displays activity details plus a list of related documents (see Figure 14-7).
Figure 14-6. The Activity Search page
Figure 14-7. The Activity Detail page showing related activities
The aggregated data allows a summary of messages by message type or receive port, optionally filtered by receive date, as you can see in Figure 14-8.
Figure 14-8. Aggregated BAM tracking data
Extending the Monitoring Solution
The solution described in this chapter can be extended in four main ways:
- Capturing more data
- Adding additional views
- Integrating system monitoring with business process monitoring
- Using BAM alerts
Capturing More Data
The set of data captured in the activities was very small; the data was kept small to minimize the impact on BizTalk’s performance. As your experience of managing BizTalk Server grows, you will probably want to add some additional data to the inbound and outbound activities. Obviously, you should add whatever data is needed by your management process, while recognizing that capturing more fields will increase the space requirements of your BAM databases.
Some of the obvious candidates for tracking are shown in Table 14-5. If you make heavy use of a specific adapter, you may want to also add context properties for that adapter. As an example, if all your data flows through IBM’s WebSphere MQ, you will probably want to add context properties provided by that adapter. If other data from the message context is useful to you, go ahead and add it.
Adding Additional Views
Once you have added additional data, additional views will also be useful to analyze the data. As an example, the aggregated view shown in Figure 14-2 displays only received data. In some cases, this may underestimate the total data processed by BizTalk Server, as one inbound message may generate many outbound messages. You may want to add a second aggregated view to show outbound message counts and sizes. Other aggregated views may also prove useful.
Integrating System Monitoring with Business Process Monitoring
In addition to the health tracking data described in this chapter, you may also be using BAM to capture data on a specific business process, with an activity that contains data that is specific to your process. You can link the business process activities with the BizTalk monitoring activities using the interchange ID. Whenever you implement a BAM activity, consider adding a relationship node that relates that activity to the inbound activity. The relationship will allow you to navigate from the business process activity to the inbound monitoring activity, and then you can navigate from the inbound activity to outbound messages associated with your business process. By creating these links, you add additional investigative capabilities to any business process view you create.
Using BAM Alerts
To create a full monitoring solution, you should supplement the BAM data with BAM alerts. Adding alerts into the solution allows administrators to be notified of unusual conditions. As an example, an administrator might want to be notified if the message load on a specific port is greater than 10,000 messages per hour.
The correct alerts for your environment will need to be determined from your specific traffic flows, but once the solution described in this chapter is in place, it’s easy to create alerts as needed.
Summary
In this chapter, you learned how to track BizTalk messages using BAM. A specific observation model was recommended for capturing statistics on messages flowing through your BizTalk group, and two implementations were provided. One of the implementations used tracking profiles, and the other used pipeline components. No matter how the data is captured, the BAM Portal can be used to view tracked data.
The last section of the chapter provided some ideas for extending the solution, including adding additional data, additional views, integrating with other BAM activities, and adding BAM alerts.
Chapter 15, the final chapter in the book, describes BAM best practices and provides some perspective on future Microsoft technologies for monitoring business events.