Happy TPE
Why BAM? Why TPE?
I’m a big believer in BizTalk’s Business Activity Monitoring (BAM) functionality, in fact I can’t think of any “real” BizTalk application that wouldn’t benefit from using it.
I generally use the BAM API along with the excellent Typed BAM Class Generator as a means of persisting business data and timings (aka milestones) to the BAM database. However, last week I had a simple message-only (no orchestrations) process to instrument and I thought “why not give the TPE another chance”?
The TPE, or Tracking Profile Editor, provides a graphical means of identifying the business data and milestones that are to be tracked. Previously, I had lost confidence with the TPE approach because I found it impossible to instrument what was a complex business process requiring multiple request / response ports. At the time, pressure was on for delivery and so I fell back using the BAM API. However, I didn’t give up on the TPE process entirely, since it does have some advantages over the BAM API.
My new process was a simple data extract from an Oracle database. This data is transformed to a flat file format as it is sent to a remote sFTP location. The trigger for the data selection from Oracle is the receipt of a particular file via the File adpater.
The first step when “BAMing” a process is to create a BAM activity model using Excel. After doing this, I then imported my BAM activity definition into the TPE - as shown below:
One thing that needs to be understood when using the TPE for a process of any complexity is the use of continuations. In my case I need a continuation for:
- the trigger file –> oracle select request
- oracle select request –> select result
- select result –> sFTP send
These continuations are illustrated by the ‘expanded’ nodes in the previous screen grabs. The 1st of the “Trigger2Oracle” continuations (without the key icon) is mapped to the File receive port for the trigger file. The 2nd “Trigger2Oracle” continuation is mapped to the send portion of the Oracle request response port. It’s the next piece I think I’ve missed previously. The 1st of the “Select2Result” continuations (without the key icon) is also mapped to the send portion of the Oracle request response port, with the 2nd continuation mapped to the response portion of the same port.
In order to give a full picture, I’ve pasted the XML of the completed tracking profile below:
<?xml version="1.0" encoding="utf-16"?>
<TrackingProfile xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" VersionGuid="00000000-0000-0000-0000-000000000000" Name="Silverpop">
<Dimension Name="ActivityID" DataType="TraceID" />
<Dimension Name="FileContentType" DataType="NVARCHAR">
<DataLevel Name="ReceivePortName" SourceTypeSelected="Messaging Context Property" SchemaName="BTS.bts_system_properties,Microsoft.BizTalk.GlobalPropertySchemas, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://schemas.microsoft.com/BizTalk/2003/system-properties']/*[local-name()='ReceivePortName' and namespace-uri()='http://schemas.microsoft.com/BizTalk/2003/system-properties']" XPath="/*[local-name()='ReceivePortName' and namespace-uri()='http://schemas.microsoft.com/BizTalk/2003/system-properties']">
<Port Name="RcvSilverpopCustServey" Direction="Receive" />
</DataLevel>
</Dimension>
<Dimension Name="TimeRequestedFromOracle" DataType="DATETIME">
<DataLevel Name="PortStartTime" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']">
<Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Send" />
</DataLevel>
</Dimension>
<Dimension Name="TimeReceivedFromOracle" DataType="DATETIME">
<DataLevel Name="PortEndTime" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortEndTime' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortEndTime' and namespace-uri()='']">
<Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Receive" />
</DataLevel>
</Dimension>
<Dimension Name="TimeSentToSilverpop" DataType="DATETIME">
<DataLevel Name="PortStartTime" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']">
<Port Name="SilverpopSendCustSurvey_sFTP" Direction="Send" />
</DataLevel>
</Dimension>
<Dimension Name="SentFilename" DataType="NVARCHAR">
<DataLevel Name="MessageID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageID' and namespace-uri()='']">
<Port Name="SilverpopSendCustSurvey_sFTP" Direction="Send" />
</DataLevel>
</Dimension>
<Dimension Name="TimeOfException" DataType="DATETIME">
<DataLevel Name="PortStartTime" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']">
<Port Name="ExceptionHandling.SubmitFault" Direction="Receive" />
</DataLevel>
</Dimension>
<Dimension Name="InterchangeId" DataType="NVARCHAR">
<DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
<Port Name="RcvSilverpopCustServey" Direction="Receive" />
</DataLevel>
</Dimension>
<Dimension Name="Trigger2Oracle" DataType="Continuation">
<DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
<Port Name="RcvSilverpopCustServey" Direction="Receive" />
</DataLevel>
</Dimension>
<Dimension Name="Trigger2Oracle" DataType="ContinuationID">
<DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
<Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Send" />
</DataLevel>
</Dimension>
<Dimension Name="Select2Result" DataType="Continuation">
<DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
<Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Send" />
</DataLevel>
</Dimension>
<Dimension Name="Select2Result" DataType="ContinuationID">
<DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
<Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Receive" />
</DataLevel>
</Dimension>
<Dimension Name="Result2sFTP" DataType="Continuation">
<DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
<Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Receive" />
</DataLevel>
</Dimension>
<Dimension Name="Result2sFTP" DataType="ContinuationID">
<DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='<Schema>' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
<Port Name="SilverpopSendCustSurvey_sFTP" Direction="Send" />
</DataLevel>
</Dimension>
</TrackingProfile>
Once deployed, this tracking profile worked perfectly! It resulted in a single “Completed” activity record (containing milestones for receipt of trigger, request & response from Oracle and send via sFTP) for each trigger file processed. My previous efforts had resulted in multiple records, some of which never made their way from the “Active” to “Completed” BAM tables. If you run into a similar issue, it may be worth checking you haven’t missed a continuation.
Ensure Send Completes
I’ve found that another cause of rows not making their way from the Bam Active to Completed table is because the process does not complete. One example of this was simple messaging only solution. I had a send port with a filter set to subscribe to any message from a particular receive port. The send made a request response to a web service. Initially, I didn’t add a subscriber for the response returned from the web service. This resulted in a routing failure report in BizTalk - to be expected. However, what was slightly unexpected was that when I checked the Bam tables, I could see a record sat in Active but nothing in Completed. I guess this is because the send port did actually “Complete”, due to the fact that its response could not be routed. Once I added a subscriber (simple send through a null pipeline) then the Bam Completed table was populated and the Active table was empty :)
Comments