bouchaet
2009-04-24 18:32:57 UTC
I like to share my spike with HL7 in case some of you would be
interested to share some thoughts.
The curious issue :
Where is the original HL7 message? The ___messages are not tracked as
an original flat-file HL7___ when using HAT with the MLLP adapter.
And the real deal:
1 Context
An integration solution with BizTalk Server 2006/2006 R2 is deployed
in production. A new version of the solution was completed and ready
to be deployed. However it is not possible to fully assert its quality
without running messages in production. I know, please do not "5whys"
me on this (http://en.wikipedia.org/wiki/5_Whys). For now, it must be
seen as a constraint. The pre-production environment cannot have
external systems that would reliably reproduce the production
environment. Production messages are highly valuable and must not be
ruined by the system. A backup plan is mandatory.
The nature of the communication, HL7 exchange, yield to another
functional constraint: the receiving sequence of the messages must be
maintain for the resubmission.
If a system back out must be held, no production messages should be
lost. Moreover, the messages must be re-injected once the back out is
completed. BizTalk Server can route ingoing messages to a dedicated
database of its tracking system.
2 Problem
How this system performs in a critical scenario and how could it be
used to resubmit messages?
3 Forces
# The archiving mechanism does not involve a new development;
# The receiving sequence is known and it can be reproduced during the
resubmission;
# The archiving mechanism is not permanent and can be de-activated;
# The original message is not modified and can be inspected as-is by
an administrator;
4 Current Finding
4.1 The Test set-up
To perform a test, an existing in-production application was used in
DEV environment. This application already has a suite of performance
tests written with loadgen [http://www.microsoft.com/downloads/
details.aspx?FamilyID=c8af583f-7044-48db-
b7b9-969072df1689&displaylang=en]. However, the tests does not clearly
identify the sequence of messages that is sent to the BizTalk engine
since it is using a file adapter on its receiving side. A new test
suite was created which runs from a batch file and works with a MLLP
receive port, an adapter that supports ordered delivery. The batch
file launches the mllpsend tool shipped with the HL7 accelerator.
4.2 Exploration One: Using HAT UI
* Tracking is enabled from the receive port;
* The messages can be saved from the UI with a right-click. However,
there is two major problem:
** the ___original messages is not saved___ in the Tracking database.
Only the XML version is available;
** the UI does not allow a multi-selection. In a scenario where there
is many messages (> 50), this is simply not workable on an
administration perspective;
* A tool on Codeplex, a plugin for HAT [http://
btsviewerhatplugin.codeplex.com] solves the previous multi-selection
problem. However, after few tests, the plugin does not work as
documented on the site. The messages are simply not shown in the UI
for no clear reason;
4.3 Exploration Two: Using Custom UI
* Tracking is enabled from the receive port;
* The UI client was download from Thiago Almeida blog [http://
connectedthoughts.wordpress.com/2008/04/02/3-ways-of-programatically-
extracting-a-message-body-from-the-biztalk-tracking-database/]. See an
image here [http://connectedthoughts.files.wordpress.com/2008/04/
trackedmessagewindowsform.jpg];
* However, to see the message, the user must know the message guid;
* No multi-selection available from the UI but the article presents
how messages can be retrieved from custom code;
** The tracking system offers theses views :
*** [BizTalkDTADb].[dbo].[btsv_Tracking_Parts] : this view holds the
data for the message (the body) in its imgPart column;
*** [BizTalkDTADb].[dbo].[btav_MessagesFacts] : this view contains all
the information related to the message like the adapter type [Event/
Adapter], the port name [Event/Port], the timestamp [Event/Timestamp]
(which could be used for the sequencing), [Event/Direction]
** The tracking system offers stored procedures for tracked messages:
*** [BizTalkDTADb].[dbo].[GetTrackedMessage]
*** [BizTalkDTADb].[dbo].[GetTrackedMessageFragments]
*** [BizTalkDTADb].[dbo].[GetTrackedMessageParts]
5 Early Conclusions
5.1 Where is the original HL7 message?
The big issue is that ___messages are not tracked as an original flat-
file HL7___. This issue may discard the BTS tracking system as a
potential backbone for an archiving-and-resubmission feature.
5.2 Multiple is not :
Out-of-the-box, HAT UI cannot provide a solution for message. The
messages are indeed archived in its database and are not so hard to
reach individually. However, it is not a workable solution for
multiple instances.
5.3 Resubmission needs thoughts :
Both HAT UI and Codeplex Tracked Message UI do nothing for
resubmission. In the best case, messages can be saved to a folder.
Nevertheless, these files are useless since messages retrieved from
the FILE adapter cannot support ordered delivery. However, the
tracking views in the BizTalkDTADb database can be queried to retrieve
an ordered sequence of messages for a given adapter or port name.
Theses messages could then be sent to an ordered aware adapter.
Any comments?
Thanks,
etienne
bouchaet.blogspot.com
interested to share some thoughts.
The curious issue :
Where is the original HL7 message? The ___messages are not tracked as
an original flat-file HL7___ when using HAT with the MLLP adapter.
And the real deal:
1 Context
An integration solution with BizTalk Server 2006/2006 R2 is deployed
in production. A new version of the solution was completed and ready
to be deployed. However it is not possible to fully assert its quality
without running messages in production. I know, please do not "5whys"
me on this (http://en.wikipedia.org/wiki/5_Whys). For now, it must be
seen as a constraint. The pre-production environment cannot have
external systems that would reliably reproduce the production
environment. Production messages are highly valuable and must not be
ruined by the system. A backup plan is mandatory.
The nature of the communication, HL7 exchange, yield to another
functional constraint: the receiving sequence of the messages must be
maintain for the resubmission.
If a system back out must be held, no production messages should be
lost. Moreover, the messages must be re-injected once the back out is
completed. BizTalk Server can route ingoing messages to a dedicated
database of its tracking system.
2 Problem
How this system performs in a critical scenario and how could it be
used to resubmit messages?
3 Forces
# The archiving mechanism does not involve a new development;
# The receiving sequence is known and it can be reproduced during the
resubmission;
# The archiving mechanism is not permanent and can be de-activated;
# The original message is not modified and can be inspected as-is by
an administrator;
4 Current Finding
4.1 The Test set-up
To perform a test, an existing in-production application was used in
DEV environment. This application already has a suite of performance
tests written with loadgen [http://www.microsoft.com/downloads/
details.aspx?FamilyID=c8af583f-7044-48db-
b7b9-969072df1689&displaylang=en]. However, the tests does not clearly
identify the sequence of messages that is sent to the BizTalk engine
since it is using a file adapter on its receiving side. A new test
suite was created which runs from a batch file and works with a MLLP
receive port, an adapter that supports ordered delivery. The batch
file launches the mllpsend tool shipped with the HL7 accelerator.
4.2 Exploration One: Using HAT UI
* Tracking is enabled from the receive port;
* The messages can be saved from the UI with a right-click. However,
there is two major problem:
** the ___original messages is not saved___ in the Tracking database.
Only the XML version is available;
** the UI does not allow a multi-selection. In a scenario where there
is many messages (> 50), this is simply not workable on an
administration perspective;
* A tool on Codeplex, a plugin for HAT [http://
btsviewerhatplugin.codeplex.com] solves the previous multi-selection
problem. However, after few tests, the plugin does not work as
documented on the site. The messages are simply not shown in the UI
for no clear reason;
4.3 Exploration Two: Using Custom UI
* Tracking is enabled from the receive port;
* The UI client was download from Thiago Almeida blog [http://
connectedthoughts.wordpress.com/2008/04/02/3-ways-of-programatically-
extracting-a-message-body-from-the-biztalk-tracking-database/]. See an
image here [http://connectedthoughts.files.wordpress.com/2008/04/
trackedmessagewindowsform.jpg];
* However, to see the message, the user must know the message guid;
* No multi-selection available from the UI but the article presents
how messages can be retrieved from custom code;
** The tracking system offers theses views :
*** [BizTalkDTADb].[dbo].[btsv_Tracking_Parts] : this view holds the
data for the message (the body) in its imgPart column;
*** [BizTalkDTADb].[dbo].[btav_MessagesFacts] : this view contains all
the information related to the message like the adapter type [Event/
Adapter], the port name [Event/Port], the timestamp [Event/Timestamp]
(which could be used for the sequencing), [Event/Direction]
** The tracking system offers stored procedures for tracked messages:
*** [BizTalkDTADb].[dbo].[GetTrackedMessage]
*** [BizTalkDTADb].[dbo].[GetTrackedMessageFragments]
*** [BizTalkDTADb].[dbo].[GetTrackedMessageParts]
5 Early Conclusions
5.1 Where is the original HL7 message?
The big issue is that ___messages are not tracked as an original flat-
file HL7___. This issue may discard the BTS tracking system as a
potential backbone for an archiving-and-resubmission feature.
5.2 Multiple is not :
Out-of-the-box, HAT UI cannot provide a solution for message. The
messages are indeed archived in its database and are not so hard to
reach individually. However, it is not a workable solution for
multiple instances.
5.3 Resubmission needs thoughts :
Both HAT UI and Codeplex Tracked Message UI do nothing for
resubmission. In the best case, messages can be saved to a folder.
Nevertheless, these files are useless since messages retrieved from
the FILE adapter cannot support ordered delivery. However, the
tracking views in the BizTalkDTADb database can be queried to retrieve
an ordered sequence of messages for a given adapter or port name.
Theses messages could then be sent to an ordered aware adapter.
Any comments?
Thanks,
etienne
bouchaet.blogspot.com