Process Message Async

Overview

In Atomic Scope pipeline activities, when many messages are to be processed, sometimes messages are queued up. This delay can impact the overall message processing. To overcome this challenge, Atomic Scope brings the ability to process the messages in an asynchronously way. In such cases, processing is done in the following two steps;

1. Capture the necessary information during the pipeline message processing, and store the information into an intermediate table in the Atomic Scope database

2. Update the tracking activities in the corresponding tables through the Atomic Scope Window NT Service

New property in the Business transaction

To be able to use this feature, Atomic Scope introduces the “Process Message Async” property. This property can be set in every Transaction in the BizTalk resources of the Business process configuration.

The value of this property will be fetched, once per hour, by the Atomic Scope pipeline components. However, to force retrieving that value, you could also restart the host instance of the port/orchestration that runs those components.

By default, this property will be disabled, and the pipeline activity will process the message inside the pipeline component. This is according to the existing (synchronous) implementation.

When the user enables the Process Message Async property, instead of processing the messages in a synchronously way, the Atomic Scope pipeline component stores the necessary information in an intermediate table in the Atomic Scope database. The messages will be further processed through the Atomic Scope Window NT service.

By keeping the work that must be done by the Atomic Scope pipeline components as limited as possible, it helps handling the message load.

Users can prefer either the existing synchronously pipeline activity message handling or the new asynchronously method to handle message load by using this Process Message Async option.

Important Note

1. The Process Message Async feature supports hybrid scenarios

2. Currently, EDI scenarios and nested orchestration scenarios are not supported.

How Atomic Scope handles the Process message async mechanism

1. Push the messages into a table in the Atomic Scope database

Atomic Scope has a table (MessageInPipeline) for the Process Message Async that will contain all details of the raw message. Pipeline and Orchestration activities will push the Message Id, activity details into the table.

2. Process the messages from the Atomic Scope database

The Atomic Scope NT Service picks the messages from the table and processes the messages as like in pipeline activity(activities), and if the message is processed, it will update the “Isprocessed” column in that table.

3. Display the processed message in Atomic Scope portal

There is no variance for displaying the messages in the portal for both mechanisms.

Retry mechanism

It might happen that due to the NT service unexpectedly being down or other technical issues, some messages might not be processed. In such scenarios, a retry mechanism will happen to complete the message processing.

The retry count column in the table MessageInPipeline is used to manage the Retry mechanism. The Retry count column is updated for every attempt of message processing. Once it reaches the Maximum Retry Count, these messages are no longer picked for processing (Dead Lettered Messages).

The maximum retry count

By default, the maximum retry count is 5. The Atomic Scope NT service updates the reason for the failure condition in the Error Description field in the MessageInPipeline table. Errors are also logged in the diagnostic logs.

Purging process

To properly maintain the Atomic Scope database, Data purging is also handled for the Process Message Async feature. As per the purging configuration, processed messages will be deleted from the MessageInPipeline table.

The failed messages will also be deleted by the purging process once the retry count reaches its maximum count.

Dead letter reprocessing

With the Process Message Async functionality, once messages reach a maximum retry count of 5, Atomic Scope considers the message as dead lettered messages, and it is no longer picked for processing.
Users can process the dead letter messages by using the reprocess dead letter option on the tracking overview page.

The user can view all dead letter messages, including message context and content, in the reprocess blade and select which dead letter messages must be processed by Atomic Scope. You can also view the error description for the specific dead letter in the blade.

If the messages, from the reprocess dead letter blade, are succesfully processed, the activities can be viewed in the tracking overview.

FAQ

1. Does the Process Message Async feature support hybrid scenarios?

   Yes, it supports hybrid scenarios.

2. Is it possible to setup both asynchronous message processing and synchronous message processing within the same business process?

Yes,we can configure both asynchronous and synchronous messaging processing within same business process.

3.Will the feature affect any performance for processing messages?

No, with this feature we have improved the performance of message processing. If messages are processed through the NT service, Atomic Scope does not miss any transactions and should not face issues under heavy load.