Chapter 16. Two-Party Coordination Patterns
Once two parties are talking to one another, typically the next challenge is to coordinate their activities. Coordination defines how the execution of an activity in one component is related to the execution of another activity in another component. How you go about coordinating activities impacts both normal behavior and the ability to detect breakdowns in the process. In terms of breakdown detection, you want to know which party (if any) is in a position to detect the breakdown, and how they know that a breakdown occurred.
This chapter examines a number of design patterns that can be used to coordinate the activities of two parties, and the next chapter explores how these patterns generalize to three or more parties. The patterns discussed in this chapter are:
• Delegation with Confirmation
• Third-Party Process Coordinator
Fire-and-Forget Coordination
The simplest possible coordination pattern is fire-and-forget (Figure 16-1). In this pattern, one party simply sends a message to the other, and there is no other communication. Note that either the In-Out or Out-In interaction pattern can be used to implement this coordination.
Figure 16-1. Fire-and-Forget Coordination
The advantage of fire-and-forget is that it is simple to implement. Its disadvantage is that neither party is in a position to know whether the interaction succeeded. The sender has no idea whether the recipient received the message or took appropriate action. The recipient has no idea whether a message was sent. Breakdown detection is impossible with this pattern. Improving the communications channel quality of service does not improve the breakdown detection—it only makes a breakdown less likely.
This pattern can be implemented with any of the ActiveMatrix-supported protocols: SOAP In-Only (any transport), and XML over JMS In-Only or Out-Only. File-based interactions also follow this pattern, along with e-mail and any other form of one-way messaging. Many TIBCO Adapter interactions are fire-and-forget as well.
Despite the limitations of this pattern, it is still possible to implement a relatively robust business process with it. Consider the process of paying your bills by mail. You receive your bill via the mail—a one-way communication. You pay your bill via the same one-way service. However, the business process is tolerant of breakdowns. If, for any reason (loss of the bill, your failure to pay, or loss of the payment), your bill does not get paid, the past-due amount is simply added to the next month’s bill.
Request-Reply Coordination
In the synchronous variation of request-reply coordination (Figure 16-2), the service consumer sends a request to the service provider and waits for a reply. With this pattern, the receipt of the reply indicates the status of the request: Either the service provider performed the required service or it failed.
Figure 16-2. Synchronous Request-Reply Coordination
There is, however, a third possible outcome in this exchange: the absence of a reply. Without a service-level agreement (SLA) specifying the time frame within which a reply is expected, the absence of a reply is ambiguous with respect to breakdown detection. The service consumer cannot draw any conclusions about the state of the process. With the SLA, however, the situation clarifies: Within the specified time frame, the absence of a reply is acceptable; beyond that time frame, there is a problem. Thus it becomes imperative in any request-reply exchange to establish a response-time SLA for the service provider.
Although the request-reply coordination pattern (with an established response-time SLA) can detect breakdowns, it does not provide sufficient information in all cases to determine the nature of the breakdown. For example, the loss of the request, the failure of the service provider to fulfill the request, or the loss of the reply will all result in the same symptom: the absence of a reply.
The synchronous pattern can be implemented with any of the ActiveMatrix-supported protocols: SOAP In-Out (any transport), and XML over JMS In-Out or Out-In. JDBC interactions follow this pattern, along with HTTP, CORBA, COM, and JRMI interactions. Some TIBCO Adapter interactions are request-reply as well.
Asynchronous variations of this pattern are also possible, but with a limited choice of protocols (see the discussion of the asynchronous variations of the In-Out interaction pattern in Chapter 11). Asynchronous designs are considerably more complicated and tend to be employed only when the time frame for performing the service is long enough to make it impractical for the service consumer to wait for completion.
Delegation
There are situations in which the length of time it takes to perform the service makes it impractical for the service consumer to wait around for a response. Yet, at the same time, you may want to ensure that the service provider got the request and agreed to perform the service. This is the purpose of the delegation pattern (Figure 16-3).
Figure 16-3. Delegation Pattern
In this pattern the service provider does, indeed, reply to the service consumer, but the reply simply indicates that the request has been received and that the service provider promises to perform the requested service at some point in the future. Note that this is a bit stronger than a mere acknowledgment that the request was received: Due diligence on the part of the service provider requires that the request be recorded in such a way that the service provider does not lose it. Presumably there is also some obligation to ensure that the pending requests are not ignored either. It is common in this pattern for the actual result to go to a third party rather than back to the service consumer.
This pattern can be viewed as a combination of the request-reply pattern (for receiving the request and returning the promise) and the fire-and-forget pattern (for performing the service and delivering the result). Its breakdown-detection capabilities reflect exactly this. Assuming that there is a response-time service-level agreement governing the return of the promise, the service consumer is in a position to determine whether the service provider has agreed to perform the work. However, no breakdown detection is possible beyond that point.
Delegation with Confirmation
The delegation with confirmation pattern extends delegation to include an eventual asynchronous response to the service consumer (Figure 16-4). The addition of this feedback (assuming that a response-time SLA is in place for this as well) now puts the service consumer in a position to verify whether or not the work was properly performed—but not necessarily to determine whether the result was properly delivered.
Figure 16-4. Delegation with Confirmation Pattern
There are several variations of this pattern worth describing. The one depicted in Figure 16-4 uses fire-and-forget to deliver the service result. Consequently, the eventual confirmation to the service consumer only indicates that the result was sent—not that it was received. In another variation, the work result is sent back to the service consumer and serves as the confirmation as well. In this case, the service consumer knows with certainty whether the entire process completed successfully. In a third variation, the service result still goes to a third party, but the third-party exchange uses request-reply. Once again, the service consumer now knows with certainty whether the entire process concluded successfully.
One challenge with this pattern is the mechanism for asynchronous delivery of the confirmation to the service consumer. Ideally, this is an Out-Only interaction, with the initial request containing the destination to which the confirmation should be sent. This, however, limits the ActiveMatrix transport choice to XML over JMS or requires the use of a non-ActiveMatrix-supported protocol such as e-mail.
Alternatively, the confirmation could be delivered as an In-Only transaction. However, this requires that the service provider be designed to call the specific interface provided by the service consumer. In other words, the service provider is customized for specific service consumers. Some flexibility can be achieved here if all service consumers are required to have the same logical interface (i.e., the same logical WSDL). In such cases a mediation flow with dynamic routing (Figure 14-15) could be used to return the confirmations.
Distributed Transactions
The intent of a transaction is to provide all-or-nothing behavior: Either all participants perform their actions or none of them perform their actions. A transaction requires a manager to govern the execution of the transaction. Depending upon the implementation choices, the manager role may be played by one of the transaction participants or it may be played by a third party. The manager interacts with the participants via some protocol, such as the Open Group’s XA standard.1
Two-Phase Commit
The standard approach to implementing distributed transactions is to use a two-phase commit protocol (Figure 16-5), with the XA standard being one of the leading examples. A transaction begins with an application program (using XA terminology) initiating the transaction. In the example, the First Party is playing the role of the application program. At this point the transaction manager issues an identifier for the transaction, which is carried through all subsequent interactions related to the transaction. The various parties involved in the transaction (XA resource managers) are each directed to perform their operations by the application program, and each registers with the transaction manager. Note that in this example the First Party is also playing the role of a resource manager. Resource managers do not make the results of their work visible at this time.
Figure 16-5. Simplified Example of a Two-Phase Commit Protocol
When all of the operations have been completed, the application program tells the transaction manager to commit the transaction. The transaction manager now interacts with the transaction participants (XA resource managers) in two phases. In the first phase, each party is asked to vote on whether the transaction should be committed. Assuming all parties vote yes, each party is then asked to make their changes permanent and visible. If anyone votes no, all parties are asked to roll back the transaction.
It is not a simple matter for a party to be a participant (resource manager) in the XA protocol. Each participant must be able to logically do the work without making the results either visible or permanent. When the party votes yes, it is obligated to make a stable (i.e., persistent) record of the information it needs to eventually commit, again without making the results visible.
The consequence of this complexity is that there are, in reality, few systems capable of participating in two-phase commit transactions, with databases being the most common. This, in itself, limits the usability of the two-phase commit protocol. At the time of this writing, composites of interacting ActiveMatrix components can only be grouped into a transaction if they are deployed in the same node.
In the meantime, this does not prevent an individual component from playing the role of the application program and possibly that of a resource manager as well. TIBCO ActiveMatrix BusinessWorks is well suited for this role. It includes an XA transaction manager and a number of activities (including JDBC database interactions, and the sending and receiving of JMS messages) that can be transaction participants. This allows BusinessWorks to play the roles of application, resource manager, and transaction manager. It can also work with an external XA transaction manager.
Messaging and Transactions
The coordination of message sending and receipt within a transaction requires careful consideration. Sending a message in a nontransactional manner will not recall that message in the event of a transaction rollback. Similarly, receiving a message in a nontransactional manner will consume the message even if the transaction is rolled back. TIBCO ActiveMatrix BusinessWorks allows you to send and receive messages as part of the transaction. For message sending, the message is given to the EMS server but not released for delivery until the transaction commits. A transaction rollback or failure (such as the restart of the BW engine in the middle of the transaction) will result in the message being deleted without being delivered. For message receipt, the acknowledgment of message receipt is not given to the EMS server until the transaction commits. A transaction rollback or failure will result in the message being re-delivered to the next eligible recipient.
As things stand today, when messaging is involved, the scope of the transaction boundary does not extend beyond the messaging server. In particular, the activities of the party receiving a message sent as part of a transaction are not within the scope of the transaction. Thus, your design must take this into consideration. It is worth noting that the TIBCO ActiveMatrix Policy Framework (discussed in Chapter 13) and emerging public standards such as WS-AtomicTransaction are beginning to lay the foundation for extending transactions in this manner, but under most circumstances such transactions are not practical today.
Distributed Transaction Limitations
The all-or-nothing behavior of transactions has some limitations as well:
• All parties must be up and running for the transaction (and thus the business process) to proceed. If any of the parties is unavailable (system down, network problems, etc.), nothing happens. For many business processes, this is not the desired behavior.
• The protocol does not address what to do if the transaction fails: The transaction manager simply informs the application program of the failure. It is then up to the application program to figure out what to do next.
• Resources involved in the transaction are locked for the duration of the transaction. This limits the usability to short-lived transactions.
Third-Party Process Coordinator
In a two-phase commit transaction, the application program (XA terminology) initiates the transaction, initiates the execution of the transaction operations (directly or indirectly), commits the transaction, and has to deal with the consequences of a failed transaction. It is, in effect, orchestrating the activities of the transaction participants and coordinating the performance of their work. However, the role of process coordinator (Figure 16-6) does not necessarily require the use of a two-phase commit transaction. The process coordinator can use any of the coordination techniques described here to orchestrate the activities of the business process participants.
Figure 16-6. Process Coordinator
Because the process coordinator is not constrained to use any particular coordination technique, it can apply whichever techniques are appropriate for the situation. This can include the use of compensating transactions (discussed in the next section) and even the implementation of alternate business processes when the primary business process is unable to execute normally.
The role of the process coordinator in a business process is as important as that of any of the participants. Thus, it is essential that the participant playing this role be made clear when defining the solution architecture. The process coordinator must, at a minimum, appear in the architecture pattern and in the mapping of the process onto that pattern. It is not uncommon for the activities of the process coordinator to show up in the process model as well.
TIBCO has three products that are designed to play the process coordinator role. TIBCO ActiveMatrix BusinessWorks provides a straight-forward means of implementing short-running automated processes. TIBCO ActiveMatrix BPM provides the capability of managing longer-running and more complex processes typically involving people as well as systems. TIBCO BusinessEvents provides the ability to implement processes that are so complex that they can only be effectively characterized by a set of rules.
Compensating Transactions
A powerful and useful alternative to the two-phase commit transaction is the compensating transaction. A compensating transaction reverses the net effect of a previous operation without modifying the previous operation. For example, when you return a purchase for credit, the initial sales transaction is not deleted or modified. Instead, a second transaction credits your account with the equivalent amount. The credit is a compensating transaction.
Approximating a Two-Phase Commit with Compensating Transactions
Compensating transactions, when used by a process coordinator, provide a means of approximating the behavior of a two-phase commit transaction (Figure 16-7). The process coordinator asks each of the participants to perform its required activity. Note that these activities are actually performed and the results are immediately visible. If all of the activities succeed (and you need to use appropriate coordination patterns to ensure that breakdowns can be detected), then the work is done. However, if any of the parties fails to perform its activity, the process coordinator then tells the other participants (those that reported success) to perform their compensating transactions. This effectively gives you the rollback of the two-phase commit.
Figure 16-7. Approximating a Two-Phase Commit with Compensating Transactions
Compensating Transaction Strengths and Limitations
Compensating transactions have the advantage that they impose virtually no overhead when the transaction completes successfully. Extra work is done only in the event that one of the parties fails to succeed in performing its assigned task. In contrast, every two-phase commit transaction adds overhead whether or not it succeeds.
Compensating transactions also have a significant limitation: Some activities do not have compensating transactions. Once you print a page, it is hard to get the ink back off the paper. Nevertheless, it is still possible to approximate a two-phase commit transaction if there is only one activity that does not have a compensating transaction (Figure 16-8). The trick is to leave the activity without the compensating transaction until last. If all the other activities succeed, then you attempt the activity without the compensating transaction. If it succeeds, you are done. If it fails, then you perform the compensating transactions for the other activities.
Figure 16-8. Coping with an Activity without a Compensating Transaction
Summary
Coordination defines how the execution of an activity is related to the execution of other activities. There are a number of patterns that can be used for this purpose. These patterns differ in their ability to detect breakdowns in the process.
Fire-and-Forget is the simplest form of coordination, but it is incapable of detecting any form of breakdown. Request-reply can detect breakdowns, but unambiguously interpreting the absence of a reply requires a response-time service-level agreement (SLA).
Delegation uses request-reply to return a promise to perform the service at some later point in time, and it uses fire-and-forget for the remainder of the process. It can detect breakdowns in making the promise, but not in the subsequent performance of the work. Delegation with Confirmation adds an eventual asynchronous confirmation that the work was completed. It extends breakdown detection to the entire process, providing that appropriate coordination patterns are used for intermediate interactions.
Distributed transactions formalize the dialog between parties required to achieve all-or-nothing behavior: Either all parties succeed in their work, or no work is performed. However, they provide no mechanism for describing what should happen if the transaction fails. Adding a Process Coordinator to the picture provides a home for additional coordination logic, which can employ compensating transactions to approximate the behavior of a distributed transaction. Some activities do not have compensating transactions. Nevertheless, one of these activities can be incorporated into an approximated distributed transaction by leaving it to last.