In this chapter, we describe three patterns that address synchronization issues of parallel business processes. In this context, synchronization means that execution in terms of the progression through the different activities of a process needs to be synchronized with other business processes running in parallel. The synchronization issues reflect requirements of complex business scenarios, and the synchronization dependencies cannot be modeled directly in the business processes via static control flow dependencies. As a result, conflicting forces arise due to the need for loosely coupling the synchronization concerns with the business process models. Besides technical forces, such as the problems of concurrent data access, supporting business agility is central. Business processes are subject to constant change. Hence, any suitable synchronization mechanism must be loosely coupled to support changes in the connected business processes.

The patterns describe architectural synchronization solutions as opposed to synchronization within a business process design. That is, other architectural components than the process engine or external systems play a role in the solutions of the patterns.

The three patterns presented are as follows:

• The REGISTER FOR ACTION pattern describes how to synchronize the execution of functionality of a target system with business processes running in parallel.

• The BUNDLE PROCESS AGENT pattern describes how business objects being generated by different parallel processes can be processed in a consolidated way.

• The PROCESS CONDUCTOR pattern addresses how to enable designers to model dependencies between parallel business processes by flexible orchestration rules.

Consider a simple example to illustrate the use of the patterns: Various business processes of a company require contacting the customer via phone, but the company wants to avoid contacting the customer too often in a specific period of time. Hence, the phone calls should be consolidated. In such business scenarios that require synchronization of multiple process instances, the patterns described in this chapter can be applied.

If only a specific action, like “put phone call into a task list” needs to be performed after synchronization has taken place, the REGISTER FOR ACTION pattern should be applied. However, the phone call might also require a business process preparing the phone call, and this business process then usually needs access to the private business objects of the synchronized processes. In this more complex scenario, the BUNDLE PROCESS AGENT pattern can be applied. Finally, if the need for synchronizing occurs within the processes and requires each of the processes to be stopped until the synchronizing action (which might be yet another business process) has happened, then PROCESS CONDUCTOR is applicable.

This simple scenario should illustrate one important design issue: Which of the patterns is chosen depends on the design of the business processes that need to be synchronized. In some scenarios, the patterns are mutual alternatives; in others, combining them makes sense. For example, the BUNDLE PROCESS AGENT pattern can be combined with the REGISTER FOR ACTION pattern to dynamically identify which business processes need to be considered by the bundling.

Figure 12.1 shows the main pattern relationships of the three patterns from the perspective of the REGISTER FOR ACTION pattern.

Business processes are executed on process engines. Sometimes, the execution of an action depends on the states of multiple business process instances running in parallel. When this is the case, the action can only be executed if those parallel business process instances have reached a certain state represented by a specific process activity.

As business processes are created and changed while the system is running, it is not advisable to define synchronization dependencies statically in the models. Instead, it should be possible to define and evaluate the synchronization dependencies at runtime while still allowing business processes to change independently.

Business processes are dynamically instantiated on process engines at different points in time. For this reason, there are usually several instances of one or more business processes running in parallel. Each of them has a different state of progression in terms of the process activity they have reached so far during execution.

When a specific action, such as a business function, service, or another business process, has a logical dependency to multiple business process instances, synchronization with all the process instances can be difficult. First, the action might not know which business process instances of the many possible parallel instances it is dependent on. But, even if it knows the instances that it is dependent on, polling them for synchronization would incur a communication overhead. The same problem of a communication overhead for synchronization would also occur if the process instances would run a synchronization protocol, for instance, before triggering a callback that executes the action.

In addition, before the action can synchronize with the process instances, it needs to know that it must wait for one or more instances. That is, a mechanism is required to communicate that there is a new dependent process instance to wait for.

Business processes can change over time, and new processes are constantly created while the overall system is running. This includes that a state at which an action must be executed might change, be added to a process, or be removed from the process. The actions that depend on business processes must be able to cope with such process changes. The effects of these changes should be minimized and should not have an impact on other components to be manageable. A consequence is that the synchronization dependencies of the actions cannot be statically modeled in the models of the business processes or the actions but must be defined and evaluated at runtime. In other words, a loose coupling between the action and the business processes it is dependent on is required.

The REGISTER FOR ACTION component offers two services:

• A registration service, by which a process instance can register itself with its instance ID

• A deregistration service that allows a process instance to deregister itself via its instance ID

Invocation of the deregistration service means that the process has reached the state that is relevant for the action. The two services are invoked by process activities. Each registration invocation must have one corresponding deregistration invocation in a business process. This design has the consequence that the place of invocations can change as the business processes change over time. In other words, the REGISTER FOR ACTION component and the business processes are loosely coupled.

The REGISTER FOR ACTION component waits until all registered business processes have deregistered themselves. After the last deregistration, the action is executed by the REGISTER FOR ACTION component.

An important detail of a REGISTER FOR ACTION design is to determine the point in time when registration ends. As most scenarios of the pattern concern long-running business processes, a registration delay is a practical solution that works in many cases. The registration delay runs a certain amount of time from the point in time when the first registration to the REGISTER FOR ACTION instance happens. For instance, if a registration delay of one day is chosen, then all registrations that accumulate throughout that day will be included. Of course, the length of the delay can be adjusted based on either previous experiences or experimentation. An alternative to a registration delay is introducing a specific registration type that ends the registration process for one REGISTER FOR ACTION instance.

Modeling the deregistration service invocation might be an issue for some business processes: Deregistration should often be invoked as early as possible so unnecessary delays for the action to be executed are not produced. If the business process contains complex decision logic, there may be various paths that may lead to a deregistration service invocation at many different positions in the process. As the process execution may follow only one case-specific path, deregistration must be found on all possible paths if a registration has been previously performed.

To place the deregistration service invocations at the right positions and to avoid multiple invocations of the deregistration service in case of loops in the process is sometimes not trivial if the business process is of higher complexity. The easiest way might be to put deregistration simply at the end of the business process and thus avoid the possible complex logic that is initiated by the different possible paths or loops. However, this is not always possible or optimal if deregistration as early as possible is required.

In the REGISTER FOR ACTION pattern, the service invocations from the business processes might be realized as SYNCHRONOUS SERVICE ACTIVITIES or FIRE-AND-FORGET SERVICE ACTIVITIES. The realization using SYNCHRONOUS SERVICE ACTIVITIES is usually better suited as it is important for the business processes to be informed whether the registration and deregistration were successful. If the target action is related to a more complex business process, then this consolidation can be achieved by using a BUNDLE PROCESS AGENT.

The ACTIVATOR pattern [SSRB00, BHS07a] has a similar structure as REGISTER FOR ACTION. It, however, solves a different problem: the on-demand activation and deactivation of service execution contexts to run services accessed by many clients without consuming resources unnecessarily. The patterns can be used together with REGISTER FOR ACTION using a shared structure. That is, the registration and deregistration services could be used for on-demand activation and deactivation.

Also, PUBLISHER/SUBSCRIBER [BMR+96, BHS07a] can be combined with REGISTER FOR ACTION using a shared structure. That is, the registration and deregistration services could be used to subscribe and unsubscribe to events for the time of registration. This way, the REGISTER FOR ACTION component can communicate with its currently registered processes.

Business processes are executed on a process engine, and during their execution business objects are created and manipulated by the business process instances. Each business process instance creates and manages its own set of business objects to avoid data inconsistencies, locking overheads, deadlocks, and other problems created by concurrent data access. Business scenarios, such as consolidated sending of postal mail in batches, require consolidating the business objects being generated by many different process instances and then processing them using a central but parallel-running business process. Hence, the usual mechanisms of a process engine, in which each process instance keeps its business objects in a private data space, are not sufficient to support such scenarios.

Business process instances running on a process engine have their own data space and are thus disjoint entities. When business objects are created during the execution of a business process, only the business process instances creating the objects know about their existence. That is, the business objects created by a business process instance are per default private to the business process instance. This helps to avoid unwanted effects, such as data inconsistencies, locking overheads, or deadlocks, when business process instances are running in parallel because the actions of the business process instances control all accesses to these business objects.

This technical concept can be applied to implement most business scenarios. However, there is a special case for which this technical solution does not work well alone: Consider that the business objects created by many different parallel business process instances are input objects to be processed by a central business process that logically gathers all these business objects and then processes the consolidated set of business objects. A typical example is that the business requires these business objects to be handled in a consolidated way, such as sending one letter per postal mail for a number of parallel transactions with a customer instead of sending multiple letters. In this case, the parallel-running consolidation process instance must gather the objects and process them. Unfortunately, usually process engines do not directly support such scenarios.

It is necessary only to centrally process those business objects that actually should be processed in a consolidated way. It might be that this is only a subset of business objects owned by a process instance. A process instance should still have control over what business objects should be processed in a consolidated way and should thus be able to publish only those objects that it considers relevant.

Each of the involved business processes can potentially change over time. Hence, the consolidation architecture should not impose restrictions on the business process design that would hinder rapid changeability.

Design an architectural component that serves as a BUNDLE PROCESS AGENT, which offers a service to be invoked by business processes to send business objects that need to be processed centrally. The BUNDLE PROCESS AGENT stores the business objects being sent to it in a container that serves as a temporary repository. The container is not intended as the actual persistence mechanism of the business objects—it is rather intended to capture only which objects need to be processed centrally.

For this reason, this container might only keep BUSINESS OBJECT REFERENCES rather than the business objects themselves. However, it is also possible to send copies of actual business objects and not just references. Often, these objects then only contain a subset of the business data of the original business objects (i.e., the subset of data relevant for processing the business objects centrally). In this case, it is advisable to introduce special types of business objects designed for these purposes.

The BUNDLE PROCESS AGENT waits until a specified end criterion is reached. For instance, this can be a deadline or specified maximum amount of business objects that can be bundled in one bundle. When the end of bundling is reached, the BUNDLE PROCESS AGENT instantiates a bundle process that processes the business objects centrally. The container with the business objects is cleared after the processing has been initiated to be ready to store new objects for the next iteration. Only one instance of the bundle process is running at a time for each bundle; that is, the processing of a set of business objects must be finished before the next instance of a bundle process can be started. Of course, different bundles can be assembled in parallel. For instance, business objects for postal mail communication with customers are bundled to send them together; then, there is one bundle per customer.

During the execution of the bundle process, new business objects are sent to the BUNDLE PROCESS AGENT by business processes running in parallel for the next iteration. These objects are again stored in the container. This way, only business objects relevant for the next iteration are kept in the container as the container is emptied when a new iteration (i.e., a new instance of the bundle process) has been started. The BUNDLE PROCESS AGENT repeats this process in a loop.

The BUNDLE PROCESS AGENT is implemented as a COMPONENT CONFIGURATOR [SSRB00] to allow controlled configuration of the agent at runtime. When it is initialized, it performs the following functions:

1. It checks whether there are new business objects in the container to be processed by a bundle process.

2. If there are new objects, it checks whether an instance of the bundle process is still running. Only if there is no instance running is a new instance created that processes the new objects in the container. The container is cleared to be empty for new objects after the process instance has been started. If an existing instance is still running, then no action is performed (i.e., no new bundle process is created and the business object container is not emptied).

3. The agent loops back to step 1 until the loop is aborted by some event to finalize the execution or to suspend the execution.

There is one concurrency issue involved in this algorithm. The service that allows business processes to send new business objects to the container might conflict with the clearing action of the container that is initiated by the algorithm described. That means a new instance of the bundle process might be created with the given objects in the container at that point in time. After the instance is created, the algorithm prescribes to clear the container. If there are new objects added to the container while the creation of the new instance is still in progress, then these objects will be deleted from the container with the clearing action without being processed by the bundle process. To avoid such a situation, the container must provide locking and unlocking functions that are used for the time a new instance of a bundle process is created.

The BUNDLE PROCESS AGENT pattern thus resolves issues according to complex bundling of business objects that need to be centrally processed and offers a general architectural solution that is both flexible and extensible. Different bundle processes can be used for different purposes, although this will increase the complexity of the architecture. However, there might be a larger effort involved to design a BUNDLE PROCESS AGENT component. For this reason, the pattern may only be suitable in projects and programs that have a larger strategic perspective.

The BUNDLE PROCESS AGENT pattern can be combined with the REGISTER FOR ACTION pattern to dynamically identify which business processes need to be considered by the bundling. As far as the service invocation from business processes is concerned, the SYNCHRONOUS SERVICE ACTIVITY pattern or the variation of the FIRE-AND-FORGET SERVICE ACTIVITY, including acknowledgment, is usually recommended to achieve some level of security that the business objects being sent have arrived at the initiated target. According to the MACRO-/MICROFLOW pattern, the bundle process can be implemented as a macroflow or microflow.

Interdependent processes are in execution on a process engine. The interdependency implies that execution of the processes needs to be synchronized at runtime. When business processes are executed on a process engine, there are often points in these processes at which dependencies to other processes need to be considered. That means a process may only move to a certain state but can only move on if other parallel processes have also reached a certain state. Further execution of these parallel-running processes need to be orchestrated at runtime as it cannot be decided at modeling time when these states are reached or what these states are. This is due to the separate execution of the processes and the fact that each process is a component that may change individually over time. In most cases, the rules for the orchestration need to be flexibly adaptable.

At some point in a process, it might be necessary that the process is only allowed to move on if other processes have also reached a certain state. However, each individual process does not know what these states of other processes are and when they are actually reached as each process runs at its own speed within its own data space. Moreover, each business process needs to be treated as a separate component and may change individually over time. Thus, processes need to be loosely coupled as far as this aspect is concerned.

The reason for this is that it is hard to specify in a single process model what states of other processes are relevant from the point of view of an individual process and when these states are reached or what the relevant dependent processes are. If this is statically modeled in a process, somehow the implementation will be inflexible, and changes to orchestration rules usually have an impact on all involved processes. That means the actual orchestration appears to be a complex concern of its own, and the rules for orchestration cannot be defined attached to an individual process model. Consequently, the orchestration rules should not be captured as some types of static relationships of a process to other processes. The dependencies that will be generated if each process should know the rules for orchestration will be hard to manage. If the rules change, then each individual process needs to be changed as well. For this reason, a tight coupling of the rules to each individual process is an inappropriate approach.

As a result, each process needs to be treated as an encapsulated component that does not know anything about the orchestration rules or the processes to which it has dependencies. Each process must only know its own points in the process at which the dependency occurs but not what this dependency is about. New processes might be created over time, which create new dependencies, and this must not affect existing process models to make the changes manageable. Each process model itself may also change (e.g., new steps are added without affecting the actual rules for orchestration as they need to be treated as a separate concern). The very problem is thus that the processes are standing in dependency but must actually not know very much about each other as the dependency needs to be separated from the process to treat it as a separate concern and to make the processes and the complexity generated by these dependencies manageable.

A central aspect of the PROCESS CONDUCTOR pattern is that the central conductor is only invoked synchronously. That is, when a business process reaches a critical state at which it may only move on if certain other dependent processes have also reached a certain state, then a SYNCHRONOUS SERVICE ACTIVITY is modeled at this point in the process that invokes a service. At this point, the process to be conducted blocks on the synchronous invocation until the conductor returns a result. The PROCESS CONDUCTOR SERVICE reports the state of a process instance and the ID of the instance to the PROCESS CONDUCTOR component. The states and corresponding process IDs are stored in a container. The PROCESS CONDUCTOR component applies orchestration rules that are configurable to determine the order of events that need to be fired to initiate dedicated process instances to move on.

The PROCESS CONDUCTOR applies its orchestration rules to the states and corresponding process IDs in the container. The orchestration rules simply define an order of the process states, that is, an order of terminating the corresponding process activities. The conductor then fires events to the process instances identified by their IDs in the order that is determined by the orchestration rules. Hence, the service implementation to report the state and the process ID can be implemented as an EVENT-BASED ACTIVITY. The process engine receives the events and terminates the activities in the order directed by the conductor. As a consequence, the processes move on to the next step in the right order. The conductor repeats this process in a loop as new processes may have registered for the next iteration.

The triggers to start one iteration of this procedure to apply the orchestration rules and to fire events to the processes can be twofold. It can happen repeatedly in defined time intervals, or it can be initiated by other dedicated event triggers (e.g., a master process has invoked the service of the PROCESS CONDUCTOR to register an initiation state that triggers the orchestration rules).

The registration of the state and process ID and the waiting position for the actual termination event to occur can also be designed as an ASYNCHRONOUS RESULT SERVICE. In this case, the business process needs to model two activities: one that places the request and a second one that gets the result (i.e., waits for the termination event).

The EVENT-BASED PROCESS SPLIT pattern can also be used in conjunction with a PROCESS CONDUCTOR in case it might take a long time until the termination event occurs, and it makes sense to split the process into two parts. That means if the termination event occurs, the second part of the process will be instantiated rather than modeling a waiting position.

One must note that the pattern generally assumes that the process engine processes the terminate events in the sequence that they are fired. This implies that the activities will terminate in the intended order (i.e., the order the terminate events have been fired), and the processes will correspondingly move on in the right order. If this cannot be assumed, then it might be that the activities of the processes do not terminate in the right order, and consequently the processes do not move on in the right order as well. To resolve this issue, the implementation can be extended by an additional service that is invoked from a business process. This additional service confirms that the activity has terminated. This is modeled as a second SYNCHRONOUS SERVICE ACTIVITY right after the first one. The next process, according to the rules, is only notified after the confirmation from the preceding business process has been received.

This may only be necessary if the order of termination is important within one iteration of notification. In many cases, this is not important as it is rather the whole iteration that represents the order (i.e., all processes of one iteration may literally move on at the same time, and slight differences do not matter). This also depends on the underlying rules and what these rules are further based on. According to the MACRO-/MICROFLOW pattern, this may also depend on whether we are at microflow or macroflow level. Transactional microflows usually run in much shorter time (sometimes milliseconds), and even slight time differences might matter, while these slight time differences might not matter at all at the macroflow level.

The pattern provides a flexible solution to determine the order of process steps that need to be synchronized by configurable rules. New processes and rules can be added without changing the architecture. Existing rules can also be modified without changing the implementation of running business processes. However, this flexible concept requires additional design and implementation effort. The design might be quite complex depending on the requirements regarding the complexity of the synchronization. For this reason, the pattern is most suitable in larger projects in which architecture evolution and business agility are required.

The PROCESS CONDUCTOR pattern is a central bottleneck, and it also incurs the risk of deadlocks in case the orchestration rules of the process conductor are misconfigured or a business process fails to signal its state. In such cases, usually manual intervention is required. It makes sense to monitor the process conductor component for such events.