Chapter 10. TIBCO BusinessEvents™

TIBCO BusinessEvents is a product suite that supports complex event processing. As with BPM, the topic of building solutions requiring complex event processing is beyond the scope of this book and will be the topic of the upcoming book Architecting Complex Event Processing Solutions with TIBCO. Here we will provide a conceptual overview of complex event processing and describe the architecture of the TIBCO BusinessEvents product.

Complex Event Processing

Events occur in many places in information systems: A message is sent, a file is created, a user initiates a login. These raw events (or technical events, as they are often called) are things that you can observe. Some of them are meaningful in their own right: A single message might tell you that a package addressed to XYZ Systems has been shipped.

Other events, however, may not be as simple to interpret. Interpretation may require some additional information—some context. The shipment of the package, for example, might mark the completion of an order in the (intentionally oversimplified) process of Figure 10-1. However, to know that the just-shipped package completes the fulfillment of order number 948756 from XYZ Systems, you need more information. If, for example, you had captured earlier raw events that marked the placement of the order, the assigning of the order number, the generation of the shipping request to the warehouse, and the assignment of the tracking number to the shipment, you might be able to infer that the shipment marked the completion of the order. Understanding that the final shipment event marks the completion of the order requires understanding the information in these events and correlating them. This event capture, information extraction, and event correlation comprise the world of complex event processing.

Figure 10-1. Order Life-Cycle Event Sequence

Complex event processing is built on a stream of raw events (Figure 10-2). Some of these events are consumed as-is: Their meaning is sufficiently clear to the event consumers that the consumer knows what they represent and can determine what action to take. In other cases, multiple events must be correlated to determine their meaning. In such cases the stream of events is fed to a complex event processor. The processor is configured to analyze the stream of events and recognize particular sequences or patterns that indicate that some business-significant event has occurred: Order 948756 is now complete. A business event is then generated and passed on to the event consumer who will decide what action to take.

Figure 10-2. Basic Complex Event Processing

Information Extraction, Caching, and Persistence

Some situations call for additional processing above and beyond simply recognizing business events. Information from events often needs to be extracted and saved for later use. In the order example you might want to capture and retain the information shown in Table 10-1. With this data you can now correlate the initial order with the shipment instructions and shipment notice to determine whether all the ordered items have been shipped.

Table 10-1. Information Extracted from Order Events

TIBCO BusinessEvents provides facilities for modeling the information in events as well as concepts. Concepts are typically abstractions such as orders, shipments, and products. Information about these abstractions is found in individual events, but the abstraction itself provides a location in which this information can be accumulated and maintained. By default, this information is cached in memory and may be distributed across multiple BusinessEvents instances and multiple machines. BusinessEvents provides two optional mechanisms for information persistence.

State Machine Modeling

It is often useful to have a milestone-level model of the process that is supposed to be executing. Figure 10-3 shows a simple order life cycle that indicates which events mark the transitions between order states (milestones). The state machine model simplifies the recording of past history and depicts the allowed courses of action. As each event comes in, the state machine is updated to reflect the current state. Then the state machine provides a convenient context for interpreting the next event.

Figure 10-3. State Model of an Order Life Cycle

BusinessEvents provides state machine modeling as part of the Data Modeling extension. This includes UML 1.2 modeling of state machines, the ability to associate the state machine with a concept to represent the concept’s life cycle, and the ability to model timeouts (the failure to take a transition within a specified period of time).

Event Channels

TIBCO BusinessEvents provides a number of configurable channels for sending and receiving events. These include:

• JMS: Send and receive JMS messages, particularly via the TIBCO Enterprise Message Service.

• HTTP, including SOAP: Send and receive HTTP messages.

• TIBCO Rendezvous^®: Send and receive TIBCO Rendezvous messages.

• TCP Channels: Connect to data sources not otherwise available through channels.

• Local: Connect to other agents co-located in the same BusinessEvents engine.

The JMS and HTTP channels can, of course, be used to interact with TIBCO ActiveMatrix components, making their full capabilities available to BusinessEvents.

Rules and Decisions

At the heart of the complex event processor sits a rules engine. Rules have the general form [condition]→[action]. Broadly speaking, the conditions allow you to examine the current event, any previous event in the cache, information in the cache, and the state of state machines. Actions include adding and removing events from the cache; adding, removing, and updating information in the cache; updating the state of state machines; and generating events.

The basic structure of rules, while powerful, is often difficult for nonprogrammers to work with. For this reason, TIBCO BusinessEvents provides the Decision Manager with alternate interfaces for designing rules that are easier for nontechnical personnel to use. It provides a spreadsheet-style format that is familiar to businesspeople, making it easier to write and maintain business rules. The state machine model is another design interface that actually creates rule structures and concept definitions (the states themselves).

Queries

When situations get complex, sometimes it is difficult to represent a situation in terms of rules and state machines. In such cases it may be simpler to query the information in the cache to determine whether a particular condition exists. Typically, these queries are executed in the rule-processing cycle either as part of the rule evaluation or just prior to rule evaluation.

Visualization

Complex event processing deals with a large volume of events and extracts higher-level information. Sometimes this higher-level information is used to initiate interactions with other systems (e.g., by generating business events), but at other times it is a human being who needs to be aware of the situation. This is the purpose of the BusinessEvents Views extension. It provides the ability to define user interfaces (typically tables and graphs) that are animated by BusinessEvents-supplied information. These displays are typically used to show real-time status and provide visual alerts for situations requiring action.

BusinessEvents Solution Roles

Basic Solution Role of a Complex Event Processor

The most basic solution role for a complex event processor is as a generator of business events that, in turn, drive activity in other solution components (Figure 10-4). These business events are essentially announcements that certain business-relevant events have occurred. These events may drive the execution of a business process managed by a BPM engine, invoke specific SOA services, or simply be inputs to other systems that are part of the solution. The key role here is that the complex event processor is interpreting raw events and generating business-meaningful events.

Figure 10-4. Complex Event Processor Solution Role

Director Role

The other major role that BusinessEvents can play is that of director (Figure 10-5). In this role BusinessEvents is not only recognizing business events but also deciding what actions to take as a result. These actions can range from simply updating a status visualization through orchestrating a complete business process execution. However, a word of caution is advisable here with respect to business processes: It is not easy to understand a business process that is defined by rules. This approach should be taken only when there are so many variations in the business process that it is not practical to represent the process using BPM or Business Works.

Figure 10-5. Complex Event Processor Director Role

TIBCO BusinessEvents Product Suite

The TIBCO BusinessEvents product suite consists of five products (Figure 10-6). The core product is TIBCO BusinessEvents. It provides basic event processing and UML modeling of concepts and events. It has a rules engine and a data grid that allows data to be distributed across multiple engines. It provides an eclipse-based design environment with an integrated debugger and has several monitoring and management options.

Figure 10-6. TIBCO BusinessEvents Product Suite

TIBCO BusinessEvents™ Views

The views extension, which requires the base BusinessEvents product, provides the ability to add real-time displays of objects in Business-Events. These displays make it convenient to visualize levels of activity or processing trends and provide a mechanism for using visual alerts to bring attention to exceptions. Figure 10-7 is an example of a BusinessEvents Views display.

Figure 10-7. TIBCO BusinessEvents Views Display Example

TIBCO BusinessEvents™ Data Modeling

This extension to BusinessEvents adds capabilities in two areas: data modeling and state machine modeling. On the data modeling side, the extension adds the ability to introspect into relational database schema definitions and automatically generate the corresponding Business-Events concepts. It also adds template-based queries and the ability to have transactional interactions with databases.

On the state machine end, the extension adds a visual UML 1.2 state machine modeling tool. The resulting state machines are used to model the life cycle of concepts (e.g., orders). It also adds the ability to model timeouts (the failure to take a transition). This makes it possible to identify events that should have happened, but did not, and take appropriate action.

Figure 10-8 shows a state machine model of the handling of luggage by an airline. This model can then be used to keep track of the state of each individual piece of luggage as it moves through the system.

Figure 10-8. State Model of Airline Luggage Handling

TIBCO BusinessEvents™ Decision Manager

The Decision Manager extension is designed to make it easier for business users to define business rules. It allows a business user to define, test, and deploy business rules using a spreadsheet-style interface. It provides facilities to import and export rules from actual spreadsheets so that business users can work with spreadsheets directly. The extension can check rules for both completeness and consistency, and can automatically generate test data from rules. Figure 10-9 shows the user interface for the Decision Manager.

Figure 10-9. TIBCO BusinessEvents Decision Manager User Interface

TIBCO BusinessEvents™ Event Stream Processing

This extension adds capabilities in two areas: pattern matching and queries. For pattern matching, the extension allows patterns in the event stream to be described in an English-like format, making it easier to define these patterns. It also provides the capability of modeling temporal relationships both between discrete points in time and between intervals. Relationships can be expressed within sliding or tumbling analysis windows. Actions can be initiated for both positive and negative outcomes.

The extension also adds query agent capabilities, placing query objects in the cache engine that allow both continuous and snapshot queries of the cached information. Queries are expressed using the Object Query Language (OQL) from the Object Data Management Group.¹

BusinessEvents Solution Deployment

BusinessEvents solutions are deployed in one or more BusinessEvents engines that are grouped together into a BusinessEvents cluster (Figure 10-10). The cluster defines a multicast group that is the mechanism by which the engines communicate with one another.

Figure 10-10. BusinessEvents Deployment Architecture Pattern

Each engine contains one or more processing units that can be of different types: inference agent, cache agent, dashboard agent, and query agent. The inference agent contains the basic event and rule processing structure. The cache agent manages the event and concept information cache and coordinates the distribution of this information across multiple engines. The dashboard agent manages the visualizations being provided by BusinessEvents. The query agent manages the queries being executed against the cache.

The early releases of BusinessEvents version 4 impose some restrictions on which combinations of agents can be run in a single engine. In particular, these releases require that the query agent be run in its own engine. With this constraint, a minimal deployment of a BusinessEvents solution involving all four agent types would involve two Business-Events engines as shown in Figure 10-11.

Figure 10-11. Minimal Business Events Deployment in Early 4.x Releases

Although it is technically possible to run the dashboard agent in the same cluster and engine as the inference and cache agents, it is a best practice to deploy it on its own cluster and engine in production. This is to keep the processing involved in creating the displays from having an adverse impact on the performance of the inference and cache agents. Figure 10-12 shows an example of a production deployment following this practice.

Figure 10-12. Example Production BusinessEvents Deployment

BusinessEvents Solution Life Cycle

A BusinessEvents solution begins its life in the BusinessEvents Studio (Figure 10-13). Here the solution design is created and saved in an EAR file, and the deployment description is captured in a CDD file. These files are then picked up by the TIBCO Administrator. During deployment, the Administrator communicates using TIBCO Rendezvous or EMS with the TIBCO Runtime Agent™ (TRA) on each machine on which a BusinessEvents engine will be deployed. The TRA places the required configuration information on the local machine’s file system, executes command lines to start and stop engines, and communicates with the engines to obtain status information.

Figure 10-13. TIBCO BusinessEvents Life Cycle Architecture Pattern

Figure 10-14 details the interactions between these components during the life cycle. The BusinessEvents Studio is used to both design the application and define its deployment. Each of these activities results in a separate file (EAR and CDD, respectively). From the TIBCO Administrator these files are uploaded. Once the EAR file has been loaded, its properties and global variables can be set in preparation for deployment. Similarly, after the CDD file has been uploaded its configuration details can be specified. Upon deployment, the administrator communicates with the TRA on each involved machine to place both configuration files and executables on the machine. When the administrator starts the application, each involved TRA executes the command line for each engine to be started. Upon start-up, each engine reads its configuration files from the local machine and begins processing.

Figure 10-14. TIBCO BusinessEvents Life Cycle

BusinessEvents provides two alternate deployment and management options. One is called BusinessEvents Monitoring and Management. It takes the same inputs (EAR and CDD files) and directly manages the engines in a cluster. It provides the ability to start, stop, and pause individual agents, and to enable and disable rules within an agent. It also provides performance metrics and monitors the health of a Business-Events cluster.

The other deployment option is to use scripts. Please consult the product manuals for more details on these options.

Summary

Complex event processing analyzes streams of directly observable events and, using rules, identifies when business-relevant events have occurred. In performing this work, additional capabilities may be required: the extraction and caching of information, the modeling of life cycles using state machines, executing queries against the cached information, and the visualization of status information. All of these capabilities are provided by the BusinessEvents product suite.

BusinessEvents can play two very different roles (or a combination of the two) in a solution. One role is simply that of a business event recognizer: BusinessEvents examines a stream of events and, when it recognizes that a business event has occurred, announces this event to other applications. In this role the other applications decide what to do as a result of that event being recognized.

The other role for BusinessEvents is to determine the appropriate actions to take once a business event has been recognized and then initiate those actions. Some of the actions may be executed by BusinessEvents itself, while in other cases BusinessEvents will make calls to other systems to initiate the actions.

The BusinessEvents product suite consists of the TIBCO Business-Events product and four extension products: TIBCO BusinessEvents Views, TIBCO BusinessEvents Data Modeling, TIBCO BusinessEvents Decision Manager, and TIBCO BusinessEvents Event Stream Processing. BusinessEvents Views adds visualization capability. Business-Events Data Modeling captures database schemas and adds state machine modeling capability. BusinessEvents Decision Manager adds spreadsheet-style interfaces for defining rules. BusinessEvents Event Stream Processing adds query capabilities and the modeling of temporal relationships.

BusinessEvents solutions are deployed on one or more Business-Events engines that are grouped together into clusters. Each engine runs one or more processors, which can be inference agents, cache agents, query agents, and dashboard agents.

BusinessEvents solutions are defined in the BusinessEvents Studio. These designs, and accompanying deployment descriptors, are picked up by the TIBCO Administrator. The Administrator is used to configure the design for deployment and, with the aid of the TIBCO Runtime Agents (TRAs) on the various machines, deploy the design and start the engines. BusinessEvents Monitoring and Management and scripts provide alternative approaches.