An ESB service container should handle the inflow and outflow of management data such as configuration, auditing, and fault handling. Management interfaces should be implemented using the Java Management eXtensions (JMX) if the ESB implementation supports Java. A more detailed discussion on the use of JMX in an ESB can be found in Chapter 10.

As illustrated in Figure 6-13, an ESB service container supports the retrieval of configuration data from a directory service, and can also have a local cache of configuration data. This means that even if other parts of the ESB, including the directory service, become temporarily unavailable, the service container can continue to operate with its current set of configuration data.

Figure 6-13. Service containers support inputs and outputs for management data such as configuration, auditing, and fault handling

Management input data can originate from a remote management console issuing commands such as start, stop, shut down, and refresh the cache. Management output data can consist of event tracking, such as notification that a message has successfully exited the service or that a failure has occurred. These inputs and outputs can be managed by a JMX management console or redirected to some other management tool using standard protocols such as the Simple Network Management Protocol (SNMP).

The ESB container provides the message flow in and out of a service. It also handles a number of facilities, such as service lifecycle and itinerary management, that we will explore in a later section. As illustrated in Figure 6-14, the container manages an entry endpoint and an exit endpoint, which are used by the container to dispatch a message to and from the service.

Figure 6-14. Message dispatch to a service uses the service’s configured entry and exit endpoints

XML messages are received by the service from a configurable entry endpoint. Upon completion of its task, the service implementation simply places its output message in the exit endpoint to be carried to its next destination using the “request/reply” or “reply-forward” pattern discussed in Chapter 5. The output message may be the same message that it received. The service may modify the message before sending it to the exit endpoint. Or, the service may create a completely new message to serve as a “response” to the incoming message and send the new message in the exit endpoint. The following code example shows what a service implementation looks like:

public void service(ESBServiceContext ctx) throws ESBServiceException {
    // Get any runtime process parameters.
    Parameters params = ctx.getParameters( );
    // Get the message.
    ESBEnvelope env = null;
    env = ctx.getNextIncoming( );
    if (env != null) {
        ESBMessage msg = env.getMessage( );
        ... // Operate on the message
    }
    // Put the message to the Exit Endpoint
    ctx.addOutgoing(env);
}

What is placed in the exit endpoint depends on the context of the situation and the message being processed. In the case of a CBR service, the message content will be unchanged, with new forwarding addresses set in the message header.

In more sophisticated cases, one input message can transform into many outputs, each with its own routing information. For example, a custom service can receive a purchase order document, split it up into multiple output messages, and send out the purchase order and its individual line items as separate messages to an inventory or order fulfillment service. The service implementation in this case does not have to be written using traditional coding practices; it can be implemented as a specialized transformation service that applies an XSLT stylesheet to the purchase order document to produce the multiple outputs (if the ESB has an XSLT extension to support multiple outputs).

Auditing and logging play an important business function within an integration strategy. Part of the reason you want to integrate across a common backbone is to gain real-time access to the business data that is flowing between departments in an organization. A reliable communications backbone will ensure that the data gets to its intended destinations. An auditing framework will allow you to track the data at a business level.

As part of its management capabilities, an ESB container provides an auditing and logging facility. This facility can have multiple sources for tracking data. System-level information about the health of the service itself and the flow of messages can be tracked and monitored. Application-level auditing, logging, and fault handling are accomplished through additional endpoints that are available to each service. As illustrated in Figure 6-15, the service implementation has three additional endpoints at its disposal: a tracking endpoint, a fault endpoint, and a rejected-message endpoint. A service can be created such that a message can be placed in the tracking endpoint in addition to its normal exit destination. A rejected-message endpoint can be used for system-level errors, such as a malformed XML document, or any case in which the service itself throws an exception. The fault endpoint can be used for any application-level errors, or faults, that can occur.

Figure 6-15. Additional tracking and fault handling endpoints are handled by the service container

There is an underlying philosophy behind the way tracking, system errors, and application faults are handled. In addition to a normal exit endpoint to handle the outgoing flow of a message, additional destinations are available to the service for auditing the message and for reporting errors. The service implementation has these endpoints available for its own use. From the service implementation’s point of view, it simply places data into the tracking endpoint or fault endpoint, and the surrounding ESB framework takes care of managing the auditing, logging, and error reporting. A tracking endpoint can be configured to point to anything—a topic or queue destination, or even a call to an external web service. This approach provides a separation between the implementation of the service and the details surrounding the fault handling. The implementer of a service need only be concerned that it has a place to put such information, whether it is information concerning the successful processing of good data, or the reporting of errors and bad data.

Tracking can be handled at both the individual service level and the business process level. A business process may make use of different implementations of individual services over time. The tracking of a fault occurrence or the auditing of an individual message can be tied to the context of the greater business process that is utilizing the service at the time.

In Chapter 7, we will explore more about how auditing facilities are used. We will also see that the use of these endpoints is one way of doing auditing and logging with an ESB. Other approaches include placing a specialized XML persistence service in the path of a message in a process flow, which can be much less intrusive to each individual service implementation.

The container is an intrinsic part of the ESB distributed processing framework. In some sense, the container is the ESB—more so than the underlying middleware that connects the containers together. The container provides the care and feeding of the service. As illustrated in Figure 6-16, the ESB container provides a variety of facilities that a service implementation may have at its disposal. Facilities such as location, routing, service invocation, and management are taken care of by the container and its surrounding framework, so the implementation of a service does not have to be concerned with those things. The container also provides facilities for lifecycle management such as startup, shutdown, and resource cleanup. Thread pooling allows multiple instances of a service to be attached to multiple listeners within a single container. Connection management facilities provide the specifics of the binding to the underlying MOM, plus additional fault tolerance and retry logic.

Figure 6-16. The ESB container provides many facilities for the service implementation

Within the Java Community Process (JCP), the Java Business Integration (JBI) effort is underway to standardize the means by which components plug into an integration infrastructure. For an ESB, this will provide conformity across implementations. More details on JBI can be found in Chapter 10.

You can think of the contrast of an ESB versus an integration broker as highly distributed and standards-based versus centralized, monolithic, and proprietary.

A hub-and-spoke integration broker engine includes four main pieces of functionality: application adapters, data transformation, routing rules, and connectivity (Figure 6-17). Today’s integration brokers were built when few standards were available to exploit, and are therefore largely proprietary. Also, as illustrated in Figure 6-17, some integration brokers have bolted on some standard interface technology, such as web services.

Figure 6-17. Hub-and-spoke integration brokers are centralized, monolithic, and largely proprietary; standard interfaces are an afterthought

An ESB makes prevalent use of standard interfaces and standard protocols throughout. Integration brokers have been largely proprietary, which, as discussed in Chapter 2, can carry a steep learning curve and result in vendor lock-in. Integration based on standards can provide a number of business benefits beyond just future-proofing your integration strategy. Chapter 3 discusses the benefits of adopting standards throughout your integration projects.

In a hub-and-spoke integration broker, the message payload must travel to the rules engine in the hub to perform data transformations and receive subsequent routing instructions (Figure 6-18). Also, each link into the hub potentially runs over a different protocol and transport, limiting reliability to the weakest link.

Figure 6-18. Hub-and-spoke integration brokers require that all messages return to the rules engine for further instruction after every step of processing

An ESB does not rely on any centralized processing engine, and lets the routing decisions occur at the remote nodes via itinerary-based routing. Hub-and-spoke has the advantage of centralized management and centralized control over routing and transformation rules. However, a good ESB should have a proper distributed management environment that allows remote containers to be updated with fresh information whenever necessary.

To date, the main function of an application server has been to host business logic and to serve up web pages that are often based on dynamic content. Application server vendors trying to get into the integration business have been building integration broker capabilities on top of their application servers.

Both ESB containers and application server containers provide a managed environment for lifecycle issues, instance management, thread management, and timer services. Security and transaction services are also common in most application servers and ESBs.

An ESB service container is intended to host integration services and to execute business process flow logic, and host them only when and where you need them. For example, process flow logic can route an invoice to the proper financial system. Process flow logic can be specified declaratively or written as code in specialized services. Other examples of integration services include hosting an adapter or a content-based router.

Integration brokers, including those built upon an application server stack, require that the entire IB/appserver stack be installed everywhere you want integration broker functionality (Figure 6-19). Think of what it would mean to install an entire application server at a remote location simply so that it can host a JCA container, which hosts a JCA adapter, which acts as a conduit to an ERP system.

Figure 6-19. The “appservers everywhere” strategy

In contrast, with the ESB approach, integration broker capabilities are distributed across a more loosely coupled integration network as integration components that are independently deployed and managed (Figure 6-20). The only software component that needs to be installed at the remote location is the ESB service container. The ESB service container can act as a JCA container directly, without the rest of the appserver or integration broker stack, which in this scenario is just excess baggage. This is one of the business reasons why the ESB is changing the economics of integration. Licensing costs aside, the real cost of deploying application servers everywhere is in the installation, configuration, and ongoing operational overhead and cost of ownership over time.

Figure 6-20. Distributed service containers provide selective deployment of integration services that are independently scalable

Managing an application server, even when using JMX, requires a direct connection between the management console and the remote JMX agent for the application server. In a highly distributed deployment, this separate management connection significantly increases the number of holes that have to be punched through the firewalls (Figure 6-21).

Figure 6-21. Managing application servers requires extra holes in firewalls

As we’ll discuss in Chapter 10, ESB management traffic is designed to share the bus with application traffic, as illustrated in Figure 6-22. This means that the firewall configuration has to be done only once.

Figure 6-22. Remote management in an ESB requires no direct connection to each container, and no additional holes in the firewalls

The main benefit of using XML standards (XPath, XSLT, XML DOM, and SAX) for transformation is that it makes the deployed processes much less brittle. For example, as shown in Figure 6-23, an ESB transformation service can be configured and deployed, merely by supplying it with the scripts and parameters it needs to use, as XML documents that are read in from a directory cache. Once deployed, the implementation is remarkably resilient to change in the XML schema. In many cases, changes in the XML schema will not invalidate the XPath selections within the XSLT, and new attributes and elements are carried along by the existing XSL templates. Even if the XSL script does change, it is easily redeployed. More information on service reuse and redeployment can be found in Chapter 7.

Figure 6-23. Configurable deployment artifacts such as XPath and XSLT are insulated from changes in XML schema

In contrast, compiled classes such as XML beans introduce a dependency on the XML schema that obviates adaptation to change. As shown in Figure 6-24, the use of the schema to generate Java code sets in motion a chain of dependencies that must then be retraced every time the schema is modified. This is because schema elements must be bound to Java objects. The problem is further compounded if the transform in question is deployed in multiple locations.

Figure 6-24. Schema changes result in downstream dependencies that must then be retraced whenever the schema is modified

Table 6-1 summarizes the comparison between compiled code in an application server and the declarative functions that are inherent in an ESB.

A key point to note is that the use of dynamically configured parameters, declarative scripts, and other runtime artifacts serves to reduce the amount of compiled code in the system, as illustrated in Table 6-1. The end result is a more flexible, loosely coupled deployment.

You shouldn’t take this negatively if you are an adopter of J2EE application server technology. Application servers are an important part of an IT organization, and an ESB is designed to integrate with them very well. In Chapter 11, we’ll see a portal application design pattern in which application servers are used for both the frontend web container display logic and the backend business logic. The ESB is used in the middle to connect the two, providing an integration weave into the rest of the corporate fabric that is not application server-based.