In this next recipe we will consider a common design pattern for processing a list of independent messages in a batch. In this scenario, a synchronous web service implemented in OSB will accept a list of messages and respond almost immediately with a response to indicate that the message was successfully received. Meanwhile, each of the individual messages will be queued for asynchronous processing by another service.
This recipe also assumes that the downstream, one-way service for processing individual messages from the batch has already been written using OSB.
This example builds on the result of the previous recipe, the "BookOrder" dynamic routing service. A sample completed version of this service is included with the code samples for the book.
- Log in to the Weblogic console and select Services | Messaging | JMS Modules. Select New.
This will open the Create JMS System Module window. Name the module
BookModuleand click on Next. Target the OSB server(s) and click on Next. Finally, check the box to add resources and click on Finish. - This will take us to the settings for
BookModule. Click on New in the Resources table. - This will open the Create a New JMS System Resource dialogue. Select Queue and click on Next. Name the queue (for example,
BookOrderQueue) and assign a JNDI name (for example,jms.queue.bookorder).Click on Next, then click on Create a New Subdeployment. Accept the default name and target the existing
wlsbJMSServer. Click on Finish. - Next, we need to modify the downstream process to use the JMS queue. Open the OSB workshop project from the previous recipe. Open the downstream proxy service (for example,
PublisherService.proxy) and select the Transport tab.i. In the Protocol drop-down list, select jms. Change Endpoint URI to the queue JNDI name we created earlier, for example:
jms://localhost:7001/weblogic.jms.XAConnectionFactory/jms.queue.bookorder
ii. Save your changes.
- Next, we need to create a new proxy service for handling our batch requests. Give it the name
PublisherBatchServiceand usePublisherBatchService_1.0.wsdlas the WSDL.This WSDL defines the operation
submitBookOrderList, which contains a list ofbookOrdersthat we want to process individually. - Select the Message Handling tab and ensure Transaction Required is Enabled.
- In the Message Flow tab, create OperationalBranch and add Pipeline Pair and Stage to the submitBookOrderList operation. Next, drag a For Each action into the Stage.
- Set the For Each Variable property with a name you would like to use to reference each element of the batch, for example,
bookOrder. Set In Variable tobody. Optionally, declare index and count reference variables.
- Next, click on the XPath link to open the XPath dialog.
- Drag the repeating list member element (for example
bookOrder) from the request body over as the XPath expression. Then click on OK.
- Drag a Publish action from the Design Palette into the body of the For Each loop. Set the Service and Operation properties of the Publish action to use the proxy service and operation of the downstream process (for example,
PublisherService.proxy, andsubmitBookOrder) respectively. - Next, drag a Routing Operations action from the Design Palette into the body of Publish action. Within the Routing Options, enable QoS (Quality of Service) and select Exactly Once.
- Finally, place a Replace action within the Publish action body and configure it to have the values listed in the following table:
Field
Value
XPath
.In Variable
body
Expression
XQuery: PublisherApp/setSubmitBookOrder
Select Replace node contents
- Configure the expression to use the XQuery resource
PublisherApp/setSubmitBookOrder.xqand pass in the variablebookOrder(set in step 10).This will create the payload required for the
submitBookOrderoperation invoked by the Publish action, based on the content of theForEachvariable defined earlier, for example,bookOrder. - Deploy and test your OSB projects by sending a request to your Batch processor.
The key to the asynchronous operation of this pattern is the separation of the initial call from the bulk of the processing with a JMS queue. If not explicitly defined, OSB will dynamically generate a generic queue when the JMS transport protocol is selected, but for finer control, including auditing, repeatability, and the potential to assign a Work Manager it has been recommended in this recipe to assign a specific named queue.
The logic of the Batch process itself is relatively straightforward. The For Each loop simply divides the batch implicitly into individual list elements and passes them on to the JMS transport downstream proxy service.
A key requirement for processing our batch, is that we want to successfully split out all messages from the batch or if something fails, roll back the entire batch (so we can re-submit once the error has been resolved). For this purpose we configured the JMS Transport as follows:
- Message handling to enable Transaction Required: This will instruct OSB to start a new transaction if one does not exist in the request received (which it won't as this is invoked over HTTP).
- Routing Options on the Publish action to have a Quality of Service of Exactly Once: This will force the publication of the message to the JMS queue to be included within the transaction started by the proxy.
- Use an XA Connection Factory for a JMS Queue: This ensures that the transaction is propagated to JMS.
In this way, each book order is written to the JMS queue as part of the same transaction. When the proxy returns a response, then the transaction will be completed. At this point each bookOrder can be processed independently by the downstream proxy in a separate transaction, without interfering with the processing of other items in the list.
Note that this pattern places very little responsibility on the Batch Processor to perform error handling. Other than syntactically validating the input request, it is not recommended to attempt any other validation on the individual list elements at this level, since one bad item might prevent the entire list from processing.
Instead, consider simply returning "success" to the caller and managing all exceptions internally, as part of the downstream process. This ensures that any valid items will still be processed and each error will be handled separately.
Fine-tuning the behavior of the JMS queue can be handled within the Weblogic console. In particular, you may wish to throttle the activity on the JMS queue to prevent the downstream process from becoming overloaded.