Throughout this Redbooks publication, we have followed the journey of Walmart as they adopted the IBM CICS asynchronous API within an offering that is heavily used in their organization. As we have read in earlier chapters, this is not their first encounter with asynchronous processing patterns in CICS applications, and they have experience with creating their own frameworks. They have a wealth of experiences to share with readers who might also be on the journey of adopting asynchronous patterns in their applications.

In this chapter, we share advice from the Walmart team, including some of their suggestions of practices, things to be aware of, and how they approached challenges.

It is likely that readers who are implementing asynchronous patterns will encounter similar challenges. Their specific nature will likely depend on the specific environments and requirements. Although further investigations will likely be required, this chapter should provide valuable food for thought.

Some advice in this section is equally applicable to asynchronous solutions that do, and do not take advantage of the CICS asynchronous API.

Use of the CICS asynchronous API greatly simplifies the efforts of creating an asynchronous processing pattern in enterprise applications. However, application developers are still required to understand the transactionality of the parent and child transactions. This is especially true when you are considering the impact of invocations that have failed midway.

Take for example an application that PROGRAM LINKs to two programs serially. If the second program ABENDs, the application can be automatically rolled back by CICS, with no additional developer foresight.

Now, consider a similar application that calls an additional two programs asynchronously. If the second of those programs ABENDs, what is the impact? The parent and two child processes are full CICS transactions in their own rights. Hence, the ABEND of second child, does not imply that the first child will be rolled back. In fact, the first child might still be executing, it might have terminated and committed; indeed, it might not have started yet. There is also a possibility that the parent has completed, or even ABENDed.

It can be safer to use only GET operations for child transactions, for example, asynchronous retrieval for stock quotes, or addresses. If something goes wrong, it is safe to simply throw away the results. However, if a child changes the state of the system (such as Child 1 'minus £200 from the current account' and Child 2 'add £200 to the savings account'), the developer must account for partial failures. Rollback algorithms and compensation flows are typically required for POST operations.

These Unit of Work (UOW) scenarios can be a source of difficulties for application developers who are new to asynchronous patterns. The CICS asynchronous API helps greatly with these combinations:

Nonetheless, the application developer needs education and skills to keep track of the interplay between parent and child transactions.

When data is passed between parent and child transactions, the data itself (or a copy of the data) should be passed, not a reference to the data. This practice ensures that each party has a sound view of the data, and that it will not be changed accidentally by another entity.

The CICS asynchronous API achieves this goal by using CICS channels and containers. The RUN TRANSID command can optionally specify a channel to pass data to a child transaction. The command gives a copy of the data to the child. Hence the parent and child have their own view of the data, and data integrity can be maintained. On a FETCH CHILD or FETCH ANY command, a channel can be returned. The returned channel is renamed by the API, so that it does not clash with an existing channel that the parent owns. It is important to remember to use the new name for the channel that the child returns!

You can issue the FETCH CHILD and FETCH ANY API commands with an optional TIMEOUT parameter. It is suggested to always code the TIMEOUT parameter and to supply the field with a variable (perhaps read in the values from some properties file). By doing this, you have future durability of code. You can always change the timeout value (or specify 0 which indicates that the timeout should be ignored), without the need to alter the source code.

The CICS asynchronous API solution might affect the amount of resources that the system consumes. Understanding the impact of design choices helps with the overall success of the application.

In this publication, we've included four examples of using the Asynchronous API. Three of those examples use request and response information that is shared between the parent and child tasks. The fourth example is a 'Fire-and-Forget' design and is covered in 7.3, “Fire-and-Forget” on page 55. In that example, the parent provides information to the child task, but does not check for the completion, order, or response information regarding the child tasks. Using this model and changing from non-threadsafe START TRANSID command to the threadsafe RUN TRANSID shows a reduction in CPU. This model also delivers an increase in throughput by reducing each child task's response time.

A performance study was conducted to determine the difference in initiating a child task with EXEC CICS RUN TRANSID versus by using the EXEC CICS START command. The test and its detailed results are documented in IBM CICS Performance Series: CICS TS for z/OS V5 Performance Report, SG24-8298. The performance study found that the EXEC CICS RUN TRANSID is no more expensive to execute than the simpler EXEC CICS START command. Yet, it provides all the benefits of the CICS asynchronous API. Here is a summary of those findings:

These assertions are further supported by the assessments that Walmart made, which are discussed in 7.3, “Fire-and-Forget” on page 55.

Chapter 4, “Our initial sequential approach” on page 19 examined the initial approach that Walmart took to determine whether sequential processing would be sufficient to accommodate the requirements of a particular application. In that case, it was determined that sequential processing was not sufficient and that a framework for parallelism that uses asynchronous methods would be necessary.

Walmart's Event Processing System (EPS) search service was initially developed for that one application. But it is a service that could also be deployed for and leveraged by other applications that might not have as extreme requirements.

Despite the benefits provided by the CICS asynchronous API, there is still a considerable amount of overhead for coordination, collation, and management, which ensure asynchrony and parallelism for that service. So, the service was built to also be deployable as a sequential solution for applications with less demanding requirements. That way, EPS does not incur extra processing unnecessarily.

This concern might not apply to all implementations of asynchronous processing. Each potential design must be assessed individually. However, it's worth noting that additional complexity and overhead might not be warranted, even in deployments of a single service with differing circumstances.

Other system resources and settings are relevant when you consider asynchronous implementations. These resources and settings can affect not only the CICS regions that are involved, but the entire system. Some prominent examples are highlighted in this section.

IBM CICS asynchronous API commands are threadsafe and do not need to switch between TCBs. This feature saves CPU time and lowers response time in your application. The MAXOPENTCB parameter in the System Initialization Table (SIT) controls the number of TCBs that are allocated in a CICS region. You use the CEMT INQUIRE MAXOPENTCB command to access and manage this parameter. Your setting for this parameter has either a positive or negative effect on the dispatching and performance of both the parent and child tasks within a region.

The OMVS parameters MAXPROCSYS and MAXPROCUSER also affect multi-threaded tasks within a system and within a region. You must review these settings and possibly adjust them to support the number of OTE TCBs that are used by the increased number of concurrent parent and child tasks.

The number of tasks that are allowed to run in the system affects system performance. If the maximum task specification (MXT) is set too high, tasks might not get resources in a timely fashion. If the maximum task specification (MXT) is too low, tasks might be delayed because they are waiting to be dispatched. The IBM Redbooks publication IBM CICS Asynchronous API: Concurrent Processing Made Simple, SG24-8411 provides a detailed discussion and formula for calculating MXT.

Along with setting MXT, attention should be given to MAXOPENTCB. As discussed in the previous topic regarding threadsafe considerations, these values affect each other.

The TRANCLASS parameter is another resource affects the dispatching of both the parent and child tasks. In Walmart's EPS design, which is described in Chapter 2, Walmart assigned a TRANCLASS for the parent task and a different TRANCLASS for the child task. The maximum child tasks that the parent task initiates in EPS can reach 255. In turn, the value of TRANCLASS for the child task is set to a value considerably higher than the value for the parent task. The separation of parent and child tasks into different TRANCLASS names is critically important to a successful parallel process.

Testing of asynchronous patterns can be tricky. There are many combinations of states for the parent and child transactions. Also, it might be difficult to force timing windows, and to be sure that niche timing windows have been tested appropriately. For example, do you know what will happen if Child 1 is delayed, Child 2 is queued on a transaction class, and the parent has terminated already?

Testing of the Walmart EPS service was performed by using two techniques. The EPS parent task is invoked by a REST request. So, both of the techniques involved some form of a REST client. The initial testing was performed through a REST client, of which there are several on the market for little or no cost.

More advanced load testing with multiple client requests was performed for stress test analysis, resource consumption, and contention identification.

Walmart used the following process to test with a single REST client:

More advanced testing was performed. A driver transaction was created in a CICS region that issued several concurrent 'Fire-and-Forget' child tasks. The tasks used either the START TRANSID or RUN TRANSID, where the child task was a REST client that issues WEB SEND and other WEB-related API commands. Each of these child tasks would create an HTTP REST request that specifies the URL of the EPS task. This EPS task can be processed by a single CICS region. Or the task can be load balanced across multiple CICS regions by using the z/OS TCP Sysplex Distributor and WLM.

Initial testing of the driver transaction was performed by using CEDF to ensure that initiation of the child tasks was performed properly. CEDX was also used in the same region to capture a REST client child task. This approach ensured a successful invocation of the EPS service through use of the WEB SEND command. The test team used the same method of capturing CEDF of the parent task and CEDX of a child task in the target EPS region. This approach ensured that the initial testing that used multiple REST clients was successful.

When this initial testing proved satisfactory, it was time for the next steps:

For all the benefits of asynchronous patterns, they do imply a higher cognitive load on viewers of the algorithm. This load affects the initial application developers, and also the defect fixers, future feature developers, servicing teams, and anyone that clones the code for use elsewhere.

As we discussed in 2.1, “Asynchronous Processing” on page 6, many clients have used different technologies to implement their own asynchronous frameworks. Typically, they have many parts, including STARTs, polling DELAYs, ECBs, TS queues, clean up transaction, and so on. Each technology brings with it a set of complexities.

The CICS asynchronous API is IBM-supported and is designed with simplicity in mind. Our advice is to keep your logic as simple as possible. For example, you might not need to issue a FREE CHILD command if your application is just about to complete.

Asynchronous processing can be greatly beneficial to increased performance and improved user experiences for your applications. However, it can also be complicated and difficult to implement. In this publication, we highlighted all of these concepts with real-world situations that the Walmart engineering team experienced. We also strove to demonstrate that the introduction of the CICS asynchronous API took z/OS developers significantly farther toward the realization of these benefits, while reducing the associated difficulties. The CICS asynchronous API brings simplicity, performance, and stability into the asynchronous processing realm. We hope that it inspires creativity and empowers all readers to build better services for our businesses.