Background
In this Redbooks publication, we follow the experience of the Walmart delivery team as they embrace asynchronous processing patterns in enterprise-grade applications.
We follow their stepped approach, starting with the initial problem space and how they assessed an ambitious set of requirements. In the end, we see how they arrived at a high-throughput, scalable service that minimizes application response times.
We see the benefits that they realized from adopting an asynchronous processing pattern. And we also share the key choices and challenges that they faced.
As with all user experience stories, the benefits that you can realize depend heavily on application architectures and existing assets that you start with. All figures that are quoted are measurements that the Walmart team took. Your results might vary. However, this fact is certain: the Walmart service transformed a 2-minute response time to just 1 second!
There are two main parts to this background chapter:
1. We introduce IBM CICS Transaction Server (CICS TS) as a multi-language application server. We explain how this platform provides an asynchronous application programming interface where you can easily develop and manage asynchronous programming patterns.
2. The Event Processing System (EPS) that Walmart developed. EPS is a large data application that provides a central point for events that are produced and consumed across the Walmart organization.
The chapters that follow this background chapter describe new requirements of the Walmart application and follow the developers' iterative solutions, which include these milestones:
•The initial "traditional-style" sequential programming implementation.
•An implementation that minimized response time, based on a home-grown asynchronous framework.
•Adoption of the IBM CICS asynchronous API capability to arrive at an implementation that minimizes response time and reduces risk and maintenance costs.
The final chapters look at other asynchronous patterns that Walmart is developing and also highlight key choices when you work with such techniques.
2.1 Asynchronous Processing
The paradigm of asynchronous processing is not new in the world of computing, nor to human behavior. Consider these contrastive examples:
•Sequential serving model: Customers wait in a queue to buy popcorn from a single cashier at the cinema. When a customer gets to the head of the queue, and they are served. They make their order, wait for the popcorn to be served, pay for it, and complete their transaction. Then, they leave the queue, and it is the turn of the next customer in line.
•Asynchronous model: A restaurant setting contrasts with the popcorn queue. Multiple groups of diners are seated at tables whenever they arrive. Each table of diners progresses at their own pace. Coordination is done by the restaurant. The restaurant processes tables to reduce response time and to maximize throughput and satisfaction. Table 2 can order and eat their food, request their bill, and complete their transaction, without the need for Table 1 to complete first.
The restaurant example reveals the crux of asynchronous processing. If you can serve/request a service, without being blocked by it, then you are able to follow other paths. This might mean continuing the algorithm, processing more data, calling another service concurrently, or even … serving another table in the restaurant.
If you can work on different parts of a service concurrently, even though those constituent parts take a set amount of effort, the overall service response time can be reduced. Basically, you make better use of your time!
Over the 50 years of CICS, application developers have implemented their own asynchronous solutions for CICS applications. Often, these frameworks involve piecing together technologies, including these:
•EXEC CICS START API
•CICS Business Transaction Services
•CICS Event processing
•Passing data by using TD queues / TS queues / Shared GETMAINs and FREEMAINs
•Returning control by using set times / polling code by using the DELAY API / posting ECBs
•Using other products, such as IBM MQ
•And other IBM and non-IBM technologies
This list shows that there is no single method to implement asynchronous patterns in CICS. Also, these technologies are often used for scenarios other than what they were originally designed for. For example, the START API might be used with proprietary GETMAINs/FREEMAINs to pass data, thereby gaining a response on a Fire-and-Forget API. And this mechanism might be coupled with suboptimal polling algorithms that use looping DELAY calls to check for response arrivals.
Research also reveals how asynchronous programming techniques tend to be alien to traditional CICS developers. Many asynchronous projects have been abandoned or had limited success, due to factors like these:
•Lack of skills
•A brittle offering with a high rate of failure
•High levels of complexity
•Higher cost of development
•Increased cost of on-going maintenance
A common difficulty is the exposure of difficult-to-test timing windows. For example, a service might be slow to respond, does not respond at all, or returns multiple responses for multiple instances. As a result, data becomes out of sync. This scenario has caused some high-profile cases where the wrong personal data was served to incorrect recipients in production.
Rare failures might be seen as acceptable in some environments. However, in enterprise-grade applications, such as those served by CICS, the high throughput and transaction rates that are required can easily expose problems that are related to timing windows.
The CICS asynchronous API is designed to be a simple to use, yet powerful, IBM-supported set of APIs that support the implementation of asynchronous processing patterns in CICS applications.
2.2 IBM CICS and the CICS Asynchronous API
2.2.1 IBM CICS Transaction Server
IBM introduced modular mainframe computers (System 360) in the mid 1960's and shipped rudimentary software at no extra charge. In 1969, IBM introduced more sophisticated software to run on their mainframes, and one of these products was called Customer Information Control System (CICS). CICS originated with the need in the public utility industry to update their repositories online from multiple sources, without locking out those repositories. Very quickly it became clear that this software was destined for a much wider community than just the public utilities.
CICS was intended to be in-the-middle software (Middleware) to manage processes across the operating system, data repositories, application and business logic, and to coordinate communication between these elements. In areas where it made sense, CICS would take management responsibilities, as in these examples:
•Originally CICS kept its own logs.
•The need for subdispatching work on a single TCB meant that CICS architecture required its own dispatcher.
Since this first release, CICS has continued to evolve. It adopts new technologies where appropriate, and it takes advantage of enhancements that are made in the operating system. Also, CICS maintains its reputation as the premier mixed-language application server. When you couple the qualities of service of running CICS with IBM Z hardware, you get a premier mixed-language application server that achieves these enterprise-grade features:
•Excellent performance
•Unimpeachable security for integrity of information
•High resilience with the ability to scale to a massive extent
To make it easy for programmers to write applications to run in CICS, the CICS Transaction Server V5.4 provides help with asynchronous processing in the form of the CICS Asynchronous API. Walmart use of this API in IBM Z with CICS resulted in throughput that was beyond belief, as compared to the existing processes.
|
"For over a decade, prophets have voiced the contention that the organization of a single computer has reached its limits and that truly significant advances can be made only by interconnection of a multiplicity of computers." IBM Fellow Dr. Gene Amdahl in 1967.
(Amdahl's research alerted computer scientists to the complexity and limitations of parallel computing.)
|
2.2.2 CICS Asynchronous API
The CICS asynchronous API is designed to be a simple, yet powerful API that application developers can use to confidently deliver asynchronous patterns into CICS applications.
The CICS asynchronous API was added to CICS Transaction Server V5.4, and the full breadth of the capability includes these features:
•A set of threadsafe EXEC CICS and JCICS API commands.
•Monitoring enhancements.
•Statistics enhancements.
•Transaction tracking enhancements.
•Trace and dump enhancements.
•New CICS policy threshold type.
•Tooling support.
•The API, which covers the three main requirements of asynchronous processing:
a. Initiation of an asynchronous transaction to execute supporting work, without blocking the initiator.
b. Efficient return of results from the work that is initiated.
c. Safe passing of data.
The parent-child relationship is at the heart of the CICS capability. A parent can have zero to many child associations. All parent and child entities are themselves CICS transactions. This approach makes their behavior predictable and easy for experienced CICS application developers to understand. The difference between these and 'regular' CICS transactions is that CICS maintains an internal understanding of scopes and relationships. That way, the CICS system can pass data through CICS channels, and also safely manage task-termination activities. Many of the hurdles that face the application developer are thereby removed.
For example, a child transaction can be initiated and it might complete and terminate before the parent is ready to fetch the results. This is not a problem for CICS. CICS maintains the child's channel data until the parent is ready to request the channel. In other situations, the parent might decide to not fetch the results of numerous child transactions. Instead it simply terminates its execution. In this situation, the CICS system is aware that the parent has terminated, and cleans up remaining channels associated with child tasks.
In addition to scenarios that involve numerous child transactions, the CICS asynchronous API also provides benefits in single-child scenarios. For example, when the parent calls an external service via HTTP, the parent is typically blocked until the external service responds. By running a child transaction to invoke the web service, the parent is not blocked by the service call. The parent can continue with other work. Perhaps the parent can take advantage of the timeout feature of the CICS asynchronous API and provide a responsive service to its initiator.
The CICS Asynchronous API includes the following set of threadsafe EXEC CICS commands:
•RUN TRANSID
•FETCH CHILD
•FETCH ANY
•FREE CHILD
RUN TRANSID
A CICS transaction can use the RUN TRANSID command to begin another child transaction. With this command, you can specify a Channel name to pass container data to the child transaction. The command returns a child token parameter that is used to reference the child transaction in other CICS asynchronous API commands.
FETCH CHILD
The FETCH CHILD command enables a parent transaction to retrieve data from a completed child transaction. The child token that is issued on the RUN TRANSID command identifies which child transaction details to retrieve. Data in containers can be returned to the parent transaction through a new Channel that is returned when the API command executes.
By default, the FETCH CHILD command blocks until the child transaction has terminated. An optional TIMEOUT parameter allows the issuing parent transaction to specify the length of time that it is willing to wait for the child transaction to complete.
FETCH ANY
The FETCH ANY command works in a similar fashion to the FETCH CHILD command. The main difference is that the FETCH CHILD command specifies the child transaction that it is interested in. In contrast, the FETCH ANY command returns the results of any one of the child transactions that have completed (that have not been fetched yet).
A return parameter on the FETCH ANY command specifies the child token of the details of the returning transactions. This can be matched against the RUN TRANSID child parameter to identify which child transaction is being returned.
You can issue multiple RUN TRANSID commands, and then issue multiple FETCH ANY commands to retrieve the results from those child transactions. The parent transaction can then process the child transactions as soon as they complete, in any order. This way, the response time of the application is minimized.
By default, the FETCH ANY command blocks until at least one child transaction has completed (or there are no further unfetched child transactions). An optional TIMEOUT parameter allows the issuing parent transaction to specify the length of time that it is willing to wait for a child transaction to complete.
FREE CHILD
The FREE CHILD command is issued by a parent transaction to disassociate the child transaction that is specified.
When a child transaction is freed, internal control blocks are removed and, if it exists, the child's unfetched channel data is not preserved after the child terminates. Issuing the FREE CHILD command does not prevent, nor halt the child transaction from executing.
In short-lived applications, it might not be necessary to issue FREE CHILD commands, because control blocks and channels are cleaned up by CICS when the parent and child transactions are completed. However, in some situations you might benefit from issuing the FREE CHILD command, in applications that include these conditions:
– Long running parent transactions
– Large data in channels
– High number of child transactions
– Unrequired information that is returned from child transactions
In this background section, we shared an overview of the CICS asynchronous API capabilities and looked at the individual API commands. For more details, including step-by-step examples, read Redbooks publication IBM CICS Asynchronous API: Concurrent Processing Made Simple, https://www.redbooks.ibm.com/abstracts/sg248411.html.
2.3 Walmart and the Event Processing System
In this section, we take a closer look at the Walmart environment and their Event Processing System (EPS). This Walmart-developed service forms the use case for most of this Redbooks publication.
2.3.1 Walmart at a Glance
Walmart is a large user of IBM's CICS technology. Walmart deployed CICS in a parallel sysplex environment to gain the ability to scale and preserve availability that is designed to keep the systems running. Walmart runs approximately 750 million CICS transactions a day that are deployed across multiple environments and data centers.
Walmart regularly shares their experiences and encourages others to do the same. They hope that shared learning will lead to more creative technological advances. Look for other Walmart experiences in IBM Redbooks publications on similar topics including these:
•How Walmart Became a Cloud Services Provider with IBM CICS, SG24-8347, https://www.redbooks.ibm.com/abstracts/sg248347.html
•IBM CICS and the Coupling Facility: Beyond the Basics, SG24-8420, https://www.redbooks.ibm.com/abstracts/sg248420.html
In addition, Walmart regularly contributes to the open source world and has contributed three IBM z/OS® cloud services that use CICS. These can be found at the following GitHub repositories. Walmart encourages downloads and interaction with other users.
•zUID: z/OS based unique identifier generator (https://github.com/walmartlabs/zUID)
•zECS: z/OS based enterprise cache service (https://github.com/walmartlabs/zFAM)
•zFAM: z/OS based file access manager (https://github.com/walmartlabs/zECS)
2.3.2 Walmart Event Processing System (EPS)
The Event Processing System (EPS) service, as its name suggests, is centered around the production and consumption of events in the Walmart organization.
Figure 2-1 on page 11 shows the overall framework for EPS. Around the outside of the architecture are the Systems of Engagement (SoE). Various business areas can sign up in EPS to produce events. Typically, these areas automate the generation of events for EPS to record. For example, delivery shipments produce an event when a truck trips a sensor on arrival at the depot, when the POD is signed as being received at the final destination, when the vehicle stops to refuel, and so on. These SoEs are known as the event producers.
The events are initially processed by a highly distributed cloud application. Then, they are sent to be stored centrally in the Systems of Record (SoR), as an entry in a file for VSAM/RLS (Virtual Storage Access Method with Record Level Sharing) on IBM Z. The qualities of service on mainframes enables the storage of the large amount of data that the event producers generate. The EPS service is projected to publish 500 million events per day. In addition, between 7 to 30 days of events are stored.
Business areas, in addition to registering as event producers, can also register as event consumers. Typically, analysis and consumption of the events are human-led activities.
Figure 2-1 Event Processing System
Each event that is logged in EPS is that of a distinct incident. There are two parts to every event entry, and these parts form the entire event in the log.
Table 2-1 Parts of an EPS event
|
Part
|
Description
|
|
Event key
|
Consists of the date and time, among other details. The key serves the following purposes:
•Searching the events based on time.
•Distributing sets of events for processing.
|
|
Event payload
|
The data stored by the event publisher. Its format and size differ between different publishers. It conforms to the publisher's schema at time of first registration to EPS.
|
2.4 Background Summary
In this chapter, we introduced the motivation for asynchronous processing and the benefits and challenges that it can bring to enterprise-grade applications. We also reviewed why large-scale operations depend on the robustness and reliability of IBM Z and on services like these:
•The CICS Transaction Server (CICS TS, the world's premier mixed-language application server) and why it makes such a good platform to develop and serve enterprise applications.
•The capabilities of the CICS Asynchronous API that CICS TS V5.4 introduced, which greatly simplify the delivery of asynchronous application patterns.
•The Walmart Event Processing System (EPS) application, especially its data lake of events that is open to all parts of the Walmart organization to not only produce but consume events from.
In Chapter 3, “Requirements and challenges” on page 13, we look at the new requirements for EPS. A staged approach in Chapter 4, “Our initial sequential approach” on page 19 shows how the delivery team investigated a traditional sequential solution to the problem. However, that approach failed to address the aggressive targets. So, we see in Chapter 5, “Homegrown asynchronous solution” on page 23 how the team rearchitected the application processing pattern to implement a 'homegrown' asynchronous architecture. Lastly, Chapter 6, “IBM CICS asynchronous solution” on page 37 shows the benefits of adopting the CICS asynchronous API.
Having proved the success of adopting the CICS asynchronous API with their EPS application, Walmart has related projects under way. In Chapter 7, “Other implementation patterns” on page 49, we explore a subset of those projects that use a range of asynchronous processing patterns.
In Chapter 8, “Considerations” on page 61, we discuss themes and areas of interest that the Walmart team discovered while it worked with asynchronous patterns in CICS applications. It is likely that many clients that have similar projects will have similar areas of interest. So, be sure to read how Walmart approached these situations in Chapter 8, “Considerations” on page 61.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.