Other implementation patterns
In addition to the implementation that is described in this publication, Walmart leverages the CICS asynchronous API in several other ways. It is not feasible to go into the same level of detail for each of these scenarios. At the same time, it is important to describe the associated architectural patterns that can make extensive use of the capability.
7.1 Unordered Responses
The unordered response scenario is related to the EPS search service that has been the main focus of this publication. After the initial requirement had been satisfied, Walmart identified this additional search pattern in a new use case.
Nearly all of the same conditions from the original EPS search service exist in this new situation. The same volume, event structure, data repository, SLAs, and so on, are in place. So, the parallelized I/O solution is also still used. As a reminder, the high-level design of this solution is illustrated in Figure 7-1.
Figure 7-1 Asynchronous Parallel I/O
So, this is effectively the same solution. However, there is one key distinction between the original and new use cases. The new use case does not require search results to be returned in chronological order. Consider this example in the new use case: The parent task initiates three child tasks in order, one by one. Unlike the EPS search service scenario in earlier chapters, the responses of the child tasks 1, 2, and 3 are unordered. Child 3, for example, can respond as soon as it finishes with its task, whether or not Child 1's task is complete.
This is a subtle, but important difference that is demonstrated in Figure 7-2.
Figure 7-2 Unordered responses
Figure 7-2 shows an example of the unordered response scenario. Parent requests move from left to right along the x-axis. Child responses move from right to left. The passage of time is visible from top to bottom in the y-axis.
The sequence of events in Figure 7-2 shows that Child 3 has a faster processing time than the other two children. There can be many reasons for this. Perhaps Child 3 has less data to process or that the data was easier to access. For whatever reason, Child 3 was able to return its data more quickly than the other children, even though it was the last child to be initiated. Eventually, Child 1 responds, then Child 2 responds. The parent task is free from the constraint of returning results to the client in chronological order. As a result, the parent task can fetch and deliver results as they are returned from the children. The FETCH ANY command is used to accommodate this behavior.
As with the main use-case covered in Chapter 6, the parent task uses the RUN TRANSID command to initiate the child tasks (Example 7-1).
Example 7-1 RUN TRANSID in parent task
***************************************************************** 00236000
* Issue RUN TRANSID for child task to check 7 event logs. * 00237000
* Store Child token in current array entry. * 00237000
***************************************************************** 00238000
1000-RUN-TRANSID. 00239000
ADD 1 TO RC-REQUEST-COUNT.
EXEC CICS PUT
CONTAINER(RC-REQUEST-CONTAINER)
CHANNEL (RC-REQUEST-CHANNEL) 02020300
FROM (RC-REQUEST)
FLENGTH (RC-REQUEST-LENGTH)
NOHANDLE
END-EXEC.
EXEC CICS RUN 02020100
TRANSID (RC-TRANSID) 02020200
CHANNEL (RC-REQUEST-CHANNEL) 02020300
CHILD (RC-CHILD) 02020300
NOHANDLE 02020400
END-EXEC. 02020500
MOVE RC-CHILD TO RC-CHILD-TOKEN (ARRAY-INDEX).
MOVE 'A' TO RC-ENTRY-STATUS(ARRAY-INDEX).
1000-EXIT. 00239000
EXIT.
00235000
Example 7-2 FETCH ANY in parent task
***************************************************************** 00236000
* Issue FETCH ANY for child tasks. * 00237000
* Process until every child is complete or time-out occurs. * 00237000
***************************************************************** 00238000
2000-FETCH-ANY. 00239000
MOVE TIME-OUT-VALUE TO RC-FETCH-TIMEOUT.
EXEC CICS FETCH 02020100
ANY (RC-FETCH-CHILD) 02020200
CHANNEL (RC-FETCH-CHANNEL) 02020300
COMPSTATUS(RC-FETCH-COMP) 02020300
TIMEOUT (RC-FETCH-TIMEOUT) 02020300
NOHANDLE 02020400
END-EXEC. 02020500
IF EIBRESP NOT EQUAL DFHRESP (NORMAL)
OR RC-FETCH-COMP NOT EQUAL DFHVALUE(NORMAL)
PERFORM 9100-SEARCH-FAILED THRU 9100-EXIT.
PERFORM 2100-CHECK-TOKEN THRU 2100-EXIT
WITH TEST AFTER
VARYING ARRAY-INDEX FROM 1 BY 1
UNTIL RC-FETCH-CHILD EQUAL RC-CHILD-TOKEN(ARRAY-INDEX).
2000-EXIT. 00239000
EXIT.
With this approach, the parent still maintains an inventory of the child tasks, but it no longer has to process the results in a particular sequence. It can simply issue the FETCH ANY and process any available results until all child tasks have reached completion. This approach typically results in improved responsiveness for the client.
Although this example is specifically related to the EPS service, this general pattern is quite common in asynchronous processing and might be applicable in a great number of situations.
7.2 Updating remote hosts (High Latency)
This example somewhat resembles the scenario that is described in 7.1, “Unordered Responses” on page 49. The difference is instead of child tasks interacting with a local data store, they must send information to other remote systems over the network via HTTP.
This implementation is related to the existing zFAM product that was developed by Walmart. The zFAM product is an online object store that functions as a NoSQL (Not only SQL) database.
|
Additional information about zFAM is available through the GitHub link and the other IBM Redbook publications, which are described in 2.3.1, “Walmart at a Glance” on page 10.
|
The product stores and retrieves objects in a local data store as requested from web-based clients. To extend its High Availability (HA) posture, the product must also replicate new or updated objects to multiple other geographically dispersed hosting sites.
The replication activity is expected to be asynchronous and non-blocking to the client. So, this function is assigned to subordinate tasks while the primary task continues on processing the client request. The general depiction of this design is included in Figure 7-3.
Figure 7-3 Replicate to remote hosts
In this example, the order that the child tasks reach completion is not important. But confirmation that they each task actually reaches completion is critical. Figure 7-4 represents the associated process for the parent task:
1. Process the request from the client.
2. Start the child tasks to perform the replication.
3. Provide a response to the client.
4. Note the responses from the child tasks when they complete.
Figure 7-4 Remote host updates
This process can be updated to take advantage of the simplicity provided by the CICS asynchronous API. So, the children tasks will be invoked with the RUN TRANSID command. Like the example in section 7.1, there is no requirement for ordered responses. So, the FETCH ANY command is used to retrieve the completion status from the child tasks. However in this scenario, there is no data to return from the child tasks. Also, the client request has likely been satisfied before the child tasks have completed, so there is no data container to retrieve in this example (Example 7-3 on page 54):
Example 7-3 FETCH ANY in parent task
***************************************************************** 01500000
* Issue FETCH ANY for child tasks. * 01510000
* Process until every child is complete or time-out occurs. * 01520000
* Channel is not required, as a GET CONTAINER is not issued * 01530021
* for the Child tasks in this process. * 01540022
***************************************************************** 01550000
2000-FETCH-ANY. 01560000
MOVE TIME-OUT-VALUE TO RC-FETCH-TIMEOUT. 01570021
01580021
EXEC CICS FETCH 01590000
ANY (RC-FETCH-CHILD) 01600000
COMPSTATUS(RC-FETCH-COMP) 01610000
TIMEOUT (RC-FETCH-TIMEOUT) 01620000
NOHANDLE 01630000
END-EXEC. 01640000
01650021
IF EIBRESP NOT EQUAL DFHRESP (NORMAL) 01660021
OR RC-FETCH-COMP NOT EQUAL DFHVALUE(NORMAL) 01670021
PERFORM 9100-SEARCH-FAILED THRU 9100-EXIT. 01680021
01690021
PERFORM 2100-CHECK-TOKEN THRU 2100-EXIT 01700021
WITH TEST AFTER 01710021
VARYING ARRAY-INDEX FROM 1 BY 1 01720021
UNTIL RC-FETCH-CHILD EQUAL RC-CHILD-TOKEN(ARRAY-INDEX). 01730021
01740021
2000-EXIT. 01750000
EXIT. 01760021
01770000
|
Note: Example 7-3 on page 54 is similar to previous examples of the FETCH ANY command. However, in this scenario the CHANNEL parameter has been omitted as no container data is expected back from the child transcation
|
Some of the details in this example are particular to Walmart's specific use-case. Nonetheless, this general pattern is seen frequently in applications. Any time that relatively high-latency communications across the network are required, asynchronous processing has value. For example, if you are using REST (Representational State Transfer) interfaces or SOAP (Simple Object Access Protocol) services, it is typically beneficial to process those communications asynchronously. A key benefit is that you avoid blocking the primary task from doing other work. The CICS asynchronous API helps to simplify this processing activity.
7.3 Fire-and-Forget
The previous examples demonstrate the use of asynchronous processing that is focused on forms of parallel or concurrent processing. In contrast, the pattern in this section involves a Fire-and-Forget model. In this example, we do not use a child task to gather data that the parent needs. Instead, we focus on initiating children that act autonomously without returning data, or even status information, to the parent.
The previous examples in this chapter were also primarily used to highlight the simplicity that the IBM CICS asynchronous API provides. This section shifts that focus to other important dimensions: efficiency and performance.
This scenario involves a system service that performs this process:
1. Capture information assets (for example, purchase orders) that flow through one application.
2. Sending them into another application for other processing.
3. As the assets are captured, initiate a child task that is associated with each asset and interacts with the receiving application.
As previously noted, these child tasks act autonomously and are never fetched by the parent task. Figure 7-5 shows a very simple view of this process.
Figure 7-5 Fire-and-Forget
In reality, a Fire-and-Forget service as described here can be very complex and well beyond the scope of this publication. The description of the process is provided only to give some basic context for the scenario. The important part is that, up to now, we used the START TRANSID command to initiate the child tasks. Now, this function is being replaced with the RUN TRANSID. As a result, we realize significant performance improvements.
It should first be noted that simplicity is still a relevant characteristic in this example. In particular, the change from START TRANSID to RUN TRANSID is a relatively easy and unintrusive modification. For example, here is the code to issue the START TRANSID (Example 7-4):
Example 7-4 Fire and Forget START TRANSID in parent task
***************************************************************** 00234400
* Start - Data for child task * 00234500
***************************************************************** 00234600
01 SC-CHANNEL PIC X(16) VALUE 'CHILD-CHANNEL '. 00100000
01 SC-CONTAINER PIC X(16) VALUE 'CHILD-CONTAINER '. 00100000
01 SC-TRANSID PIC X(04) VALUE 'RICH'. 00100000
01 SC-LENGTH PIC S9(08) COMP VALUE ZEROES. 00100000
00110000
01 SC-REQUEST. 00100000
02 SC-PO-NUMBER PIC 9(10) COMP-3 VALUE ZEROES.
02 SC-PO-SEQUENCE PIC 9(08) COMP VALUE ZEROES.
02 SC-PO-SEGMENT PIC 9(02) COMP VALUE ZEROES.
02 SC-PO-SUFFIX PIC 9(08) COMP VALUE ZEROES.
00110000
***************************************************************** 00234400
* End - Data for child task * 00234500
***************************************************************** 00234400
***************************************************************** 00236000
* Issue START TRANSID for child task to process PO numbers. * 00237000
***************************************************************** 00238000
1000-START-TRANSID. 00239000
MOVE LENGTH OF SC-REQUEST TO SC-LENGTH.
EXEC CICS PUT
CONTAINER(SC-CONTAINER)
CHANNEL (SC-CHANNEL) 02020300
FROM (SC-REQUEST)
FLENGTH (SC-LENGTH)
NOHANDLE
END-EXEC.
EXEC CICS START 02020100
TRANSID (SC-TRANSID) 02020200
CHANNEL (SC-CHANNEL) 02020300
NOHANDLE 02020400
END-EXEC. 02020500
1000-EXIT. 00239000
EXIT.
00235000
Notice the similarity of the RUN TRANSID code that replaces the preceding code (Example 7-5):
Example 7-5 Fire and Forget RUN TRANSID in parent task
***************************************************************** 00236000
* Issue RUN TRANSID for child task to process PO numbers. * 00237000
***************************************************************** 00238000
1000-RUN-TRANSID. 00239000
MOVE LENGTH OF SC-REQUEST TO SC-LENGTH.
EXEC CICS PUT
CONTAINER(SC-CONTAINER)
CHANNEL (SC-CHANNEL) 02020300
FROM (SC-REQUEST)
FLENGTH (SC-LENGTH)
NOHANDLE
END-EXEC.
EXEC CICS RUN 02020100
TRANSID (SC-TRANSID) 02020200
CHANNEL (SC-CHANNEL) 02020300
CHILD (SC-CHILD) 02020300
NOHANDLE 02020400
END-EXEC. 02020500
1000-EXIT. 00239000
EXIT.
00235000
Now, compare the effect of these innocuous code adjustments on performance metrics for the process. The baseline metrics with using the START command are as follows (Example 7-6):
Example 7-6 Fire and Forget Performance START TRANSID in parent task
15FEB2019 17:56:12 ------ MAINVIEW WINDOW INTERFACE (V6.3.00) ----------------
COMMAND ===> SCROLL ===> CSR
CURR WIN ===> 1 ALT WIN ===>
>W1 =CHIST=============CICS001==*========15FEB2019==17:56:12====MVCICS===D===88
Task CICS Lcl Task Lcl Task Tran Response CPU Storage I/O Term
Num System End Date End Time ID Time Time HWM Calls ID
45010 CICS001 15FEB2019 17:55:24 JOHN 0.00303 0.00062 58976 0 ZAAA
45020 CICS001 15FEB2019 17:55:24 RICH 0.00106 0.00008 55392 1
45019 CICS001 15FEB2019 17:55:24 RICH 0.00097 0.00008 55392 1
45017 CICS001 15FEB2019 17:55:24 RICH 0.00113 0.00010 55392 1
45018 CICS001 15FEB2019 17:55:24 RICH 0.00091 0.00009 55392 1
45016 CICS001 15FEB2019 17:55:24 RICH 0.00107 0.00009 55392 1
45013 CICS001 15FEB2019 17:55:24 RICH 0.00132 0.00009 55392 1
45014 CICS001 15FEB2019 17:55:24 RICH 0.00128 0.00009 55392 1
45015 CICS001 15FEB2019 17:55:24 RICH 0.00062 0.00009 55392 1
45012 CICS001 15FEB2019 17:55:24 RICH 0.00080 0.00012 55392 1
45011 CICS001 15FEB2019 17:55:24 RICH 0.00059 0.00020 55392 1
The metrics that are associated with using the RUN TRANSID command are as follows (Example 7-7):
Example 7-7 Fire and Forget Performance RUN TRANSID in parent task
15FEB2019 17:56:12 ------ MAINVIEW WINDOW INTERFACE (V6.3.00) ----------------
COMMAND ===> SCROLL ===> CSR
CURR WIN ===> 1 ALT WIN ===>
>W1 =CHIST=============CICS001==*========15FEB2019==17:56:12====MVCICS===D===88
Task CICS Lcl Task Lcl Task Tran Response CPU Storage I/O Term
Num System End Date End Time ID Time Time HWM Calls ID
44941 CICS001 15FEB2019 17:52:35 JOHN 0.00108 0.00054 58976 0 ZAAA
44939 CICS001 15FEB2019 17:52:33 RICH 0.00144 0.00009 55392 1
44940 CICS001 15FEB2019 17:52:33 RICH 0.00140 0.00008 55392 1
44934 CICS001 15FEB2019 17:52:33 RICH 0.00145 0.00008 55392 1
44937 CICS001 15FEB2019 17:52:33 RICH 0.00136 0.00008 55392 1
44938 CICS001 15FEB2019 17:52:33 RICH 0.00134 0.00009 55392 1
44936 CICS001 15FEB2019 17:52:33 RICH 0.00134 0.00009 55392 1
44935 CICS001 15FEB2019 17:52:33 RICH 0.00114 0.00008 55392 1
44931 CICS001 15FEB2019 17:52:33 RICH 0.00117 0.00015 55392 1
44933 CICS001 15FEB2019 17:52:33 RICH 0.00092 0.00009 55392 1
44932 CICS001 15FEB2019 17:52:33 RICH 0.00079 0.00012 55392 1
In comparing these metrics, you see significant differences in the parent task (Tran ID = JOHN).
Table 7-1 Comparative metrics for the START and RUN TRANSID commands
|
|
START command
|
RUN TRANSID command
|
|
Response Time
|
0.00303 (3.03 ms)
|
0.00108 (1.08 ms)
|
|
CPU Time
|
0.00062 (.62 ms)
|
0.00054 (.54 ms)
|
These values significantly improve performance for the parent task. A 13% reduction in CPU consumption is observed, which is enough to be beneficial. More importantly, we see a 65% reduction in response time. These improvements arise because the RUN TRANSID command is threadsafe. Thus, much of the task switching of the non-threadsafe START command is eliminated. You can verify this improvement by inspecting the AUXTRACE of the tasks.
At scale, with large numbers of transactions, these improvements have a substantial positive impact on system throughput. Any pattern that involves a task that starts groups of subordinate tasks, you should consider a change to the RUN TRANSID command. This humble, low-investment change can dramatically improve performance with no additional, or even slightly lower CPU consumption.
7.4 Summary
This chapter looked into a few other examples where the Walmart team is leveraging the CICS asynchronous API to improve processing. These examples also represent implementation patterns that are common in many applications and that might benefit from employing the CICS asynchronous API. Potential benefits include ease-of-use, simplification, and better performance. However, there are always issues to consider when you adopt new technology. Chapter 8, “Considerations” on page 61 will highlight some of these issues.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.