IBM CICS asynchronous solution
This chapter explores the version of the search function that uses the IBM CICS asynchronous API to perform parallel I/O processing. We also describe the actions that were needed to migrate from the homegrown solution to this new design.
|
"The way the processor industry is going, is to add more and more cores, but nobody knows how to program those things. I mean, two, yeah; four, not really; eight, forget it." --Steve Jobs, Apple
|
The preceding quotation reflects the complexity and difficulty of programming for parallelism. The complexity of the homegrown solution in Chapter 5, “Homegrown asynchronous solution” on page 23 illustrates this fact. The design in this chapter demonstrates how the CICS asynchronous API reduces that difficulty.
6.1 Design
The basic sequence of events for processing each search request is unchanged from what was described in 5.3, “Design” on page 25 and is repeated here:
1. Receive request from client
2. Use search parameters to identify and start I/O tasks
3. Monitor I/O tasks for completion
4. Gather and return results to client in the proper order
Also, the same general components are used to accomplish these steps, including this:
•A parent task that receives the client request and determines how many child tasks to start.
•The child tasks that search sections and return relevant results to the parent task.
The parent task will then return the results to the client. This general design is depicted in Figure 6-1
Figure 6-1 High-Level Design
The simplification that the CICS asynchronous API brings becomes apparent when we explore the details of the design components. These details are highlighted in the following sections. Like Chapter 5., “Homegrown asynchronous solution” on page 23, these details include the following general activities of a standard asynchronous pattern:
•Prepare Data for Child
•Initiate Child
•Check for Completion
•Retrieve Data from Child
•Perform Housekeeping
6.1.1 Prepare Data for Child
Only slight changes are needed for this stage of the process, as compared to the homegrown solution. The ingest handling and fundamental steps for child data preparation that are described in 5.3.1, “Prepare Data for Child” on page 26 are the same and can be summarized as follows:
1. The WEB RECEIVE and WEB EXTRACT commands retrieve search criteria from the client HTTP GET request, and control is passed to the parent program.
2. Based on the total time range of the search request, a proprietary algorithm determines the quantity and duration of intervals to search, and therefore the number of child tasks to prepare.
3. The parent task creates an in-memory array that includes various bits of identifying information that are used to manage each child task.
4. The parent task uses this identifying information (along with its EIBTRNID and EIBTASKN values) to create a unique channel name. Then it stores search criteria by using a PUT CONTAINER command for each child task.
From this perspective, the high-level design is similar to the homegrown solution as it was previously depicted in Chapter 5., “Homegrown asynchronous solution” on page 23.
Figure 6-2 Prepare Data for Child
However, when you use the CICS asynchronous API, there is one key difference to this section of the solution. That difference is related to the array that contains identifying information for each child task. Now, the array also includes the child token and container name for each entry. The array must be defined to accommodate these values, which become available at child initiation. This information is used by various invocations of the asynchronous API in subsequent stages of the process. See Example 6-1.
Example 6-1 Array in parent task
*********************************************************************** 00581362
* Start TSQ Table List. * 00581489
* Maximum 255 entries. * 00581489
*********************************************************************** 00581562
TT_DSECT DSECT
TT_CHILD DS CL16 Child Token (RUN TRANSID)
TT_CHAN DS CL16 Child channel (FETCH)
TT_STAT DS F Child COMPSTAT (FETCH)
TT_RESP DS F Child Response (FETCH)
TT_T_OUT DS F Child Timeout (FETCH)
*
TT_START DS 0CL09
TT_S_HH DS CL02 TS Start HH
TT_S_MM DS CL02 TS Start MM
TT_S_SS DS CL02 TS Start SS
TT_S_MS DS CL03 TS Start MS (not part of TSQ name)
DS CL07 Align
*
TT_END DS 0CL09 TS End time 00476045
TT_E_HH DS CL02 TS End HH
TT_E_MM DS CL02 TS End MM
TT_E_SS DS CL02 TS End SS
TT_E_MS DS CL03 TS End MS
DS CL07 Align
*
TT_209 DS 0CL09 TS Resume time 00476045
TT_R_HH DS CL02 TS Resume HH
TT_R_MM DS CL02 TS Resume MM
TT_R_SS DS CL02 TS Resume SS
TT_R_MS DS CL03 TS Resume MS
DS CL03 Align
*
*
TT_STAT DS CL01 TS Table entry status
* A - active
* I - inactive
* R - resume (after 209)
* S - started
* C - completed
*
TT_IDX DS CL03 TS Index number
*
TT_E EQU *-TT_DSECT Entry length
6.1.2 Initiate Child
This stage of the process includes more substantial changes. This stage of the homegrown solution included these initial actions:
1. Issuing a START TRANSID with the CHANNEL option for each child task.
2. Upon completion of each START TRANSID command, the parent task updates a status flag in the control array.
Now, in the asynchronous API solution this process is replaced with the following actions:
2. Upon completion of each RUN TRANSID command, the parent task updates a status flag in the control array.
3. The array is also updated with the token from the CHILD parameter of the RUN TRANSID command.
Example 6-2 RUN TRANSID CHANNEL for parent task
*********************************************************************** 00790110
* Issue RUN for Child task providing the Channel name, which will * 00790219
* be used to provide request CONTAINER information. * 00790219
*********************************************************************** 00791019
SY_0138 DS 0H 00791110
MVC PP_TRAN,EIBTRNID Move current TranID
MVC PP_TRAN+1,EC_CHILD Move child Identifier
*
MVI TT_COMP,C'S' Move 'started' indicator
MVC PP_IDX,TT_IDX Move index number
*
EXEC CICS RUN X
TRANSID (PP_TRAN) X
CHILD (PP_CHILD) X
CHANNEL (PP_CHAN) X
NOHANDLE
* 00791223
MVC TT_CHILD,PP_CHILD Move Child token to table
As before, the channel option specifies a name that corresponds with the array index of the particular child task. And the status flag is used by the parent task to manage the child tasks. The value of the child token is used to simplify some of the remaining steps of the process.
In the homegrown solution, each child task ran an ASSIGN CHANNEL command to obtain the unique channel name that was created by the parent task for that particular child. That name was used by the child task to create a TS queue name for response status information. This technique is eliminated in the asynchronous API solution. Instead, a PUT CONTAINER command is used to achieve this purpose. The GETMAIN SHARED is also replaced with a local (non-shared) GETMAIN and another PUT CONTAINER operations.
In the new solution, the child task issues a GET CONTAINER to acquire the search criteria from the parent task. Then it issues a local GETMAIN to house its result set. Then, each child task runs STARTBR and READNEXT commands, which remain identical to the original process. These actions peruse the records in the child task's assigned interval. Any matches that it finds are placed into the GETMAIN storage
After the child task processes its assigned interval, the two PUT CONTAINERs commands run (Example 6-3):
•One PUT CONTAINER provides status information to the parent task.
•The other PUT CONTAINER passes the result set from the local GETMAIN to the parent task.
Example 6-3 Issue PUT CONTAINER for response in child task
*********************************************************************** 01111599
* Put response information in Container for Parent task to process. * 01112099
*********************************************************************** 01120010
TS_0030 DS 0H 00791110
EXEC CICS PUT X
CONTAINER(C_STATUS) X
CHANNEL (PP_CHAN) X
FROM (PP_REC) X
FLENGTH (PP_FLEN) X
NOHANDLE
* 01077147
CLC PP_CODE,=C'204' Status 204?
BRC B'1000',TS_0099 ... yes, no response set
L R4,RA_ADDR Load response array address
USING RA_DSECT,R4 ... tell assembler
* 01077147
EXEC CICS PUT X
CONTAINER(C_RESULT) X
CHANNEL (PP_CHAN) X
FROM (RA_DSECT) X
FLENGTH (RA_LEN) X
NOHANDLE
* 01077147
DROP R4 ... tell assembler
* 01077147
TS_0099 DS 0H 00791110
L R14,TS_REG Load return register 01070893
BCR B'1111',R14 Return to caller 01070893
*
These changes make a design that looks very similar to the original homegrown solution. But the management of differing storage constructs is now simplified by the use of containers. This is reflected in Figure 6-3.
Figure 6-3 Initiate Child
6.1.3 Check for Completion
The basic approach for this step is similar to the homegrown solution. After all child tasks are started, the parent program begins attempting to process the response from each child task in the order they appear in the array.
However, the details of how this is accomplished are quite different. The techniques from the homegrown solution are almost entirely replaced with CICS asynchronous API commands.
Following the order of tasks in the array, a FETCH CHILD command is issued. The command uses the child token that was captured when each particular task was run. Upon successful response from CICS (and subsequent data retrieval), this action is simply repeated with the child token of the next item in the array. The READQ TS and the STIMERM-based polling loop have been eliminated.
An overall completion time of all tasks must still be considered. The TIMEOUT parameter of the FETCH CHILD command is used for this purpose. The parent task uses a simple internal timer to set the timeout values. The pre-defined timeout for the service is 30 seconds. So, the task uses this value for the TIMEOUT parameter on the initial FETCH CHILD. Then, it decrements the value (based on the internal timer) on each iteration of the process. This approach ensures that the cumulative processing is accounted for when a request times out after running for a long time. See Example 6-4.
Example 6-4 Synchronicity in parent task
*********************************************************************** 00790110
* Issue FETCH for the TT_CHILD token and specify a TIMEOUT default of * 00790219
* 30 seconds. Set status 534 for TIMEOUT, then end the request. * 00790219
* Resources for the FETCH will be stored in the current TT_DSECT * 00790219
* array index. * 00790219
* The CHANNEL returned from the FETCH will be used for subsequent * 00790219
* GET CONTAINER commands to retrieve result set and response status * 00790219
* from the CHILD task. * 00790219
*********************************************************************** 00791019
SY_0220 DS 0H
OC TT_T_OUT,TT_T_OUT Time-out already set?
BRC B'0111',*+10 ... yes, bypass default
MVC TT_T_OUT,=F'30000' ... no, set 30 second default
*
EXEC CICS FETCH X
CHILD (TT_CHILD) X
CHANNEL (TT_CHAN) X
COMPSTATUS(TT_STAT) X
RESP (TT_RESP) X
TIMEOUT (TT_T_OUT) X
NOHANDLE
*
L R1,EC_C_SEC Load current seconds
S R1,EC_S_SEC Subtract starting seconds
C R1,SM_MAX Max time exceeded?
BRC B'1010',SY_0282 ... yes, set HTTP status 534
*
CLC TT_RESP,DFHRESP(NOTFINISHED) Timeout?
BRC B'1000',SY_0282 ... yes, set HTTP status 534
*
CLC TT_STAT,DFHVALUE(NORMAL) Normal response?
BRC B'0111',ER_50303 ... no, set HTTP status 503
*
LA R1,PP_L Load PP record length
ST R1,PP_FLEN Save PP record length
MVC PP_IDX,TT_IDX Move PP index number
*
EXEC CICS GET X00791223
CONTAINER(C_STAT) X00791223
CHANNEL (TT_CHAN) X00791223
INTO (PP_STAT) X00791223
FLENGTH (PP_FLEN) X00791223
NOHANDLE 00791223
*
OC EIBRESP,EIBRESP Normal response?
BRC B'0111',ER_50304 ... no, set HTTP status 503
*
EXEC CICS GET X00791223
CONTAINER(C_RESULT) X00791223
CHANNEL (TT_CHAN) X00791223
SET (R4) X00791223
FLENGTH (PP_FLEN) X00791223
NOHANDLE 00791223
*
OC EIBRESP,EIBRESP Normal response?
BRC B'0111',SY_0284 ... no, must be 204
*
ST R4,PP_RA_A Save response array address
MVC PP_RA_L,PP_FLEN Save response array length
BRC B'1111',SY_0230 Continue process
No additional items are added to the high-level design with this solution because there is no need to read TS queues or add a polling mechanism. Figure 6-4 illustrates this fact.
Figure 6-4 Check for Completion
6.1.4 Retrieve Data from Child
The parent no longer uses the READQ TS or GETMAIN operations to retrieve data from the child tasks in the new solution. Upon completion of the FETCH CHILD command, the parent task simply issues these commands:
•A GET CONTAINER for the status information.
•Another GET CONTAINER for the result set that the child task provides.
From this point, the actions are the same as the homegrown solution. The parent task uses chunked message transfer through the WEB SEND command to return the result set to the client. It then goes to the next entry in the array and processes the next child task. This activity is repeated until all child tasks have been processed. Relationships for the GET CONTAINERs are added to the high-level diagram in Figure 6-5.
Figure 6-5 Check for Completion
6.1.5 Perform Housekeeping
The CICS asynchronous API brought substantial reductions in complexity to the housekeeping phase. A large part of the homegrown solution is simply removed. The parts that remain are simplified considerably.
In the homegrown solution, the parent task issues DELETEQ TS and FREEMAIN commands to clean up the storage that was used for status responses and result sets from the child tasks. Now that all data that passes between parent and children is performed with containers, only two DELETE CHANNEL commands are required to clean up two types of stored data (Example 6-5):
•Search request information that was passed from parent to child.
•Response information that was passed from child to parent.
Example 6-5 Housekeeping in parent task
*********************************************************************** 01070292
* Issue DELETE CHANNEL for response CONTAINERS. * 01070392
*********************************************************************** 01070292
DC_0010 DS 0H 00973499
ST R14,DC_REG Save return register 01070893
* 00806542
EXEC CICS DELETE X00806642
CHANNEL(TT_CHAN) X00806764
NOHANDLE 00806986
* 00806542
L R14,DC_REG Load return register 01070893
BCR B'1111',R14 Return to caller 01070893
* 00948799
*********************************************************************** 01070292
* Issue DELETE CHANNEL for request CONTAINERS. * 01070392
*********************************************************************** 01070292
DC_0020 DS 0H 00973499
ST R14,DC_REG Save return register 01070893
* 00806542
EXEC CICS DELETE X00806642
CHANNEL(PP_CHAN) X00806764
NOHANDLE 00806986
* 00806542
L R14,DC_REG Load return register 01070893
BCR B'1111',R14 Return to caller 01070893
* 00948799
* 00948799
* 00948799
The process changes dramatically for abnormal end conditions. The homegrown solution employed a background ICE transaction. This transaction ran periodically to issue various INQUIRE commands, apply logic to identify orphaned constructs, and execute DELETEQ TS and FREEMAIN commands. Any orphaned storage that was found was thereby released.
With the CICS asynchronous API and the exclusive use of channels and containers, all storage is managed and implicitly freed by CICS. The concerns about orphaned storage are removed. The background housekeeping task from the homegrown solution is entirely eliminated, thereby reducing code, resource usage, and complexity. Also, the high-level design view remains unchanged from the previous stage.
6.2 Migration
The differences between the homegrown solution and the new solution that uses the CICS asynchronous API have been described in the sections of this chapter. However, the general steps that we took to move to the new solution might not be obvious. So, here's a summary of the changes to the migration process:
•Initiate child tasks with RUN TRANSID, not START TRANSID.
•Add channel and child token information to the internal control array.
•In child tasks, use GET/PUT CONTAINERs, instead of TS queue and GETMAIN SHARED.
•Make the following changes in the parent task:
– Use FETCH CHILD TIMEOUT, instead of the READQ TS and STIMERM-based polling mechanism
– Use GET CONTAINER, instead of GETMAIN SHARED.
– Use DELETE CHANNEL, instead of DELETEQ TS and FREEMAIN for housekeeping.
•Remove the background process for further housekeeping.
This list demonstrates that the changes needed to convert the homegrown solution to using the CICS asynchronous API are relatively minor. Most of the service remains unchanged.
6.3 Summary
Replacing the homegrown asynchronous mechanisms with the CICS asynchronous API required relatively minor changes, but provided significant benefit. The solution was greatly simplified, while maintaining the throughput performance from the original design.
The simplification comes from the removal of numerous items, including these:
•Management of TS queues and shared GETMAINs
•STIMERM-based polling mechanism
•Many housekeeping functions
Hundreds of lines of code were removed from the original solution during the conversion.
|
Flexibility is another benefit with this solution. For example, removal of the STIMERM-based (or other common techniques, like Event Control Blocks or Compare and Swap instructions) polling mechanism eliminates assembly language as a requirement. This change adds options for future maintenance. Also, the success of this change suggests the viability of using higher level languages to develop asynchronous solutions.
|
Additional expected benefits of the CICS asynchronous API include these:
•Quicker and lower-risk development cycles for similar projects. Some of the "heroics" of the engineering team now might not be needed, because the API abstracts away much of the low-level mechanics that used to be required for these types of solutions.
•Reduced burden of ongoing maintenance of code. The API leads to simpler and easier-to-understand code that can be maintained by developers who are less skilled.
•Greater focus on business value. IBM-supported mechanisms replace proprietary code. This change allows teams to direct more attention to processes that directly add value.
The success of using the CICS asynchronous API in this situation led to plans for using it in other solutions. Chapter 7, “Other implementation patterns” on page 49 shows some of these other design patterns where the API is applicable.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.