Integrating IBM zAware with other IBM products
This chapter discusses the options for integrating IBM zAware with other IBM products. To obtain the maximum value from IBM zAware, it is important to integrate it into your systems management environment. IBM zAware provides an application programming interface (API) that you can use to provide that integration. This chapter provides an example of the use of that API.
6.1 Introduction
The IBM zAware application provides a graphical user interface that is accessed using a web browser. This is useful for enabling you to periodically look at and review its results. However, you also need a way to monitor IBM zAware without having to constantly watch yet another console. Therefore, you will probably want to integrate IBM zAware with your other systems management products.
By linking the information from IBM zAware with other products, the data can be made more useful and queried close to real time, for analysis with your automation programs.
This chapter describes two types of integration that currently are possible with the IBM zAware application. The first type of integration is a direct URL connection from the z/OSMF user interface to IBM zAware. This lets you quickly open the IBM zAware graphical interface while you are reviewing the incident logs in z/OSMF, for example.
The second type of integration is with IBM Tivoli NetView for z/OS. NetView provides samples that can be used to interface between user-written programs and the IBM zAware API. This integration gives you the ability to programmatically retrieve the same information that is presented in the Analysis view in the IBM zAware graphical interface.
The information retrieved into NetView can be used in various ways. You can use the information to verify that the IBM zAware application is still recording information from your system. You can also pass the information to your automation to have it raise an alert if the anomaly score for an interval exceeds a certain threshold. And you can use the CANZLOG consolidated log feature of NetView to perform problem determination to determine the cause of the anomalies that IBM zAware highlighted.
It is expected that eventually other IBM products will be extended to integrate information from IBM zAware.
6.2 z/OSMF
In z/OSMF, you can link to other web pages from the z/OSMF navigation tree. This allows you to easily launch IBM zAware from within z/OSMF, and keeps your desktop organized so that problem determination tools are grouped together.
Figure 6-1 shows the z/OSMF home window after you have logged onto z/OSMF using a z/OSMF administrator user ID.
Figure 6-1 z/OSMF home window
1. To add the IBM zAware GUI as a link to z/OSMF, open z/OSMF Administration in the navigation tree and select Links as shown in Figure 6-1 on page 193.
Figure 6-2 Select New from z/OSMF Links
2. When Links opens, the list of existing links appears. Click Actions → New as shown in Figure 6-2 on page 194.
Figure 6-3 Create new link for IBM zAware
3. Enter the required data as shown in Figure 6-3:
a. Enter a name, the SAF resource name suffix, and the URL for the IBM zAware GUI.
b. Select the category based on where you want IBM zAware to be located in your z/OSMF navigation tree.
c. Select your choice as to where the window opens.
d. Select the authority required to use the link.
e. Click OK.
In our example, we choose to have IBM zAware appear in the Problem Determination category.
Figure 6-4 IBM zAware links added to z/OSMF
Figure 6-4 shows the results, with the links to the IBM zAware partitions contained in the Problem Determination category of z/OSMF.
Use these links whenever you want to access IBM zAware while signed in to z/OSMF.
6.3 IBM Tivoli NetView for z/OS
In relation to IBM zAware, NetView plays two roles. One role is that it provides an interface between a program running on z/OS and the IBM zAware API. The other role is that it provides an infrastructure (perhaps in tandem with your automation product) to do something with the information that is returned. First we discuss the interface role.
The Analysis results contained in the IBM zAware file system can be retrieved using an API provided with IBM zAware (the API is discussed in more detail in 6.4, “IBM zAware Application Programming Interface” on page 205). The API can deliver two levels of information back to your programs:
•A detailed list of message IDs and their attributes for a specific interval
•A summary of the anomaly score and message ID count for each analysis interval
|
Message ID count: When retrieving the summary information for a system, be aware that the message ID count is a count of the unique message IDs that occurred in that period. It is not a count of the total number of messages that were issued in the period.
|
The data returned from the API is formatted as XML records.
NetView provides a set of sample programs to make it easier for your programs to communicate with the IBM zAware API. The sample programs, together with information about their installation and use, can be downloaded from the NetView Tivoli System z Monitoring and Application Management wiki, available on the web at:
The sample programs and their function are listed and briefly described in Table 6-1.
Table 6-1 Sample programs to retrieve IBM zAware XML data
|
Program
|
Description
|
|
ZAITIMER
|
The ZAITIMER program is designed to be called from a timer every 10 minutes. It will build the parameters for the ZAIPROC exec to retrieve the details of the previous 10-minute interval and then call ZAIPROC with those parameters. It retrieves the results from the previous 10-minute interval because those will be complete. The current interval results are still being calculated and might change.
Detailed information about how to use the ZAITIMER program is provided in “Using ZAITIMER” on page 199.
|
|
ZAIPROC
|
The ZAIPROC program accepts parameters that control the type of information to be returned. ZAIPROC calls the ZAIGET program to retrieve information from the IBM zAware API. The output will be in single-line and multi-line messages, each of which is described in the comments of the program.
Detailed information about how to use the ZAIPROC program is provided in “Using ZAIPROC” on page 200.
|
|
ZAIGET
|
The ZAIGET program performs the HTTPS connection to the IBM zAware application, and performs the API actions to retrieve the data. When the data is returned, the output is placed in a NetView safe1 named ZAIGET, which the ZAIPROC program uses.
Information about the use of ZAIGET is included in the download package.
|
1 For more information about safes refer to the section about “PIPE SAFE” in the NetView “Programming: Pipes” manual, available online at: http://pic.dhe.ibm.com/infocenter/tivihelp/v3r1/index.jsp?topic=%2Fcom.ibm.itnetviewforzos.doc%2Fdqsl2mst192.htm
The ZAIPROC program creates single-line and multi-line messages that are exposed to the NetView Automation table, and that will also be available for browsing in the NetView CANZLOG. The messages contain the information retrieved from the IBM zAware application. You can use your automation table to process these messages. The various NetView functions are shown in Figure 6-5 on page 198.
Figure 6-5 Retrieving data from IBM zAware API
6.3.1 Using the ZAI sample programs
The concept of the ZAITIMER and ZAIPROC sample programs is that ZAITIMER will be called on a regular basis (we suggest using a 10-minute interval to be in synch with IBM zAware). ZAITIMER calls ZAIPROC, passing it parameters to identify the interval that is to be retrieved. ZAIPROC then uses the ZAIGET sample program to send the request to the IBM zAware API.
The ZAIPROC program can provide results in one of the following two forms, depending on the setting of the REQUEST parameter when ZAIPROC is called:
•If REQUEST(INTERVAL) is specified, a detailed list of message IDs with an anomaly score greater than zero (0) that occurred in a particular interval is returned.
– The default anomaly scores that will be returned are those above 96.5. You can override this using the ASCORE parameter on the ZAITIMER and ZAIPROC programs.
– If there was data returned for an interval, but the anomaly score was less than the ASCORE value (default 96.51) the results will not display. Message ZAI0004I will be issued instead, stating that the result was filtered due to a low anomaly score. Setting the ASCORE value to zero (0) will cause all the records for that interval to be displayed.
•If REQUEST(LPAR) is specified, a summary list of anomaly scores and message ID counts for each interval in a day is returned.
When ZAIPROC is passed the parameters for a specific time interval, the results are returned in a multi-line message for automation in the NetView message automation table.
Example 6-1 shows sample output from ZAIPROC when the parameters specify to retrieve the previous 10-minute period from the IBM zAware application. The ZAI0001I message is used to return the requested information for any INTERVAL requests. The message includes the sysplex and system that the information relates to, and the time period that was retrieved. The times are always shown in UTC time. The message then shows the interval anomaly score for that period and each distinct message ID and its anomaly score. You can use the ZAI0001I message to drive your automation.
Example 6-1 ZAI0001I INTERVAL results message
ZAI0001I Interval Results.
System : UTCPLXSB-SP0
Interval: 2012-08-04T12:40:00.000Z
Anomaly : 57.4
-------------------------------------------------------------------------------
Anomaly Message Count Cluster Contribution Rarity
1.000000 CSQX520E 2 NEW 2.226 101.0
0.940000 CSQX470E 4 UNCLUSTERED 2.822 90.0
0.871000 IXG284I 1 UNCLUSTERED 2.051 20.0
0.865000 IEF285I 1 UNCLUSTERED 2.012 2.0
0.839000 IEC070I 30 UNCLUSTERED 1.832 20.0
0.801000 DFS2864I 8 UNCLUSTERED 1.618 5.0
0.670000 IGD104I 1 UNCLUSTERED 1.11 18.0
0.645000 IEF237I 2 UNCLUSTERED 1.038 2.0
0.618000 DFS994I 2 UNCLUSTERED 0.964 21.0
0.202000 CSQX004I 1 UNCLUSTERED 0.226 26.0
Alternatively, when ZAIPROC is called with REQUST(LPAR), the information is returned in message ZAI0005I. The message lists the sysplex and system and the intervals for the date given to the program.
If the date given is the same as the current UTC date, the ZAIPROC program will filter the results to return the last six intervals, starting from the current UTC time and going back six intervals. If the date is not the same as the current UTC date, ZAIPROC will not filter the result and every interval score for that date, 144 entries, will be displayed.
Example 6-2 shows an LPAR request for the current UTC date. This returns the anomaly score and message ID count for the last six intervals.
Example 6-2 ZAI0005I LPAR results message
ZAI0005I Interval Results.
System : UTCPLXSB-SP0
Interval: 2012-08-04T00:00:00.000Z
-------------------------------------------------------------------------------
Anomaly MessageIDs
37.70000 21 Oldest interval
51.00000 39
14.20000 20
57.40000 43
57.40000 62
51.00000 30 Newest interval
Using ZAITIMER
The main function of the ZAITIMER program is to calculate the previous time period using UTC time, and supply that and other derived parameters to the ZAIPROC program. It uses the previous interval rather than the current one because the results are complete, and therefore have a final anomaly score for that interval.
You typically code the call to ZAITIMER on a timer in NetView2 and run it every 10 minutes. It does not need any parameters. However, you can specify any of the parameters listed in Table 6-2. These parameters are then passed to the ZAIPROC program, which is called from the ZAITIMER program.
Table 6-2 ZAITIMER parameters
|
Parameter
|
Value
|
|
PLEX(......)
|
This contains the sysplex name to retrieve information about.
|
|
SYS(....)
|
This contains the system name to retrieve information about.
|
|
SERVER(....)
|
This is the IP address of the IBM zAware server.
|
|
ASCORE(....)
|
This is the interval anomaly score filter value. This will change the default filter, so you can retrieve intervals that have anomaly scores lower than the default but higher than the value specified here. If you specify a value of zero (0), the results for the interval will be displayed regardless of the interval anomaly score.
|
The commands shown in Example 6-3 include both methods of executing ZATIMER. Firstly without parameters, meaning that the program will determine the PLEX and SYS parameters itself. And secondly with parameters, where the parameters are supplied for the sysplex, system, and anomaly score parameters.
Example 6-3 ZATIMER execution examples
ZAITIMER
ZAITIMER PLEX(UTCPLXSB) SYS(SP0) ASCORE(50)
Using ZAIPROC
The ZAIPROC program has two main purposes:
•To retrieve the requested information from IBM zAware using the sample ZAIGET program
•To parse the XML data that is retrieved from IBM zAware and send the results to the NetView netlog for viewing and processing through the NetView automation table
ZAIPROC issues ZAInnnna messages to present the information received from the API.
ZAIPROC can be called from other programs, or directly from the command line.
ZAIPROC has certain default parameters that are required for it to execute successfully. These are retrieved from NetView common global variables. The common global variables can be set in the CNMSTYLE processing3 when NetView starts, or dynamically with the NetView RESTYLE COMMON command.
The required global variables are documented in Table 6-3 and in the program. The common global values can be overridden by supplying them as parameters to the program when it is executed.
Table 6-3 ZAIPROC global variables in CNMSTYLE
|
Common global variable name
|
Value description
|
|
zaiproc.zAwareServer
|
[Mandatory] The IP address or host name of the IBM zAware application.
|
|
zaiproc.zAwareUserid
|
[Mandatory] The user ID that will logon to the IBM zAware application to execute the API.
•It can be overridden by a parameter passed to the program.
•The password for the user ID will be retrieved from the NetView password file using GETPW, or you can specify the password as a parameter on the program call.
•The user ID need only have USER authority in the IBM zAware application.
|
|
zaiproc.zAwareAscore
|
[Optional] The value of the Anomaly score for which results that are greater than this value will be returned. If the value is zero (0), then all results are returned.
|
The ZAIPROC program has a number of parameters that are used to build the API request used by ZAIGET. Each parameter is explained in Table 6-4.
|
Important: When calling ZAIPROC, the command needs to be prefixed with the NETVASIS keyword. This is because the password character case needs to be retained.
|
Table 6-4 ZAIPROC parameters
|
Parameter
|
Value description
|
|
REQUEST(...)
|
[Mandatory] INTERVAL or LPAR
|
|
DATE(...)
|
[Mandatory when using REQUEST(LPAR)]
Format UTC: mmddyyyy
|
|
TIME(...)
|
[Mandatory when using REQUEST(INTERVAL)
Format UTC: yyyymmddhhmm00
|
|
PLEX(...)
|
[Mandatory] Sysplex name
|
|
SYS(...)
|
[Mandatory] System name
|
|
SERVER(...)
|
[Optional] IP address or host name of the IBM zAware application.
|
|
ASCORE(...)
|
[Optional] Anomaly score filter applied to data returned on the REQUEST(INTERVAL) calls.
•Any anomaly score for an interval that is below this value will not be displayed.
•Setting the value to zero (0) will return all results with no filtering.
|
|
USER(...)
|
[Optional] The user ID that will logon to the IBM zAware application to retrieve the XML data.
|
|
PASS(....)
|
[Optional] The password for the user ID. Case sensitive.
•When not specified, the password for the user ID will be retrieved from the NetView password file using GETPW.
•When specified, use the NETVASIS command before ZAIPROC.
|
Next we show a few examples of calling the ZAIPROC program with different parameter combinations.
First, Example 6-4 uses a password on the parameters. Because the password is case sensitive, the NETVASIS command has been added to the front of the ZAIPROC command. This makes NetView keep the case the command was entered in. Otherwise, NetView will convert the command and parameters to uppercase.
Example 6-4 Using ZAIPROC to retrieve a specific interval result from IBM zAware
netvasis zaiproc request(interval) time(20120804090000) plex(utcplxsb) sys(sp0)
pass(rabbit)
ZAI0001I Interval Results.
System : UTCPLXSB-SP0
Interval: 2012-08-04T09:00:00.000Z
Anomaly : 37.7
-------------------------------------------------------------------------------
Anomaly Message Count Cluster Contribution Rarity
1.000000 DSNB260I 1 NEW 0.226 101.0
0.940000 CSQX470E 4 UNCLUSTERED 2.822 90.0
0.871000 IXG284I 1 UNCLUSTERED 2.051 20.0
0.865000 IEF285I 1 UNCLUSTERED 2.012 2.0
0.839000 IEC070I 30 UNCLUSTERED 1.832 20.0
0.819000 DFS2864I 6 UNCLUSTERED 1.714 5.0
0.670000 IGD104I 1 UNCLUSTERED 1.11 18.0
0.645000 IEF237I 2 UNCLUSTERED 1.038 2.0
Here, Example 6-5 shows a request for the LPAR information for the current day. This means that the current UTC time of the system is retrieved, the number of 10-minute periods since the start of the day is calculated, then the Anomaly scores for the last six intervals are returned.
Example 6-5 Using ZAIPROC to retrieve LPAR summary of last six intervals from IBM zAware
zaiproc request(lpar) date(08042012) plex(utcplxsb) sys(sp0)
ZAI0005I Interval Results.
System : UTCPLXSB-SP0
Interval: 2012-08-04T00:00:00.000Z
-------------------------------------------------------------------------------
Anomaly MessageIDs
20.10000 18 Oldest interval
51.00000 39
37.70000 27
73.00000 75
31.50000 22
31.50000 22 Newest interval
Define NetView password file
To store a password in NetView for the user ID that logs onto the IBM zAware application, you need to use the NetView (AON) password data set.
The sample JCL for defining this data set can be found in the NetView CNMSAMP data set, member CNMSJ004. The member with the IDCAMS statements to define the password data set is EZLSI101.
After you have defined the data set, you can allocate it to the NetView CNMSJ009 procedure using DD EZLPSWD.
To define the password for the user ID that logs onto the IBM zAware application, you use the GETPW command. You also need to use the NETVASIS command in front of GETPW, to ensure the case of the password is retained. When you use NETVASIS, make sure the parameter keywords are in uppercase, because GETPW expects them to be that way.
Imagine you have a user ID of A1234B defined in the IBM zAware application, with a password of 12@aBcE!, which is mixed case. And the NetView domain that executes the ZAIPROC program, which uses GETPW to retrieve the password, is CNM01.
Example 6-6 Defining a password with GETPW
netvasis getpw a1234b cnm01,INIT=12@aBcE!
Retrieving XML data from IBM zAware with ZAIGET
|
AT-TLS function of TCP/IP must be activated: Because the IBM zAware API uses an HTTPS connection, the AT-TLS function of TCP/IP needs to be activated on z/OS. Refer to Appendix B, “Activating TCP/IP AT-TLS” on page 213 for a sample configuration.
|
The ZAIGET program performs all the TCP/IP communications with the IBM zAware application. It accepts parameters passed on the command line.
The ZAIGET program needs to be called from another program when you want to view the data retrieved by it. The data is returned into a SAFE, and the SAFE only lives as long as the REXX execution environment that called ZAIGET exists.
In this book, we use ZAIPROC as the front-end to ZAIGET. It is ZAIPROC that does the parsing of the data returned from the API calls that ZAIGET performs.
ZAIGET expects a task global variable, ZAWARE.URL, to be set with the HTTP details of the API request, and expects the user ID and password to be passed as parameters to the program. The details of its parameters are contained in the program comments.
ZAIGET performs the following functions:
1. It connects to the IBM zAware application and logs on the user ID.
2. It send the API parameters to the server.
3. It captures the returned data.
4. It outputs the data into a named SAFE, called ZAIGET.
6.3.2 Using the information retrieved from the IBM zAware API
A typical z/OS system has many monitors including MVS, IBM CICS, IBM DB2, and MQ performance monitors. It also has health check monitors, network performance monitors, network management consoles, workload scheduler consoles, automation consoles, and possibly many others. The last thing you need is yet another console to have to monitor.
Alternatively, the information that is provided by IBM zAware might be critical in helping you avoid an outage. How do you balance the need to limit the number of consoles that operators must monitor with the need to be aware of anomalies that IBM zAware might detect?
By using the sample programs documented in this book, you can integrate the information from IBM zAware into your operations management environment. The ZAInnnna messages can be processed by your automation product, and alerts can be raised based on thresholds that you specify.
For example, you might set up ZAITIMER to use a relatively low interval Anomaly score so that information about most anomalous messages in the interval is returned to NetView. This means that the information is available in CANZLOG for quick viewing if you need it.
Alternatively, you can set up your automation to only raise an alert if the interval Anomaly score is greater than a higher value such as 96.5, for example. This way, you have all the information you are likely to want in CANZLOG, but you do not have automation constantly raising alerts that operators will soon learn to ignore.
Another important use of NetView and the IBM zAware API is to regularly check the health of the IBM zAware LPAR. You probably will not have someone constantly monitoring the IBM zAware GUI, but you want to be aware if some problem causes the IBM zAware application or one of its components to stop working. You will not want to turn to IBM zAware to help debug a potential problem situation, only to find that it has not been collecting data for the last three hours.
You can monitor the health of the IBM zAware application by using the REQUEST(LPAR) flavor of the ZAIPROC program to retrieve information for the whole day. In this case you will probably be more interested in the fact that you actually received a response, than in the information returned. Because the ZAITIMER exec is designed to call the REQUEST(INTERVAL) flavor of ZAIPROC, you need to provide some automation to build the appropriate parameters to call ZAIPROC, but that should be a minor effort.
As part of monitoring the health of the IBM zAware environment, you will check that the connections from System Logger to IBM zAware are active. However, that checking uses System Logger commands rather than the IBM zAware API. That monitoring is discussed in Chapter 5, “Maintaining and managing IBM zAware” on page 153.
If you have IBM Tivoli System Automation for z/OS, you can then perhaps create a logical resource to track the status of the IBM zAware application. For example, you might track whether it is running or not, or whether the connection from OPERLOG to IBM zAware is active. There are many ways the information can be transformed and used to extend the management of IBM zAware.
6.4 IBM zAware Application Programming Interface
The IBM zAware application provides an API that enables you to retrieve information about the Analysis view results. There are two types of results available, INTERVAL and LPAR, as described in Table 6-5. The details of the API are documented in IBM System z Advanced Workload Analysis Reporter (IBM zAware) Guide, SC27-2623.
Table 6-5 API request types
|
Request type
|
Description
|
|
INTERVAL
|
Return a 10-minute analysis interval result as an XML package.
|
|
LPAR
|
Return all the summary results for every interval for a full day, all 144, 10-minute periods in an XML package.
|
The IBM zAware API requires the use of Secure Sockets (HTTPS). Therefore, as previously mentioned, if you are going to use the API from z/OS you will need to activate the TCP/IP AT-TLS function. Details about AT-TLS and how we activated it in the ITSO systems are contained in Appendix B, “Activating TCP/IP AT-TLS” on page 213.
1 When you analyze the information returned from IBM zAware, the anomaly score is used to determine how important the results of a particular interval are. The important anomaly scores are generally those above 96.5. They are important because they have the greatest deviation from the model of past behavior.
2 Refer to “Timer Commands” in the “NetView Program Automation Facilities” chapter of the NetView “Automation Guide”, online at: http://pic.dhe.ibm.com/infocenter/tivihelp/v3r1/index.jsp?topic=%2Fcom.ibm.itnetviewforzos.doc%2Fdqal2mst203.htm
3 Refer to “Updating the CNMSTYLE Member” in the “Getting Ready to Start NetView” chapter of the NetView “Installation: Getting Started” manual, available online at: http://pic.dhe.ibm.com/infocenter/tivihelp/v3r1/index.jsp?topic=%2Fcom.ibm.itnetviewforzos.doc%2Finql0mst102.htm&path%3D65_1_8_4
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.