Server Time Protocol
 
Server Time Protocol (STP) provides synchronized time-of-day (TOD) clocks among several connected systems. IBM Z Program Development Tool (IBM zPDT) GA6 (and later) provides this function for use among multiple zPDT systems. The zPDT implementation is not designed for connection to larger zSystems STP configurations. Figure 18-1 provides an overview of the function.
Figure 18-1 STP overview
An STP implementation is more complex than indicated in Figure 18-1, but it is useful as a starting point in describing the function.
Two terms are important in this description:
Coordinated Timing Network (CTN): Indicates a collection of servers that are time-synchronized. On larger zSystems machines, the coordination can be provided by STP or (on older systems) by a Sysplex Timer or both. For zPDT, only STP is involved.
Server Time Protocol (STP): Describes the protocol operating in and between the servers in the CTN. In a rough sense, STP is an implementation of a CTN.
STP and CTN terminology is sometimes used interchangeably, which is technically incorrect but the meanings are usually clear. Figure 18-1 on page 347 shows three zPDT instances (each in a separate PC) in a CTN. Figure 18-1 on page 347 includes the following high-level elements:
A CTN has stratum levels. Stratum level n provides timer control to level n+1. Stratum 1 is the top layer in a CTN environment and must exist. In practice, a zPDT CTN must have a stratum 1 server and one or more stratum 2 clients. Stratum 3 clients (fed by stratum 2 clients) are possible but are not described here and were not tested.
The stratum 1 server might obtain its TOD from various sources, including the machine BIOS clock, Network Time Protocol (NTP) connections, or an external high-precision clock. These sources can feed or initialize the Linux clocks, and then STP synchronizes with the Linux clocks.
The STP server (in the stratum 1 machine) connects to clients. These client are local clients (in the same Linux as the server) or remote clients (in other machines that are connected by TCP/IP).
STP servers and clients provide log functions. Our implementation (described later) places them in /tmp by default, but this location can be changed.
The STP executable program is named newcct. The same program is used for the server and clients.
An STP client maintains a small file that is named CCTpage, which is in /tmp by default. It contains local timer adjustment data, and it is updated as needed by the STP client.
Information from CCTpage is used to adjust the TOD clock that is used by zPDT. Multiple elements contribute to the TOD clock that is used by the STP server, including these elements:
 – The Linux REALTIME clock (maintained by Linux software)
 – The Linux MONOTONIC clock (maintained by Linux software)
 – The PC BIOS hardware clock, which is referenced when Linux starts
 – The PC time-stamp counter (TSC), if present
 – The libCCT library package and the ReadCCT routine
The interaction of these elements is complex and not described in this document.
Figure 18-1 on page 347 shows a single zPDT instance in each machine. Multiple instances are possible, and each instance requires a separate STP client instance.
The STP functions run at the Linux level. The functions are started before staring zPDT. In practice, the STP functions are typically started when Linux starts and are left running while Linux is running.
The remote clients use TCP/IP connections to the server. In a larger zSystems environment, these connections are typically through Coupling Facility (CF) channels, which are high-speed connections. The accuracy and usability of the zPDT STP implementation might be affected by delays in the IP network.
The zPDT STP implementation cannot be connected to a larger zSystems CTN.
This chapter describes the practical configuration and operation of STP in a zPDT system. For more information about in general, see Server Time Protocol Planning Guide, SG24-7280.
18.1 CCT uses
The coordinated TOD that is provided to multiple z/OS systems can be used for various purposes, such as later comparisons or reconciliation of log files. However, the most common use is for sysplex operation. A z/OS sysplex, either a basic sysplex or an IBM Parallel Sysplex, must have coordinated TOD clocks. For zPDT, Parallel Sysplex operation is provided only when all z/OS systems are guests within a single z/VM environment. In this case, a simulated clock is used, and a CCT is not relevant.
18.2 Configuration
To use the STP function, complete the following steps:
1. Create and customize the /usr/z1090/bin/CCT_data file while working as root. The pattern is in /usr/z1090/bin/CCT_data.MASTER, which contains the following text:
ThisNode = xxx.xxx.xxx.xxx
MasterNode = yyy.yyy.yyy.yyy
CCTid = zPDTbasic
ClientLogEnabled = Y
CCTdir = /tmp
 – ThisNode is the address of your Linux system. It may be expressed as an IP address or as a domain name. For all parameters, a space must exist before and after the equal sign.
 – MasterNode is the address (IP address or domain name) of the Linux system running the STP server. ThisNode and MasterNode are the same value for the Linux system running the server, and it also is a client function.
 – CCTid is the name of your STP network. This parameter should be the same for all your systems. The sample name, zPDTbasic, is an arbitrary but workable name.
 – ClientLogEnabled must be set to Y or N to control client logs. If enabled, a line is added to each client log every minute. These logs might be useful when setting up a basic sysplex complex, but might not be useful in an established operational system.
 – CCTdir indicates where the log files and an internal file (CCTpage) file are placed. A good place is /tmp unless you have specific requirements.
 – The log files (if enabled) have names indicating the starting date and time of that file. Existing log files are overwritten.
 – Place a space before and after the equal sign in this file. The IP addresses are for the base Linux systems, and not for the z/OS systems.
In our example, we copied /usr/z1090/bin/CCT_data.MASTER to /usr/z1090/bin/CCT_data and working as root, configured the following lines for our first Linux server:
ThisNode = 192.168.1.80 (Our second Linux server uses 192.168.1.90.)
MasterNode = 192.168.1.80
CCTid = zPDTbasic
ClientLogEnabled = Y
CCTdir = /tmp
2. Use the stpserverstart command to start the script. The command also automatically adds commands to the Linux cron function so that the server is started automatically when Linux starts or fails. The stpserverstop command can be used to stop the server and remove the automatic cron function. In our example, we start our first Linux server because it contains the STP server. The stpserverquery command may be used to determine the state of your CCT/STP network. After the command is added to the cron function, the STP clients and server are automatically started whenever you start Linux. It is not harmful if you are not using the command for zPDT.
3. Update your devmaps to include an [stp] stanza. After this stanza is included, zPDT cannot be started with this devmap unless the CCT function is operational. You might want to keep a separate copy of your devmaps without the [stp] stanzas for use in a stand-alone mode.
[stp]
ctn 00000000F1F0F9F0 #16 hex digits beginning with 00
node 1 W520 * #asterisks marks this node
node 2 W510
There is one node statement for each Linux PC in your complex. The number (1 or 2) indicates the stratum level of the machine. You should have one stratum 1 server and one or more stratum 2 clients. The names of the nodes (W520 and W510 in the example) are arbitrary, but z/OS messages might be more meaningful if these names are the names of the Linux systems, as expressed in /etc/HOSTNAME.
The asterisk (*) denotes the current system that is processing this devmap. In the example, our second system has the asterisk after the W510 name. Except for the location of the asterisk, all the zPDTs in your complex have the same [stp] stanza in their devmap.
4. The z/OS CLOCKxx members that you use (in PARMLIB) must be altered to use the STP function:
(Member CLOCKB1 in USER.PARMLIB in our test system)
OPERATOR NOPROMPT
TIMEZONE W.05.00.00
ETRMODE NO
ETRZONE NO
ETRDELTA 10
STPMODE YES
STPZONE NO
STP operation in z/OS can be verified with the MVS command D ETR:
D ETR
.....
SYNCHRONIZATION MODE = STP
.....
You cannot use a devmap (as the operand of an awsstart command) that has an [stp] stanza unless the CCT/STP function is running. You can check the status by using the stpserverquery command.
If you display Linux processes directly with, for example, a ps aux | grep cct command, you see a line that ends with newcct information, as in the following example:
newcct 192.168.1.80 -L -e1000 -E300000 -f -I zPDTbasic -n0 -r120
newcct -v1
The CCT/STP function does not affect the Linux TOD clock. It affects only clocks that use the CCT interface library routines; in practice, the zPDT (when a [stp] stanza is present in the devmap) and z/OS (when the CLOCKxx parameter is set).
18.3 More details
The z/Architecture places tight tolerances on STP clock synchronization and uses high-speed coupling channels as part of the design to achieve the specified tolerances. zPDT, by using IP communication, cannot duplicate this environment, so the z/Architecture tolerances are relaxed for the zPDT STP function. You must evaluate the usefulness and correctness of the zPDT STP function for your environment.
Both the client and server create log files. Our sample configuration places them in /tmp. Existing log files are overwritten when a server or client is started. The server log files typically have little in them. A client log file might have entries such as the following ones:
Wed Oct 8 11:14:54 EDT 2014 (<-- this is when this log file started)
%CCT (TAI seconds) Offset (us) Dispersion Skew (ppm) Steering RTT (us)
1412781391.767832 -0.088 0.009 0.00 0.00 0.228
1412781451.774840 -0.102 0.006 0.00 0.00 0.183
1412781511.782228 -0.103 0.006 0.00 0.00 0.181
1412781571.789614 -0.097 0.026 -0.00 0.00 0.212
With the default parameters, a client log line is written every minute. As a general statement, you do not need any information from the logs except an indication of how often the client might have failed. (The default parameters cause the client to restart itself after a failure.) The log files are named STPServerLog and STPClientLog and placed in the Linux directory that you specify in the STP configuration file. These two files are open whenever the STP function is active. If you want to capture the contents of a log file for later inspection (they are overwritten whenever the STP function is started), use the Linux cp command, as in this example (the cp command may be used against an active Linux file):
$ cp /tmp/STPClientLog /tmp/STPClint.monday3PM
The fields in the client log are as follows:
International Atomic Time (TAI) Number of seconds since 1 January 1972, including leap seconds. (The leap second data is not normally available to the zPDT newcct program, and this definition might not be completely accurate.)
Offset The difference (in microseconds) between the master CCT clock (the stratum 1 server) and the client clock, as calculated by the client.
Dispersion The uncertainty in the calculated offset value.
Skew The difference in the clock stepping rate between the master and the subordinate.
Steering The rate at which the client clock is being steered to reduce the offset toward zero.
RTT The minimum measured round-trip delay for a set of CCT sample “ping-pong messages”. It is the sum of the minimum forward delay time and minimum backward delay time for a set of CCT sample ping-pong messages,
 
Note: “Ping-pong” messages are short message exchanges between a client and server.
The TOD that is provided to z/OS by the CCT/STP function ultimately depends on the TOD that is returned by the effective Linux clock.1 A user might have a substantially inaccurate TOD in the PC BIOS clock that is used to set the Linux clocks when Linux starts. Another user might have an NTP2 connection for Linux, resulting in a Linux clock that is accurate to within a second or so. At the zPDT level, there is no control over the accuracy of the time data that is provided by Linux. The STP function keeps multiple zPDT TOD values synchronized, but does not control the accuracy of the time that is presented.
The log files are not intended for normal user inspection and may contain messages with whimsical phrasing, such as this example:
Incomplete PingPong exchange on client: Resource temporarily unavailable
This message indicates a client failure, but the client immediately restarts itself. We have seen these messages at random intervals on zPDT CCT networks that are closely coupled (private 1 GB Ethernet), but we have not seen them on other systems that are connected through the internet. In our test environment, these transient failures appear to have no effect on the z/OS basic sysplex operation. Corresponding log messages on the CCT server include the following ones:
Failed to read doorbell msg
Welcome (Denotes a client restart.)
When a remote client connects to the STP server, another server instance is automatically started. No action is needed to manage this task, but you might see multiple STP servers when using the Linux ps command.
If you have an [stp] stanza in a devmap, you cannot start this devmap (by using the awsstart command) unless you have an STP function active. If you do not have STP active, the zPDT startup fails. The most common error message is INVALID REGISTER NUMBER.
The zPDT STP facility relies on a fast, low-jitter TCP/IP connection between host Linux systems to synchronize the zPDT TOD clock values. Insufficient TCP/IP connection quality results in STP reporting either unsynchronized or unusable TOD clock-source machine checks to the zPDT-hosted OS. You must provide a physical TCP/IP connection with sufficient quality to support machine-check-free STP operation. In a virtualized environment, the hypervisor increases the TCP/IP latency and jitter, as perceived by the STP facility, which increases the risk of disruptive zPDT STP machine checks. We have performed limited STP testing in a virtualized environment with mixed results, so as a best practice, do not use STP in a virtualized environment.
18.3.1 Leap seconds
 
Tip: There is no need to deal with leap second details unless you (or your applications) are sensitive to them. We expect 99% of zPDT users will ignore leap seconds.
You may update the leap second information block by including an extra statement in the devmap, as in the following example:
[stp]
ctn 00000000F1F0F9F0 #16 hex digits beginning with 00
node 1 W520 * #asterisks marks this node
node 2 W510
LEAPSECONDS 25 26 2015 6 30 23 59 59
The format for the LEAPSECONDS statement is as follows:
LEAPSECONDS <active> <new> <year> <month> <day> <hour> <minute> <second>
<active> is the number of leap seconds present.
<new> is the replacement number of leap seconds. It is not an extra value, but a complete replacement of the active number. It potentially can be a smaller number than the <active> value, although this situation has not happened yet. The difference between <active> and <new> should never be more than 1.
<year><month><day><hour><minute><second> is the time at which the <new> value is effective.
Two new zPDT commands (issued in a Linux command-line interface (CLI)) are provided for leap second details:
$ d lso (displays the leap second information block)
$ st lso <active><new><year><month><day><hour><minute><second>
$ st lso 25 26 2015 6 30 23 59 59 (an example)
We understand that the z/OS CLOCKxx parameter STPZONE must be set to YES for z/OS to process leap seconds when using an STP time source. The leap second information block is displayed, as in the following example:
LEAP SECOND OFFSET INFORMATION BLOCK
word(0): 80000000
lso active: 25
lso new: 26
update time: 73061E173B3B00 - 2015: 6:30:23:59:59
 

1 This topic is complex because Linux has several clocks. We ignore this detail here.
2 NTP references an accurate TOD service that is available over the internet.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.141.0.61