Server Time Protocol (STP)
 
Server Time Protocol (STP) provides synchronized time-of-day (TOD) clocks among several connected systems. zPDT GA6 (and later) provides this function for use among multiple zPDT systems. The zPDT implementation is not designed for connection to larger System z STP configurations. Figure 20-1 provides an overview of the function.
Figure 20-1 STP Overview
STP implementation is considerably more complex than indicated in the figure, but the figure is useful as a starting point in describing the function.
Two terms are important in this description:
Coordinated Timing Network (CTN): This indicates a collection of servers that are time synchronized. On larger z System machines, the coordination can be provided by STP or (on older systems) by a Sysplex Timer or both. For zPDT, only STP is involved.
Server Time Protocol (STP): This describes the protocol operating in and between the servers in the CTN. In a rough sense, STP is an implementation of a CTN.
STP and CTN terminology is sometimes mixed; this is technically incorrect but the meanings are usually clear. The figure shows three zPDT instances (each in a separate PC) in a CTN. The figure includes the following high-level elements:
A CTN has stratum levels. Stratum level n provides timer control to level n+1. Stratum 1 is the top layer in an CTN environment and must exist. In practice, a zPDT CTN must have a stratum 1 server and one or more stratum 2 clients. Stratum 3 clients (fed by stratum 2 clients) are possible but are not described here and have not been tested.
The stratum 1 server might ultimately obtain its time-of-day from a variety of sources, including the machine BIOS clock, Network Time Protocol (NTP) connections, or an external high-precision clock. These sources can feed or initialize the Linux clocks and STP then works from the Linux clocks.
The STP server (in the stratum 1 machine) connects to clients. These are local clients (in the same Linux as the server) or remote clients (in other machines, connected by TCP/IP).
STP servers and clients provide log functions. Our implementation (described later) places these in /tmp by default, but this is configurable.
The STP executable program is named newcct. The same program is used for the server and clients.
An STP client maintains a small file named CCTpage, which is in /tmp by default. This contains local timer adjustment data, and is updated as needed by the STP client.
Information from CCTpage is used to adjust the time-of-day (TOD) clock that is used by zPDT. Multiple elements contribute to the TOD clock used by the STP server, including these elements:
 – The Linux REALTIME clock (maintained by Linux software)
 – The Linux MONOTONIC clock (maintained by Linux software)
 – The PC BIOS hardware clock, which is referenced when Linux is booted
 – The PC time-stamp counter (TSC), if present
 – The libCCT library package and the ReadCCT routine
The interaction of these elements is complex and is not described in this document.
The figure shows a single zPDT instance in each machine. Multiple instances are possible and each instance requires a separate STP client instance.
The STP functions run at the Linux level. They are started before staring zPDT. In practice, the STP functions are typically started when Linux is booted and are left running while Linux is running.
The remote clients use TCP/IP connections to the server. In a larger z System environment, these connections are typically through coupling facility channels, which are very high-speed connections. The accuracy and usability of the zPDT STP implementation might be affected by delays in the IP network.
The zPDT STP implementation cannot be connected to a larger z System CTN.
This chapter describes the practical configuration and operation of STP in a zPDT system. Much more general STP detail is available in Server Time Protocol Planning Guide, SG24-7280.
20.1 CCT uses
The coordinated time of day provided to multiple z/OS systems can be used for a variety of purposes, such as later comparisons or reconciliation of log files. However, the most common use is for sysplex operation. A z/OS sysplex, either a basic sysplex or a parallel sysplex, must have coordinated time-of-day clocks. For zPDT, parallel sysplex operation is provided only when all z/OS systems are guests within a single z/VM environment. In this case, a simulated clock is used and a CCT is not relevant.
20.2 Configuration
To use the STP function, complete these steps:
1. Create and customize the /usr/z1090/bin/CCT_data file, working as root. The pattern is given in /usr/z1090/bin/CCT_data.MASTER, which contains the following text:
ThisNode = xxx.xxx.xxx.xxx
MasterNode = yyy.yyy.yyy.yyy
CCTid = zPDTbasic
ClientLogEnabled = Y
CCTdir = /tmp
 – ThisNode is the address of your Linux system. It may be expressed as an IP address or as a domain name. For all parameters, a space must exist before and after the equal sign.
 – MasterNode is the address (IP address or domain name) of the Linux system running the STP server. ThisNode and MasterNode are the same value for the Linux system running the server and also a client function.
 – CCTid is the name of your STP network. This parameter should be the same for all your systems. The sample name, zPDTbasic, is arbitrary but is a workable name.
 – ClientLogEnabled must be set to Y or N to control client logs. If enabled, a line is added to each client log every minute. These logs might be useful when setting up a basic sysplex complex, but might not be useful in an established operational system.
 – CCTdir indicates where the log files and an internal file (CCTpage) file are to be placed. A good place is /tmp unless you have specific requirements.
 – The log files (if enabled) have names indicating the starting date and time of that file. Existing log files are overwritten.
 – Remember to place a space before and after the equal sign in this file. Remember that the IP addresses are for the base Linux systems, not for the z/OS systems.
We copied /usr/z1090/bin/CCT_data.MASTER to /usr/z1090/bin/CCT_data and, working as root, configured the following lines for our first Linux:
ThisNode = 192.168.1.80 (Our second Linux uses 192.168.1.90)
MasterNode = 192.168.1.80
CCTid = zPDTbasic
ClientLogEnabled = Y
CCTdir = /tmp
2. Use the stpserverstart command to start the script; it also automatically adds commands to the Linux cron function so that the server is started automatically when Linux is booted or fails. The stpserverstop command can later be used to stop the server and remove the automatic cron function. We then start our first Linux, because it contains the STP server. The stpserverquery command may be used to determine the state of your CCT/STP network. After it is added to the cron function, the STP server/clients are automatically started whenever you boot Linux. They do no harm if you are not using them for zPDT.
3. Update your devmaps to include an [stp] stanza. After this stanza is included, zPDT cannot be started with this devmap unless the CCT function is operational. You might want to keep a separate copy of your devmaps without the [stp] stanzas, for use in a stand-alone mode.
[stp]
ctn 00000000F1F0F9F0 #16 hex digits beginning with 00
node 1 W520 * #asterisks marks this node
node 2 W510
There is one node statement for each Linux PC in your complex. The number (1 or 2) indicates the stratum level of the machine. You should have one stratum 1 server and one or more stratum 2 clients. The names of the nodes (W520 and W510 in the example) are arbitrary, but z/OS messages might be more meaningful if these are the names of the Linux systems, as expressed in /etc/HOSTNAME.
The asterisk (*) denotes the current system which is processing this devmap. In the example, our second system would have the asterisk after the W510 name. Except for the location of the asterisk, all the zPDTs in your complex have the same [stp] stanza in their devmap.
4. The z/OS CLOCKxx members you use (in PARMLIB) must be altered to use the STP function.
(Member CLOCKB1 in USER.PARMLIB in our test system)
OPERATOR NOPROMPT
TIMEZONE W.05.00.00
ETRMODE NO
ETRZONE NO
ETRDELTA 10
STPMODE YES
STPZONE NO
STP operation in z/OS can be verified with the MVS command D ETR:
D ETR
.....
SYNCHRONIZATION MODE = STP
.....
You cannot use a devmap (as the operand of an awsstart command) that has an [stp] stanza unless the CCT/STP function is running. You can check the status by using the stpserverquery command.
If you display Linux processes directly, with a ps aux | grep cct command for example, you see a line that ends with newcct information, as in the following example:
newcct 192.168.1.80 -L -e1000 -E300000 -f -I zPDTbasic -n0 -r120
newcct -v1
The CCT/STP function does not affect the Linux time-of-day clock. It affects only clocks that use the CCT interface library routines; in practice, this means zPDT (when a [stp] stanza is present in the devmap) and z/OS (when the CLOCKxx parameter is set).
20.3 Additional details
The z/Architecture places tight tolerances on STP clock synchronization and uses the high speed coupling channels as part of the design to achieve the specified tolerances. zPDT, using IP communication, cannot duplicate this environment and the z/Architecture tolerances are relaxed for the zPDT STP function. You must evaluate the usefulness and correctness of the zPDT STP function for your environment.
Both the client and server create log files. Our sample configuration places these in /tmp. Existing log files are overwritten when a server or client is started. The server log files typically have little in them. A client log file might have entries such as these:
Wed Oct 8 11:14:54 EDT 2014 (<-- this is when this log file started)
%CCT (TAI seconds) Offset (us) Dispersion Skew (ppm) Steering RTT (us)
1412781391.767832 -0.088 0.009 0.00 0.00 0.228
1412781451.774840 -0.102 0.006 0.00 0.00 0.183
1412781511.782228 -0.103 0.006 0.00 0.00 0.181
1412781571.789614 -0.097 0.026 -0.00 0.00 0.212
With the default parameters, a client log line is written every minute. As a general statement, you do not need any information from the logs except, perhaps, an indication of how often the client might have failed. (The default parameters cause the client to restart itself after a failure.) The log files are named STPServerLog and STPClientLog and are placed in the Linux directory you specify in the STP configuration file. These two files are open whenever the STP function is active. If you want to capture the contents of a log file for later inspection (remembering that they are overwritten whenever the STP function is started) use the Linux cp command, as in this example (the cp command may be used against an active Linux file):
$ cp /tmp/STPClientLog /tmp/STPClint.monday3PM
The fields in the client log are as follows:
TAI International Atomic Time (TAI) is the number of seconds since 1 January 1972, including leap seconds. (The leap second data is not normally available to the zPDT newcct program, and this definition might not be completely accurate.)
Offset The difference (in microseconds) between the master CCT clock (the stratum 1 server) and the client clock, as calculated by the client.
Dispersion The uncertainty in the calculated offset value.
Skew The difference in the clock stepping rate between the master and the slave.
Steering The rate at which the client clock is being steered in order to reduce the offset toward zero.
RTT The minimum measured round-trip delay for a set of CCT sample “ping-pong messages”. More precisely, it is the sum of the minimum forward delay time and minimum backward delay time for a set of CCT sample ping-pong messages,
The “ping-pong” messages mentioned are short message exchanges between a client and server.
The time-of-day provided to z/OS by the CCT/STP function ultimately depends on the time-of-day returned by the effective Linux clock.1 A user might have a substantially inaccurate time-of-day in the PC BIOS clock that is used to set the Linux clocks when Linux is booted. Another user might have an NTP2 connection for Linux, resulting in a Linux clock that is accurate to within a second or so. At the zPDT level, there is no control over the accuracy of the time data provided by Linux. The STP function keeps multiple zPDT TOD values synchronized, but does not control the accuracy of the time presented.
The log files are not intended for normal user inspection and may contain messages with whimsical phrasing, such as this example:
Incomplete PingPong exchange on client: Resource temporarily unavailable
This indicates a client failure, but the client immediately restarts itself. We have seen these messages at random intervals on zPDT CCT networks very closely coupled (private 1 GB Ethernet) but have not seen them on other systems connected through the “real” Internet. In our test environment, these transient failures appear to have no effect on z/OS basic sysplex operation. Corresponding log messages on the CCT server include these:
Failed to read doorbell msg
Welcome (this denotes a client restart)
When a remote client connects to the STP server, another server instance is automatically started. No action is needed to manage this, but you might see multiple STP servers when using the Linux ps command.
If you have an [stp] stanza in a devmap, you cannot start this devmap (using the awsstart command) unless you have an STP function active. If you do not have STP active, the zPDT startup will fail. The most common error message is INVALID REGISTER NUMBER.
The zPDT STP facility relies on a fast, low-jitter TCP/IP connection between host Linux systems in order to synchronize the zPDT TOD clock values. Insufficient TCP/IP connection quality results in STP reporting either unsynchronized or unusable TOD clock-source machine checks to the zPDT-hosted OS. You must provide a physical TCP/IP connection with sufficient quality to support machine-check-free STP operation. In a virtualized environment, the hypervisor increases the TCP/IP latency and jitter, as perceived by the STP facility, increasing the risk of disruptive zPDT STP machine checks. We have performed limited STP testing in a virtualized environment with mixed results and therefore recommend that STP not be used in a virtualized environment.
20.3.1 Leap seconds
 
Tip: There is no need to deal with leap second details unless you (or your applications) are sensitive to them. We expect 99% of zPDT users will ignore leap seconds.
You may update the leap second information block by including an additional statement in the devmap, as in the following example:
[stp]
ctn 00000000F1F0F9F0 #16 hex digits beginning with 00
node 1 W520 * #asterisks marks this node
node 2 W510
LEAPSECONDS 25 26 2015 6 30 23 59 59
The format for the LEAPSECONDS statement is as follows:
LEAPSECONDS <active> <new> <year> <month> <day> <hour> <minute> <second>
where:
<active> is the number of leap seconds currently present. (For the first half of 2017 this number is 27.)
<new> is the replacement number of leap seconds. (This is not an additional value, but a complete replacement of the active number. It potentially can be a smaller number than the <active> value, although this has not happened yet.) The difference between <active> and <new> should never be more than one.
<year><month><day><hour><minute><second> is the time at which the <new> value is to be effective.
Two new zPDT commands (issued in a Linux command window) are provided for leap second details:
$ d lso (display the leap second information block)
$ st lso <active><new><year><month><day><hour><minute><second>
$ st lso 25 26 2015 6 30 23 59 59 (an example)
We understand that the z/OS CLOCKxx parameter STPZONE must be set to YES for z/OS to process leap seconds when using an STP time source. The leap second information block is displayed, as in the following example:
LEAP SECOND OFFSET INFORMATION BLOCK
word(0): 80000000
lso active: 25
lso new: 26
update time: 73061E173B3B00 - 2015: 6:30:23:59:59
 

1 This is a slightly complex topic because Linux has several clocks; we ignore this detail here.
2 Network Time Protocol (NTP) references an accurate time-of-day service available over the Internet.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.117.183.172