Disaster recovery and high availability
This chapter describes the various data and volume replication techniques that are implemented in DS8000 Copy Services, which provide the foundations for disaster recovery (DR) operations and enable high data availability.
DS8000 Copy Services are complemented by management frameworks and functions that are built directly into the IBM Z software and firmware. This functional complementarity allows a further enhanced automation approach that can handle almost any potential incident, even in an unattended environment that is composed of IBM Z servers and DS8880 storage systems.
This chapter includes the following topics:
2.1 DS8880 Copy Services functions
The DS8880 storage system provides broad and rich copy services functions. They can be used for 2-, 3-, or 4-site solutions. These scenarios are described in this chapter.
2.1.1 Metro Mirror 2-site synchronous volume replication
Metro Mirror (MM) is the synchronous volume replication approach that uses DS8880 firmware. The technology that is used is known as Peer-to-Peer Remote Copy (PPRC).
IBM introduced this technology in the mid-1990s and released it to customers in 1996 with the IBM 3990-9 cached storage control unit. Throughout the years, the technology was continuously enhanced, starting with the IBM RAMAC Virtual Array (IBM RVA), followed by the IBM Enterprise Storage Server®, and finally, the DS8000 series.
Figure 2-1 shows the basic sequence of operations. The goal is to ensure the safe and consistent arrival of the data at the receiving site of MM with the least cycles possible between both sites. MM achieves this goal quickly.
Figure 2-1 Metro Mirror basic operation
A balanced approach between pre-deposited MM writes at the receiving site and occasional feedbacks from the receiving control unit back to the sending control unit allows the use of the Fibre Channel links and MM paths between both sites as efficiently as possible.
2.1.2 Global Mirror 2-site asynchronous volume replication
IBM offers a software-based solution to provide a replication technology that can bridge any distance and still ensure consistent data at the remote site. It was first introduced as Extended Remote Copy (XRC). XRC was rebranded as IBM z/OS Global Mirror (IBM zGM).
The IBM Z software holds the key component of IBM zGM, which is the System Data Mover (SDM). SDM is a part of Data Facility Storage Management Subsystem Data Facility Product (DSSMSdfp) that is included in z/OS and can handle all data that is written by using
IBM Extended Count Key Data (IBM ECKD™) channel programs. Otherwise, SDM cannot be used with open system volumes or LUNs.
To cover any storage server volume type, IBM developed a solution that is independent of any host type or volume type, which is known as Global Mirror (GM). The goal was to provide an asynchronous replication technology that can run in an autonomic fashion and provide data currency (within no more than 5 seconds) at a distant site.
The data consistency at a distant site is ensured for a single storage server pair and across as many as up to 16 storage systems. This consistency is made possible without any other constraints, such as imposing time stamps with each single write I/O.
The basic operational sequence of GM is shown in Figure 2-2.
Figure 2-2 Global Mirror basic operation managed out of the storage system
From a host perspective, the write I/O behaves as though it is writing to a non-mirrored volume. The host receives an I/O completion event when the write data arrives in the cache and non-volatile cache portion of the DS8000 cache. The DS8000 storage system then asynchronously replicates the data and sends it to the remote site. When the data is secured in the remote cache and remote non-volatile cache, the replication I/O is completed.
GM combines the Global Copy and FlashCopy functions. Global Copy performs the data replication and FlashCopy secures the previous data from H2 onto J2 before the respective track on H2 is overwritten by the replication I/O. Therefore, J2 behaves as a journal for H2.
The GM consistency group creation process is solely performed within the DS8000 storage systems. Synergy comes into play when managing such a configuration through IBM Copy Services Manager (CSM) or IBM Geographically Dispersed Parallel Sysplex (GDPS).
2.1.3 z/OS Global Mirror 2-site asynchronous volume replication
IBM zGM essentially is z/OS software-based asynchronous volume replication. The design goal for IBM zGM was to not exceed 5 seconds in remaining current with the primary site.
IBM zGM relies on timestamped ECKD write I/Os for each write I/O to each IBM zGM primary Count Key Data (CKD) volume. IBM zGM can manage only CKD volumes.
IBM zGM is a powerful session-based architecture that is based on z/OS system software. This approach allows easy management of IBM zGM based replication configurations. Although IBM zGM is simple to implement and configure, other considerations in fine-tuning the environment are required, especially when bridging long distances between sites and with limited bandwidth. For long distances, the use of channel-extending devices is required.
The basic components of IBM zGM operations are shown in Figure 2-3.
Figure 2-3 IBM zGM basic operation that is managed through IBM Z software
Figure 2-3 on page 10 shows how closely IBM Z software cooperates with the DS8000 storage system:
1. Application write I/Os perform at the same speed as with writing to an unmirrored volume in H1. Each write I/O also contains a unique time stamp. The Parallel Sysplex Timer clock is used.
2. Immediately after successfully storing the data to cache and non-volatile cache storage in the DS8000 storage system, the I/O is complete from an application perspective.
3. SDM is a highly parallel working driver that fetches the data from the H1 site as fast as possible by using particular enhancements in z/OS and its Input/Output Supervisor (IOS). Any bandwidth between sites can be used by the SDM and through its multiple-reader support.
4. SDM internally sorts all write I/Os according to the applied time stamp during application write I/O processing to ensure the same write order to the secondary volumes as they occurred to the primary volumes.
5. To resume operations after an unplanned outage of any component within the remote site, SDM applies first the consistency groups onto a journal. Next, SDM writes the same consistency group (or groups) to the secondary volumes and then frees up the corresponding journal space.
After IBM zGM reaches a balanced system level (combining all involved components), it is a firm solution that runs unnoticed. The key requirement is to provide enough bandwidth between sites and on the storage back end at the recovery site to manage the amount of write data that is arriving at the local site.
A limiting factor is the primary storage system cache size and the number of Storage Control sessions. A Storage Control session is created for each logical control unit (LCU) that holds IBM zGM primary volumes. Here, the maximum number is 40 LCUs at the primary site. Regarding the primary storage cache size, a best practice is to upgrade to the maximum available cache size. The guideline is for an IBM zGM session not to exceed 1500 - 2000
IBM zGM volume pairs.
An SDM instance is a z/OS address space. The level of parallelism within an address space cannot go beyond 255 tasks, which can run in parallel within that address space. These tasks are subdivided between all these SDM functions, such as parallel reading record sets from the primary site, parallel writing to journal and secondary volumes, parallel establishment of volume pairs, and parallel running control tasks.
To overcome the limit within a single SDM instance, IBM zGM scales to up to
182 interconnected SDM instances (as shown in Figure 2-4) and still can provide data consistency across all involved volumes. You might cluster SDM sessions within a logical partition (LPAR) or couple individual SDM instances across LPARs or a combination of clustered SDM instances with individual SDM instances.
Figure 2-4 IBM zGM scales almost unlimited with up to 182 SDM instances
For more information about planning and best practices, see IBM z/OS Global Mirror Planning, Operations, and Best Practices, REDP-4878.
The performance of SDM highly depends on the amount of real storage that SDM gets, which is ensured in the corresponding z/OS image (LPAR). A best practice is that more is better than less. Approximately 1.7 GB of real storage for an SDM instance is useful in a larger configuration. For more information about precise calculations, see z/OS DFSMS Advanced Copy Services, SC35-0428.
IBM zGM is a perfect example of synergy between IBM Z and the DS8000 storage system.
IBM zGM and GM overlap in their goal to provide consistent data at the remote site at any time. Also, the design goal to ensure a gap of no more that 5 seconds, or even less for the recovery point objective (RPO) between both sites, is similar. Each approach has its particular strength. Which solution to choose depends on the individual customer needs and IT environment.
2.1.4 Metro/Global Mirror 3-site solution
Metro/Global Mirror (MGM) is solely based on the DS8000 firmware. It is a cascaded approach that spans over three sites.
The first leg is an MM relationship from site 1 to site 2. Then, it continues to a potentially distant site 3 with GM. The role of site 2 volumes is a cascaded status. The same volume in site 2 is an MM secondary volume in a DUPLEX state. At the same time, it is a Global Copy primary volume with a PENDING status.
The basic components of an MGM configuration are shown in Figure 2-5.
Figure 2-5 Metro/Global Mirror cascaded 3-site solution
Often, site 1 and site 2 are in a campus or metropolitan area within MM acceptable distances. In a typical IBM Z environment, both sites are usually at a distance that is also supported by the Parallel Sysplex architecture and spread the IBM Z server across site 1 and site 2.
Both sites might be only a few kilometers apart from each other to allow an efficient data sharing approach within the coupling facility. MM acceptable distances are often measured from approximately 100 meters (328 feet) to approximately 5 - 6 km (3.10 - 3.72 miles) because the synergy of this configuration relies on Parallel Sysplex functions and MM in combination with IBM HyperSwap.
When coupling facility-based data sharing is not a potential issue, the distance can be as much as the supported distance by the Parallel Sysplex Timer, which is up to 100 km
(62.13 miles) between site 1 and site 2. Then, another synergy item plays a significant role. When a HyperSwap occurs and application writes run from site 1 application servers to the site 2 storage systems, the new High Performance FICON for IBM Z (zHPF) Extended Distance II support improves the performance of large write I/Os.
Site 1 and site 2 fulfill the role of DR covering site 1 and site 2 and high availability (HA) when site 1 components fail or the storage server (or servers) experience any type of outage. Site 3 is a pure DR site when site 1 and site 2 are no longer available.
 
Important: A management framework, such as CSM or GDPS, is required to manage such a 3-site volume replication configuration.
2.1.5 Multiple Target Peer-to-Peer Remote Copy 3-site solutions
Multiple Target Peer-to-Peer Remote Copy (MT-PPRC) is available with DS8880 or DS8870 configurations with firmware levels 7.4 or later. It is also a 3-site Copy Services configuration.
It is called Multiple Target PPRC because it can have two Copy Services that are based on volume copies in site 2 and another in site 3. These Copy Services relationships can be either of the following approaches:
•Two MM relationships of the same site 1 volume
•A combination of an MM relationship between site 1 and site 2 and a second GM or Global Copy relationship from site 1 to another site 3
Both approaches are described next.
MT-PPRC can be used to migrate data from primary or secondary DS8000 storage systems in PPRC configuration. The use of MT-PPRC allows for a migration procedure with little or no periods in which the system is not protected by mirroring.
MT-PPRC 3-site with two synchronous targets
The basic mode of operation and configuration of MT-PPRC with two MM relationships is shown in Figure 2-6.
Figure 2-6 MT-PPRC with two synchronous targets
With MT-PPRC and two MM relationships, the DS8000 storage system provides another level of DR and, combined with HyperSwap, another level of HA.
The primary storage system schedules two parallel and synchronous replication writes to a target DS8000 storage system in site 2 and to another target DS8000 storage system in site 3. After both replication writes succeed, the application write is considered to be successfully completed by the host server.
Depending on the available SAN infrastructure, site 2 and site 3 also can be connected to potentially allow synchronous replication from site 2 to site 3 or the opposite configuration if a HyperSwap event occurs in site 1. This configuration is indicated by the HyperSwap action that is shown in Figure 2-6 on page 14 and requires FICON connectivity from the IBM Z server in site 1 to site 2 or site 3.
Managing such a 3-site MT-PPRC configuration is supported by GDPS (GDPS/Multi-Target Metro Mirror (MTMM)) or by CSM. This support is also proof of how closely IBM Z based Copy Services software and HyperSwap interact with the connected DS8000 storage systems.
MT-PPRC 3-site configuration with a synchronous and asynchronous target
Another possible MT-PPRC configuration, which implies a Parallel Sysplex configuration across site 1 and site 2, is shown in Figure 2-7.
Figure 2-7 MT-PPRC with synchronous and asynchronous target of H1
In this configuration, the storage system in site 1 synchronously replicates disk storage volumes over MM to site 2.
Although the SAN fabric configuration that is shown in Figure 2-7 on page 15 also allows a cascaded configuration from site 2 to site 3, the implication here is that GM is the second Copy Services relationship from site 1 to site 3 and running through a SAN fabric, which might have SAN switches in all three sites. This configuration allows for the highest flexibility.
You must also plan for a useful redundancy in the SAN fabric, which is not shown in Figure 2-7 on page 15.
Similar considerations apply, as described in “MT-PPRC 3-site with two synchronous targets” on page 14. HyperSwap might transfer the active site from site 1 to site 2 and carry the GM relationship from site 1 to site 2. This configuration can keep the GM relationship active and preserve DR capability in site 3 following a potential HyperSwap event.
When returning the active site to site 1 (if the storage system in site 1 is running again),
CSM supports an incremental resynch approach: from site 2 to site 1 and returning to site 1 through a planned HyperSwap operation from site 2 to site 1 when H2 and H1 are back in a FULL DUPLEX state.
Another subsequent incremental resynch from H1 to H2, and automatically enabling HyperSwap when H1 and H2 are back in FULL DUPLEX state, establishes the original configuration, including returning GM back to H1 to H3/J3.
This configuration includes some potential to provide a powerful HA and DR configuration.
Again, this configuration combines IBM Z server-based services (such as HyperSwap and hosting Copy Services management software) and combines it with the unique DS8880 Copy Services capabilities.
2.1.6 Symmetrical HA/DR 4-site solutions
Many customers conduct regular failover operations to DR sites to comply with regulation requirements, perform data center maintenance at primary locations, or exercise DR capabilities. In such environments, having the ability of still maintaining HA while systems are running on DR sites is a requirement.
The combination of cascading and MT-PPRC technologies that are present on DS8880 machines can provide 4-site solutions so that clients can have sites spanning two different metropolitan areas while still maintaining HA and DR capabilities between them independent of where their systems are running. How this solution can be accomplished is shown in Figure 2-8.
Figure 2-8 Symmetrical HA/DR 4-site solution
In this example, sites 1 and 2 are running a production workload on “Metropolitan Area A” while Sites 3 and 4 are a DR environment on “Metropolitan Area B”.
H1 includes an active HyperSwap capable MM relationship with H2, and H2 includes an active GM relationship with H3. H1 also features a “stand-by” GM relationship with H3 (denoted with a doted line in Figure 2-8).
In this case, if H2 encounters any issues, the GM relationship can be taken over by H1 without the need of a full synchronization to H3. H3 also has an active Global Copy relationship with H4, which allows H4 to be close to a “synchronized state” with H3.
If a planned or unplanned DR situation is declared, the systems can be failed over from Metropolitan Area A to Metropolitan Area B. In this case, an H3 to H4 replication relationship can be then converted from Global Copy to MM and HyperSwap can be enabled between them, which provides data HA.
When Metropolitan Area A is ready to be reactivated, H1 becomes the target site for the GM relationship and features a Global Copy relationship that is started from H1 to H2. After the four sites are synchronized, the systems can remain running on Metropolitan Area B with DR protection on Metropolitan Area A, or can be failed back to Metropolitan Area A making Metropolitan Area B again the DR environment.
This configuration is fully supported by GDPS and can be implemented with CSM by using a combination of different CSM session types. These IBM products can automate all of the management and failover steps that are involved to meet your requirements. For more information, see IBM GDPS Family: An Introduction to Concepts and Capabilities, SG24-6374 or contact your IBM representative.
2.2 z/OS HyperSwap
For many years, z/OS provided the capability to swap transparently the access from one device to another device. First uses for disk storage volumes occurred with PPRC dynamic address switching (P/DAS). Through system commands such as IOACTION and SWAP, transparently switching I/O from a primary MM volume to a secondary MM volume is possible. This capability was implemented in the last editions of IBM 3990-6 and IBM RVA disk storage control units.
For more information, see the following IBM Knowledge Center pages:
The z/OS based swap process that redirects I/Os from device H1 to another device H2 (if these devices are in an MM relationship and in the FULL DUPLEX state) is shown in Figure 2-9.
Figure 2-9 z/OS swap process
The core of this process is to exchange (swap) the content of the two unit control blocks (UCBs), which represent the disk storage devices. Among the many details it contains about a device, the UCB also includes one or more channel paths or one or more CHPIDs that connect to the two devices.
Figure 2-9 on page 18 also shows the status just after the UCB swap operation. Before the swap, all I/O to the device on 123 (which is the MM primary device) ran through channel-path identifier (CHPID) 6E.
Eventually, all I/O traffic to 123 is stopped before the actual swap operation occurs. After all I/O to 123 is quiesced, the swap process exchanges the UCB content of device 123 and device 456. After the swap is completed, IOS resumes I/O operations, and the UCB eventually directs the resumed I/O to CHPID BA, which connects to device 456. An earlier step of the swap process is also to change the MM status of the device on 456 from the SECONDARY DUPLEX to the PRIMARY SUSPENDED state.
IBM enhanced this swap process and raised the number of swap operations that are running in parallel. With today’s processor speed and creating dedicated highly parallel running swap services within the HyperSwap address space, many thousands of swap operations can occur in a single-digit number of seconds. Highly cultivating this swap process to today’s standard is now called HyperSwap and performs in its own address space.
In addition to the actual swap operation that the z/OS HyperSwap service provides, certain DS8000 Copy Services commands can be issued during the swap operation to trigger freeze and failover functions within the DS8000 storage system.
Also, IOS understands HyperSwap triggers to perform autonomically a HyperSwap operation after such a trigger is raised, based on an issue to or within the primary storage server in H1.
Because this HyperSwap service is not an externalized interface, another authorized user must enable this service and exercise close cooperation with this z/OS based HyperSwap service.
Currently, authorized users of HyperSwap services are CSM and GDPS. Both solutions manage Copy Services configurations and closely interact with the HyperSwap address space to provide a Copy Services configuration to HyperSwap services after the configuration is in a proper FULL DUPLEX state.
2.3 Copy Services Manager and HyperSwap
CSM, formerly known as IBM Tivoli® Storage Productivity Center for Replication, is required to use HyperSwap within z/OS for an MM configuration. This section does not describe CSM beyond the fact that it manages sessions. Such a session contains all MM volume pairs that are set up and defined within a Parallel Sysplex configuration. From a user perspective, the entity of management is the session only.
CSM is server-based and includes two interfaces: a graphical user interface (GUI) and a command-line interface (CLI). The CSM server often is preinstalled on the DS8880 Hardware Management Console (HMC). It also can run on all common server platforms, such as IBM AIX®, Linux, Microsoft Windows, and on z/OS within the UNIX System Services or UNIX System Services shell.
CSM can handle all z/OS based CKD volumes within its MM session, even when the
CSM server is hosted on the HMC or a distributed server. However, a best practice is to use the robust IBM Z server platform and place the CSM server in a z/OS LPAR. Also, when possible, you can host the CSM stand-by server in another z/OS LPAR at the other site.
As shown in Figure 2-9 on page 18, CSM appears disabled because the CSM is not necessary when z/OS performs a HyperSwap operation. This fact is also true when HyperSwap is enabled within a Parallel Sysplex configuration. Therefore, CSM is also disabled, as shown in Figure 2-10.
Figure 2-10 HyperSwap enabled: CSM is passive
As shown in Figure 2-10, HyperSwap is represented in an LPAR by the following address spaces:
•HyperSwap API (HSIBAPI) address space that handles the actual swap process
•HyperSwap Management (HSIB) address space that is the communication handler to its peers in other Parallel Sysplex members
Figure 2-10 also shows normal operations with the active I/O paths connected to H1 volumes, which are MMed to H2 and are all in a healthy FULL DUPLEX state. This requirement must be met to reach the HyperSwap enabled state.
You can also query the HyperSwap status by using the z/OS display command, as shown in Example 2-1.
Example 2-1 Querying the HyperSwap status by using a z/OS system command
D HS,STATUS
 
IOSHM0303I HyperSwap Status 671
Replication Session: MGM
HyperSwap enabled
New member configuration load failed: Disable
Planned swap recovery: Disable
Unplanned swap recovery: Partition
FreezeAll: Yes
Stop: No
Example 2-2 shows another z/OS system command that you can use to control HyperSwap and disable or enable HyperSwap when a HyperSwap session exists. Because only one potential HyperSwap session is allowed within a Parallel Sysplex, you do not need to refer to a specific session name in the SETHS z/OS system command.
Example 2-2 z/OS system commands to disable or enable HyperSwap
RO *ALL,SETHS DISABLE
 
RO *ALL,SETHS ENABLE
Sometimes, you might want to disable HyperSwap in a planned fashion to avoid a HyperSwap operation during a controlled and planned activity that might trigger a HyperSwap operation. After such a controlled activity, you can again enable HyperSwap so that z/OS HyperSwap regains control.
Example 2-3 shows another z/OS system command that you can use to query a complete HyperSwap configuration. However, that command might not be useful when thousands of MM volume pairs are within the HyperSwap session.
Example 2-3 Querying the complete HyperSwap configuration
D HS,CONFIG(DETAIL,ALL)
IOSHM0304I HyperSwap Configuration 495
Replication Session: MGM
Prim. SSID UA DEV# VOLSER Sec. SSID UA DEV# Status
A0 31 0A031 A#A031 40 31 04031
A0 7F 0A07F A#A07F 40 7F 0407F
A0 AF 0A0AF A#A0AF 40 AF 040AF
A0 72 0A072 A#A072 40 72 04072
A0 A2 0A0A2 A#A0A2 40 A2 040A2
A0 13 0A013 A#A013 40 13 04013
. . . . . . . . . . . . . . . .
To see why a HyperSwap session is disabled, a basic approach is to start with another D HS z/OS system command, as shown in Example 2-4.
Example 2-4 Querying HyperSwap for exceptions
D HS,CONFIG(EXCEPTION,ALL)
IOSHM0304I HyperSwap Configuration 840
Replication Session: MGM
None Duplex
2.3.1 HyperSwap to site H2
A healthy and enabled HyperSwap session is shown in Figure 2-10 on page 20. What happens if this session faces difficulties and HyperSwap cannot remain enabled?
For some reason (planned or unplanned), a HyperSwap trigger changed the HyperSwap configuration to the configuration that is shown in Figure 2-11.
Figure 2-11 After HyperSwap: CSM is still passive
After a planned or unplanned HyperSwap switched the active volumes from H1 to H2,
CSM remains passive and not involved. HyperSwap eventually notifies CSM about the session state change after the HyperSwap operation is completed.
During the actual swap operation, HyperSwap is also responsible for issuing all the necessary Copy Services commands to perform the complete failover to H2. This failover also leads to the MM state change of the H2 volumes from secondary DUPLEX to primary SUSPENDED.
2.3.2 Returning to site H1
After the decision is made to return the active site to H1 and if the DS8000 storage system in H1 is recovered and still holds all the data at a level when the HyperSwap occurred, CSM is required to perform the necessary steps.
The first steps to return the active site to H1 are shown in Figure 2-12.
Figure 2-12 Reestablish Metro Mirror and eventually return to H1: CSM required
When H2 was the active site, all relevant updates to the H2 configuration were logged within the DS8000 storage system and the corresponding bitmap.
You must return to the corresponding CSM session. There, you discover that the session changed and is no longer in a green OK status. To start the process and to return the active volumes to H1, complete the following steps:
1. Modify the CSM session to allow the volumes to be replicated in the opposite direction that it was using. This action enables the session to replicate from H2 to H1.
2. Start the replication to resynchronize incrementally the volumes from H2 to H1 by using a CSM START_H2_H1 command.
After the incremental resynchronization process is completed for all volumes within the session and everything is back to DUPLEX, CSM returns the configuration to HyperSwap, which then switches back to enable the session for HyperSwap ready.
To return the active volumes back to H1, complete the following steps:
1. Issue a planned HyperSwap through the CSM or by using the SETHS SWAP z/OS system command. This command again performs a swap operation and puts the active volumes back to H1, including the MM status of primary SUSPENDED.
2. CSM performs the following actions as before:
a. Allows the session to replicate from H1to H2.
b. Resynchronizes all the volume pairs from H1to H2 through another START_H1_H2 command.
After all MM pairs are in the full DUPLEX state, CSM again signals the new configuration to HyperSwap, which in turn enables HyperSwap ready and the replication continues now from H1 to H2.
2.3.3 Summary
CSM follows the high IBM standards that also apply for the enhancements that are made to IBM Z and the DS8000 storage system.
CSM is the enabler for z/OS and its HyperSwap function. This synergy allows a 2-, 3-, or 4-site MM-based disk volume configuration to achieve high standards in data availability and DR readiness in a fully transparent fashion to the application I/Os.
2.4 Geographically Dispersed Parallel Sysplex
GDPS is a solution that manages complex multisite DR and HA IBM Z environments. GDPS simplifies DS8000 storage system replication and parallel sysplex management while providing end-to-end application business resilience.
To address an entire site failure, GDPS can perform a site-switch to another local site or to a remote (out-of-region) location that is based on predefined, automated scripts. Various GDPS offerings are available (see Figure 2-13); each addresses specific DR and HA goals that can be customized to meet various RPO and recovery time objective (RTO) requirements.
Figure 2-13 GDPS offerings
One difference between options is in the type of DS8000 Copy Services that are used as a building block for DR and HA design. The following Copy Services are used:
•GDPS/PPRC HyperSwap Manager (HM) and GDPS/PPRC are based on DS8000 synchronous data replication MM (known as PPRC).
•GDPS/GM is based on the DS8000 GM, which is an asynchronous form of remote copy.
•GDPS/XRC uses asynchronous data replication XRC (also known as IBM zGM).
•GDPS/MGM uses MM and GM disk replication for a 3-site or 4-site DR and HA environment.
•GDPS/MTMM supports MTMM on DS8000 storage systems. GDPS/MTMM provides similar capabilities as the capabilities that are available in GDPS/PPRC while extending PPRC management and HyperSwap capabilities to cover the two replication legs.
•GDPS/MzGM uses MM and XRC or IBM zGM disk replication for a 3-site or 4-site DR and HA environment.
•GDPS Active/Active is a multisite HA/DR solution at virtually unlimited distances. This solution is based on software-based asynchronous mirroring between two active production sysplexes that are running the same applications with the ability to process workloads in either site.
For more information about GDPS and each option, see IBM GDPS Family: An Introduction to Concepts and Capabilities, SG24-6374.
2.4.1 GDPS and DS8000 synergy features
Almost all GDPS solutions (except for GDPS active/active) rely on IBM disk replication technologies that are used in the DS8000 storage family. This section provides more information about the key DS8000 technologies that GDPS supports and uses.
Metro Mirror (PPRC) failover/failback support
When a primary disk failure occurs and the disks are switched to the secondary devices, failover/failback support eliminates the need to perform a full copy when reestablishing replication in the opposite direction. Because the primary and secondary volumes are often in the same state when the freeze occurred, the only differences between the volumes are the updates that occur to the secondary devices after the switch.
Failover processing sets the secondary devices to primary suspended status and starts change-recording for any subsequent changes that are made. When the mirror is reestablished with failback processing, the original primary devices become secondary devices and changed tracks are resynchronized.
GDPS/PPRC transparently uses the failover/failback capability. This support mitigates RPO exposures by reducing the amount of time that is needed to resynchronize mirroring after a HyperSwap. The resynchronization time depends on how long mirroring was suspended and the number of changed tracks that must be transferred.
GDPS now also supports MTMM on the IBM DS8880 and IBM DS8870 storage systems. Initial support is for two synchronous copies from a single primary volume, also known as an MTMM configuration. GDPS/MTMM provides similar capabilities as the capabilities that are available in GDPS/PPRC while extending PPRC management and HyperSwap capabilities to cover the two replication legs.
Global Copy
Global Copy (formerly known as PPRC-XD) is an asynchronous form of the DS8000 advanced copy functions. GDPS uses Global Copy rather than synchronous MM (PPRC) to reduce the performance effect of certain remote copy operations that potentially involve a large amount of data. The replication links are typically sized for steady state update activity, but not for bulk synchronous replication, such as initial volume copy or resynchronization.
Initial copy or resynchronizations by using synchronous copy do not need to be performed because the secondary disks cannot be made consistent until all disks in the configuration reach the duplex state. Therefore, GDPS supports initial copy and resynchronization by using asynchronous Global Copy.
When GDPS starts copy operations in asynchronous copy mode, GDPS monitors the progress of the copy operation. When the volumes are near full duplex state, GDPS converts the replication from the asynchronous copy mode to synchronous. Initial copy or resynchronization by using Global Copy eliminates the performance effect of synchronous mirroring on production workloads.
The use of asynchronous copy allows clients to establish or resynchronize mirroring during periods of high production workload. It also might reduce the time during which the configuration is exposed.
DS8000 Health Message Alert
An unplanned HyperSwap is started automatically by GDPS if a primary disk failure occurs.
In addition to a disk problem being detected as a result of an I/O operation, a primary disk subsystem can proactively report that it is experiencing an acute problem. The DS8000 storage system features a special microcode function that is known as the Storage Controller Health Message Alert capability. It alerts z/OS when hardware events occur and generates a message and Event Notification Facility (ENF) signal, as shown in Example 2-5.
Example 2-5 DS8880 health message alert
IEA074I STORAGE CONTROLLER HEALTH,MC=20,TOKEN=1004,SSID=AB01, DEVICE NED=2107.961.IBM.75.0000000ABCD1.0100,PPRC SECONDARY CONTROLLER RECOVERY ACTION
Problems of different severities are reported by the DS8000 storage system. Those problems that are classified as acute are also treated as HyperSwap triggers. After systems are swapped to use the secondary disks, the disk subsystem and operating system can attempt to perform recovery actions on the former primary without affecting the applications that are using those disks.
One main benefit of the Health Message Alert function is to reduce false freeze events. GDPS Freeze and Conditional Stop actions query secondary disk subsystem to determine whether systems can be allowed to continue in a freeze event.
Metro Mirror (PPRC) suspension (Summary Event Notification)
An MM suspension generates a message aggregation that is also known as Summary Event Notification. This aggregation dramatically reduces host interrupts and operator messages when MM volume pair is suspended.
When GDPS performs a freeze, all primary devices in the MM configuration suspend. This suspension can result in significant state change interrupt (SCI) traffic and many messages in all of the systems. GDPS with z/OS 1.13 (and later) and microcode on the DS8000 storage system supports reporting suspensions in a summary message per DS8000 LCU instead of at the individual device level.
When compared to reporting suspensions on a per devices basis, the Summary Event Notification dramatically reduces the message traffic and extraneous processing that is associated with MM suspension events and freeze processing. Examples exist where 10,000 operator messages were reduced to under 200.
Soft Fence
After a GDPS HyperSwap or an unplanned site switch, potential exposures exist to systems that are connected to the original primary MM (PPRC) volumes. As shown in Figure 2-14, after a planned or unplanned HyperSwap, the GDPS changes the secondary volumes to primary suspended; however, the former primary volumes statuses remain unchanged. Therefore, these devices remain accessible and usable to any system within or outside the sysplex that is connected to them. In this way, the possibility exists to update or perform an IPL accidentally from the wrong set of disks, which can result in a potential data integrity or data loss problem.
Figure 2-14 GDPS and DS8000 Soft Fence feature
GDPS uses a DS8000 capability that is called Soft Fence to fence (block access to a selected device). GDPS uses Soft Fence when appropriate to fence devices that otherwise might be exposed to accidental update, for example, after a GDPS HyperSwap event, as shown in Figure 2-14.
Although GDPS includes built-in protection features that prevent an IPL of the systems from the wrong set of disks, the DS8880 Soft Fence function is more protection. If an IPL of any system is done manually (without GDPS), the attempt of an IPL from the wrong set of disks (fenced former primary MM volumes) is prohibited.
Also, other systems that are outside the sysplex and therefore outside GDPS control can access the former primary MM volumes. Soft Fence protection blocks any attempt to update these volumes.
On-Demand Dump
When problems occur with disk subsystems, such as problems that result in an unplanned HyperSwap, a mirroring suspension, or performance issues, a lack of diagnostic data from the time that the event occurs can result in difficulties in identifying the root cause of the problem. Taking a full statesave can lead to temporary disruption to host I/O.
The On-Demand Dump (ODD) capability of the DS8880 storage system facilitates taking a nondisruptive statesave (NDSS) at the time such an event occurs. The DS8000 microcode performs this statesave automatically for certain events, such as generating a dump of the primary disk system that triggers a freeze event, and allows an NDSS to be requested by a user. This feature enables first failure data capture (FFDC) and ensures that diagnostic data can help in problem determination efforts.
GDPS supports taking an NDSS that uses the remote copy pages (or web GUI). In addition to this support, GDPS autonomically takes an NDSS if an unplanned freeze or HyperSwap event occurs.
Query Host Access
When an MM (PPRC) disk pair is being established, the device that is the target (secondary) must not be used by any system. The same is true when establishing a FlashCopy relationship to a target device. If the target is in use, the establishment of the PPRC or FlashCopy relationship fails.
When such failures occur, identifying which system is delaying the operation can be a tedious task. The DS8000 Query Host Access function provides the means to query and identify what system is using a selected device. This function is used by the IBM Device Support Facilities (ICKDSF) utility (for more information, see 4.5, “Volume formatting overwrite protection” on page 56) and by GDPS.
GDPS has the following capabilities:
•Query Host Access identifies the LPAR that is using the selected device through the CPU serial number and LPAR number. For the operations staff to convert this information to a system or CPU and LPAR name is still a tedious job. GDPS performs this conversion and presents the operator with more readily usable information, which avoids this extra conversion effort.
•When GDPS is requested to perform a PPRC or FlashCopy establish operation, GDPS first performs Query Host Access to determine whether the operation is expected to succeed or fail as a result of one or more target devices being in use. GDPS alerts the operator if the operation is expected to fail, and identifies the target devices in use and the LPARs holding them.
•GDPS continually monitors the target devices that are defined in the GDPS configuration and alerts operations that target devices are in use when they should not be in use. This alert allows operations to fix the reported problems in a timely manner.
•With GDPS, the operator can perform ad hoc Query Host Access to any selected device by using the GDPS pages (or GUI).
IBM DS8000 Easy Tier Heat Map Transfer
IBM DS8000 Easy Tier Heat Map Transfer (HMT) can transfer the Easy Tier learning from an MM (PPRC) primary to the secondary disk subsystem such that the secondary disk system can also be optimized based on this learning and have similar performance characteristics in the HyperSwap event. For more information, see 5.6, “Easy Tier” on page 86.
GDPS integrates support for HMT. The appropriate HMT actions (such as the starting and stopping of processing, and reversing transfer direction) are incorporated into the GDPS managed processes. For example, if MM is temporarily suspended by GDPS for a planned or unplanned secondary disk outage, HMT is also suspended. If MM direction is reversed as a result of a HyperSwap, the HMT direction is also reversed.
As of GDPS V3.12 and later, HMT is supported for all available GDPS options (2-, 3-, and 4-site environments).
2.4.2 GDPS and DS8000 synergy summary
GDPS is specifically designed for complex multi-site or single-site IBM Z environments. It provides the capability to manage disk remote copy, automate Parallel Sysplex operation tasks, and perform failure recovery from a single point of control in an easy and automated way. Continuous collaboration over many years between IBM Z, GDPS, and DS8000 development teams delivered a robust HA/DR design that is commonly used among IBM Z clients.
With its HyperSwap capability, GDPS is the ideal solution if you target 99.9999% availability.
Moreover, it also allows clients to run DR tests more frequently without affecting production. The more you practice your DR process, the more confident you become in recovering your systems and applications if a real disaster strikes.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.17.76.72