Differences between IBM PowerHA SystemMirror Enterprise Edition version 6.1 and version 7.1.2
In this chapter we cover:
2.1 Architecture changes
Earlier versions of PowerHA (version 5.1 to version 6.1) used the reliable scalable clustering technologies (RSCT) topology services for heartbeat communications. With the introduction of PowerHA 7.1 and Cluster Aware AIX (CAA) we use Storage Interconnected Resource Collection (SIRCOL). SIRCOL is the equivalent of a local cluster or site in terms of PowerHA.
2.1.1 Cluster Aware AIX
PowerHA SystemMirror 7.1.2 uses the Cluster Aware AIX (CAA) services to configure, verify and monitor the cluster topology. This is a major reliability improvement because core functions of the cluster services such as topology related services, now run in the kernel space. This makes it much less susceptible to be affected by the workload generated in the user space.
Communication paths
This includes the important process of sending and processing the cluster heartbeats by each participant node. Cluster communication is achieved by communicating over multiple redundant paths. The following redundant paths provide a robust clustering foundation that is less prone to cluster partitioning:
•TCP/IP
PowerHA SystemMirror and Cluster Aware AIX, via multicast, uses all network interfaces that are available for cluster communication. All of these interfaces are discovered by default and used for health management and other cluster communication. You can use the PowerHA SystemMirror management interfaces to remove any interface that you do not want to be used by specifying these interfaces in a private network.
•SAN-based
A redundant high-speed path of communication is established between the hosts by using the storage area network (SAN) fabric that exists in any data center between the hosts. Discovery-based configuration reduces the burden for you to configure these links.
•Repository disk
Health and other cluster communication is also achieved through the central repository disk.
Of these three, only the SAN-based one is optional for a local site cluster. However, none of these communication paths can be used between two sites in a linked cluster. In this case, you have to use unicast communication between the sites. More information can be found in “Linked cluster and multiple site support” on page 19.
Repository disk
This is one of the core components of CAA cluster topology services containing vital information about cluster topology and resources. What is new specifically in PowerHA SystemMirror 7.1.2 Enterprise Edition is defining two respository disks, one for each site, when configuring a linked cluster. The repositories between sites are kept in sync internally by CAA. For more information, see 3.1.4, “Cluster repository disk” on page 47.
When sites sunder or split, and then merge, CAA provides a mechanism to reconcile the two repositories. This can be done either through rebooting all the nodes on the losing side or through APIs implemented exclusively for RSCT. More information on this topic can be found in 10.4.2, “Configuring the split and merge policy” on page 441.
New quorum rule
Although the cluster continues operating if one or more nodes lose access to the repository disk, the affected nodes are considered to be in degraded mode. If, in addition, the heartbeat communication is also affected, then there is the potential for the nodes to form an independent cluster (partition) by seeing other nodes register an abnormal failure.
Therefore, PowerHA SystemMirror 7.1.2 Enterprise Edition does not allow a node to operate if it no longer has access to the repository disk and also registers an abnormal node down event. This allows a double failure scenario to be tolerated.
Tie breaker disk
The tie breaker disk is a new feature that can be configured in a PowerHA SystemMirror 7.1.2 Enterprise Edition cluster. However, to utilize it, you must use any disk that supports SCSI-3 persistent reservation.
It is an optional feature you can use to prevent a partitioned cluster, also known as split brain. If specified as an “arbitrator” in the split and merge policy of a cluster, the tie breaker decides which partition of the cluster survives. The one containing a node that succeeds in placing a SCSI-3 persistent reserve on the tie breaker disk wins, and hence survives. The loser is rebooted.
Similar behavior happens while merging the partitioned cluster. The nodes belonging to the partition that is unable to place the SCSI-3 persistent reserve belong to the losing side and will be rebooted.
Refer to 10.2, “Methods to avoid cluster partitioning” on page 437 for a full explanation and examples of configuring split/merge options using a tie breaker disk.
2.1.2 ODM updates
The following ODM classes have been added to PowerHA 7.12 to support new features.
HACMPcluster
The following new stanzas have been added to the class HACMPcluster:
•remote_HB_factor
•link_timeout
•multi_site_lc
These are shown in Example 2-1.
Example 2-1 HACMPcluster new ODM stanzas
HACMPcluster:
id = 1595970743
name = "xdtest"
nodename = "hacmp21"
sec_level = "Standard"
sec_level_msg = ""
sec_encryption = ""
sec_persistent = ""
last_node_ids = "1, 2"
highest_node_id = 2
last_network_ids = ""
highest_network_id = 0
last_site_ids = ""
highest_site_id = 0
handle = 2
cluster_version = 14
reserved1 = 0
reserved2 = 0
wlm_subdir = ""
settling_time = 0
rg_distribution_policy = "node"
noautoverification = 0
clvernodename = ""
clverhour = 0
clverstartupoptions = 0
node_down_delay = 10
node_timeout = 20
remote_HB_factor = 0
link_timeout = 0
multi_site_lc = 0
•remote_HB_factor - Ratio of remote (unicast) heartbeats to local (multicast) heartbeat messages. The default is 10.
•link_timeout - Delay (milliseconds) added to the node_timeout + node_down_delay before marking the remote node down. The default is 30000.
•multi_site_lc - Definition to define new cluster types:
-1 = local cluster
 0 = stretched cluster
 1 = linked cluster
HACMPsircol
This ODM class has the addition of an extra repository disk stanza because linked clusters require two separate CAA repository disks (one stanza for local and stretched clusters and two stanzas for a linked cluster). Example 2-2 shows the file contents of a linked cluster.
Example 2-2 Linked cluster HACMPsircol
HACMPsircol:
name = "xdtest_sircol"
id = 0
uuid = "0"
ip_address = "228.1.10.10"
repository = "00ccfe7445cd455c"
repository = "00ccfe7445Rd421J"
backup_repository = ""
HACMPsite
No changes to this stanza.
2.2 New features and functionalities
The IBM PowerHA SystemMirror 7.12 Enterprise Edition introduces the following features:
2.2.1 Stretched and linked clusters
PowerHA 7.12 Enterprise Edition introduced two new cluster types: stretched and linked clusters.
Stretched cluster
The term stretched cluster denotes a cluster that has sites defined in the same geographic location, for example, a campus style cluster. The key aspect about a stretched cluster is that it uses a shared repository disk. This means, extended distance sites with IP only connectivity are not possible with this configuration. A stretched cluster can support cross-site LVM mirroring, HyperSwap, and Geographical Logical Volume Mirroring (GLVM). Refer to Chapter 8, “Migrating to PowerHA SystemMirror 7.1.2 Enterprise Edition” on page 329 for an example of a GLVM snapshot migration to a stretched cluster. Refer to Figure 2-1 for an illustration of a typical stretched cluster.
Figure 2-1 Stretched cluster example
A stretched cluster configuration can also be utilized with PowerHA 7.1.2 Standard Edition with the use of LVM Cross Site mirroring. See 5.1, “Cross-site LVM mirroring overview” on page 152. This is similar to what some clients have configured in previous versions.
The stretched cluster configuration is capable of utilizing all three levels of cluster communication provided by 2.1.1, “Cluster Aware AIX” on page 16.
Linked cluster and multiple site support
A linked cluster is expected to be the most common Enterprise Edition configuration (see Figure 2-2 on page 20). It allows the configuration of a traditional extended distance cluster between two sites, for example in London and New York. The key aspect of a linked cluster that makes it different from extended distance clusters in previous versions is the use of SIRCOL in CAA. This means that each site has its own CAA repository disk, which is replicated automatically between sites by CAA. Linked cluster sites communicate with each other using unicast and not multicast. However, local sites internally still use multicast. Multicast still needs to be enabled in the network at each site.
Figure 2-2 Linked cluster illustration
All interfaces are defined in this type of configuration as CAA “gateway addresses.” CAA maintains the repository information automatically across sites via unicast address communication. Refer to Table 2-1 for details about stretched and linked clusters.
Table 2-1 Stretched vs. linked cluster cross reference
Function
Stretched
Linked
Site communication
Multicast
Unicast
Repository disk
Shared
One local in each site
Cluster communication
- IP network
- SAN fabric
- Disk (repository)
IP network
Cross site logical volume mirroring
Available
Available
Geographical LVM mirroring (GLVM)
Available
Available
HyperSwap
Available
Available
Concurrent resource group with HyperSwap
Available
Not supported
SVC replicated resources
V7000 replicated resources
Available
Available
DS8K replicated resources
Available
Available
XIV replicated resources
Available
Available
EMC SRDF*
Not supported
Available
Hitachi TrueCopy/HUR*
HP Continuous Access
Not supported
Available
 
Restriction: *Though these third party storage replication options are supported with the IBM PowerHA SystemMirror 6.1 Enterprise Edition, they are not supported with the initial version 7.1.2 offering. IBM intends to remove this restriction in upcoming releases of the PowerHA SystemMirror software solution.
2.2.2 Split/merge handling overview
In the current version of PowerHA, there are exactly two options to choose from how the cluster will behave in the case that a partitioned or split condition occurs. PowerHA 7.1.2 allows us to configure this while we are defining the cluster.
Cluster split
The default behavior, called None, means that each partition becomes an independent cluster. This can be very dangerous because the resources will be activated in both partitions!
The other option is tie breaker. The partition that survives is the one that is able to perform a SCSI-3 persistent reserve on the tie breaker disk. All nodes in the losing partition will reboot.
Cluster merge
When nodes rejoin the cluster it is called a merge operation. What happens next depends on which split policy is chosen. You have to use Majority if the split policy was None, and Tie Breaker if the split policy was Tie Breaker.
For a detailed description of this topic see 10.4.2, “Configuring the split and merge policy” on page 441.
2.2.3 HyperSwap overview
HyperSwap is a function that provides continuous availability against storage errors. It is based upon storage-based synchronous replication. When directed (or upon disk errors), AIX hosts accessing the primary disk subsystem automatically switch over to the backup copy of the data.
HyperSwap technology enables PowerHA SystemMirror to support the following capabilities for you:
•Enable storage maintenance without any application downtime.
•Enable migration from old to new storage.
•No disruption to the dependent applications.
•Natural extension of disaster recovery configurations.
Thus, HyperSwap helps eliminate primary disk subsystems as the single point of failure to provide the next level of continuous operations support within metro distances.
For complete details and examples of exploiting HyperSwap, see Chapter 4, “Implementing DS8800 HyperSwap” on page 59.
2.2.4 IPv6 support
PowerHA has been supporting IPv6 in previous releases. The following is the history of PowerHA and IPv6 development:
•PowerHA 5.4.1 or before - No IPv6 support was available.
•PowerHA 5.5 - Added support for IPv6 services and persistent labels. Since the underlying topology service (RSCT) did not have full native IPv6 support, boot IP labels were delayed.
•PowerHA 6.1 - Full native IPv6 support with the following limitations:
 – Ethernet and IPAT via aliasing only
 – Cannot change prefix length or netmask of service label via dare
 – Smart Assists not supported
•PowerHA 7.1 and PowerHA 7.1.1 - Due to CAA implementation, IPv6 boot label supports were again dropped.
PowerHA 7.1.2 re-introduced support for IPv6. This feature is introduced with an update to CAA. The IPv6 support includes the following:
•Support communications of IPv6 packets through CAA.
•Multicast IPv6.
•Gathering and displaying of IPv6 statistics and addresses in use.
•Support for new cluster and migration from IPv4.
•All interfaces of TCP/IP sockets support the AF_INET6 socket family.
•Derivation of the IPv6 address takes place in user space.
•IPv6 multicast address is derived from the IPv4 multicast address and conversion is handled in the user space.
•Mping support added for IPv6.
•During discovery autoconf6 is no longer executed.
•Smart Assists support for IPv6 label.
Table 2-2 shows a quick overview of IPv6 capability for each version.
Table 2-2 PowerHA and IPv6 support overview
 
PowerHA 5.4.1 or before
PowerHA 5.5
PowerHA 6.1
PowerHA 7.1
PowerHA 7.1.1
PowerHA 7.1.2
IPv6 boot IP
Not supported
Not supported
Supported
Not supported
Supported
IPv6 service IP
or persistent IP
Not supported
Supported
Supported
Supported
Supported
For more information on implementing IPv6 with PowerHA, see “Configuring IBM PowerHA SystemMirror with IPv6” on page 466.
2.2.5 PowerHA Smart Assists
PowerHA SystemMirror 7.1.2 provides the following Smart Assists for application support:
•DB2 UDB non-DPF Smart Assist
•DHCP Smart Assist
•DNS Smart Assist
•Lotus® Domino® Smart Assist
•Filenet P8 Smart Assist
•IBM HTTP Server Smart Assist
•SAP MaxDB Smart Assist
•Oracle Database Smart Assist
•Oracle Application Server Smart Assist
•Print Subsystem Smart Assist
•SAP Smart Assist
•Tivoli Directory Server Smart Assist
•TSM Admin Smart Assist
•TSM Client Smart Assist
•TSM Server Smart Assist
•Websphere Smart Assist
•Websphere MQ Smart Assist
The Smart Assists are provided as part of the licensed product filesets. When you install PowerHA, you need to install the appropriate Smart Assist required to support your application.
 
Note: Review the Smart Assist documentation carefully for any application-specific requirements.
Table 2-3 Smart Assist application version support
 
SystemMirror 7.1.0
SystemMirror 7.1.2
DB2 Enterprise Edition
9.5
9.7
WAS
6.1
6.1
WAS N/D
6.1
6.1
HTTP Server
6.1
6.1
TSM
6.1
6.2
TDS
5.2
6.3
Filenet
4.5.1
4.5.1
Lotus Domino Server
8.5.1
8.5.1
Oracle Database Server
11gR1
11gR1
Oracle Application Server
10gR2
10gR2
SAP
 
 
 
 
 
- MaxDB
- Oracle
- DB2
SAP ERP Netweaver 2004s
 
 
 
 
 
N/A
10gR2
N/A
SAP SCM 7.0 with Netweaver EHP1 for FVT
 
SAP SCM 7.0 with Netweaver EHP2 for SVT
 
7.6
10gR2
9.7
MQSeries®
7.0.1.5
7.0.1.5
AIX Print Server
AIX 6.1
AIX 6.1
AIX DHCP
AIX 6.1
AIX 6.1
AIX DNS
AIX 6.1
AIX 6.1
2.2.6 IBM Systems Director
A new plug-in is provided for PowerHA management with IBM Systems Director 6.3. Enhancements include support for:
•Enterprise Edition
•Replicated storage
•Mirror pool support
•Volume group wizard
•Federated security wizard
•Capacity on demand
•Events management
•Reports management
PowerHA Enterprise Edition support
Support has now been added for PowerHA SystemMirror 7.12 Enterprise Edition. This includes the cluster configuration wizard allowing the defining of sites and additions to the cluster resource group management view.
Replicated storage support
A new wizard has been provided for replicated storage support called the Replicated Mirror Group Wizard. Support is provided for:
•DS8000® Series, Global Mirror and HyperSwap
•SAN Volume Controller (SVC) [7.1.2.1]
•XIV
Mirror pool support
Mirror pools are displayed for the volume group they reside in. There are a number of management options available, including the ability to create a new mirror pool.
Volume group wizard
This new wizard allows for the easy creation of a volume group.
Federated security wizard
This wizard is to provide end-to-end server security configuration such as LDAP server/client, encrypted file systems (EFS), user groups, users, and roles in a single wizard. Configuration of LDAP users, user groups and roles is also possible.
Capacity On Demand support
Support for adding a communication path to the HMC and creating Capacity On Demand object application controller has been added.
Event management
Accessible from the Events and Alerts Tab, this feature allows for the editing, viewing and creation of custom event actions. Users can customize commands, notifications, and recovery scripts of the event. The recovery counter can also be changed.
Reports management
This provides the ability to run and save reports through the Director plug-in. Saved reports are stored on the directory server. See Chapter 9, “PowerHA 7.1.2 for IBM Systems Director plug-in enhancements” on page 379 for more information.
2.2.7 New or changed cluster administration tools
This section covers command updates to CAA to support PowerHA SystemMirror 7.1.2 Enterprise Edition, mainly for site support.
mkcluster [-S sitename{ [cle_uuid=<UUID>:cle_globid=<id>:cle_prio=<prio>] } ]
Specifies the name of the local site. If not specified, a default site with the name local is created.
Currently, a cluster can support only two sites. To create a second site, use the chcluster command.
The following site information may be specified:
cle_uuid - The site UUID, which is honored as long as it is unique across the cluster. If not specified, the site UUID is automatically generated.
cle_globid - The short ID of site, which must be a unique unsigned number greater than zero. If not specified, the site short ID is automatically generated.
The following site attribute may be specified:
cle_prio - The priority of a site. A lower value indicates a higher priority. The priority is mainly used in the context of synchronizing the repository metadata.
chcluster [-S sitename{ [cle_uuid=<UUID>:cle_globid=<id>:cle_prio=<prio>] } ] This uses the same attributes as mkcluster.
-r + remote_reposdisk
lscluster -m has been enhanced to include site information.
-s enhanced to include remote pkts sent and remote pkts recv.
-d shows the disks known to each node along with the repository. Shared disks are site specific, shown by the disks discovered by each node.
clras dumprepos -v (undocumented) option added for verbose repos disk content.
mping Support for AF_INET6 (IPv6) socket family has been added.
2.2.8 Features at a glance added since PowerHA 6.1
Figure 2-3 on page 26 shows a list of features and the corresponding PowerHA levels in which they were added.
Figure 2-3 PowerHA version features list
2.3 Limitations
In this section we talk about limitations in PowerHA SystemMirror 7.1.2 Enterprise Edition. We also discuss the restrictions and unsupported changes in the product.
2.3.1 Deprecated features
Starting with PowerHA SystemMirror 7.1, the following features are no longer available:
•IP address takeover (IPAT) via IP replacement
•Locally administered address (LAA) for hardware MAC address takeover (HWAT)
•Heartbeat over IP aliases
•The following IP network types:
 – ATM
 – FDDI
 – Token ring
•The following point-to-point (non-IP) network types:
 – RS232
 – TMSCSI
 – TMSSA
 – Disk heartbeat (diskhb)
 – Multinode disk heartbeat (mndhb)
•Two-node configuration assistant
•WebSMIT (replaced with the IBM Systems Director plug-in)
Though PowerHA Enterprise Edition was never supported with WebSMIT, the 7.1.2 version is supported with the IBM Systems Director plug-in.
2.3.2 Restrictions
In this section, we list the restrictions in the 7.1.2 version at the time of writing this book. Some of these restrictions might change in the future. Users are advised to contact IBM sales and support to get the latest information. The restrictions are as follows:
•OEM storage features
•Network tunables
OEM storage replicated support
At the time of writing, PowerHA SystemMirror 7.1.2 Enterprise Edition does not support EMC SRDF and Hitachi TrueCopy. This restriction is expected to be removed in a future update.
Network tunables
Configure NIC not to use autonegotiate, but to run at the desired speed and duplex value. For the switches turn off the following options:
•Spanning tree algorithm
•portfast
•uplinkfast
•backbonefast
PowerHA SystemMirror 7.1 2 Enterprise Edition ignores network tunables.
2.3.3 Unsupported changes
Here we list some of the unsupported changes in PowerHA SystemMirror cluster:
•The hostname of a cluster node cannot be changed after the cluster is configured. To change the hostname, you must first remove the Cluster Aware AIX (CAA) cluster definition, update PowerHA SystemMirror and AIX operating system configurations, and then synchronize the changes to recreate the CAA cluster with the new hostname.
•You cannot change the IP address that corresponds to the hostname of a cluster node after the cluster is configured in Cluster Aware AIX (CAA).
•In PowerHA SystemMirror 7.1.2 Enterprise Edition, you cannot change linked cluster to local or stretched cluster. There are limitations on changing local and stretched cluster to a linked cluster.
2.4 Hardware and software prerequisites
For information about supported hardware and required AIX levels for PowerHA SystemMirror 7.1.2 Enterprise Edition, see 3.2, “Hardware and software requirements” on page 50.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.22.27.45