Migrating from Symantec Cluster Server powered by Veritas
This chapter presents a migration scenario followed by a cluster administration scenario.
The following topics are discussed in this chapter:
4.1 Terminology
The following subsections contain terminology and basic concepts to establish a link between the Symantec Cluster Server (VCS) and PowerHA for those who are not familiar with VCS terminology.
4.1.1 Cluster communication
Starting in IBM PowerHA 7.1, the Cluster Aware AIX (CAA) has been one of the key requirements to deploy a new cluster. CAA is an AIX operating system component that established common cluster capabilities for applications such as RSCT and PowerHA. PowerHA uses CAA for base cluster communication of events, heartbeating, and cluster-wide configuration management.
With CAA, the cluster communication for events and configuration is no longer limited to network availability. The use of shared disks across all nodes enhances failure detection and event notification among the nodes.
 
Note: CAA has been available since AIX 6.1 TL6 and 7.1 and is well described in IBM PowerHA SystemMirror Standard Edition 7.1.1 for AIX Update, SG24-8030, published in October 2012.
Unlike PowerHA, Symantec Cluster Server (VCS) is restricted to network communication over Ethernet interfaces for most of its cluster management and event notification, which is indeed a way to keep a common communication layer for multiple platforms.
Although this communication method may appear antiquated when compared to the CAA capabilities, it has components to make it reliable. First, VCS requires redundant private network for cluster-wide and cluster-to-nodes communication. Further, the Group Membership Services and Atomic Broadcasting (GAB) and Low Latency Transport (LLT) features provide membership control and high-performance communication across the private network to ensure resiliency.
LLT is a proprietary replacement for the IP stack, which provides low-latency high-performance communication. LLT features traffic distribution of communication packets across the available channels and is responsible for the cluster heartbeat. GAB stays on top of LLT using its communication features to control the node membership across the cluster.
 
Note: Detailed information about Symantec Cluster Server communication can be found in the documentation section of the Symantec Operations Readiness Tools (SORT) website:
http://sort.symantec.com
4.1.2 Seeding
Seeding is a protection feature of GAB to verify that all nodes configured in the cluster are available and joined the cluster properly. A node is required to be seeded before it has VCS services activated and be enabled to act as a member node.
4.1.3 Coordination points
In a Symantec Cluster Server, the coordination points are components that provide additional references to the cluster nodes when a problem occurs. The coordination points play an important role during fencing; however, although it is desired that a cluster has coordination points they are actually not required.
Coordination points can be deployed either with disks (Coordination Disks) or with servers (Coordination Point Servers or CP Servers). In the first method, VCS requires at least three disks in an exclusive diskgroup, for fencing. For a CP Server, Symantec suggests an exclusive server with VCS installed.
 
Note: Since Coordination Point Servers require an installation of the Symantec Cluster Server, additional licensing costs may imply in this configuration.
The role of the coordination points in VCS is similar to what is achieved when netmon.cf is configured in PowerHA.
4.1.4 Fencing
Fencing isolates a failed node from the cluster preventing it from acquiring resources and causing problems.
In PowerHA, fencing capabilities are provided by the CAA layer which has embedded network and disk communication capabilities.
The Symantec Cluster Server provides fencing in two different ways: disk-based and server-based. Disk-based and server-based fencing are achieved by using coordination points, which can be Coordinator Disks or Coordination Point Servers (CP Servers), respectively.
4.1.5 Resources and groups
In Symantec Cluster Server (VCS), the resources are configured through the hares command. There are different classes of resources known as resource types, each having a different set of attributes.
By default, resources in VCS have to be enabled to work in the cluster. In the event of problems after the configuration of a resource, it may be necessary to perform a probe operation. Probing a resource will test it for errors and activate it or point errors. The command hares can be used to probe the resources.
Once the resources are configured, they must be bound to a group with the hagrp command. Groups are collection of resources to be managed across the nodes of the cluster. Each group in VCS has a set of global attributes and another set of attributes tied to each node, which can indicate the status of that group in each of the nodes.
Resources dependency
In VCS, the dependency relationship between the resources is defined as parent and child in a way that the parent resource depends on the child resource. Child resources are started before parent resources.
 
Let us pick a DiskGroup and Mount resources called res_dg01 and res_mnt01 as an example. From a logical volume perspective, the diskgroup can be seen as the parent resource while the mount point would be a child resource. In this relationship, the child depends on the parent.
In VCS however, from a resource dependency perspective, that relationship is seen in the opposite way. The parent depends on the child resources. Therefore, res_mnt01 is considered the parent resource and depends on res_dg01, the child resource, to be made available first.
4.1.6 Storage foundation
The volume manager in Storage Foundation, formerly known as Veritas Volume Manager (VxVM) is basically built of physical and virtual objects.
Physical objects are mainly represented by the disks available to the operating system. Virtual objects are represented by VxVM disks, subdisks, plexes, volumes, and diskgroups, which we briefly describe below.
VxVM disks
In order to use a physical object or a disk in Storage Foundation, the disks must be captured or encapsulated as an VxVM Disk, which is virtual object. These objects now can be used to compose VxVM diskgroups and volumes.
In most cases VxVM disks will be a representation of an operating system disk, but Storage Foundation allows for other types of storage entities to be encapsulated as an VxVM Disk. For example, an LVM logical volume (LV) can be encapsulated as a VxVM Disk to be used in VxVM Volumes. This scenario can be seen in the online migration of LVM to VxVM.
Diskgroups
Diskgroups in Storage Foundation are composed of VxVM Disks and have a number of configurations and attributes that describe the configuration of its volumes, plexes, and subdisks. For comparison, diskgroups are similar to volume groups in LVM.
Volumes
The concept of VxVM volumes is similar to the logical volumes in LVM. However, while in LVM, LVs are composed of a set of physical partitions, VxVM Volumes are built of plexes, which are an entity that does not exist in LVM.
Plexes
Plexes are an additional layer between the volumes and subdisks in which the layout of the diskgroup is defined. Plexes can be arranged in different ways (stripped, mirrored, concatenated, and so on) and also provide a software layer for RAID layouts.
The plex layer plays an important role in the VxVM structure by adding flexibility to the product. Plexes allows layout changes. For example, a VxVM Volume initially created with a RAID 0+1 layout can be changed to RAID 1+0 without difficulties.
LVM does not have any kind of entity that corresponds to a VxVM Plex.
Subdisks
Subdisks are logical disk devices bound to a VxVM Disk. In most cases, a subdisk will be bound to a disk. There are specific situations where the subdisks may have different devices as their back-end devices. On “LVM to VxVM migration” on page 28, we describe the online migration from LVM to VxVM in which a logical volume is configured as a VxVM Disk and attached to a subdisk to configure a VxVM volume.
4.2 Introduction
In the following sections, we illustrate a process to migrate resources running under Symantec Cluster Server to PowerHA.
We have chosen to perform an in-place cluster migration on the same nodes running the current Symantec Cluster Server. We have decided to perform this type of migration to understand and demonstrate how much of the process can be accomplished without required downtime.
The current state of the virtualization technologies makes the creation of additional resources fairly easy. However, there are a number of other factors that will influence decisions when preparing a cluster migration.
Strict service management policies may represent a constraint on schedules when they define processes to add and retire servers in the IT environment. In such cases, utilizing available tools for migration can save a considerable number of labor hours.
Many different factors affect the time required to perform a full cluster migration. A significant one is if a large data migration is also required. We have split the migration procedure into different processes, so that the migration can be performed in different phases or maintenance windows:
•Cluster configuration migration only
This step is to migrate only the cluster configuration from Symantec Cluster Server to PowerHA, preserving the Volume Manager storage resources (diskgroups and file systems).
User-defined resources will be used in PowerHA to manage the non-LVM diskgroups and file systems. This generally would not be required because PowerHA does have integrated support for VxVM. However, at the time of writing it did not support the latest levels (6.1) we used during for our migration.
 
Note: The configuration step itself can be performed with the Symantec Cluster Server still active, without any downtime. However, the cluster must be fully stopped before any resource groups are started in PowerHA.
•Cluster configuration plus VxVM to LVM migration
On this step, we migrate the volume group data, migrating data from Volume Manager volumes to new LVM logical volumes.
During this process, the file systems type and data are fully preserved. This means that at the end of the process, LVM will contain logical volumes in the vxfs format.
At this time, we are able to allow PowerHA to manage the LVM volume groups. The vxfs file systems will still be managed as user-defined resources.
 
•Cluster configuration and file system data migration
This is the final step, where all data is migrated from one file system to another either by copy, or backup and restore processes.
 
Note: Unless data can be migrated by using snapshots, all applications are required to be down and file systems should not be updated during the copy of data.
4.3 Daily administration
Administration of both products are similar and require the same level of attention and care.
Administration of PowerHA can be performed through clmgr command-line interface (CLI), SMIT, or IBM Systems Director PowerHA plug-in. All task should be available through all three methods.
Symantec Cluster Server mostly utilizes a graphical interface that is freely available. Further VCS can be also managed by the Veritas Operation Manager (VOM) if available on the environment. Also administration can be performed through a command-line interface.
For PowerHA users preparing for a migration, it will be fairly easy to use VCS commands. A number of the commands used in VCS are illustrated throughout this chapter.
4.4 Cluster environment
Both the Symantec and IBM products offer a wide range of implementation options. Since our tests are intended to be a proof of concept, we have decided to keep the cluster as simple as possible.
The chosen configuration for the migration is a Symantec Cluster Server managing one Volume Manager disk group and one service IP address within the same service group. The cluster infrastructure is made of two private networks for the heartbeating and three VFC-attached disks for fencing.
This book will not cover the steps of creating a cluster since the procedure is well explained in other publications. Instead, we cover specific steps that are specific to this migration scenario.
 
Note: For detailed information about creating PowerHA clusters, refer to PowerHA SystemMirror for AIX Cookbook Update, SG24-7739-01.
4.5 Planning
Before starting the cluster migration, ensure that all resources required for the migration are available within your environment.
Most of the PowerHA cluster setup can be performed offline. This allows you to configure the cluster in advance and ultimately reduce the required outage to perform the transition from one product to another.
The migration of data is probably the most important and sensitive part of the transition. Especially when there is not a tool to perform the migration from VxFS back to JFS/JFS2.
Depending on the type of data to be migrated, different approaches may apply. For example, file systems, which hold mostly static files, can be replicated in advance while still online by using snapshots or tools like rsync and just resynchronized at the time of the transition. Database files, however, which are kept open for updates most of the time, should be migrated with the applications down. Raw devices can be migrated from VxVM to LVM, but also require that applications are taken down before the migration.
As mentioned in 4.2, “Introduction” on page 51, Symantec diskgroups and file systems can be managed by PowerHA by using user-defined resources. Although customized scripts are required to manage such resources, they play an important role during the migration of the cluster.
4.5.1 Network considerations
The existing network infrastructure used with Symantec Cluster Server can be used by PowerHA if an in-place migration is being performed. A similar network infrastructure will be required if PowerHA is being configured in different boxes.
In complex networks, obtaining new IP addresses for an entire new setup may not be wanted. This makes utilizing the existing one an easier choice.
However, if the new cluster is being deployed in a new infrastructure, make sure that all VLANs are properly configured and that any firewall rules required for the applications are properly configured.
4.5.2 Storage considerations
The PowerHA setup requires at least one additional disk drive shared between the nodes for the Cluster Aware AIX (CAA) repository disk and volume group.
For our test, we created a new 1 Gb Shared Storage Pool LUN and mapped to both nodes of the cluster.
The storage requirements for the migration will depend on the plans for data migration. While migrating only the cluster configuration does not add many storage requirements, copying or restoring data to fresh volume groups and file systems may require duplication of the current available disks.
 
 
Note: At the time of writing, the HMC interface to manage Shared Storage Pool resources does not allow mapping a LUN to multiple nodes. The mapping must be performed through the CLI in the VIOS.
4.5.3 Application resources
Application resources are used to manage the application availability on the cluster. They usually perform tasks like start and stop of applications, but can also perform other tasks like application monitoring.
In Symantec Cluster Server, a resource of the Application type is used to start, stop, and monitor applications. In PowerHA, the start and stop tasks are configured in Application Resources and the monitors are configured in the Application Monitors Resources.
Theoretically, the scripts used to manage application resources in the Symantec cluster can be migrated to PowerHA if they use the same convention for return codes and are not bound to the Storage Foundation suite.
4.6 Converting a Symantec Cluster Server to an IBM PowerHA cluster
In this section, we cover how to convert Symantec Cluster Server (VCS) to a PowerHA cluster. Our intention is to document it so that anyone who has not worked with VCS before will be able to perform these steps. It is always recommended to perform these steps in a test or pre-production environment before ever performing in production. Also, valid backups should also exist before beginning.
In our example, we make a copy of the Symantec Storage Foundation disk pool on LVM, so that we can take a copy of the data on VxVM and then migrate the system to PowerHA. This LVM disk will be first controlled by VCS until all data copies and migrations are completed or until you are ready to swap clusters. At which point, VCS can then be stopped and PowerHA started, taking ownership of the LVM disk and the related applications.
Overview of the following tasks:
2. Create LVM disk pool
 • Define linked cluster with sites
 • Configure volume groups
 • Create logical volumes
 • Create file systems
 • Add new LVM into VCS Service Groups
 • Test Failover
3. Install PowerHA
4. Add LVM into PowerHA
5. Test PowerHA
6. Migrate to PowerHA from VCS
7. Roll back and removal instructions
8. Remove VCS
4.7 Test environment overview
In our example, we have a local two-node cluster using a single IP network. Each systems’ rootvg is mapped over via shared storage pools (SSPs). The application data is utilizing NPIV-attached IBM DS3400 shared storage as shown in Figure 4-1 on page 55.
What is not shown is that our test environment utilizes dual VIO Servers within each of the IBM Power P750 machines. The boot volumes are presented up to the virtual machines (LPARs) through VIOS via SSP. There is not a technical requirement for this; however, we chose to do so because at the time of writing there was other existing documentation of such examples. The shared disk resources between the systems are mapped to each via NPIV. This is due to limitations with SCSI-3 reserves that the Volume Manager requires.
 
Note: Because the IBM AIX operating system does not support full SCSI-3 persistent reserve capabilities, SSP implements additional options. Details can be found in the IBM PowerVM Enhancements What is New in 2013, SG24-8198.
Figure 4-1 Test environment configuration
4.7.1 Environmental details
The VIOS and virtual machines software details are as follows:
•IBM AIX 7.1 TL3 SP3
•IBM PowerHA 7.1.3.1
•Symantec Cluster Server (VCS) 6.1
•Symantec Storage Foundation (VxSF) 6.1
•IBM VIO Server 2.2.3.2
4.7.2 Symantec Cluster Server configuration
This section describes the Symantec Cluster Server configuration.
Topology
The details of the main cluster topology configuration are shown in Figure 4-2 on page 56.
[josie112:root] / # lltstat -n
LLT node information:
Node State Links
0 peter111 OPEN 2
* 1 josie112 OPEN 2
[josie112:root] / # hares -display webip |grep ArgListValues
webip ArgListValues josie112 Device 1 en0 Address 1 192.168.100.110 NetMask 1 255.255.252.0 Options 1 "" RouteOptions 1 "" PrefixLen 1 0
webip ArgListValues peter111 Device 1 en0 Address 1 192.168.100.110 NetMask 1 255.255.252.0 Options 1 "" RouteOptions 1 "" PrefixLen 1 0
Figure 4-2 Test environment topology
The cluster is set up as a two-node (Peter111 and Josie112) cluster. There is one service IP (corben110) address.
The cluster resources and resource group defined are shown in Figure 4-3.
[josie112:root] / # hastatus -sum
 
-- SYSTEM STATE
-- System State Frozen
 
A josie112 RUNNING 0
A peter111 RUNNING 0
 
-- GROUP STATE
-- Group System Probed AutoDisabled State
 
B ClusterService josie112 Y N OFFLINE
B ClusterService peter111 Y N ONLINE
B corbenSG01 josie112 Y N ONLINE
B corbenSG01 peter111 Y N OFFLINE
B corbenSG02 josie112 Y N ONLINE
B corbenSG02 peter111 Y N OFFLINE
B vxfen josie112 Y N ONLINE
B vxfen peter111 Y N ONLINE
[josie112:root] / # hares -display |grep ONLINE
RES_phantom_vxfen State josie112 ONLINE
RES_phantom_vxfen State peter111 ONLINE
coordpoint State josie112 ONLINE
coordpoint State peter111 ONLINE
corbenDG01 State josie112 ONLINE
corbenSG01_mnt1 State josie112 ONLINE
corbenSG01_mnt2 State josie112 ONLINE
corbenSG01_mnt3 State josie112 ONLINE
csgnic State josie112 ONLINE
csgnic State peter111 ONLINE
nfsd State josie112 ONLINE
webip State peter111 ONLINE
Figure 4-3 Beginning cluster resource group configuration
The service group (corbenSG01) contains three VxFS striped mounts called /share/data<1/2/3>, then the second service group (corbenSG02) is an example of a simple test application that starts, stops, and monitors the NFS daemon.
4.7.3 Collecting Cluster specifications
Now that we have the basics of the Symantec Cluster Server, we need to obtain the related information in order to perform our migration. Now we need to get the information about the type of resources that we have, as shown in Example 4-1.
Example 4-1 Collecting the cluster resource specifications
[peter111:root] / # hares -display -group corbenSG01 -attribute Type
#Resource Attribute System Value
corbenDG01 Type global DiskGroup
corbenSG01_mnt1 Type global Mount
corbenSG01_mnt2 Type global Mount
corbenSG01_mnt3 Type global Mount
[peter111:root] / # hares -display -group corbenSG02 -attribute Type
#Resource Attribute System Value
nfsd Type global Application
From the resource list shown in Example 4-1, we can see that our two service groups corbenSG01 and corbenSG02 are type DiskGroup and Application, so we can customize our queries as appropriate. First the DiskGroup, which we select from the information as shown in Example 4-2.
Example 4-2 DiskGroup information
[peter111:root] / # hares -display -group corbenSG01 -attribute BlockDevice MountPoint FSType FsckOpt State
#Resource Attribute System Value
corbenDG01 State josie112 ONLINE
corbenDG01 State peter111 OFFLINE
 
corbenSG01_mnt1 State josie112 ONLINE
corbenSG01_mnt1 State peter111 OFFLINE
corbenSG01_mnt1 BlockDevice global /dev/vx/dsk/corbendg01/stripe01
corbenSG01_mnt1 FSType global vxfs
corbenSG01_mnt1 FsckOpt global %-n
corbenSG01_mnt1 MountPoint global /share/data1
 
corbenSG01_mnt2 State josie112 ONLINE
corbenSG01_mnt2 State peter111 OFFLINE
corbenSG01_mnt2 BlockDevice global /dev/vx/dsk/corbendg01/stripe02
corbenSG01_mnt2 FSType global vxfs
corbenSG01_mnt2 FsckOpt global %-n
corbenSG01_mnt2 MountPoint global /share/data2
 
corbenSG01_mnt3 State josie112 ONLINE
corbenSG01_mnt3 State peter111 OFFLINE
corbenSG01_mnt3 BlockDevice global /dev/vx/dsk/corbendg01/stripe03
corbenSG01_mnt3 FSType global vxfs
corbenSG01_mnt3 FsckOpt global %-n
corbenSG01_mnt3 MountPoint global /share/data3
Example 4-2 on page 57 shows the information that we need to create a VxFS and add it into the cluster. But also some of this is what we need to migrate to PowerHA LVM. However, the application information is slightly different as shown in Example 4-3.
Example 4-3 Application information details
[peter111:root] / # hares -display -group corbenSG02 -attribute MonitorProgram StartProgram StopProgram
#Resource Attribute System Value
nfsd MonitorProgram global /home/veritas/monitor/nfsd.monitor.sh
nfsd StartProgram global /usr/bin/startsrc -s nfsd
nfsd StopProgram global /usr/bin/stopsrc -s nfsd
Now that we have all the required information, we can advance to the next stage of migration.
4.8 Creating the LVM volume group and the file systems
In this section, we create the definitions needed in VCS and for our LVM volume group and journal file systems.
The steps we will be taking are as follows:
1. Create the volume group in LVM
2. Create the logical volumes
3. Create the file systems
4. Import the volume group across the cluster
Then, within the Symantec Cluster Server:
5. Define the new Service Group
6. Add the LVM volume group
7. Add the file system mounts
8. Link the resources together
9. Bring the file systems online
10. Test Failover
4.8.1 LVM creation
In this example, we cover the commands needed to create the LVM structure that we tested. This example gives a simple framework to start from:
[josie112:root] / # mkvg -V 52 -f -y corbenvg01 -s 256 -n -C hdisk7 hdisk8 corbenvg01
mkvg: This concurrent capable volume group must be varied on manually.
[josie112:root] / # varyonvg corbenvg01
[josie112:root] / # mklv -y lvstripe01 -t jfs2 -x 4096 -S1M corbenvg01 12 hdisk7 hdisk8
lvstripe01
Repeating the steps for lvstripe02 and lvstripe03 shows:
[josie112:root] / # crfs -v jfs2 -d lvstripe01 -m /data1_new -A no -a logname=INLINE -a logsize=10
File system created successfully.
3135188 kilobytes total disk space.
New File System size is 6291456
Repeat again for /data2_new.
Once completed, varyoff the VG and import it into the other nodes in your cluster as follows:
[josie112:root] / # varyoffvg corbenvg01
[peter111:root] / # importvg -V 52 -y corbenvg01 hdisk7
corbenvg01
0516-783 importvg: This imported volume group is concurrent capable.
Therefore, the volume group must be varied on manually.
[peter111:root] / #
The previous step imports all the mount points for you and gets them ready and fresh for the file systems. Remember that at the end, you need to leave the VG varied off and ready for the cluster to bring it online.
4.8.2 Add LVM to VCS
In this section, we cover all the commands that we need to create an LVM volume group within the Symantec Cluster Server. Though this can also be accomplished via the “Cluster Manager GUI”, not everyone may have access to it. So we show how to perform these tasks via the CLI.
1. First, we need to set the cluster to read/write:
# haconf -makerw
2. Then, add the Service Group, specifying the wanted name. In our case, it is corbenSG3lvm with our two nodes peter111 and josie112:
# hagrp -add corbenSG3lvm
VCS NOTICE V-16-1-10136 Group added; populating SystemList and setting the Parallel attribute recommended before adding resources
# hagrp -modify corbenSG3lvm SystemList peter111 0 josie112 1
# hagrp -modify corbenSG3lvm Parallel 0
3. Add the volume group. The resource is called corbenSG3vg, which is to be part of the service group corbenSG3lvm.
 
Note: In order to create this resource, you need to have the volume group major number. Without it, you will not be able to bring the service group online. The Enabled 1 parameter means this resource is enabled and ready to use, 0(Zero) is disabled, which cannot be bought online without enabling first.
# hares -add corbenSG3vg LVMVG corbenSG3lvm
VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
# hares -modify corbenSG3vg MajorNumber 52
# hares -modify corbenSG3vg ImportvgOpt n
# hares -modify corbenSG3vg VolumeGroup corbenvg01
# hares -modify corbenSG3vg Enabled 1
4. Then, we add the related mounts; you will need to ensure that you have the correct parameters, those being the mount point, device path, file system type, mount options, and FSCK option. Note the percent(%) is required before the parameter so that the minus(-) is added to the parameter and not passed as part of the whole command. The speech marks(“) are also required to ensure the ‘jfs2’ parameter is passed with the mount options:
# hares -add corbenSG3lvmMNT1 Mount corbenSG3lvm
VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
# hares -modify corbenSG3lvmMNT1 MountPoint /data1_new
# hares -modify corbenSG3lvmMNT1 BlockDevice /dev/lvstripe01
# hares -modify corbenSG3lvmMNT1 FSType jfs
# hares -modify corbenSG3lvmMNT1 MountOpt "%-V jfs2"
# hares -modify corbenSG3lvmMNT1 FsckOpt %-p
# hares -modify corbenSG3lvmMNT1 Enabled 1
In this example, we have three file systems. Therefore, these commands are repeated twice more for the mounts and devices.
5. Then link the resources together. This is the parent resource linking to the child resource (see “Resources dependency” on page 49). In our example, the mount is linked to the volume group:
# hares -link corbenSG3lvmMNT1 corbenSG3vg
6. Save the configuration and set the cluster back to read-only:
# haconf -dump -makero
7. Finally, bring the resources online on a node of your choice:
# hagrp -online corbenSG3lvm -sys peter111
Figure 4-4 shows an example of the new service group online.
Figure 4-4 New Service Group Online
We also need to test that all is working correctly with the resource group on our cluster. A good simple test is to move the resources to another node in the cluster:
# hagrp -switch corbenSG3lvm -to josie112
We can see an example of the move in Figure 4-5 on page 61.
Figure 4-5 VCS LVM file system move
We recommend to perform full failover testing of the cluster now that we have added this new resource. This will enable us to determine if there are any resource conflicts. Details about how to back out of your changes are detailed in 4.11, “Roll-back and removal” on page 72, if needed.
4.9 Installing the PowerHA software
In our scenario, PowerHA Standard Edition is to be installed on the two local nodes.
When installing it, you can choose to install all, or only those filesets specific to your environment. The IBM PowerHA Standard Edition requires the installation and acceptance of license agreements for the Standard Edition cluster.license fileset.
This cluster was configured utilizing standard SMIT sysmirror menus. There is also the option of utilizing the PowerHA Systems Director plug-in to configure, monitor, and manage the cluster. More information about the Systems Director plug-in option can be found in the IBM PowerHA SystemMirror 7.1.2 Enterprise Edition for AIX, SG24-8106.
To start creating the cluster, enter smitty sysmirror → Cluster Nodes and Networks → Standard Cluster Deployment → Setup a Cluster, Nodes and Networks.
Complete the options as wanted and press Enter. Upon execution, it will perform a discovery to gather both IP and shared disk information to be used in the cluster configuration.
The next step is to define a cluster repository disk and multicast address. We can use fast paths in SMIT to bypass additional menus. Execute smitty cm_setup_menu → Define Repository Disk and Cluster IP Address.
For the repository disk field, press F4 and get a pick list to choose the wanted disk. Our repository disk was made available between the nodes in the cluster with Shared Storage Pools, something that is now fully supported with PowerHA. This data is gathered during the discovery in the first step of creating the cluster. The available disks list is created by finding all shared disks, with PVIDs, not currently in a volume group. See the troubleshooting section in 4.13, “Troubleshooting, and known issues” on page 73 if you cannot find any disk or get errors.
Since this is going to be a PowerHA 7.1.3 cluster, we utilize unicast instead of the previously required multicast as the primary heartbeat mechanism. The Cluster IP address is not required to be entered. If you want one, you can either manually enter it or let PowerHA create one. It does this by taking the last three octets of the host name IP address from the node in which the cluster is being created on and replacing the first octet with 228.
After performing the previous two steps, it is recommended to synchronize the cluster. Execute smitty sysmirror → Cluster Nodes and Networks → Verify and Synchronize Cluster Configuration and press Enter twice. The main reason being is that the first time the cluster is synced, the CAA cluster is created automatically. This way if a problem is encountered, it can be addressed before adding all the additional cluster components. Figure 4-6 shows the disk and CAA volume group information after the synchronization and CAA cluster was created successfully.
Figure 4-6 CAA cluster created successfully
Though optional, and not used in this test configuration, it is considered a best practice to also configure SAN-based communications. This requires setting the appropriated FC adapter attributes on VIO servers, and adding virtual Ethernet adapters using VLAN 3358.
We now need to create our resources. This will consist of an application controller, service address, and shared volume group. They will then be added into a resource group.
In our scenario, we have no real application to utilize. So we created a dummy application controller by simply having it execute a banner command. We can add it by executing smitty sysmirror → Cluster Applications and Resources → Resources → Configure User Applications (Scripts and Monitors) → Application Controller Scripts.
In our case, we will be using the information we collected from 4.7.3, “Collecting Cluster specifications” on page 57.
Optionally, you can also add a monitor for your application, though it is not required for the cluster to start. This differs from VCS, which has a need for the monitor to configure the application resource.
Now we need to add a service IP address. To do so, execute the fast path of smitty cm_resources_menu → Configure Service IP Labels/Addresses → Add a Service IP Label/Address (choose “Network Name” from pop-up). In our cluster, we created it on Peter111 with host name address of 192.168.100.110, as this is the cluster address defined for VCS.
After adding the service IP, we can see it has been added to the cluster topology as shown from the cllsif output below in Figure 4-7.
Figure 4-7 cllsif output
Next, we create a resource group and add the resources to it. To create a new resource group, execute the fast path smitty cm_add_resource_group.
To add the resources to the resource group, execute the same fast path of smitty cm_resource_groups → Change/Show Resources and Attributes for a Resource Group and choose the previously created resource group. Then, for the fields of Service IP Labels/Addresses, Application Controllers, and Volume Groups, press F4 and a pop-up window appears with the ones previous created. Choose them, and press Enter.
If you need to add any additional storage, this can be accomplished by using the Cluster Single Point of Control facility (C-SPOC). Enter smitty cspoc → Storage → Volume Groups → Create a Volume Group (choose both nodes).
For your environment, repeat the previous steps as needed. Once completed, just to be sure, go ahead and synchronize the cluster.
We now have a two-node “hot-standby” cluster created consisting of the following:
•Two nodes, peter111 and josie112)
•One IP-network, net_ether_01)
•One repository disk, hdisk8
•One resource group, corben110rg, peter111 is primary, josie112 is backup.
•One application server, nfsd
•One service address, corben110, with IP of 192.168.100.110 via IP aliasing
•One shared VG, corbenvg01
The cluster configuration details can be seen in Figure 4-8.
Figure 4-8 cltopinfo output
We leave the cluster down ready for testing as PowerHA and VCS are now sharing resources. We have successfully had both instances working together ready to take control but this is not a supported or recommended configuration.
4.9.1 Testing the cluster
To execute the cluster test tool enter smitty hacmp_testtool_menu, then choose “Execute Automated Test Procedure” as shown in Figure 4-9.
Figure 4-9 Cluster test tool menu
When you press Enter, the final menu is displayed. The detailed results of each test are displayed in the SMIT window during execution (Example 4-4), and are also saved in /var/hacmp/log/cl_testtool.log.
Example 4-4 Output while testing the cluster
[peter111:root] / # cat /var/hacmp/log/cl_testtool.log|egrep "Complete|Completion"
|| Test 1 Complete - NODE_UP: Start cluster services on all available nodes
25/06/2014_07:17:49: || Test Completion Status: PASSED
|| Test 2 Complete - NODE_DOWN_GRACEFUL: Stop cluster services gracefully on a node
25/06/2014_07:18:35: || Test Completion Status: PASSED
|| Test 3 Complete - NODE_UP: Restart cluster services on the node that was stopped
25/06/2014_07:19:31: || Test Completion Status: PASSED
|| Test 4 Complete - NODE_DOWN_TAKEOVER: Stop cluster services with takeover on a node
25/06/2014_07:21:17: || Test Completion Status: PASSED
|| Test 5 Complete - NODE_UP: Restart cluster services on the node that was stopped
25/06/2014_07:23:44: || Test Completion Status: PASSED
|| Test 6 Complete - NODE_DOWN_FORCED: Stop cluster services forced on a node
25/06/2014_07:24:18: || Test Completion Status: PASSED
|| Test 7 Complete - NODE_UP: Restart cluster services on the node that was stopped
25/06/2014_07:25:55: || Test Completion Status: PASSED
## Cluster Testing Complete: Exit Code 0
|| Test 1 Complete - VG_DOWN: Bring down volume group
25/06/2014_07:27:28: || Test Completion Status: PASSED
## Cluster Testing Complete: Exit Code 0
|| Test 1 Complete - CLSTRMGR_KILL: Kill the cluster manager on a node
25/06/2014_07:29:50: || Test Completion Status: PASSED
## Cluster Testing Complete: Exit Code 0
The overall test time was 14 minutes and each of the following events were executed successfully:
1. NODE_UP - Each node one at a time.
2. NODE_DOWN_GRACEFUL – Same as above.
3. NODE_UP – Same as above.
4. NODE_DOWN_TAKEOVER – Graceful down and moves resource group from peter111 to josie112.
5. NODE_UP – Restart services on previously down node (peter111).
6. NODE_DOWN_FORCED – On peter111.
7. NODE_UP – Restart services on previously down node (peter111).
8. VG_DOWN – Simulates volume group loss (rg_move runs from josie112 to peter111).
9. CLSTRMGR_KILL – Creates hard fallover via halt on peter111.
While this testing does cover the core basic functionality of the cluster, additional granular level testing via the Custom Test Procedure is often wanted to include such common events as:
•FAIL_LABEL – (Both IP and Non-IP)
•NETWORK_DOWN_LOCAL – (Both IP and Non-IP)
•JOIN_LABEL – (Both IP and Non-IP)
•NETWORK_UP_LOCAL – (Both IP and Non-IP)
•SERVER_DOWN – (nice test when application monitoring is being used)
There are several specific events related to additional configuration options within PowerHA. This includes, but is not limited to, sites and global networks. Manually creating failures for testing is also encouraged, for example, disabling ports, pull cables, and so on.
4.10 Performing the migration
The migration of the servers is relatively simple once all the devices and cluster have been configured. When you can schedule downtime, you should be able to migrate the cluster services from VCS to PowerHA.
 
Note: We mentioned previously that this work can be done with the cluster live, but it is not supported. Without testing and knowing the application/environment involved, it is impossible for us and the related support teams to guarantee anything. All actions and commands performed are taken from well documented and supported processes. We have consolidated the actions together in order to perform our migrations. We can confirm in the scenarios mentioned in this IBM Redbooks publication that we were able to migrate a live cluster from VCS to PowerHA without any downtime or outages, but we cannot make this same guarantee in any other situation.
1. First, ensure all data has been synchronized over to the new file systems.
2. Stop VCS processes and disable the applications from auto starting on restart:
# haconf -makerw
# hagrp -disable corbenSG3lvm
# hagrp -disable corbenSG02
# hagrp -disable corbenSG01
# hagrp -disable vxfen
# hagrp -disable ClusterService
# hagrp -offline corbenSG3lvm -any
# hagrp -offline corbenSG02
# hagrp -offline corbenSG01
# hagrp -offline vxfen
# hagrp -offline -force ClusterService -any
# hagrp -disableresources vxfen
# hagrp -disableresources ClusterService
# haconf -dump -makero
In the previous steps, we are disabling the resources in order, then taking them in turn offline, and finally disabling and locking the vxfen and ClusterService so that they do not restart automatically on restart. Otherwise, VCS on a reboot/failover will grab cluster resources before or during PowerHA events.
3. Confirm the state of VCS:
[peter111:root] / # hastatus -sum
 
-- SYSTEM STATE
-- System State Frozen
 
A josie112 RUNNING 0
A peter111 RUNNING 0
 
-- GROUP STATE
-- Group System Probed AutoDisabled State
 
B ClusterService josie112 Y N OFFLINE
B ClusterService peter111 Y N OFFLINE
B corbenSG01 josie112 Y N OFFLINE
B corbenSG01 peter111 Y N OFFLINE
B corbenSG02 josie112 Y N OFFLINE
B corbenSG02 peter111 Y N OFFLINE
B corbenSG3lvm josie112 Y N OFFLINE
B corbenSG3lvm peter111 Y N OFFLINE
B vxfen josie112 Y N OFFLINE
B vxfen peter111 Y N OFFLINE
As you can see now everything is offline and ready to move to PowerHA.
4. Now we can start PowerHA knowing that the file systems and IP address are not in use elsewhere, as shown in Figure 4-10.
# smitty clstart
Figure 4-10 Start the cluster
5. Once this is complete, you can confirm the cluster state, first the cluster daemon:
[peter111:root] / # lssrc -ls clstrmgrES
Current state: ST_STABLE
sccsid = "@(#)36 1.135.1.118 src/43haes/usr/sbin/cluster/hacmprd/main.C,hacmp.pe,61haes_r713,1343A_hacmp713 10/21/"
build = "May 6 2014 15:08:06 1406D_hacmp713"
i_local_nodeid 1, i_local_siteid -1, my_handle 1
ml_idx[1]=1 ml_idx[2]=0
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 15
local node vrmf is 7131
cluster fix level is "1"
The following timer(s) are currently active:
monitor nfsd_mon:::RestartTimer
Current DNP values
DNP Values for NodeId - 2 NodeName - josie112
PgSpFree = 259592 PvPctBusy = 0 PctTotalTimeIdle = 95.663888
DNP Values for NodeId - 1 NodeName - peter111
PgSpFree = 259529 PvPctBusy = 0 PctTotalTimeIdle = 95.510347
CAA Cluster Capabilities
CAA Cluster services are active
There are 4 capabilities
Capability 0
id: 3 version: 1 flag: 1
Hostname Change capability is defined and globally available
Capability 1
id: 2 version: 1 flag: 1
Unicast capability is defined and globally available
Capability 2
id: 0 version: 1 flag: 1
IPV6 capability is defined and globally available
Capability 3
id: 1 version: 1 flag: 1
Site capability is defined and globally available
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was JOIN_NODE_CO on node 2
 – Then, the cluster IP:
[peter111:root] / # /usr/es/sbin/cluster/utilities/cllsif
Adapter Type Network Net Type Attribute Node IP Address Hardware Address Interface Name Global Name Netmask Alias for HB Prefix Length
josie112 boot net_ether_01 ether public josie112 192.168.100.112 en0 255.255.252.0 22
corben110 service net_ether_01 ether public josie112 192.168.100.110 255.255.252.0 22
peter111 boot net_ether_01 ether public peter111 192.168.100.111 en0 255.255.252.0 22
corben110 service net_ether_01 ether public peter111 192.168.100.110 255.255.252.0 22
 – Followed by the cluster topology:
[peter111:root] / # /usr/es/sbin/cluster/utilities/cltopinfo
Cluster Name: corben110
Cluster Type: Standard
Heartbeat Type: Unicast
Repository Disk: hdisk8 (00f6f5d0d0135ac2)
 
There are 2 node(s) and 1 network(s) defined
 
NODE josie112:
Network net_ether_01
corben110 192.168.100.110
josie112 192.168.100.112
 
NODE peter111:
Network net_ether_01
corben110 192.168.100.110
peter111 192.168.100.111
 
Resource Group corben110rg
Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The List
Fallback Policy Fallback To Higher Priority Node In The List
Participating Nodes peter111 josie112
Service IP Label corben110
 – And the local state of the cluster, confirming the mounts and IP address on the adapter:
[peter111:root] / # df -g|grep data
/dev/lvstripe01 3.00 2.99 1% 4 1% /data1_new
/dev/lvstripe02 3.00 2.99 1% 4 1% /data2_new
/dev/lvstripe03 3.00 2.99 1% 4 1% /data3_new
[peter111:root] / # ifconfig -a
en0: flags=1e084863,10480<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),CHAIN>
inet 192.168.100.110 netmask 0xfffffc00 broadcast 192.168.103.255
inet 192.168.100.111 netmask 0xfffffc00 broadcast 192.168.103.255
tcp_sendspace 262144 tcp_recvspace 262144 rfc1323 1
lo0: flags=e08084b,c0<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,LARGESEND,CHAIN>
inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255
inet6 ::1%1/0
tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1
6. Finally, we do a test failover to make sure our resources respond as expected, so on the current node holding the cluster resources:
# halt -q
7. Then, monitor the system failover. See Figure 4-11, Figure 4-12 on page 71, and Figure 4-13 on page 71.
Figure 4-11 Stable cluster - qha
Figure 4-12 Cluster failing over - qha
Figure 4-13 Cluster failover complete - qha
In the examples above, we used qha to monitor the cluster, which is a nice simple shell script that you can obtain from the following site:
Check to ensure that you have the latest version of qha. Version 9.01 works with PowerHA 7.1.3. Earlier versions work with PowerHA 6.1 and before as they use the clcomdES deamon, which is no longer part of PowerHA 7.1.3.
 
Note: In a production environment, we always recommend that you perform full cluster testing. This could involve the removal of disks, cables, failure of servers, and more. Details of examples of this testing can be found in the IBM PowerHA SystemMirror 7.1.2 Enterprise Edition for AIX, SG24-8106.
4.11 Roll-back and removal
If any issues are encountered with our new resource group, mounts, or volume groups, we may need to remove everything. This could be due to conflicts with other resources, applications, or file systems that we could not have foreseen. So in this section, we cover the commands needed to remove everything that we created in 4.6, “Converting a Symantec Cluster Server to an IBM PowerHA cluster” on page 54.
First, we take the resources offline across the cluster:
# hagrp -offline corbenSG3lvm -any
Ensure that we put the cluster into read/write mode, otherwise, our changes will not work:
# haconf -makerw
We then delete our LVM volume group details:
# hares -delete corbenSG1vg
Followed by deleting all the file system mounts:
# hares -delete corbenSG3lvmMNT1
# hares -delete corbenSG3lvmMNT2
# hares -delete corbenSG3lvmMNT3
Then, delete the service group that everything was owned by:
# hagrp -delete corbenSG3lvm
Finally, save and set the cluster back to read-only, otherwise, the changes will not take affect:
# haconf -dump -makero
Now the cluster is back to how it was set up before we added the details. We can then go and remove the related LVM file systems, logical volumes, and volume groups as needed.
4.12 Deleting the Symantec Cluster Server
Our last action will be the removal of the cluster. We can of course leave the service on until we decide it is time to remove it, so that we can ensure that everything is working first.
First, delete the Service Group and the related resources:
# haconf -makerw
# hares -delete <mounts>
Repeat as required:
#hares -delete <service_group>
Repeat as required, then delete the disk group:
# hares -delete RES_phantom_vxfen
# hares -delete coordpoint
# hares -delete vxfen
When that is deleted, you can then delete the cluster service:
# hares -delete csgnic
# hares -delete webip
# hares -delete ClusterService
Finally, save the config:
# haconf -dump -makero
Stop any remaining cluster services:
# hastop -all -force
Disable fencing:
# /sbin/vxfenconfig -U
Then, remove the packages in order based on their dependencies:
# installp -u VRTSamf VRTSaslapm VRTScps VRTSdbed VRTSfsadv VRTSfssdk VRTSgab VRTSob VRTSodm VRTSperl VRTSsfcpi61 VRTSsfmh VRTSspt VRTSvbs VRTSvcs VRTSvcsag VRTSvcsea VRTSvcswiz VRTSvxfen VRTSvxfs VRTSvxvm
# installp -u VRTSllt
# installp -u VRTSveki
# installp -u VRTSvlic
4.13 Troubleshooting, and known issues
As VxVM is the primary storage controller on these systems, when moving to LVM during these tests we noticed some technical issues. So in this section we cover the problems that we have seen.
4.13.1 Volume Manager holds the disk
After creating a new volume group for PowerHA on a system with an existing Symantec Cluster, we found that we could not find any disks to access, or it displayed the following errors when creating a volume group:
1800-050 Error exit status (1) returned by
Command_to_List; the output is:
"# No free disks found.
# Check each node to see if any disks need to have PVIDs allocated"
(The Command_to_List is:
"/usr/es/sbin/cluster/cspoc/cllspvids -O -n 'peter111' ".)
or
0516-1254 /usr/sbin/mkvg: Changing the PVID in the ODM.
0516-1397 /usr/sbin/mkvg: The physical volume hdisk6, will not be added to
the volume group.
0516-1254 /usr/sbin/mkvg: Changing the PVID in the ODM.
0516-1397 /usr/sbin/mkvg: The physical volume hdisk7, will not be added to
the volume group.
0516-862 /usr/sbin/mkvg: Unable to create volume group.
This is because the Symantec Storage Foundation is holding on to them:
[peter111:root] / # vxdisk -e list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
disk_0 auto:LVM - - LVM hdisk0 -
disk_1 auto:LVM - - LVM hdisk8 -
ds3400-0_0 auto:cdsdisk - - online hdisk4 -
ds3400-0_1 auto:cdsdisk - - online hdisk2 -
ds3400-0_2 auto:cdsdisk - - online hdisk5 -
ds3400-0_3 auto:cdsdisk - - online hdisk1 -
ds3400-0_4 auto:cdsdisk - - online hdisk3 -
ds3400-0_5 auto:none - - online invalid hdisk6 -
ds3400-0_6 auto:none - - online invalid hdisk7 -
We need to remove the disk from VxVM so that LVM is allowed access to the disk:
# vxdisk rm <disk>
The disk name can be either the hdisk name or the device name.
Make sure that we run this on all nodes in our cluster:
[peter111:root] / # vxdisk -e list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
disk_0 auto:LVM - - LVM hdisk0 -
disk_1 auto:LVM - - LVM hdisk8 -
ds3400-0_0 auto:cdsdisk - - online hdisk4 -
ds3400-0_1 auto:cdsdisk - - online hdisk2 -
ds3400-0_2 auto:cdsdisk - - online hdisk5 -
ds3400-0_3 auto:cdsdisk - - online hdisk1 -
ds3400-0_4 auto:cdsdisk - - online hdisk3 -
Now we have access again and we can create our volume group:
[peter111:root] / # mkvg -s 128 -y testvg hdisk6 hdisk7
0516-1254 mkvg: Changing the PVID in the ODM.
testvg
4.13.2 Volume group will not varyon with VCS after PowerHA test
After testing of the PowerHA cluster software, you may find that the Symantec Cluster Server is unable to start the LVM Service. This might be because of some locks placed on the new LVM volume group. These can be cleared by varying on and off the volume group as follows:
[josie112:root] / # varyonvg corbenvg01
0516-1972 varyonvg: The volume group is varied on in other node in concurrent
mode; you cannot vary on the volume group in non-concurrent mode. Use
-O flag to force varyon the volume group if needed.
[josie112:root] / # varyonvg -O corbenvg01
[josie112:root] / # varyoffvg corbenvg01
4.14 Shared storage pools
Throughout this IBM Redbooks publication, all the servers rootvg were built on disk using Shared Storage Pools (SSPs). A number of reasons were chosen for doing this:
1. IBM PowerHA supports SSP but it has not been used before.
2. Flexibility of the storage support enables us to quickly remove, move, add, and manage the disk locally without outside support.
3. SSP snapshots allowed us to roll back changes to the environment with minimal outage time. This enabled quick retesting of scenarios and recovery from issues.
While Shared Storage Pools are not supported by Symantec Cluster Server due to the SCSI3 reserve requirements on the resource disks, we would urge that in most scenarios the benefits outweigh this issue. Like us, you can put your operating system (AIX) boot disk on SSP along with the PowerHA cluster disk using NPIV for the Symantec Cluster Server.
The following are examples of the commands that we used for the creation, and recovery of the servers from the VIO servers. Listing your snapshots:
# snapshot -list -clustername <cl_name> -spname <pool_name>
Create a snapshot:
# snapshot -clustername <cl_name> -spname <pool_name> -create <file_name> -lu <node_name>
Rolling back to a previous snapshot (ensuring the virtual server is powered-down first):
# snapshot -clustername <cl_name> -spname <pool_name> -rollback <file_name> -lu <node_name>
4.15 Cluster migration
This section describes the cluster migration.
4.15.1 PowerHA and Storage Foundation
In PowerHA, third-party volume groups and file systems are treated as OEM and usually require specific configuration of additional management methods to make them manageable within a PowerHA resource group. Preinstalled methods for managing Symantec volume groups and file systems until Veritas Volume Manager version 4.0 are bundled into PowerHA. For other versions and products, a set of scripts must be created and configured in PowerHA.
As mentioned before, newer versions of Storage Foundation (above 4.0) are not supported in PowerHA. Although we found that statement in the official PowerHA documentation, we tried to use the embedded feature with our recent version in an attempt to check how much of the functionalities were still working.
While most of the scripts were found to work individually with minor or no changes at all, we had problems while running the verification and synchronization processes of PowerHA, which proved the official statements about the support and forced us to abort the trial.
In order to integrate the Symantec diskgroups and file systems with our PowerHA cluster, we had to choose using the user-defined resources features of PowerHA. The method used will be explained in the next sections.
 
Note: Typically when adding an LVM volume group to a resource group, the F4 key may be used to list the available volume groups in the system. Since our tests were performed with an unsupported version of Storage Foundation, we were not able to verify this functionality when using Storage Foundation diskgroups.
4.15.2 Configuring User-Defined Resources
As mentioned before, PowerHA does not support recent versions of VxVM volumes as OEM resources. As the test environment is running Storage Foundation version 6.1, one of our major concerns was related to how data could be managed in order to allow the migration of the cluster software.
In PowerHA 7.1, the Storage Foundation volumes and file systems can be configured as User-Defined Resources. These types of resources require that a set of scripts is created to accomplish the wanted tasks.
The SMIT menu to define the user-defined resources can be accessed through: smitty sysmirror → Custom Cluster Configuration → Resources → Configure User Defined Resources and Types. The menu listing can be seen in Example 4-5.
Example 4-5 SMIT - Configure User Defined Resources and Types
Configure User Defined Resources and Types
 
Move cursor to desired item and press Enter.
 
Configure User Defined Resource Types
Configure User Defined Resources
 
Import User Defined Resource Types and Resources Definition from XML file
 
F1=Help F2=Refresh F3=Cancel F8=Image
F9=Shell F10=Exit Enter=Do
The menu as shown in Example 4-5 has two different entries: Configure User Defined Resource Types and Configure User Defined Resources. The first entry will be used to define a new resource type handler. That is where you define all the scripts and parameters that will control the behavior of all resources associated to that type. The second entry is basically where you create a resource associated to the resource type previously defined.
The next sections illustrate how Storage Foundation resources can be enabled to work with PowerHA.
4.15.3 Enabling Storage Foundation Resources in PowerHA
Now that we talked about the User Defined Resources, we illustrate how to enable the Storage Foundation diskgroups and file systems in our PowerHA cluster.
Storage Foundation Diskgroups
In Example 4-6 on page 77, we create a new resource type named SFDG that performs all the activation and deactivation of the Storage Foundation diskgroups. The assistant requires four fields to be filled, all marked with an asterisk and shown in bold in the example.
First, the name of the new resource type is required. Then, the Processing Order, which defined the moment at which the resource will be processed during the startup or shutdown of the resource group. PowerHA provided a number of options for this field and we selected FIRST as our value because we want it to be processed when the resource acquisition starts.
Next, the startup and stop methods are required. These fields require the full path of the scripts created to handle the resources of this type. Since we are creating a resource to handle Storage Foundation diskgroups, we add two scripts that perform whatever tasks are necessary to start and stop the diskgroups.
Example 4-6 PowerHA - Adding a custom resource type for the diskgroups
Add a User Defined Resource Type
 
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Resource Type Name [SFDG]
* Processing order [FIRST] +
Verification Method          []
Verification Type [Script] +
* Start Method [/usr/es/sbin/cluster/OEM/MyResources/dgstart]
* Stop Method [/usr/es/sbin/cluster/OEM/MyResources/dgstop]
Monitor Method []
Cleanup Method []
Restart Method []
Failure Notification Method []
Required Attributes []
Optional Attributes []
Description []
 
 
F1=Help F2=Refresh F3=Cancel F4=List
F5=Reset F6=Command F7=Edit F8=Image
F9=Shell F10=Exit Enter=Do
As you can see, the menu allows the defining additional methods to perform other tasks. It is important that these parameters are used and set wisely. Of course, there are different aspects to be considered when deciding which methods will be used and skipped.
The scripts dgstart and dgstop are simple wrappers for the vxdg command to import and deport the diskgroups. The scripts must receive a parameter from PowerHA, which will be the name of the diskgroup. Therefore, it is important that the name of the diskgroups of your cluster are not hard-coded.
The contents of dgstart script are shown in Example 4-7, which illustrates how the scripts may look. The complexity of the scripts is dictated by how your systems are configured.
Example 4-7 dgstart script to import Storage Foundation diskgroups during resource group acquisition
#!/usr/bin/ksh93
# This script will import a given Storage Foundation diskgroup. PowerHA will call # this script during the activation of the resource group.
#
 
/usr/sbin/vxdg import ${1}
if [ $? -ne 0 ]; then
return 1
fi
 
return 0
With the new resource type created, we can now proceed and add a User Defined Resource to start our Storage Foundation diskgroups. To add the resource, use smitty sysmirror → Custom Cluster Configuration → Resources → Configure User Defined Resources → Add a User Defined Resource. Then, select the type of resource from the list, in this case, SFDG and press Enter.
The creation of a new resource is shown in Example 4-8. The Resource Name must be the exact name of the resource being handled. This name will be passed to the start and stop scripts as a parameter to be processed.
 
Hint: Custom naming conventions can be created to make it easier to identify the type of the resources according to their name, as long as the scripts are able to handle the name and process it properly during all cluster operation.
In Example 4-8, our resource is called testdg just like the real name of our diskgroup.
Example 4-8 PowerHA - Add a User Defined Resource
Add a User Defined Resource
 
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
 
[Entry Fields]
* Resource Type Name SFDG
* Resource Name [testdg]
Attribute data []
 
F1=Help F2=Refresh F3=Cancel F4=List
F5=Reset F6=Command F7=Edit F8=Image
F9=Shell F10=Exit Enter=Do
The configuration of the User Defined Resource is that simple. You can press Enter again and finish the creation of the resource.
 
Note: In this example, PowerHA will display a warning about the lack of a monitor method for the resource. For our demonstration, we will just ignore that message.
Storage Foundation file systems
The tasks to enable Storage Foundation file systems in PowerHA is very similar to the tasks we used for the diskgroups, except that this time our Processing Order parameter will be different.
The SMIT menu (Example 4-9 on page 79) to define the user-defined resources can be accessed through: smitty sysmirror → Custom Cluster Configuration → Resources → Configure User Defined Resources and Types.
Example 4-9 PowerHA - Adding a custom resource type for the file systems
Add a User Defined Resource Type
 
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* Resource Type Name [SFFS]
* Processing order [SFDG] +
Verification Method []
Verification Type [Script] +
* Start Method [/usr/es/sbin/cluster/OEM/MyResources/mount]
* Stop Method [/usr/es/sbin/cluster/OEM/MyResources/umount]
Monitor Method []
Cleanup Method []
Restart Method []
Failure Notification Method []
Required Attributes []
Optional Attributes []
Description []
 
 
F1=Help F2=Refresh F3=Cancel F4=List
F5=Reset F6=Command F7=Edit F8=Image
F9=Shell F10=Exit Enter=Do
Notice that the Processing Order parameter is now set to SFDG. This tells PowerHA that all resources of the SFFS type are to be processed after SFDG resources. In other words, the file systems will be mounted only after the diskgroups are made available.
Next, we create the resources to activate the file systems. The process is the same as illustrated in Example 4-8 on page 78, except that this time the type of the resource must be SFFS and the resource name must be the entire file system name, as defined in the /etc/filesystems file.
In our test, we created the SFFS resource /test/havol01. Now, we are going to add the resources to the PowerHA resource group hagrp01, as shown in Example 4-10. The resources cannot be added in the standard way that we usually do for native LVM volume groups and file systems. Instead, our new resources must be configured in the “User Defined Resources” field.
 
Note: The steps to create resource groups in PowerHA are not covered in this book. For more information, see PowerHA SystemMirror for AIX Cookbook Update, SG24-7739-01.
Example 4-10 Creating the resources
Change/Show All Resources and Attributes for a Resource Group
 
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[MORE...28] [Entry Fields]
Raw Disk UUIDs/hdisks [] +
Disk Error Management? no +
Fast Connect Services [] +
Primary Workload Manager Class [] +
Secondary Workload Manager Class [] +
Miscellaneous Data []
WPAR Name [] +
User Defined Resources [testdg /test/havol01] +
SVC PPRC Replicated Resources [] +
EMC SRDF(R) Replicated Resources [] +
DS8000 Global Mirror Replicated Resources +
[MORE...3]
F1=Help F2=Refresh F3=Cancel F4=List
F5=Reset F6=Command F7=Edit F8=Image
F9=Shell F10=Exit Enter=Do
With all the resources configured, the resource group can now be started, stopped, and moved from one node to another. The Storage Foundation resources are now under PowerHA management.
4.15.4 Mixed volume manager environment
Systems are not expected to always be static. Instead, data growth is expected as the business grows.
It is important to have a strategy in mind for the moment at which your data needs to grow. Although the ideal scenario would be to have all data migrated before adding new disks to the system or expanding file systems, eventually, the need for additional space may appear before we are able to migrate all data.
When the time comes to add more data to the system, the decision about how to make it will be important. If Storage Foundation diskgroups are still used on the system, they can be expanded. Alternatively, if the new data requirements allow, new LVM volume groups can be created along with JFS2 file systems.
No matter what choices are made when data grows, at some point, the system may end up with a mix of different resource types.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.16.50.60