System Operations	Storage Administrator	Operations Administrator
Pool, volume, and snapshot management	X
Migration and replication	X
User and group management	X
Host definition and connectivity	X
Define iSCSI connectivity	X	X
Deploy and upgrade IBM Spectrum Accelerate system	X	X
Upgrade IBM Spectrum Accelerate system	X	X
Add new modules to system		X
Replacement for modules and hard disks		X
Resolving service flags		X

Note: The IBM Spectrum Accelerate update package is made available in a compressed file from IBM Passport Advantage. Within the compressed file is the actual upgrade package, which has an extension of tar.gz. Extract and place the upgrade package tar.gz file on the Windows workstation that is used to complete the upgrade.

Note: Ensure that the tar.gz upgrade package is selected and not the archive, which was downloaded from IBM Passport Advantage.

Note: Modules cannot be added to a system that uses a newer version of the IBM Spectrum Accelerate system software than what is already in use on the IBM Spectrum Accelerate system.

Important: The new VMware server must be configured to use the same vSwitch network names and be accessible on the same subnets as the IBM Spectrum Accelerate system.

Tip: The use of a deployment configuration XML file populates the configuration into the new module deployment utility and helps prevent errors.

Note: Before proceeding with the module deployment, export the configuration to preserve the new system configuration in case more modules are added in the future. Select Export Current Configuration from the General tab to save the deployment configuration XML file.

Note: The redistribution process is run in the background and completes after all the data on the system is redistributed to account for the new modules. The amount of time it takes to complete the redistribution process can be many hours and depends on the amount of data, number of modules and disks, and the interconnect bandwidth of the system.

Note: Modules cannot be added to a system if the modules have a newer IBM Spectrum Accelerate system software version than what is installed on the IBM Spectrum Accelerate system.

Note: The VMware ESXi server must be configured to use the same vSwitch network names. The server also must be accessible on the same subnets and virtual LAN networks as the IBM Spectrum Accelerate system.

Note: The VMware ESXi administrative user password can be changed after the system is successfully deployed. If more modules are added to the system or module maintenance must be completed, modify the deployment configuration XML file to reflect the new password.

Note: Example 9-2 also shows the deployment configuration XML file in the same folder as the deployment kit.

Tip: The deployment process outputs any errors that are encountered during the deployment to the Linux command-line. Review this output to help diagnose failures that can occur during the deployment process.

Note: Ensure that the correct deployment configuration XML file is used and that it contains only the stanza for the modules that are being added. The use of an incorrect deployment configuration XML file can cause problems on a system up to and including potential data loss.

Note: The redistribution process is run in the background and completes after all the data on the system is redistributed to account for the new modules. The amount of time it takes to complete the redistribution process can span many hours and depends on the amount of data, number of modules and disks, and the interconnect bandwidth of the system.

Note: Modules cannot be added to a system that uses a newer version of the IBM Spectrum Accelerate system software than what is already in use on the IBM Spectrum Accelerate system.

Important: The new VMware server must be configured to use the same vSwitch network names and be accessible on the same subnets as the IBM Spectrum Accelerate system.

Note: The interconnect subnet exchanges data only between modules of the same IBM Spectrum Accelerate system. Therefore, there is no need for these TCP/IP addresses to be routable outside of their assigned subnet.

Note: Take care when the Overwrite previous deployment option is used because it deletes any virtual machines that feature the same name as the virtual machines that are defined in the deployment configuration.

Note: The redistribution process is run in the background and completes after all the data on the system is redistributed to account for the new modules. The amount of time it takes to complete the redistribution process can be many hours and depends on the amount of data, number of modules and disks, and the interconnect bandwidth of the system.

Tip: If connected to the IBM Spectrum Accelerate system by using IBM XIV Management XCLI utility, run the component_list filter=notok command to verify component status. If any components are in a failed state, resolve the issue before moving forward.

Note: The system enters redistribution (see Figure 9-28 on page 257) when the data that is on the module is migrated to other modules. This configuration provides data redundancy during the concurrent repair process. The phase out process is run in the background and completes after all the data on the system is migrated to the other modules. The amount of time it takes to complete the phase out process can span many hours. This time depends on the amount of data, number of modules and disks, and the interconnect network bandwidth that is allocated to the system.

Tip: The module is the only one that is marked as Failed.

Note: The phase in process is run in the background and completes after all the data on the system is redistributed across the entire system. The amount of time it takes to complete the phase in process can be many hours, depending on the amount of data, number of modules and disks, and the interconnect bandwidth that is allocated to the system.

Note: The arguments for this command require the after= value that specifies a start time for the event search. The after= value specifies current date.hour minus 8,16, 24, or 32 days. Removal of a failed disk that has the good copy of data when a functioning disk does not prevent proper recovery and must be avoided.

Tip: The correct argument to specify a disk for the disk_list command is 1:Disk:<Module#>:<Disk#>. These values can be found by using the IBM XIV Management GUI.

Note: In addition to the HDDs that are labeled “Mapped Raw Lun”, there are HDDs that are labeled “Virtual Disk.” These HDDs are used to support IBM Spectrum Accelerate system, but are not used by the IBM Spectrum Accelerate system to store data.

Important: Make note of the Virtual Device Node of the failed disk and record it for use when the replacement disk is added to the virtual machine. As shown in Figure 9-34, the Virtual Device Node is 0:0.

Tip: For more information about HDD location recording and labeling, see “Important considerations” on page 19.

Note: Some disk controllers increment the Identifier (WWN) so that the last digits do not match the logically reported disk Identifier and what is physically printed on the disk, as shown in Figure 9-36.

Important: Because VMware ESXi servers add devices that are based on controller and port number, it is important that replacement disks are in the same physical location as the disk that is being replaced.

Note: Depending upon the hardware disk controller, the WWN can be altered or replaced with a WWN that is issued from the controller.

Note: The RDM mapping file name must use the following form to comply with the IBM Spectrum Accelerate system: /vmfs/volumes/MOD1_ds/data_rdm_disk_paths/DISK_XX_RDM.vmdk When the physical device is specified in the command, the disk device node can be used instead of the Physical LUN device if the Physical LUN device is unknown.

Note: If the new disk shows functioning=yes, the disk is ready to be component tested and phased into the IBM Spectrum Accelerate system. If the new disk shows functioning=no, verify the replacement steps, re-create the RDM mapping file, and check again. If the disk is still listed as functioning=no after completing the checks, contact IBM Technical Support for assistance.

Note: The phase in process is run in the background and completes after all the data on the system is redistributed to account for the new disk. The amount of time it takes to complete the phase in process can span many hours, depending on the amount of data, number of modules and disks, and the interconnect bandwidth of the system.

Note: The arguments for this command require the after= value to specify a start time for the event search. The after= value specifies current date.hour minus 8,16, 24, or 32 days. Removal of a failed disk that has the undamaged partition when the functioning disk does not prevent proper recovery and must be avoided.

Tip: The correct argument to specify a disk for the disk_list command is 1:Disk:<Module#>:<Disk#>. These values can be found by using the IBM XIV Management GUI or running the component_list filter=notok IBM Spectrum Accelerate XCLI command.

Note: The vim-cmd vmsvc/device.getdevices $VMID command gathers all of the basic information for the hardware components in the server. It also displays information about how they are mapped to the virtual machines.

Important: Record for future reference the Virtual Device Node of the failed disk. The Virtual Device Node is used mapping the replacement HDD to the virtual machine.

Note: Use the VMID of the virtual machine and the controllerKey and unitNumber as the identifying arguments to remove the HDD from the virtual machine.

Tip: It is suggested that you connect to the system to ensure that no other HDDs were marked as failed after running the disk removal.

Note: The disk number matches the drive number that is marked failed by IBM Spectrum Accelerate system.

Tip: Run the ls -l /vmfs/volumes/{data store}/data_rdm_disk_paths command to ensure that data store mapping file was removed.

Note: Some disk controllers logically increment the Identifier (WWN) so that the last digits no longer matches what is physically printed on the disk. Figure 9-41 shows an example of an incremented reported Identifier compared to the WWN number that is physically printed on the HDD.

Important: Because VMware adds devices that are based on controller and port number, it is important that replacement disks are installed in the same physical location as the disk that is being replaced.

Note: Some hardware disk controllers can alter or even replace the disk Identifier (WWN) with a different WWN that is issued by the disk controller.

Note: The RDM mapping file must use the following naming convention be compliant with the IBM Spectrum Accelerate system: /vmfs/volumes/MOD1_ds/data_rdm_disk_paths/DISK_XX_RDM.vmdk When the physical device is specified in the command, the disk device node can be used instead of the Physical LUN device if the Physical LUN device is unknown.

Note: If the system is in a rebuild or redistribution process, do not proceed with physical removal of the SSD until the system is in a Full Redundancy status.

Tip: The SSD information can be obtained from within the IBM XIV Management GUI.

Note: The user is presented with several devices, including many HDDs. In addition to the HDDs that are labeled “Mapped Raw Lun,” there are more disks that are labeled “Virtual Disk” that are used to support IBM Spectrum Accelerate but are not used for storing data.

Note: If there are multiple data stores that are mounted to the VMware ESXi host, ensure that the correct one is used to create the RDM mapping file for the SSD drive.

Note: The SSD device node can be used within the command directly instead of the Physical LUN device if the Physical LUN device is unknown.

Note: This value often must be changed from SCSI 0:# to SCSI 1:0.

Note: Unless the SSD is set to use a separate Virtual Device Node SCSI controller (In this case, SCSI 1:0), the IBM Spectrum Accelerate system cannot find the SSD during the testing phase.

Tip: The ssd information can be obtained from within the IBM XIV Management GUI.

Note: The vim-cmd vmsvc/device.getdevices [VMID#] command gathers all of the basic hardware information of the hardware components within the server and how they are mapped to the virtual machines.

Important: If there are multiple SSDs, this method does not help identify the correct SSD. In this case, use the VMware vSphere client to correctly identify the SSD.

Note: Example 9-27 shows a controllerKey value of 1001 (Controller 1) and a unitNumber value of 0 (Slot 1 on Controller 1).

Note: Use the VMID of the virtual machine and the controllerKey and unitNumber as the identifying arguments to remove the SSD from the virtual machine.

Tip: It is suggested to connect to the system to ensure that no other SSDs are marked as failed.

Note: If there are multiple data stores that are mounted to the VMware ESXi server, ensure that the correct data store is used to create the RDM mapping file for the SSD drive.

Note: The SSD device node can be used within the command directly instead of the Physical LUN device if the Physical LUN device is unknown.

Note: If Proactive Support is configured on the IBM Spectrum Accelerate system, IBM Technical support automatically is updated with the events and can notify the primary contact in these types of instances.

Note: Without interconnect connections, a module cannot send or receive data to or from any other module and is of no use to the system.

Note: During initialization, the module undergoes various tests to validate the module configuration and test that the module components are functioning and ready to be used. The initialization process can take a long time to complete as a battery of diagnostic tests are completed on the module being tested.

Note: The phase in process is run in the background (as shown in Figure 9-55 on page 295) and completes after all the data on the system is redistributed to account for the new disk. The amount of time it takes to complete the phase in process can span many hours, depending on the amount of data, number of modules and disks, and the interconnect bandwidth of the system.

Tip: System recovery actions can be completed only by an IBM Spectrum Accelerate system user who is a member of the Operations Administrator user category. Ensure that there is access to the IBM XIV Management GUI with a user of this category.

Note: The initialization process can take a long time to complete as many diagnostic tests are run on the component that is being tested.

Note: The phase in process is run in the background and completes after all the data on the system is redistributed to account for the new disk (see Figure 9-70). The amount of time it takes to complete the phase in process can span many hours, depending on the amount of data, number and of disks, and the interconnect bandwidth of the system.

Note: The arguments for this command require the after= value to specify a start time for the event search. The after= value specifies current date.hour minus 8,16, 24, or 32 days. Removing a failed disk that includes the undamaged partition when the functioning disk does not include the partition prevents proper recovery and must be avoided.

Note: The reported times are estimations only and might increase or decrease, depending on the workload of the system. Example 9-33 shows that 24% of the capacity to redistribute is done and the remaining time is 54 seconds; however, the process can take longer.

Important: Example 9-34 shows the <server> stanza for Module:1 with the option module_id="1" to define the deployment as that specific module.

Note: If a deployment configuration XML file that was exported from the IBM XIV Management GUI during system deployment is used, make sure to add the correct VMware ESXi server user name and password to the server definition. To ensure security of the VMware ESXi servers, the VMware ESXi administrative passwords are not retained in the exported deployment configuration XML files. If the password is not added, the deployment fails with the following error messages: Verifying SSH credentials and connection *** Node Verification Error ***

Note: Allow the module to complete the testing process and have a state of Ready. This process can take time because of the various tests that are conducted on the module.

Note: The phase in process is run in the background and completes after all the data on the system is redistributed to account for the new disk. The amount of time it takes to complete the phase in process can span many hours, depending on the amount of data, number of modules and disks, and the interconnect bandwidth of the system.

Note: The arguments for this command require the after= value to specify a start time for the event search. The after= value specifies current date.hour minus 8,16, 24, or 32 days. Removing a failed disk that includes the undamaged data when the functioning disk does not include the data prevents proper recovery and must be avoided.

Note: The reported times are estimations only and might increase or decrease, depending on the workload of the system. Example 9-39 shows that 24% of the capacity to redistribute is done and the remaining time is 54 seconds, but the process can take longer.

Note: The disk that holds the old data store is not inserted at this stage to ensure a clean installation of the VMware ESXi to a new disk. The disk with the old data store was inserted into its slot after customization was finished and before the VMware ESXi was restarted.

Note: If the network device was not active, the Speed column shows Down as can be seen for vmnic0, vmnic1, and vmnic2.

Note: If a deployment configuration XML file that was exported from the IBM XIV Management GUI during system deployment is used, make sure to add the correct VMware ESXi server user name and password to the server definition. To ensure security of the VMware ESXi servers, VMware ESXi administrative passwords are not retained in exported deployment configuration XML files. If the password is not added, the deployment fails with the error messages: Verifying SSH credentials and connection *** Node Verification Error ***

Note: According to the last message lines, the deployment kit assigned a new non-unique serial number and unique serial number (UUID) to the system. This message can be disregarded as the cluster assigns the original system serial and UUID number to the module.

Note: Allow the module to complete the testing process and change into a state of Ready. This process can take time because of the various tests that are conducted on the module.

Note: The phase in process is run in the background and completes after all the data on the system is redistributed to account for the new disk. The amount of time it takes to complete the phase in process can span many hours, depending on the amount of data, number of and disks, and the interconnect bandwidth of the system.

Note: The Export option is now available. It can be used to export the current settings to a deployment configuration XML file, as shown in Figure 9-120.

Note: The duration of the deployment process depends on several factors, including bandwidth from the deployment workstation to the VMware ESXi server, the number of modules that are being deployed, and the resources that are available on the VMware ESXi servers.

Note: Equipping a module is necessary only if a new module is added to the system. Module replacement does not include adding modules.

Note: Allow the module to complete the testing process and have a state of Ready. This process can take time because of the battery of tests that are being conducted on the module.

Note: The phase in process is run in the background and completes after all the data on the system is redistributed to account for the new disk. The amount of time it takes to complete the phase in process can span many hours, depending on the amount of data, number of modules and disks, and the interconnect bandwidth of the system.