Guidelines: Port utilization in an IBM FlashSystem V9000 scalable environment
To maximize the full potential of performance and low latency of IBM FlashSystem V9000 with IBM FlashCore technology in a scalable environment, several important items must be considered in configuring your environment. For example, host, FlashSystem storage, additional storage, intra-cluster, and optional remote copy ports must be properly designated and zoned in order to optimize performance and properly isolate the types of Fibre Channel traffic.
This appendix covers the following topics:
A.1 Overview
|
Note: When an FlashSystem V9000 with AE2 storage enclosure is upgraded to 8.1.x or higher, AE2 will remain configured as internal storage, while AE3 storage enclosure are mapped to the V9000 control enclosure as external storage. This section will use the term FlashSystem storage enclosure when there is no difference between AE2 and AE3 storage enclosure from a configuration point of view.
|
IBM FlashSystem V9000 provides a flexible architecture for assigning port resources. Two primary methods of port utilization that can be implemented in a Fibre Channel environment are suggested depending on your needs. This appendix compares and provides guidelines for these two methods:
•IBM FlashSystem V9000 port utilization for infrastructure savings
This method reduces the number of required customer Fibre Channel ports that are attached to the customer fabrics. This method provides high performance and low latency but performance might be port-limited for certain configurations. Intra-cluster communication and FlashSystem storage enclosure traffic occur over the internal switches.
•IBM FlashSystem V9000 port utilization for performance
This method uses more customer switch ports to improve performance for certain configurations. Only ports that are designated for intra-cluster communication are attached to private internal switches. The private internal switches are optional and all ports can be attached to customer switches.
Random workloads can also experience performance benefits with the performance method because more host ports are available when compared with the infrastructure savings method. This benefit is more pronounced for a IBM FlashSystem V9000 solution that includes dedicated ports for remote copy.
|
Note: AC3 controller can have up to four 4-port 16 Gb cards. This appendix focuses on the three card configuration.
|
A.2 Comparison of port utilization methods
The infrastructure savings method has dedicated internal switches for the IBM FlashSystem V9000 storage enclosures connections and also intra-cluster communication with a reduced number of customer host facing ports. Eight ports per I/O group are available for storage traffic to the FlashSystem storage enclosures.
The performance method uses the customer fabric for all connections (with the option to use dedicated internal switches for intra-cluster communication). The ports have designated purposes based on fabric attachment, zone assignments, and port masking. This method provides shared-use ports that use the full bidirectional capabilities of Fibre Channel along with the V9000 excellent load-balancing capabilities across all available ports. Up to sixteen ports per I/O group are available for storage traffic to FlashSystem storage enclosures.
Both methods can designate host ports for remote copy and mirroring. The performance method is more efficient when ports are designated to remote copy.
Both methods support attachment to additional external storage. In both cases, zones in the customer fabric are required for attaching external storage.
The infrastructure savings method requires up to four types of zones:
•Open zoning
•Host zones
•Optional remote copy zones
•External storage zones
The performance method requires up to four types of zones; host zones, storage zones (internal and external), intra-cluster zones, with optional remote copy zones.
The infrastructure savings method has specific port cabling suggestions for the FlashSystem storage enclosures and AC3 control enclosures that are calculated to support connections and non disruptive scalability for up to eight AC3 controller enclosure and eight FlashSystem storage enclosures (four building blocks plus four storage enclosures). For details, see A.4, “Guidelines: The infrastructure savings method” on page 249.
The performance method requires planning to ensure non disruptive scalability. For details, see A.3, “Guidelines: The performance method” on page 247.
A.3 Guidelines: The performance method
The performance method for port utilization provides shared use of ports that use the full bidirectional capabilities of Fibre Channel.
Table A-1 on page 248 lists the connections to the two AC3 control enclosures, and host and storage connections to switches that are external to the building block. The building blocks connect to two customer fabrics external to the building block.
If you are doing remote copy (Metro Mirror or Global Mirror), then some ports designated for host and storage can be used for remote copy traffic. Another possibility is to get a fourth 16 Gb card and use some of those ports for remote copy. Depending on the remote copy bandwidth, you might use only one port per node.
Table A-1 provides an example of port utilization that designates ports for intra-cluster traffic, ports for host and storage traffic, and optional ports for remote copy.
|
Note: The performance method requires customer planning to ensure nondisruptive scalability.
|
Table A-1 Port connections in scalable environment with port utilization for performance
|
|
12 x 16 Gb per AC3
|
|||
|
Source port on AC3 (slot:port)
|
#
|
Type
|
SAN
|
Usage
|
|
3:1
|
1
|
16 Gb
|
C (S1)
|
Intra-cluster
|
|
3:2
|
2
|
16 Gb
|
D (S2)
|
Intra-cluster
|
|
3:3
|
3
|
16 Gb
|
C
|
Host/Storage (or remote copy)
|
|
3:4
|
4
|
16 Gb
|
D
|
Host/Storage
|
|
4:1
|
5
|
16 Gb
|
C (S1)
|
Intra-cluster
|
|
4:2
|
6
|
16 Gb
|
D (S2)
|
Intra-cluster
|
|
4:3
|
7
|
16 Gb
|
C
|
Host/Storage
|
|
4:4
|
8
|
16 Gb
|
D
|
Host/Storage (or remote copy)
|
|
7:1
|
9
|
16 Gb
|
C
|
Host/Storage
|
|
7:2
|
10
|
16 Gb
|
D
|
Host/Storage
|
|
7:3
|
11
|
16 Gb
|
C
|
Host/Storage
|
|
7:4
|
12
|
16 Gb
|
D
|
Host/Storage
|
|
localfcportmask
|
000000110011
|
|||
|
partnerfcportmask
|
000010000100
|
|||
|
|
|
|
|
|
|
SAN - C / D customer fabric
SAN - Optional S1 / S2 internal switches
|
32 ports per building block connect to customer fabric: 12 ports per AC3 and 8 ports per FlashSystem storage enclosures.
|
|||
|
Note: When comparing the port utilization for infrastructure savings in Table A-2 on page 250 with the port utilization for performance listing in Table A-1 on page 248, you see that the customer fabric port connection mapping and usage is much higher in the port utilization for performance method. This method provides shared-use ports that take advantage of the full bidirectional capabilities in Fibre Channel, resulting in higher performance.
|
Port mask
With Fibre Channel port masking, you can control whether the ports are used to communicate to other local nodes and if they are used to communicate to nodes in remote, partnered systems. Fibre Channel port masking does not affect host or storage traffic.
The system has two Fibre Channel port masks. The local port mask controls connectivity to other nodes in the same system, and the partner port mask control connectivity to nodes in partnered systems. By default, all ports are enabled for both local and partner connectivity. The port masks apply to all nodes in the same local system.
For example, if the local port mask is set to 101101 on a node with eight Fibre Channel ports, ports 1, 3, 4 and 6 are able to connect with other local nodes in the same system. Ports 2, 5, 7 and 8 are not used for connection to other local nodes.
Table A-1 on page 248 also lists the port masks (localfcportmask and partnerfcportmask) to use with these port designations.
Figure A-1 shows an example from the IBM FlashSystem V9000 GUI that displays Fibre Channel port masks (localfcportmask and partnerfcportmask). This information is displayed when you select Settings → Network → Fibre Channel Ports → Actions → Modify Connection → View details.
Deviations from the port assignments shown in Table A-1 on page 248 require corresponding changes to the port masks. Proper zoning is still required.
Figure A-1 Modify Fibre Channel port mask example
A.4 Guidelines: The infrastructure savings method
The infrastructure savings method for port utilization provides a dedicated port setup. The ports are used either for host, storage, or intra-cluster communication. This method has dedicated internal switches for the IBM FlashSystem storage enclosures connections and also intra-cluster communication with a reduced number of customer host facing ports.
Table A-2 shows the connections to the two AC3 control enclosures, including the cluster connections to the switch that is internal to the building block. Host, remote copy, and external storage connections to switches that are external to the building block are shown.
The building blocks connect to two SAN fabrics within the building block, and two customer fabrics external to the building block.
Table A-2 Port connections in scalable environment for minimum infrastructure effect
|
|
12 x 16 Gb per AC3
|
|||
|
Source port on AC3 (slot:port)
|
#
|
Type
|
SAN
|
Usage
|
|
3:1
|
1
|
16 Gb
|
S1
|
Intra-cluster/FlashSystem storage
|
|
3:2
|
2
|
16 Gb
|
S2
|
Intra-cluster/FlashSystem storage
|
|
3:3
|
3
|
16 Gb
|
C
|
Remote copy (variable)
|
|
3:4
|
4
|
16 Gb
|
D
|
Host/External storage (Variable)
|
|
4:1
|
5
|
16 Gb
|
S1
|
Intra-cluster/FlashSystem storage
|
|
4:2
|
6
|
16 Gb
|
S2
|
Intra-cluster/FlashSystem storage
|
|
4:3
|
7
|
16 Gb
|
C
|
Host/External storage (variable)
|
|
4:4
|
8
|
16 Gb
|
D
|
Remote copy (variable)
|
|
7:1
|
9
|
16 Gb
|
C
|
Host
|
|
7:2
|
10
|
16 Gb
|
D
|
Host
|
|
7:3
|
11
|
16 Gb
|
C
|
Host
|
|
7:4
|
12
|
16 Gb
|
D
|
Host
|
|
localfcportmask
|
000000110011
|
|||
|
partnerfcportmask
|
000010000100
|
|||
|
|
|
|
|
|
|
SAN - C / D Customer Fabric
SAN - S1 / S2 Internal Switches
|
16 ports per building block connect to customer fabric: 8 per AC3
|
|||
The infrastructure savings method has specific port cabling guidelines for the FlashSystem storage enclosures and AC3 controller enclosures. The guidelines are calculated to support connections and non disruptive scalability for up to eight AC3 controller enclosures and eight FlashSystem storage enclosures (four building blocks plus four FlashSystem storage enclosures).
A.5 Guidelines: Zoning and pathing
This section covers general guidelines concerning N_Port ID Virtualization (NPIV) and pathing and also zoning guidelines for the performance and infrastructure savings methods.
A.5.1 NPIV
N-Port Virtualization ID (NPIV) is a method for virtualizing a physical Fibre Channel port that is used for host I/O. When NPIV is enabled, you must use the virtualized WWPN in your host zones. The non-virtualized WWPN continue to be used for the storage, cluster, and remote copy zones.
From the CLI, the lsiogrp command returns a detailed view of I/O groups that are visible to the system. If the resulting output is fctargetportnode:enabled then NPIV is enabled.
Figure A-2 shows the Fibre Channel Ports view that can be displayed from the GUI by selecting Settings → Network → Fibre Channel Ports and then opening the selection for ports with ID 1. The figure shows the four worldwide port names (WWPNs) that correspond to ID 1 for node 1. The WWPN that must be used for the host zones is indicated by Yes in the Host IO Permitted and Virtualized columns.
Figure A-2 Fibre Channel Ports view
|
Note: For more details about NPIV, see the Implementing the IBM System Storage SAN Volume Controller with IBM Spectrum Virtualize V8.1, SG24-7933 chapter about N-Port Virtualization ID (NPIV) support.
|
A.5.2 Pathing
The number of paths to a storage volume, commonly referred to as a logical unit number (LUN) is a composition of a number of physical and logical elements:
•Physical number of these:
– IBM FlashSystem V9000 host accessible ports (target ports)
– Ports per host (initiator ports)
•Logical number of these:
– IBM FlashSystem V9000 host objects
– Initiator ports (WWPNs) per IBM FlashSystem V9000 host object
– IBM FlashSystem V9000 mappings per volume
– Target ports per initiator ports (zoning)
Calculating the number of paths to a storage volume
The goal is to have no more than a maximum of eight redundant paths between the hosts and the volumes. This provides good performance and protection from single point of failures. Increasing the number of zones or increasing the number of host objects per host decreases the number of paths to a volume.
The first row in Table A-3 represents a way to keep the number of paths to four. This scales with hosts with more than two ports. In this case, there would be a host object for every pair of ports. These port pairs would each be connected to a separate fabric. An equal portion of volumes would be mapped to each host object.
Although eight redundant paths are supported, limiting the number of paths from each host to four reduces the performance impact of error handling by the host, and demands on resources within the host. Limiting the number of paths to four also helps avoid issues with high port fan-outs and fabric state changes.
The last row is not a recommended configuration and shows a high number of paths that can result from lumping all the host ports and all the target ports into one zone per fabric.
Table A-3 Calculating the number of paths to a volume
|
Host with 16 initiator ports
AC3s with 16 target ports
2 redundant fabrics
|
Number of initiator ports (WWPNs) per host object
|
Number of host object mappings per volume
|
Number of target ports per initiator port
|
Number of paths
|
|
4 host zones, 8 host objects
|
02
|
1
|
2
|
004
|
|
2 host zones, 1 host object
|
16
|
1
|
8
|
128
|
The calculation is simple arithmetic as follows:
# of initiator ports per host object * number of host object mappings per volume * number of target ports per initiator port = number of paths
|
Note: For a host with more than two ports, create multiple host definitions, each with no more than two ports.
For management simplification, multiple host objects may grouped into a single host cluster, as documented in Implementing the IBM System Storage SAN Volume Controller with IBM Spectrum Virtualize V8.1, SG24-7933 chapter about Hosts.
|
A.5.3 Port utilization method for performance
The method for performance requires up to four types of zones: host zones, storage zones (internal and external), intra-cluster zones, with optional remote copy zones. They can be defined as follows:
•One intra-cluster zone per fabric
•Two host zones per fabric
•One storage zone per fabric
•One remote copy zone per fabric (if needed)
Figure A-3 shows an overview of the zoning that is configured for the performance method.
Figure A-3 High level zone requirements
Figure A-4 shows an example for zoning storage and host, based on Table A-1 on page 248 for 12 x 16 Gbps building blocks.
Figure A-4 Zoning for performance detailed example
The same physical AC3 ports are used for both storage and host connections but the WWPNs are different due to NPIV.
|
Tip: You must use the virtualized WWPN for the host zones with NPIV enabled.
|
A.5.4 Port utilization method for infrastructure savings
The infrastructure savings method requires up to four types of zones:
•Open zoning and FlashSystem storage
•Host zones
•Remote copy zones (optional)
•Additional external storage zones
These zones are described as follows:
•Open zoning on internal switches
AC3 and Flash storage enclosures components are connected as documented in the “Installing” topic of IBM Knowledge Center.
•Two host zones per fabric
•One external storage zone per fabric
•One remote copy zone per fabric (if needed)
Figure A-5 shows an overview of the zoning configured for the infrastructure savings method.
Figure A-5 IBM FlashSystem V9000 scalable solution: Zoning for infrastructure savings
A.6 Summary
This appendix provides guidelines for two methods or port utilization in a scalable IBM FlashSystem V9000 environment. Depending on customer requirements, the following methods of port utilization can be put into practice:
•IBM FlashSystem V9000 port utilization for performance
This method uses more customer switch ports to improve performance for certain configurations. Only ports designated for intra-cluster communication are attached to private internal switches. The private internal switches are optional and all ports can be attached to customer switches.
•IBM FlashSystem V9000 port utilization for infrastructure savings
This method reduces the number of required customer Fibre Channel ports that are attached to the customer fabrics. This method provides high performance and low latency but performance might be port-limited for certain configurations. Intra-cluster communication and FlashSystem storage traffic occur over the internal switches.
By following the port utilization cabling and zoning guidelines in this appendix, you can maximize the full potential of performance and low latency of IBM FlashSystem V9000 in enterprise-scalable environments.
|
Note: It is beyond the intended scope of this IBM Redpaper publication to provide an in-depth understanding of port configuration and SAN zoning setup. For more information, see the Implementing the IBM System Storage SAN Volume Controller with IBM Spectrum Virtualize V8.1, SG24-7933, chapter about SAN configuration planning.
|
A.7 Supported environments
IBM FlashSystem V9000 can be used in different port utilizations. To check for supported switches, see the IBM System Storage Interoperation Center (SSIC).
For your search, use the values listed in Table A-4.
Table A-4 Values for the search fields
|
Field
|
Value to select
|
|
Storage Family
|
IBM System Storage Enterprise Flash
|
|
Storage Version
|
FlashSystem V9000 8.1.x
|
|
Connection Protocol
|
Fibre Channel
|
Validate that the intended FC switches are listed in the SAN or Networking field.
If there are additional host-side details, such as platform, OS, or multipathing, that restricts your switch selection, then choose the selections in the appropriate lists.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.