FlashSystem V9000 architecture
This chapter describes the IBM FlashSystem V9000 architecture, detailing the components, capabilities, and features that make up this product. An introduction to the IBM FlashSystem V9000, product features, a comparison to the IBM FlashSystem V840, and an overview of the architecture and hardware are included.
This chapter includes the following topics:
IBM Knowledge Center has more details about IBM FlashSystem V9000 architecture:
|
Note: This chapter refers to both AC2 and AC3 control enclosure models. Where fundamental differences exist between the two models, images of both control enclosures are provided. Where no significant differences exist, only the AC2 model is shown, but the action and resulting display are the same on an AC3 control enclosure of an IBM FlashSystemV9000.
|
2.1 Introduction to IBM FlashSystem V9000
IBM FlashSystem V9000 is an all-flash storage array that provides extreme performance and large capacity while also delivering enterprise-class reliability and “green” data center power and cooling requirements. The IBM FlashSystem V9000 building block holds up to twelve 5.7 terabytes (TB) IBM MicroLatency modules in only 6U of rack space, making it an extremely dense all-flash storage array solution.
IBM FlashSystem V9000 uses a fully featured and scalable all-flash architecture that performs at up to 3 million input/output operations per second (IOPS) with IBM MicroLatency modules, is scalable up to 68 gigabytes per second (GBps), and delivers up to 2.28 petabytes (PB) of internal flash effective capacity. With its flash-optimized design, IBM FlashSystem V9000 can provide response times of 180 microseconds. This high capacity, extreme performance, and enterprise reliability are powered by the patented IBM FlashCore Technology.
IBM FlashSystem V9000 offers the advantages of software-defined storage at the speed of flash. This all-flash storage platform combines the high performance, ultra-low latency, superior efficiency and extreme reliability of IBM FlashCore technology with a rich set of virtualization and storage features such as dynamic tiering, thin provisioning, data copy services and high-availability configurations.
Advanced data services that are provided include copy services, mirroring, replication, external virtualization, IBM HyperSwap, Microsoft Offloaded Data Transfer (ODX) and VMware vSphere Storage APIs - Array Integration (VAAI) support. Host interface support includes 8 gigabit (Gb) and 16 Gb FC, and 10 Gb Fibre Channel over Ethernet (FCoE) and Internet Small Computer System Interface (iSCSI). Advanced Encryption Standard (AES) 256 hardware-based encryption adds to the rich feature set.
The IBM FlashSystem V9000 building block is made up of the two control enclosures, referred to as AC2s or AC3s, and one storage enclosure, referred to as AE2. The IBM FlashSystem V9000 core attributes are described next. Figure 2-1 shows the front view of the IBM FlashSystem V9000.
Figure 2-1 IBM FlashSystem V9000
2.1.1 Capacity
IBM FlashSystem V9000 supports a maximum of four building blocks and four additional storage enclosures. Each building block or storage enclosure can accommodate up to twelve 5.7 TB IBM MicroLatency modules, which provide a capacity of 57 TB (RAID 5). IBM FlashSystem V9000, with 8 storage enclosures, therefore supports a maximum physical capacity of 456 TB of internal MicroLatency flash storage.
By using the optional IBM Real-time Compression and other design elements, the FlashSystem V9000 provides up to 285 TB effective capacity in only 6U. With 8 storage enclosures, the effective capacity goes up to 2.28 PB in only 36U of rack space.
IBM FlashSystem V9000 also supports up to 32 PB of external storage virtualization and also standard and high density expansion enclosures scaling up to 29.4 PB raw capacity by using nearline SAS (NL-SAS) HDDs or 32 PB raw capacity by using SSDs.
|
Note: For detailed capacity information about IBM FlashSystem V9000 and expansion enclosures see Table 2-7 on page 58.
|
Each IBM FlashSystem V9000 building block can be ordered with 4, 6, 8, 10, or 12 MicroLatency modules. The MicroLatency modules available are either 1.2 TB, 2.9 TB, or 5.7 TB storage capacity.
|
Important: 1.2 TB, 2.9 TB, and 5.7 TB IBM MicroLatency modules cannot be intermixed in the same IBM FlashSystem V9000 storage enclosure.
|
IBM FlashSystem V9000 supports RAID 5 configurations.
|
Note: The maximum usable capacity of IBM FlashSystem V9000 in RAID 5 mode is
51.8 tebibytes (TiB) per building block. |
IBM FlashSystem V9000 supports the creation of up to 2,048 logical unit numbers (LUNs) per building block. The size of the LUNs can be 1 MiB - 51.8 TiB in size (not to exceed the total system capacity). The IBM FlashSystem V9000 supports up to 2,048 host connections and up to 256 host connections for each interface port. The IBM FlashSystem V9000 supports the mapping of multiple LUNs to each host for Fibre Channel, Fibre Channel over Ethernet (FCoE), and iSCSI protocols.
IBM FlashSystem V9000 supports up to 256 host connections for the iSCSI protocol.
Table 2-1 lists all the combinations of storage capacities for various configurations of the IBM FlashSystem V9000 building block.
Table 2-1 IBM FlashSystem V9000 capacity in TB and TiB for RAID 5
|
IBM FlashSystem 900 AE2 configuration
|
RAID 5 TB
|
RAID 5 TiB
|
|
Four 1.2 TB flash modules
|
2.2
|
2.0
|
|
Six 1.2 TB flash modules
|
4.5
|
4.1
|
|
Eight 1.2 TB flash modules
|
6.8
|
6.2
|
|
Ten 1.2 TB flash modules
|
9.1
|
8.3
|
|
Twelve 1.2 TB flash modules
|
11.4
|
10.4
|
|
Six 2.9 TB flash modules
|
11.4
|
10.3
|
|
Eight 2.9 TB flash modules
|
17.1
|
15.5
|
|
Ten 2.9 TB flash modules
|
22.8
|
20.7
|
|
Twelve 2.9 TB flash modules
|
28.5
|
28.9
|
|
Six 5.7 TB flash modules
|
22.8
|
20.7
|
|
Eight 5.7 TB flash modules
|
34.2
|
31.0
|
|
Ten 5.7 TB flash modules
|
45.6
|
41.4
|
|
Twelve 5.7 TB flash modules
|
57.0
|
51.8
|
2.1.2 Performance and latency
IBM FlashSystem V9000 uses all hardware field-programmable gateway array (FPGA) components in the AE2 storage enclosure data path, which enables fast I/O rates and low latency. IBM FlashSystem V9000 provides extreme performance of up to 3 million IOPS and up to 68 GBps in bandwidth. The IBM FlashSystem V9000 provides response times as low as 180 microseconds (μs).
2.1.3 IBM FlashCore technology
IBM FlashSystem V9000 provides enterprise class reliability and serviceability that are unique for all-flash storage arrays. IBM FlashSystem V9000 uses the patented IBM FlashCore Technology to provide data protection and maximum system uptime:
•IBM Advanced Flash Management improves flash endurance 9x over standard implementations:
– Proprietary garbage collection, relocation, and block-picking algorithms that were invented by IBM.
– Flash wear leveling includes the following functions:
• ECC algorithms that correct very high bit error rates.
• Variable voltage and read-level shifting to maximize flash endurance.
• Health binning and heat segregation continually monitor the health of flash blocks and perform asymmetrical wear leveling and sub-chip tiering.
• Hot-data placement provides up to 57% improvement in endurance. Heat-level grouping provides up to 45% reduction in write amplification.
•IBM Variable Stripe RAID is a patented IBM technology that provides an intra-module RAID stripe on each flash module.
•With two-dimensional (2D) Flash RAID, system-wide RAID 5 along with Variable Stripe RAID helps reduce downtime and maintain performance, and enables the provisioning of an entire flash module as a spare to be used in another flash module failure.
Terminology
The following terms are mentioned in this book:
Wear leveling An algorithm that assures even usage of all blocks.
Garbage collection Erasing blocks, which are not used anymore, so that they can be rewritten.
Relocation Moving a block to another location.
Block picking The first step of the garbage collection process. Using proprietary algorithms, the best block is picked for garbage collection.
More reliability and serviceability features of IBM FlashSystem V9000
In addition to the standard features, IBM FlashSystem V9000 includes the following reliability and serviceability features:
•Hot-swappable IBM MicroLatency modules with tool-less front panel access
If a MicroLatency module failure occurs, critical client applications can remain online while the defective module is replaced.
Because client application downtime does not need to be scheduled, you can typically perform this service immediately versus waiting for days for a service window. The Directed Maintenance Procedure (DMP), accessible from the GUI, can be used to prepare the IBM FlashSystem V9000 for a MicroLatency module replacement. You can easily remove the MicroLatency modules from the front of the IBM FlashSystem V9000 unit without needing to remove the top access panels or extend cabling.
•Concurrent code loads
IBM FlashSystem V9000 supports concurrent code load, enabling client applications to remain online during firmware upgrades to all components, including the flash modules.
•Redundant hot-swappable components
RAID controllers (called canisters), management modules, and interface cards (all contained in the canister), and batteries, fans, and power supplies are all redundant and hot-swappable. All components are easily accessible through the front or rear of the unit so the IBM FlashSystem V9000 does not need to be moved in the rack, and top access panels or cables do not need to be extended. This makes servicing the unit easy.
|
Tip: Concurrent code loads require that all connected hosts have at least two connections, at least one to each control enclosure, to the IBM FlashSystem V9000.
|
2.1.4 Overview of IBM Variable Stripe RAID and 2D Flash RAID
Storage systems of any kind are typically designed to perform two main functions: to store and protect data. The IBM FlashSystem V9000 includes the following features for data protection:
•RAID data protection:
– IBM Variable Stripe RAID
– Two-dimensional (2D) Flash RAID
•Flash memory protection methods
•Optimized RAID rebuild times
Variable Stripe RAID
Variable Stripe RAID is a unique IBM technology that provides data protection on the page, block, or chip level. It eliminates the necessity to replace a whole flash module when a single chip or plane fails. This, in turn, expands the life and endurance of flash modules and reduces considerably maintenance events throughout the life of the system.
Variable Stripe RAID provides high redundancy across chips within a flash module. RAID is implemented at multiple addressable segments within chips, in a 15+1 or 12+1 RAID 5 fashion, and it is controlled at the flash controller level (up to four in each flash module). Due to the massive parallelism of direct memory access (DMA) operations controlled by each FPGA and parallel access to chip sets, dies, planes, blocks, and pages, the implementation of Variable Stripe RAID has minimal effect on performance.
The following information describes some of the most important aspects of Variable Stripe RAID implementation:
•Variable Stripe RAID is managed and controlled by each of the up to four flash controllers within a single module.
•A flash controller is in charge of only 13 or 16 flash chips (IBM MicroLatency module capacity size-dependent).
•Data is written on flash pages of 8 kilobytes (KB) and erased in 1 megabyte (MB) flash blocks.
•Variable Stripe RAID is implemented and managed at flash chip plane levels.
•There are 16 planes per chip.
•Before a plane fails, at least 256 flash blocks within a plane must be deemed failed.
•A plane can also fail in its entirety.
•Up to 64 planes can fail before a whole module is considered failed.
•Up to four chips can fail before a whole module is considered failed.
•When a flash module is considered failed, 2D Flash RAID takes control of data protection and recovery.
•When a plane or a chip fails, Variable Stripe RAID activates to protect data while maintaining system-level performance and capacity.
How Variable Stripe RAID works
Variable Stripe RAID is an IBM patented technology. It includes but is more advanced than a simple RAID of flash chips. Variable Stripe RAID introduces two key concepts:
•The RAID stripe is not solely across chips; it actually spans across flash layers.
•The RAID stripe can automatically vary based on observed flash plane failures within a flash module. For example, stripes are not fixed at n+1 RAID 5 stripe members, but they can go down to 15+1, 14+1, or even 13+1 based on plane failures.
This ability to protect the data at variable stripes effectively maximizes flash capacity even after flash component failures. Figure 2-2 on page 31 shows an overview of the IBM FlashSystem Variable Stripe RAID.
Figure 2-2 IBM FlashSystem Variable Stripe RAID (VSR)
Figure 2-3 shows the benefits of IBM Variable Stripe RAID.
Figure 2-3 The value of the IBM FlashSystem Variable Stripe RAID
An important aspect to emphasize is that Variable Stripe RAID has an effect at only the plane level. Therefore, only the affected planes within a plane failure are converted to N-1. Variable Stripe RAID maintains the current stripe member count (N+1) layout through the remainder of the areas of all other planes and chips that are not involved in the plane failure.
To illustrate how Variable Stripe RAID functions, assume that a plane fails within a flash chip and is no longer available to store data. This might occur as a result of a physical failure within the chip, or some damage is inflicted on the address or power lines to the chip. The plane failure is detected and the system changes the format of the page stripes that are used.
The data that was previously stored in physical locations across chips in all 16 or 13 lanes using a page stripe format with 10 pages is now stored across chips in only nine lanes using a page stripe format with nine pages. Therefore, no data stored in the memory system was lost, and the memory system can self-adapt to the failure and continue to perform and operate by processing read and write requests from host devices.
This ability of the system to automatically self-adapt, when needed, to chip and intra-chip failures makes the FlashSystem flash module extremely rugged and robust, and capable of operating despite the failure of one or more chips or intra-chip regions. It also makes the system easier to use because the failure of one, two, or even more individual memory chips or devices does not require the removal and potential disposal of previously used memory storage components.
The reconfiguration or reformatting of the data to change the page stripe formatting to account for chip or intra-chip failures might reduce the amount of physical memory space that is held in reserve by the system and available for the system for background operation. Note that in all but the most extreme circumstances (in which case the system creates alerts), it does not affect performance. Even in the case of extreme circumstances, the usable capacity is not affected and the system fails the module first.
Reliability, availability, and serviceability
The previous explanation points out an increase in reliability, availability, and serviceability (RAS) levels and the IBM FlashSystem RAS levels over other technologies.
In summary, Variable Stripe RAID has these capabilities:
•Patented Variable Stripe RAID allows RAID stripe sizes to vary.
•If one plane fails in a chip stripe, only the failed plane is bypassed, and then data is restriped across the remaining chips. No system rebuild is needed.
•Variable Stripe RAID reduces maintenance intervals caused by flash failures.
Two-dimensional (2D) Flash RAID
Two-dimensional (2D) Flash RAID refers to the combination of Variable Stripe RAID (at the flash module level) and system-level RAID 5.
The second dimension of data protection is implemented across flash modules of RAID 5 protection. This system-level RAID 5 is striped across the appropriate number of flash modules in the system based on the selected configuration. System-level RAID-5 can stripe across four (2D+1P+1S - IBM MicroLatency 1.2 TB module only), six (4D+1P+1S), eight (6D+1P+1S), ten (8D+1P+1S), or twelve flash modules (10D+1P+1S).
The architecture enables you to designate a dynamic flash module hot spare.
Figure 2-4 on page 33 shows the IBM FlashSystem V9000 2D RAID.
Figure 2-4 IBM FlashSystem 2D RAID
The 2D Flash RAID technology within the IBM FlashSystem V9000 provides two independent layers of RAID 5 data protection within each system:
•The module-level Variable Stripe RAID technology
•An additional system-level RAID 5 across flash modules
The system-level RAID 5 complements the Variable Stripe RAID technology implemented within each flash module, and it provides protection against data loss and data unavailability resulting from flash module failures. It also enables data to be rebuilt onto a hot-spare flash module, so that flash modules can be replaced without data disruption.
Other reliability features
In addition to 2D Flash RAID and Variable Stripe RAID data protection, the IBM FlashSystem family storage systems incorporate other reliability features:
•Error-correcting codes to provide bit-level reconstruction of data from flash chips.
•Checksums and data integrity fields designed to protect all internal data transfers within the system.
•Overprovisioning to enhance write endurance and decrease write amplification.
•Wear-leveling algorithms balance the number of writes among flash chips throughout the system.
•Sweeper algorithms help ensure that all data within the system is read periodically to avoid data fade issues.
Understanding 2D Flash RAID enables you to visualize the advantage over other flash memory solutions. Both Variable Stripe RAID and 2D Flash RAID are implemented and controlled at FPGA hardware-based levels. Two-dimensional flash RAID eliminates single points of failure and provides enhanced system-level reliability.
2.1.5 Scalability
IBM FlashSystem V9000 supports the ability to grow both the storage capacity and performance after deployment, which is referred to as scale up and scale out. IBM FlashSystem V9000 scale up or scale out is achieved by using scalable building blocks and additional storage enclosures. IBM FlashSystem V9000 supports a maximum configuration of twelve 1.2 TB, 2.9 TB, or 5.7 TB IBM MicroLatency modules per AE2 storage enclosure. The IBM FlashSystem V9000 can be purchased with 4, 6, 8, 10, or 12 modules of 1.2 TB, 2.9 TB, or 5.7 TB sizes.
Capacity can be scaled up with external storage virtualization or with standard or high density (HD) expansion enclosures that support NL-SAS HDDs or SDDs. You can attach up to 20 standard expansion enclosures per controller pair (up to 80 total) scaling up to 9.6 PB raw capacity using NL-SAS HDDs or 29.4 PB raw capacity using SSDs. You can attach up to 8 HD expansion enclosures per controller pair (up to 32 total) scaling up to 29.4 PB raw capacity using NL-SAS HDDs or 32PB raw capacity using SSDs. Up to 32PB of external storage can be virtualized.
IBM FlashSystem V9000 offers these upgrade options:
•Systems that are purchased with 4 MicroLatency modules can be expanded to 6, 8, 10, or 12 of the same capacity MicroLatency modules.
•Systems that are purchased with 6 MicroLatency modules can be expanded to 8, 10, or 12 of the same capacity MicroLatency modules.
•Systems that are purchased with 8 MicroLatency modules can be expanded to 10 or 12 of the same capacity MicroLatency modules.
•Systems that are purchased with 10 MicroLatency modules can be expanded to 12 of the same capacity MicroLatency modules.
|
Note: Adding MicroLatency modules to an existing AE2 storage enclosure is a disruptive activity for the IBM FlashSystem V9000 building block. Storage expansions without solution outage are possible with careful planning.
|
IBM FlashSystem V9000 delivers up to 57 TB per building block, scales to four building blocks, and offers up to four more 57 TB V9000 storage enclosure expansion units for large-scale enterprise storage system capability. Building blocks can be either fixed or scalable. You can combine scalable building blocks to create larger clustered systems in such a way that operations are not disrupted.
A scalable building block can be scaled up by adding IBM FlashSystem V9000 AE2 storage enclosures for increased storage capacity. You can add a maximum of four extra storage enclosures, one extra storage enclosure per building block, to any scaled solution. A scalable building block can be scaled out by combining up to four building blocks to provide higher IOPS and bandwidth needs for increased performance as shown in Figure 2-5.
Figure 2-5 IBM FlashSystem V9000 scalability options
With IBM Real-time Compression technology, IBM FlashSystem V9000 further extends the economic value of all-flash systems. IBM FlashSystem V9000 provides up to two times the improvement in Real-time Compression over the model it is replacing, by using dedicated Compression Acceleration Cards. Using the optional Real-time Compression and other design elements, IBM FlashSystem V9000 provides up to 57 TB usable capacity and up to 285 TB effective capacity in only 6U. This scales to 456 TB usable capacity and up to 2.28 PB effective capacity in only 36U. These capacity numbers are not considering the use of SAS expansion enclosures.
A fixed building block contains one IBM FlashSystem V9000. The AE2 storage enclosure is cabled directly to each AC2 or AC3 control enclosure using 8 Gb or 16 Gb links, and each AC2 or AC3 control enclosure is connected to switches or to a host. The AC2 or AC3 control enclosures are directly connected without the use of switches or a SAN fabric, to form the cluster links. A fixed building block can be upgraded to a scalable building block, but the upgrade process is disruptive to operations.
Figure 2-6 shows the relationship between fixed and scalable building blocks.
Figure 2-6 IBM FlashSystem V9000 fixed versus scalable building blocks
For more details about cabling for a fixed building block, see 6.4.1, “Connecting the components in a fixed building block” on page 234.
Scalable building blocks can contain multiple AC2 or AC3 control enclosure pairs and multiple AE2 storage enclosures. The building block components communicate with each other by using Fibre Channel. Management of the environment is provided by using the management ports on the control enclosures, to create a private management local area network (LAN). In a scalable building block, AC2 or AC3 control enclosures are not cabled point to point to each other. This infrastructure means that you can add building blocks or storage enclosures nondisruptively. Dedicated Fibre Channel switches facilitate efficient coordination between the control enclosure resources.
The Fibre Channel switch fabric is dedicated, and is not shared with hosts or server-side storage area networks (SANs). After connecting the components in a scalable building block, no physical cable connects any host to any switch in the internal Fibre Channel switch fabric. This private fabric is therefore not affected by traditional host-side SAN traffic, saturation issues, or accidental or intentional zoning issues, therefore providing maximum availability.
To support a flash-optimized tiered storage configuration for mixed workloads, up to 20 optional 9846/9848-12F or 9846/9848-24F and up to 8 9846/9848-92F SAS expansion enclosures can be connected to each building block in the system. To intermix the models 12F, 24F, and 92F see Table 2-14 on page 83.
Consider the following information:
•A 9846/9848-12F SAS expansion enclosure contains up to 12 3.5-inch nearline SAS drives.
•A 9846/9848-24F SAS expansion enclosure contains up to 24 2.5-inch read-intensive SAS flash drives.
•A 9846-92F or 9848-92F SAS expansion enclosure contains up to 92 3.5-inch SAS drives.
To support SAS expansion enclosures, an AH13 - SAS Enclosure Attach adapter card must be installed in expansion slot 2 of each AC3 control enclosure in the building block.
For more details about cabling for a scalable building block, see 6.4.2, “Connecting the components in a scalable building block” on page 235.
For a comparison and the configuration guidelines of the following two suggested methods for port utilization in an IBM FlashSystem V9000 scalable environment, see Appendix A, “Guidelines: Port utilization in an IBM FlashSystem V9000 scalable environment” on page 657:
•IBM FlashSystem V9000 port utilization for infrastructure savings
This method reduces the number of required customer Fibre Channel ports attached to the customer fabrics. This method provides high performance and low latency but performance might be port-limited for certain configurations. Intra-cluster communication and AE2 storage traffic occur over the internal Fibre Channel switches.
•IBM FlashSystem V9000 port utilization for performance
This method uses more customer switch ports to improve performance for certain configurations. Only ports designated for intra-cluster communication are attached to private internal switches. The private internal switches are optional and all ports can be attached to customer switches.
|
Note: The Fibre Channel internal connection switches are ordered together with the first IBM FlashSystem V9000 scalable building block. IBM also supports the use of customer-supplied Fibre Channel switches and cables, provided it is supported by IBM. See the list of supported Fibre Channel switches:
|
Remember these important considerations:
•Mixing different capacity MicroLatency modules (1.2 TB, 2.9 TB, or 5.7 TB) in a single AE2 storage enclosure is not supported.
•Expanding an IBM FlashSystem V9000 unit with 2, 4, 6, or 8 extra MicroLatency modules requires that the flash array is deleted and recreated. A backup of the system configuration and data migration, if needed, must be planned before the expansion.
2.1.6 Host adapter protocol support
IBM FlashSystem V9000 supports the following interface protocols:
•8 Gbps Fibre Channel
•16 Gbps Fibre Channel
•10 Gbps Fibre Channel over Ethernet (FCoE)
•10 Gbps iSCSI
The interface protocols can auto-negotiate down to slower speeds.
AC2 control enclosure interface options
For the AC2 control enclosure different interface configuration options are available based on whether the AE2 has 8 Gb or 16 Gb optics and whether the configuration is a fixed or a scalable configuration.
For more details about hardware of the AC2 control enclosure, see 2.3, “Control enclosure (AC2)” on page 59.
Table 2-2 shows the three adapter combinations that are allowed in the AC2 control enclosures when the AE2 storage enclosure is ordered with 8 Gb optics. The adapter combinations are identical for both scalable and fixed building block order types.
Table 2-2 8Gb AE2 fixed or scalable building block allowed AC2 I/O adapter combinations
|
Host connections
|
16 Gb FC
4-port |
8 Gb FC
4-port |
16 Gb FC
2-port |
10Gb Ethernet 4-port
|
SAS enclosure attach
|
|
8 Gb FC
|
-
|
3
|
-
|
-
|
-
|
|
16 Gb FC
|
-
|
2
|
2
|
-
|
-
|
|
10Gb Ethernet
|
-
|
2
|
-
|
1
|
-
|
Table 2-3 shows the four adapter combinations that are allowed in the AC2 control enclosures when the AE2 storage enclosure is ordered with 16 Gb optics and the AC2 control enclosure is ordered as a fixed building block. The AC2 control enclosure has six adapter slots, four of which can be used for interface adapters. The SAS enclosure attach adapter takes up one of these four slots.
Table 2-3 16Gb AE2 fixed building block allowed AC2 I/O adapter combinations
|
Host connections
|
16 Gb FC
4-port |
8 Gb FC
4-port |
16 Gb FC
2-port |
10Gb Ethernet 4-port
|
SAS enclosure attach
|
|
16 Gb FC
|
4
|
-
|
-
|
-
|
-
|
|
16 Gb FC
|
3
|
-
|
-
|
-
|
0 or 1
|
|
16 Gb FC
10Gb Ethernet
|
3
|
-
|
-
|
1
|
-
|
|
16 Gb FC
10Gb Ethernet
|
2
|
-
|
-
|
1
|
0 or 1
|
Table 2-4 shows the nine adapter combinations that are allowed in the AC2 control enclosures when the AE2 storage enclosure is ordered with 16 Gb optics and the AC2 control enclosure is ordered as a scalable building block. A main differentiator is whether you order the 16 Gb adapter with two or four ports.
Table 2-4 16Gb AE2 scaled building block allowed AC2 I/O adapter combinations
|
Host connections
|
16 Gb FC
4-port |
8 Gb FC
4-port |
16 Gb FC
2-port |
10Gb Ethernet 4-port
|
SAS enclosure attach
|
|
16 Gb FC
|
4
|
-
|
-
|
-
|
-
|
|
16 Gb FC
|
3
|
-
|
-
|
-
|
0 or 1
|
|
16 Gb FC
10Gb Ethernet
|
3
|
-
|
-
|
1
|
-
|
|
16 Gb FC
10Gb Ethernet
|
2
|
-
|
-
|
1
|
0 or 1
|
|
16 GB FC
|
-
|
-
|
4
|
-
|
-
|
|
16 Gb FC
10Gb Ethernet
|
-
|
-
|
3
|
1
|
-
|
|
10 Gb Ethernet
|
-
|
-
|
2
|
1
|
-
|
|
8 Gb FC
|
-
|
2
|
2
|
-
|
-
|
|
8 Gb FC
10Gb Ethernet
|
-
|
1
|
2
|
1
|
-
|
AC3 control enclosure interface options
For the AC3 control enclosure there are less interface combinations since the AC3 is only ordered when the AE2 has 16Gb optics. The interface combinations are the same for both fixed and scalable order types. The AC3 control enclosure also has more adapter slots than the AC2 control enclosure which allows the AC3 control enclosure to have a dedicated slot for the SAS enclosure attach adapter. The SAS enclosure attach adapter does not take up one of the four allowed adapters that are used for host and AE2 storage communication.
Table 2-5 shows the four adapter combinations that are allowed in the AC3 control enclosures when the AE2 storage enclosure is ordered with 16 Gb optics.
Table 2-5 16Gb AE2 fixed or scalable building block allowed AC3 I/O adapter combinations
|
Host connections
|
16Gb FC
4-port |
8Gb FC
4-port |
16 Gb FC
2-port |
10Gb Ethernet 4-port
|
SAS enclosure attach
|
|
16 Gb FC
|
4
|
-
|
-
|
-
|
0 or 1
|
|
16 Gb FC
|
3
|
-
|
-
|
-
|
0 or 1
|
|
16 Gb FC
10Gb Ethernet
|
3
|
-
|
-
|
1
|
0 or 1
|
|
16 Gb FC
10Gb Ethernet
|
2
|
-
|
-
|
1
|
0 or 1
|
For more details about the AC3 control enclosure hardware, see 2.4, “Control enclosure (AC3)” on page 64.
2.1.7 Encryption support
IBM FlashSystem V9000 provides optional encryption of data at rest, which protects against the potential exposure of sensitive user data and user metadata that are stored on discarded or stolen flash modules. Encryption of system data and metadata is not required, so system data and metadata are not encrypted.
Configuring encryption
You can activate encryption with the easy setup wizard during initialization or the hot key activation process after the IBM FlashSystem V9000 is already initialized, when an encryption feature code is purchased. If encryption is activated, an encryption key is generated by the system to be used for access to the system. The processes start a wizard that guides the user through the process of copying the encryption key to multiple USB keys. For details about setting up encryption, see 9.4, “Security menu” on page 426.
IBM FlashSystem V9000 supports Encryption Rekey to create new encryption keys that supersede the existing encryption keys.
|
Note: If you plan to implement either hot key activation or encryption rekey, be sure to inform IBM Support so that it can monitor the operation. IBM Support personnel will guide you through this process.
|
Accessing an encrypted system
At system start (power on) or to access an encrypted system, the encryption key must be provided by an outside source so that the IBM FlashSystem V9000 can be accessed. The encryption key is provided by inserting the USB flash drives that were created during system initialization into one of the AC2 or AC3 control enclosures in the solution. Starting with FlashSystem V9000 Version 7.8 encryption keys can be managed by an IBM Security Key Lifecycle Manager (SKLM) key server.
Encryption technology
Key encryption is protected by an Advanced Encryption Standard (XTS-AES) algorithm key wrap using the 256-bit symmetric option in XTS mode, as defined in the IEEE1619-2007 standard. An HMAC-SHA256 algorithm is used to create a hash message authentication code (HMAC) for corruption detection, and it is additionally protected by a system-generated cyclic redundancy check (CRC).
IBM Security Key Lifecycle Manager (V7.8 and higher)
IBM FlashSystem V9000 Software V7.8 adds improved security with support for encryption key management software that complies with the Key Management Interoperability Protocol (KMIP) standards, such as IBM Security Key Lifecycle Manager (SKLM) to help centralize, simplify, and automate the encryption key management process.
Prior to IBM FlashSystem V9000 Software V7.8, encryption was enabled only by using USB flash drives to copy the encryption key to the system. USB flash drives have the following characteristics:
•Physical access to the system is required to process a rekeying operation.
•No moving parts with almost no read operations or write operations to the USB flash drive.
•Inexpensive to maintain and use.
•Convenient and easy to have multiple identical USB flash drives available as backups.
Starting with IBM FlashSystem V9000 Software V7.8 you have the option to enable encryption by configuring a key server.
Key servers can have the following characteristics:
•Physical access to the system is not required to process a rekeying operation.
•Support for businesses that have security requirements not to use USB ports.
•Strong key generation.
•Key self-replication and automatic backups.
•Implementations follow an open standard that aids in interoperability.
•Audit detail.
•Ability to administer access to data separately from storage devices.
|
Note: If you are creating a new cluster with V 7.8 you have the option to either use USB-based encryption or key server encryption but not both. The USB flash drive method and key server method cannot be used in parallel on the same system. Existing customers that are currently using USB-based encryption must wait for a future release before they can move to key server encryption. The migration of a local (USB) key to a centrally managed key (SKLM key server) is not yet available at the time of this writing.
|
For more information about encryption technologies supported by other IBM storage devices, see IBM DS8880 Data-at-rest Encryption, REDP-4500.
2.1.8 Management
IBM FlashSystem V9000 includes a single state-of-the-art IBM storage management interface. The IBM FlashSystem V9000 single graphical user interface (GUI) and command-line interface (CLI) are updated from previous versions of the IBM FlashSystem products to include the IBM SAN Volume Controller CLI and GUI, which resembles the popular IBM XIV GUI.
IBM FlashSystem V9000 also supports Simple Network Management Protocol (SNMP), email notification (Simple Mail Transfer Protocol (SMTP)), and syslog redirection.
Figure 2-7 on page 42 shows the IBM FlashSystem V9000 GUI for a fixed building block system.
Figure 2-7 IBM FlashSystem V9000 GUI for a fixed building block system
For more details about the use of the IBM FlashSystem V9000 GUI and CLI, see 2.7.1, “System management” on page 84.
Figure 2-8 IBM FlashSystem V9000 GUI for a scale up and scale out scalable building block system
The IBM Mobile Storage Dashboard, version 1.5.4, also supports IBM FlashSystem V9000. IBM Storage Mobile Dashboard is a no-cost application that provides basic monitoring capabilities for IBM storage systems. Storage administrators can securely check the health and performance status of their IBM Storage systems by viewing events and also real-time performance metrics. You can download this application for your Apple iPhone from this page:
Figure 2-9 shows examples of the IBM Storage Mobile Dashboard.
Figure 2-9 IBM Storage Mobile Dashboard
2.2 Architecture of IBM FlashSystem V9000
The IBM FlashSystem V9000 architecture is explained in the following section together with key product design characteristics, performance, and serviceability. Hardware components are also described.
2.2.1 Overview of architecture
The IBM FlashSystem V9000 AC2 or AC3 control enclosure combines software and hardware into a comprehensive, modular appliance that uses symmetric virtualization. Single virtualization engines, which are known as AC2 or AC3 control enclosures, are combined to create clusters. In a scalable solution, each cluster can contain between two and eight control enclosures.
Symmetric virtualization is achieved by creating a pool of managed disks (MDisks) from the attached storage systems. Those storage systems are then mapped to a set of volumes for use by attached host systems. System administrators can view and access a common pool of storage on the storage area network (SAN). This functionality helps administrators to use storage resources more efficiently and provides a common base for advanced functions.
The design goals for the IBM FlashSystem V9000 are to provide the client with the fastest and most reliable all-flash storage array on the market, while making it simple to service and support with no downtime. The IBM FlashSystem V9000 uses hardware acceleration techniques incorporating Field Programmable Gate Array (FPGA) components to reduce the software stack which keeps I/O latency to a minimum and I/O performance to a maximum.
IBM Spectrum Virtualize software
IBM FlashSystem V9000 is built with IBM Spectrum Virtualize software, which is part of the IBM Spectrum Storage family.
Virtualization is a radical departure from traditional storage management. In traditional storage management, storage is attached directly to a host system that controls the storage management. SAN introduced the principle of networks of storage, but storage is still primarily created and maintained at the RAID system level. Multiple RAID controllers of different types require knowledge of, and software that is specific to, the specific hardware. Virtualization provides a central point of control for disk creation and maintenance.
IBM Spectrum Virtualize is a key member of the IBM Spectrum Storage portfolio. It is a highly flexible storage solution that enables rapid deployment of block storage services for new and traditional workloads, on-premises, off-premises, and in a combination of both, and it is designed to help enable cloud environments.
For more information about the IBM Spectrum Storage portfolio, see the following website:
AE2 storage enclosure architecture
Figure 2-10 on page 45 illustrates the IBM FlashSystem V9000 AE2 storage enclosure design. At the core of the system are the two high-speed non-blocking crossbar buses. The crossbar buses provide two high-speed paths, which carry the data traffic, and they can be used by any host entry path into the system. There is also a slower speed bus for management traffic.
Connected to the crossbar buses are high-speed non-blocking RAID modules and IBM MicroLatency modules. There is also a passive main system board (midplane) to which both the RAID canisters and all the flash modules connect, and also connections to battery modules, fan modules, and power supply units.
The two RAID canisters contain crossbar controllers, management modules, interface controllers and interface adapters, and fan modules. The two RAID canisters form a logical cluster, and there is no single point of failure in the design (assuming that all host connections have at least one path to each canister).
Figure 2-10 AE2 storage enclosure architecture
IBM FlashSystem V9000 software
The IBM FlashSystem V9000 software provides the following functions for the host systems that attach to IBM FlashSystem V9000:
•Creates a pool of storage.
Two choices are available when the system comprises more than one AE2 storage enclosure:
– Create a separate pool for each AE2 storage enclosure.
– Create one storage pool that spans all AE2 storage enclosures.
|
Important: Before deciding whether to create a single or multiple storage pools, carefully evaluate which option best fits your solution needs, considering data availability and recovery management.
|
•Provides logical unit virtualization.
•Manages logical volumes.
IBM FlashSystem V9000 software also provides these advanced functions:
•Large scalable cache
•Copy services:
– IBM FlashCopy (point-in-time copy) function, including thin-provisioned FlashCopy to make multiple targets affordable
– Metro Mirror (synchronous copy)
– Global Mirror (asynchronous copy)
•Data migration
•Space management
•IBM Easy Tier function to automatically migrate the most frequently used data to higher-performance storage
•Thin-provisioned logical volumes
•Compressed volumes to consolidate storage
•HyperSwap, which enables each volume to be presented by two I/O groups
•Microsoft Offloaded Data Transfer (ODX)
•VMware and vSphere 6.0 support
•Enhanced FlashCopy bitmap space increased
For more information about the IBM FlashSystem V9000 advanced software features, see Chapter 3, “Advanced software functions” on page 97.
MDisks
A managed disk (MDisk) is a logical unit of physical storage. MDisks are either arrays (RAID) from internal storage or volumes from external storage systems. MDisks are not visible to host systems.
An MDisk might consist of multiple physical disks that are presented as a single logical disk to the storage area network (SAN). An MDisk always provides usable blocks of physical storage to the system even if it does not have a one-to-one correspondence with a physical disk.
Each MDisk is divided into a number of extents, which are sequentially numbered starting at 0 (zero), from the start to the end of the MDisk. The extent size is a property of pools. When an MDisk is added to a pool, the size of the extents that the MDisk is divided into depends on the attribute of the pool to which it was added. The access mode determines how the clustered system uses the MDisk.
|
Attention: If you observe intermittent breaks in links or if you replaced cables or connections in the SAN fabric or LAN configuration, you might have one or more MDisks in degraded status. If an I/O operation is attempted when a link is broken and the I/O operation fails several times, the system partially excludes the MDisk and changes the status of the MDisk to excluded. You must include the MDisk to resolve the problem.
|
The MDisks are placed into storage pools where they are divided into several extents, which are 16 - 8192 MB, as defined by the IBM FlashSystem V9000 administrator. For more information about the total storage capacity that is manageable per system regarding the selection of extents, see the following web page:
A volume is host-accessible storage that was provisioned from one storage pool. Or, if it is a mirrored volume, it was provisioned from two storage pools. The maximum size of an MDisk is 1 PB. One IBM FlashSystem V9000 supports up to 4096 MDisks.
MDisks consideration for IBM FlashSystem V9000
Each MDisk from external storage has an online path count, which is the number of nodes that have access to that MDisk. The path count represents a summary of the I/O path status between the system nodes and the storage device. The maximum path count is the maximum number of paths that were detected by the system at any point in the past. If the current path count is not equal to the maximum path count, the MDisk might be degraded. That is, one or more nodes might not see the MDisk on the fabric.
Previously with the IBM Spectrum Virtualize (2145 SAN Volume Controller Model DH8) and previously with IBM FlashSystem V840, the leading practices stated that the back-end storage (on SAN Volume Controller) or internal storage (in FlashSystem V840) should be divided into 16 MDisks for the best performance.
On the IBM FlashSystem V9000, one MDisk per AE2 array is automatically created, rather than the 16 MDisks previously used on older products.
The reason for this change can be explained in the relationship of the I/O throughput on the machine, versus the number of cores and threading on the control enclosure architecture.
The control enclosures assign workloads to different cores, depending on the object that is associated with the workload. The three categories of objects are as follows:
•Interface channel (I/O) ports
•VDisk
•MDisks
When an I/O comes in, this input is assigned to the core associated with an interface channel port. It moves to the VDisk thread and then to the MDisk thread and finally back to an interface channel thread, for de-staging back out of the system.
The VDisk has the most amount of work associated with it.
The redesign for IBM FlashSystem V9000 was done to enable the interface ports to use all eight threads, but VDisks are restricted to seven threads and MDisks must all use the thread that VDisks do not use. Tests showed that the VDisk work is approximately seven times more than the MDisk work.
|
Note: Interface I/O is actually handled on all eight threads. If you do not assign in this way, core 1 runs at only about 70%.
|
Storage pool
In general, a pool or storage pool is an allocated amount of capacity that jointly contains all of the data for a specified set of volumes. The system supports two types of pools:
•Parent pools receive their capacity from MDisks. All MDisks in a pool are split into extents of the same size. Volumes are created from the extents that are available in the pool. You can add MDisks to a pool at any time either to increase the number of extents that are available for new volume copies or to expand existing volume copies. The system automatically balances volume extents between the MDisks to provide the best performance to the volumes.
•Child pools, instead of being created directly from MDisks, are created from existing capacity that is allocated to a parent pool. As with parent pools, volumes can be created that specifically use the capacity that is allocated to the child pool. Child pools are similar to parent pools with similar properties and can be used for volume copy operation.
Child pools are created with fully allocated physical capacity. The capacity of the child pool must be smaller than the free capacity that is available to the parent pool. The allocated capacity of the child pool is no longer reported as the free space of its parent pool.
Consider the following general guidelines when you create or work with a child pool:
•Child pools can be created and changed with the command-line interface or through the IBM Spectrum Control when creating VMware vSphere Virtual Volumes. You can use the management GUI to view child pools and their properties.
•On systems with encryption enabled, child pools can be created to migrate existing volumes in non-encrypted pool to encrypted child pools. When you create a child pool after encryption is enabled, an encryption key is created for the child pool even when the parent pool is not encrypted. You can then use volume mirroring to migrate the volumes from the non-encrypted parent pool to the encrypted child pool.
•As with parent pools, you can specify a warning threshold that alerts you when the capacity of the child pool is reaching its upper limit. Use this threshold to ensure that access is not lost when the capacity of the child pool is close to its allocated capacity.
•Ensure that any child pools that are associated with a parent pool have enough capacity for the volumes that are in the child pool before removing MDisks from a parent pool. The system automatically migrates all extents that are used by volumes to other MDisks in the parent pool to ensure data is not lost.
•You cannot shrink the capacity of a child pool below its real capacity. The system uses reserved extents from the parent pool that use multiple extents. The system also resets the warning level when the child pool is shrunk and issues a warning if the level is reached when the capacity is shrunk.
•The system supports migrating a copy of volumes between child pools within the same parent pool or migrating a copy of a volume between a child pool and its parent pool. Migrations between a source and target child pool with different parent pools are not supported. However, you can migrate a copy of the volume from the source child pool to its parent pool. The volume copy can then be migrated from the parent pool to the parent pool of the target child pool. Finally, the volume copy can be migrated from the target parent pool to the target child pool.
To track the space that is available on an MDisk, the system divides each MDisk into chunks of equal size. These chunks are called extents and are indexed internally. Extent sizes can be 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, or 8192 MB. The choice of extent size affects the total amount of storage that is managed by the system.
At any point in time, an MDisk can be a member in one storage pool only, except for image mode volumes. Image mode provides a direct block-for-block translation from the MDisk to the volume by using virtualization. Image mode enables the virtualization of MDisks that already contain data that was written directly and not through an IBM FlashSystem V9000; rather, it was created by a direct-connected host.
Each MDisk in the storage pool is divided into several extents. The size of the extent is selected by the administrator when the storage pool is created and cannot be changed later. The size of the extent is 16 - 8192 MB.
|
Tip: A preferred practice is to use the same extent size for all storage pools in a system. This approach is a prerequisite for supporting volume migration between two storage pools. If the storage pool extent sizes are not the same, you must use volume mirroring to copy volumes between pools.
|
IBM FlashSystem V9000 limits the number of extents in a system to 222 = ~4 million. Because the number of addressable extents is limited, the total capacity of an IBM FlashSystem V9000 system depends on the extent size that is chosen by the administrator.
The capacity numbers that are specified in Table 2-6 for an IBM FlashSystem V9000 assume that all defined storage pools were created with the same extent size.
Table 2-6 Extent size-to-address ability matrix
|
Extent size (MB)
|
Maximum non thin-provisioned volume capacity in GB
|
Maximum thin-provisioned volume capacity in GB
|
Maximum MDisk capacity in GB
|
Total storage capacity manageable per system
|
|
16
|
2,048 ( 2 TB)
|
2,000
|
2,048 ( 2 TB)
|
64 TB
|
|
32
|
4,096 ( 4 TB)
|
4,000
|
4,096 ( 4 TB)
|
128 TB
|
|
64
|
8,192 ( 8 TB)
|
8,000
|
8,192 ( 8 TB)
|
256 TB
|
|
128
|
16,384 ( 16 TB)
|
16,000
|
16,384 ( 16 TB)
|
512 TB
|
|
256
|
32,768 ( 32 TB)
|
32,000
|
32,768 ( 32 TB)
|
1 PB
|
|
512
|
65,536 ( 64 TB)
|
65,000
|
65,536 ( 64 TB)
|
2 PB
|
|
1024
|
131,072 (128 TB)
|
130,000
|
131,072 ( 128 TB)
|
4 PB
|
|
2048
|
262,144 (256 TB)
|
260,000
|
262,144 ( 256 TB)
|
8 PB
|
|
4096
|
262,144 (256 TB)
|
262,144
|
524,288 ( 512 TB)
|
16 PB
|
|
8192
|
262,144 (256 TB)
|
262,144
|
1,048,576 (1,024 TB)
|
32 PB
|
|
Notes:
•The total capacity values amount assumes that all of the storage pools in the system use the same extent size.
•For most systems, a capacity of 1 - 2 PB is sufficient. A preferred practice is to use 256 MB for larger clustered systems. The default extent size is 1024 MB.
|
For more information, see IBM System Storage SAN Volume Controller and Storwize V7000 Best Practices and Performance Guidelines, SG24-7521.
Volumes
A volume is a logical disk that the system presents to attached hosts.
Hosts and application servers access volumes instead of directly connecting to flash storage modules.
You can create different types of volumes, depending on the type of topology that is configured on your system. For example, in standard topology, which is single-site configuration, you can create basic, mirrored, or custom volumes. If you have a HyperSwap topology, which is two-site configuration, you can create basic, HyperSwap, or custom volumes. For each of these volume types you can specify specific details, such as a method of capacity savings for the volumes. The system supports compression and thin provisioning to save space on volumes. With compressed volumes, data is compressed as it is written to the volume, which saves capacity on the volume. Thin provisioning creates a volume with more virtual than real capacity which allows the capacity to grow as it is needed.
Each volume copy can be one of the following types:
•Striped
Striped volumes have the following characteristics:
– A volume copy that has been striped is at the extent level. One extent is allocated, in turn, from each MDisk that is in the storage pool. For example, a storage pool that has 10 MDisks takes one extent from each MDisk. The 11th extent is taken from the first MDisk, and so on. This procedure, known as round-robin, is similar to RAID-0 striping.
– You can also supply a list of MDisks to use as the stripe set. This list can contain two or more MDisks from the storage pool. The round-robin procedure is used across the specified stripe set.
|
Attention: By default, striped volume copies are striped across all MDisks in the storage pool. If some of the MDisks are smaller than others, the extents on the smaller MDisks are used up before the larger MDisks run out of extents. Manually specifying the stripe set in this case might result in the volume copy not being created.
|
If you are unsure if enough sufficient free space is available to create a striped volume copy, select one of the following options:
– Check the free space on each MDisk in the storage pool by using the lsfreeextents command.
– Let the system automatically create the volume copy by not supplying a specific stripe set.
•Sequential
When extents are selected, they are allocated sequentially on one MDisk to create the volume copy if enough consecutive free extents are available on the chosen MDisk.
•Image
Image volumes have the following characteristics:
– Image-mode volumes are special volumes that have a direct relationship with one MDisk. If you have an MDisk that contains data that you want to merge into the clustered system, you can create an image-mode volume. When you create an image-mode volume, a direct mapping is made between extents that are on the MDisk and extents that are on the volume. The MDisk is not virtualized. The logical block address (LBA) x on the MDisk is the same as LBA x on the volume.
– When you create an image-mode volume copy, you must assign it to a storage pool. An image-mode volume copy must be at least one extent in size. The minimum size of an image-mode volume copy is the extent size of the storage pool to which it is assigned.
– The extents are managed in the same way as other volume copies. When the extents have been created, you can move the data onto other MDisks that are in the storage pool without losing access to the data. After you move one or more extents, the volume copy becomes a virtualized disk, and the mode of the MDisk changes from image to managed.
|
Attention: If you add a managed mode MDisk to a storage pool, any data on the MDisk is lost. Ensure that you create image-mode volumes from the MDisks that contain data before you start adding any MDisks to storage pools.
|
Volume states
A volume can be in one of three states:
•Online
At least one synchronized copy of the volume is online and available if both nodes in the I/O group can access the volume. A single node can access a volume only if it can access all the MDisks in the storage pool that are associated with the volume.
•Offline
The volume is offline and unavailable if both nodes in the I/O group are missing, or if none of the nodes in the I/O group that are present can access any synchronized copy of the volume. The volume can also be offline if the volume is the secondary of a Metro Mirror or Global Mirror relationship that is not synchronized. A thin-provisioned volume goes offline if a user attempts to write an amount of data that exceeds the available disk space.
•Degraded
The status of the volume is degraded if one node in the I/O group is online and the other node is either missing or cannot access any synchronized copy of the volume.
|
Note: If you have a degraded volume and all of the associated nodes and MDisks are online, call the IBM Support Center for assistance.
|
Cache modes options
You can select to have read and write operations stored in cache by specifying a cache mode. You can specify the cache mode when you create the volume. After the volume is created, you can change the cache mode. The following cache mode options are available:
•Readwrite
All read and write I/O operations that are performed by the volume are stored in cache. This is the default cache mode for all volumes.
•Readonly
All read I/O operations that are performed by the volume are stored in cache.
•None
All read and write I/O operations that are performed by the volume are not stored in cache.
Compressed volumes
When you create volumes, you can specify compression as a method to save capacity for the volume. With compressed volumes, data is compressed as it is written to disk, saving more space. Real-time Compression is licensed through the IBM FlashSystem V9000 base license option 5639-RB7.
Fully allocated volumes
A fully allocated volume contains both virtual capacity and real capacity, which are set when you create the volume.
Mirrored volumes
By using volume mirroring, a volume can have two physical copies. Each volume copy can belong to a different pool, and each copy has the same virtual capacity as the volume. In the management GUI, an asterisk (*) indicates the primary copy of the mirrored volume. The primary copy indicates the preferred volume for read requests.
HyperSwap volumes
HyperSwap volumes create copies on separate sites for systems that are configured with HyperSwap topology. Data that is written to a HyperSwap volume is automatically sent to both copies so that either site can provide access to the volume if the other site becomes unavailable. HyperSwap volumes are supported on Storwize systems that contain more than one I/O group.
Thin-provisioned volumes
When you create a volume, you can designate it to be thin-provisioned to save capacity for the volume. A thin-provisioned volume typically has higher virtual capacity than used capacity.
Virtual volumes
The system supports VMware vSphere Virtual Volumes, sometimes referred to as VVols, which allow VMware vCenter to automate the management of system objects like volumes and pools.
IBM Knowledge Center has more detailed information:
Remote Mirror and HyperSwap
Remote Mirroring Software allows the use of Metro Mirror and Global Mirror functions. This function enables you to set up a relationship between volumes on two systems, so that updates that are made by an application to one volume are mirrored on the other volume. The volumes can be in the same system or on two different systems. The function provides storage system-based data replication by using either synchronous or asynchronous data transfers over Fibre Channel communication links.
Remote Mirroring Software provides the following advanced functions:
•Metro Mirror
Maintains a fully synchronized copy at metropolitan distances (up to 300 km).
•Global Mirror
Operates asynchronously and maintains a copy at much greater distances (up to 8000 km).
•Global Mirror with change volumes
This is the term for the asynchronous remote copy of a locally and remotely created FlashCopy. All functions support VMware Site Recovery Manager to help speed disaster recovery.
IBM FlashSystem V9000 remote mirroring interoperates with other IBM FlashSystem V9000, V840, SAN Volume Controller and V7000 storage systems.
•HyperSwap
HyperSwap capability enables each volume to be presented by two I/O groups. The configuration tolerates combinations of node and site failures, using a flexible choice of host multipathing driver interoperability. In this usage, both the IBM FlashSystem V9000 control enclosure and the storage enclosure identify and carry a site attribute.
The site attributes are set during the initial cluster formation where the human operator designs the site in which the equipment is. This is then used later when performing provisioning operations to easily automate the creation of a VDisk that has multi-site protection.
The HyperSwap function uses the following capabilities:
– Spreads the nodes of the system across two sites, with storage at a third site acting as a tie-breaking quorum device.
– Locates both nodes of an I/O group in the same site. Therefore, to get a volume resiliently stored on both sites, at least two I/O groups are required.
– Uses additional system resources to support a full independent cache on each site, enabling full performance even if one site is lost. In some environments, a HyperSwap topology provides better performance than a stretched topology.
Hosts, IBM FlashSystem V9000 control enclosures, and IBM FlashSystem V9000 storage enclosures are in one of two failure domains (sites), and volumes are visible as a single object across both sites (I/O groups).
Figure 2-11 shows an overview of the HyperSwap capability.
Figure 2-11 HyperSwap overview
Figure 2-11 on page 53 shows the following components:
– Each primary volume (denoted by the “p” in the volume name) has a secondary volume (denoted by the “s” in the volume name) on the opposite I/O group.
– The secondary volumes are not mapped to the hosts.
– The dual write to the secondary volumes is handled by the IBM FlashSystem V9000 HyperSwap function, and is transparent to the hosts.
The following list summarizes the main characteristics of the HyperSwap function:
•The HyperSwap function is available on an IBM FlashSystem V9000 running software version 7.6 and later, and with two or more I/O groups.
•IBM FlashSystem V9000 software version 7.7.1 supports HyperSwap management through GUI and CLI.
•Data is stored on two sites in parallel.
•The maximum distance between sites is 300 kilometers (km).
•Two independent copies of data are maintained (four if you use additional volume mirroring to two pools in each site).
•HyperSwap uses a standard host multipathing driver.
•Cache data is retained if only one site is online.
•Automatically synchronizes and resynchronizes copies.
•Automatic host-to-storage-system path optimization, based on host site (requires Asymmetric Logical Unit Access/Target Port Groups Support (ALUA/TPGS) support from the multipathing driver.
•Stale-consistent data is retained during resynchronization for disaster recovery.
•The maximum number of highly available volumes is 1024.
•Requires a remote mirroring license for volumes. Exact license requirements can vary by product.
For additional information and examples of the HyperSwap function, see Chapter 11, “IBM HyperSwap” on page 485.
System management
The IBM FlashSystem V9000 AC2 or AC3 control enclosures in a clustered system operate as a single system. A single point of control is provided for both management and service activities. System management and error reporting are provided through an Ethernet interface to one of the AC2 or AC3 control enclosures in the system, called the configuration node. The AC2 or AC3 control enclosures run a web server and provides a CLI.
The configuration node is a role that any AC2 or AC3 control enclosures can take. If the current AC2 or AC3 control enclosures fails, a new configuration node is automatically selected from the remaining nodes. Each node also provides a CLI and web interface for initiating hardware service actions.
2.2.2 Hardware components
Each IBM FlashSystem V9000 AC2 or AC3 control enclosures is an individual server in an IBM FlashSystem V9000 clustered system (I/O group) on which the IBM FlashSystem V9000 software runs. These control enclosures are organized into I/O groups; each I/O group is made up of a pair of either AC2 or AC3 control enclosures.
An I/O group takes the storage that is presented to it by the AE2 storage enclosures as MDisks, adds these to pools, and translates the storage into logical disks (volumes) that are used by applications on the hosts. An AC2 or AC3 control enclosure is in only one I/O group and provides access to the volumes in that I/O group.
These are the core IBM FlashSystem V9000 components:
•Canisters
•Interface cards
•IBM MicroLatency modules
•Battery modules
•Power supply units
•Fan modules
Figure 2-12 shows the IBM FlashSystem V9000 front view. The 12 IBM MicroLatency modules are in the middle of the unit.
Figure 2-12 IBM FlashSystem V9000 front view
Figure 2-13 shows the IBM FlashSystem V9000 rear view. The two AC2 or AC3 control enclosures are at the top and bottom, with the AE2 storage controller in the middle. All power supply units are to the right (small units).
Figure 2-13 IBM FlashSystem V9000 rear view
Figure 2-14 shows the IBM FlashSystem V9000 AC3 control enclosure rear view.
Figure 2-14 IBM FlashSystem AC3 control enclosure rear view
|
Note: Several interface options are available for the AC2 and AC3 control enclosures, which are not shown in Figure 2-13 on page 56 and Figure 2-14 on page 56.
|
2.2.3 Power requirements
IBM FlashSystem V9000 is green data center friendly. The IBM FlashSystem V9000 building block uses only 3100 W of power under maximum load, and uses six standard single phase (100v - 240v) electrical outlets, two per AC2 orAC3 storage controller and two for the AE2 storage enclosure. Plan to attach each of the two power supplies in each of the enclosures, to separate main power supply lines.
The IBM FlashSystem V9000 maximum configuration, with four scalable building blocks and four additional AE2 storage enclosures, consumes 17900 W of power under maximum load.
|
AE2 storage enclosure: The 1300 W power supply for high-line voltage provides the AE2 storage enclosure with high power to run at maximum performance for longer durations during power supply servicing, resulting in more predictable performance under unexpected failure conditions. Optimal operation is achieved when operating between 200V - 240V (nominal). The maximum and minimum voltage ranges (Vrms) and associated high line AC ranges are as follows:
•Minimum: 180V
•Nominal: 200 - 240V
•Maximum: 265V
Using two power sources provides power redundancy. The suggestion is to place the two power supplies on different circuits.
Important: The power cord is the main power disconnect. Ensure that the socket outlets are located near the equipment and are easily accessible.
|
2.2.4 Physical specifications
The IBM FlashSystem V9000 installs in a standard 19-inch equipment rack. The IBM FlashSystem V9000 building block is 6U high and 19 inches wide. A standard data 42U 19-inch data center rack can be used to be populated with the maximum IBM FlashSystem V9000 configuration to use up to 36U.
The IBM FlashSystem V9000 has the following physical dimensions:
•IBM FlashSystem V9000 control enclosure (AC2) each:
– Width: 445 mm (17.5 in); 19-inch Rack Standard
– Depth: 746 mm (29.4 in)
– Height: 86 mm (3.4 in)
– Weight: 22.0 kg (48.4 lb)
– Airflow path: Cool air flows into the front of unit (intake) to rear of unit (exhaust)
– Heat dissipation: 3480.24 BTU per hour
•IBM FlashSystem V9000 control enclosure (AC3) each:
– Width: 447.6 mm (17.62 in); 19-inch Rack Standard
– Depth:801 mm (31.54 in)
– Height: 87.5 mm (3.44 in.)
– Weight: 23.8 kg (52.47 lb)
– Airflow path: Cool air flows into the front of unit (intake) to rear of unit (exhaust)
•Heat dissipation: 3480.24 BTU per hour
•IBM FlashSystem V9000 storage enclosure (AE2):
– Width: 445 mm (17.6 in); 19-inch rack standard
– Depth: 761 mm (29.96 in)
– Height: 86.2 mm (3.39 in)
– Weight (maximum configuration is 12 flash modules): 34 kg (75 lb)
– Airflow path: Cool air flows into the front of unit (intake) to rear of unit (exhaust)
– Heat dissipation: 1194 BTU (maximum configuration RAID 5)
Table 2-7 lists the specifications for the configuration of IBM FlashSystem V9000.
Table 2-7 IBM FlashSystem V9000 configuration specifications
|
IBM FlashSystem V9000
|
|
|
Models
|
9846/8-AC3 and 9846/8-AE2
|
|
Flash type
|
MLC enhance by IBM
|
|
Flash module configuration
|
4 x 1.2 TB, 6 x 1.2 TB, 8 x 1.2 TB, 10 x 1.2 TB, 12 x 1.2 TB, 6 x 2.9 TB, 8 x 2.9 TB, 10 x 2.9 TB, 12 x 2.9 TB, 6 x 5.7 TB, 8 x 5.7 TB, 10 x 5.7 TB, 12 x 5.7 TB
|
|
Maximum internal flash capacity
|
•Scalable from 2.2 TB (usable) up to 456 TB with full scale-out of 8 control and 8 storage enclosures (8x8).
•From 12 TB to 2.2 PB with full scale-out of control enclosures and storage enclosures (at 80% reduction with Real-time Compression).
|
|
Maximum expansion enclosure capacity
|
•Up to 80 standard expansion enclosures (up to 20 per controller pair)
•A 9846/9848-12F SAS expansion enclosure contains up to 12 3.5-inch NearLine SAS HDDs (8 TB or 10 TB)
•Up to 9.6PB raw capacity using NL-SAS HDDs ( 9846/8-12F )
•A 9846/9848-24F SAS expansion enclosure contains up to 24 2.5-inch SSDs. (1.9 TB, 3,8 TB, 7.7 TB, 15.4 TB)
•Up to 29.4 PB raw capacity using SSDs ( 9846/8-24F )
•Up to 32 high density expansion enclosures (up to 8 per controller pair)
•9846/8-92F SAS expansion enclosure contains up to 92 3.5-inch or 2.5-inch NL-SAS or SSD drives. (8 TB, 10 TB, 1.9 TB, 3,8 TB, 7.7 TB, 15.4 TB)
•Up 29.4 PB of raw NL-SAS HDD capacity and 32PB of raw SSD capacity is supported. (9846/8-92F)
|
|
Maximum external storage capacity
|
•Up to 32 PB usable capacity (requires External Virtualization).
|
|
Maximum Performance: Per building block (100% read, cache miss)
|
|
|
Latency (4 K)
|
180 µs
|
|
IOPS (4 K)
|
750,000
|
|
Bandwidth (256 K)
|
9.5 GBps
|
|
Maximum Performance: Scaled out (100% read, fully scaled out with four building blocks)
|
|
|
Minimum Latency (4 K)
|
180 µs
|
|
IOPS (4 K)
|
3,000,000
|
|
Bandwidth (256 K)
|
68 GBps
|
|
Data reduction IOPS (4 K)
|
1,200,000
|
|
Reliability, availability, and serviceability (RAS) features
|
•Two-dimensional flash RAID
•Module-level IBM Variable Stripe RAID
•System-level RAID 5 across modules
•Hot-swappable flash modules
•Tool-less module installation/replacement
•Concurrent code load
•Redundant and hot-swappable components
|
|
Supported platforms
|
Information about servers, operating systems, host adapters, and connectivity products that are supported by FlashSystem products is available at the SSIC web page:
|
|
Encryption
|
Data-at-rest AES-XTS 256
|
|
IBM FlashSystem V9000 host connectivity options per building block
|
•32 x 16/8/4 Gb Fibre Channel
•8 x 10 Gb Fibre Channel over Ethernet (FCoE)
•8 x 10 Gb iSCSI
|
|
Virtualization software model
|
5639-RB7
|
|
Tiered Solution Models
|
9846/8-12F, 9846/8-24F, 9846/8-92F
|
|
Shared symmetric multiprocessing (SMP) processor configuration
|
Two Intel Xeon E5 v4 series 8-core 3.2 GHz processors
|
|
Controller memory
|
64 GB standard, up to 256 GB option (per controller and supported in future releases of code)
|
|
Dimensions (height x width x depth)
|
6U x 445 mm x 761 mm (6U x 17.5 in. x 29.96 in.)
|
|
Weight (V9000 single block)
|
78 kg (171.8 lb.) fully loaded
|
|
Weight (Expansion enclosure Model 12F)
|
Fully configured: 26.7 kg (58.76 lb)
|
|
Weight (Expansion enclosure Model 24F)
|
Fully configured: 27.3 kg (60.19 lb)
|
2.3 Control enclosure (AC2)
The IBM FlashSystem V9000 AC2 control enclosures are based on IBM System x server technology. Each AC2 control enclosure has the following key hardware features:
•Two Intel Xeon E5 v2 Series eight-core processors with 64 GB memory
•16 Gb FC, 8 Gb FC, 10 Gb Ethernet, and 1 Gb Ethernet I/O ports for FC, iSCSI, and FCoE connectivity
•Hardware-assisted compression acceleration (optional feature)
•Two integrated battery units
•2U, 19-inch rack mount enclosure with ac power supplies
The AC2 control inclosure includes three 1 Gb Ethernet ports standard for iSCSI connectivity. It can be configured with up to four I/O adapter features providing up to eight 16 Gb FC ports, up to twelve 8 Gb FC ports, or up to four 10 Gb Ethernet (iSCSI or Fibre Channel over Ethernet (FCoE)) ports. See the IBM FlashSystem V9000 documentation:
Real-time Compression workloads can benefit from the IBM FlashSystem V9000 with two 8-core processors with 64 GB of memory (total system memory). Compression workloads can also benefit from the hardware-assisted acceleration offered by the addition of two Compression Acceleration Cards.
The front panel unit contains a dual-battery pack with its backplane acting as an uninterruptible power supply to the AC2. Batteries are fully redundant and hot-swappable. An AC2 is able to operate with one healthy battery, however it is logged and the control enclosure becomes degraded, but it still continues I/O operations.
The AC2 control enclosure hardware layout is presented in Figure 2-15.
Figure 2-15 AC2 control enclosure
Figure 2-16 illustrates the back view of the AC2 control enclosure.
Figure 2-16 Rear view of AC2 control enclosure
2.3.1 I/O connectivity
IBM FlashSystem V9000 offers various options of I/O cards for installation and configuration. The 2U rack-mount form factor of the AC2 enables the control enclosure to accommodate up to six PCIe Gen3 cards for I/O connectivity or compression support. The rear view of the AC2 is shown in Figure 2-17.
Figure 2-17 Rear view of AC2 control enclosure
Slots 4 - 6 are internally attached to processor 2, and are available with both processors and 64 GB of total memory installed in the AC2 control enclosure node.
The I/O card installation options for an IBM FlashSystem V9000 fixed building block are outlined in Table 2-8.
Table 2-8 Layout for IBM FlashSystem V9000 fixed building block
|
Top of node
|
|
|
Processor 1 attachment
|
Processor 2 attachment
|
|
Slot 1: I/O card (8 Gbps)
|
Slot 4: Compression Acceleration Card (optional)
|
|
Slot 2: I/O card (8 Gbps)
|
Slot 5: I/O card (8 Gbps)
|
|
Slot 3: Not used
|
Slot 6: Compression Acceleration Card (optional)
|
The I/O card installation options for an IBM FlashSystem V9000 16 Gbps scalable building block are outlined in Table 2-9.
Table 2-9 Layout for IBM FlashSystem V9000 16 Gbps scalable building block
|
Top of node
|
|
|
Processor 1 attachment
|
Processor 2 attachment
|
|
Slot 1: I/O card (16 Gbps)
|
Slot 4: Compression Acceleration Card (optional)
|
|
Slot 2: Adapter for host connections only. Options include:
•Four port 8 Gbps FC adapter
•Two port 16 Gbps FC adapter
•Four port 10 Gbps Ethernet adapter (FCoE or iSCSI)
•Four port SAS expansion enclosure adapter
|
Slot 5: Adapter for host connections only. Options include:
•Four port 8 Gbps FC adapter
•Two port 16 Gbps FC adapter
•Four port 10 Gbps Ethernet adapter (FCoE or iSCSI)
|
|
Slot 3: I/O card (16 Gbps)
|
Slot 6: Compression Acceleration Card (optional)
|
|
Important: For IBM FlashSystem V9000 V7.5 release of code, the AC2 supports direct FC adapter connection to the hosts at 16 Gbps without a switch. At the time of the writing of this book, the only restriction is that AIX hosts are not supported with this direct connect.
|
For details, see Chapter 6, “Installation and configuration” on page 231 and 6.3, “Installing the hardware” on page 233.
2.3.2 Compression Acceleration Card
Compressed volumes are a special type of volume where data is compressed as it is written to disk, saving more space. The AC2 control enclosures must have two Compression Acceleration Cards installed to use compression. Enabling compression on the IBM FlashSystem V9000 does not affect non-compressed host to disk I/O performance. Figure 2-18 shows the Compression Acceleration Card and its possible placement in the AC2 control enclosures.
Figure 2-18 Placement of Compression Acceleration Cards
|
Remember: To use compression, two Compression Acceleration Cards are compulsory for each AC2 control enclosure in an IBM FlashSystem V9000.
|
For an IBM FlashSystem V9000 with no Compression Acceleration Cards, an attempt to create a compressed volume fails.
A fully equipped IBM FlashSystem V9000 building block with four compression accelerators supports up to 512 compressed volumes.
2.3.3 Technician port
The purpose and key benefit of the IBM FlashSystem V9000 technician port is to simplify and ease the initial basic configuration of the system by the local administrator or by service personnel. The technician port is marked with the letter “T” (Ethernet port 4) as depicted in Figure 2-19.
Figure 2-19 Location of technician port
To initialize a new system, you must connect a personal computer to the technician port on the rear of one of the AC2 control enclosures in the solution and run the initialization wizard. This port runs a Dynamic Host Configuration Protocol (DHCP) server to facilitate service and maintenance that is ready to use in lieu of the front panel. Be sure that your computer has DHCP enabled, otherwise manually provide these settings:
•IP address set to 192.168.0.2
•Network mask set to 255.255.255.0,
•Gateway set to 192.168.0.1.
|
Attention: Never connect the technician port to a switch. If a switch is detected, the technician port connection might shut down, causing an AC2 error 746 in the log.
|
If the node has Candidate status when you open the web browser, the initialization wizard is displayed. Otherwise, the service assistant interface is displayed. The procedure of the initial configuration is described in Chapter 6, “Installation and configuration” on page 231.
2.3.4 Battery backup
The AC2 control enclosure has two hot-swappable batteries in the front of the enclosure, with the battery backplane at the back of battery drawers. See Figure 2-20 for details.
Figure 2-20 Position of batteries in AC2 control enclosure
The AC2 battery units provide these items:
•Dual batteries per AC2.
•They are hot-swappable.
•Designed as redundant within as AC2 control enclosure.
•Batteries incorporate a test load capability.
•Each battery has its own fault LED indicator.
The AC2 control enclosure is designed for two batteries, but continues to operate on a single battery. To achieve maximum redundancy and to get the full life rating of the cells, the system needs to run with both batteries. Running with a single battery results in almost a full discharge and places a higher discharge current on the cells, which leads to a reduced capacity after several cycles. Running with just one battery is a degraded state and a node error event is logged to ensure the missing or failed battery is replaced.
An AC2 control enclosure is able to continue operation with one failed battery, although after an AC power failure, the node might have to wait for the battery to charge before resuming host I/O operation.
2.4 Control enclosure (AC3)
IBM FlashSystem V9000 control enclosure model AC3 is a component of the IBM FlashSystem V9000 storage system that provides increased performance and additional storage capacity.
The IBM FlashSystem V9000 control enclosure is a purpose-built 2U 19-inch rack-mount enclosure with two AC power supplies, two backup batteries, and dual SSD boot drives. The control enclosure provides up to eight 16 Gb Fibre Channel ports to connect to IBM FlashSystem V9000 storage enclosures, either directly (with the fixed building block) or through SAN switches (with the scalable building block).
Figure 2-21 shows the front view of the IBM FlashSystem V9000 AC3 controller.
Figure 2-21 IBM FlashSystem V9000 AC3 control enclosure front view
IBM FlashSystem V9000 control enclosure model AC3 has the following features:
•Two eight-core processors with 64 GB memory standard, and options to increase memory up to 256 GB.
•16 Gb Fibre Channel (FC) and 10 Gb iSCSI and Fibre Channel over Ethernet (FCoE) connectivity options.
•Hardware-assisted compression acceleration for Real-time Compression workloads.
•Capability for adding into existing clustered systems with previous generation IBM FlashSystem V9000 control enclosures.
•Up to 20 SAS attached expansion enclosures are supported per IBM FlashSystem V9000 controller pair, providing up to 480 HDD type drives with expansion Model 24F and up to 240 low cost SSD drives with expansion Model 12F.
•Up to eight high-density enclosures are supported per IBM FlashSystem V9000 controller pair, providing up to 736 HDD or SDD drives with expansion Model 92F.
Figure 2-22 shows the IBM FlashSystem V9000 AC3 control enclosure rear view.
Figure 2-22 Rear view of IBM FlashSystem V9000 AC3 control enclosure
IBM FlashSystem V9000 Control Enclosure Model AC3 requires IBM FlashSystem V9000 Software V7.7.1 or later for operation. Use of the software is entitled through the IBM FlashSystem V9000 base license option 5639-RB7.
Battery backup units and front view information
Figure 2-23 shows the IBM FlashSystem AC3 control enclosure front view.
Figure 2-23 AC3 control enclosure front view
The numbers in Figure 2-23 refer to the following items:
1. Battery backup unit 1
2. Battery backup unit 2
3. SSDs drive slots from 1 to 8
4. Control enclosure LEDs
Battery backup unit considerations:
•Each AC3 control enclosure in the system contains two batteries that provide backup power to that enclosure. The battery within a control enclosure supplies power only to that control enclosure.
•If power to an AC3 control enclosure node is lost, the system starts to save critical data after a 5-second wait. The saving of critical data runs to completion, even if power is restored during this time. If the power outage is shorter than 5 seconds, the battery continues to support the node and critical data is not saved.
The canister batteries in an AC3 control enclosure periodically discharge and charge again to maintain the battery life.
Figure 2-24 shows AC3 control enclosure detailed rear view information.
Figure 2-24 AC3 control enclosure detailed rear view
AC3 control enclosure internal view
The internal components of the AC3 control enclosure (for example, a fan, CPU, DIMM, or PCIe adapter) only need to be accessed if they must be replaced.
Figure 2-25 shows the location and ID numbers of the fan modules in the AC3 control enclosure, in this case fan IDs from 1 to 6.
Figure 2-25 AC3 control enclosure view from the front of the unit with fan IDs 1 - 6
Figure 2-26 shows the location and some of the slot numbers of the DIMM modules, in this case bank slots A0 and C0. The black air baffle was removed to reveal the CPU and DIMM.
Figure 2-26 AC3 control enclosure internal view from the rear of the unit, with 4 DIMM installed using bank0
Figure 2-27 shows the location and ID numbers of the processors, in this case processor 0 and processor 1. The black air baffle was removed to reveal the CPU and the DIMM.
Figure 2-27 AC3 control enclosure internal view from the rear of the unit, with Processor IDs 0 and 1
AC3 control enclosure features
The AC3 control enclosures have the following features:
•2U form factor.
•Redundant hardware components and modules.
•Hot-replaceable modules.
•Two (mirrored) boot drives.
•Integrated Ethernet ports.
•PCIe adapter slots.
•Boot drives: 2 SSDs.
•Dual redundant power supplies.
•Dual redundant batteries.
•A dedicated technician port to initialize or service the system.
•A 19-inch rack-mounted enclosure.
•Two eight-core processors.
•64 GB base memory per processor. Optionally, by adding 64 GB of memory, the processor can support 128 GB, 192 GB, or 256 GB of memory.
•Eight small form factor (SFF) drive bays at the front of the control enclosure.
•Support for various optional host adapter cards, including these:
– 4-port 16 Gbps Fibre Channel adapter cards
– 4-port 10 Gbps Fibre Channel over Ethernet (FCoE) adapter cards for host attachment
– 4-port 12 Gbps SAS cards to attach to expansion enclosures
•Support for iSCSI host attachment (10 Gbps Ethernet).
•Support for expansion enclosures to attach additional drives:
– 9846/9848-24F to house up to 24 SFF flash drives
– 9846/9848-12F to house up to 12 large form factor LFF HDD
– 9846/9848-92F to house up to 92 SFF or LFF drives
•Support for optional Compression Accelerator cards for IBM Real-time Compression.
•Support for up to four adapter cards, including:
– 8 Gbps and 16 Gbps Fibre Channel adapter cards
– 10 Gbps iSCSI and 10 Gbps Fibre Channel over Ethernet adapter cards for host attachment
– (Optional) IBM Real-time Compression
I/O connectivity
IBM FlashSystem V9000 offers various options of I/O cards for installation and configuration. The 2U rack-mount form factor of the AC3 control enclosure enables it to accommodate up to seven available PCIe Gen3 cards for I/O connectivity or compression support. The AC3 control enclosure has eight physical PCIe adapter slots, but slot 1 is not supported for use. Therefore, up to seven PCIe adapters can be installed in the enclosure.
Figure 2-28 shows the AC3 control enclosure PCIe slot numbers 1 - 8.
Figure 2-28 AC3 control enclosure PCIe slot numbers 1 - 8
Table 2-10 shows the PCIe slot number and adapter type.
Table 2-10 AC3 control enclosure PCIe slot number and adapter type
|
PCIe slot
|
Adapter type
|
|
1
|
Not supported for use
|
|
2
|
SAS
|
|
3
|
Fibre Channel or Ethernet
|
|
4
|
Fibre Channel or Ethernet
|
|
5
|
Compression accelerator
|
|
6
|
Fibre Channel or Ethernet
|
|
7
|
Fibre Channel or Ethernet
|
|
8
|
Compression accelerator
|
Integrated Ethernet port
Ethernet ports that can be used for management connections are provided on the system board, accessed from the rear of the control enclosure.
Figure 2-29 shows the location and numbers of the integrated Ethernet ports.
Technician port
The purpose and key benefit of the IBM FlashSystem V9000 technician port is to simplify and ease the initial basic configuration of the system by the local administrator or by service personnel. The technician port is marked with the letter “T” (Ethernet port 4) as depicted in Figure 2-29.
Figure 2-29 AC3 control enclosure rear view, technician port
AC3 control enclosure Fibre Channel port numbers and WWPNs
The Fibre Channel (FC) port numbers and worldwide port names (WWPNs) depend on the type of adapters that are installed in the control enclosure.
The WWPNs are assigned as follows: 500507680f<P>XXXX
In that assignment, the values have the following meanings:
5 The IEEE Network Address Authority field format number. This value identifies a registered port name.
005076 The Organizationally Unique Identifier (OUI) for IBM.
80f The product unique identifier for an AC3 control enclosure.
<P> The port ID number.
XXXX A unique number for each AC3 control enclosure in the system.
Figure 2-30 shows the FC port numbers for the IBM FlashSystem AC3 control enclosure.
Figure 2-30 AC3 control enclosure FC port numbers
Table 2-11 lists the WWPNs for each FC port in an AC3 control enclosure.
Table 2-11 Standard WWPNs
|
ID
|
Slot
|
Port
|
Primary
|
Host
|
|
1
|
3
|
1
|
500507680f01xxxx
|
500507680f81xxxx
|
|
2
|
3
|
2
|
500507680f02xxxx
|
500507680f82xxxx
|
|
3
|
3
|
3
|
500507680f03xxxx
|
500507680f83xxxx
|
|
4
|
3
|
4
|
500507680f04xxxx
|
500507680f84xxxx
|
|
5
|
4
|
1
|
500507680f05xxxx
|
500507680f85xxxx
|
|
6
|
4
|
2
|
500507680f06xxxx
|
500507680f86xxxx
|
|
7
|
4
|
3
|
500507680f07xxxx
|
500507680f87xxxx
|
|
8
|
4
|
4
|
500507680f08xxxx
|
500507680f88xxxx
|
|
9
|
6
|
1
|
500507680f09xxxx
|
500507680f89xxxx
|
|
10
|
6
|
2
|
500507680f0axxxx
|
500507680f8axxxx
|
|
11
|
6
|
3
|
500507680f0bxxxx
|
500507680f8bxxxx
|
|
12
|
6
|
4
|
500507680f0cxxxx
|
500507680f8cxxxx
|
|
13
|
7
|
1
|
500507680f0dxxxx
|
500507680f8dxxxx
|
|
14
|
7
|
2
|
500507680f0exxxx
|
500507680f8exxxx
|
|
15
|
7
|
3
|
500507680f0fxxxx
|
500507680f8fxxxx
|
|
16
|
7
|
4
|
500507680f10xxxx
|
500507680f90xxxx
|
2.5 Storage enclosure (AE2)
The AE2 storage enclosure components include flash modules, battery modules, and power supplies.
Each IBM FlashSystem AE2 storage enclosure contains two fully redundant canisters. The fan modules are at the bottom and the interface cards are at the top. Each canister contains a RAID controller, two interface cards, and a management controller with an associated 1 Gbps Ethernet port. Each canister also has a USB port and two hot-swappable fan modules.
Figure 2-31 shows the components of the AE2 storage enclosure. One of the two canisters was removed, and now you can see the interface cards and fan modules. The power supply unit to the right of the fans provides redundant power to the system. All components are concurrently maintainable except the midplane and the power interposer, which has no active components. All external connections are from the rear of the system.
Figure 2-31 AE2 storage enclosure components
To maintain redundancy, the canisters are hot-swappable. If any of the components (except the fans) within a canister fail, the entire canister is replaced as a unit. Both fan modules in each canister are hot-swappable.
|
Notes:
•If either interface card in a canister fails, the entire canister (minus the fans) must be replaced as an entire unit. When replacing hardware in the AE2 storage enclosure, follow the DMP that is accessible through the GUI.
•For more details about the IBM FlashSystem canisters, including canister state LEDs, see the IBM FlashSystem V9000 web page at IBM Knowledge Center:
|
2.5.1 Interface cards
The AE2 storage enclosure supports the following interface cards:
•Fibre Channel 8 Gbps
•Fibre Channel 16 Gbps
Figure 2-32 shows a four-port FC interface card, which is used for 16 Gbps FC (only two ports used), and 8 Gbps (four ports used).
Figure 2-32 AE2 storage enclosure FC interface card
Support of 16 Gbps Fibre Channel
The AE2 storage controller supports the new 16 Gbps FC connection speed through the standard FC interface card. The following rules apply to supporting 16 Gbps FC on the AE2:
•If using 16 Gbps FC, only two (of the four) ports on the FC modules can be used. The two leftmost ports (1 and 2) on each interface card are used for 16 Gbps support. The two right-most ports (3 and 4) are disabled when 16 Gbps is sensed on any port in the AE2.
•If using 16 Gbps FC, the interface is configured as either 16 Gb FC (only two ports active), or 8 Gb FC (4 ports active). This is configured at the factory and is not changeable by the client.
2.5.2 MicroLatency modules
The IBM FlashSystem AE2 storage enclosure supports up to 12 IBM MicroLatency modules, accessible from the enclosure front panel. Each IBM MicroLatency module has a usable capacity of either 1.06 TiB (1.2 TB), 2.62 TiB (2.9 TB), or 5.24 TiB (5.7 TB) of flash memory. IBM MicroLatency modules without the daughterboard are either half-populated with 1.06 TiB (1.2 TB) or fully populated with 2.62 TiB (2.9 TB). The optional daughterboard adds another 2.62 TiB (2.9 TB) for a total of 5.24 TiB (5.7 TB).
Figure 2-33 illustrates an AE2 storage enclosure MicroLatency module (base unit and optional daughterboard).
Figure 2-33 AE2 storage enclosure MicroLatency module
|
Note: All MicroLatency modules in the AE2 storage enclosure must be ordered as 1.2 TB, 2.9 TB, or 5.7 TB. IBM MicroLatency modules types cannot be mixed in a single enclosure. The daughterboard cannot be added after deployment.
|
The maximum storage capacity of the IBM FlashSystem V9000 is based on the following factor:
•In a RAID 5 configuration, one IBM MicroLatency module is reserved as an active spare, and capacity equivalent to one module is used to implement a distributed parity algorithm. Therefore, the maximum usable capacity of a RAID 5 configuration is 57 TB (51.8 TiB), which is 10 MicroLatency modules x 5.7 TB (5.184 TiB).
IBM MicroLatency modules are installed in the AE2 storage enclosure based on the following configuration guidelines:
•A minimum of four MicroLatency modules must be installed in the system. RAID 5 is the only supported configuration of the IBM FlashSystem V9000. RAID 10 is not supported on the IBM FlashSystem V9000.
•The system supports configurations of 4, 6, 8, 10, and 12 MicroLatency modules in RAID 5.
•All MicroLatency modules that are installed in the enclosure must be identical in capacity and type.
•For optimal airflow and cooling, if fewer than 12 MicroLatency modules are installed in the enclosure, populate the module bays beginning in the center of the slots and adding on either side until all 12 slots are populated.
Table 2-12 lists suggestions to populate MicroLatency module bays.
Table 2-12 Supported MicroLatency module configurations
|
No. of installed
flash modules1
|
Flash
mod.
slot 1
|
Flash
mod.
slot 2
|
Flash
mod.
slot 3
|
Flash
mod.
slot 4
|
Flash
mod.
slot 5
|
Flash
mod.
slot 6
|
Flash
mod.
slot 7
|
Flash
mod.
slot 8
|
Flash
mod.
slot 9
|
Flash
mod.
slot 10
|
Flash
mod.
slot 11
|
Flash
mod.
slot 12
|
|
Four
|
|
|
|
|
X
|
X
|
X
|
X
|
|
|
|
|
|
Six
|
|
|
|
X
|
X
|
X
|
X
|
X
|
X
|
|
|
|
|
Eight
|
|
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
|
|
Ten
|
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
|
Twelve
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
1 RAID 5 is supported with configurations of 4, 6, 8, 10, and 12 MicroLatency modules.
|
Notes:
•If fewer than 12 modules are installed, module blanks must be installed in the empty bays to maintain cooling airflow in the system enclosure.
•During system setup, the system automatically configures RAID settings based on the number of flash modules in the system.
•All MicroLatency modules installed in the enclosure must be identical in capacity and type.
Important:
•MicroLatency modules are hot swappable. However, to replace a module, you must power down the MicroLatency module by using the management GUI before you remove and replace the module. This service action does not affect the active logical unit numbers (LUNs), and I/O to the connected hosts can continue while the MicroLatency module is replaced. Be sure to follow the DMP from the IBM FlashSystem V9000 GUI before any hardware replacement.
•The suggestion is for the AE2 storage enclosure to remain powered on, or be powered on periodically, to retain array consistency. The AE2 storage enclosure can be safely powered down for up to 90 days, in temperatures up to 40 degrees C. Although the MicroLatency modules retain data if the enclosure is temporarily disconnected from power, if the system is powered off for a period of time exceeding 90 days, data might be lost.
•FlashSystem V840 MicroLatency modules are not supported in the IBM FlashSystem V9000 AE2 storage enclosure and installation should not be attempted.
|
2.5.3 Battery modules
The AE2 storage enclosure contains two hot-swappable battery modules. The function of the battery modules is to ensure that the system is gracefully shut down (write buffer fully flushed and synchronized) when AC power is lost to the unit. The battery modules are hot-swappable. Figure 2-34 shows Battery Module 1, which is in the leftmost front of the AE2 storage enclosure. A battery module can be hot-swapped without software intervention; however, be sure to follow the DMP from the IBM FlashSystem V9000 GUI before any hardware replacement.
Battery reconditioning
The AE2 storage enclosure has a battery reconditioning feature that calibrates the gauge that reports the amount of charge on the batteries. On systems that have been installed for 10 months or more, or systems that have experienced several power outages, the recommendation to run “battery reconditioning” is displayed in the Event Log shortly after upgrading. For more information, see the IBM FlashSystem V9000 web page at IBM Knowledge Center:
Figure 2-34 AE2 storage enclosure battery module 1
Power supply units
The AE2 contains two hot-swappable power supply units. The system can remain fully online if one of the power supply units fails. The power supply units are accessible from the rear of the unit and are fully hot swappable.
Figure 2-35 on page 76 shows the two hot-swappable power supply units. The IBM FlashSystem V9000 GUI and alerting systems (SNMP and so on) will report a power supply fault. The power supply can be hot-swapped without software intervention; however, be sure to follow the DMP from the IBM FlashSystem V9000 GUI before any hardware replacement.
Figure 2-35 AE2 storage enclosure hot swappable power supply units
Fan modules
The AE2 contains four hot-swappable fan modules. Each canister holds two hot swappable fan modules. Each fan module contains two fans. The system can remain fully online if one of the fan modules fails. The fan modules are accessible from the rear of the unit (in each canister) and are fully hot swappable.
Figure 2-36 shows a hot-swappable fan module. The IBM FlashSystem V9000 GUI and alerting systems (SNMP and so on) will report a fan module fault. The fan module can be hot-swapped without software intervention; however, be sure to follow the DMP from the IBM FlashSystem V9000 GUI before any hardware replacement.
Figure 2-36 AE2 storage enclosure fan module
2.6 Expansion enclosures (12F, 24F, 92F)
Three expansion enclosures models are available:
•Model 12F
•Model 24F
•Model 92F
To support a flash-optimized tiered storage configuration for mixed workloads, up to 20 optional 9846/9848-12F or 9846/9848-24F, or up to eight 9846/9848-92F SAS expansion enclosures can be connected to each control enclosure pair (building block) in the system.
The maximum capacity of the expansion enclosures is as follows:
•A 9846/9848-12F standard expansion enclosure contains up to 12 3.5-inch nearline SAS drives and scales up to 9.6 PB raw capacity with 80 enclosures
– Model 12F supports twelve 8 TB or 10 TB SAS 3.5-inch nearline SAS drives.
– 20 standard expansion enclosures supported per control enclosure pair
– 9.6 PB = 20 expansion enclosures * 4 control enclosure pairs * 12 drives * 10 TB
•A 9846/9848-24F standard expansion enclosure contains up to 24 2.5-inch read-intensive flash drives and up to 29.4 PB raw capacity with 80 enclosures.
– Model 24F supports twenty-four 1.92 TB, 3.84 TB, 7.68 TB, or 15.36 TB flash drives
– 20 standard expansion enclosures supported per control enclosure pair
– 29.4PB = 20 expansion enclosures * 4 control enclosure pairs * 24 drives * 15.36 TB
•A 9846/9848-92F high density expansion enclosure contains up to 92 3.5-inch nearline SAS drives or 92 2.5-inch read-intensive flash drives and up to 29.4 PB raw capacity using NL-SAS HDDs or 32 PB raw capacity using SSDs:
– Model 92F supports ninety-two 8 TB or 10 TB NL-SAS. It also supports ninety-two 1.92 TB, 3.84 TB, 7.68 TB, or 15.36 TB flash drives.
– Eight high-density expansion enclosures are supported per control enclosure pair.
– Using NL-SAS HDDs:
29.4 PB of raw capacity = 8 expansion enclosures * 4 control enclosure pairs * 92 drives * 10 TB
– Using SSDs:
32 PB of raw capacity = 8 expansion enclosures * 4 control enclosure pairs
* 92 drives * 10 TB
* 92 drives * 10 TB
There are enough physical slots to technically house more than 32 PB of capacity, but only up to 32 PB are supported.
|
Note: To support SAS expansion enclosures, an AH13 - SAS Enclosure Attach adapter card must be installed in expansion slot 2 of each AC3 control enclosure in the building block (only for version 7.7.1 or later). The expansion enclosure Model 9846/9848-92F can be used on only version 7.8 or later.
|
Expansion enclosure model 12F
IBM expansion enclosure model 12F offers new tiering options and up to twelve slots for 3.5-inch low cost NL-SAS HDDs.
High capacity nearline drives enables high value tiered storage with hot data stored in flash and warm data on lower cost nearline SAS HDDs all managed by IBM Easy Tier. 8 TB or 10 TB SAS 3.5-inch nearline drives are available for IBM FlashSystem V9000 LFF storage expansion enclosure model 12F for a maximum of 9.6 PB raw capacity with four control enclosure pairs.
Figure 2-37 shows the IBM FlashSystem expansion enclosure model 12F.
Figure 2-37 Front view of expansion enclosure model 12F
Expansion enclosure model 24F
IBM expansion enclosure model 24F offers new tiering options and up to 24 slots for 2.5-inch low-cost read-intensive SAS flash drives.
The 1.92 TB, 3.84 TB, 7.68 TB, 15.36 TB SAS 2.5-inch SSD flash drive options are available for IBM FlashSystem V9000 SFF expansion enclosure model 24F for a maximum of 29.4 PB raw capacity with four control enclosure pairs.
Figure 2-38 shows the front view of the IBM FlashSystem expansion enclosure model 24F.
Figure 2-38 Front view of expansion enclosure model 24F
Both models of expansion enclosures have the same common features:
•Two expansion canisters
•12 Gb SAS ports for attachment to the IBM FlashSystem V9000 controllers
•2U, 19-inch rack-mount enclosure with AC power supplies
Figure 2-39 shows the back of IBM FlashSystem expansion enclosure models 12F and 24F.
Figure 2-39 Rear view of the expansion enclosure models 12F and 24F
Figure 2-40 shows the maximum possible configuration with a single building block using a combination of native IBM FlashSystem V9000 flash storage expansion enclosures and SAS attached storage expansion enclosures.
Figure 2-40 Maximum configuration with single building block for 12F and 24F expansion enclosures
Expansion enclosure model 92F
Figure 2-41 shows the front view of the IBM FlashSystem expansion enclosure model 92F.
Figure 2-41 Front view of the expansion enclosure model 9848-92F
Figure 2-42 shows the rear view of the IBM FlashSystem expansion enclosure model 92F.
Figure 2-42 Rear view of the expansion enclosure model 9848-92F
Figure 2-43 shows the inside of the 9848-92F expansion enclosure.
Figure 2-43 Inside view of the 9848-92F expansion enclosure
IBM FlashSystem V9000 HD expansion enclosure model 92F delivers increased storage density and capacity in a cost-efficient way.
IBM FlashSystem HD expansion enclosure model 92F offers the following features:
•5U, 19-inch rack mount enclosure with slide rail and cable management assembly
•Support for up to ninety-two 3.5-inch large-form factor (LFF) 12 Gbps SAS top-loading drives or ninety-two 2.5-inch small-form factor (SFF) drives
•High-capacity nearline disk drives, and flash drive support
•High-capacity, archival-class nearline disk drives in 8 TB and 10 TB 7, 7200 rpm
•Flash drives in 1.92 TB, 3.84 TB, 7.68 TB, and 15.36 TB
•Redundant 200 - 240VA power supplies (new PDU power cord required)
Up to eight HD expansion enclosures are supported per IBM FlashSystem V9000 building block, providing up to 736 drives with expansion model 92F for up to 29.4 PB of raw SAS HDD or 32 PB SSD capacity in each building block. With four building blocks a maximum of 32 high-density expansion enclosures can be attached giving a maximum 29.4 PB of raw NL-SAS capacity and 32PB of raw SSD capacity is supported.
2.6.1 SAS expansion enclosures intermix
IBM FlashSystem V9000 control enclosures with the SAS enclosure attach adapters support up to two SAS chains of expansion enclosures. The SAS chains have limits depending on the number of standard and dense expansion enclosures. Table 2-13 shows the allowed intermix of expansion enclosures per SAS chain.
Table 2-13 Number of expansion enclosures allowed per SAS chain; two chains per building block
|
Number of expansion
enclosures
|
Config 1
|
Config 2
|
Config 3
|
Config 4
|
Config 5
|
|
1
|
Standard
|
Dense
|
Dense
|
Dense
|
Dense
|
|
2
|
Standard
|
Standard
|
Dense
|
Dense
|
Dense
|
|
3
|
Standard
|
Standard
|
Standard
|
Dense
|
Dense
|
|
4
|
Standard
|
Standard
|
Standard
|
Standard
|
Dense
|
|
5
|
Standard
|
Standard
|
Standard
|
Standard
|
|
|
6
|
Standard
|
Standard
|
Standard
|
|
|
|
7
|
Standard
|
Standard
|
Standard
|
|
|
|
8
|
Standard
|
Standard
|
|
|
|
|
9
|
Standard
|
|
|
|
|
|
10
|
Standard
|
|
|
|
|
|
Note: Standard refers to either Model 12F LFF or Model 24F SFF expansion enclosures. Dense refers to the Model 92F HD expansion enclosure.
|
|||||
Table 2-14 shows the allowed intermix of expansion enclosures per building block (control enclosure pair). Any cell in the table is valid but you must balance the enclosures across both chains up to the maximum limits specified per chain.
Table 2-14 Number of expansion enclosure types that can be intermixed on one building block; two SAS chains
|
Number
of expansion
enclosures
|
Config 1
|
Config 2
|
Config 3
|
Config 4
|
Config 5
|
|
1
|
Standard
|
Dense
|
Dense
|
Dense
|
Dense
|
|
2
|
Standard
|
Standard
|
Dense
|
Dense
|
Dense
|
|
3
|
Standard
|
Standard
|
Standard
|
Dense
|
Dense
|
|
4
|
Standard
|
Standard
|
Standard
|
Standard
|
Dense
|
|
5
|
Standard
|
Standard
|
Standard
|
Standard
|
Dense
|
|
6
|
Standard
|
Standard
|
Standard
|
Dense
|
Dense
|
|
7
|
Standard
|
Standard
|
Standard
|
Dense
|
Dense
|
|
8
|
Standard
|
Standard
|
Dense
|
Dense
|
Dense
|
|
9
|
Standard
|
Dense
|
Dense
|
Standard
|
|
|
10
|
Standard
|
Standard
|
Standard
|
Standard
|
|
|
11
|
Standard
|
Standard
|
Standard
|
|
|
|
12
|
Standard
|
Standard
|
Standard
|
|
|
|
13
|
Standard
|
Standard
|
Standard
|
|
|
|
14
|
Standard
|
Standard
|
Standard
|
|
|
|
15
|
Standard
|
Standard
|
|
|
|
|
16
|
Standard
|
Standard
|
|
|
|
|
17
|
Standard
|
|
|
Legend:
|
|
|
18
|
Standard
|
|
|
Chain 1
|
|
|
19
|
Standard
|
|
|
Chain 2
|
|
|
20
|
Standard
|
|
|
|
|
|
Note: Standard refers to either Model 12F LFF or Model 24F SFF expansion enclosures. Dense refers to the Model 92F HD expansion enclosure.
|
|||||
Because IBM FlashSystem V9000 supports up to 4 building blocks, the maximum standard expansion enclosures supported in a full scale-out configuration is 80 (20 x 4). The maximum number of dense expansion enclosures in a full scale-out configuration is 32 (8 x 4).
2.7 Administration and maintenance
This section describes the IBM FlashSystem V9000 storage system capabilities for administration and maintenance.
2.7.1 System management
The IBM FlashSystem V9000 control enclosures in a system operate as a single system and present a single point of control for system management and service. System management and error reporting are provided through an Ethernet interface to one of the nodes in the system, which is called the configuration node. The configuration node runs a web server and provides a command-line interface (CLI). Any node in the system can be the configuration node. If the current configuration node fails, a new configuration node is selected from the remaining nodes. Each node also provides a command-line interface and web interface for initiating hardware service actions.
IBM FlashSystem V9000 includes the use of the popular IBM SAN Volume Controller CLI and GUI, which deliver the functions of IBM Spectrum Virtualize, part of the IBM Spectrum Storage Family. The IBM FlashSystem V9000 supports SNMP, email forwarding (SMTP), and syslog redirection for complete enterprise management access.
Graphical user interface (GUI)
IBM FlashSystem V9000 includes the use of the standard IBM Spectrum Virtualize GUI.
The IBM FlashSystem V9000 GUI is started from a supported Internet browser when you enter the systems management IP address. The login window then opens (Figure 2-44).
Figure 2-44 IBM FlashSystem V9000 GUI login window
Enter a valid user name and password. A system overview window opens:
•Figure 2-45 on page 85 is for fixed building block.
•Figure 2-46 on page 85 is for scalable building block configurations.
The middle of the window displays a real-time graphic of the IBM FlashSystem V9000.
Figure 2-45 shows the system overview window for a fixed building block.
Figure 2-45 System overview window (fixed building block)
Figure 2-46 shows the system overview window for a scale-up and scale-out environment.
Figure 2-46 System overview window (scale-up and scale-out environment)
The bottom of the window has three dashboard icons:
•Capacity
•Performance
•System status
The left side of the window displays seven function icons:
Those functions are briefly described next. Also, see the following chapters:
•Details about the GUI: Chapter 8, “Using IBM FlashSystem V9000” on page 321
•Details about the Settings function: Chapter 9, “Configuring settings” on page 405
Monitoring function
Figure 2-47 shows the Monitoring icon and the associated branch-out menu. Click the Monitoring icon if you want to select any of these actions:
•System: Monitor the system health of the IBM FlashSystem V9000 hardware.
•Events: View the events log of the IBM FlashSystem V9000.
•Performance: Start the system I/O performance graphs.
Figure 2-47 IBM FlashSystem V9000 GUI: Monitoring icon and branch-out menu
Pools function
Figure 2-48 on page 87 shows the Pools icon and the associated branch-out menu. Click the Pools icon if you want to select any of these actions:
•Pools: View list of pools, create new pools, edit existing pools, and delete pools.
•Volumes by Pool: View a list of volumes (LUNs) that are associated with pools, create new associations, or delete associations.
•Internal Storage: View all internal storage associated with the IBM FlashSystem V9000.
•External Storage: View and manage all external storage associated with the IBM FlashSystem V9000.
•MDisks by Pools: View a list of MDisk that are associated with pools, create new associations, or delete associations.
•System Migration: Perform storage migration actions for data from externally virtualized storage.
Figure 2-48 IBM FlashSystem V9000 GUI: Pools menu
Volumes function
Figure 2-49 shows the Volumes icon and the associated branch-out menu. Click the Volumes icon if you want to do any of these actions:
•Volumes: View a list of all system storage volumes (LUNs), create new volumes, edit existing volumes, and delete volumes.
•Volumes by Pools: View a list of volumes that are associated with pools, create new associations, or delete associations.
•Volumes by Host: View a list of volumes that are associated with hosts, create new associations, or delete associations.
Figure 2-49 IBM FlashSystem V9000 GUI: Volumes icon and branch-out menu
Hosts function
Figure 2-50 shows the Hosts icon and the associated branch-out menu. Click the Hosts icon if you want to select any of these actions:
•Hosts: View a list of all hosts, create new hosts, edit existing hosts, and delete hosts.
•Ports by Host: View a list of ports that are associated with a host, create new hosts, edit existing hosts, and delete hosts.
•Host Mappings: View mappings per host regarding volumes.
•Volumes by Host: View a list of volumes that are associated with hosts, create new associations, or delete associations.
Figure 2-50 IBM FlashSystem V9000 GUI: Hosts icon and branch-out menu
Copy Services function
Figure 2-51 on page 89 shows the Copy Services icon and associated branch-out menu. Click the Copy Services icon if you want to select any of these actions:
•FlashCopy: View a list of all volumes and their associated flash copies, create new FlashCopy relationships, and edit or delete existing relationships.
•Consistency Groups: View the consistency groups created for remote copy partnerships, create new groups, edit existing groups, delete groups.
•FlashCopy Mappings: View a list of current FlashCopy mappings and their status, create new mappings, edit existing mappings, and delete mappings.
•Remote Copy: View the consistency groups created for remote copy partnerships, create new groups, edit existing groups, and delete groups.
•Partnerships: View the system partnerships with secondary system, create a new partnership, edit a partnership, and delete a partnership.
Figure 2-51 IBM FlashSystem V9000 GUI: Copy Services icon and branch-out menu
Access function
Figure 2-52 shows the Access icon and associated branch-out menu. Click the Access icon if you want to select any of these actions:
•Users: View a list of current users, create new users, edit existing users, and delete users.
•Audit Log: View the system access log and view actions by individual users.
Figure 2-52 IBM FlashSystem V9000 GUI: Access icon and branch-out menu
Settings function
Figure 2-53 shows the Settings icon and associated branch-out menu. Click the Settings icon if you want to configure system parameters, including alerting, open access, GUI settings, and other system-wide configuration.
Figure 2-53 IBM FlashSystem V9000 GUI: Settings icon and branch-out menu
Command-line interface (CLI)
IBM FlashSystem V9000 uses the standard IBM Spectrum Virtualize storage CLI. This CLI is common among several IBM storage products, including IBM Spectrum Virtualize and the IBM Storwize family of products: the V7000, IBM V5000, IBM V3700, and IBM V3500 disk systems. IBM Spectrum Virtualize CLI is easy to use with built-in help and hint menus.
To access the IBM FlashSystem V9000 Spectrum Virtualize CLI, a Secure Shell (SSH) session to the management IP address must be established. You are then prompted for a user name and password. For information about using the FlashSystem V9000 CLI, see Chapter 13, “Hints and tips” on page 597.
Call home email SMTP support
IBM FlashSystem V9000 supports setting up a Simple Mail Transfer Protocol (SMTP) mail server for alerting the IBM Support Center of system incidents that might require a service event. These emails can also be sent within the client’s enterprise to other email accounts that are specified. After it is set up, system events that might require service are emailed automatically to an IBM Service account specified in the IBM FlashSystem V9000 code.
The email alerting can be set up as part of the system initialization process or added or edited at anytime through the IBM FlashSystem V9000 GUI. Also, a test email can be generated at anytime to test the connections. Figure 2-54 on page 91 shows the IBM FlashSystem V9000 Email setup window.
|
Tip: Be sure to set up Call Home. For details, see 9.2.1, “Email and call home” on page 407.
|
Figure 2-54 IBM FlashSystem V9000 Email alerting setup window
SNMP support
IBM FlashSystem V9000 supports SNMP versions 1 and 2. The GUI is used to set up SNMP support on the IBM FlashSystem V9000.
To set up SNMP support on the IBM FlashSystem V9000, click the Settings icon at the left side of the window, click the Notifications tab and click the SNMP tab to enter the SNMP trap receiver IP address and community access information. Figure 2-55 shows the IBM FlashSystem V9000 SNMP setup window.
Figure 2-55 IBM FlashSystem V9000 SNMP setup window
|
Note: The IBM FlashSystem V9000 CLI can also be used to program the SNMP settings.
|
Redirection of syslog
You can redirect syslog messages to another host for system monitoring. Use the GUI to set up syslog redirection on the IBM FlashSystem V9000. To set up syslog redirection, click the Settings icon on the lower left of the window, click the Notifications tab, and then click the Syslog tab to enter the remote host trap IP address and directory information. Figure 2-56 shows the Syslog redirection setup window.
Figure 2-56 IBM FlashSystem V9000 Syslog redirection setup window
|
Note: The IBM FlashSystem V9000 CLI can also be used to set up syslog redirection.
|
2.7.2 Software and licensing
IBM FlashSystem V9000 uses the advanced software features of IBM Spectrum Virtualize. IBM FlashSystem V9000 data services are provided through IBM FlashSystem V9000 software. IBM FlashSystem V9000 has both base and optional software licenses.
For more information about IBM FlashSystem V9000 advanced software functionality, see these resources:
•IBM FlashSystem V9000 Version 7.7 Product Guide, REDP-5409
Base licensed features
The following functions are provided with the IBM FlashSystem V9000 base software license 5639-RB7:
•Virtualization of IBM FlashSystem V9000 storage and expansion enclosures
Enables rapid, flexible provisioning, and simple configuration changes.
One IBM FlashSystem V9000 base license option 5639-RB7 is needed for each storage and expansion enclosure.
•Thin provisioning
Helps improve efficiency by allocating disk storage space in a flexible manner among multiple users, based on the minimum space that is required by each user at any time.
•Data migration
Enables easy and nondisruptive moves of volumes from another storage system onto the IBM FlashSystem V9000 system by using Fibre Channel connectivity. Dynamic migration helps speed data migrations from weeks or months to days, eliminating the cost of add-on migration tools and providing continuous availability of applications by eliminating downtime.
•Simple GUI
Simplified management with the intuitive GUI enables storage to be quickly deployed and efficiently managed. The GUI runs on the IBM FlashSystem V9000 system, so there is no need for a separate console. All you need to do is point your web browser to the system.
•IBM Easy Tier technology
This feature provides a mechanism to seamlessly migrate data to the most appropriate tier within an IBM FlashSystem V9000 storage pool. This migration can be to and from the internal IBM FlashSystem V9000 storage or expansion enclosure, or to and from external storage systems that are virtualized by IBM FlashSystem V9000. Easy Tier technology adds more blended economy of capacity and is useful for cost effective expansion and usage of your existing storage capacity investment.
Easy Tier supports up to three tiers of storage. For example, you can set up a storage pool intended for Easy Tier volumes where the pool consists of the IBM FlashSystem V9000 storage enclosures, 15,000 RPM Fibre Channel disk drives, and SAS disk drives.
Software version 7.8 introduces a fourth tier so that you can separate the flash storage into two tiers.
•Automatic restriping of data across storage pools
When growing a storage pool by adding more storage to it, IBM FlashSystem V9000 software can restripe your data on pools of storage so you do not need to implement any manual or scripting steps. This feature helps grow storage environments with greater ease while retaining the performance benefits that come from striping the data across the disk systems in a storage pool.
The following functions are provided with the IBM FlashSystem V9000 base software license for internal storage only. Internal storage includes IBM FlashSystem V9000 storage enclosures and expansion enclosures:
•FlashCopy provides a volume level point-in-time copy function for any storage that is virtualized by IBM FlashSystem V9000. FlashCopy and snapshot functions enable you to create copies of data for backup, parallel processing, testing, and development, and have the copies available almost immediately.
•Real-time Compression helps improve efficiency by compressing data by as much as 80%, enabling storage of up to 5x as much data in the same physical space. Unlike other approaches to compression, Real-time Compression is designed to be used with active primary data such as production databases and email systems, dramatically expanding the range of candidate data that can benefit from compression.
•Microsoft Windows Offloaded Data Transfer (ODX) is supported with IBM FlashSystem V9000 Software V7.5 and later. This functionality in Windows improves efficiencies by intelligently managing IBM FlashSystem V9000 systems to directly transfer data within or between systems, bypassing the Windows host system.
•VMware and vSphere 6.0. IBM FlashSystem V9000 Software V7.7 supports vCenter Site Recovery Manager (SRM) and vCenter Web Client (IBM Spectrum Control Base 3.0.2 functionality). Also supported are vStorage application programming interfaces (APIs) for storage awareness.
•Remote Mirroring provides storage system-based data replication by using either synchronous or asynchronous data transfers over Fibre Channel communication links:
– Metro Mirror maintains a fully synchronized copy at metropolitan distances (up to 300 km).
– Global Mirror operates asynchronously and maintains a copy at much greater distances (up to 8000 km).
Both functions support VMware Site Recovery Manager to help speed disaster recovery.
IBM FlashSystem V9000 remote mirroring interoperates with other IBM FlashSystem V9000, FlashSystem V840, SAN Volume Controller, and Storwize V7000 storage systems.
Optional licensed features
The following optional licensed features are offered with the IBM FlashSystem V9000 software for external storage:
•External Virtualization 5641-VC7 FC 0663
The system does not require a license for its own control and expansion enclosures; however, a capacity-based license is required for any external systems that are being virtualized. The system does not require an external virtualization license for external enclosures that are only being used to provide managed disks for a quorum disk and are not providing any capacity for volumes.
Enables IBM FlashSystem V9000 to manage capacity in other Fibre Channel SAN storage systems. When IBM FlashSystem V9000 virtualizes a storage system, its capacity becomes part of the IBM FlashSystem V9000 system and it is managed in the same way as capacity on internal flash modules within IBM FlashSystem V9000. Capacity in external storage systems inherits all the functional richness of the IBM FlashSystem V9000.
•Real-time Compression for external storage 5641-CP7 FC 0708
With the compression function data is compressed as it is written to the drive, saving additional capacity for the system. This license is capacity-based.
Helps improve efficiency by compressing data by as much as 80%, enabling storage of up to 5x as much data in the same physical space. Unlike other approaches to compression, Real-time Compression is designed to be used with active primary data such as production databases and email systems, dramatically expanding the range of candidate data that can benefit from compression.
|
Note: IBM FlashSystem V9000 V7.5 and later has revised licensing rules for IBM Spectrum Virtualize Real-time Compression for external storage (5641-CP7 FC 0708).
As it pertains to externally virtualized storage, rather than using the volume size as the measure for determining how many terabytes of IBM Spectrum Virtualize Real-time Compression for external storage 5641-CP7 FC 0708 to license, the measured terabyte capacity now applies to the actual managed disk capacity consumed by the compressed volumes.
|
For example, suppose that you want to store 500 TB of data where 300 TB of that data cannot be compressed (so it is not configured on compressed volumes), but 200 TB of that data can be compressed and is configured on compressed volumes.
Rather than needing to license 200 TB of Real-time Compression, the compression ratio can be applied to determine how much storage the 200 TB of volumes actually uses. The compression ratio can be obtained in advance using the IBM Comprestimator tool, or it can be shown in the system later as the actual amount of managed disk space used by those compressed volumes.
If, for example, the compression ratio is 3:1 for that 200 TB of data, meaning that only 1 TB of managed storage is consumed for every 3 TB of data, the user would license only 1/3 of the 200 TB, or 67 TB of the 5641-CP7 license. The 5641-CP7 license continues to not be licensed to a specific IBM Spectrum Virtualize hardware device, but is licensed to the customer within a country, in the same way that SAN Volume Controller standard (5639-VC7) software is licensed today.
|
Note: The 5641-VC7 (External Virtualization, FlashCopy, and Remote Mirroring features) and 5641-CP7 FC 0708 (Compression) are licensed per enterprise within one country and are the same licenses as for SAN Volume Controller. Existing SAN Volume Controller licenses can be used for the IBM FlashSystem V9000 for these features.
|
•FlashCopy for external storage 5641-VC7 FC 0671
The FlashCopy function copies the contents of a source volume to a target volume. This license is capacity-based.
FlashCopy provides a volume level point-in-time copy function for any storage that is virtualized by IBM FlashSystem V9000. FlashCopy and snapshot functions enable you to create copies of data for backup, parallel processing, testing, and development, and have the copies available almost immediately.
•Remote Mirroring Software for external storage 5641-VC7 FC 0679
The remote-copy function allows the use of Metro Mirror and Global Mirror functions. This function enables you to set up a relationship between volumes on two systems, so that updates that are made by an application to one volume are mirrored on the other volume. The volumes can be in the same system or on two different systems. This license is capacity-based. Provides storage system-based data replication by using either synchronous or asynchronous data transfers over Fibre Channel communication links.
Starting with version 7.7 of IBM Spectrum Virtualize, Differential Licensing is used to calculate the license needed for a given configuration. With Differential Licensing, licenses change from per terabyte to per storage capacity unit (SCU). Differential Licensing and how to calculate SCUs is explained in IBM FlashSystem V9000 Version 7.7 Product Guide, REDP-5409.
2.7.3 Serviceability and software enhancements
IBM FlashSystem V9000 includes several design enhancements for the administration, management, connectivity, and serviceability of the system:
•Concurrent code load
IBM FlashSystem V9000 supports upgrading the system firmware on the AC2 or AC3 control enclosures and AE2 storage enclosures (RAID controllers, management modules, and interface cards) and flash modules without affecting the connected hosts or their applications.
•Easily accessible hot swappable modules with no single point of failure
IBM FlashSystem V9000 design enables the easy replacement of any hardware module through the front or rear of the unit. The IBM FlashSystem V9000 does not require the top panel to be removed nor does it need to be moved in the rack to replace any component.
•Standard IBM CLI and GUI
IBM FlashSystem V9000 uses the latest IBM Spectrum Virtualize CLI and GUI for simple and familiar management of the unit.
•Encryption support
IBM FlashSystem V9000 supports hardware encryption of the flash modules to meet the audit requirements of enterprise, financial, and government clients.
•Sixteen Gbps FC support
IBM FlashSystem V9000 supports 16 Gbps FC, enabling clients to take advantage of the latest high-speed networking equipment while increasing performance.
2.8 Support matrix for IBM FlashSystem V9000
The IBM FlashSystem V9000 supports a wide range of operating systems (Windows Server 2008 and 2012, Linux, and IBM AIX), hardware platforms (IBM System x, IBM Power Systems, and x86 servers not from IBM), host bus adapters (HBAs), and SAN fabrics.
For specific information, see the IBM System Storage Interoperation Center (SSIC):
Contact your IBM sales representative or IBM Business Partner for assistance or questions about the IBM FlashSystem V9000 interoperability.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.