IBM FlashSystem 9100 architecture
This chapter describes the FlashSystem 9100 architectural components, available models, enclosure, software features, and licensing options.
This chapter includes following topics:
2.1 FlashSystem 9100 hardware components
Each IBM FlashSystem 9100 consists of a control enclosure and IBM FlashCore module drives. The control enclosure is the storage server that runs Spectrum Virtualize Software that controls and provides features to store and manage data on the FlashCore module or industry standard NVMe drives (see Figure 2-1).
Figure 2-1 FlashSystem 9100
The IBM FlashSystem FS9100 includes the following core components:
•IBM FlashSystem FS9100 control enclosure:
– Power supply units
– Battery modules
– Fan modules
– Interface cards
– Skylake CPUs and Memory Slots
•IBM FlashSystem FlashCore modules
•IBM FlashSystem FS9100 expansion enclosures (SAS attached)
2.2 IBM FS9100 Control Enclosure
IBM FS9100 is a 2U model and can support up to 24 IBM FCMs (IBM built NVMe drives) with hardware compression and encryption or industry standard NVMe drives of various capacities. IBM FS9100 can be configured with up to 1.5 TB of cache.
Figure 2-2 shows the internal architecture.
Figure 2-2 Internal architecture
An IBM FlashSystem 9100 clustered system can contain up to four IBM FlashSystem 9100 systems and up to 3,040 drives. IBM FlashSystem 9100 systems can be added to clustered systems that include Storwize V7000 systems.
Figure 2-3 shows the rear view of the enclosure.
Figure 2-3 Enclosure rear view
As shown in Figure 2-3, the IBM FS9100 enclosure consists of redundant power supply units, node canisters, and fan modules to provide redundancy and high availability.
Figure 2-4 shows the internal hardware components of a node canister. To the left is the front of the canister where fan modules and battery backup are installed, followed by two Skylake CPUs, memory DIMM slots, and PCIe risers for adapters on the right.
Figure 2-4 Internal hardware components
2.2.1 Model 9110 Control Enclosure AF7
IBM FS9100 model 9110 offers the following features:
•Two node canisters with 4 x 8 cores 1.7 GHz Skylake CPUs with compression assist up to 40 Gbps
•Cache options from 128 GB (64 GB per canister) to 1.5 TB (768 GB per canister)
•Eight 10 Gb Ethernet ports standard for iSCSI connectivity
•16 Gb FC, 25 Gb Ethernet, and 10 Gb Ethernet ports for FC and iSCSI connectivity
•12 Gb SAS ports for expansion enclosure attachment
•A total of 24 slots for 2.5-inch NVMe flash drives
•2U, 19-inch rack mount enclosure with AC power supplies
•One Boot drive
2.2.2 Model 9150 Control Enclosure AF9
IBM FS9100 model 9150 offers 4 x14 cores 2.2 GHz Skylake CPUs with dual boot drive and the following hardware features and software functions that are common to both models of FS9100:
•Two node canisters, each with 4 x14 cores 2.2 GHz Skylake CPUs with compression assist up to 100 Gbps
•Cache options from 128 GB (64 GB per canister) to 1.5 TB (768 GB per canister)
•Eight 10 Gb Ethernet ports standard for iSCSI connectivity
•16 Gb FC, 25 Gb Ethernet, and 10 Gb Ethernet ports for FC and iSCSI connectivity
•12 Gb SAS ports for expansion enclosure attachment
•A total of 24 slots for 2.5-inch NVMe flash drives
•2U, 19-inch rack mount enclosure with AC power supplies
•Two Boot drives
2.2.3 Model 9150 Expansion Enclosure Models AFF and AF9
All Flash expansions models AFF and AF9 can be attached to FS9100 control enclosure by using the SAS adapter.
2.2.4 Model AFF
Model AFF holds up to 24 2.5-inch SAS flash drives in a 2U, 19-inch rack mount enclosure. An intermix of capacity drives is allowed in any drive slot and up to 20 AFF enclosures can be attached to the Control enclosure (490) drives.
2.2.5 Model AF9
Model A9F holds up to 92 3.5-inch SAS flash drives in a 5U, 19-inch rack mount enclosure. An intermix of capacity drives is allowed in any drive slot and up to 8 A9F enclosures can be attached to the Control enclosure (736) drives.
2.2.6 FlashSystem 9100 Utility Models UF7 and UF8
IBM FlashSystem 9100 utility models UF7 and UF8 provide a variable capacity storage offering. These models offer a fixed capacity, with a base subscription of 35% of the total capacity.
IBM Storage Insights (free edition or pro) is used to monitor system usage, and capacity that is used beyond the base 35% is billed on a per month, per terabyte basis. This feature enables you to grow or shrink usage, and pay for the configured capacity only.
IBM FlashSystem utility models are provided for customers who can benefit from a variable capacity system, where billing is based on provisioned space only. The hardware is leased through IBM Global Finance on a three-year lease, which entitles the customer to use up to 35% of the total system capacity at no extra cost. If storage needs increase beyond that 35% capacity, usage is billed based on the average daily provisioned capacity per terabyte, per month, on a quarterly basis.
Total system capacity of 115 TB example
A customer has a FlashSystem 9100 utility model with 4.8 TB NVMe drives, for a total system capacity of 115 TB. The base subscription for such a system is 40.25 TB. During the months where the average daily usage is below 40.25 TB, no extra billing is needed.
The system monitors daily provisioned capacity and averages those daily usage rates over the month term. The result is the average daily usage for the month.
If a customer uses 45 TB, 42.5 TB, and 50 TB in three consecutive months, Storage Insights calculates the overage as listed in Table 2-1, rounding to the nearest terabyte.
Table 2-1 Billing calculations that are based on customer usage
|
Average daily
|
Base
|
Overage
|
To be billed
|
|
45 TB
|
40.25 TB
|
4.75 TB
|
5 TB
|
|
42.5 TB
|
40.25 TB
|
2.25 TB
|
2 TB
|
|
50 TB
|
40.25 TB
|
9.75 TB
|
10 TB
|
The total capacity that is billed at the end of the quarter is 17 TB per month in this example.
Flash drive expansions can be ordered with the system, in all supported configurations.
The feature codes that are associated with the UF7 and UF8 utility model billing are listed in Table 2-2.
Table 2-2 9100 UF7 and UF8 utility model billing feature codes
|
Feature code
|
Description
|
|
# AE00
|
Variable Usage 1 TB per month
|
|
# AE01
|
Variable Usage 10 TB per month
|
|
# AE02
|
Variable Usage 100 TB per month
|
These features are used to purchase the variable capacity that is used in the utility models. The features (feature code AE00, AE01, and AE02) provide TBs of capacity beyond the base subscription on the system. Usage is based on the average capacity that is used, per month. The total of the prior three months’ usage is totaled, and the corresponding number of AE00, AE01, and AE02 features ordered quarterly.
2.3 FlashCore Module and NVMe drives
Figure 2-5 shows an IBM FlashCore Module (NVMe) with a capacity of 19.2 TB built that uses 64-layer TLC flash memory and an Everspin MRAM cache into a U.2 form factor.
Figure 2-5 FlashCore Module (NVMe)
IBM FCMs (NVMe) are designed for high parallelism and optimized for 3DTLC and updated FPGAs. IBM also enhanced the FCMs by adding read cache to reduce latency on highly compressed pages, and four plane programming to lower the overall power during writes. FCMs offer hardware assisted compression up to 3:1 and FIPS 140-2 complaint.
carry the IBM patented Variable Stripe RAID™ at the FCM level and use DRAID to protect data at system level. VSR and DRAID together optimize raid rebuilds by offloading rebuilds to DRAID and offers protection against FCM failures.
FCMs on FS9100 can be configured to use 4.8 TB, 9.6 TB, and 19.2 TB.
2.3.1 Industry-standard NVMe drives
FS9100 provides an option to use industry standard NVMe drives that are sourced from Samsung and Toshiba and available in the following capacity variations:
•NVMe 1.92 TB
•3.84 TB
•7.68 TB
•15.36 TB
2.4 NVMe and adapter support
NVMe is a NUMA optimized, high-performance, and highly scalable storage protocol that is designed to access non-volatile storage media by using host PCIe bus. NVMe uses low latency, available parallelism, and reduces I/O overhead.
NVMe supports multiple IO queues up to 64 K queues and each queue can support up to 64 K entries. Older SAS and SATA support single queue with only 254 and 32 entries and use many more CPU cycles to access data. NVMe handles more workload for the same Infrastructure footprint.
NVMe over Fabrics (NVMe-oF) is a technology specification that is designed to enable nonvolatile memory express message-based commands to transfer data between a host computer and a target solid-state storage device or system over a network, such as Ethernet, Fibre Channel (FC), or InfiniBand.
2.4.1 Support for adapters, host platforms, and switches
The following adapters, host platforms, and switches are supported:
•Adapters:
– Fibre Channel 4x16Gb
– iSCSI 8x10Gb
– Ethernet 2x1Gb System management
– iSER over RoCE with 2x25G Mellanox ConnectX4-LX
– iSER over iWARP with 2X25G Chelsio T6 adapters
– SAS expansion 2x12Gb
•Host platforms:
– RHEL 7.4
– CentOS 7.4
– ESX 6.7
•Switches:
– Cisco 3232C
– Arista 7060
– Dell
2.5 Software features and licensing
Figure 2-6 shows the software offerings that are orderable with IBM FlashSystem 9100.
Figure 2-6 IBM FS9100 software that is included for base and optional licensing
2.5.1 IBM Spectrum Virtualize for IBM FlashSystem 9100
IBM FS9100 uses IBM Spectrum Virtualize™ software that combines the following software-defined functions for Flash Storage to manage data:
•Deduplication
•Compression
•Thin provisioning
•Easy Tier (automatic and dynamic tiering)
•Encryption for internal and virtualized external storage
•SCSI Unmap
•HyperSwap (high availability active-active)
•FlashCopy (snapshot)
•Remote data replication
For more information about IBM FlashSystem 9100 capabilities and software, see Implementing the IBM System Storage SAN Volume Controller with IBM Spectrum Virtualize V8.1, SG24-7933.
2.5.2 IBM Multi-Cloud starter software for IBM FlashSystem 9100
IBM FS9100 Multi-Cloud starter software along with Spectrum Virtualize includes the following software stack:
•IBM Spectrum Protect™ Plus
•IBM Spectrum Copy Data Management
•IBM Spectrum Virtualize for Public Cloud
The software bundle that is included with IBM FlashSystem 9100 control enclosures enables the development of multi-cloud strategy to harness the power of data. It increases the flexibility to manage data through choice, security, and protection. Licensing includes 5 TB of managed capacity and provides a base to migrate to a complete IBM FlashSystem 9100 Multi-Cloud Solution by following an IBM validated blueprint.
2.5.3 IBM FlashSystem 9100 Multi-Cloud solutions
FlashSystem 9100 Multi-Cloud solutions are a set of proven solutions that are designed to support today’s data driven multi-cloud architectures. These NVMe-ready, cloud-enabled software solutions are validated according to a set of multi-cloud blueprints. These solutions also enable modernizing infrastructure by expanding IBM FlashSystem 9100 to be used for multi-cloud architectures that include data protection, business continuity, and data reuse.
The following solutions are available:
•IBM FlashSystem 9100 Multi-Cloud solution for data reuse, protection, and efficiency
This solution consists of IBM Spectrum Protect Plus and IBM Spectrum Copy Data Management. It is designed to secure and reuse your company’s most precious asset, protect your data, and drive secondary data efficiencies within multi-cloud environments.
•IBM FlashSystem 9100 Multi-Cloud solution for business continuity and data reuse
This solution consists of IBM Spectrum Virtualize for Public Cloud and IBM Spectrum Copy Data Management. By using IBM Spectrum Virtualize for Public Cloud, data can be copied by using synchronous or asynchronous real-time replication from FlashSystem 9100 to IBM Cloud™. IBM Spectrum Copy Data Management can be used to create secondary data reuse snapshots of data that is managed by IBM Spectrum Virtualize for Public Cloud in IBM Cloud.
•IBM FlashSystem 9100 Solution for Private Cloud Flexibility and Data Protection
This solution consists of IBM Spectrum Copy Data Management and is designed to simplify and transform multi-cloud environments by combining private cloud management with enabling tools, all managed through a single user interface.
•IBM Storage Insights
Cloud-based IBM Storage Insights provides a single dashboard that gives you a clear view of all IBM block storage. It enables predictive analysis and displays real-time and historical charts to monitor performance and capacity.
Storage health information enables customers to focus on areas that need attention. When IBM support is needed, Storage Insights simplifies uploading logs, speeds resolution with online configuration data, and provides an overview of open tickets all in one place.
Storage Insights Pro is a subscription service that provides longer historical views of data, more reporting and optimization options, and supports IBM file and block storage together with EMC VNX and VMAX.
2.5.4 IBM FS9100 high availability
IBM FlashSystem 9100 is designed to offer high system and data availability with the following features:
•HyperSwap support
•Dual-active, intelligent node canisters with mirrored cache
•Dual-port flash drives with automatic drive failure detection and RAID rebuild
•Redundant hardware, including power supplies and fans
•Hot-swappable and customer replaceable components
•Automated path failover support for the data path between the server and the drives
2.6 Data protection on IBM FS9100
Data protection from NAND chip and controller failures are managed by using two IBM technologies: Variable Stripe RAID and DRAID. VSR protects failures at IBM FlashCore modules chip level and DRAID protects data from failure of FlashCore modules and industry-standard NVMe drives.
2.6.1 Variable Stripe RAID
Variable Stripe RAID is a patented IBM technology that provides data protection at the page, block, or chip level. It eliminates the need to replace an entire flash module when a single chip or plane fails. This feature, in turn, expands the life and endurance of flash modules and considerably reduces maintenance events throughout the life of the system.
For more information about VSR, see Introducing and Implementing IBM FlashSystem V9000, SG24-8273.
2.6.2 DRAID
Distributed RAID functionality is managed by Spectrum Virtualize, which enables a storage array to distribute RAID5 or RAID6 to a largest set of drives. For example, if eight drives were used on traditional RAID5, the data was striped across seven and the parity was on 8. DRAID enhanced this method by specifying the stripe width and the number of drives separately and as a result still has seven data stripes that are protected by a parity stripe but the eight drives are selected from the larger set. In addition, with distributed sparing, each drive in the array gives up some of its capacity to make a spare instead of an unused spare drive.
The benefit of DRAID is improved rebuild performance. During a drive failure, the data rebuild is done from a larger set of drives. This ability increases the number of reads and is rebuilt to a larger set of distributed sparing drives, which also increases the number of writes as compared to traditional RAID where reads are done from a smaller set of drives that are written to a single drive.
DRAID on IBM FS9100
DRAID6 is recommended for IBM FS9100 and is only allowed option from GUI. DRAID5 is configurable by using CLI only. DRAID6 creates spare space across all NVMe SSDs or FCMs on the array. During failure, the array rebuilds data by using the spare space faster than traditional RAID rebuilds.
DRAID rebuild
The spare area for rebuild on FS9100 is reserved against the physical capacity of the drives. As the rebuild progresses, the data is copied to remaining drives, which increases the capacity threshold, as shown in Figure 2-7.
Figure 2-7 Physical capacity threshold
DRAID copyback
DRAID copyback is a similar process to the rebuild process. Upon completion, FS9100 releases the space area by UNMAPing the area that was used, as shown in Figure 2-8.
Figure 2-8 DRAID copyback
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.