IBM FlashSystem V9000 introduction
This chapter introduces the IBM FlashSystem V9000 storage system and its core values, benefits, and technological advantages.
This chapter includes the following topics:
1.1 IBM FlashSystem V9000 storage overview
The IBM FlashSystem V9000, shown in Figure 1-1, delivers high capacity and fully integrated management for the enterprise data center. IBM FlashSystem V9000 uses a fully featured and scalable all-flash architecture that performs at up to 3.0 million input/output operations per second (IOPS) with IBM MicroLatency, is scalable to 68 gigabytes per second (GBps), and delivers an effective flash capacity of up to 2.28 petabytes (PB).
Figure 1-1 IBM FlashSystem V9000
Beyond its base all-flash architecture, the IBM FlashSystem V9000 also addresses tiered capabilities as described below, while not losing its focus on full integration.
With the release of IBM FlashSystem V9000 Software V7.8, extra functions and features are available, including support for new and more powerful IBM FlashSystem V9000 Control Enclosure Model AC3 and new SAS-based small form factor (SFF) and large form factor (LFF) storage enclosures, providing a mixture of nearline SAS hard disk drives (HDDs) and flash managed disks (MDisks) in a pool, which can be used for IBM Easy Tier®.
Up to 20 serial-attached SCSI (SAS) expansion enclosures are supported per IBM FlashSystem V9000 controller pair, providing up to 240 drives with expansion enclosure Model 12F, and up to 480 drives with expansion enclosure Model 24F.
The new IBM FlashSystem V9000 LFF expansion enclosure Model 92F supports up to 92 drives per enclosure, with a mixture of HDD and SSD drives in various capacities.
Using its flash-optimized design, IBM FlashSystem V9000 can provide response times of 180 microseconds. It delivers better acquisition costs than a high-performance spinning disk for the same effective capacity while achieving five times the performance, making it ideal for environments that demand extreme performance.
The new IBM FlashSystem V9000 LFF expansion enclosure Model 12F offers new tiering options with 8 TB or 10 TB nearline SAS hard disk drives (HDDs).
The new IBM FlashSystem V9000 SFF expansion enclosure Model 24F offers new tiering options with low-cost solid-state drives (SSDs).
Figure 1-2 shows IBM FlashSystem V9000 expansion enclosure Model 12F.
Figure 1-2 IBM FlashSystem V9000 expansion enclosure Model 12F
Figure 1-3 shows IBM FlashSystem V9000 expansion enclosure Model 24F.
Figure 1-3 IBM FlashSystem V9000 expansion enclosure Model 24F
Figure 1-4 shows IBM FlashSystem V9000 expansion enclosure Model 92F.
Figure 1-4 IBM FlashSystem V9000 expansion enclosure Model 92F
With IBM Real-time Compression™ technology, IBM FlashSystem V9000 further extends the economic value of all-flash systems. IBM FlashSystem V9000 provides up to two times the improvement in Real-time Compression over the model it is replacing. Using the optional Real-time Compression and other design elements, IBM FlashSystem V9000 provides up to 57 terabtyes (TB) usable flash capacity, and up to 285 TB effective flash capacity in only 6U. This scales to 456 TB usable flash capacity and up to 2.28 PB effective flash capacity in only 34U.
The addition of the new IBM FlashSystem V9000 expansion enclosures offers these features:
•SAS LFF and SFF expansion enclosures
Up to 20 expansion enclosures, (up to 80 in total) with twelve 3.5-inch HDDs or twenty-four 2.5-inch flash drives (SSDs) per enclosure:
– Up to 120 TB per enclosure and a total of 9.6 PB raw capacity using nearline HDDs
– Up to 367 TB per enclosure and a total of 29.4 PB raw capacity using SSDs
•SAS high-density (HD) expansion enclosures
Up to eight are supported per IBM FlashSystem V9000 controller pair (up to 32 in total), providing up to 92 drives (HDD or SSD mixed) per enclosure:
– Up to 920 TB per enclosure and a total of 29.4 PB raw capacity using nearline HDDs
– Up to 1.4 PB per enclosure and a total of 32 PB raw capacity using SSDs
IBM FlashSystem V9000 delivers enterprise-class advanced storage capabilities, including these among others:
•IBM Real-time Compression Accelerators
•IBM Easy Tier
•Thin provisioning
•Copy services
•Data virtualization
•IBM HyperSwap® Split-Clusters
•Highly available configurations
•N_Port ID Virtualization (NPIV) support
•Distributed redundant array of independent disks (DRAID) Component in Doubt (CID)
•iSCSI virtualization support
•SKLM Encryption support (at code level 7.8)
•Transparent Cloud Tiering (at code level 7.8)
Advanced data services that are provided include copy services, mirroring, replication, external virtualization, IBM HyperSwap capabilities, Microsoft Offloaded Data Transfer (ODX)-capable features, and VMware vSphere Storage application programming interfaces (APIs) Array Integration (VAAI) support.
Host interface support includes 8 gigabit (Gb) and 16 Gb Fibre Channel (FC), and 10 Gb Fibre Channel over Ethernet (FCoE) or Internet Small Computer System Interface (iSCSI). Advanced Encryption Standard (AES) 256 hardware-based encryption adds to the rich feature set.
|
Note: The AC3 control enclosure supports only the 16 Gb 4-port Fibre Channel adapter, however it can negotiate down to both 8 Gb and 4 Gb, so this book uses the reference “16/8/4” to indicate that the three speeds are supported by this adapter.
|
IBM FlashSystem V9000, including its IBM MicroLatency module (flash modules), is covered by up to seven years of total hardware support through the applicable warranty period.
1.2 Why flash matters
Flash is a vibrant and fast growing technology. Clients are looking to solve data center problems, optimize applications, reduce costs, and grow their businesses.
Here are several reasons why flash is a must in every data center, and why an IBM FlashSystem changes storage economics:
•Reduces application and server licensing costs, especially those related to databases and virtualization solutions.
•Improves application efficiency, that is, an application’s ability to process, analyze, and manipulate more information, faster.
•Improves server efficiency. Helps you get more out of your existing processors, use less random access memory (RAM) per server, and consolidate operations by having server resources spend more time processing data as opposed to waiting for data.
•Improves storage operations. Helps eliminate costly application tuning, wasted developer cycles, storage array hot spots, array tuning, and complex troubleshooting. Decreases floor space usage and energy consumption by improving overall storage environment performance.
•Enhances performance for critical applications by providing the lowest latency in the market.
Almost all technological components in the data center are getting faster, including central processing units, network, storage area networks (SANs), and memory. All of them have improved their speeds by a minimum of 10x; some of them by 100 times (100x), such as data networks. However, spinning disk has only increased its performance 1.2x.
The IBM FlashSystem V9000 provides benefits that include a better user experience, server and application consolidation, development cycle reduction, application scalability, data center footprint savings, and improved price performance economics.
Flash improves the performance of applications that are critical to the user experience, such as market analytics and research applications, trading and data analysis interfaces, simulation, modeling, rendering, and so on. Server and application consolidation is possible because of the increased process utilization resulting from the low latency of flash memory, which enables a server to load more users, more databases, and more applications. Flash provides or gives back time for further processing within the existing resources of such servers. Clients soon realize that there is no need to acquire or expand server resources as often or as soon as was previously expected.
Development cycle reduction is possible because developers spend less time designing an application to work around the inefficiencies of HDDs and less time tuning for performance.
Data center footprint savings are realized due to the high density and high performance of the IBM flash solutions, these systems are replacing racks and cabinet bays of spinning HDDs. Reducing the data center footprint also translates into power and cooling savings, making flash one of the greenest technologies for the data center.
|
Improved price: Performance economics are because of the low cost for performance value from the IBM FlashSystem. The cost savings result from deploying fewer storage enclosures, fewer disk drives, fewer servers with fewer processors, and less RAM while using less power, space, cooling and fewer processor licenses. Flash is one of the best tools for the data center manager for improving data center economics.
|
1.3 IBM FlashSystem family: Product differentiation
Flash is used widely in the data center, either within a server (Peripheral Component Interconnect Express (PCIe) cards or internal SSDs), in storage arrays (hybrid or all-flash), appliances, or platform solutions (hardware, software, and network). Flash can be used as cache or as a data tier. Because of the vast and wide adoption of flash, several flash architectures and, therefore, criteria can be applied to compare flash options. See Figure 1-5.
Figure 1-5 The various deployments of flash
Most storage vendors manufacture and market some type of flash memory system. The difference is how it is implemented, and the effect that such implementation has on the economics (cost reduction and revenue generation) for clients.
Flash technology is used to eliminate the storage performance bottleneck. The IBM FlashSystem family is a key shared-storage market leader and provides extremely low latency and consistent response times. It is designed and purpose-built specifically to harness what flash technology has to offer.
Some other vendors create flash appliances based on commodity server platforms and use software-heavy stack. Also, they use hardware technologies designed and created for disk, not flash. Others have designed their products using hybrid arrays combining existing storage designs, spinning HDDs, and SSD. The IBM storage portfolio includes SSD and flash on a variety of storage platforms; however, these alternative solutions do not have the same low latency (MicroLatency) as the hardware-accelerated IBM FlashSystem.
IBM FlashSystem family versus SSD-based storage arrays
Flash memory technologies appeared in the traditional storage systems some time ago. These SSD-based storage arrays help to successfully address the challenge of increasing I/Os per second (IOPS) needed by applications, and the demand for lower response times in particular tasks. An implementation example is the IBM Easy Tier technology. For an overview of this technology, see 3.2.1, “IBM Easy Tier” on page 99.
However, these technologies typically rely on flash in the format of Fibre Channel (FC), serial-attached SCSI (SAS), or Serial Advanced Technology Attachment (SATA) disks, placed in the same storage system as traditional spinning disks, and using the same resources and data paths. This approach can limit the advantages of flash technology because of the limitations of traditional disk storage systems.
However, SAS attached storage is still advantageous under the control of the IBM FlashSystem V9000 and can be considered as a good second tier level of storage in this environment.
IBM FlashSystem storage provides a hardware-only data path that realizes all of the potential of flash memory. These systems differ from traditional storage systems, both in the technology and usage.
An SSD device with an HDD disk form factor has flash memory that is put into a carrier or tray. This carrier is inserted into an array, such as an HDD. The speed of storage access is limited by the following technology because it adds latency and cannot keep pace with flash technology:
•Array controllers and software layers
•SAS controllers and shared bus
•Tiering and shared data path
•Form factor enclosure
IBM FlashSystem purpose-built MicroLatency modules are fast and efficient, designed using hardware-only data path technology that has a minimum number of software layers. Using this technology, IBM implements a mostly firmware component data path, and management software that is separated from the data path enabling the lowest latency modules on
the market.
the market.
The only other family of products with hardware-only access to flash technology is the PCI Express (PCIe) flash product family, where products are installed into a dedicated server. With the appearance of the IBM FlashSystem, the benefits of PCIe flash products to a single server can now be shared by many servers.
1.4 IBM FlashSystem V9000: IBM Tier 1 storage
The market for all-flash arrays is saturated with products aiming to replace enterprise storage arrays but consistently failing to deliver the breadth of data lifecycle, storage services, or the scalability delivered by incumbent solutions. Alternatively, hybrid arrays loaded with storage services consistently lack the low latency and performance scalability delivered by all-flash arrays.
The IBM FlashSystem V9000 merges IBM software-defined storage with the scalable performance of IBM FlashSystem storage to accelerate critical business applications and decrease data center costs simultaneously. As a result, your organization can gain a competitive advantage through a more flexible, responsive, and efficient storage environment.
The IBM FlashSystem V9000 provides a true paradigm shift in enterprise storage. Powered by IBM FlashCore Technology, IBM FlashSystem V9000 provides three dimensions of value, as shown in Figure 1-6.
•Versatile performance
•Enduring economics
•Agile integration
Figure 1-6 IBM FlashSystem V9000 dimensions of value
Versatile performance
IBM FlashSystem V9000 has the following versatile performance attributes:
•Scale-up or scale-out, independently
•Scalable to 3 million IOPS
•Scalable to 68 GBps bandwidth
•Sustained IBM MicroLatency
•Quality of service
•Faster applications
Enduring economics
IBM FlashSystem V9000 provides the following enduring economics attributes:
•Scalable to 2.2 PB effective capacity using native flash storage
•Expandable with up to 480 low-cost SSDs
•Expandable with up to 240 high capacity nearline drives
•Flash for less than the cost of disk with IBM Real-time Compression
•Low power and cooling requirements
•Virtualized storage
•Flash wear warranty
•Infrastructure continuity with space efficient snapshots, cloning, and replication
Agile integration
IBM FlashSystem V9000 has the following agile characteristics:
•Fully integrated system management
•Application-aware data services
•Advanced Encryption Standard (AES), data at rest encryption
•Tier or mirror to existing storage
•Mixed workload consolidation
•Nondisruptive data migrations
•Concurrent code load
By accelerating applications, both physical and virtual, IBM FlashSystem V9000 can help organizations reduce costs, increase revenue, and improve customer satisfaction for all types of applications, including the following categories:
•Transactional
•Enterprise resource planning and supply chain management (ERP and SCM)
•Big data and analytics
•Server and desktop virtualization
•Cloud
1.5 IBM FlashCore technology
The IBM FlashCore technology, used in the IBM FlashSystem V9000, employs several new and patented mechanisms to achieve greater capacity and throughput, at a lower cost than the previous IBM FlashSystem V840. Figure 1-7 shows the three major areas within IBM FlashCore technology, and the unique IBM attributes of each one.
Figure 1-7 IBM FlashCore Technology
To learn more about IBM FlashCore technology, visit the following web page:
1.5.1 Hardware accelerated I/O
IBM FlashSystem V9000 hardware design offers several unique IBM components including Hardware RAID, Non-blocking Crossbar Switch, Hardware Only Data Path, Single Box Highly Available Architecture, Concurrent Code Load and Concurrent Maintenance.
1.5.2 IBM MicroLatency module
IBM FlashSystem V9000 uses the new 20 nanometer (nm) multi-level cell (MLC) flash card memory chips and either 1.2 TB, 2.9 TB, or 5.7 TB capacity IBM MicroLatency modules, as shown in Figure 1-8. The IBM FlashSystem V9000 design also employs the use of IBM Engineered Massively Parallel Design, Field Programmable Gate Arrays (FPGAs) in the Data Path, Distributed RAM, and High-Speed Interfaces plus Hardware-based Data-at-Rest Encryption.
Figure 1-8 IBM MicroLatency module
1.5.3 Advanced flash management
The IBM FlashSystem V9000 has unique patented designs to ensure maximum availability. These include IBM Variable Stripe RAID™, IBM engineered error correction code (ECC), IBM Optimized Over-provisioning, Advanced Wear Leveling on IBM MicroLatency modules, Write Buffer And Hardware Offload, and Garbage Collection. See “Terminology” on page 29.
All this is made possible because of the following IBM patented and world class innovations:
•ECC algorithms that correct very high bit-error rates
•Variable voltage and read level shifting that help to maximize flash endurance
•Health binning and heat segregation, which continually monitor the health of flash blocks and perform asymmetrical wear leveling and sub-chip tiering
This all results in providing up to 57% improvement in endurance with a potential 45% reduction in write amplification.
1.5.4 Flash wear assurance
Through close collaboration between IBM Research, software development, and flash engineering, IBM created an advanced flash characterization platform to test thousands of flash devices over a lifetime of wear.
As a result, IBM has made major strides in advanced flash management. This improves MLC flash endurance 9x over standard implementations, and provides enterprise reliability and performance with IBM MicroLatency.
The IBM FlashSystem V9000, including its IBM MicroLatency modules, is covered by up to seven years of total hardware support through the applicable warranty period plus up to six years of optional post-warranty hardware maintenance. Clients can purchase the post-warranty hardware maintenance either at the time of system purchase or up until IBM announces withdrawal from marketing or withdrawal from service.
1.6 Architectural design overview
This section provides an overview of the IBM FlashSystem V9000 architecture.
1.6.1 IBM FlashSystem V9000 building blocks
The IBM FlashSystem V9000 consists of two control enclosures, (either two model AC2 or two models AC3), one storage enclosure (AE2), and software and hardware features, to make up a building block. A building block can be either fixed or scalable. You can combine scalable building blocks to create larger clustered systems in such a way that operations are not disrupted.
Figure 1-9 on page 12 shows the IBM FlashSystem V9000 fixed versus scalable building block capacity.
Figure 1-9 IBM FlashSystem V9000 scalability
|
Note: You can mix the AC2 control enclosure-based building blocks with the newer AC3 control enclosure building blocks, but each building block must have either two AC2s or two AC3s. The control enclosure types cannot be inter-mixed within a building block.
|
A scalable building block can be scaled up by adding IBM FlashSystem V9000 AE2 storage enclosures for increased storage capacity. You can add a maximum of four extra storage enclosures.
A scalable building block can be scaled out by combining up to four building blocks to provide higher IOPS and bandwidth needs for increased performance.
Figure 1-9 illustrates the scalable capacity of IBM FlashSystem V9000. It also shows that extra IBM FlashSystem V9000 storage enclosures (SEs) can be added to a single building block, and also to two, three, or four building blocks.
1.6.2 IBM FlashSystem V9000 expansion enclosures
With the introduction of IBM FlashSystem V9000 storage expansion enclosures, even greater capacity offerings are now available.
The IBM FlashSystem V9000 large form factor (LFF) expansion enclosure Model 12F offers new tiering options with high capacity nearline SAS hard disk drives (HDDs). Each LFF expansion enclosure supports up to twelve 8 TB or 10 TB drives.
The IBM FlashSystem V9000 small form factor (SFF) expansion enclosure Model 24F offers new tiering options with low-cost SSDs. Each SFF expansion enclosure supports up to 24 2.5-inch low-cost SSD drives.
Up to 20 LFF or SFF expansion enclosures are supported per IBM FlashSystem V9000 controller pair, providing up to 480 drives with expansion enclosure Model 24F (SFF) and up to 240 drives with expansion enclosure Model 12F (LFF).
|
Note: If you require to intermix the storage expansion enclosure types, see the supported configurations in 2.6.1, “SAS expansion enclosures intermix” on page 82.
|
IBM FlashSystem V9000 HD expansion enclosure Model 92F delivers increased storage density and capacity in a cost-efficient way.
IBM FlashSystem HD expansion enclosure Model 92F offers the following features:
•5U, 19-inch rack mount enclosure with slide rail and cable management assembly
•Support for up to ninety-two 3.5-inch LFF 12 Gbps SAS top-loading drives
•High-performance disk drives, high-capacity nearline disk drives, and flash drive support:
– High-capacity, archival-class nearline disk drives in 8 TB and 10 TB 7,200 rpm
– Flash drives in 1.92 TB, 3.84 TB, 7.68 TB, and 15.36 TB
•Redundant 200 - 240VA power supplies (new PDU power cord required)
Up to eight Model 92F high-density (HD) expansion enclosures are supported per IBM FlashSystem V9000 controller pair, providing up to 736 drives with expansion Model 92F. With four controller pairs, a maximum of 32 HD expansion enclosures with up to 2,944 drives can be attached.
If a mix of SFF, LFF, and HD enclosures is required, see 2.6.1, “SAS expansion enclosures intermix” on page 82.
Figure 1-10 on page 14 shows the maximum possible configuration with a single building block (controller pair) using a combination of native IBM FlashSystem V9000 storage enclosures and LFF or SFF expansion enclosures.
|
Note: If you require a mix of storage expansion enclosure types, see the supported configurations in 2.6.1, “SAS expansion enclosures intermix” on page 82.
|
Figure 1-10 Single scalable building block maximum configuration
High-density, low-cost SSDs allow applications to scale and achieve high read performance while maintaining traditional reliability and endurance levels. The 1.92 TB and 3.84 TB SAS 2.5-inch low-cost SSDs options are available for IBM FlashSystem V9000 SFF expansion enclosure.
High-capacity nearline SAS drives enables high value tiered storage with hot data stored in flash and warm data on lower cost nearline SAS HDDs all managed by IBM Easy Tier. The 8 TB and 10 TB SAS 3.5-inch nearline drives are available for IBM FlashSystem V9000 LFF storage expansion enclosure model 12F.
The IBM FlashSystem V9000 Version 7.8 also supports Model 92F 5U-high, 92drive bay, and supports the following drive types:
• High-capacity, nearline HDDs in 8 TB and 10 TB 7,200 rpm
• SSD Flash drives in 1.92 TB, 3.84 TB, 7.68 TB, and 15.36 TB
RAID 5 with standby hot spare is the only available RAID option for IBM FlashSystem V9000 native flash storage expansion. However, the SAS attached storage expansion enclosures can be configured with various RAID options. The preference for SAS attached storage is however distributed RAID (DRAID 5 and DRAID 6), which offers improved RAID rebuild times.
|
Note: To support SAS expansion enclosures, an AH13 - SAS Enclosure Attach adapter card must be installed in slot 3 of each AC2 control enclosure or slot 2 of each AC3 control enclosure in the building block.
|
1.6.3 IBM FlashSystem V9000 AE2 flash storage array
The IBM FlashSystem V9000 AC2 or AC3 control enclosures combines software and hardware into a comprehensive, modular appliance that uses symmetric virtualization.
Symmetric virtualization is achieved by creating a pool of managed disks (MDisks) from the internal storage and externally attached storage systems. Volumes are created from the MDisk pools and mapped to the attached host systems. System administrators can view and access a common pool of storage on the storage area network (SAN). With this functionality, administrators can use storage resources more efficiently and it provides a common base for advanced functions.
Also with IBM FlashSystem V9000, you can migrate data across MDisks without interfering with any host applications that are simultaneously accessing or writing data.
The IBM FlashSystem V9000 includes a single easy-to-use management graphical user interface (GUI) to help you monitor, manage, and configure your system.
The AE2 flash storage enclosure components include flash modules, battery modules, and canisters. The AE2 storage enclosure, with an all-hardware data path using FPGA modules, is engineered to deliver the lowest possible latency. The modules incorporate proprietary flash controllers and use numerous patented technologies. The flash controllers have a proprietary logic design, firmware, and system software.
No commodity 2.5-inch SSDs, PCIe cards, or any other significant non IBM assemblies are within the AE2 flash storage enclosure. The flash chips, Field Programmable Gate Array (FPGA) chips, processors, and other semiconductors in the system are carefully selected to be consistent with the purpose-built design, which is designed from the “ground up” for high performance, reliability, and efficiency.
The AE2 storage enclosures offer the following notable architectural concepts:
•Hardware-only data path.
•Use of FPGAs extensively.
•Field-upgradable hardware logic.
•Less expensive design cycle.
•Extremely high degree of parallelism.
•Intelligent flash modules.
•Distributed computing model.
•Low-power IBM PowerPC® processors.
•Interface and flash processors run thin real-time operating systems.
•With minimal management communication, the management processor communicates with the interface and flash processors through an internal network.
Hardware-only data path
The hardware-only data path design of the AE2 storage enclosures eliminates software-layer latency. To achieve extremely low latencies, the IBM FlashSystem advanced software functions are carefully assessed and implemented.
In the AE2 storage enclosures, data traverses the array controllers through FPGAs and dedicated, low-power processors (CPUs). No cycles are wasted on interface translation, protocol control, or tiering.
The AE2 storage enclosures, with an all-hardware data path design, have an internal architecture that differs from other hybrid (SSD and HDD) or SSD-only based disk systems.
The 20 nm flash card memory chips
The flash chip is the basic storage component of the IBM MicroLatency module. A maximum of 64 multi-level cell (MLC) flash chips can exist on each flash module. To maintain consistent wearing and reliability, combining flash chips of different flash technologies is not supported in the same flash module or storage system.
The IBM FlashSystem V9000 employs the new 20 nm MLC chips, which are of a higher density than the 24 nm enterprise MLC (eMLC) chips used in the IBM FlashSystem V840. This new design of chips enables the IBM FlashSystem V9000 to package greater densities of flash memory per card while retaining the same if not better performance and wear.
IBM patented ECC correction and checking algorithms ensure the same or greater performance from the MCL-based chips, with a greater capacity for the same footprint and at a lower cost per terabyte.
Flash module capacities
The IBM FlashSystem V9000 uses either 1.2 TB, 2.9 TB, or 5.7 TB IBM MicroLatency modules. This is a 40% increase in capacity per module over the IBM FlashSystem V840. They must be of the same capacity throughout the AE2 storage enclosure and cannot be intermixed with the older 24 nm flash modules.
Only RAID 5 is supported on the IBM FlashSystem V9000 with configurations of 4, 6, 8, 10, and 12 modules when using the 1.2 TB IBM MicroLatency modules. RAID 5 is supported with configurations of 6, 8, 10, and 12 modules when using the 2.9 TB or 5.7 TB IBM MicroLatency modules.
If fewer than 12 modules are installed, flash module fillers must be installed in the empty bays to maintain cooling airflow in the system enclosure.
Gateway interface FPGA
The gateway interface FPGA is responsible for providing I/O to the flash module and direct memory access (DMA) path. It is on the flash module and has two connections to the backplane.
Flash controller FPGA
The flash controller FPGA of the flash module provides access to the flash chips and is responsible for the following functions:
•Provides data path and hardware I/O logic
•Uses lookup tables and a write buffer
•Controls 13 or 16 chips (module-size-dependent)
•Operates independently of other controllers
•Maintains write ordering and layout
•Provides write setup
•Maintains garbage collection
•Provides error handling
Figure 1-11 shows the flash controller design details.
Figure 1-11 Flash controller design
The concurrent operations performed on the flash chips include moving data in and out of the chip through DMA, and by internally moving data and performing erasures. While actively transferring user data in the service of host-initiated I/O, the system can simultaneously run garbage collection activities without affecting the I/O. The ratio of transparent background commands running concurrent to active data transfer commands is 7:1.
A maximum of four flash controllers per IBM MicroLatency module exist: two for each primary board and two for each expansion board.
1.6.4 IBM Variable Stripe RAID and two-dimensional flash RAID overview
Storage systems of any kind are typically designed to perform two main functions: storing and protecting data. IBM FlashSystem V9000 includes the following options for data protection:
•RAID data protection:
– IBM Variable Stripe RAID
– Two-dimensional (2D) Flash RAID
•Flash memory protection methods
•Optimized RAID rebuild times
Table 1-1 lists the various methods of protection.
Table 1-1 Various types of IBM FlashSystem protection
|
Layer
|
Managed by
|
Protection
|
|
System-level RAID 5
|
Centralized RAID controllers
|
Module failure
|
|
Module-level RAID 5
|
Each module across the chips
|
Chip failure and page failure
|
|
Module-level Variable Stripe RAID
|
Each module across the chips
|
Sub-chip, chip, or multi-chip failure
|
|
Chip-level error correction code (ECC)
|
Each module using the chips
|
Bit and block error
|
|
Note: The proprietary 2D Flash RAID data protection scheme of the IBM FlashSystem V9000 storage system combines system-level RAID 5 and module-level Variable Stripe RAID (not only module-level RAID).
|
1.6.5 Fixed and scalable configurations
IBM FlashSystem V9000 can be configured as a fixed building block or a scalable building block.
A fixed building block contains one IBM FlashSystem V9000. The AE2 storage enclosure is cabled directly to each model AC2 control enclosure using 8Gb or 16Gb links or model AC3 control enclosure using 16Gb links. Each control enclosure is connected to switches or directly attached to a host. The control enclosures are directly connected, without the use of switches or a SAN fabric, to form the cluster links. A fixed building block can be upgraded to a scalable building block, but the upgrade process is disruptive to operations.
Scalable building blocks can contain multiple control enclosure pairs and multiple AE2 storage enclosures. In a scalable building block, the control enclosures are not cabled to each other. This infrastructure means that you can add building blocks or storage enclosures non disruptively. Fibre Channel switches are used to create a private storage fabric.
The Fibre Channel switch fabric does not have to be dedicated, and can be shared with hosts or server-side storage area networks (SANs). After connecting the components in a scalable building block, no physical cable connects any host to any switch in the internal Fibre Channel switch fabric. Care must be taken to ensure correct zoning of the back-end storage fabric to prevent interaction with any hosts or server-side storage area network traffic.
For more guidelines of port utilization techniques in a scalable environment, see Appendix A, “Guidelines: Port utilization in an IBM FlashSystem V9000 scalable environment” on page 657.
The back-end storage switch fabric is isolated, through the zoning, from the host or server-side SAN for these reasons:
•So that any host or server does not have access to the AE2 storage enclosures directly
•So that the I/O from the controller to the storage does not interfere with the I/O from the host to the controllers
1.6.6 Scale-up and scale-out solutions
IBM FlashSystem V9000 offers the flexibility of the purchase of an all flash solution and hybrid enclosures that can be upgraded in the future, by the ability to scale-up for increased capacity, scale-out for increased performance, or both.
Clients can start with a fixed building block, or opt for a scale-up scale-out (SUSO) solution, that includes two 16 Gb FC switches, which enables you to add extra storage enclosures and building blocks with minimal effect to the existing systems.
Figure 1-12 shows the IBM FlashSystem V9000 scale-up and scale-out capabilities.
Figure 1-12 IBM FlashSystem V9000 scale-up and scale-out capabilities
Figure 1-13 shows a scale-out solution with four IBM FlashSystem V9000 building blocks, using the 16 Gb FC switches for interconnections.
Figure 1-13 Scale out IBM FlashSystem V9000 solution
Figure 1-14 shows a scale-up solution with one IBM FlashSystem V9000 scalable building block and four IBM FlashSystem V9000 AE2 storage systems.
Figure 1-14 Scale up IBM FlashSystem V9000 solution
Figure 1-15 shows a scale-up and scale-out solution with four IBM FlashSystem V9000 building blocks and four IBM FlashSystem V9000 AE2 storage systems, indicating a maximum supported configuration.
Figure 1-15 Scale-up and scale-out IBM FlashSystem V9000 solution
|
Note: The FC internal connection switches are ordered together with the first IBM FlashSystem V9000 scalable building block. IBM also supports the use of customer-supplied FC switches and cables, if they are supported by IBM. See the latest information about supported FC switches at the IBM System Storage® Interoperation Center (SSIC):
|
The IBM FlashSystem V9000 capacity can be expanded further by the inclusion of additional SAS attached expansion enclosures. See more details in 1.6.2, “IBM FlashSystem V9000 expansion enclosures” on page 13.
For more details about IBM FlashSystem V9000 scale-up or scale-out solutions, see Chapter 5, “Scalability” on page 179.
1.7 Advanced software features
The IBM FlashSystem V9000 can function as a feature-rich, software-defined storage layer that virtualizes and extends the functionality of all managed storage. These include Real-time Compression, dynamic tiering, thin provisioning, snapshots, cloning, replication, data copy services and high-availability configurations. In this capacity, it acts as the virtualization layer between the host and other external storage systems, providing flexibility and extending functionality to the virtualized external storage capacity.
Up to 32 PB of storage can be managed by a single IBM FlashSystem V9000 array, and because the storage is virtualized, volumes can be nondisruptively moved between external and internal storage capacity. This functionality enables agile integration into existing storage environments with seamless data migration between IBM FlashSystem V9000 and existing storage systems. When using Real-time Compression for active data sets, IBM FlashSystem V9000 can increase the effective capacity of your flash memory up to five times.
1.7.1 Advanced functions for data reduction
The IBM FlashSystem V9000 employs several features to assist with the reduction of data and the ability to increase its effective capacity.
IBM Real-time Compression
The IBM Real-time Compression within the IBM FlashSystem V9000 addresses the requirements of primary storage data reduction, without sacrificing performance by the use of dedicated compression acceleration hardware. It does so by implementing a purpose-built technology called Real-time Compression using the Random Access Compression Engine (RACE).
Customers can expect data reduction and effective capacity increases of up to 5x for relevant data sets. When the initial virtual disk (VDisk) volume, also known as the logical unit number (LUN), is created and a thin provisioned volume is allocated, then as data is stored into the VDisk it is compressed in real time.
Thin provisioning
In a shared storage environment, thin provisioning is a method for optimizing the use of available storage. It relies on allocation of blocks of data on demand versus the traditional method of allocating all of the blocks up front. This methodology eliminates almost all white space, which helps avoid the poor usage rates (often as low as 10%) that occur in the traditional storage allocation method where large pools of storage capacity are allocated to individual servers but remain unused (not written to).
Thin-provisioned flash copies
Thin-provisioned IBM FlashCopy® (or snapshot function in the GUI) uses disk space only when updates are made to the source or target data and not for the entire capacity of a volume copy.
1.7.2 Data migration
The IBM FlashSystem V9000 provides online volume migration while applications are running, which is possibly the greatest single benefit for storage virtualization. This capability enables data to be migrated on and between the underlying storage subsystems without any effect on the servers and applications. In fact, this migration is performed without the knowledge of the servers and applications that it even occurred. The IBM FlashSystem V9000 delivers these functions in a homogeneous way on a scalable and highly available platform over any attached storage and to any attached server.
1.7.3 Advanced copy services
Advanced copy services are a class of functionality within storage arrays and storage devices that enable various forms of block-level data duplication locally or remotely. By using advanced copy services, you can make mirror images of part or all of your data eventually between distant sites. Copy services functions are implemented within an IBM FlashSystem V9000 (FlashCopy and Image Mode Migration), or between one IBM FlashSystem V9000 and another IBM FlashSystem V9000 in three different modes:
•Metro Mirror
•Global Mirror
•Global Mirror with Change Volumes
Remote replication can be implemented using both Fibre Channel and Internet Protocol (IP) network methodologies.
FlashCopy
FlashCopy is the IBM branded name for point-in-time copy, which is sometimes called time-zero (T0) copy. This function makes a copy of the blocks on a source volume and can duplicate them on 1 - 256 target volumes.
Remote mirroring
The three remote mirroring modes are implemented at the volume layer within the IBM FlashSystem V9000. They are collectively referred to as remote copy capabilities. In general, the purpose of these functions is to maintain two copies of data. Often, but not necessarily, the two copies are separated by distance. The remote copy can be maintained in one of two modes: synchronous or asynchronous, with a third asynchronous variant:
•Metro Mirror is the IBM branded term for synchronous remote copy function.
•Global Mirror is the IBM branded term for the asynchronous remote copy function.
•Global Mirror with Change Volumes is the IBM branded term for the asynchronous remote copy of a locally and remotely created FlashCopy.
1.7.4 External virtualization
The IBM FlashSystem V9000 includes data virtualization technology to help insulate hosts, hypervisors, and applications from physical storage. This enables them to run without disruption, even when changes are made to the underlying storage infrastructure. The IBM FlashSystem V9000 functions benefit all virtualized storage. For example, Easy Tier and Real-time Compression help improve performance and increase effective capacity, where high-performance thin provisioning helps automate provisioning.
These benefits can help extend the useful life of existing storage assets, reducing costs. And because these functions are integrated into the IBM FlashSystem V9000, they can operate smoothly together, reducing management effort.
1.7.5 Easy Tier
Easy Tier is a performance function that automatically migrates or moves extents of a volume to or from one storage tier to another storage tier. Starting with IBM FlashSystem V9000 Version 7.8, Easy Tier supports four kinds of storage tiers.
Consider the following information about Easy Tier:
•Easy Tier monitors the host volume I/O activity as extents are read and migrates the most active extents to higher performing tiers.
•The monitoring function of Easy Tier is continual but, in general, extents are migrated over a 24-hour period. As extent activity cools, Easy Tier moves extents to slower performing tiers.
•Easy Tier creates a migration plan that organizes its activity to decide how to move extents. This plan can also be used to predict how extents will be migrated.
For more information about Easy Tier see 3.2.1, “IBM Easy Tier” on page 99.
1.8 IBM HyperSwap
IBM HyperSwap is new as of IBM FlashSystem V9000 firmware 7.5. HyperSwap capability enables each volume to be presented by two IBM FlashSystem V9000 I/O groups. The configuration tolerates combinations of node and site failures, using host multipathing driver based on the one that is available for the IBM FlashSystem V9000.
IBM FlashSystem V9000 V7.7.1 and later provides GUI management of the HyperSwap function.
For details about implementation and HyperSwap capability, see Chapter 11, “IBM HyperSwap” on page 485.
1.9 Transparent cloud tiering (V7.8)
IBM FlashSystem V9000 V7.8 includes transparent cloud tiering technology. This capability provides increased flexibility to protect data by leveraging the cloud as snapshot targets and restore snapshots from the cloud. Snapshots are encrypted and compressed before being uploaded to the cloud for more security, lower telecommunication costs, and lower cloud storage costs.
Transparent cloud tiering allows you to configure a cloud account on the system to create and restore cloud snapshots of system volumes.
Configuring transparent cloud tiering on the system, requires you to enable a cloud connection to a supported cloud service provider. The system supports IBM SoftLayer®, OpenStack Swift, and Amazon S3 cloud service providers.
1.10 Licensing
The base license that is provided with your system includes the use of its basic functions. However, extra licenses can be purchased to expand the capabilities of your system. Administrators are responsible for purchasing extra licenses and configuring the systems within the license agreement, which includes configuring the settings of each licensed function on the system.
The base 5639-RB7 license entitles IBM FlashSystem V9000 (machine type 9846/9848) to all of the licensed functions, such as Virtualization, FlashCopy, Global Mirror, and Metro Mirror, and Real-time Compression. Any connected storage that is not an IBM FlashSystem V9000 requires the External Virtualization license that is a per-terabyte (TB) capacity unit of metric. You use the Licensed Functions window in the System Setup wizard to enter External Virtualization licenses purchased for your system.
For more details about licensing, see 2.7.2, “Software and licensing” on page 92.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.