IBM Elastic Storage System models
In this appendix, we describe information and differences between the three generations of IBM Elastic Storage Systems (IBM ESS) that are used today. Understanding the nomenclature and the differences between these generations is helpful when you are designing your IBM ESS environment, especially if different generations of IBM ESS coexist in the same IBM Spectrum Scale cluster.
This appendix includes the following topics:
IBM Elastic Storage System model specifications
This section describes the specifications of the various IBM ESS models.
First-generation IBM Elastic Storage System specifications
The first-generation IBM ESS models were first announced and shipped in 4Q2014. These models were withdrawn from marketing in January 2018.
The end-of-service date for the first-generation IBM ESS was announced in July 2020, with an effective date of 31 December 2021. First-generation IBM ESS models are no longer serviced or supported by IBM.
The first-generation IBM ESS consisted of the models that are listed in Table A-1.
Table A-1 First-generation IBM ESS building block models
|
IBM ESS model
|
Enclosure type /U/ IBM machine type - model
|
Enclosures
|
Number of drives1
|
|
GS1
|
EXP24S
2U
No machine/type.
Specified as feature
code on POWER processor-based
IBM ESS servers. |
1
|
24 solid-state drives (SSDs)
|
|
GS2
|
2
|
46 serial-attached SCSI (SAS) + 2 SSDs or 48 SSDs
| |
|
GS4
|
4
|
94 SAS + 2 SSDs or 96 SSDs
| |
|
GS6
|
6
|
142 SAS + 2 SSDs
| |
|
GL2
|
DCS3700
4U
Machine type
1818-80E
|
2
|
116 NL-SAS + 2 SSDs
|
|
GL4
|
4
|
232 NL-SAS + 2 SSDs
| |
|
GL6
|
6
|
348 NL-SAS + 2 SSDs
|
1 On the hard disk drive (HDD) configurations, the two SSDs are not visible user capacity. These SSDs hold data that is used internally for IBM Spectrum Scale RAID metadata only.
Consider the following points regarding network interface cards (NICs):
•The IBM ESS GSx and GLx models support three high-speed data NICs per
IBM POWER8 processor-based data server.
IBM POWER8 processor-based data server.
•Because two POWER8 processor-based data servers are used per IBM ESS building block, a total of six high-speed data NICs are used per IBM ESS GSx or GLx building block.
Second-generation IBM Elastic Storage System specifications
This section describes the second-generation IBM ESS specifications.
The second-generation IBM ESS models were announced in April 2017. At the time of writing, these models are still actively supported and serviced.
The second-generation IBM ESS does not include a solution IBM machine type. The second-generation IBM ESS server and storage hardware components feature the following IBM ESS-unique machine types:
•IBM machine type 5148 for the IBM ESS POWER8 processor-based servers
•IBM machine type 5147 for the IBM ESS IBM Storage Enclosures
These machine types uniquely identify these hardware components as part of an IBM ESS solution.
The second-generation IBM ESS models are available in the following types:
•GSxS models for all SSD storage
•GHxx models hybrids of some SSD storage and some HDD storage
•GLxS models for high-capacity HDD storage
•GLxC models for ultra high-capacity and density HDD storage
These GLxS and GSxS models were first announced and available in the second quarter of 2017. They are fully supported by IBM Service and Support for hardware, firmware, software updates, and maintenance.
The SSD flash storage second-generation IBM ESS models GSxS specifications are listed in Table A-2.
Table A-2 Second-generation IBM ESS SSD flash building block models
|
IBM ESS model
|
Enclosure U/Number drives IBM machine type - model
|
Enclosures
|
Number of drives
|
|
GS1S
|
2U24
Machine type 5147-024
|
1
|
24 SSDs
|
|
GS2S
|
2
|
48 SSDs
| |
|
GS4S
|
4
|
96 SSDs
|
The HDD disk drive storage second-generation IBM ESS models “GLxS” specifications are shown in Table A-3.
Table A-3 Second-generation IBM ESS HDD hard disk building block models
|
IBM ESS model
|
Enclosure U /Number drives
IBM machine type - model
|
Enclosures
|
Number of drives1
|
|
GL1S
|
5U84
Machine type 5147-084
|
1
|
82 NL-SAS + 2 SSDs
|
|
GL2S
|
2
|
166 NL-SAS + 2 SSDs
| |
|
GL3S
|
3
|
250 NL-SAS + 2 SSDs
| |
|
GL4S
|
4
|
334 NL-SAS + 2 SSDs
| |
|
GL5S
|
5
|
418 NL-SAS + 2 SSDs
| |
|
GL6S
|
6
|
502 NL-SAS + 2 SSDs
|
1 On the HDD configurations, the two SSDs are not visible user capacity. These SSDs hold data that is used internally for IBM Spectrum Scale RAID metadata only.
Consider the following points regarding NICs:
•The IBM ESS GSxS and GLxS models support three high-speed data NICs per POWER8 processor-based data server.
•Because two POWER8 processor-based data servers are used per IBM ESS building block, a total of six high-speed data NICs are used per IBM ESS GSxS or GLxS building block.
The ultra-high capacity HDD storage second-generation IBM ESS models GLxC specifications are as listed in Table A-4.
Table A-4 Second-generation IBM ESS HDD hard disk building block models
|
IBM ESS model
|
Enclosure U /Number drives
IBM machine type - model
|
Enclosures
|
Number of drives1
|
|
GL1C
|
4U106
Machine type 5147-106
|
1
|
104 NL-SAS + 2 SSDs
|
|
GL2C
|
2
|
210 NL-SAS + 2 SSDs
| |
|
GL3C
|
3
|
316 NL-SAS + 2 SSDs
| |
|
GL4C
|
4
|
422 NL-SAS + 2 SSDs
| |
|
GL5C
|
5
|
528 NL-SAS + 2 SSDs
| |
|
GL6C
|
6
|
634 NL-SAS + 2 SSDs
| |
|
GL8C
|
8
|
846 NL-SAS + 2 SSDs
|
1 On the HDD configurations, the two SSDs are not visible user capacity. These SSDs hold data that is used internally only for IBM Spectrum Scale RAID metadata.
Consider the following points regarding NICs:
•The IBM ESS GLxC models include another SAS adapter per POWER8 processor-based server, compared to the GSxS and GLxS models. This addition provides enough SAS connections for the GL8C model to be built.
•The GLxC models support two high-speed data NICs per POWER8 processor-based data server.
•Because two POWER8 processor-based data servers are used per IBM ESS building block, a total of four high-speed data NICs are used per IBM ESS GLxC building block.
When the ultra-high-capacity IBM ESS GLxC models are configured, ensure that the NIC capacity and bandwidth is sufficient for your workload. If you need higher network bandwidth capability than four high-speed data NICs per POWER8 processor-based data server can provide, choose one of the other IBM ESS models that has three high-speed data NICs per POWER8 processor-based data server.
Second-generation IBM Elastic Storage System Hybrid models
A primary use of the GHxy hybrid models is for a less expensive option to combine an IBM Spectrum Scale SSD storage tier with an IBM Spectrum Scale HDD capacity storage tier.
The IBM ESS GHxy Hybrid models combined one or two drawers of SSD with two or four drawers of 5U84 HDD storage. You can think of these models as a combination of the GSxS models and the GLxS models.
IBM ESS Hybrid models are IBM ESS storage building blocks that consist of SSD flash 5147-024 Storage Enclosures and HDD 5147-084 Storage Enclosures. The IBM ESS Hybrid models (model GHxy) are a blend of 2U24 SSD enclosures and 5U84 HDD enclosures, where “x” is the number of 2U24 SSD enclosures, and “y” is the number of 5U84 HDD enclosures. The following models are available:
•GH12
•GH22
•GH14
•GH24
The IBM ESS GHxy models use the same POWER8 processor-based data server. They also feature the same NIC capacity as the IBM ESS GSxS and GLxS models.
Third-generation IBM Elastic Storage System 3000 specifications
The IBM Elastic Storage System 3000 (IBM ESS 3000) is one of the third-generation
IBM ESS models.
IBM ESS models.
The specifications of the IBM ESS 3000 model are listed in Table A-5.
Table A-5 Third-generation IBM ESS NVMe flash building block model
|
IBM ESS model
|
Enclosure U/Number drives
IBM machine type - model
|
Enclosures
|
Number of drives
|
|
IBM ESS 3000
|
2U24
Machine type 5141-AF8
|
1
|
12 or 24 NVMe.
Available NVMe drive sizes are 3.84 TB, 7.68 TB, or 15.36 TB.
|
Third-generation IBM Elastic Storage System 3200 specifications
The IBM Elastic Storage System 3200 (IBM ESS 3200) is one of the third-generation
IBM ESS models.
IBM ESS models.
The specifications of the IBM ESS 3200 model are listed in Table A-6.
Table A-6 Third-generation IBM ESS NVMe flash building block model
|
IBM ESS model
|
Enclosure U/Number drives IBM machine type - model
|
Enclosures
|
Number of drives
|
|
IBM ESS 3200
|
2U24
Machine type 5141-FN1
|
1
|
12 or 24 NVMe.
Available NVMe drive sizes are 3.84 TB, 7.68 TB, or 15.36 TB.
|
Third-generation IBM Elastic Storage System 3500 specifications
The IBM Elastic Storage System 3500 is one of the third-generation IBM ESS offerings.
The IBM ESS 3500 model includes NVMe flash storage.
The specifications of the IBM ESS 3500 model are listed in Table A-7.
Table A-7 Third-generation IBM ESS NVMe flash building block model
|
IBM ESS model
|
Enclosure U/Number drives IBM machine type - model
|
NVMe drives
|
Drive size
|
|
IBM ESS 3500 or
IBM ESS 3500 H0 |
2U24
Machine type 5141-FN2
|
NVMe 12 or 24
|
3.84 TB, 7.68 TB, or 15.36 TB
|
The IBM ESS 3500Hx, shown in Table A-8, includes machine type 5141-FN2 and 1-4 model 5147-102.
Table A-8 IBM ESS 3500Hx enclosure specifications
|
IBM ESS Model
|
Enclosure U/Number drives IBM machine type - model
|
NVMe
|
HDD enclosure
|
HDD drives
|
Drive size
|
|
IBM ESS 3500H1
|
2U24 Machine type 5141-FN2 + 1@4U external storage Model 5147-102
|
Quantity:
12 or 24 Size: 3.84 TB, 7.68 TB, or
15.36 TB |
1
|
52 or 102
|
HDD:
10 TB, 14 TB, 18 TB, or 20 TB |
|
IBM ESS 3500H2
|
2U24 Machine type 5141-FN2 + 2@4U external storage Model 5147-102
|
2
|
204
| ||
|
IBM ESS 3500H3
|
2U24 Machine type 5141-FN2 + 3@4U external storage Model 5147-102
|
3
|
306
| ||
|
IBM ESS 3500H4
|
2U24 Machine type 5141-FN2 + 4@4U external storage Model 5147-102
|
4
|
408
|
The IBM ESS 3500Cx, shown in Table A-9, includes machine type 5141-FN2 logtip only NVMe drives and 1 - 4 model 5147-102.
Table A-9 IBM ESS 3500Cx enclosure specifications
|
IBM ESS Model
|
Enclosure U/Number drives IBM machine type - model
|
NVMe
|
HDD enclosure
|
HDD drives
|
Drive size
|
|
IBM ESS 3500C1
|
2U24 Machine type 5141-FN2 + 1@4U external storage Model 5147-102
|
Quantity: 4
(used for logtip)
|
1
|
52 or 102
|
HDD: 10 TB, 14 TB, 18 TB, or
20 TB |
|
IBM ESS 3500C2
|
2U24 Machine type 5141-FN2 + 2@4U external storage Model 5147-102
|
2
|
204
| ||
|
IBM ESS 3500C3
|
2U24 Machine type 5141-FN2 + 3@4U external storage Model 5147-102
|
3
|
306
| ||
|
IBM ESS 3500C4
|
2U24 Machine type 5141-FN2 + 4@4U external storage Model 5147-102
|
4
|
408
|
Third-generation IBM Elastic Storage System 5000 specifications
The IBM Elastic Storage System 5000 is part of the third generation of IBM ESS offerings.
The third-generation IBM ESS 5000 model with IBM POWER9 processor-based data servers and 5147-092 or 5147-106 Storage Enclosures was announced in July 2020 and general availability was August 2020.
The specifications of the IBM ESS 5000 SL model are listed in Table A-10.
Table A-10 Third-generation IBM ESS 5000 SL model - HDD hard disk building block
|
IBM ESS model
|
Enclosure U/Number drives
IBM machine type - model
|
Enclosures
|
Number of drives1
|
|
SL1
|
5U92 Machine type 5147-092
HDD drive sizes that are available:
6 TB, 10 TB, 14 TB, or 16 TB |
1
|
90 NL-SAS + 2 SSDs
|
|
SL2
|
2
|
182 NL-SAS + 2 SSDs
| |
|
SL3
|
3
|
274 NL-SAS + 2 SSDs
| |
|
SL4
|
4
|
366 NL-SAS + 2 SSDs
| |
|
SL5
|
5
|
458 NL-SAS + 2 SSDs
| |
|
SL6
|
6
|
550 NL-SAS + 2 SSDs
| |
|
SL7
|
7
|
642 NL-SAS + 2 SSDs
|
1 On the HDD configurations, the first storage enclosure has two slots that are taken up by SSDs, which are not visible user capacity. These SSDs hold data that is used internally by
IBM Spectrum Scale RAID metadata only.
IBM Spectrum Scale RAID metadata only.
The specifications of the IBM ESS 5000 SC model are listed in Table A-11.
Table A-11 Third-generation IBM ESS 5000 SC model - HDD hard disk building block
|
IBM ESS model
|
Enclosure U/Number drives
IBM machine type - model
|
Enclosures
|
Number of drives1
|
|
SC1
|
4U106 Machine type 5147-106
HDD drive sizes that are available: 10 TB, 14 TB, or
16 TB |
1
|
104 NL-SAS + 2 SSDs
|
|
SC2
|
2
|
210 NL-SAS + 2 SSDs
| |
|
SC3
|
3
|
316 NL-SAS + 2 SSDs
| |
|
SC4
|
4
|
422 NL-SAS + 2 SSDs
| |
|
SC5
|
5
|
528 NL-SAS + 2 SSDs
| |
|
SC6
|
6
|
634 NL-SAS + 2 SSDs
| |
|
SC7
|
7
|
740 NL-SAS + 2 SSDs
| |
|
SC8
|
8
|
846 NL-SAS + 2 SSDs
| |
|
SC9
|
9
|
952 NL-SAS + 2 SSDs
|
1 On the HDD configurations, the first storage enclosure has two slots that are taken up by SSDs, which are not visible user capacity. These SSDs hold data that is used internally by
IBM Spectrum Scale RAID metadata only.
IBM Spectrum Scale RAID metadata only.
First-generation IBM Elastic Storage System models overview
You might encounter IBM Spectrum Scale clusters that include the first-generation IBM ESS models. Although these models continue to operate, IBM support for these models ceased on 31 December 2021.
First-generation IBM Elastic Storage System models
The first-generation IBM ESS models were first announced and shipped in 4Q2014. These models, including upgrades, were withdrawn from marketing in January 2018. As of
31 December 2021 the first-generation IBM ESS models are no longer supported by
IBM Service and Support for hardware and software maintenance. First-generation IBM ESS models also no longer receive IBM ESS solution stack hardware, firmware, and software updates.
31 December 2021 the first-generation IBM ESS models are no longer supported by
IBM Service and Support for hardware and software maintenance. First-generation IBM ESS models also no longer receive IBM ESS solution stack hardware, firmware, and software updates.
The first-generation IBM ESS models were available in the following types:
•GSx models
•GLx models
Depending on the first-generation IBM ESS model, the following configurations were available:
•2U24 or storage enclosures hold SSDs or 10K RPM HDD storage drives.
•4U60 of storage enclosures to hold high capacity Nearline SAS HDD storage drives.
The first-generation IBM ESS has a solution IBM machine type: 5146. This machine type applies to the first-generation IBM ESS only. Each first-generation IBM ESS building block consists of two POWER8 processor-based servers and a specific number of storage enclosures (1, 2, 4, or 6), depending on the model.
The POWER8 processor-based data servers were model S822L (IBM machine type 8247-22L). The storage enclosures can be one of the following configurations:
•IBM EXP24S SFF (small form factor) Gen2-bay drawer (2U 24 drives), which are used in models GS1, GS2, GS4, or GS6.
•IBM DCS3700 (IBM machine type 1818-80E) is 4U and contain 58 HDD drives each, which is used in models GL2, GL4, and GL6.
The first-generation IBM ESS GS models can use 2.5-inch 10 K rpm HDD (1. 2 TB) or 2.5-inch SSD (400 GB or 800 GB).
The first-generation IBM ESS GL models can use 3.5-inch NL-SAS HDDs (2 TB, 4 TB, or 6 TB).
All IBM ESS models can be mixed in an IBM Spectrum Scale cluster to meet the specific needs of client applications that use IBM ESS storage. For example, a common best practice is to use SSDs for IBM Spectrum Scale metadata and NL-SAS for IBM Spectrum Scale data storage.
First-generation IBM Elastic Storage System GSx models
The GSx models that used 2U24 storage enclosures were available with all SSD or all 10K RPM, high-speed SAS HDD storage.
The first-generation IBM ESS GSx models are shown in Figure A-1.
Figure A-1 First-generation IBM ESS GSx models
A primary use of the GSx models with SSD was for IBM Spectrum Scale metadata, or for high-speed HDD storage tier.
First-generation IBM Elastic Storage System GLx models
The HDD models of the first-generation IBM ESS (see Figure A-2) used 4U60 Storage enclosures (IBM 1818-80E).
Figure A-2 First-generation IBM ESS GLx models
The primary usage of the GSx models with NL-SAS HDDs was for IBM Spectrum Scale capacity tiers.
First-generation IBM Elastic Storage System Data Servers
The first-generation IBM ESS uses a pair of POWER8 processor-based (machine type 8247-22L) servers as IBM Spectrum Scale Data servers, which are cross-connected to all storage for high availability (HA). These 8247-22L servers run Red Hat Enterprise Linux (RHEL) Big Endian operating system.
The first-generation IBM ESS Management Server (IBM EMS) is also an POWER8 processor-based (machine type 8247-21L) server. This 8247-21L server also runs RHEL Big Endian operating system.
The first-generation POWER processor-based IBM ESS servers were managed by an
IBM Hardware Management Console (HMC), which is a hardware component that is commonly used to manage IBM Power server environments.
IBM Hardware Management Console (HMC), which is a hardware component that is commonly used to manage IBM Power server environments.
HMC for the first-generation IBM Elastic Storage System
The first-generation IBM ESS used an IBM HMC to manage the IBM Power server portion of the first-generation IBM Elastic Storage Server (IBM ESS) environment.
|
Note: IBM ESS required an HMC for the first-generation only. In later IBM ESS generations, the function of the HMC was integrated into the IBM Electronic Service Agent (ESA) running on the second-generation IBM EMS.
|
First-generation IBM Elastic Storage System Storage Enclosures
In this section, we describe the storage enclosures that were available on the first-generation IBM ESS models.
2U24 (feature code on 8247-22L)
The first-generation IBM ESS 2U24 Storage Enclosures can be ordered with SSDs or HDDs.
These storage enclosures do not have a separate IBM machine type. They were feature codes of POWER8 processor-based 8247-22L Data Servers in the first-generation IBM ESS.
The IBM 2U24 Storage Enclosure is shown in Figure A-3.
Figure A-3 IBM 2U24 Storage Enclosure
The following storage drives are available in these 2U24 enclosures:
•2.5-inch 10 K rpm HDD (1.2 TB)
•2.5-inch SSD (400 GB or 800 GB)
All 2U24 enclosures within one first-generation IBM ESS building block must have the same SSD or HDD size. Different IBM ESS building blocks can have different storage enclosures or drive sizes.
4U60 (1818-80E)
The first-generation IBM ESS HDD models (GL2, GL4, and GL6) used IBM machine type 1818-80E Storage Enclosures for HDD storage. All 1818-80E Storage Enclosures are 4U60 in size (4 rack units, 60 HDD slots available) and were populated with 58 HDDs (no partial population was available).
The first 1818-80E in a first-generation IBM ESS also contained two SSDs that hold internal IBM Spectrum Scale RAID metadata. The capacity of these two SSDs is not visible nor available as user capacity for the IBM Spectrum Scale and IBM ESS file system. The
IBM 1818-80 4U60 Storage Enclosure is shown in Figure A-4.
IBM 1818-80 4U60 Storage Enclosure is shown in Figure A-4.
Figure A-4 IBM 1818-80E 4U60 Storage Enclosure
Each IBM 1818-80E used in the first-generation IBM ESS, which held 58 HDDs.
All HDDs in a first-generation IBM ESS building block with 4U60 must have same HDD size. In the 1818-80E, 2 TB, 4 TB, and 6 TB HDDs were available.
Second-generation IBM Elastic Storage System overview
The second generation of IBM ESS was announced in April 2017 and was withdrawn from marketing in December 2021. IBM no longer provides upgrades for second-generation
IBM ESS. At the time of writing, IBM continues to provide full hardware and software service and support for second-generation IBM ESS.
IBM ESS. At the time of writing, IBM continues to provide full hardware and software service and support for second-generation IBM ESS.
The second-generation IBM ESS does not include a solution IBM machine type. The
IBM ESS server and storage hardware components feature the following IBM ESS-unique machine types:
IBM ESS server and storage hardware components feature the following IBM ESS-unique machine types:
•IBM machine type 5148 for the IBM ESS POWER8 processor-based servers
•IBM machine type 5147 for the IBM ESS IBM Storage Enclosures
These machine types uniquely identify these hardware components as part of an IBM ESS solution.
The available models of second-generation IBM ESS were available in the following categories:
•SSD flash storage IBM ESS models
•HDD IBM ESS models
•Hybrid IBM ESS models that consist of enclosures of SSD flash drives and HDDs.
IBM Elastic Storage System GSxS models
The second-generation IBM ESS GSxS models are all-SSD models. GSxS models are not available with HDD storage.
Figure A-5 shows the three IBM ESS GSxS models. A primary usage of the GSxS models with SSD is for IBM Spectrum Scale metadata, or to provide a high-speed SSD storage tier.
Figure A-5 Second-generation IBM ESS GSxS models
|
Note: IBM ESS performance data is available on request to your IBM or IBM Business Partner representative. They use the IBM File and Object Solution Design Engine to estimate performance that is based on your workload and network environment.
Optimum IBM ESS performance is derived from an unconstrained IOR benchmark for 100% sequential read numbers by using unconstrained InfiniBand networks. Other networks (such as 100 Gb Ethernet (GbE), 40 GbE, and 10 GbE) have more overhead than InfiniBand and typically lower aggregate bandwidth capabilities results. For more information, contact your IBM or IBM Business Partner representative.
|
IBM Elastic Storage System GHxy Hybrid models with SSD and HDD
The IBM ESS GHxy Hybrid models combined 1 - 2 drawers of SSDs with 2 - 4 drawers of HDD storage. A primary usage of the GHxy hybrid models is for a less expensive option to combine an IBM Spectrum Scale SSD storage tier with an IBM Spectrum Scale HDD capacity storage tier.
Second-generation IBM Elastic Storage System Hybrid models
IBM ESS Hybrid models are IBM ESS storage building blocks that consist of SSD flash 5147-024 Storage Enclosures and HDD 5147-084 Storage Enclosures. The IBM ESS Hybrid models (model GHxy) are a blend of 2U24 SSD enclosures and 5U84 HDD enclosures; “x” is the number of 2U24 SSD enclosures, and “y” is the number of 5U84 HDD enclosures.
The following models are available:
•GH12
•GH22
•GH14
•GH24
Figure A-6 shows the IBM ESS GHxy models.
Figure A-6 Second-generation IBM ESS GHxy models
|
Note: IBM ESS performance data is available on request to your IBM or IBM Business Partner representative. Use the IBM File and Object Solution Design Engine to estimate performance that is based on your workload and network environment.
Optimum IBM ESS performance is derived from an unconstrained IOR benchmark for 100% sequential read numbers by using unconstrained InfiniBand networks. Other networks (such as 100 GbE, 40 GbE, and 10 GbE) have more overhead than InfiniBand and typically lower aggregate bandwidth capabilities results. For more information, contact your IBM or IBM Business Partner representative.
|
IBM Elastic Storage System GLxS models
The IBM ESS GLxS models provided high capacity HDD storage by using a 5U84 storage enclosure. A primary use of the GLxS models is to provide a high capacity IBM Spectrum Scale HDD storage tier.
Figure A-7 shows the various models that are available.
Figure A-7 Second-generation IBM ESS GLxS models
|
Note: IBM ESS performance data is available on request from your IBM or IBM Business Partner representative. They use the IBM File and Object Solution Design Engine to estimate performance that is based on your workload and network environment.
Optimum IBM ESS performance is derived from an unconstrained IOR benchmark for 100% sequential read numbers by using unconstrained InfiniBand networks. Other networks (such as 100 GbE, 40 GbE, and 10 GbE) have more overhead than InfiniBand and typically lower aggregate bandwidth capabilities results. For more information, contact your IBM or IBM Business Partner representative.
|
IBM Elastic Storage System GLxC models
IBM ESS GLxC models use the 4U106 storage enclosure that was pioneered and designed for use in the Sierra and Summit supercomputers at the US Department of Energy1.
Figure A-8 shows the IBM ESS GLxC models.
Figure A-8 Second-generation IBM ESS GLxC models
A primary use of the GLxC models is to provide an ultra high-capacity, high-density
IBM Spectrum Scale HDD storage tier.
IBM Spectrum Scale HDD storage tier.
You can expect performance of the IBM ESS GLxC models to be similar to the corresponding IBM ESS GLxS model.
If you want to upgrade an existing IBM ESS GLxC model to an equivalent IBM ESS 5000 SC model, submit a Request for Price Quotation (RPQ) to IBM to replace the POWER8 processor-based data servers with POWER9 processor-based data servers. The upgrade is a destructive upgrade. Data must be backed up from the IBM ESS GLxC; the upgrade and replacement of the POWER8 processor-based data servers with POWER9 processor-based data servers is performed; the IBM ESS software stack must be reloaded; and the
IBM Spectrum Scale file system is redefined on the upgraded IBM ESS 5000 SC model.
IBM Spectrum Scale file system is redefined on the upgraded IBM ESS 5000 SC model.
For more information about comparison tables with IBM ESS model specifications for SSD and HDD drive sizes and counts by model, see Appendix A, “IBM Elastic Storage System models” on page 65.
IBM Elastic Storage System 3200
The IBM ESS 3200 is a third-generation IBM ESS model. IBM ESS 3200 was announced in May 2021 and includes the following features:
•Based on a 2U24 storage enclosure with PCI Gen4-based x86 server canisters to provide improvements in throughput and bandwidth capability.
•Uses enterprise class NVMe drives:
– Can be ordered half-populated with 12 NVMe drives or fully populated with 24 NVMe drives.
– If ordered half-populated, you can add the remaining 12 NVMe drives non-disruptively (the added drives must be the same size as the first 12 drives).
•Designed to provide the following benefits:
– High performance: NVMe flash storage with up to 80 GBps read throughput per 2U building block.
– Designed to provide edge capability and global data access: This solution can be deployed in data centers or at the edge, incorporating and processing data that then uses IBM Spectrum Scale Active File Management (AFM) to share the data globally.
– Simplicity: Containerized software installation and upgrade, plus a powerful management GUI, minimize the demands on IT staff time and expertise.
•Deployed by using containerized Red Hat Ansible playbooks that provide improved ease of use and orchestration of complex IBM ESS administration tasks, such as cluster configuration, file system creation, and code update.
The third-generation IBM ESS-3200 addresses the challenges of managing today’s data.
IBM ESS 3200 delivers high-performance, software-defined flash storage. The IBM ESS 3200 builds on years of experience and combines IBM Spectrum Scale software with fast NVMe storage technology to offer industry-leading file management capabilities.
IBM ESS 3200 delivers high-performance, software-defined flash storage. The IBM ESS 3200 builds on years of experience and combines IBM Spectrum Scale software with fast NVMe storage technology to offer industry-leading file management capabilities.
The IBM ESS 3200 builds on and extends a track record of meeting the needs of the smartest and most demanding organizations. The IBM ESS 3200 is up to 100% faster than the previous generation of IBM ESS NVMe storage.
Figure A-9 shows the IBM ESS 3200 NVMe storage building block.
Figure A-9 Third-generation IBM ESS 3200 model
IBM ESS 3200 provides a high-performance tier of IBM Spectrum Scale file storage for various AI, analytics, and big data applications. IBM ESS 3200 keeps GPUs in AI workloads running at peak performance.
Like all IBM ESS models, IBM ESS runs IBM Spectrum Scale RAID erasure coding, which provides superior consistent high performance; mitigation of storage hardware failures; and intelligent monitoring, management, and dynamic tuning of all IBM ESS offerings for
IBM Spectrum Scale data.
IBM Spectrum Scale data.
|
Note: IBM ESS performance data is available on request from IBM or an
IBM Business Partner representative. The IBM File and Object Solution Design Engine is used to estimate performance that is based on your workload and network environment. Optimum IBM ESS performance is derived from an unconstrained IOR benchmark for 100% sequential read numbers by using unconstrained InfiniBand networks. Other networks (such as 200 GbE, 100 GbE, 40 GbE, and 10 GbE) feature more overhead than InfiniBand and typically lower aggregate bandwidth capabilities results.
For more information, contact your IBM or IBM Business Partner representative.
|
IBM ESS 3200 is the simplest way yet for users to deploy IBM Spectrum Scale.
IBM Spectrum Scale is included in a pre-configured system. Installations and updates are delivered by using containerized software that speeds and simplifies the process.
IBM Spectrum Scale is included in a pre-configured system. Installations and updates are delivered by using containerized software that speeds and simplifies the process.
A storage specialist from IBM System Lab Services implementation is not required if you have an IBM ESS and IBM Spectrum Scale system and you are comfortable with IBM ESS implementations. It is much easier to install, and maintenance can be performed by your IT staff.
If you are unfamiliar with IBM Spectrum Scale and IBM ESS, as a best practice, use
IBM System Lab Services to ensure high satisfaction with your initial IBM ESS 3200 installation.
IBM System Lab Services to ensure high satisfaction with your initial IBM ESS 3200 installation.
IBM Elastic Storage System 3000
The IBM ESS 3000 model was the initial model of the third-generation IBM ESS.
IBM ESS 3000 was announced in October 2019 and includes the following features:
IBM ESS 3000 was announced in October 2019 and includes the following features:
•Based on IBM FlashSystem® 9150 Storage Enclosure and x86 server canister technology.
•Uses enterprise class NVMe drives:
– Can be ordered half-populated with 12 NVMe drives or fully populated with 24 NVMe drives.
– If ordered half-populated, you can add the remaining 12 NVMe drives non-disruptively (the additional drives must be of the same size as the first 12 drives).
•Designed to provide the following benefits:
– High-performance NVMe storage for High-Performance Tier (HPT) for IBM Spectrum Scale in a compact, energy-efficient 2U24 form factor with two highly available (HA), cross-connected, and x86-based IBM Spectrum Scale Data Servers.
– The necessary bandwidth to fully use compute GPUs for AI and analytics workloads, which provide up to 40 GBps of throughput in a 2U24 form factor.
– Improved ease of use, faster time to installation, and faster time to value.
•Deployed by using containerized Red Hat Ansible playbooks that provide improved ease of use and orchestration of complex IBM ESS administration tasks, such as cluster configuration, file system creation, and code update.
The third-generation IBM ESS 3000 model provides high-performance NVMe storage. Figure A-10 shows the IBM ESS 3000 NVMe storage building block.
Figure A-10 Third-generation IBM ESS 3000 model
A primary use of the IBM ESS 3000 model with NVMe is for fast IBM Spectrum Scale metadata, or to provide a dense, high-speed, and potentially mobile NVMe storage tier.
|
Note: IBM ESS performance data is available on request from your IBM or IBM Business Partner representative. They use the IBM File and Object Solution Design Engine to estimate performance that is based on your workload and network environment.
Optimum IBM ESS performance is derived from an unconstrained IOR benchmark for 100% sequential read numbers by using unconstrained InfiniBand networks. Other networks (such as 200 GbE, 100 GbE, 40 GbE, and 10 GbE) have more overhead than InfiniBand and typically lower aggregate bandwidth capabilities results.
For more information, contact your IBM or IBM Business Partner representative.
|
For ease of management and upgrading, IBM ESS 3000 provides a simplified, faster installation experience compared to previous-generation IBM ESS models. IBM ESS 3000 hardware is set up by an IBM System Services Representative (IBM SSR). If the client is unfamiliar with IBM Spectrum Scale and IBM ESS, as a best practice, use IBM System Lab Services to ensure high satisfaction with your initial IBM ESS 3000 installation.
Differences between the IBM Elastic Storage System generations
The following section describes the component differences between the first-, second-, and third-generation IBM ESS models. It is helpful if these different generations of IBM ESS are in the same IBM Spectrum Scale cluster.
Hardware
The IBM ESS generations feature the following major hardware differences:
•First generation:
– Uses POWER8 processor-based 8247-21L for IBM EMS and POWER8 processor-based 8247-22L for the IBM ESS Data Servers, all running RHEL Big Endian. This generation IBM ESS has an overall solution machine type 5146.
– The first-generation IBM ESS for 2U24 storage used the IBM EXP24 2U24 Storage Enclosure, which is specified as feature codes on the 8247-22L Data Server. The HDD storage used IBM 1818-80E Storage Enclosures for large capacity.
– It required an IBM Power HMC to manage the IBM Power servers.
– This generation requires an RHEL Big Endian-based IBM EMS (8247-21L). One RHEL Big Endian-based IBM EMS is required per IBM Spectrum Scale cluster that includes first-generation IBM ESS.
•Second generation:
– Uses POWER8 processor-based 5148-21L for IBM EMS and POWER8 processor-based 5148-22L for the IBM ESS Data Servers, all running RHEL Little Endian. These machine types are unique to the IBM ESS and uniquely identify these POWER8 processor-based servers as part of an IBM ESS solution. The second-generation IBM ESS does not have an overall solution machine type.
– This generation uses IBM 5147-024 Storage Enclosures for SSD storage. For high capacity HDD storage, it uses IBM 5147-084 Storage Enclosures or 5147-106 Storage Enclosures. These machine types are unique to IBM ESS and uniquely identify these storage enclosures as part of an IBM ESS solution.
•Third generation:
– The IBM ESS 3000 has machine type 5141-AF8, and can be managed by a 5148-21L POWER8 processor-based IBM EMS or a 5105-22E POWER9 processor-based
IBM EMS.
IBM EMS.
– The IBM ESS 3200 has machine type 5141-FN1, and can be managed a 5105-22E POWER9 processor-based IBM EMS.
– The IBM ESS 5000 has a common server IBM machine type of 5105-22E POWER9 processor-based servers for Data Servers, protocol nodes, and IBM EMS.
– An IBM ESS 5000 storage building block can use 5147-092 Storage Enclosures (SL model) or 5147-106 Storage Enclosures (SC model). You cannot intermix 5147-092 and 5147-106 in the same IBM ESS 5000 storage building block.
– IBM ESS 5000 must be managed by a 5105-22E POWER9 processor-based IBM EMS. The second-generation 5148-21L POWER8 processor-based management server cannot support the IBM ESS 5000 or the IBM ESS 3200.
Consider the following points:
•Only the first-generation IBM ESS models require an HMC for the IBM Power servers. In the second- and third-generation IBM ESS models, the HMC function was integrated into the IBM EMS ESA.
•IBM fully supports all three multiple generations of IBM ESS in the same IBM Spectrum Scale cluster. If doing so, make sure you satisfy the requirement that the appropriate
IBM EMS generation also is attached to the IBM Spectrum Scale cluster.
IBM EMS generation also is attached to the IBM Spectrum Scale cluster.
|
Note: All IBM ESS generations can all coexist in a IBM Spectrum Scale cluster.
|
A typical first-generation IBM ESS deployment is shown in Figure A-11.
Figure A-11 Typical first-generation IBM ESS components and deployment
A typical IBM ESS second-generation deployment is shown in Figure A-12.
Figure A-12 IBM ESS second-generation components and deployment
The components and deployments of IBM ESS first generation is compared to IBM ESS second generation in Figure A-13. Consider the following major differences that are featured in the second generation:
•Does not have a solution machine type that is equivalent to “5146”. Instead, the second-generation IBM ESS components are uniquely identified as machine type 5148 servers or machine type 5147 Storage Enclosures.
•Runs RHEL Little Endian.
•Does not require an HMC for the IBM Power servers.
Figure A-13 Differences between IBM ESS first- and second-generation components
IBM third-generation IBM ESS 5000 models are similar in configuration to second-generation IBM ESS models, with the following differences or similarities:
•IBM 5148 POWER8 processor-based servers are replaced with IBM 5105-22E POWER9 processor-based servers.
•IBM ESS 5000 SL models use IBM 5147-092 Storage Enclosures, whereas the second-generation IBM ESS GLxS models used IBM 5147-084 Storage Enclosures.
•IBM ESS 5000 SC models use the same IBM 5147-106 Storage Enclosures, as the second-generation IBM ESS GLxC models.
•No third-generation IBM ESS Hybrid model exists that is equivalent to the second-generation IBM ESS GHxx models. Instead, use combinations of IBM ESS 3200, IBM ESS 5000, or IBM ESS 3000 as suitable.
IBM fully supports first-, second-, and third-generation IBM ESS models co-existing in an
IBM Spectrum Scale cluster.
IBM Spectrum Scale cluster.
Software
The IBM ESS generations feature the following major software differences and nomenclature:
•First-generation IBM ESS runs RHEL Big Endian.
•Second- and third-generation IBM ESS runs RHEL Little Endian.
•IBM ESS software stacks are under the nomenclature IBM ESS 5.x.x.
•IBM ESS software stacks with the nomenclature IBM ESS 5.2.x have (now end of service) IBM Spectrum Scale 4.2.x.
•IBM ESS software stacks with the nomenclature IBM ESS 5.3.x have IBM Spectrum Scale 5.0.x.
•IBM ESS software stacks with the nomenclature IBM ESS 6.x.x have IBM Spectrum Scale 5.1.x.
•The various IBM ESS software stack component levels for IBM ESS software stacks are documented in the IBM Spectrum Scale RAID FAQ in the Online IBM Documentation.
|
Note: An IBM Spectrum Scale cluster can coexist nodes that run Big Endian and Little Endian operating systems.
|
IBM Elastic Storage System Data Server IO node
The IBM Spectrum Scale Data Server machine type and model for first-generation IBM ESS is IBM machine type 8247-22L. This machine type and model is the same as the equivalent POWER8 processor-based compute servers.
The second-generation IBM ESS uses IBM machine type 5148-22L for the IBM ESS Data Servers. This machine type is unique to IBM ESS and is a different machine type from the stand-alone POWER8 processor-based compute servers.
Two identical data servers always must be used in each IBM ESS building block.
The third-generation IBM ESS 3000 and IBM ESS 3200 Data Servers are based on x86 servers. They are integrated in a single 2U24 enclosure system with the NVMe storage that also is within the 2U24 IO enclosures.
The third-generation IBM ESS 5000 uses IBM machine type 5105-22E for the IBM ESS Data Servers. This POWER9 processor-based server is unique to IBM ESS.
IBM Elastic Storage System Management Server
The EMS machine type and model for first generation is IBM machine type 8247-21L. This machine type and model is the same as the equivalent POWER8 processor-based compute servers.
The second-generation IBM ESS uses IBM machine type 5148-21L for the IBM EMS. This IBM machine type is unique to IBM ESS to differentiate it from standard 8247-21L POWER8 processor-based compute servers.
An IBM Spectrum Scale cluster with IBM ESS must have at least one IBM EMS for each “Endianness”.
If you have first-generation IBM ESS in an IBM Spectrum Scale cluster, you must have a Big Endian IBM EMS. This Big Endian EMS 8247-21L supports the first-generation IBM ESS systems only.
If you have second- and third-generation IBM ESS systems in an IBM Spectrum Scale cluster, you must have at least one Little Endian IBM EMS. Use the suitable Little Endian EMS to support your second- and third-generation IBM ESS systems.
Consider the following points”
•The IBM ESS 3000 has machine type 5141-AF8 and can be managed by a 5148-21L POWER8 processor-based IBM EMS or a 5105-22E POWER9 processor-based
IBM EMS.
IBM EMS.
•The IBM ESS 5000 has a common server IBM machine type of 5105-22E POWER9 processor-based servers for data servers, protocol nodes, and IBM EMS.
•An IBM ESS 5000 storage building block can use 5147-092 Storage Enclosures (SL model) or 5147-106 Storage Enclosures (SC model). You cannot intermix 5147-092 and 5147-106 in the same IBM ESS 5000 storage building block.
IBM ESS 5000 must be managed by a 5105-22E POWER9 processor-based IBM EMS. The second-generation 5148-21L POWER8 processor-based management server cannot support the IBM ESS 5000 or the IBM ESS 3200.
Hardware Management Console
An HMC is used by the first-generation IBM ESS systems only. For the second- and third-generation IBM ESS, the HMC features are integrated into the Little Endian IBM EMS.
IBM Spectrum Scale for IBM Elastic Storage System licensing
IBM Spectrum Scale and IBM ESS licensing is described in detail in the IBM Spectrum Scale FAQ Online at IBM Documentation, in Section 13 “Licensing and Pricing Questions”
of IBM Spectrum Scale Overview.
of IBM Spectrum Scale Overview.
For more information about IBM Spectrum Scale and IBM ESS Licensing, see IBM Spectrum Scale: IBM Elastic Storage System Licensing Information.
Capacity licensing on second- and third-generation IBM Elastic Storage System
All second- and third-generation IBM ESS models are licensed by capacity. Instead of the standard IBM Spectrum Scale software license with a “Per TiB” metric, a specific
IBM Spectrum Scale for IBM ESS license with a specific IBM Program ID and with a
“Per Disk” metric is used.
IBM Spectrum Scale for IBM ESS license with a specific IBM Program ID and with a
“Per Disk” metric is used.
The “Per Disk” metric licenses the IBM Spectrum Scale for IBM ESS by the number of SSDs or HDDs in the IBM ESS. No difference exists in the license charge for the size of the SSD or HDD. Therefore, the licensing is the same for an IBM ESS model, regardless if it has small
4 TB HDDs or large 14 TB HDDs.
4 TB HDDs or large 14 TB HDDs.
|
Note: IBM Spectrum Scale for IBM ESS (current IBM Program IDs are 5765-DAE for Data Access Edition and 5765-DME for Data Management Edition) is a specific IBM Program ID for use on IBM ESS. It uses a different metric than the IBM Spectrum Scale software-only capacity license (5641-DAx or 5641-DMx), which is licensed by TiB. (The “x” indicates 1, 3, or 5 years of IBM software service and support.)
Also, unlike IBM Spectrum Scale software (which IBM clients can manage through
IBM Passport Advantage®), IBM Spectrum Scale for IBM ESS does not have a Passport Advantage part number. Renewals with 5765-DAE and 5765-DME can be managed with by working with your IBM Technical Support Services (TSS) representative. It is possible to license your IBM ESS by using IBM Spectrum Scale “Per TiB” capacity pricing. For more information about how to use this licensing model on your IBM ESS, see IBM Spectrum Scale Licensing question 2.2 and IBM Spectrum Scale: IBM Elastic Storage System Licensing Information.
|
IBM Spectrum Scale RAID technical overview
The following section provides a high-level, technical overview of the IBM Spectrum Scale RAID that is used in all IBM ESS models.
The IBM Spectrum Scale RAID software that is used in the IBM ESS solution runs on SAS disks in just a bunch of disks (JBOD) arrays. IBM Spectrum Scale RAID on IBM ESS provides JBOD cost reduction while simultaneously providing enterprise class reliability.
Different IBM ESS models provide solid-state drives (SSDs or NVMe) when more performance is needed. The IBM ESS solution does not require or use any kind of external RAID controller or acceleration.
IBM Spectrum Scale RAID supports multiple RAID codes and distributes client data, redundancy information, and spare space across the disks in such a way that if a physical disk loss or even a group of physical disk loss occurs, it does not affect data availability.
Instead of relying on the disks or conventional RAID controller only, IBM Spectrum Scale RAID implements erasure coding, which is embedded within the IBM Spectrum Scale file system to provide reliability and high performance. This configuration detects and reports storage media faults and read or write errors, and resolves other data integrity problems.
IBM Spectrum Scale RAID also implements an end-to-end checksum from the storage media all the way out to the IBM Spectrum Scale client.
IBM Spectrum Scale RAID erasure codes
IBM Spectrum Scale RAID in the IBM ESS supports different data protection algorithms and can detect and correct up to two or three concurrent storage media faults per erasure code array.
The options for RAID configuration are eight stripes of data plus two or three parity stripes that use Reed-Solomon codes or one stripe of data plus two or three replica stripes. The data plus parity or replica stripes, which are known as tracks, are shown in Figure A-14.
Figure A-14 IBM Spectrum Scale RAID tracks
End-to-end checksum
If you use the IBM Spectrum Scale software client to access data on the IBM ESS, the
IBM Spectrum Scale client is aware if the IBM Spectrum Scale file system is based on IBM Spectrum Scale RAID Network Shared Disks. If this configuration is used, an 8-byte checksum is calculated during a write operation, which is appended to the data, and sent over the network to the IBM Spectrum Scale RAID server. The checksum is verified and then
IBM Spectrum Scale RAID writes the data along with its checksum on the disks and logs the version number on its metadata.
IBM Spectrum Scale client is aware if the IBM Spectrum Scale file system is based on IBM Spectrum Scale RAID Network Shared Disks. If this configuration is used, an 8-byte checksum is calculated during a write operation, which is appended to the data, and sent over the network to the IBM Spectrum Scale RAID server. The checksum is verified and then
IBM Spectrum Scale RAID writes the data along with its checksum on the disks and logs the version number on its metadata.
When a read operation is requested, IBM Spectrum Scale RAID verifies checksum and version on its metadata. If it is OK, it sends the data to the client. If it is not OK, the data is rebuilt based on parity or replication and then sent to the client along with newly generated checksum.
The end-to-end checksum feature provides a robust means to prevent and correct silent disk errors or missing disk writes. This feature provides an essential level of reliability to ensure data integrity and prevent file system checks, which ensure reliability and uptime for ever larger petabyte-scale file systems.
Declustered RAID arrays
IBM Spectrum Scale RAID implements its own data and spare disk layout scheme that reduces overhead and mitigates the performance impact to users when recovering from disk failures. IBM Spectrum Scale RAID spreads or declusters user data, redundancy information, and spare space across all the disks of the array instead of leaving all spare space in a single disk. A conventional 1+1 RAID layout is compared to a declustered array in Figure A-15.
Figure A-15 Declustered array versus a 1+1 array
Consider an example where seven stripes of data are on each disk. Figure A-15 shows the left three arrays of two disks in a replicated 1+1 configuration and a spare. On the left, you can see the data stripes spreading all over the seven disks of the declustered array.
In a failure of one disk, all data from the remaining disks of the array must be replicated to the spare disk on the traditional 1+1 array. On the declustered array, the replication occurs on spare space of all the remaining disks, which can decrease the rebuild impact from three to four times.
The IBM Spectrum Scale RAID overhead reduction is shown in Figure A-16.
Figure A-16 Array rebuild operation
On the IBM ESS, using RAID 8+2 or 2-way replication (1+2), if one disk loss occurs, the rebuild operation starts with low priority with even lower impact for the clients. With this array configuration, the rebuild is considered critical and run on high priority only if two concurrent disk losses occur.
By using 8+3 RAIDs or 3-way replication (1+3), the rebuild operation becomes critical only if three concurrent disk losses occur in the same declustered array.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.