IBM Spectrum Storage family
This chapter provides an overview of IBM software-defined storage (SDS) product offerings with a focus on the IBM Spectrum Storage family of products and their capabilities and benefits.
The products are organized by their functions within the SDS control plane or SDS data plane. The control plane is the software layer that manages administrative functions such as configuration, monitoring, replication, policy automation, and provisioning, for software-defined storage resources. Data is processed and stored in the data plane.
This chapter includes the following sections:
2.1 SDS architecture
Figure 2-1 shows the IBM SDS architecture with a mapping of the Spectrum Storage family of products across the SDS control plane and data plane.
Figure 2-1 IBM Spectrum Storage family mapped to the SDS control plane and data plane
Table 2-1 is a high-level overview of the IBM Spectrum Storage family. It pairs descriptions with the products that provide the functions.
Table 2-1 IBM Spectrum Storage Family descriptions
 
 
 
Spectrum Storage family member
 
Description
Former name
SDS Control Plane
Spectrum Control
Automated control and optimization of storage and data infrastructure
IBM Tivoli® Storage Productivity Center, management layer of IBM Virtual Storage Center, and IBM Spectrum Control™ Base Edition (IBM Storage Integration Server)
 
 
 
IBM Spectrum Protect™
Optimized data protection for client data through backup and restore capabilities
IBM Tivoli Storage Manager Suite for Unified Recovery
IBM Spectrum Protect Snapshot
SDS Data Plane
Spectrum Virtualize
Core SAN Volume Controller function is virtualization that frees client data from IT boundaries
IBM SAN Volume Controller
 
 
 
Spectrum Accelerate
Enterprise storage for cloud that is deployed in minutes instead of months
IBM XIV as software
 
 
 
Spectrum Scale
Storage scalability to yottabytes and across geographical boundaries
IBM General Parallel File System (GPFS™)
 
 
 
Spectrum Archive
Enables long-term storage of low activity data
IBM Linear Tape File System™ Enterprise Edition, Library Edition, and Single Drive Edition
2.2 SDS control plane
The control plane is a software layer that manages the virtualized storage resources. It provides all the high-level functions that are needed by the customer to run the business workload and enable optimized, flexible, scalable, and rapid provisioning storage infrastructure capacity. These capabilities span functions like storage virtualization, policy automation, analytics and optimization, backup and copy management, security, and integration with the API services, including other cloud provider services.
This section describes the IBM software product offerings that provide the building blocks for the SDS control plane:
•IBM Spectrum Control
•IBM Spectrum Protect
2.2.1 IBM Spectrum Control
IBM Spectrum Control provides efficient infrastructure management for virtualized, cloud, and software-defined storage to simplify and automate storage provisioning, capacity management, availability monitoring, and reporting.
Key capabilities
Spectrum Control helps organizations transition to new workloads and updated storage infrastructures by providing these tools:
•A single console for managing all types of data on disk, flash, file, and object storage systems
•Simplified visual administration tools that include an advanced web-based user interface, a VMware vCenter plug-in, and IBM Cognos® Business Intelligence with pre-designed reports
•Analytics-driven tiered storage optimization that automatically moves data to the most cost-effective tier to help support business required service levels
•Near-instant data protection and recovery capabilities that protect data without affecting application performance
•Choice of license plans with three deployment options that can fit any size organization, including mid-sized organizations.
Benefits
The Spectrum Control benefits address the two most significant storage costs: Physical capacity and storage administration. Here are several highlights of those broad based benefits:
•Volume-level, cross-platform automated storage tiering that reduces users’ cost of storage by up to 50%1
•Cloud and software defined storage that provides a service catalog, automated provisioning, and optimized utilization of pools of storage
•Intelligent Performance Management with application and department views of the storage infrastructure
Figure 2-2 shows the Spectrum Control dashboard window where all the managed resources in your data server are presented in an aggregated view.
Figure 2-2 Single dashboard for monitoring all storage components
IBM Spectrum Control data management and storage management solutions provide comprehensive monitoring, automation, and analytics. They address the two most significant storage costs: Physical capacity and storage administration. You can get important insights into cloud, virtualized, and software defined storage environments.
IBM data management and storage management solutions can help with these tasks:
•Identify and categorize storage assets for file, block, and object data
•Generate departmental and application views of storage
•Optimize data placement within the infrastructure
•Identify unused and wasted storage space
IBM Spectrum Control is a comprehensive, end-to-end data and storage management solution that monitors, automates, and analyzes multivendor storage environments.
IBM Spectrum Control provides a single point of control that helps administrators manage every aspect of the storage infrastructure, between the hosts, through the fabric, and down to the physical disks, across multi-site storage environments. IBM Spectrum Control helps consolidate management of file, object, SAN, server-based, and software- defined storage and enables you to manage storage your way. Optimized interfaces are included for storage specialists, VMware vCenter users, and remote management with open application programming interfaces (APIs) such as OpenStack and IBM Cloud Orchestrator.
IBM Spectrum Control delivers a comprehensive solution that can significantly improve monitoring, automation, and analytics capabilities in multivendor storage environments:
•Heterogeneous storage systems, ports, and switches
IBM Spectrum Control provides device-level, integrated storage infrastructure management capabilities for managing both IBM and non-IBM storage systems that are managed with IBM SAN Volume Controller. Agent and agentless options are available. Heterogeneous storage support is offered by using the Storage Networking Industry Association (SNIA) Storage Management Initiative Specification (SMI-S) v1.0.2, v1.1, and v1.5 or later.
•Device management
IBM Spectrum Control helps monitor and manage performance and measure service levels by storing received performance statistics in database tables for future use. IBM Spectrum Control sets performance thresholds for devices based on selected performance metrics, generating alerts when those thresholds are exceeded. These capabilities help the solution simplify the complex monitoring of multiple SAN-attached storage devices.
IBM Spectrum Control includes these device management capabilities:
 – Proactive performance management from a single, integrated console for monitoring storage devices
 – Monitoring of metrics, such as I/O rates and cache utilization, and support for storage optimization through the identification of the best logical unit numbers (LUNs) across multiple storage subsystems
 – The measuring and tracking of service levels by storing historical performance statistics for analysis and report generation
 – Generation of timely alerts that enable event action by setting performance thresholds based on business policies
•SAN fabric management
IBM Spectrum Control supports multivendor SANs and includes automatic resource and topology discovery, monitoring and alerts, zone control, and SAN error-prediction capabilities. IBM Spectrum Control diagnostic capabilities show which resources are impacted by an availability or performance issue in the SAN.
IBM Spectrum Control includes these SAN fabric monitoring capabilities:
 – Automated device discovery, topology rendering, and zone control for SANs
 – Multiple SAN views, including physical, logical, and zone
 – Diagnostic capabilities that pinpoint resources affected by an availability or performance issue
 – Performance monitoring at the port and switch level
 – Enterprise scalability that enables upgrading from SAN islands to enterprise SANs
•Storage provisioning
IBM Spectrum Control includes optimized interfaces for storage specialists, VMware vCenter users, IBM Cloud Orchestrator environments, and open APIs such as Storage Management API for Cloud (SMAC). It can help simplify and standardize storage provisioning, so users can quickly get the storage that they need to do their jobs.
•Advanced alert management
Advanced alert management in IBM Spectrum Control can help administrators identify and resolve problems fast. Policy-based automation enables automated responses, based on business policies. Event processing can reduce the complexity of managing alerts in large storage environments.
•Scalable copy services management
IBM Spectrum Control includes copy services management capabilities that help simplify the data replication process. It can automate complex replication tasks without scripts, and can scale to support hundreds of replication sessions across thousands of volumes. IBM Spectrum Control supports the following session types: IBM FlashCopy®, IBM XIV snapshots, IBM Basic HyperSwap®, Metro Mirror, Global Mirror, Metro/Global Mirror, and Global Copy.
For copy services management, IBM Spectrum Control can use IBM DS8000®, XIV, IBM SAN Volume Controller (SVC), IBM Storwize V7000 Unified, IBM Storwize V7000, IBM Storwize V3700, and IBM Storwize V3500.
IBM Spectrum Control integrates with IBM Tivoli System Automation to enable a single point of control for both multi-tiered business applications and the corresponding copy services between two sites. This integration offers the following capabilities:
 – Integrated monitoring and control of storage hardware replication sessions
 – Automatic start, stop, and move of tiered applications
 – Lower-priority business application shutdown to keep higher-priority business applications running
 – End-to-end application recovery
•Tiered storage analysis and optimization
IBM Spectrum Control Tiered Storage Optimizer helps organizations optimize price and performance without manual tuning. Tiered Storage Optimizer uses proprietary analytics from IBM Research that can help keep data on the correct storage tier over time. Storage tier decisions are based on data usage patterns, rather than on initial projections or guesswork. Tiered Storage Optimizer moves data volumes between storage pools and systems.
•SAN Error Predictor
SAN Error Predictor is an autonomic computing capability that is designed to help predict SAN network outages before they occur. This functionality incorporates predictive failure analysis into the storage network environment that allows administrators to be more proactive in ensuring SAN availability. SAN Error Predictor enables network errors to be resolved before they affect data and application availability.
•Performance analysis and management
IBM Spectrum Control integrates built-in, context- sensitive performance management that is accessible to any administrator. Performance management is server-centric, so administrators can more easily visualize storage performance problems from the user perspective. Administrators can quickly generate performance graphs from several perspectives and align them with a simple mouse click. Performance scenarios can be cloned, so administrators can look back to better understand how problems first developed
IBM Spectrum Control is available in a range of options to fit practically any size environment and budget. For example, IBM Spectrum Control Storage Insights is a cloud-based solution that deploys in minutes and includes advanced analytics for storage tier optimization. On-premises solutions include standard and advanced editions, available in capacity or per-enclosure license options. IBM Virtual Storage Center bundles IBM Spectrum Control Advanced Edition with multi-brand storage virtualization and snapshot management. IBM Spectrum Control Base consolidates VMware integration for IBM storage systems into a single download.
IBM Spectrum Control Standard Edition
IBM Spectrum Control Standard Edition is available in capacity or per-enclosure license options. The IBM Spectrum Control Standard Edition license is priced by the capacity of the storage systems that it manages. The IBM Spectrum Control Standard Select Edition license is priced by the number of storage enclosures that it manages.
These licenses provide the following features:
•Single dashboard view of the storage environment, which enables you to manage storage systems, hypervisors, servers, and networks
•Capacity and usage monitoring in addition to management and planning for the heterogeneous storage environment
•Performance monitoring for storage systems and networks
•Health and alerting for storage systems, hypervisors, servers, and networks
•Modeling of departments and applications to understand the consumer view of the environment
•Administrative tools, which includes the following:
 – VMware vCenter plug-in, which enables you to manage and provision virtual storage in a vSphere environment, and to view reports about storage systems that are monitored by IBM Spectrum Control
 – Reports from IBM Cognos Business Intelligence, which provides reporting and analytics information
This license includes a license for IBM Copy Services Manager (formerly known as IBM Tivoli Storage Productivity Center for Replication). Copy Services Manager manages 2-site replication, 3-site replication, and advanced copy services.
For more information about Spectrum Control Standard, see IBM Spectrum Family: IBM Spectrum Control Standard Edition, SG24-8321.
IBM Spectrum Control Advanced Edition
IBM Spectrum Control Advanced Edition is available in capacity or per-enclosure license options like Standard Edition. The IBM Spectrum Control Advanced Edition license is priced by the capacity of the storage systems that it manages. The IBM Spectrum Control Advanced Select Edition is priced by the number of storage enclosures that it manages.
These licenses contain everything that is in the IBM Spectrum Control Standard Edition license, and the following additional features:
•Analytics-driven, tiered-storage optimization that automatically moves data to the most cost-effective tier
•Pool balancing
•Block storage provisioning
•Application-aware snapshot-based protection that is offered by IBM Spectrum Protect Snapshot (formerly known as IBM FlashCopy Manager)
For more information, see the IBM Knowledge Center:
The IBM Tivoli Storage Productivity Center license is now IBM Spectrum Control Standard Edition.
IBM Spectrum Control Advanced Edition is a new software-only product that offers the same functions as the Storage Analytics Engine product that was part of IBM Virtual Storage Center.
For a complete list of devices that can be used with IBM Spectrum Control, see the following website:
IBM Spectrum Control Base Edition
IBM Spectrum Control Base Edition is a centralized server that consolidates a range of IBM storage provisioning, automation, and monitoring solutions through a unified server platform. As shown in Figure 2-3, it provides a single-server back-end location and enables centralized management of IBM storage resources for the use of independent software vendor (ISV) platforms and frameworks. These frameworks currently include VMware vCenter Server, VMware vSphere Web Client (vWC), and VMware vSphere Storage APIs for Storage Awareness (VASA), VMware vCenter Operations Manager (vCOps), VMware vCenter Orchestrator (vCO). Spectrum Control Base Edition is available for no extra fee to storage-licensed clients.
Figure 2-3 Spectrum Control Base Edition
 
Control plane: IBM Spectrum Control Base Edition is not in the data path. It runs in the control plane as shown in Figure 2-1 on page 14.
Spectrum Control Base Edition integrates between environments like OpenStack components, VMware, and SNIA SMI-S driven environments, and the storage subsystems. Spectrum Control Base supports requests from APIs like VASA and Next Generation Client (NGC).
Spectrum Control Base Edition provides common services such as authentication, high availability, and storage configuration for all IBM Storage in homogeneous and heterogeneous multiple target environments. Spectrum Control Base Edition manages IBM XIV Storage System, IBM DS8000 series, IBM SAN Volume Controller, the IBM Storwize family, and third-party storage subsystems.
Support for VMware virtual volumes
IBM Spectrum Control Base Edition delivers comprehensive storage virtualization support, by using VMware Virtual Volume (VVOL) technology.
 
Note: The virtual volume functionality is supported by the IBM FlashSystem V9000 (7.6 or later), IBM XIV (11.5.1 or later), and IBM Spectrum Virtualize/Storwize (7.6 or later) storage systems.
The VVOL architecture, introduced in VMware VASA 2.0, preserves the concept of a traditional datastore, maintaining familiarity and compatibility with previous data storage implementations. However, the virtual disks in a VVOL datastore can use different storage attributes (services), which can include thin/thick provisioning, snapshot support, encryption, and so on. Moreover, each VVOL can be managed independently. This implementation improves system scalability, ensures granular management, uses hardware features, and improves performance of storage systems at the VM level, providing a complete end-to-end cloud solution. Another entity, a storage space, includes one or more services, and can be assigned to different storage customers.
For more information, see the IBM Spectrum Control Base Edition information in the IBM Knowledge Center:
For more information about VVOL architecture implementation in XIV, see IBM Spectrum Control Base: Enabling VMware Virtual Volumes with IBM XIV Storage System, REDP-5183.
For more information about VVOL implementation with IBM Spectrum Virtualize, see Quick-start Guide to Configuring VMware Virtual Volumes for Systems Powered by IBM Spectrum Virtualize, REDP-5321.
Spectrum Control in an OpenStack environment
The Spectrum Control OpenStack Cinder driver enables your OpenStack powered cloud environment to use your Spectrum Control installation for block storage provisioning. Spectrum Control provides block storage provisioning capabilities that a storage administrator can use to define the properties and characteristics of storage volumes within a particular service class. For example, a block storage service class can define RAID levels, tiers of storage, and various other storage characteristics.
IBM Storage Insights
IBM Storage Insights is an analytics-driven, storage resource management solution that is delivered over the cloud. The solution uses cloud technology to provide visibility into on-premises storage to help you optimize you storage environments. This SaaS solution, which runs on SoftLayer®, can deploy in as little as 5 minutes and show actionable insights in 30 minutes.
Storage Insights is a cloud data and storage management service that is deployed in a secure and reliable cloud infrastructure that provides the following features:
•Accurately identify and categorize storage assets
•Monitor capacity and performance from the storage consumer's view, including server, application, and department-level views
•Increase capacity forecasting precision by using historical growth metrics
•Reclaim unused storage to delay future purchases and improve utilization
•Optimize data placement based on historical usage patterns that can help lower the cost of storage by 50% per GB
Figure 2-4 shows an example of the Storage Insights dashboard.
Figure 2-4 Storage Insights dashboard
For more information, see the following websites:
•IBM Spectrum Control Storage Insights product page
•IBM Spectrum Control Storage Insights
2.2.2 IBM Spectrum Protect
IBM Spectrum Protect is an integrated, industry-leading, data protection solution that covers your entire enterprise from small systems to virtual environments to enterprise applications. It provides a set of product features that allow you to design adaptive and comprehensive data protection solutions. It is a comprehensive data protection, recovery, and resiliency solution for software-defined, virtual, physical, and cloud environments.
Key capabilities
IBM Spectrum Protect has the following key capabilities:
•Protects virtual, physical, and cloud data with one solution.
•Native object storage support.
•Simplified user experience that simplifies backups by consolidating administration tasks under a single pane of glass.
•Reduction of backup and recovery infrastructure costs with inline data deduplication.
•Incremental forever backups with progressive incremental methods that back up only new or changed versions of files, which can greatly reduce data redundancy, network bandwidth, and storage pool consumption as compared to traditional methods.
•Supports many file systems and platforms, including IBM DB2®, Oracle, SAP, SAP HANA, Microsoft SQL Server, Microsoft Exchange, and IBM Lotus® Domino®. Support for extra platforms is available from IBM Business Partners through the partner ecosystem.
•Delivers full capabilities at significant savings for small and mid-sized organizations.
For an updated list of the available products and support in the IBM Spectrum Protect family, see the following website:
Benefits
IBM Spectrum Protect provides a single point of control and administration for storage management needs.
IBM Spectrum Protect data deduplication
Inline data deduplication improves storage utilization, and eliminates the need for a dedicated data deduplication appliance. It provides the following benefits:
•Significant scalability increase with up to 100 TB daily deduplication ingest throughput including ingest, deduplication, replication, and maintenance.2
•Improved efficiency with inline data deduplication reduces storage requirements for server-side data deduplication. New, efficient, server-side inline data deduplication in IBM Spectrum Protect 7.1.3 and later allows the use of inexpensive backup storage pool disks.
•No need for identify and reclamation processing and a reduced need for offline reorganization.
•Deduplicated storage pools can natively use on-premises object storage or off-premises cloud-based object storage.
When the IBM Spectrum Protect server receives data from a client, the server identifies duplicate data extents and stores unique instances of the data extents in a directory-container storage pool. If the same byte pattern occurs many times, data deduplication greatly reduces the amount of data that must be stored or transferred. In addition to whole files, IBM Spectrum Protect can also deduplicate parts of files that are common with parts of other files.
IBM Spectrum Protect provides the following types of data deduplication:
•Server-side data deduplication
The server identifies duplicate data extents and moves the data to a directory-container storage pool. The server-side process uses inline data deduplication, where data is deduplicated at the same time that the data is written to a directory-container storage pool. Deduplicated data can also be stored in other types of storage pools. With inline data deduplication on the server, the need for reclamation is eliminated.
•Client-side data deduplication
With this method, processing is distributed between the server and the client during a backup process. The client and the server identify and remove duplicate data to save storage space on the server. In client-side data deduplication, only compressed, deduplicated data is sent to the server. The server stores the data in the compressed format that is provided by the client. Client-side data deduplication can reduce the amount of data that is sent over the local area network (LAN). It can also reduce processing power and time that is required to remove duplicate data on the server. It can also improve database performance because the client-side data deduplication is also inline. You can combine both client-side and server-side data deduplication in the same production environment. The ability to deduplicate data on either the client or the server provides flexibility in terms of resource utilization, policy management, and data protection.
•Progressive incremental backup
In a progressive incremental backup process, the IBM Spectrum Protect server monitors client activity and backs up any files that have changed since the initial full backup. Entire files are backed up so that the server does not need to reference base versions of the files. This backup technique eliminates the need for multiple full backups of client data, thus saving network resources and storage space.
Server-side compression
With IBM Spectrum Protect 7.1.5, you can increase the amount of available space in a storage pool by enabling inline compression. When you enable inline compression, data is compressed as it is written to a storage pool. Data compression is available for directory-container and cloud-container storage pools.
Inline compression works on non-encrypted data in both cloud and directory container pools. It has negligible impact on performance.
IBM Spectrum Protect Operations Center
IBM Spectrum Protect Operation Center is a light-weight management application that offers the daily dashboard and management interface for the IBM Spectrum Protect servers. It provides overview, visibility, automation, and control over the environment, and enables administrators to manage multiple IBM Spectrum Protect servers, including server diagnostics and email reporting on a single window as shown in Figure 2-5.
Figure 2-5 Operation Center
The Operation Center follows a hub-spoke model, with the first IBM Spectrum Protect server connected to it acting as the hub server. This system in turn connects to the spoke servers to query information and run commands. There is a correlation between the version of the Operation Center and the version of the hub server as new functions are introduced over time, requiring updates on the IBM Spectrum Protect servers.
The IBM Spectrum Protect Operation Center can be deployed on the system that hosts the primary IBM Spectrum Protect server or on, for example, a virtual machine.
For information about compatibility, see the following website:
Also, explore the IBM Spectrum Protect Operation Center live demonstration environment at the following website:
For information about Operation Center hardware and software requirements, see the following website:
IBM Spectrum Protect for Virtual Environments
IBM Spectrum Protect for Virtual Environments (also known as Data Protection for VMware) protects VMware and Microsoft Hyper-V virtual machines.
It can be quickly and easily installed on virtual machines or physical hosts. The web-based portal provides self-service file recovery. No software is required on the user’s system and the user does not need to know about IBM Spectrum Protect.
IBM Spectrum Protect provides the following benefits:
•Improves efficiency with deduplication, incremental “forever” backup, and other advanced IBM technology to help reduce costs
•Simplifies backups and restores for VMware with a practical interface that you can access from within VMware vCenter or a self-service restore portal
•Integrates with VMware snapshots and OpenStack clouds for faster, more frequent backups of your most critical virtual machines
•Flexible recovery and copy options from image-level backups give you the ability to perform recovery at the file, mailbox, database object, volume, or VM image level by using a single backup of a VMware image
•Eliminates overhead with optimized virtual machine backup by supporting VMware vStorage APIs for Data Protection and Microsoft Hyper-V technology, which simplifies and optimizes data protection.
•Instant access and restore that enables the IBM Spectrum Protect server to act as a virtual datastore
•Object-level recovery from image-level backups (MS Exchange, MS SQL Server, and files)
•The ability to use all IBM Spectrum Protect enterprise-class features including Operations Center
Figure 2-6 shows the file-level recovery capabilities of IBM Spectrum Protect for Virtual Environments.
Figure 2-6 Web-based portal for self-service file recovery
For more information, see the following website:
Support for IBM Cleversafe as a target
With IBM Spectrum Protect Version 7.1.5, you can configure cloud-container storage pools to use Cleversafe, an IBM object storage system, to back up data.
The Cleversafe object storage system uses the Simple Storage Service (S3) protocol to communicate with a cloud server. The Cleversafe object storage system can help you simplify storage management, improve server performance, and secure data by using encryption.
For more information, see the version 7.1.5 information in the IBM Knowledge Center:
IBM Spectrum Protect Snapshot
IBM Spectrum Protect Snapshot (formerly Tivoli Storage FlashCopy Manager) delivers high levels of protection for key applications and databases by using advanced integrated application snapshot backup and restore capabilities.
IBM Spectrum Protect Snapshot provides support for IBM Spectrum Scale, IBM Storwize family, IBM DS8000, IBM XIV Storage systems, storage managed with IBM SAN Volume Controller and Spectrum Virtualize, and various virtual environments. It also manages application-aware snapshots on EMC, Hitachi, NetApp, and IBM and VSS-compatible Windows storage.
Traditional backups of files, databases, and similar objects that are sent across networks (LAN or SAN) to backup servers provide a degree of recoverability. However, the speed of backup and recovery is constrained by resource capacity (server, network, storage, and so on). The ability to make point-in-time snapshots with limited impact to the application while rapidly recovering large data sets by reverting to a snapshot greatly reduces the recovery time for these data sets.
The snapshots that are captured by IBM Spectrum Protect Snapshot can be retained as backups on local disk. With optional integration with IBM Spectrum Protect, you can use the full range of advanced data protection and data reduction capabilities such as data deduplication, progressive incremental backup, hierarchical storage management, and centrally managed policy-based administration as shown in Figure 2-7.
Figure 2-7 IBM Spectrum Protect Snapshot storage snapshot capabilities
Because a snapshot operation typically takes much less time than the time for a tape backup, the window during which the application must be aware of a backup can be reduced. This reduction facilitates more frequent backups, which reduces the time that is spent performing forward recovery through transaction logs, increases the flexibility of backup scheduling, and eases administration.
For more information, see the following website:
For hardware and software requirements for IBM Spectrum Protect Snapshot, see the following website:
IBM Spectrum Protect Blueprints
Blueprints can be thought of as the instructions for how to build a physical appliance. A blueprint consists of a prescriptive “cookbook” document that describes up to three reference architectures in detail (including IBM hardware model numbers and configuration requirements) and scripts to help with deployment.
Blueprints help improve time to value with faster server deployment. They can reduce implementation time from hours to as little as 10 minutes with systematic design and build instructions that align with software-defined data protection.
The following platforms are supported:
•Linux and Microsoft Windows
•IBM AIX® on POWER®
•Linux on POWER, big endian
The following storage is supported:
•IBM Storwize V7000, V5000, and V3700
•IBM Elastic Storage Server GL4, GL2 (powered by IBM Spectrum Scale)
Blueprints are available for small (up to 6 TB), medium (6 - 20 TB), and large (20 - 100 TB) environments.
For more information, see the following website:
2.3 SDS data plane
The data plane encompasses the infrastructure where data is processed. It consists of all basic storage management functions that can be started and managed by the control plane, which include:
•Virtualization
•RAID protection
•Tiering
•Copy services (remote, local, synchronous, asynchronous, and point-in-time)
•Encryption
•Data deduplication
The data plane is the interface to the hardware infrastructure where the data is stored. It provides a complete range of data access possibilities, spanning traditional access methods like block I/O (for example, iSCSI) and File I/O (POSIX compliant), to object-storage and Hadoop Distributed File System (HDFS).
This section describes the IBM software product offerings that provide the building blocks for the SDS data plane:
•IBM Spectrum Virtualize
•IBM Spectrum Accelerate™
•IBM Spectrum Scale
•IBM Spectrum Archive™
2.3.1 IBM Spectrum Virtualize
IBM Spectrum Virtualize provides industry-leading storage virtualization that enhances existing storage to improve resource utilization and productivity to achieve a simpler, more scalable, and cost efficient IT infrastructure. IBM SAN Volume Controller delivers the functions of IBM Spectrum Virtualize.
SAN Volume Controller, part of the IBM Spectrum Storage family, has been improving infrastructure flexibility and data economics for more than 10 years. SAN Volume Controller’s innovative data virtualization capabilities provide the foundation for the entire IBM Storwize family. When using SAN Volume Controller integrated into the IBM Virtual Storage Center, you have SDS that spans both the control plane and the data plane.
Key capabilities
The highly scalable storage virtualization system provides common functions, management, and mobility across heterogeneous storage types. Spectrum Virtualize has the following key capabilities:
•Pools storage from multiple systems
•Compresses data with high performance IBM Real-time Compression for extraordinary efficiency
•Enables nondisruptive movement of data among storage systems
•Supports high availability, multi-site configurations
Benefits
The sophisticated virtualization, management, and functions of Spectrum Virtualize provide these storage benefits, among others:
•Improves storage utilization up to 100%
•Supports up to 5x as much data in the same physical space
•Simplifies management of heterogeneous storage systems
•Enables rapid deployment of new storage technologies for greater return on investment (ROI)
•Improves application availability with virtually zero storage-related outages
The SAN Volume Controller combines software and hardware into a comprehensive, modular appliance that uses symmetric virtualization.
Symmetric virtualization is achieved by creating a pool of managed disks (MDisks) from the attached storage systems. Those storage systems are then mapped to a set of volumes for use by the attached host systems. System administrators can view and access a common pool of storage on the SAN. This function helps administrators to use storage resources more efficiently and provides a common base for advanced functions. Figure 2-8 shows the Spectrum Virtualize functions.
Figure 2-8 IBM Spectrum Virtualize functions
Table 2-2 describes the Spectrum Virtualize features and benefits.
Table 2-2 Spectrum Virtualize features and benefits
Feature
Benefits
Single point of control for storage resources
•Designed to increase management efficiency
•Designed to help support business application availability
Pools the storage capacity of multiple storage systems on a SAN
•Helps you manage storage as a resource to meet business requirements, not just as a set of boxes
•Helps administrators better deploy storage as required beyond traditional “SAN islands”
•Can help increase utilization of storage assets
•Insulates applications from physical changes to the storage infrastructure
Clustered pairs of IBM SAN Volume Controller data engines
•Highly reliable hardware foundation
•Designed to avoid single points of hardware failure
IBM Real-time Compression
•Increases effective capacity of storage systems up to five times, helping to lower costs, and floor-space, power, and cooling requirements
•Can be used with a wide range of data, including active primary data, for dramatic savings
•Hardware compression acceleration helps transform the economics of data storage
Innovative and tightly integrated support for flash memory
•Designed to deliver ultra-high performance capability for critical application data
•Move data to and from flash memory without disruption; make copies of data onto hard disk drives (HDDs)
Support for IBM FlashSystem
Enables high performance for critical applications with IBM MicroLatency, coupled with sophisticated functions
Easy-to-use IBM Spectrum Storage family management interface
•Single interface for storage configuration, management, and service tasks regardless of storage vendor
•Helps administrators use their existing storage assets more efficiently
IBM Storage Mobile Dashboard
Provides basic monitoring capabilities to securely check system health and performance
Dynamic data migration
•Migrate data among devices without taking applications that use that data offline
•Manage and scale storage capacity without disrupting applications
Manage tiered storage
Helps balance performance needs against infrastructure costs in a tiered storage environment
Advanced network-based copy services
•Copy data across multiple storage systems with IBM Spectrum Protect Snapshot
•Copy data across metropolitan and global distances as needed to create high-availability storage solutions
Integrated Bridgeworks SANrockIT technology for IP replication
•Optimize use of network bandwidth
•Reduce network costs or speed replication cycles, improving the accuracy of remote data
IBM HyperSwap
•Provide highly available, concurrent access to a single copy of data from data centers up to 300 km (186 miles) apart
•Enable nondisruptive storage and virtual machine mobility between data centers
Thin provisioning and snapshot replication
•Dramatically reduce physical storage requirements by using physical storage only when data changes
•Improve storage administrator productivity through automated on-demand storage provisioning
IBM Spectrum Protect Snapshot application-aware snapshots
•Performs near-instant application-aware snapshot backups, with minimal performance impact for IBM DB2, Oracle, SAP, VMware, Microsoft SQL Server, and Microsoft Exchange
•Provides advanced, granular restoration of Microsoft Exchange data
Virtualizing storage with SAN Volume Controller helps make new and existing heterogeneous storage arrays more effective by including many functions that are traditionally deployed within disk array systems. By including these features in a virtualization system, SAN Volume Controller standardizes functions across virtualized storage for greater flexibility and potentially lower costs.
Figure 2-9 shows how SAN Volume Controller stretch virtual volume standardizes heterogeneous storage across data centers.
Figure 2-9 SAN Volume Controller stretches virtual volume across heterogeneous storage
IBM Spectrum Virtualize and SAN Volume Controller support attachment to servers through Fibre Channel (FC) protocols and Internet Small Computer System Interface (iSCSI) protocols over IP networks at 1 Gbps and 10 Gbps speeds. These configurations can help reduce costs and simplify server configuration. Spectrum Virtualize also supports Fibre Channel over Ethernet (FCoE) protocol.
IBM Spectrum Virtualize and SAN Volume Controller combines hardware and software in an integrated, modular solution that is highly scalable. An I/O group is formed by combining a redundant pair of storage engines that are based on IBM System x server technology. Highly available I/O groups are the basic configuration element of a Spectrum Virtualize cluster.
The configuration flexibility means that your implementation can start small and grow with your business to manage large storage environments. The scalable architecture and tight integration enable your business to take advantage of the high throughput of flash drives. This configuration supports high performance for critical applications.
IBM Spectrum Virtualize also includes the IBM System Storage Easy Tier function, which helps improve performance at lower cost through more efficient use of flash drives. The Easy Tier function automatically identifies highly active data within volumes and moves only the active data to a flash drive. It targets flash use to the data that benefits the most, which delivers the maximum benefit even from small amounts of flash capacity, and helps move critical data to and from flash drives as needed without disruption to applications.
IBM Spectrum Virtualize helps increase the amount of storage capacity that is available to host applications by pooling the capacity from multiple disk systems within the SAN. In addition, it combines various IBM technologies that include thin provisioning, automated tiering, storage virtualization, IBM Real-time Compression, clustering, replication, multiprotocol support, and an updated graphical user interface (GUI). Together, these technologies enable a IBM Spectrum Virtualize and SAN Volume Controller to deliver exceptional storage efficiency.
IBM Spectrum Virtualize and SAN Volume Controller includes a dynamic data migration function to move data from one storage system to another while still maintaining access to that data. The Volume Mirroring function stores two copies of a volume on different storage systems. This configuration helps improve application availability during a failure or disruptive maintenance to an array or disk system. The controller’s stretched cluster configuration automatically uses whichever copy of the data remains available.
With the stretched cluster, administrators can apply a single set of advanced network-based replication services that operate in a consistent manner. This set is applied regardless of the type of storage that is being used. The Metro Mirror and Global Mirror functions operate between systems at different locations. They help create copies of data for use during a catastrophic event at a data center. For even greater flexibility, Metro Mirror and Global Mirror also support replication between SAN Volume Controller systems and IBM Storwize V7000 Unified systems.
The IBM FlashCopy function stretched cluster quickly creates a copy of active data that can be used for backup purposes or for parallel processing activities. This capability enables disk IBM Spectrum Virtualize and SAN Volume Controller Enhanced Stretched Cluster with VMware backup copies to be used to recover almost instantly from corrupted data, which significantly speeds application recovery.
What is new with IBM Spectrum Virtualize Software V7.6
IBM Spectrum Virtualize software delivers a powerful solution for storage efficiency in environments where IBM is the primary storage. When used in combination with SVC, Storwize V7000 (Unified), Storwize V5000, Storwize V3700, and Storwize V3500, IBM Spectrum Virtualize provides the benefits of storage virtualization and advanced storage capabilities, including HyperSwap, VVOL support, FlashCopy, replication, Real-time Compression, Easy Tier, and Encryption.
Spectrum Virtualize 7.6 includes the following new functionalities:
•Encryption
For improved data security and confidentiality, encryption is now available for both internal capacities of SVC, Storwize V7000 and FlashSystem V9000 and externally virtualized capacities of Storwize V7000, SVC, and FlashSystem V9000. Encryption can be applied to virtualized storage arrays, even if the virtualized array does not have encryption capabilities. Encrypted volumes are transparent to applications, easing implementation and operation.
For supported externally virtualized storage configurations, data encryption can be enabled by the Storwize V7000 or SVC controllers. The data is encrypted and sent over the storage network to the virtualized storage devices. Encryption by Spectrum Virtualize is transparent to hosts and applications, and the data is encrypted and protected across the network between the Storwize and SVC controllers and the storage devices.
•Distributed RAID
For higher availability, Distributed RAID is now available for internal capacities, offering up to 10 times faster RAID array rebuild times. As drives grow in capacity, rebuild times increase, increasing the possibility of a second drive failure and resulting in a loss of data. Distributed RAID addresses this issue by distributing the data across a larger number of physical drives, reducing the load on each individual drive from rebuild activity. Distributed RAID can also deliver increased performance because data can be read from/written to more drives for each I/O. Distributed RAID volumes are transparent to applications for simplified implementation and operation.
•IP quorum base support
Current Stretched Cluster (SVC) and HyperSwap (SVC, Storwize V7000, and Storwize V5000) implementations require FC storage on a third site to cope with tie-break situations during an inter-site link failing. IP quorum base support enables the use of a lower-cost IP network-attached host as quorum disk for simplified implementation and operation.
•HyperSwap GUI support
Current local HyperSwap is functionally complete, but requires complex configuration through the existing command-line interface. This new release of HyperSwap offers a new command-line interface that simplifies the set-up process to a handful of commands, and also adds the ability to configure and manage local HyperSwap through the GUI.
•VMWare VVOL integration
Migration of space efficient volumes between storage containers maintains the space efficiency of volumes and delivers resiliency for VMs in case the volumes start running out of space. Cloning a virtual machine will achieve a full independent set of virtual volumes.
•Increase maximum quantity of iSCSI hosts
For more scalable iSCSI host support, version 7.6 increases the maximum number of iSCSI host sessions from 256 to 1024 per node, offering more fanout for each SVC I/O group.
•Integrated Comprestimator
Real-time Compression is a key differentiator of the Storwize Family, and the Comprestimator is its key sizing tool to estimate how much capacity savings the customer can expect. The Comprestimator can analyze the patterns of the actual customer data, and estimate the compressibility of data per volume. Currently, storage administrators have to get server administrator collaboration and authorization to install Comprestimator on servers, potentially on hundreds of servers. The integration of Comprestimator within Spectrum Virtualize code eases the process of estimating capacity savings by having this sizing tool run natively in SVC, Storwize V7000, and FlashSystem V9000, avoiding the need to install Comprestimator on hundreds of servers and enabling estimates of Real-time Compression effectiveness from a central console.
IBM Spectrum Virtualize and VersaStack solution by Cisco
VersaStack, an IBM and Cisco integrated infrastructure solution, combines computing, networking, and storage into a single integrated system. It combines the Cisco Unified Computing System (Cisco UCS) Integrated Infrastructure with IBM Spectrum Virtualize, which includes IBM FlashSystem V9000 and IBM Storwize storage offerings. This combination allows quick deployment and rapid time to value for the implementation of modern infrastructures.
For more information about VersaStack solutions, visit this webpage:
2.3.2 IBM Spectrum Accelerate
A key member of the IBM Spectrum Storage portfolio, IBM Spectrum Accelerate is a highly flexible, software-defined storage solution that enables rapid deployment of block data storage services for new and traditional workloads, both on and off premises. Spectrum Accelerate allows you to run hotspot-free, grid-scale software that deploys on-premises on x86 commodity servers and on the optimized XIV Storage System, and off-premises as a public cloud service on SoftLayer. It offers proven grid- scale technology, mature features, and ease of use, and is already deployed on over 100,000 servers worldwide.
IBM Spectrum Accelerate delivers a design that was made for the cloud, ensuring hotspot and tuning-free service levels across mixed workloads to help meet the unpredictability of today's cloud world. It delivers a single management experience across software defined infrastructure with the Hyper-Scale Manager GUI. It also helps cut costs through reduced administration effort and training, reduced procurement, standardization of data center storage hardware operations and services, and licensing flexibility that enables cost-efficient cloud building.
Figure 2-10 shows how straightforward scaling is by building a storage grid with Spectrum Accelerate.
Figure 2-10 Spectrum Accelerate iSCSI storage grid
Key capabilities
Spectrum Accelerate gives organizations these capabilities:
•Enterprise cloud storage in minutes, using commodity hardware
•Hotspot-free performance and QoS without any manual or background tuning needed
•Advanced remote replication, role-based security, and multi-tenancy
•Deploy on-premises or on the cloud
•Management of dozens of petabytes
•Best in class VMware and OpenStack integrations
Spectrum Accelerate runs as a virtual machine on vSphere ESXi hypervisor. This configuration enables you to build a server-based SAN from commodity hardware that includes x86 servers, Ethernet switches, solid-state drives (SSDs), and direct-attached, high-density disks. Spectrum Accelerate essentially acts as an operating system for your self-built SAN storage, grouping virtual nodes and spreading the data across the entire grid.
IBM Spectrum Accelerate release 11.5.3 manages up to 15 nodes in a grid. It provides a single point of management of up to 144 grids of 15 nodes connected through Hyper-Scale Manager, for a total of 2160 nodes. It supports 6 TB HDDs for 50% more capacity in the same footprint than 4 TB drives and support for 600 GB 10K rpm drives.
IBM Hyper-Scale mobility over iSCSI networks provides the following capabilities:
•Delivers online, transparent, nondisruptive data mobility between IBM Spectrum Accelerate storage clusters
•Helps maximize utilization and effective performance balancing across multiple clusters and facilitate aggressive thin-provisioning
Spectrum Accelerate allows you to deploy storage services flexibly across different delivery models, including customer-choice hardware, existing data center infrastructure, and IBM storage systems.
Benefits
Spectrum Accelerate provides the following benefits:
•Cost reduction by delivering hotspot-free storage to different deployment models, on and off premises, paying less for the same capacity, acquiring less hardware, reducing maintenance outlay, and minimizing administrative effort
•Increased operational agility through easy cloud building, faster provisioning, small capacity increments, and licensing flexibility that enables you to easily repurpose servers
•Rapid response through enterprise-class storage availability, data protection, and security for new and traditional workload needs in the data center and other sites while flexibly balancing capital and operational expenses
•Simplified management with seamless management across 144 virtual systems, the XIV Storage System, or the infrastructure of your off- premises services provider
•An administrator can deploy the IBM Spectrum Accelerate storage solution on existing virtualized servers in minutes, enabling a tuning-free storage environment to be built quickly with outstanding ease of management and agility
Table 2-3 describes Spectrum Accelerate features with their associated benefits.
Table 2-3 Spectrum Accelerate features and benefits
Feature
Benefit
Performance
•Ensures even data distribution through massive parallelism and automatic load balancing, including capacity add
•Distributed cache
Reliability and Availability
•Grid redundancy maintains two copies of each 1-MB data partition with each copy on a different virtual machine (VM), proactive diagnostics, fast and automatic rebuilds, event externalization
•Advanced monitoring; network monitoring; disk performance tracking/reporting; data center monitoring; shared monitoring for some components; data and graphical reports on I/O, usage, and trends
•Self-healing, which minimizes the rebuild process by rebuilding only actual data
•Automated load balancing across components with minimized risk of disk failure due to rapid return to redundancy
Management
•Intuitive GUI: Scales to over 40 PB with IBM Hyper-Scale Manager; extensive CLI; RESTful API; mobile app support with push notifications; multi-tenancy with quality of service by domain, pool, or host
Cloud automation and self-service
•OpenStack; VMware vRealize Orchestrator through IBM Spectrum Control Base
Snapshot management
•Space efficient snapshots: Writable, snapshot of snapshot, restore from snapshot, snapshots for consistency groups, mirroring
Thin provisioning and space reclamation
•Thin provisioning per pool, thick-to-thin migration; VMware, Microsoft, Symantec space reclamation support
Mirroring
•Synchronous/asynchronous; volumes and consistency groups, recovery point objective (RPO) of seconds; online/offline initialization; failover/failback; mirroring across platforms, including with XIV Storage System
Security
•Role-based access management, multi-tenancy, iSCSI Challenge Handshake Authentication Protocol (CHAP), and auditing; integrates with Lightweight Directory Access Protocol (LDAP) and Microsoft Active Directory servers
For more information about IBM Spectrum Accelerate, see the following resources:
•IBM Spectrum Accelerate
•IBM Spectrum Accelerate Deployment, Usage, and Maintenance, SG24-8267
•Deploying IBM Spectrum Accelerate on Cloud, REDP-5261
2.3.3 IBM Spectrum Scale
IBM Spectrum Scale is a proven, scalable, high-performance file management solution that is based on IBM GPFS. IBM Spectrum Scale provides world-class storage management with extreme scalability, flash accelerated performance, and automatic policy-based storage tiering from flash to disk, then to tape. IBM Spectrum Scale reduces storage costs up to 90% while improving security and management efficiency in cloud, big data, and analytics environments.
First introduced in 1998, this mature technology enables a maximum volume size of 8 YB, a maximum file size of 8 EB, and up to 18.4 quintillion (two to the 64th power) files per file system. IBM Spectrum Scale provides simplified data management and integrated information lifecycle tools as software-defined storage for cloud, big data, and analytics. It introduces enhanced security, flash accelerated performance, and improved usability. It also provides capacity quotas, access control lists (ACLs), and a powerful snapshot function.
Key capabilities
IBM Spectrum Scale adds elasticity with the following capabilities:
•Global namespace with high performance access scales from departmental to global
•Automated tiering, data lifecycle management from flash (6x acceleration) to tape (10x savings)
•Enterprise ready: Data security (encryption), availability, reliability, large scale proven
•Open: POSIX compliant, integrated with OpenStack components and Hadoop
Benefits
IBM Spectrum Scale provides these benefits:
•Improves performance by removing data-related bottlenecks
•Automated tiering, data lifecycle management from flash (acceleration) to tape (savings)
•Lowers cost by eliminating duplicate data
•Enables sharing of data across multiple applications
•Reduces cost per performance by placing data on most applicable storage (flash to tape)
IBM Spectrum Scale is part of the IBM market-leading software defined storage family and has the following characteristics:
•As a software-only solution: Runs on virtually any hardware platform and supports almost any block storage device. IBM Spectrum Scale runs on Linux (including Linux on IBM z Systems), AIX, and systems based on Windows.
•As an integrated IBM Elastic Storage Server solution: A bundled hardware, software, and services offering that includes installation and ease of management with a graphical user interface. Elastic Storage Server provides unsurpassed end-to-end data availability, reliability, and integrity with unique technologies that include IBM Spectrum Scale RAID.
•As a cloud service: IBM Spectrum Scale delivered as a service provides high performance, scalable storage, and integrated data governance for managing large amounts of data and files in the IBM SoftLayer cloud.
IBM Spectrum Scale features enhanced security with native encryption and secure erase. It can increase performance by using server-side flash cache to increase I/O performance up to six times. IBM Spectrum Scale provides improved usability through data replication capabilities, data migration capabilities, Active File Management (AFM), File Placement Optimizer (FPO), and IBM Spectrum Scale Native RAID.
Figure 2-11 shows an example of the Spectrum Scale architecture.
Figure 2-11 Spectrum Scale architecture
IBM Spectrum Scale is based around the following concepts:
Storage pools
A storage pool is a collection of disks or arrays with similar attributes. It is an organizational structure that allows the combination of multiple storage locations that are of identical characteristics. There are three different types of storage pools:
System Pool The system pool contains file system metadata and can be used to store data. One system pool is needed per file system.
Data Pool A data pool is used to store file data. A data pool is optional.
External Pool An external pool is used to attach auxiliary storage, such as tape, to IBM Spectrum Scale. An external pool is optional.
File sets
IBM Spectrum Scale creates a single name space. Tools can provide a fine grained management of the directory structure. A file set acts like a partition of a file system, a subdirectory tree. File sets can be used for operations such as quotas or used in management policies. It is a directory tree that behaves like a “file system” within a file system and has the following characteristics:
•Part of the global namespace
•Can be linked and unlinked (like mount/unmount)
•Policy scan can be restricted to only scan file sets. This feature can be helpful when the file system has billions of files
•A file set can be assigned to a storage pool
There are two kinds of file sets:
Dependent file sets A dependent file set allows for a finer granularity of administration. It shares the inode space with another file set.
Independent file set An independent file set has a distinct inode space. An independent file set allows file set level snapshots, independent file scans, and enables advanced features like Active File Management.
Policy engine
The policy engine uses an SQL style syntax to query or operate on files based on file attributes. Policies can be used to migrate all data that has not been accessed in 6 months to less expensive storage or just used to query the contents of a file system. Management policies support advanced query capabilities, though what makes the policy engine most useful is the performance. The policy engine is capable of scanning billions of objects as shown in Table 2-4.
Table 2-4 Speed comparison for GPFS policy engine
Search through 1000000000 (1 billion) files
find
~ 47 hours
GPFS policy engine
~ 5 hours
Table 2-4 shows the power of the GPFS policy engine. Although an average find across 1 billion files took approximately 47 hours, the GPFS policy engine could satisfy the request within five hours. The GPFS policy engine can also create a candidate list for backup applications to use to achieve a massive reduction in candidate identification time.
IBM Spectrum Scale has next generation availability with features that include rolling software and hardware upgrades. You can add and remove servers to adapt the performance and capacity of the system to changing needs. Storage can be added or replaced online, and you can control how data is balanced after storage is assessed.
Mirroring, replication, and migration capabilities
In IBM Spectrum Scale, you can replicate a single file, a set of files, or the entire file system. You can also change the replication status of a file at any time by using a policy or command. Using these capabilities, you can achieve a replication factor of two, which equals mirroring, or a replication factor of three.
A replication factor of two in Spectrum Scale means that each block of a replicated file is in at least two failure groups. A failure group is defined by the administrator and contains one or more disks. Each storage pool in a file system contains one or more failure groups. Failure groups are defined by the administrator and can be changed at any time. So when a file system is fully replicated, any single failure group can fail and the data remains online.
For migration, IBM Spectrum Scale provides the capability to add storage to the file system, migrate the existing data to the new storage, and remove the old storage from the file system. All of this can be done online without disruption to your business.
Active File Management
AFM enables the sharing of data across unreliable or high latency networks. With AFM, you can create associations between IBM Spectrum Scale clusters and define the location and flow of file data. AFM allows you to implement a single name space view across clusters, between buildings, and around the world.
AFM operates at the file set level. This means that you can create hundreds of AFM relationships in each file system. AFM is a caching technology though inode, and file data in a cache file set is the same as an inode and file data in any IBM Spectrum Scale file system. It is a “real” file that is stored on disk. The job of the cache is to keep the data in the file consistent with the data on the other side of the relationship.
AFM can be implemented in these modes:
•Read-Only (ro): Create a flash cache for heavily read data
•Local-Update (lu): Provide a development copy of data
•Single-Writer (sw): Collect data at a remote location
•Independent Writer (iw): Create a global interactive name space
•Asynchronous DR: Create asynchronous copies of file data
File Placement Optimizer
FPO is optimized for Hadoop in a shared-nothing deployment. FPO can use the local storage of the individual cluster members to form a high performance, resilient storage environment. Using policies, you can control the placement of data and replicas within the cluster to optimize for your access patterns.
IBM Spectrum Scale Native RAID
IBM Spectrum Scale Native RAID provides next generation performance and data security. IBM Spectrum Scale native RAID just a bunch of disks (JBOD) are directly attached to the systems running IBM Spectrum Scale software. This technology uses declustered RAID to minimize performance degradation during RAID rebuilds and provides extreme data integrity by using end-to-end checksums and version numbers to detect, locate, and correct silent disk corruption. An advanced disk hospital function automatically addresses storage errors and slow performing drives so your workload is not affected.
IBM Spectrum Scale native RAID is available with the IBM Power8 architecture in the IBM Elastic Storage Server.
Licensing
IBM Spectrum Scale V4.2 offers different editions so you only pay for the functions that you need:
•Express Edition contains the base IBM Spectrum Scale data access function
•Standard Edition includes the base function plus information lifecycle management (ILM), AFM, and either integrated multiprotocol support, which includes Ganesha NFS or traditional clustered NFS
•Advanced Edition includes encryption of data at rest, secure erase, asynchronous multisite disaster recovery, and all the features of Standard Edition
For each of these editions, you can choose an IBM Spectrum Scale license for Server, Client, and FPO.
For more information, see the following resources:
•IBM Spectrum Scale (IBM developerWorks)
•IBM Spectrum Scale
•IBM Spectrum Scale (IBM Knowledge Center)
•IBM Spectrum Scale Wiki
•IBM Elastic Storage Server
2.3.4 IBM Spectrum Archive
IBM Spectrum Archive, a member of the IBM Spectrum Storage family, enables direct, intuitive, and graphical access to data stored in IBM tape drives and libraries by incorporating the Linear Tape File System (LTFS) format standard for reading, writing, and exchanging descriptive metadata on formatted tape cartridges. Spectrum Archive eliminates the need for additional tape management and software to access data. Spectrum Archive offers three software solutions for managing your digital files with the LTFS format: Single Drive Edition (SDE), Library Edition (LE), and Enterprise Edition (EE). With Spectrum Archive Enterprise Edition and Spectrum Scale, tape can now add savings as a low-cost storage tape tier.
Network attached unstructured data storage with native tape support using LTFS delivers the best mix of performance and lowest cost storage.
Key capabilities
Spectrum Archive options can support small, medium, and enterprise businesses with these advantages:
•Seamless virtualization of storage tiers
•Policy-based placement of data
•Single universal namespace for all file data
•Security and protection of assets
•Open, non-proprietary, cross platform interchange
•Integrated functionality with IBM Spectrum Scale
Benefits
IBM Spectrum Archive enables direct, intuitive, and graphical access to data stored in IBM tape drives and libraries. It does so by incorporating the LTFS format standard for reading, writing, and exchanging descriptive metadata on formatted tape cartridges. Spectrum Archive eliminates the need for additional tape management and software to access data.
Spectrum Archive takes advantage of the low cost of tape storage while making it easy to use. Spectrum Archive provides these benefits, among others:
•Access and manage all data in stand-alone tape environments as easily as though it were on disk
•Enable easy-as-disk access to single or multiple cartridges in a tape library
•Improve efficiency and reduce costs for long-term, tiered storage
•Optimize data placement for cost and performance
•Enable data file sharing without proprietary software
•Scalable and low cost
IBM Linear Tape File System
In 1997, IBM invented a new file system for files stored on tape cartridges, which was initially called IBM Long Term File System. This invention allowed it to use a tape drive as if it were a hard disk drive or a removable flash drive to store and access data. Because of the nature of the tape technology, the file access is still sequential.
In 2010, this technology was adopted by the Linear Tape-Open (LTO) consortium that defines the standard for all LTO technology specifications. For more information, see 8.1.1, “LTO technology” on page 178). The LTFS format specification and file system implementation were released on April 12, 2010 with the support of IBM, HP, Quantum, and the LTO Consortium. From this time, the file system was called LTFS.
LTFS is the first file system that works with LTO generation 7, 6, and 5 tape technology (or IBM TS1150 and TS1140 tape drives). It sets a new standard for ease of use and portability for open systems tape storage. With this application, accessing data that is stored on an IBM tape cartridge is as easy and intuitive as using a USB flash drive. Tapes are self-describing, and you can quickly recall any file from a tape without having to read the whole tape.
Further, any LTFS-capable system can read a tape that is created by any other LTFS-capable system regardless of the operating system and platform. Any LTFS-capable system can identify and retrieve the files that are stored on it. LTFS-capable systems have the following characteristics:
•Files and directories are displayed as a directory tree listing.
•More intuitive searches of cartridge and library content are now possible due to the addition of file tagging.
•Files can be moved to and from LTFS tape by using the familiar drag-and-drop metaphor common to many operating systems.
•Many applications that were written to use files on disk can now use files on tape without any modification.
•All standard File Open, Write, Read, Append, Delete, and Close functions are supported.
Spectrum Archive editions
As shown in Figure 2-12, Spectrum Archive is available in different editions that support small, medium, and enterprise businesses.
Figure 2-12 IBM Spectrum Archive Singe Drive Edition, Library Edition, and Enterprise Edition implementations
IBM Spectrum Archive Enterprise Edition
IBM Spectrum Archive EE gives organizations an easy way to use cost-effective IBM tape drives and libraries within a tiered storage infrastructure. By using tape libraries instead of disks for Tier 2 and Tier 3 data storage (data that is stored for long-term retention), organizations can improve efficiency and reduce costs. In addition, Spectrum Archive EE seamlessly integrates with the scalability, manageability, and performance of IBM Spectrum Scale, which is an IBM enterprise file management platform that enables organizations to move from simply adding storage to optimizing data management.
Here are some of the Spectrum Archive Enterprise Edition highlights:
•Simplify tape storage with the IBM LTFS format, combined with the scalability, manageability, and performance of IBM Spectrum Scale
•Help reduce IT expenses by replacing tiered disk storage (Tier 2 and Tier 3) with IBM tape libraries
•Expand archive capacity by simply adding and provisioning media without affecting the availability of data already in the pool
•Add extensive capacity to IBM Spectrum Scale installations with lower media, floor space, and power costs
•Support for attaching up to two tape libraries to a single Spectrum Scale cluster
Spectrum Archive EE for the IBM TS4500, IBM TS3500, and IBM TS3310 tape libraries provides seamless integration of Spectrum Archive with Spectrum Scale by creating an LTFS tape tier. You can run any application that is designed for disk files on tape by using Spectrum Archive EE. Spectrum Archive EE can play a major role in reducing the cost of storage for data that does not need the access performance of primary disk. This configuration improves efficiency and reduces costs for long-term, tiered storage.
With Spectrum Archive EE, you can enable the use of LTFS for the policy management of tape as a storage tier in a Spectrum Scale environment and use 7, 6, and 5 tape drives as a critical tier in the storage environment. Spectrum Archive EE supports IBM LTO Ultrium 6 and 5 tape drives, IBM System Storage TS1150, and TS1140 tape drives that are installed in TS4500, and TS3500 tape libraries, or LTO Ultrium 7, 6, and 5 tape drives that are installed in the TS3310 tape libraries.
The use of Spectrum Archive EE to replace disks with tape in Tier 2 and Tier 3 storage can improve data access over other storage solutions because it improves efficiency and streamlines management for files on tape. Spectrum Archive EE simplifies the use of tape by making it transparent to the user and manageable by the administrator under a single infrastructure. Figure 2-13 shows the integration of Spectrum Archive EE archive solution with Spectrum Scale.
Figure 2-13 Integration of IBM Spectrum Scale and Spectrum Archive Enterprise Edition
The seamless integration offers transparent file access in a continuous name space. It provides file level write and read caching with disk staging area, policy-based movement from disk to tape, creation of multiple data copies on different tapes, load balancing, and high availability in multi-node clusters. It also offers data exchange on LTFS tape by using import and export functions, fast import of file name space from LTFS tapes without reading data, built-in tape reclamation and reconciliation, and simple administration and management.
For more information, see IBM Spectrum Archive Enterprise Edition at the following link:
IBM Spectrum Archive Library Edition
IBM Spectrum Archive LE extends the file management capability of the IBM Spectrum Archive SDE. Spectrum Archive LE is introduced with Version 2.0 of LTFS. It enables easy-as-disk access to single or multiple cartridges in a tape library.
LTFS is the first file system that works with IBM System Storage tape technology to optimize ease of use and portability for open-systems tape storage. It manages the automation and provides operating system-level access to the contents of the library. Spectrum Archive LE is based on the LTFS format specification, enabling tape library cartridges to be interchangeable with cartridges that are written with the open source SDE version of Spectrum Archive. IBM Spectrum Archive LE supports most IBM tape libraries:
•TS2900 tape autoloader
•TS3100 tape library
•TS3200 tape library
•TS3310 tape library
•TS3500 tape library
•TS4500 tape library
IBM TS1150 and IBM TS1140 tape drives are supported on IBM TS4500 and IBM TS3500 tape libraries only.
Spectrum Archive LE enables the reading, writing, searching, and indexing of user data on tape and access to user metadata. Metadata is the descriptive information about user data that is stored on a cartridge. Metadata enables searching and accessing of files through the GUI of the operating system. Spectrum Archive LE supports both Linux and Windows.
Spectrum Archive LE provides the following product features:
•Direct access and management of data on tape libraries with LTO Ultrium 7, 6, and 5 tape drives, and TS1150 and TS1140 tape drives
•Tagging of files with any text, allowing more intuitive searches of cartridge and library content
•Use of the partitioning of the media in LTO-5 tape format standard
•One-to-one mapping of tape cartridges in tape libraries to file folders
•Capability to create a single file system mount point for a logical library that is managed by a single instance of LTFS and runs on a single computer system
•Capability to cache tape indexes, and to search, query, and display tape content within an IBM tape library without having to mount tape cartridges
The IBM Spectrum Archive LE offers the same basic capabilities as the IBM Spectrum Archive SDE with additional support of tape libraries. Each LTFS tape cartridge in the library appears as an individual folder within the file space. The user or application can go to these folders to access the files that are stored on each tape. The Spectrum Archive LE software automatically controls the tape library robotics to load and unload the necessary LTFS Volumes to provide access to the stored files.
For more information, see IBM Spectrum Archive Library Edition at the following link:
IBM Spectrum Archive Single Drive Edition
The IBM Spectrum Archive SDE implements the LTFS Format and allows tapes to be formatted as LTFS Volumes. These LTFS Volumes can then be mounted by using LTFS to allow users and applications direct access to files and directories that are stored on the tape. No integration with tape libraries exists in this edition. You can access and manage all data in stand-alone tape environments as simply as though it were on disk. The limitations of having access to tape as a sequential medium apply.
Use cases for IBM Spectrum Archive SDE are local data import/export; data exchange between locations; all applications that deal with large files like video surveillance, video editing, and streaming; image archiving; and many more.
For more information, see IBM Spectrum Archive Single Drive Edition at the following link:
1 Based on IBM experience using storage analysis
2 IBM internal tests. Includes ingest, deduplication, replication, and maintenance.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.145.111.41