Appendix H: A Novel A/V Storage System

In high-end broadcast and postproduction environments, real-time, high-availability storage systems offer significant workflow advantages. This appendix reviews a new storage system architecture¹ designed specifically for the needs of A/V. The design blurs the lines between network attached storage (NAS) and storage area network (SAN) systems. It addresses a number of key needs, including support for a large number of real-time editing clients and I/O channels, high data availability, hot-swap components, heterogeneous clients (Windows and MacOS), and wide-range scalability in data rates and storage capacity.

Commercially available, enterprise-class NAS and SAN systems are hard pressed to meet these functional requirements. A NAS, in which client data pass through a single NAS server, does not scale easily and suffers from intrinsic data rate and latency constraints. Fibre Channel-based SAN topologies are also problematic and have limited client-attach flexibility and scalability. Finally, commercially available clustered file systems are not designed to support data requests with real-time QoS requirements.

H.0 ARCHITECTURAL OVERVIEW

An entirely new networked file system and storage infrastructure was developed to overcome these deficiencies. The file system is unique in several ways. Every editing client and I/O channel requires consistent, glitch-free, A/V data streams delivered to/from storage. The file system arbitrates data transfer deadlines between clients and storage, ensuring clients complete transfers to storage within a prescribed time window. Without such functionality, a client’s internal buffers would either overflow or starve. Further, the file system enables storage configurations to be modified dynamically, allowing system administrators to reassign, add, or remove storage from user groups without interrupting client operations.

To deliver the required performance and scalability, the file system is implemented as a clustered file system with intelligence distributed among the file system metadata managers (the system directors), the A/V clients, and intelligent storage blade servers. A/V clients access low data rate file metadata from the directors and access bulk file data via an Ethernet iSCSI-like connection to the storage blades. The director is mirrored offering a NSPOF design.

The storage infrastructure uses one or more blade chassis, each containing up to 16 intelligent storage blades (see ). Each blade contains two disks (250 or 500GB capacity each), a CPU, and dual gigabit Ethernet ports to connect to the backplane. Further, two independent Ethernet switch blades are integral components to each chassis, enabling clients to directly attach to the storage as well as interconnect multiple blade chassis. For the 24/7 world of broadcast, switch blades and power supplies are redundant and support hot swap. Figure H.1 illustrates a typical configuration of a small number of blade chassis and several real-time clients.

FIGURE H.1 Sample unity ISIS configuration.

User data are protected using a redundant array of independent nodes (RAIN) rather then RAID methods. When a client writes data to a storage blade, the blade is responsible for making a redundant copy on other blades. If any blade fails, the system director notifies all blades of the failure. A new copy of any lost blade data is made immediately. The data replication process occurs in parallel across all the blades, resulting in exceedingly short rebuild times and improving overall reliability.

H.1 SUMMARY

The methods discussed in this appendix illustrate a new approach to A/V realtime storage. The combination of Ethernet-based storage access, an intelligent file system director, and distributed storage creates a system designed specifically for A/V use. The system uses NSPOF design principles and clever use of blades to create a highly reliable and scalable real-time storage system.