5.4. Conclusions

This chapter introduced and evaluated three different non-partitioning techniques that guarantee a continuous display of objects with a heterogeneous disk subsystem. These techniques are Disk Grouping, Staggered Grouping, and Disk Merging. All three techniques conceptualize the physical heterogeneity of a storage system as a set of logical disks with homogeneous characteristics. As a result of this abstraction, conventional scheduling and data placement algorithms can be applied.

Disk Grouping aggregates sets of physical disks into groups to form logical disks. Each set must consist of the same mix of physical disk drives. Data is placed on and retrieved from the logical disks. Each data block is in effect declustered across the physical disks that form a logical disk. Partial blocks are called fragments and their size is adjusted according to the performance of the physical disk they reside on. If a block retrieval is initiated from a logical disk, all appropriate fragments are read in parallel, recombined in memory, and transmitted to the client display station.

Staggered Grouping operates similar to Disk Grouping, except that it reduces the amount of main memory that is required for the same throughput. The improvement is based on the observation that not all fragments of a logical block are needed at the same time at the client side. By staggering the retrieval of the fragments in time, memory frames can be reused.

Finally, Disk Merging separates the concept of logical disks from the physical disks altogether. Logical disks are formed from fractions of the bandwidth and storage capacity of the available physical devices. Consequently, Disk Merging is more flexible than the other two techniques because it can support an arbitrary mix of physical disks. This flexibility increases the effort necessary to find the best possible configuration for a specific application. Consequently, a configuration planner was introduced to automate and speed up the configuration process.

To evaluate our techniques we first compared the three non-partitioning schemes with one another using analytical methods. All utilize system resources well, with Staggered Grouping and Disk Merging providing a slightly lower cost per stream (by minimizing the amount of required memory). While Disk Merging provides the lowest minimum startup time, the variance in startup latency is reduced with both Disk Grouping and Staggered Grouping.

Next, we chose Disk Merging as the representative of the non-partitioning schemes to compare it with a simple partitioning technique. Similar to a real system, our simulation modeled shifts in the user access pattern as a function of time. Tests were conducted at system loads of 70%, 80%, and 90%, respectively. The 90% load revealed the fundamental problem of any partitioned system: some resources become a bottleneck while others remain idle, waiting for work. In the specific case of the partitioned CM server, one of the three sub-servers became overloaded and as a result the average startup latency for all users increased. Disk Merging was able to handle the situation more gracefully (both Disk and Staggered Grouping are expected to provide a performance similar to Disk Merging). It was sensitive to neither the frequency of access to clips nor their assignment. As a result the latency increase was much less dramatic.

One disadvantage of non-partitioning schemes as compared with partitioned systems is their increased vulnerability to disk failures. Hence, we investigated fault-tolerance and high availability techniques and their application to non-partitioned storage systems. The result was a framework of broad observations that can build the foundation for a more in-depth investigation. We provided one example, where we analyzed how the class of non-overlapping parity techniques can be applied to a non-partitioning storage approach (i.e., Disk Merging). We specified both design rules and algorithms that provide the necessary independence of logical disks among each other and across parity groups for the successful application of parity-based data redundancy.