The computer storage pendulum has swung from people asking, “What would I do with all that storage?” about a decade ago to people now using personal storage media in amounts once only possible for the largest organizations in the world. As you are probably well aware, the average smartphone today has as much storage in it as a PC did in 1997. Data has proliferated in type, size, and audience over the last few years. This data proliferation over the last decade has jokingly been referred to as “datageddon.” Data storage and data management have become a real challenge for organizations worldwide, as they struggle to keep up with webcam, voice, e-mail, transaction, and other data feeds that they are required to retain as part of their business’s intellectual property or work product.

This chapter discusses the various types of storage available in Microsoft Azure. Azure Storage refers to long-term disk storage, not cache or RAM, which is closer to the processor and resides within the hosts running the virtual machines. Another key differentiator is that not all storage is created equal. There are capabilities to be considered, such as drive-access speed and redundancy. Because Azure is created to meet the needs of the enterprise, the type of storage used is enterprise grade as well. Therefore, at a minimum, these are physically 15K spindles and solid-state drives (SSDs) deployed in locally and geographically redundant setups.

Azure Storage is elastic, meaning it can grow and shrink with you, and you’re only billed for what you are currently using. Even if a storage account or disk is provisioned at a substantially larger size than what is actually used, only the blocks used are counted for billing purposes.

Azure Storage is available from anywhere there is an Internet connection, like the rest of Azure. Many services already use Azure Storage, such as the Microsoft Windows Server 2012 native backup utility and the Microsoft System Center Data Protection Manager 2012. Azure Storage also uses a system that automatically load-balances your storage based on traffic, just like the most expensive storage area networks (SANs) on the market today. All of these Azure Storage capabilities can be provisioned quickly, and you can have customers up and running using them in about five minutes.

Azure Storage Services: Terms and Concepts

Microsoft’s Azure Storage service is available in all Azure datacenters. In general, several types of Azure Storage options are available; they are defined after you create an Azure Storage account, with the exception of databases and StorSimple. The different types of storage are depicted in Figure 7-1.

• Azure Storage accounts: An Azure Storage account must be used to create blobs, tables, queues, or file storage services.
• BLOB (binary large object) storage: Blobs are generally used to store files accessed via a browser using the HTTP/HTTPS GET and POST methods.
• SQL databases: A SQL database is a type of storage used for structured data. But instead of managing a SQL server, Azure SQL Database is an alternative for consuming the database without having to manage the SQL server that is providing the service. This is different from consuming SQL Server as an IaaS, because in the latter scenario, you still need to manage and maintain the SQL Servers in IaaS.
• Queues: As the name implies, a queue provides messaging and communication between different applications. Queue storage provides such a mechanism without having to maintain a separate messaging bus.
• Tables: Azure Table Storage is a NoSQL key-value store that is useful for applications that must store large amounts of non-relational data without schemas. This is different from Azure databases in that table storage does not provide any way to represent relationships between data. It is a low-cost way to provide a fault-tolerant store for structured data that may not require the complexities of a relational database. For more information about the differences between Azure Table Storage and Azure SQL Database, see https://msdn.microsoft.com/library/azure/jj553018.aspx.
• File storage: File storage provides the ability to access files via standard SMB 2.1.
• Hybrid storage (StorSimple): Microsoft acquired StorSimple, which is a storage solution that deploys on-premises 2U devices that use cloud storage for overflow, backup, and archiving capabilities. The StorSimple device is an ISCSI unit with SSDs and traditional drives for fast, low-latency access to frequently used files.

Azure Storage Account

Before going into the details of the different types of storage, it is important to note that with the exception of StorSimple and Azure SQL Database, all storage types are created from an Azure Storage account. The Azure Storage account in turn determines certain characteristics for the storage, such as whether the storage is locally redundant or georedundant, and whether the storage is based on standard hard drives or SSDs.

Premium Storage

Azure Storage is generally provisioned on enterprise-class spinning hard drives. For higher performance, Azure offers Premium Storage. Premium Storage is provisioned on SSDs. As a result, Premium Storage delivers high-performance, low-latency disk support for I/O intensive workloads.

Locally Redundant Storage

Locally redundant storage (LRS), which is the base-level redundancy for storage, ensures that hardware is provisioned on three separate replicas in the same datacenter. This guarantees that no localized hardware failure results from an interruption in data services.

Georedundant Storage

Georedundant storage (GRS) takes LRS storage in one datacenter and replicates it to LRS in another datacenter that is more than 500 miles away. This ensures true georedundancy. Thus if an entire datacenter becomes unavailable, there is no interruption to data services because the second datacenter can continue providing services. Technically, GRS has six replicas of the data (LRS × 2).

Queue Storage

Queues are used primarily for messaging between services and applications. Azure Queue is part of the Azure Storage family of services and uses REST-based protocols such as GET and PUT to transfer messages.

Azure provides two types of queue methods: Azure queues and service bus queues. Although both methods are messaging services, they have different features and capacities in terms of message size, maximum queue size, and message time-to-live (TTL). Application and service developers primarily use Azure queues, which are beyond the scope of this book. For detailed information about Azure queues and service bus queues, and the appropriate use-case scenarios for each, visit https://azure.microsoft.com/en-us/documentation/articles/service-bus-azure-and-service-bus-queues-compared-contrasted. You can also find a detailed discussion of Azure Queue storage at http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-queues.

File Storage

Azure File storage offers shared storage for applications using the standard Server Message Block (SMB) 2.1 protocol. Azure VMs and cloud services can share file data across application components via mounted shares, and on-premises applications can access file data in a share via the Azure File storage API. Azure VMs can connect to Azure Storage by simply mounting the file storage as a shared drive via the SMB protocol. Multiple clients can access the file storage via SMB simultaneously.

You can also achieve high-performance, low-latency file storage by building the file storage on the new Azure Premium Storage, which is based on SSD drives.

Blob Storage

Azure Blob storage is designed for storing unstructured data, such as data from backups, rich content such as images and streaming video or audio, text, or binary data. It is designed to be accessed over the HTTP or HTTPS protocol, which makes it ideal for anywhere access over the Internet. Because Azure Blob storage uses the HTTP/HTTPS standards, many third-party solutions use this storage option to offload the data that they generate.

The following are the core concepts of Azure Blob storage:

• There are two types of blobs:
  • Block blob: Comprises data blocks that are identified by their unique block ID. Blocks can be of different sizes, with a maximum of 4 MB per block. There is a maximum size of 200 GB for block blobs. As such, block blobs are ideal for large, efficient uploads; hence their use in backups. Files of up to 64 MB can be written in single-write operations.
  • Page blob: As the name implies, essentially a collection of 512-byte page files. A page blob can grow to a maximum size of 1 TB.
• You may define a blob as a block or page blob at the time of creation, but once it has been created, the blob type cannot be changed.
• All blobs reflect committed changes immediately.
• Blobs can be duplicated in a snapshot.
• To limit the risk of a blob being unintentionally overwritten, you can lease it for exclusive write access. When you do so, a lease ID is generated. Henceforth, any write requests that do not include the current or correct lease ID are not allowed to modify the blob’s existing contents.
• Management of Azure Blob storage is done through an Azure Storage account, covered in the next section of this chapter.
• Azure Blob storage must reside in containers, which are groupings of Azure blobs. Containers can have unlimited blobs; likewise, Azure Storage accounts can contain an unlimited number of containers.
• Blobs are addressable using the following URL format: http://<storageaccount>.blob.core.windows.net/<container>/<blob>.

Figure 7-5 illustrates the concepts and relationships of the different Azure Blob components.

In addition to accessing Azure Blob storage through the browser, Microsoft also makes available a free Azure Storage Explorer through CodePlex. Azure Storage Explorer is a graphical user interface (GUI) application that provides user-friendly access to Azure Blob storage. You can download the Azure Storage Explorer at http://azurestorageexplorer.codeplex.com.

Azure Storage Analytics

It is sometimes helpful to analyze how storage is being used. This is especially useful for optimizing the use of storage and finding opportunities to lower the cost of consumption or for troubleshooting purposes.

Azure Storage Analytics logs the access and actions of users accessing Azure Storage: for example, users storing content in Azure. Azure Storage Analytics then analyzes the logs to provide metrics data for a storage account.

To use Azure Storage Analytics, you must enable it individually for each service. You do so from the Azure Management Portal; for details, see Chapter 13. You can also enable Storage Analytics programmatically via the REST API or the client library.

The aggregated data is stored in a well-known blob (for logging) and in well-known tables (for metrics), which may be accessed using the Azure Table service APIs or through the Azure Management Portal. Storage Analytics has a 20 TB limit on the amount of stored data that is independent of the total limit for your storage account.

Storage Analytics logs detail information about successful and failed requests. This information can be used to monitor individual requests and to diagnose issues with a storage service. Requests are logged on a best-effort basis. Log entries are created only if there is storage service activity. For example, if a storage account has activity in its Blob service but not in its Table or Queue services, only logs pertaining to the Blob service are created.

Azure Storage Analytics is not enabled by default and can only be turned on by the storage account owner. Each write operation performed by Storage Analytics is a billable activity in addition to the amount of storage used to store metrics data.

All logs are stored in block blobs in a container named $logs, which is automatically created when Storage Analytics is enabled for a storage account. The $logs container is located in the blob namespace of the storage account, such as http://<accountname>.blob.core.windows.net/$logs. This container cannot be deleted once Storage Analytics has been enabled, although its contents can be deleted.

Azure Import/Export Service

The Azure Import/Export service allows you to transfer data to and from Azure by shipping hard disk drives. In scenarios with large amounts of data, shipping it via the Import/Export service may be faster and more cost effective than electronically transferring the data over the network or Internet. The hard disk drives that are shipped to or from Microsoft datacenters are encrypted.

Azure SQL Database

Azure SQL Database is a relational database-as-a-service type of storage for structured data.

The benefit of using the Azure SQL Database service is the ability to spin up and consume a relational database quickly; it can then easily be replicated for georedundancy without all the infrastructure deployment and management normally required. When you are considering whether to use Azure SQL Database for a solution, it is important to understand the SQL Server features that are not available. However, Azure SQL Database should be able to meet the needs of most solutions that require a relational database in the back end.

StorSimple

StorSimple is a hybrid on-premises and cloud storage solution. It consists of a 2U rackmount appliance that has SSDs, hard drives, and the ability to use Azure as a storage source. Figure 7-20 is a schematic of the rear panel of a StorSimple 8100 storage appliance.

The idea behind StorSimple is the need for high-speed, low-latency access to frequently used data—hence the local storage on the appliance. Then, as data ages out, the StorSimple device moves the data from higher-speed local SSDs to the traditional hard drives that are also still on the appliance. Finally, once the device has reached 95% of capacity, as the data ages out even more, the content is moved to Azure storage. If the file is accessed after it has been moved to Azure storage, it is “promoted” back into the on-premises local StorSimple SSDs, and the aging process starts over. This entire process is transparent to the administrator and end user. It is depicted in Figure 7-21.

A StorSimple-specific storage account known as the StorSimple Manager is used to configure StorSimple. The StorSimple Manager is an extension of the Azure Management Portal. At the time of this writing, the new Preview Portal does not yet support the creation of a StorSimple Manager, so you need to use the Azure Portal. When you purchase StorSimple, this option becomes available, as shown in Figure 7-22. Also, at the time of this writing, Microsoft provides an onboarding service for StorSimple customers. As part of the service, Microsoft helps customers configure the StorSimple Manager and the appliance.

StorSimple 8000 appliances provide automatic compression and de-duplication, so there is nothing for an administrator to configure with respect to these capabilities. The appliances have 10 GB Ethernet connections and come in two versions:

• StorSimple 8100: Provides 15 TB to 40 TB of storage on the appliance, depending on the level of compression achieved. The SSD on the 8100 is 800 GB in size. The maximum available storage that an 8100 appliance can handle, including Azure storage, is 200 TB.
• StorSimple 8600: Provides 40 TB to 100 TB of storage on the appliance, depending on the level of compression achieved. The SSD on the 8600 is 2 TB in size. The maximum available storage that an 8600 appliance can handle, including Azure storage, is 500 TB.

Summary

This chapter introduced the different types of Azure Storage accounts. These accounts determine certain hardware and deployment characteristics, such as LRS versus GRS, and standard HDDs versus high-performance SSDs.

Building on an Azure Storage account, you can define block blob, page blob, or file storage. You can also use storage accounts to create table and queue storage.

You were also introduced to relational databases as a service in the form of Azure SQL Database. But other non-SQL relational databases as a service, like MySQL, are also available.

Finally, you visited StorSimple as a hybrid on-premises and cloud storage solution. StorSimple is a unique solution that takes advantage of the best of on-premises and cloud storage.

The next chapter covers how to extend your network into Azure.