Changing Allocations

Typically, capacity is allocated in discrete steps (a server costs a certain amount per hour; DynamoDB capacity units cost a certain amount per hour). You can change the number of servers allocated to your application or the number of capacity units allocated to your DynamoDB table, but only in discrete steps (the size of your server or the size of a capacity unit). Although there can be steps of various sizes available (such as different server sizes), you must allocate a whole number of units at a time.

It is your responsibility to ensure that you have enough capacity at hand. This might involve performing capacity planning exercises similar to those that you perform for traditional data center–based servers. You may very well allocate capacity based on expected demand and leave the number alone until you perform a review and determine that your capacity requirements have changed. This is typical of non-cloud-based server allocation.

However, cloud allocation changes are easier to perform than traditional capacity changes in a data center. As such, other algorithms can be used to perform your allocation. For instance, because allocation changes can be typically performed (almost) immediately, you can wait until you have consumed (almost) all your capacity before you decide to increase your capacity allocation.

Another option available is to change your allocation on a fixed schedule that matches your use patterns. For instance, increase the number of servers available during heavily used daylight hours and decrease the number of servers during lesser-used nighttime hours.

Yet another option is to change your allocation dynamically and automatically, based on current usage patterns. You might, for instance, monitor CPU usage on your servers and as soon as their usage reaches a certain threshold, you can automatically allocate additional servers.² You might build these automated mechanisms into your application or your service infrastructure yourself, or you might take advantage of cloud services such as AWS’s AutoScaling to automatically change your allocation based on current usage criteria that you specify.

Whatever mechanism you choose to determine and change capacity, it is important to note that whatever capacity you currently have allocated is all that is available to you, and you could still end up with capacity allocated (and charged) to you that is not being used. Even worse, you could find yourself resource starved because you do not have enough capacity. Even if you use an automated allocation scheme such as AutoScale to give your application additional capacity when it is needed, that does not mean that the algorithm AutoScale uses to change your capacity can notice the need fast enough before your application becomes resource starved by, for instance, a sudden resource usage increase.

Whether you change allocations manually or via an automated method, your usage is defined and constrained by your allocation. You pay for the entire allocation, whether you use it or not. If your application requires a higher allocation than is currently allocated, your application will be resource starved. Proper capacity planning, either manual or automated, is essential for managing these resources.

Reserved Capacity

You typically can change your allocated capacity as often as you want,⁴ increasing and decreasing as your needs require.

This is one of the advantages of the cloud. If you need 500 servers one hour and only 200 the next hour, you are only charged for 500 for one hour and 200 for the next hour. It’s clean and simple.

However, because of this essentially infinite flexibility in the amount of capacity you can allocate, you pay a premium price.

But what if your needs are more stable? What if you will always need at least 200 servers allocated? Why pay for the ability to be flexible in how many servers you need on an hour-by-hour basis when your needs are much more stable and fixed?

This is where reserved capacity comes into play. Reserved capacity is the ability for you to commit to your cloud provider up front that you will consume a certain quantity of resources for a period of time (such as one to three years). In exchange, you receive a favorable rate for those resources.

Reserved capacity provides a way for you to receive capacity at a lower cost in exchange for committed allocation of those resources.⁵

The “Magic” of Usage-Based Resource Allocation

These services are easy to manage and scale because no capacity planning is required. This seemingly “magic” materialization of the resources necessary for your application using a usage-based resource is one of the true benefits of the cloud. It is made possible by the multitenant nature of the cloud service.

Behind a service like Amazon S3 is a huge amount of disk storage and a huge number of servers, which are allocated as needed to individual requests from individual users. If your application has a spike in the amount of resources it requires, the necessary resources can be allocated from a shared availability pool of services.

This availability pool is shared by all customers, and so it is a potentially huge pool of resources. As your application’s resource spike ebbs, another user’s application might begin to spike, and those resources are then allocated to that user’s application. This is done completely transparently.

As long as the pool of available capacity is large enough to handle all the requests and all the resource usage spikes occurring across all users, there is no starvation by any consumer. The larger the scale of the service (the more users that are using the service), the greater the ability of the cloud provider to average out the usage spikes and plan enough capacity for all the users’ needs.

This model works as long as no single user represents a significant portion of the total resources available by the cloud provider. If a single customer is large enough to represent a significant portion of the resources available for the service by the cloud provider, that customer can experience resource starvation during peak usage and potentially affect the capacity available to other customers, as well.

But for services like Amazon S3, the scale of the service is so massive,⁶ that no single customer represents a significant portion of usage, and the resource allocation of S3 remains magical.