Chapter 24. Scalable Computing Options
Setting up, configuring, and managing servers, and deploying your applications to them, is only one way of deploying your high-scale application. There are other alternatives, each with a different set of advantages and disadvantages. In this chapter, we’ll discuss various alternatives, from cloud-based servers to AWS Lambda and talk about how they differ, how they are the same, and how to decide which to choose for your application’s needs.
In this chapter, we compare various cloud-based1 application execution environments, specifically:
- Cloud-based servers
-
This is basic server technology. Examples include Amazon EC2 servers.
- Compute slices
-
These are traditional software applications running in compute environments independent of servers. Examples include Heroku Dynos and Google App Engine.
- Dynamic containers
-
Full server capabilities wrapped in the ability to quickly start, stop, and migrate to new systems. Docker is the best example of this.
- Microcompute
-
Small size, highly scaled, event-driven execution environments. AWS Lambda is the best example of this.
Before we compare these methods, let’s look at each of them individually.
Cloud-Based Servers
Cloud-based servers are the simplest way to acquire scalable computing, and are the most consistent with traditional programming and system architecture models. Cloud-based servers are available everywhere, with Amazon EC2 instances being the most popular and best-known example.
The primary benefit of using cloud-based servers is their ability to be quickly brought online and utilized, which is especially useful for scaling. However, they require an application to be constructed in such a way that adding additional servers to a deployment can actually improve the application scalability. Although this might be true in many environments for many applications, it is not universally true.
Advantages
-
Least expensive option per compute cycle
-
Very few limits on functionality; most server capabilities are available
-
Run continuously
Disadvantages
-
Allocated capacity.2 Scaling is about provisioning new servers and deploying an application to that server.
-
You pay for allocated capacity, not used capacity.
-
Application architecture must support scalability by allowing additional servers to be brought online.
-
Many advantages of standalone servers cannot be utilized effectively in a scaled environment (such as local storage capabilities).
-
The server must be managed and maintained by the application owner, including software and security upgrades.
Optimized Use Cases
Cloud-based servers are good for a wide variety of general-purpose use cases and can be utilized for most scaling needs.
Compute Slices
Compute slices are an alternative execution model that involves executing applications without knowledge of which server they are running on. They involve taking an application’s software and deploying it to a Platform-as-a-Service (PaaS) infrastructure that will execute the stack in a managed way. This is done without exposing the specifics of the server on which software is running.
There are several examples of compute slice–based compute engines, but Heroku Dynos is a classic example.
Advantages
-
Easy to vary allocated capacity at a relatively granular scale.
-
Providers of compute slices have a strategy of over-provisioning of slices. This facilitates potentially lower per-slice cost points, which is especially useful for low-traffic applications.
-
No server management required.
Disadvantages
-
More expensive per-compute cycle than cloud-based servers.
-
Allocated capacity. Scaling is about provisioning new slices to run your application.
-
You pay for allocated capacity, not used capacity.
-
You have no control over the server and infrastructure components that are being utilized.
Optimized Use Cases
A traditional use case is a low-traffic web application.
It’s most effective where a traditional application or service needs to be run, but the owner does not want to deal with managing servers or other infrastructure requirements.
In applications for which usage is relatively low and a small number of slices can run the entire application, slices can provide a cost-effective option. However, the larger the application, the more costly it is compared to basic servers.
It should be noted that Google App Engine is another example of compute slice–based programming and provides similar advantages/disadvantages to what we just described, except their payment mechanism is different and allocation is handled more dynamically. The point here is that many of the advantages and disadvantages of slice-based computing depend on the implementation, not the general concept.
Dynamic Containers
Dynamic containers is a specific use case of container technology that involves dynamically allocating and migrating containers to different servers to provide a highly scaled, managed compute environment. Note that containers can be utilized as a deployment technology in any of these compute environments. What we are speaking about here is a dynamic model for easily moving and scheduling containers to optimize system resources and make adding new containers easy.
In a dynamic container environment, an application, wrapped in a container, is designed to be easily started, stopped, and relocated quickly and easily. Combined with a microservice-based application architecture, dynamic containers can provide a highly dynamic and fluid application that can be scaled easily and effectively, and existing compute resources can be optimized more effectively than with traditional compute or more static container deployment strategies.
Advantages
-
Inexpensive and easy to effectively optimize existing server capabilities.
-
Scaling is easier because containers can be deployed dynamically wherever compute resources are available.
-
Containers place few limits on the application with the exception, perhaps, of slow application startup time, which can negatively affect advantages of container scheduling).
-
Containers can run either continuously or only when needed, conserving resources.
-
Containers are easy to deploy and servers require minimum configuration for them to be utlized.
-
Starting up new services in an environment is easy and can be automated.
Disadvantages
-
Requires active container management and software for managing container deployment and scheduling. Some cloud providers have services that help with this, but it is still a relatively new field and no single solution for managing dynamic containers has proven to be better than other approaches. The sweet spot in container scheduling and management has not yet been realized.
Optimized Use Cases
Dynamic containers are great for easy deployment of complex applications in a microservice-based architecture. They also are great for rapidly starting and stopping service instances (container start/stop is very similar in scope to starting/stopping a Linux process).
Microcompute
Microcomputing is the pinnacle of scalable compute technology. Microcomputing is a compute model wherein a relatively simple (micro) piece of code is available to execute, and is run whenever necessary in response to some incoming event, such as a user request or a service call. Each event causes a single execution of the microcompute function to run and act on the event. An arbitrary number of microcompute functions can be executing independently at a time, handling all incoming requests as they occur. This effectively provides a nearly limitless scaling runtime environment. AWS Lambda is the best example of microcompute technology.
Microcompute is similar to the compute slice model of program execution. The biggest difference is that compute slices are allocated and scheduled in discrete units. You can change the number of running discrete units to handle incoming requests, but ultimately your ability to scale to the number of incoming requests is limited by your ability to scale-up discrete compute slices. Microcompute is scaled completely dynamically with the number of instances running only a function of the current number of requests to process.
To make this possible, compute functions used within Lambda must be small, nimble, and require virtually no initializtion or destruction to execute effectively.
Advantages
-
Automatic and near limitless scaling as needed3
-
No server management, no system upgrades
-
Charges based on actual number of events processed; limited overhead
-
Easily supports rapidly changing scaling needs (scale up/down is transparent)
Disadvantages
-
Significantly limited functionality and languages.
-
Applications must be architected around the use of the microcompute technology.
-
Most expensive on a per-compute-cycle basis, but pricing models and the ability to charge at a very granular basis can help with this.
-
Current implementations (AWS Lambda) do not provide good deployment, rollback, or A/B testing capabilities, making them more difficult to utilize in a highly available application without additional application architecture or external tooling.
Optimized Use Cases
Microcompute is optimized for high-volume data streaming and event processing. It is useful for data validation, transformation, and filtering. On the network edge, it is useful for validation and regulation of inbound data.
This is most effectively used when small operations utilizing a small code base is necessary to perform specific event-based programming.
Now What?
With the wide variety of scalable compute options available now, almost any application and IT operational need can be successfully met by one or more of the available alternatives.
But how do you determine which one you should use? Many of these techniques are available only on specific cloud providers. As such, the availability and desire to use these compute options may be a piece of your cloud provider selection process.
Still, almost any major cloud provider can support many of these options, which is one of the advantages the cloud offers today. Don’t be afraid to use multiple options within your application, as different parts of your application will often have different needs. As always, consider your application’s availability and scalability needs, and pick the solutions that best match the needs of your application and the teams that support that application.
1 Some of these techniques apply to non-cloud-based deployment environments, but because our discussion is focused on scalability, we’ll examine scalable cloud-based uses only.
2 See Chapter 23 for a discussion of the pros and cons of allocated capacity.
3 Obviously it is not limitless scaling, but for any given practical application needs, it can be considered near limitless.