Managing Workloads
Management, in this context, refers to how resources are assigned in order to process workloads. Assignments might be based on resource availability, business priorities, or event scheduling.
The idea of managing workloads has been around for decades. In the unified mainframe computing era, workload management was pretty straightforward. When a task had to be executed, a job was scheduled to run on that system. The instructions for running that task or job were typically written in a complex job-control instruction language. This set of commands helped the IT organization carefully plan the execution of workloads. If a mission-critical workload required a huge amount of time to run, a set of instructions could be established to stop that workload and allow another workload to run. When the second workload finished executing its task, the long-running workload could resume. If there were dependencies that the workload needed to complete a task, a command could be issued to go find that task so it could be executed and then the result added to the workload.
Here are a few principles to consider as you start to think about managing workloads in a cloud model:
Understand processing requirements. In other words, you need to understand how your computing resources can execute your workloads on average and at peak demand. In general, IT often engineers its computing resources to meet the peak workload.
Use modeling resources. You need to work out what CPU, disk, and memory are needed to execute workloads. Generally, you create some sort of model to do this. Your model might be a simple linear model that calculates the amount of CPU per service, or it might be something more complex.
Determine the capacity you need. Optimize your resources based on required response time, number of services, and numerous other variables that need to be considered depending on what you’re trying to accomplish with your workload.
The challenge in managing any workload is making sure that it can be executed and delivered at the right performance level. The principle is not that difficult if you’re dealing with applications running on one server. However, as IT infrastructures become more complex and heterogeneous (such as in the hybrid cloud), this becomes harder to do.
The load balancer
In distributed computing environments, such as the cloud, a load balancer is often deployed to help ensure that no single machine is overwhelmed. Load balancing is the process where IP traffic is distributed across multiple servers. The figure illustrates a simple example of load balancing. Here, the load balancer sits in front of three servers and distributes the load across them. All three servers utilize the same database — the load balancer helps to optimize resource allocation across the servers.
How does it do this? First, it needs to know where the data is coming from. Second, it needs to know which servers it’s supposed to distribute the load across. Then it uses an algorithm to determine which machine to give the load to. For example, a popular algorithm is the round-robin algorithm, where a specific region on the distributed platform is used for the first request. Then, the next region is used for the next request, and so on. When the end of the list is reached, the next region used is the first one. And, so on. Of course, there are numerous different algorithms that can be used to balance workloads across servers. The round robin is just one of them.
