To answer this question, Little's Law comes to the rescue. This law explains how to calculate the number of requests that are processed simultaneously (or simply how many parallel workers there should be) to handle a predefined throughput at a particular latency level. In other words, using this formula, we can calculate the system capacity, or how many computers, nodes, or web application instances running in parallel we need in order to handle the required number of users per second with a stable response time:

The preceding formula may be explained as: the mean number of requests resident (or the number of requests processed simultaneously) in a system or queue (N) is equal to the throughput (or users per second) (X) multiplied by the mean response time or latency (R).

That means that if our system has an average response time R of 0.2 seconds and throughput X of 100 requests per second, then it should be able to process 20 requests simultaneously, or 20 users in parallel. We either need 20 workers on one machine, or 20 machines with one worker. That is an ideal case, where there are no intersections between workers or simultaneous requests. This is shown in the following diagram:

As we can see in the preceding diagram, there are three workers in the system, which can handle six actors or requests per second. In this case, all the actors are balanced between the workers, which means that there is no coordination required between them in order to choose the worker.

However, the preceding case is not actually very realistic, because any system, such as a web application, requires concurrent access to shared resources such as CPU or memory. Hence, there is a growing list of amendments to the overall throughput, which is described in Amdahl's Law and its extension, the Universal Scalability Law.