Throughput is the quantity of data that is sent and received at a given time, while latency is defined as the time between when the user initiates a request in the application and receives the response from the application. When it comes to networks, bandwidth plays an important role.

Throughput and latency have a direct relationship as they work together. Lower latency means high throughput as more data can transfer in less time. To understand this better, let's take the analogy of a country's transportation infrastructure.

Let's say that highways with lanes are network pipelines and cars are data packets. Suppose a given highway has 16 lanes between 2 cities, but that not all vehicles can reach the destination at the desired time; they may get delayed because of traffic congestion, lanes closing, or accidents. Here, latency determines how fast a car can travel from one city to another, while throughput tells us how many cars can reach their destinations. For a network, using full bandwidth is challenging because of error and traffic congestion.

Network throughput transfers a significant amount of data over the network in bits per second (bps). Network bandwidth is the maximum size of the network pipeline, which it can process. The following diagram illustrates the amount of data transferred between client and server:

In addition to the network, the throughput is applicable at the disk level. Disk throughput is determined by a factor of input/output per second (IOPS) and the amount of data requested (I/O size). Disk throughput is determined in megabytes per second using the following formula:

Average IO size x IOPS = Throughput in MB/s

So, if your disk IOPS is 20,000 IOPS and the I/O size is 4 KB (4096 bytes), then the throughput will be 81.9 MB/s (20,000 x 4096 and converted from bytes to megabytes).

Input/output (I/O) requests and disk latency have a direct relationship. I/O means write and read respectively, while disk latency is the time taken by each I/O request to receiving the response from the disk. Latency is measured in milliseconds and should be minimal. This is impacted by disk revolution per minute (RPM). The IOPS is the number of operations that the disk can serve per second.

The throughput, also applicable at CPU and memory level, is determined by the amount of data transfer between the CPU and RAM per second. At the database level, throughput is determined by the number of transactions a database can process per second. At the application level, your code needs to handle transactions that can be processed every second by managing the application memory with the help of garbage collection handling and efficient use of the memory cache.

As you learned when you looked at latency, throughput, and bandwidth, there is another factor called concurrency, which is applicable to the various components of architecture and helps to improve application performance. Let's learn more about concurrency.