As much as Nginx will help your servers hold the load, there are always limits to what a single machine can process: an aging hard drive or limited bandwidth will eventually induce a bottleneck, resulting in longer request-serving times, which in turn leads to the disappointment of your visitors.

As your websites grow more popular and your single machine begins to suffer, you will be tempted to simply get a bigger and more expensive server. But this would not be a cost-efficient approach in the long run, and remember that the more strain a server is exposed to, the more likely it is to suffer from hardware failure.

In this chapter, we will investigate two concepts, the first of which is load balancing: the art of distributing a load across several servers and managing this distribution efficiently. The second part will explore the subject of thread pools: a new mechanism relieving servers under heavy loads (more specifically, loads induced by blocking operations) by serving requests in a slightly different manner.

This chapter covers the following topics:

• An introduction to load balanced architectures
• Common concepts and issues of load balancing
• Dealing with the session affinity problem
• A demonstration of load balancing with MySQL
• Optimizing your setup with the help of thread pools