A rock pile ceases to be a rock pile the moment a single man contemplates it, bearing within him the image of a cathedral.
As application developers, we hope that the software we painstakingly design, code, test, and deploy will be used and loved by many people. As the quote suggests, it is our vision and desire for great software that drives us to build successful applications from mere ones and zeros. We often measure the success of these applications with criteria such as user satisfaction or bug count. Part of our software’s success criteria can also be measured by the number of people who use it. When we design these applications, we must ensure that our end-to-end architecture will always support the needs of our users, even as the adoption and usage of the application grows. This is especially true for Web applications, which require specific design considerations to scale properly.
Designing Web applications for small audiences can be quite easy to accomplish, given the plethora of great tools and resources available to application developers. However, designing applications that scale to meet the needs of large numbers of users is quite challenging. Certain techniques that work well for applications that support small audiences can become problematic as the application’s usage begins to grow. For example, a Web application that performs very well for 10,000 concurrent users may produce a reasonably good user experience when growth doubles to 20,000 users. However, the same application may become unusable when growth again doubles, thus requiring additional hardware resources or code changes to ensure a consistent user experience as growth continues to trend upward. Quality and design issues that are the result of an application’s inability to scale properly are extremely costly to find and fix, and they may actually encourage users to switch to a competitor whose application properly scales.
As software has evolved beyond the desktop to a server-based delivery model, application scalability has become a critical element to high-quality software designs. Due to the nature of server-based applications, scale factors such as hardware infrastructure design, load balancing, deployment, and application manageability have a direct influence on both the application design and the cost associated with scaling to meet user demands. Scale factors should always be considered as part of the application design in order to prevent quality issues in real-world production scenarios.
Scaling and scalability are terms used to describe a Web application’s ability to handle increased usage or load from a growing demand for application resources. An application is considered "scalable" when it has the capability to increase total throughput under an increased load after additional hardware is added to the application’s hardware infrastructure. By contrast, applications whose designs fail as usage demand increases, despite the availability or addition of hardware resources, are not considered scalable. While this is a sufficient and general definition of scalability, it does not completely represent the characteristics of an application that is truly scalable. Let us examine some more specific characteristics of a scalable Web application.
Scalable applications can handle increased usage or data. As user demand for Web application features increase, the application should be able to expand to accommodate the additional user load in a linear way. This means that additional hardware infrastructure in the form of servers, data storage, or networking equipment could be added to handle the potential application usage overload. This also implies that users of the application should experience a consistency in the overall performance and behavior of the application regardless of load.
Scalable applications are always available and fault tolerant. Application scalability goes beyond just being able to apply hardware to address growth and consumption. Applications must always be available to their users to remain successful. Hardware and software problems will arise, application maintenance will be required, and users should not be affected by either scenario. Designing redundancy and fault tolerance into Web applications and deployment topologies ensures that both applications and infrastructure are resilient to everyday occurrences like disk failures or subsystem dependency outages.
Scalable applications are manageable and maintainable. As applications scale to meet user demand, hardware must be added to accommodate demand. An expanded hardware and software deployment footprint adds maintenance and manageability complexity that can lead to quality of service issues with the application. It is critically important that application developers design software that can be easily configured, deployed, and monitored once released.
An application that is designed to be scalable should possess the characteristics described above. When applied, they ensure that application growth can be a linear progression that is not encumbered by application design or operational issues. These characteristics should not be confused with other features of the application such as the overall performance or the choice of technology. The performance of an application is measured in terms of raw speed and does not indicate the application’s ability to accommodate additional demand. In fact, as previously mentioned, application performance should be consistent under a broad spectrum of usage load. Additionally, application scalability is not a problem that can be solved by a particular technology like .NET, PHP, or Java, but rather through development best practices. Web applications built in any language can be designed to scale provided they possess the characteristics described above.
Now that we have discussed a few characteristics of a scalable application, let us consider an example that illustrates the importance of ensuring that application designs meet scale requirements. Consider a Web application that is positioned to deliver social networking features to high school and college students (not an uncommon product these days). If this application’s usage is projected to be around 20 million page views per day, then the application architecture needs to support an average of about 230 page requests per second. (For simplicity, let’s assume an even distribution of load. In reality, application development teams need to plan for traffic peaks and even spikes depending on the scenario.) Additionally, suppose this application is going to store user data in a relational or structured data store. If each page reads the database two times, then the application data store needs to scale to accommodate approximately 460 queries per second. Let’s assume that an average Web or database server would individually not be able to accommodate these usage projections. Therefore, the application design must account for a strategy to support these projections and potentially scale beyond them as the demand requires. Let us explore the basic options for approaching scalability of applications.
Designing Web applications requires developers to consider all aspects of the end-to-end system design, including the hardware requirements of the application. Scalable application designs allow hardware to be added to the application infrastructure to meet the demand of increased user load. Therefore, application developers should be familiar with the appropriate tactics for scaling applications and incorporate them within their designs. Let us review the most common strategies for scaling applications properly.
There are two basic strategies for scaling an application to expand its hardware capacity. They are sometimes referred to as vertical and horizontal scaling but are more commonly known as scaling up (vertical) and scaling out (horizontal) strategies. Each of these approaches has certain benefits and drawbacks, which include factors such as cost of ownership and administration. In some cases, scaling up may be a better solution for a particular application than scaling out. It is important to understand the characteristics of each approach in order to best inform the end-to-end application design, so that the correct choice for achieving scale can be made.
The scaling up strategy for increasing an application’s hardware capacity is fairly simple. It involves either adding resources to a single machine or moving the application to a larger machine. Under typical circumstances, this could involve adding additional CPUs, memory, or disk to a single machine that is running the application or moving the application to a more powerful machine. As the applications usage grows, so does the need to purchase more resources for a single machine or, simply, a larger machine. This strategy is illustrated in 5-1.
Figure 5-1. An example of scaling up by upgrading existing hardware to meet the increased user demand.
The benefit of this approach is that it is perhaps the simplest from a design and implementation perspective. Applications can be designed and deployed to run on a single machine that has the necessary capacity to meet the user demand. As demand increases, the machine will require upgrading to accommodate the increased load. As long as there is hardware available to meet the needs of the application, the model will work effectively. By contrast, however, this model could become very costly as applications require bigger, more expensive hardware to meet their demands. In fact, if the application grows large enough, it may be cost prohibitive to maintain the application as the expense of the "big iron" increases. Additionally, the field of vendors who are building the equipment may also prove to be a limiting factor.
For some applications, scaling up might be the best option if the scope of the applications usage is thought to be small enough to maintain cost effectiveness. However, if application usage is broad and growth is thought to be constant and ever increasing, then scaling out is probably a better strategy in the long run. Let us review the specifics of the scaling out approach.
The scaling out strategy for increasing an application’s hardware capacity is the opposite of the scaling up strategy. Scaling out involves expanding the application’s deployment footprint by adding additional machines to help balance the increased user load. Unlike the scale up model that continually requires the addition of more powerful machines, the scale out model requires the addition of one or several servers with identical specifications for components like CPU, memory, and disk. As the application grows, adding capacity to handle the increased demand becomes almost as simple as racking a new server and powering it up. 5-2 illustrates this approach.
Figure 5-2. An example of scaling out with the addition of identical hardware to meet the increased user demand.
There are several benefits to pursuing a scale out strategy over a scale up. Without question, the biggest benefit is the lower cost associated with scaling out versus that of the scale up strategy. This can be achieved by either implementing clusters of high-quality, higher priced, modestly powerful servers or buying larger clusters of less expensive, less powerful, commodity servers. Clusters of commodity servers can be much more cost effective to purchase and operate than a smaller number of more powerful servers. The cost savings for these smaller commodity servers is realized in terms of lower cost of the equipment and the required power consumption. However, choosing commodity hardware over higher quality, higher priced equipment requires an investment in a solid failover solution, because commodity equipment tends to have a higher failure rate. Scaling out with higher priced, higher quality equipment will certainly achieve the core goals of scaling out, while also ensuring better equipment reliability. The best choice will likely vary by application and situation. In comparing the scale up versus scale out models, in certain cases the cost difference between them could be inconsequential. However, for larger-scale application deployments, the scale out model will likely be the more cost-effective choice.
Note
Power consumption may sound trivial in this particular explanation, but it should definitely not be underestimated. Companies like Microsoft, Google, and Yahoo! spend considerable time strategizing the expansion of their datacenters into locations that offer the most affordable power. When considering the impact of the power cost to run thousands of machines on the profit margin of a Web service, every kilowatt of power saved contributes to the overall profitability of the service.
In addition to the overall cost effectiveness of the scale out model, there are additional manageability advantages, as well. Given that this model expects our application load to be spread across several servers, it also distributes the risk across those servers. For instance, 1 server out of 20 may fail, and while that represents 5 percent of the traffic, the other 19 servers will balance the load effectively. This provides system administrators the flexibility to investigate a problematic piece of hardware and rebuild or replace it if need be, without affecting the users of the application.
By contrast to the benefits of this model, there are perceived disadvantages to the scale out model, as well. The most common of disadvantages is higher administration costs, or increased resource overhead associated with managing tens, hundreds, or thousands of servers. For large, industry-leading Web applications at Microsoft, these concerns have largely been mitigated by automated administration software that allows the ratio of servers to administrators to be very high. This approach could also be applied to small to mid-sized Web applications, as well, given the availability of application programming interfaces (APIs) for managing all aspects of server administration.
As previously discussed, applications with broad usage and constant, upward growth are good candidates for a scale out approach to addressing capacity needs. In general, the scale out model is much more flexible, cost effective, and lower risk than the scale up strategy. In certain cases, however, application developers may have specific design requirements that will not necessarily scale linearly. An example of this might be an application that needs to aggregate data from all database nodes across all clusters, which would not necessarily scale well using this approach. It is important that these requirements get surfaced early in the design cycle, so that scalability decisions can be addressed long before the application is ready to be deployed.
Thus far, we have discussed the philosophies of scaling up and scaling out an application’s server infrastructure to address increased user load on Web applications. While the aforementioned strategies will definitely help to improve the capacity of the servers handling the inbound Web page requests, they do little to help address potential capacity concerns of a database infrastructure. It is a fair assumption that most Web applications today rely heavily on a database management system (DBMS) like Microsoft SQL Server. Therefore, it is critical that application developers, who are designing their applications to scale, consider how to also scale the database infrastructure on which their applications depend. Let us review a common approach to scaling databases.
When considering design options for scaling the database tier of a Web application, the same basic options apply—scale up or scale out. For applications whose scope is fairly limited in terms of the number of users, page views, and database queries or updates, choosing the scale up approach is likely the right choice. However, for applications that expect broad usage, increasing growth, and a large volume of data, then scaling out is the better choice. Scaling databases across several servers is quite a bit more challenging than just buying more powerful equipment and growing the database on a single server or network-connected disk array. Let us review how best to accomplish this.
The most widely accepted method used to scale an application’s database infrastructure across multiple servers is data partitioning. This is a process that involves logically and physically dividing the application’s data set across multiple servers. Microsoft SQL Server 2008 enables this using a combination of database federation, which transparently integrates several databases into a single federated database, and partitioning. This is accomplished by establishing a group of independent servers that cooperate to process database access requests. The application then interacts with the database through mid-tier routing rules that understand the partitioning scheme and can connect each request with the appropriate member server and partition. 5-3 provides an illustration of how federation, partitioning, and routing are applied to an application that needs to scale its data source.
The strategies previously enumerated are very common approaches to achieving scalable Web application designs, but the detailed requirements for implementation are complex and not to be underestimated. For example, features and capabilities of networking gear can vary by manufacturer, and effectively architecting and configuring hardware infrastructure requires specific expertise in the selected equipment. Scaling services both at the application code tier and data tier can be extremely challenging and requires intense planning and design prior to implementation. The important thing is to address these design considerations up front, rather than after the application is developed. This approach will ensure that your Web application will scale with user demand, and any post-release quality issues related to excessive and unhandled load will be avoided after the application is released. Now that we have reviewed some common scalability strategies that can be applied to your Web application designs, let us review some more specific methods for designing Web applications with scalability in mind.