Creating a new Web cluster isn’t difficult, but there are a number of preliminary steps that you should complete so that Application Center can create a cluster that works correctly the first time out. These steps include

When you’ve prepared your soon-to-be cluster members, you’re ready to form your new cluster. You can either install Application Center on each cluster member now, or install it as you join each member to the cluster. At the very least, you’ll need to start by installing Application Center on the computer that will become the cluster controller. Once Application Center (and any recent Application Center service packs) is ready to go, follow these steps to create your new cluster:

Congratulations! You’ve got a new, single-member Web cluster, all ready to go. Although it only contains one member, the cluster will behave like any other cluster. You can begin using it immediately or, more likely, you can start adding members to it.

Joining a new member to a Web cluster is easy. Simply launch the Application Center console, right click Application Center, and select Connect from the pop-up menu. Type the name of the new member, and select the option to Manage cluster for the specified server. A dialog box will pop up offering to join the server to an existing cluster, and you just have to enable Application Center to do so. Keep in mind that the new member must already have Application Center installed.

There’s another way to add new members that I find to be a little faster:

If you’ve already configured Web applications on the cluster’s controller, and you’ve configured automatic synchronization, Application Center will automatically deploy the applications to the new member. This is a great feature, as it enables you to quickly add new members to a cluster without having to spend the time to manually add the Web sites’ content to the new member. Basically, you just install Windows and Application Center on a new server, add the server to an existing cluster, and you’re done.

Application Center monitoring is accomplished in part by the Microsoft Health Monitor, which is included with the Application Center default installation. Application Center also collects operating system and Application Center events from cluster members, enabling you to view these events in a single location.

To view a cluster member’s current status, simply select the member in the Application Center console. The member’s status appears in the right pane, and is updated every 30 seconds. To view events associated with a cluster or member, expand the cluster or member in the console, and select Events. The events will be listed in the right pane of the console, as shown in Figure 16.6.

Figure 16.6. Viewing events for a cluster displays the events from all cluster members; viewing a single member only displays that member’s events.

You can also view performance counters for a particular cluster member, or aggregate performance counters for the entire cluster. To do so, simply select the cluster or member in the Application Center console. As shown in Figure 16.7, performance counters are displayed in the right pane of the console.

Figure 16.7. Viewing a cluster’s performance is a great way to get a feel for how the cluster members are performing as a team.

Viewing events and performance data for an entire cluster from a single window is very useful. However, both assume that you’ve got the time to sit and watch the cluster on a regular basis, looking for problems. Unfortunately, few of us have that kind of time! Busy Application Center administrators learn to take advantage of the Microsoft Health Monitor, which is designed to watch a variety of performance and other data and categorize each as “healthy” or “unhealthy.” Basically, Health Monitor sits and watches performance data for you, and lets you know when performance data starts to move from acceptable ranges into ranges that you need to worry about. At that time, Health Monitor can take actions to alert you, execute corrective actions, and so forth.

You use Health Monitor by configuring one or more monitors. Each monitor contains a data collector, which retrieves performance and other data about a server. Each monitor includes thresholds, which define performance conditions that you consider to be acceptable, marginal, or critical. Monitors can also include actions—such as notifications—that are triggered when the data collector moves from normal to warning, or from warning to critical.

Start using Health Monitor by creating a basic notification action, which can be used to inform you of a problem with a particular monitor. To do so

Once created, this action can be reused by multiple monitors. The next step is to create a data collector to watch a particular performance point:

You’ve now created a collector that uses your action. All you need to do now is define the thresholds that indicate Normal, Warning, or Critical status for your collector. To do so, follow these steps:

That’s it! You’ve now created a new monitor which will watch a specific performance condition, change its status from OK to Warning or Critical as appropriate, and notify you (or take other actions) as appropriate. You can view the status of individual monitors in the Application Center console, as shown in Figure 16.8.

Figure 16.8. Be sure to occasionally view the status of your monitors to make sure they (and their associatedactions) are functioning correctly.

Maintaining a Web cluster is just as easy as creating one. Again, most of the maintenance you do is going to be normal IIS stuff, such as configuring new Web sites or adding Web content. Application Center plays a role in that type of administration, but it’s a background role: Application Center handles content and IIS configuration synchronization from the cluster controller to all cluster members.

There are a few common Application Center-specific tasks that you’ll need to perform from time to time, including taking a member out of the cluster for maintenance (or permanently), moving the cluster controller role, and restarting members after you’ve deployed new content or performed some kind of maintenance. I’ll cover those tasks in the next three sections.

Removing a Member

Sometimes you’ll need to remove a member from the cluster so that you can install service packs or perform other maintenance; sometimes you’ll want to completely remove a member from the cluster, either to redeploy the server elsewhere in your organization, or to retire an older server completely.

If you just need to temporarily remove a member from the cluster (say, for maintenance), just right-click the member in the Application Center console and select Set Offline from the pop-up menu. Offline members can still synchronize with the cluster controller, but they won’t accept any new requests from clients. When you set the member offline, you can specify a drain time, as shown in Figure 16.9. The default is 20 minutes, which is designed to allow all existing client connections to close naturally. That way, clients won’t suddenly be cut off; instead, they’ll be able to finish whatever they’re doing. In most cases, “finishing” simply means receiving whatever Web page they just requested; once the member is set to offline, clients will re-load balance to another cluster member for subsequent page requests.

Figure 16.9. Specify a drain time of at least 10 minutes to ensure that all in-process requests are completed normally.

Caution

Setting a member offline can have negative effects for clients if you’re using NLB’s client affinity feature. The now-offline member contained the only copy of clients’ ASP session variables. Although clients will still re-balance to a new cluster member, that member won’t contain the clients’ session information, so it will appear to clients as if they are starting over all of a sudden. Unfortunately, there’s no way around this problem other than avoiding session variables (and, hence, client affinity) completely.

If you need to take a member offline immediately, select Take offline now in the Set Offline dialog box. The member will immediately stop responding to client requests, even ones that it was in the middle of processing. Clients who were waiting on a response from the member will receive a 401 or 404 HTTP error message, which indicates that the server was unavailable. If clients retry their request, they will be re-balanced to a different cluster member, which should be able to fulfill their request normally.

When you’re done working on the offline member, use the Application Center console to set the member back online: Right-click the member and select Set Online from the pop-up menu. The member will immediately re-enter the load balancing loop and begin accepting new client requests.

If you want to permanently remove a member from a cluster, right-click the cluster member, select All Tasks, and then select Remove Cluster Member. Removing the member won’t delete any Web content; you’ll need to manually remove the Web content if you wish to do so. After you remove a member from the cluster, there is no way to add it back in other than re-joining it to the cluster from scratch.

If you’re trying to permanently remove a cluster member, and receive error messages indicating that settings could not be removed from the member, see “Couldn’t Remove Settings from Member” in the “Troubleshooting” section at the end of this chapter.

Although Web farms are often built with identical servers, that isn’t always the case. As you expand your Web farm, newer servers will likely be more powerful than your original ones. NLB enables you to assign each cluster member a weight, which indicates each member’s relative ability to handle the Web farm’s traffic.

To begin with, you’ll need to identify your least powerful Web servers and assign them an arbitrary weight of, say, 10. The actual number isn’t important; what’s important is the relationship between other server’s weights. After configuring your least-powerful servers, move on to the next–most-powerful servers. If those are twice as powerful as the least-powerful servers, assign them double the weight: 20. This tells Application Center that the server can handle twice the load as a server with a weight of 10.

Configure each of your cluster members with an appropriate weight. Your numbers are sure to be somewhat arbitrary, as it’s difficult to determine when a server is exactly twice as powerful (for example) as another server. Do your best, and don’t worry about being too exact. The idea is to give Application Center a rough idea of which servers are more powerful, and approximately how much more powerful they are.

Once you’ve determined the server weights for each cluster member, you can configure Application Center with the appropriate weight values. To do so

The ideal configuration for client affinity is the default: Off. Remember that client affinity’s entire purpose is to partially defeat load balancing in order to provide ASP session variable compatibility. If you don’t use session variables in your Web applications, leave client affinity completely off. If you do, however use session variables in your applications then you’re going to have to use client affinity, too.

You configure client affinity in the properties of a Web cluster. Open the Application Center console, right-click a cluster, and select Properties from the pop-up menu. You’ll see the dialog box shown in Figure 16.11. Select either Single Client or Class C affinity, as appropriate, and click OK. The cluster controller will synchronize the change to all cluster members.

Figure 16.11. Make sure the cluster controller is available before modifying any of the cluster’s properties, including client affinity.

Which client affinity setting should you choose? If your Web site operates in an intranet, select Single Client affinity. Each new user that connects to the Web farm will be load balanced to the least-busy server, and will continue to use that server for the remainder of their session. On the Internet, you’ll need to select Class C affinity, because a single client may appear to be using multiple IP addresses due to their Internet service provider’s (ISP) Network Address Translation techniques. In Class C affinity, each request from a new class C IP address range (such as 131.127.15.x) will be load balanced to the least-busy cluster member, and that member will be responsible for handling all future requests from all clients using that same class C IP address range.

Practically speaking, Class C affinity means that a single server will handle all requests from a given ISP, or at least from a large portion of the ISP’s customers. This means a single server could end up handling all of the traffic from mid-Atlantic America Online members, which can be a significant load. While required for ASP session variable compatibility, Class C affinity can so undermine Application Center’s capability to load balance user connections that it might not seem to be worth using load balancing at all, especially on busy Web sites. I recommend that you eliminate your site’s dependency on session variables as quickly as possible.

The Request Forwarder is designed to examine incoming Web traffic after it has been load balanced to a particular cluster member, and then redirect the traffic if necessary. To enable the request forwarder, open the cluster’s properties dialog box and select the check box to enable Web request forwarding, as shown in Figure 16.12.

Figure 16.12. The request forwarder can forward different types of requests, as shown here.

Web request forwarding is an alternative to client affinity for ASP session variable compatibility. Essentially, each incoming HTTP request contains a cookie that indicates which cluster member is handling that client’s session information. The request forwarder examines this cookie and redirects the incoming request to the appropriate member. As shown in Figure 16.12, you can also configure the request forwarder to forward publishing requests, which include content updates and administrative traffic. These requests are always redirected to the cluster controller.

The request forwarder places a significant additional burden on Application Center clusters. The cluster must still use NLB to load balance requests, and only then will the request forwarder examine each incoming request and redirect it if necessary. If at all possible, you should avoid the use of the request forwarder: Use client affinity, if necessary, to support ASP session variables, and manage your clusters from their back-end network adapters (ensuring that you can explicitly connect to the controller without the need for request forwarding).

You may choose to use the request forwarder if you have only a couple of ASP pages that rely on session variables, and if the majority of your site’s content is contained in static HTML files. As shown in Figure 16.13, you can configure the request forwarder to ignore certain types of content based on their file extension, such as HTM or HTML for static Web pages. Doing so will reduce the negative performance impact of the forwarder, since it won’t bother examining those pages for cookies that indicate a redirect is necessary.

Figure 16.13. Include all non-dynamic content in the forwarder’s exclusion list to minimize the forwarder’s impact on cluster performance.

As shown in Figure 16.14, the Application Center console enables you to work with applications. When you install Application Center, it automatically creates several default applications, which are listed in Figure 16.14.

Figure 16.14. Applications are managed for an entire cluster, not for individual cluster members.

The default applications include one application for each Web site that is configured in IIS when Application Center is installed, including Application Center’s own administrative Web site. Application Center also creates an application that includes all Web sites on the server, and automatically updates the contents of that application as you add and delete Web sites under IIS.

You can remove any of the default applications except for the “all sites” default application, and you can, of course, configure your own applications to represent the Web sites that the cluster will host. You can also modify the default applications to include additional resources. Since Application Center creates a default application for each Web site on the server, you’ll generally just modify one of the default applications to make it more complete.

To add a new application, just click the New button in the toolbar, shown in Figure 16.14. All you’ll provide is a name for the application; it starts out with no resources included. To modify the resources in an application—whether a default application or one that you’ve just created—you’ll use the Add and Remove buttons at the bottom half of the right-hand side of the console.

To modify an application’s resources, you’ll need to start by selecting the resource type, as shown in Figure 16.15. You can’t use the Add button when All Resources is selected in the drop-down list box; you must select a specific type of resource in order to add the resource to the application.

Figure 16.15. Make sure each application includes all the resources that it needs to function correctly once deployed.

Don’t include COM+ applications along with your Web site applications. Any application including a COM+ application (which I’ll call a COM+ component) cannot be synchronized, because installing a COM+ component on a server requires the server to restart. Instead, applications that include COM+ components must be deployed, which is a manual process. The best strategy is to create one application for all of your Web site’s resources except COM+ components, and then create standalone applications for each of the COM+ components required by your Web site.

When you create an application and add a COM+ resource, you’ll only be able to select from a list of COM+ components that are already installed on the computer, as shown in Figure 16.16.

Figure 16.16. Be sure your COM+ components are properly installed and configured before trying to add them to an application.

I always create a separate application for each COM+ component, or for each group of closely related components. That may seem like overkill, but it enables me to easily deploy new versions of the components without having to deploy every other component associated with my Web site.

Synchronization is the process of ensuring that cluster members contain the same application resources as the cluster controller. You can synchronize an application on demand by selecting the application in the Application Center console and clicking the Synchronize button, shown previously in Figure 16.14. Generally, though, you’ll have your applications synchronize with the controller automatically. To configure automatic synchronization, modify the cluster properties as shown in Figure 16.17.

Figure 16.17. You can specify an interval for full synchronization. The default of every hour is sufficient for most environments.

By default, Application Center’s automatic synchronization is triggered by file changes on the cluster. Whenever a file changes, Application Center synchronizes that file. When a large number of files change at once, it’s possible for Application Center to “miss” some of them, which is where the full-synchronization interval comes in. This interval specifies how often Application Center will attempt to completely synchronize each cluster member with the controller. Note that only changes are actually synchronized, but Application Center does check each and every resource to see if a synchronization is necessary on each member. This time interval acts as a backup for the change-based synchronization, ensuring that all changes are eventually deployed to each cluster member.

You use the Application Center console to manage application deployments. You can create a new deployment by right-clicking the cluster name in the console and selecting New Deployment from the pop-up menu. You’ll create a name for the deployment and, as shown in Figure 16.18, determine whether the deployment will occur between clusters or within a cluster.

Figure 16.18. You’ll often use intra-cluster deployments to deploy COM+ applications; inter-cluster deployments enable you to deploy entire Web sites.

You’ll also need to specify the deployment’s target servers, as shown in Figure 16.19. Here are some tips for using deployments:

Figure 16.19. Other deployment properties enable you to specify a deployment time and automatic restart (if necessary) of the server or Web site.

In an NLB cluster (which Application Center calls a General Web cluster), each cluster member communicates with the others, enabling the members to determine which server is the least busy and will handle the next incoming user request. CLB clusters, however, don’t include this information. Instead, each cluster client is responsible for maintaining a list of CLB cluster members, called the CLB routing list. Each client is also responsible for communicating with each CLB server to determine which one is the least busy. The routing list is maintained as part of a Web cluster’s properties, as shown in Figure 16.20.

Figure 16.20. CLB clusters don’t maintain a virtual name or IP address, so you’ll have to manually list each cluster member on each cluster client’s routing list.

Application Center doesn’t perform any validation to ensure that the servers you enter are, in fact, Application Center CLB cluster members. Application Center simply verifies that it can connect to each server, and enables you to add it to the list.

CLB clusters are created in much the same way as NLB clusters—right-click Application Center in the Application Center console and select Connect from the pop-up menu. Then, provide the name of the new cluster’s controller, and select the option to create a new cluster. As shown in Figure 16.21, you’ll select the COM+ cluster option instead of the General Web cluster option to begin creating a CLB cluster.

Figure 16.21. Once created, you can join additional servers to the cluster, just as you would for an NLB cluster.

Application Center will then prompt you to select a load balancing option, as shown in Figure 16.22. Your choices are to use CLB or NLB. Generally, you’ll select CLB, which assumes that the new cluster will be accessed by other Application Center computers (most often Web clusters). However, you can create COM+ clusters that use NLB instead, making them accessible to non-Application Center computers. If you select NLB, then you’re really just creating a different type of Web cluster, as opposed to a true CLB cluster.

Figure 16.22. Select the CLB option if your new cluster will be the middle-tier for Application Center-based Web servers.

COM+ routing clusters use a combination of NLB and CLB. The cluster uses a virtual name and IP address, just like an NLB cluster, to accept client requests on load balanced network adapters. The cluster then uses CLB to forward COM+ requests from its back-end network adapters to a standard CLB cluster. The purpose of routing clusters is to provide better accessibility to CLB clusters for non-Application Center clients, and to enable a single Application Center Web cluster to access multiple CLB-based middle tiers.

When you create a new cluster in Application Center, you have the option to create a COM+ routing cluster, as shown in Figure 16.23.

Figure 16.23. Routing clusters often contain a controller and a single member, which provide redundancy for the cluster’s operations.

You’ll also need to select a front-end load balancing option, just as you would for a Web cluster. If the computer doesn’t have at least two network adapters installed, you won’t be able to select NLB, and will be required to use an external load balancing solution. Figure 16.24 shows the selection screen.

Figure 16.24. NLB provides the easiest to configure and most efficient load balancing solution for routing clusters.

You’ll finish configuring the new cluster in much the same way as a Web cluster, including selecting the load balanced and back-end network adapters, and so forth. You’ll also need to configure the new cluster’s CLB routing list with the names of the CLB cluster members that the routing cluster will route requests to. Keep in mind that any COM+ components that will be routed must be locally installed on the routing cluster and configured for load balancing, which I’ll cover next.

The COM+ components used in a CLB cluster must be installed on both the CLB cluster members, where the components will physically execute, and on the CLB cluster’s clients. The clients must have the components so that they have the components’ registration information, which is configured when the component is installed. The clients must also specifically configure the components for load balancing, using the Windows Component Services console. As shown in Figure 16.25, each component that can be load balanced will include the appropriate checkbox on its Activation tab. You must select this checkbox on each CLB cluster client. Doing so ensures that, when the component is instantiated on the client, the COM+ request is intercepted and sent to a server on the CLB routing list instead of executing locally.

Figure 16.25. Not all components support dynamic load balancing; components that don’t will disable the checkbox.

No special COM+ component configuration is required on the CLB cluster members.

Application Center is used to build scale-out applications, where you grow the application by adding additional cluster members. This type of scaling contrasts with scale-up applications, where growth is enabled by buying bigger and more powerful server hardware, but not necessarily increasing the number of servers in use. The big disadvantage to scale-up applications is that all of your eggs are in one basket: the one server you’re using. Scale-out applications, on the other hand, have significantly lower risk because each server in the application is identical. In other words, if a Web cluster member fails, it’s no big deal. Sure, the other servers may have to work harder to take up the slack, but you won’t lose any data, because each cluster member is essentially an identical copy of the others. The only way you can really lose all of your application’s files is in a major disaster, such as a meteor strike on your data center that wipes out all of your cluster members at once.

The primary strategy in Application Center disaster recovery planning, then, is to back up the cluster controller. This ensures that the cluster configuration remains intact, and provides you with a copy of your application files, which you can store offsite. Actual disaster recovery operations, however, will almost never need to utilize that offsite backup. If the cluster controller fails, you have a couple easy-to-implement options:

If a cluster member other than the controller fails, just fix it and re-add it to the cluster. Or, if it’s a really bad failure that you can’t repair, just replace it with a new server. In either case, Application Center will make sure the cluster members receive the correct content, although you’ll need to use deployments to set up new members with any COM+ components.

The core disaster recovery tasks for Application Center are easy: Use Windows Backup or your favorite backup software to regularly make a backup tape of the cluster controller. Be sure to capture the controller’s system state, which includes the system registry. I never bother backing up the other cluster members, since they’re essentially already backups of the controller itself.

If you need to forcibly transfer the cluster controller role to another server, log on to that server and use the AC CLUSTER /SETCONTROLLER /FORCE command. Doing so will make that server the cluster controller.

My cluster members are connected to a switch, but only one of them is receiving incoming Web traffic.

Let me briefly review how switches work, and why they can break NLB. Normally, a switch accepts incoming traffic and examines the MAC address that the traffic is destined to. The first time the switch sees a MAC address, it sends the traffic out to all of its ports. When a computer replies, the switch remembers the port that the reply came from. Future traffic to the same MAC address will only be sent to that port. The idea is to reduce traffic broadcasts by only sending the traffic to the switch port that contains the recipient.

NLB, however, uses a virtual MAC address to represent all of a cluster’s members. This means the switch’s normal behavior will send all incoming cluster traffic to a randomly selected cluster member. The easiest resolution for this is to connect all cluster members to one switch port, using a hub, or simply connect the members to a hub to begin with. Hubs aren’t as smart as switches, and send all incoming traffic to all hub ports without examining the MAC address. The other solution is to configure the switch ports containing cluster members to be promiscuous, which means all traffic received by the switch will be sent to all cluster members. How you configure your switch (or whether or not you even can configure it) depends on the model of switch you use; consult the switch’s documentation for more details.

Unfortunately, promiscuous operation places a lot of overhead on some less-expensive switches, because they simply aren’t designed to operate at peak efficiency in that mode. If you’re not using a high-quality switch (such as one from Cisco or Foundry), you might achieve better network throughput by simply using a hub.

I’ve tried to permanently remove a cluster member, but Application Center says it couldn’t remove the member’s settings.

This is a pretty common error when you’re removing a cluster member. Application Center is simply unable, for some reason, to remove all of the cluster’s settings from the member. If you plan to format the member and reinstall Windows, you can safely ignore the error messages, because formatting will definitely remove the settings!

On the other hand, if you’d like to make the member a spare Application Center server without reinstalling everything, you’ll need to clean up the prior cluster’s settings. Until you do, you won’t be able to successfully use Application Center on the server again. Fortunately, removing the settings is easy. Just open a command line window and run ac cluster /clean on the recently-removed member. AC is Application Center’s command-line administrative tool, and the /clean switch forcibly removes all cluster configuration information.