Manageability

In the section on availability, we discussed the need for applications that communicate with the system administrators when bad things happen. Reporting was yet another item that helps with availability. Not too surprisingly, availability and manageability items are closely related. To make your Web Service more manageable, you will have to design how reporting will work and specify what tools will be purchased or created so that administrators can find out about server health. In this section, we will look at a number of items that will make your Web Service easier to manage. In particular we will look at the following items:

• Using performance monitor— Performance monitor can track counters and send alerts when the counters start reaching “warning” levels.

• Deployment packages— If the Web Service has to be installed, you can save yourself a lot of time by making installations easy for the installer.

• Reporting— If you are auditing the Web Service, reports will help you extract the information into a more meaningful dataset.

This section does not cover all the management items that you will need to do a good job. For example, you need to make sure that the right people are in charge of your servers. Get the operations people the training they need to understand what you are trying to do with a Web Service. A Web Service is like running an RPC server and a Web site. My experience indicates that the best candidates will have experience running larger Web sites. You can teach the administrator the transactional, Web Service stuff pretty easily. Now, let's look at the items that you can control as a developer.

Performance Monitor

The performance monitor and related counters can be a big help in monitoring the Web Service and the hardware on which it runs. Using these counters can give you a good picture of the health of the system. Decide what “acceptable” looks like for your system. For example, you may decide that a sustained CPU load of 80 percent indicates that you need to add servers. By monitoring the page file use, you will know when it is time to add more physical memory to a machine. To give you a start in identifying counters to watch, the following is a list of some that ought to be applicable most of the time, including a brief explanation of why the counter is useful. The list is enumerated as [Object Name].[Counter Name].

• Processor.% ProcessorTime This counter reports back how busy the processor has been. If the processor use stays too high for too long, odds are pretty good that the server is getting overloaded. For example, if the CPU sits at a sustained 80 percent and above use, you may need a faster machine or more machines to handle the request load.

• Memory.Available Mbytes The memory object actually exposes three counters of interest—Available MBytes, Available KBytes, and Available Bytes. Each counter is just a change in how tight you need to be. This counter shows how much physical memory is available on the computer. If this number starts getting close to 0, say only 10MB left of physical memory, you should look at increasing the amount of RAM in the machine. If the Web Service is memory bound, you may need more machines to handle the request load.

• ASP.NET.Requests Queued This states how many requests are waiting to be processed. If the queue grows too long, more machines need to be added to the Web Farm.

• ASP.NET.Request Wait Time States how long the most recent request was waiting in the queue. If this number gets to be too long (you should decide what is acceptable), take a look at the application and see what functions might need further optimization work.

• ASP.NET.Requests Rejected If the queue grows to a point that requests are being turned away, you should look at either speeding the Web Service up or at adding more machines.

• ASP.NET Applications.Requests/sec This counter indicates how many requests are executed per second. Monitoring this value will tell you how accurate your test plan was. When this value drifts above the threshold you tested for, you will want to pay attention to how well the Web Service is handling the additional, unplanned load. Make sure that the Web Service can handle the added stress. Note: there are counters for each application running on the server. Make sure you monitor the correct instance of the ASP.NET Applications object. You can monitor the _Total object when that makes the most sense.

Besides these items, you may find that other counters make sense. For example, if you have a large number of objects that are not being picked up by the garbage collector, you will want to look at the counters exposed by the .NET CLR Memory object. When using this one, look at the aspnet_wp instance; it contains data pertaining to your ASP.NET Web Service. Take some time to familiarize yourself with the other objects added by Visual Studio .NET and see if they give you any extra information. If you cannot find what you need, you may even want to consider providing your own objects and counters. To view these counters, follow these simple steps:

After you select the objects that you want to watch, you can save that selection for later use. Just choose File, Save, and then save the selection. When you want to use that profile later, find the file in Windows Explorer and double-click it. Everything will come up just as you saved it.

Deployment Packages

Whenever possible, script the installation procedures for the administrator. By scripting, I do not mean writing out a long list of instructions that must be followed.

Instead, go to the trouble of writing a Microsoft Installer package (MSI). These packages can run other batch files, executables, or Windows Scripting Host compatible scripts (VBScript or JScript). Through script, you can create virtual directories, install or modify databases, or any number of things. By providing an MSI, you make the administrator's job a lot easier and improve his or her chances of success.

How long does it take to put together an installation script? That all depends on how much you need to set up and how complicated the installation is. After you create this package, you will have something that you can give to your testers as well as your server administrators. This also guarantees that everyone installs the Web Service the same way, and that no crucial steps get skipped.

For manageability, the MSI makes it easy for the administrator to set up new boxes as the need arises. Need an installation on the other side of town or around the world? No problem. Just make sure the administrator of the Web Service box has access to the MSI and you are set.

Reporting

The operations staff will need a way to regularly check on the health on the Web Service. For one Web Service deployment, the operations staff I worked with used a combination of reports generated by some Web site monitoring software and our own custom reports to monitor the health and usage of the Web Service. The packaged software reported on which endpoints were getting called and how frequently that was happening. The software also generated an analysis of when the site had the highest traffic. A lot of packages exist that can do basic Web Site monitoring. These packages do not come cheap, but they are worth the money because they help prevent a large number of expensive problems.

You will not be able to escape writing custom reports. Find out what information the systems operations staff needs to know, as well as the information you need to know as a developer. If you plan ahead, you can easily instrument your Web Service to report how much time it takes to service a request, as well as the status of the request. Knowing who your heaviest users are will allow you to talk to them and find out what features the Web Service is lacking. Finally, custom reports can deliver extra information regarding the health of the system.