Internet Service Provider Workloads

The Internet provides a challenge for managing computer systems. Instead of a small population of known users who can connect to a server system, millions of users can connect to any server that is on the Internet. External synchronizing events can cause a tidal wave of users to arrive at the server at the same time, as shown in the following examples.

• Sports-related web sites in the USA get a peak load during key games in the “March madness” college basketball season. In general, any television or news media event may advertise a URL to which users can connect, and cause a spike in activity.

• In countries where local phone calls are charged by the minute, the onset of cheap rate calls at 6:00 p.m. each day causes a big spike in the number of dial-in connections at an ISP, creating a major load on the local servers at that time.

• Proxy caching web servers sit between a large number of users and the Internet and funnel all activity through the cache. They are used in corporate intranets and at ISPs. When all the users are active at once, regardless of where they are connecting to, these proxy cache servers get very busy.

On the other hand, many web sites get little or no activity most of the time. It is too expensive to dedicate a single computer system to each web site, so a single system may be configured to respond to hundreds or thousands of internet addresses. This is sometimes known as virtual web hosting. From a user perspective it is not possible to tell that accesses to the different web sites are going to the same physical system. If one of the virtual web sites becomes very busy, the performance of accesses to all the other virtual sites can be affected, and resource management tools are needed to help maintain a fair quality of service for all the sites.

This section looks at three case studies: a proxy web cache workload, virtual web hosting, and a content provider site.

Proxy Web Cache Workload

A caching web server acts as an invisible intermediary between a client browser and the servers that provide content. It cuts down on overall network traffic and provides administrative control over web traffic routing. Performance requirements are quite different from those for a regular web server. After discussing the issues, we'll look at several metrics that must be collected and analyzed to determine the workload mix and the management of resources.

We start by remembering the caching principle of temporal and spacial locality. In the case of a web cache, cacheable objects are always separate and are always read in their entirety with no prefetching. The proxy web cache mostly works by using temporal locality. If cached items are read more than once by different users in a reasonably short time interval, the cache will work well. If every user reads completely different pages, the cache will just get in the way. If one user rereads the same pages, that browser will tend to cache the pages on the client. So the proxy server cache won't be used effectively.

Not all web content is cacheable. Some of the busiest traffic is dynamic by nature, and caching it would prevent the browser from seeing an update.

Cache Effectiveness

Caches commonly have a hit rate of about 20 to 30 percent, with only 60 percent of the data being cacheable. That figure does not take into account the size of each access—just the total number of accesses. Each cache transaction takes some time to complete and adds significant latency to the connection time. In other words, the cache slows down end users significantly, and only a small proportion of the data read is supplied from the cache. All the users go through the cache regardless of their final destination. This funnel effect can increase network load at a local level because accesses to local web servers will normally go via the cache, causing two trips over the local network rather than one.

Direct web service, as shown in FIGURE 4-21, allows the browser to contact web servers directly whenever possible. This contact may involve a firewall proxy that does not provide caching to get out to the public Internet.

In contrast, when a proxy is introduced, as shown in FIGURE 4-22, all the browsers connect to the proxy by default. An exception can be made for named systems and domains that can be accessed more efficiently by a direct connection.

Caching for Administrative Control

The reduction in wide-area network traffic from a 20 to 30 percent hit rate is worth having, especially since wide-area links are expensive and may be saturated at peak times.

The real reason to set up a proxy cache intranet infrastructure is the administrative control. Security is a big problem. One approach is to set up the firewall gateways to the Internet so that they route web traffic only to and from the proxy caches. The proxy caches can make intelligent routing decisions. For example, Sun has connections to the Internet in Europe and the USA. If a user in Europe accesses an Internet web site in Europe, that access is routed via the local connection. When an access is made to a site in the USA, a choice can be made. The route could go to the USA over Sun's private network to the USA-based gateway, or it could go directly to the Internet in Europe and find its own way to the USA. The second option reduces the load on expensive private transatlantic links, but the first option can be used as a fallback if the European connection goes down.

Restricted routing also forces every end user who wants to get out to the Internet to do so via a proxy cache. The cache makes routing and filtering decisions and logs every access with the URL and the client IP address (which is usually sufficient to identify the end user). If the corporate policy is “Internet access is provided for business use only during business hours,” then employees who clog up the networks with non-business traffic can be identified. It is probably sufficient to make it known to the employees that their activity is being logged and that they can be traced. Posting an analysis of the most visited sites during working hours for everyone to look at would probably have a sobering effect. Filtering can be added to deny access to popular but unwelcome sites.

The point is that you have a limited amount of expensive wide-area network capacity. There is a strong tendency for end users to consume all the capacity that you provide. Using gentle peer pressure to keep the usage productive seems reasonable and can radically improve the performance of your network for real business use.

Solaris Bandwidth Manager software can be used effectively in this environment to implement policies based on the URL, domain or protocol mix that is desired. For example, if traffic to an Internet stock quote server is crowding out more important traffic in the wide area links of a company intranet, Solaris Bandwidth Manager software can throttle access to that site to a specified percentage of the total. The combination of proxy cache controls and bandwidth management is a powerful and flexible way to optimize use of wide area network and Internet gateway bandwidth.

Clustered Proxy Cache Architectures

The Apache and Netscape™ proxy caches are extensions of a conventional web server. They are used singly, and although they can be used in a hierarchy of caches, there is no optimization between the caches. An alternative is a clustered cache. The Harvest Cache was the original development and is now a commercial product. The Squid cache is a freely available spin-off and is the basis for some commercial products. Squid is used at the biggest cache installations, caching traffic at the country level for very large Internet service providers.

Clustered caches use an intercache protocol (ICP) to talk among themselves and form an explicit hierarchy of siblings and parents. If the load would overwhelm a single machine or if high availability is important, multiple systems are configured as siblings. Each sibling stores data in its cache but also uses the ICP to search the caches of other siblings. The net result: the effective cache size is that of all the siblings combined, the hit rate is improved, and it doesn't matter which sibling a client visits. The parent-child relationships also form a more efficient hierarchy because the ICP-based connections are much more efficient than individual HTTP transfers. ICP connections are kept open, and transfers have lower latency. When using bandwidth management in a clustered cache architecture, give the inter-cache protocol high priority so that the response time for cache queries does not get affected when the caches are generating a lot of network traffic.

Virtual Web Hosting

A huge number of people and businesses have registered their own personal domain names and want the domain to be active on the internet with a home page. In many cases, there is little activity on these names, but they are assigned a fixed IP address, and somewhere a web server must respond to that address. In the Solaris operating environment, it is possible to configure more than one IP address on a single network interface. The default limit is 256 virtual addresses per interface, but it has tested up to 8000 addresses per interface and could go higher. Use the ifconfig command to assign IP addresses to an interface and specify virtual interfaces with a trailing colon and number on the interface name. Use the ndd command to query or set the maximum number of addresses per interface. In the example below, two addresses are configured on hme0, and the default of up to 256 addresses is set.

% ifconfig -a
lo0: flags=849<UP,LOOPBACK,RUNNING,MULTICAST> mtu 8232
        inet 127.0.0.1 netmask ff000000
hme0: flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST> mtu 1500
        inet 129.136.134.27 netmask ffffff00 broadcast 129.136.134.255
hme0:1: flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST> mtu 1500
        inet 129.136.134.26 netmask ffffff00 broadcast 129.136.134.255
hme1: flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST> mtu 1500
        inet 129.136.23.11 netmask ffffff00 broadcast 129.136.23.255

# /usr/sbin/ndd -get /dev/ip ip_addrs_per_if
256

The real limit to the number of virtual addresses is the time it takes to run ifconfig command on all of them. When thousands of virtual addresses are configured, the reboot time is affected, which can be a significant problem in a high availability failover scenario. The Solaris 2.6 operating environment was tuned to speed up this process.

Web servers can be configured in several ways to manage many virtual addresses. The most efficient way is to use a feature of the web server so that a single instance of the server listens to many addresses and responds with the appropriate home page. With a multithreaded web server, one process can support many addresses. With web servers like Apache that use a separate process for each connection, more processes are spawned for the addresses that are most active at any point.

Administrative problems may rule out this approach. It may be necessary to run a completely separate web instance of the web server for each address. This way the web server runs using a distinct userid. The administrator of the web site can log in as that user to configure their own site. The web server and any search engine or cgi-bin scripts that it starts all run as that user so that accounting, performance monitoring and resource management tools can manage each virtual web site's activity separately. It is common for cgi-bin scripts to have errors and be resource intensive. They can get stuck in infinite loops and consume an entire CPU. With thousands of customer web sites setup on a single system, the central administrator cannot control the quality of the custom cgi-bin code that is set up on each site.

In the end, this workload is similar in many ways to an old-fashioned central timeshare system. Resource management using the SRM software to control the CPU usage and the number of processes and logins on a per-user basis is extremely effective. The Solaris Bandwidth Manager software can also be used effectively, as there is an explicit administrative mapping from each network address to the userid that is running that web server. Users can be allocated shares of the CPU and network resources according to their need, or can pay more for a larger share and be confident that they will get a better quality of service if the server saturates.

Managing Web Servers with SRM Software

The SRM software manages resources on web servers by controlling the amount of CPU and virtual memory. Three basic topologies are used on systems hosting web servers.

Resource Managing a Consolidated Web Server

A web server can be managed by controlling the resources it can use. In an environment where a web server is being consolidated with other workloads, this basic form of resource management prevents other workloads from affecting the performance of the web server, and vice versa.

In FIGURE 4-23, the web server is allocated 20 shares, which means that it is guaranteed at least 20 percent of the processor resources should the database place excessive demands on the processor. In addition, if a cgi-bin process in the web server runs out of control with a memory leak, the entire system will not run out of swap space; only the web server will be affected.

Managing Resources within a Single Web Server

It is often necessary to use resource management to control the behavior within a single web server. For example, a single web server may be shared among many users, each with their own CGI-BIN programs. An error in one CGI-BIN program could cause the entire web server to run slowly or, in the case of a memory leak, could even bring down the web server. To prevent this from happening, use the per-process limits.

Resource Management of Multiple Virtual Web Servers

Single machines often host multiple virtual web servers. In such cases, there are multiple instances of the httpd web server process, and far greater opportunity exists to exploit resource control using the SRM software.

You can run each web server as a different UNIX userid by setting a parameter in the web server configuration file. This effectively attaches each web server to a different lnode in the SRM hierarchy. For example, the Solaris Web Server has the following parameter, as shown in FIGURE 4-25, in the configuration file, /etc/http/httpd.conf

By configuring each web server to run as a different UNIX userid, you can set different limits on each web server. This is particularly useful for control and for accounting for resource usage on a machine hosting many web servers. You can make use of most or all of the SRM resource controls and limits: