Server Congestion

An important scaling problem for the Internet is server congestion. The antidote is a technique referred to as server caching, which is the replication of content at geographically dispersed places. Instead of millions of users converging on a single server, content of a popular server is replicated and accessed by users elsewhere. Figure 8-1 illustrates the concept of server caching, with the original cache being replicated via the Internet and the Access Network. The user accesses the server cache through the Access Network.

The following list details the basic requirements of server caching:

• The original server and the caching server have the same content.

• The process of synchronizing the databases doesn't generate excessive traffic.

• The consumer reaches the nearest server without having to explicitly identify the server.

• Cache hits are counted reliably, just as if the hit had occurred at the original server,

A number of approaches can be used with server caching. The first is to have the original source periodically multicast updates to the replicating servers. This requires IP multicast in the network and creates the potential problem of updating servers unnecessarily. Even servers that service relatively few requesters would receive the periodic updates.

A second approach is to have the local router forward requests to the nearest server cache, as illustrated by Path 1 in Figure 8-1. If an updated record exists, it is returned to the user. If no match exists, the request for information is forwarded to the original server via Path 2 in Figure 8-1, which answers the request. In so doing, the replicating server notices the response and caches it. The cached entry is aged in a time appropriate for the nature of the content, the request rate for the page, and the cache server resources. All requests in the interim are answered by the replicating server until the record is removed. The entry then is refreshed, possibly with a modified record after the next cache miss. This technique enables the replicating server to keep a minimum amount of information, which minimizes backbone congestion.

One problem with either approach is that the hit to the cache may not be counted as if the hit were made directly to the original server. If the transaction occurs at the cache, the originating server may be unaware of the transaction. This is important to advertisers who want to be sure they have the highest count of viewers.

An approach to solving this has been devised by Akamai Networks . (Akamai means cool in Hawaiian.) Akamai caches the highest bit rate content, normally video streams and complex still images. The textual, low-bit-rate parts of a Web page are retrieved from the original server. This technique permits an accurate count of page hits, while at the same time the high-bit-rate images are offloaded to the caches.

Caching high-bit-rate content separately from low-bit-rate content represents the type of subtle challenge to system integrators in scaling the Internet.