Identifying the Elements of a Business Continuity Plan

A complete Business Continuity Plan consists of several components that handle not only the continuation of critical business functions, but also all the functions and resources that support those critical functions.

Emergency response

Emergency response teams must be identified for every possible type of disaster. These response teams need written procedures and checklists to keep critical business functions operating.

Written procedures are vital for two reasons. First, the people who perform critical functions after a disaster may not be familiar with them: They may not usually perform those functions. (During a disaster, the people who ordinarily perform the function may be unavailable.) Second, the team probably needs to use different procedures and processes for performing the critical functions during a disaster than they would under normal conditions. Also, the circumstances surrounding a disaster might have people feeling out-of-sorts; having a written procedure guides them into action (kind of like the “break glass” instructions on some fire alarms, in case you forget what to do).

Damage assessment

When a disaster strikes, experts need to be called in to inspect the premises and determine the extent of the damage. Typically, you need experts who can assess building damage, as well as damage to any special equipment and machinery.

Depending on the nature of the disaster, you may have to perform damage assessment in stages. A first assessment may involve a quick walkthrough to look for obvious damage, followed by a more time-consuming and detailed assessment to look for problems that you don’t see right away.

Damage assessments determine whether an organization can still use buildings and equipment, whether they can use those items after some repairs, or whether they must abandon those items altogether.

Personnel safety

In any kind of disaster, the safety of personnel is the highest priority, ahead of buildings, equipment, computers, backup tapes, and so on. Personnel safety is critical not only because of the intrinsic value of human life, but also because people — not physical assets — make the business run.

Personnel notification

The Business Continuity Plan must have some provisions for notifying all affected personnel that a disaster has occurred. An organization needs to establish multiple methods for notifying key business-continuity personnel in case public communications infrastructures are interrupted.

Not all disasters are obvious: A fire or broken water main is a local event, not a regional one. And in an event such as a tornado or flood, employees who live even a few miles away may not know the condition of the business. Consequently, the organization needs a plan for communicating with employees, no matter what the situation.

Throughout a disaster and the recovery from it, management must be given regular status reports as well as updates on crucial tactical issues so that management can align resources to support critical business operations that function on a contingency basis. For instance, a manager of a corporate Facilities department can loan equipment that critical departments need so that they can keep functioning.

Backups and off-site storage

Things go wrong with hardware and software, resulting in wrecked or unreachable data. When it’s gone, it’s gone! Thus IT departments everywhere make copies of their critical data on tapes, removable discs, or external storage systems.

These backups must be performed regularly, usually once per day. The backup media must also be stored off-site in the event that the facility housing the original systems is damaged. Having backup tapes in the data center may be convenient for doing a quick data restore but of little value if backup tapes are destroyed along with their respective systems.

For systems with large amounts of data, that data must be well understood in order to determine what kinds of backups need to be performed (full, differential, and incremental) and how frequently. Consider these factors:

The time that it takes to perform backups

The effort required to restore data

The procedures for restoring data from backups, compared with other methods for recovering the data

For example, consider whether you can restore application software from backup tapes more quickly than by installing them from their release media (the original CD-ROMs or downloaded install files). Just make sure you can recover your configuration settings if you re-install software from release media. Also, if a large part of the database is static, do you really need to back it all up every day?

You must choose off-site storage of backup media and other materials (documentation, and so on) carefully. Factors to consider include survivability of the off-site storage facility, as well as the distance from the off-site facility to the data center, airports, and alternate processing sites. The facility needs to be close enough so that media retrieval doesn’t take too long (how long depends on the organization’s recovery needs), but not so close that the facility becomes involved in the same natural disaster as the business.

Remote backup services are a viable alternative to off-site backup media storage. Broadband Internet access makes it possible to back up critical data to a backup service provider — often faster than magnetic tapes can be returned from an off-site facility and data recovered from them.

Some organizations have one or more databases so large that the organizations literally can’t (or, at any rate, don’t) back them up to tape. Instead, they keep one or more replicated copies of their databases on other computers in other cities. BCP planners need to consider this possibility when developing continuity plans.

The purpose of off-site media storage is to ensure that up-to-date data is available in the event that systems in the primary data center are damaged.

Software escrow agreements

Your organization should consider software escrow agreements (wherein the software vendor sends a copy of its software code to a third-party escrow organization for safekeeping) with the software vendors whose applications support critical business functions. In the event that an insurmountable disaster (which could include bankruptcy) strikes the software vendor, your organization must consider all options for the continued maintenance of those critical applications, including in-house support.

External communications

The Corporate Communications, External Affairs, and (if applicable) Investor Relations departments should all have plans in place for communicating the facts about a disaster to the press, customers, and public. You need contingency plans for these functions if you want the organization to continue communicating to the outside world. Open communication during a disaster is vital so that customers, suppliers, and investors don’t panic (which they might do if they don’t know the true extent of the disaster).

The emergency communications plan needs to take into account the possibility that some corporate facilities or personnel may be unavailable. Thus you need to keep even the data and procedures related to the communications plan safe so that they’re available in any situation.

Utilities

Data-processing facilities that support time-critical business functions must keep running in the event of a power failure. Although every situation is different, the principle remains the same: The BCP team must determine for what period of time the data-processing facility must be able to continue operating without utility power. A power engineer can find out the length of typical (we don’t want to say routine) power outages in your area and crunch the numbers to arrive at the mean time of outages. By using that information, as well as an inventory of the data center’s equipment and environmental equipment, you can determine whether the organization needs an uninterruptible power supply (UPS) alone, or a UPS and an electric generator.

A business can use uninterruptible power supplies (UPSs) and emergency electric generators to provide electric power during prolonged power outages. A UPS is also good for a controlled shutdown, if the organization is better off having their systems powered off during a disaster.

In a really long power outage (more than a day or two), it is also essential to have a plan for the replenishment of generator fuel.

Logistics and supplies

The BCP team needs to study every aspect of critical functions that must be made to continue in a disaster. Every resource that’s needed to sustain the critical operation must be identified and then considered against every possible disaster scenario to determine what special plans must be made. For instance, if a business operation relies upon a just-in-time shipment of materials for its operation and an earthquake has closed the region’s only highway (or airport or sea/lake port), then alternative means for acquiring those materials must be determined in advance. Or, perhaps an emergency ration of those materials needs to be stockpiled so that the business function can continue uninterrupted.

Fire and water protection

Many natural disasters disrupt public utilities, including water supplies or delivery. In the event that a disaster has interrupted water delivery, new problems arise. Your facility may not be allowed to operate without the means for fighting a fire, should one occur.

In many places, businesses could be ordered to close if they can’t prove that they can effectively fight a fire using other means, such as FM-200 inert gas. Then again, if water supplies have been interrupted, you have other issues to contend with, such as drinking water and water for restrooms. Without water, you’re hosed!

We discuss fire protection in more detail in Chapter 13.

Documentation

Any critical business function must be able to continue operating after a disaster strikes. And to make sure you can sustain operations, you need to make available all relevant documentation for every piece of equipment, as well as every critical process and procedure that the organization performs in a given location.

Don’t be lulled into taking for granted the emerging trend of hardware and software products that don’t come with any documentation. Many vendors deliver their documentation only over the Internet, or they charge extra for a hard copy. But many types of disasters may disrupt Internet communications, thereby leaving an operation high and dry with no instructions for how to use and manage tools or applications.

At least one set of hard copy (or CD-ROM soft copy) documentation (including your Business Continuity Plan and Disaster Recovery Plan) should be stored at the same off-site storage facility that stores the organization’s backup tapes. It would also be smart to issue soft copies of BCP and DRP documentation to all relevant personnel on USB storage devices (with encryption if needed).

Continuity and recovery documentation must exist in hard copy in the event that it’s unavailable via electronic means.

Data processing continuity planning

Data processing facilities are so vital to businesses today that a lot of emphasis is placed on them. Generally this comes down to these variables: where and how the business will continue to sustain its data processing functions.

Because data centers are so expensive and time-consuming to build, better business sense dictates having an alternate processing site available. The types of sites are

Cold site: A cold site is basically an empty computer room with environmental facilities (UPS; heating, ventilation, and air conditioning [HVAC]; and so on) but no computing equipment. This is the least-costly option, but more time is required to assume a workload because computers need to be brought in from somewhere and set up, and data and applications need to be loaded. Connectivity to other locations also needs to be installed.

Warm site: A warm site is basically a cold site, but with computers and communications links already in place. In order to take over production operations, you must load the computers with application software and business data.

Hot site: Indisputably the most expensive option, you equip a hot site with the same computers as the production system, with application changes, operating system changes, and even patches kept in sync with their live production-system counterparts. You even keep business data up-to-date at the hot site by using some sort of mirroring or transaction replication. Because the organization trains its staff in how to operate the organization’s business applications (and staff members have documentation), the operations staff knows what to do to take over data processing operations at a moment’s notice.

Reciprocal site: Your organization and another organization sign a reciprocal agreement in which you both pledge the availability of your organization’s data center in the event of a disaster. Back in the day, when data centers were rare, many organizations made this sort of arrangement, but it’s fallen out of favor in recent years.

Multiple data centers: Larger organizations can consider the option of running daily operations out of two or more regional data centers that are hundreds (or more) of miles apart. The advantage of this arrangement is that the organization doesn’t have to make arrangements with outside vendors for hot/warm/cold sites, and the organization’s staff is already onsite and familiar with business and computer operations.

A hot site provides the most rapid recovery capability, but it also costs the most because of the effort required to maintain its readiness.

Table 11-1 compares these options side by side.