The Disaster Recovery Plan

The disaster recovery plan serves as the “backup plan” to the business continuity plan. The BCP addresses issues that help prevent disasters from interrupting your business. The DRP, on the other hand, kicks into effect when your business is actually interrupted by an emergency situation.

This section examines three major aspects of the DRP—selection of a recovery facility, recovery of operational data, and restoration of normal operations. The disaster planning team should write the DRP with the intent of providing a handbook that guides personnel through the entire recovery process until operations resume under predisaster conditions.

Recovery Facilities

Selection of an alternative data processing facility is arguably the most important part of the disaster recovery plan. Your DRP must provide at least one alternative computing facility that takes over operations should your present facility become unavailable.

There are several options for recovery facilities:

• Hot sites. Offer a fully redundant, ready-to-go data center. Larger organizations sometimes maintain their own hot sites at a geographically distant company office, but the majority of hot site users outsource site maintenance to an outside contractor. These sites maintain all the facilities you need to restore operations almost instantaneously, including fully mirrored copies of your critical business data.

• Warm sites. Offer a cost-effective compromise. These shared facilities host all the hardware needed to restore your operations in a functional environment. However, the software configurations and data are stored on magnetic media, ready for fast setup. Under normal conditions, warm sites take at least 24 hours to assume data processing responsibilities. When selecting a warm site vendor, be certain to investigate their resources carefully. Vendors usually depend on assumptions that all of their clients will not need the facilities simultaneously. Do they have the resources in place to handle a major regional disaster? If insufficient resources are available, how will use of the remaining resources be prioritized?

• Cold sites. Are environmentally controlled facilities with sufficient power to support data center operations. However, cold sites do not maintain hardware or software in a ready-to-use environment. The vendor may allow you to store equipment onsite, but actual installation and configuration do not take place until after you declare an emergency. Expect restoration of operations at these sites to take at least several days and possibly weeks if you have difficulty locating the necessary hardware and software to ramp up operations.

• Reciprocal Assistance Agreements. Formalize “I'll watch your back if you watch mine” arrangements between businesses. RAAs are legal contracts or informal arrangements that obligate two companies to render assistance to each other in the event of a disaster. RAAs vary drastically in scope from maintaining mirrored servers in each other's data centers to simply reserving floor space and power/network resources for emergency operations. Carefully investigate any potential RAA partners before entering into a relationship. You're granting them a large degree of trust when you place your data in their hands. Also consider their geographic location—how likely is it that the disasters high on your prioritized BCP list would impact both locations? If the same hurricane knocks out your data center and that of your RAA partner, you're both up the creek without a paddle.

  WARNING

  Beware of RAAs Be extremely careful when considering signing an RAA with another organization, especially if you plan to maintain mirrored servers or data backups in each other's data center. Consult with upper management and ensure that they're aware that you'll be placing your company's critical data in the hands of another organization. Management may feel that RAAs simply create too great a vulnerability to industrial espionage or acts of sabotage.

  
  

• Improvised Facilities. Are the lowest level of recovery facility. Under this type of arrangement, a business plans to construct a new data center from scratch after an emergency. Space may already be secured but no infrastructure is in place to handle day-to-day operations. Improvised facilities are an extremely poor recovery option for two reasons. First, it could take weeks to restore normal operations after an emergency. Second, it is next to impossible to test an improvised facilities arrangement during periodic DRP drills. You should consider improvised facilities only if you have no budget for a backup facility and can tolerate significant downtime after a disaster.

Data Recovery

The second leg of the DRP triad is data recovery. After you've established operations in a recovery facility, it's time to restore your business-critical data.

Your data recovery options are limited by the choices you made while designing the backup strategies in your business continuity plan (see the “Data Backup” section of this chapter) and the recovery facility chosen in the disaster recovery plan (see the previous section on “Recovery Facilities”). If you were fortunate enough to implement a hot site recovery facility with mirrored servers, you wouldn't have much data recovery work if everything went according to plan. Simply verify that your mirrored servers are functioning properly and you're back in business!

If you need to restore data from tape or other magnetic media,make sure that your DRP makes provisions to have the appropriate restoration equipment on hand in the recovery facility. There's nothing worse than not having the proper tape drive when you need to restore data. Also, if you have a large amount of data to restore and limited restoration equipment, you'll need to draw up a prioritized list of what gets restored first. Restoration of massive amounts of data can take a significant amount of time. Plan accordingly!

Restoration of Normal Operations

The work of the disaster recovery team doesn't end when the business is up and running at an alternative location. In most cases, the business wants to eventually restore normal operations at the original facility. The disaster recovery team must be prepared to facilitate this transition (and, therefore, the DRP must address restoration issues). Restoration may either be the responsibility of the entire disaster recovery team (undertaken after the situation is under control) or there may be a subset of the team dedicated to restoration operations (beginning immediately after a disaster strikes). The selection of one of these manpower allocation options depends upon the urgency of returning to a permanent worksite.

There are three main issues that must be addressed as part of the restoration operation:

• Salvage equipment from the original data center. Depending upon the type of disaster that affected operations, a good deal of the equipment in the permanent data center may require repair. The restoration team should identify pieces of equipment that are still in working order and equipment that can be repaired in a cost- and time-efficient manner.

• Restore the data center to working condition. This step involves everything from cleaning the physical facilities to bringing up the servers. You'll need to locate replacements for any critical machines that are beyond repair.

• Transition from the recovery site back to the permanent data center. The transition is a complex process—it's actually quite similar to the disaster itself except you have the luxury of advance knowledge. You need to ensure that operations are not down longer than necessary and that any transactions processed at the recovery site are reflected at the permanent site.

The DRP Document

As with the BCP, there is no set format for the disaster recovery plan. Keep in mind that the DRP is a dynamic document that changes on a regular basis as you revise your disaster recovery strategies. It's probably a good idea to keep it in a loose-leaf binder in a location accessible to all employees involved in the plan.