Determining Your Required Level of Disaster Tolerance

After the uptime requirements of each business group are understood, reviewed, rationalized, and finally documented, the SA can begin the task of reviewing these requirements with an eye toward cost. That is, a good SA will not just design a $50 million solution capable of delivering five nines of availability simply because the business told him that’s what they need! Instead, he’ll work with the business and IT to help them understand the costs inherent in such a solution, as he also works toward better understanding high-availability and disaster recovery options that can be implemented up and down the solution stack.

DR and Return on Investment Calculations

Disaster recovery-related TCO and ROI exercises represent valuable methods of comparing various solution approaches. Such an approach boils down to math—each solution must be evaluated in terms of the following costs:

• Infrastructure costs associated with the floor space that will be required (and thus lost from other data center applications), as well as power, cabling, potentially increased network capacity, and more.

• Acquisition/procurement costs of the actual hardware or software solution.

• Other costs related to procuring the solution that might not be as obvious, like factoring in a specific version of a high-availability supporting operating system, database, or specific hardware component upgrades/delta necessary to realize the value the solution presents.

• Costs related to implementing each solution. This often amounts to a combination of SAP Technical Support Organization, or TSO training, plus HA/DR solution-provider consulting hours, and any costs specific to knowledge transfer enabling the customer to implement the solution themselves in the future.

• Support and other steady-state management costs inherent in each HA/DR approach. Some solutions require special enterprise management bolt-ons (that of course add to the cost of the enterprise management utility); other solutions benefit from specific versions of enterprise management and other supporting software.

• A value should be assigned to each HA/DR approach, such that a “cost” can be assigned related to the general complexity of each solution. When all other things are close to equal, the less complex solution is always more appealing.

• Finally, weighing each solution in terms of its unique cost/benefit ratio is advantageous. One solution, for example, might provide 99.95% availability, whereas another might provide 99.9%—both meet a “three nines of availability” requirement, but if all other costs are close to equal, it will probably make sense to go with the solution supporting the higher level of availability.

▸	To learn more about the costs of downtime related to disasters, as well as DR site TCO considerations, see “Disaster Recovery Requirements That Drive TCO,” p. 130 in Chapter 5.

In some cases, “doing the math” clearly supports one alternative over another. More often, though, quite a bit of work needs to go into understanding different vendors’ approaches to solving availability challenges.

Calculating the Cost of Downtime

Another key area that the SA must understand is related to the average cost of downtime, both planned and unplanned. As the graphs in Figure 6.3 show, planned downtime is inherently less expensive to an organization than unplanned downtime. Nonetheless, the numbers start out pretty big, and grow quickly as an organization incurs more and more downtime. Consider the cost of planned downtime at the end of the millennium, for example. At more than $35K per average outage per week, and something like $10K/hour, even the “typical” small ERP organization lost a small mint over the course of the year. Manufacturing organizations lost even more—an hour of downtime alone racked up $28K. And yet even that was a paltry sum compared to banking and similar financial institutions, who might lose $2.5 million per hour of downtime, or brokerages houses, who might lose $6.5 million per hour of downtime. Today, although I have no new figures to share, it seems reasonable to assume that these numbers are probably much higher—an incremental dollar in revenue is certainly harder to capture now than it was a few years ago, and an organization’s customers are even more likely to shop around, thereby weighting the cost of lost customers and lower customer satisfaction more heavily than I’ve ever before experienced.

With dollars like these at stake, the cost of both planned and unplanned downtime needs to be well understood by all of the members of the SAP project’s steering Committee. In particular, the SA needs to understand how availability impacts costs to the company, so that a solution design begins to emerge that addresses these needs. These costs can manifest themselves as lost revenue streams, spoiled/unfit inventory, lost end-user productivity, decreased customer satisfaction, decreased vendor or distributor faith, less-optimized supply chains, increased compensation, damaged reputation, and so on—in each case, for each specific mySAP solution, the costs need to be characterized, understood, and documented.

In the next section, I take a look at how the solution architect can begin educating himself about potential HA/DR solutions, where the goal is simple—to begin eliminating as many non-applicable or substandard approaches as makes sense, so that you have an idea as to what sizing information to share with your hardware and software partners going forward.

▸	Solution design and other “sizing” considerations are covered in depth in the next chapter. See “Overview—The Sizing and Blueprinting Process,” p. 221 in Chapter 7.

Educating the Solution Architect on HA/DR Options

When the solution architect has his arms around the “nines” and therefore understands both basic budget constraints as well as uptime needs, he can then begin researching in earnest the various HA and DR approaches and solutions at his disposal. At this point, I assume that the specific hardware and software vendors upon which the SAP Solution Stack will be formed have not been determined. However, the list has probably been narrowed down to some extent as we worked through technical staffing considerations in Chapter 4, TCO considerations in Chapter 5, and now have begun to understand basic availability drivers.

Based on my own experience designing and supporting SAP solutions that have sat on top of numerous hardware platforms, across five very different operating systems, and leveraging five very different relational database systems, I drafted the following “Top Ten” list for educating oneself on various SAP high-availability and disaster recovery offerings:

• Vendor-supplied general high-availability papers/best-practices documents for SAP.

• Vendor-specific documentation for HA solution offerings for SAP, like the installation PDF for HP’s MC/ServiceGuard for SAP Extension.

• Reading and research via SAP technical journals, the World Wide Web, and other such resources—SAP AG-provided HA and DR documents are a good example.

• Vendor-supplied SAP questionnaires, which generally detail a vendor’s offerings in support of highly available SAP solutions.

• SAP and partner-sponsored SAP-specific technical conferences, user group meetings, seminars, and so on, where material can be collected and relationships with key SAP and partner consulting/technology organizations can be forged.

• Leveraging a “circle of support” that can be constructed by building relationships with other SAP customers who have similar HA requirements, or belong to a specific user group or industry vertical, and so on.

• Formal training or less formal workshops where official training curriculum is available, and a portion of the training focuses on hands-on implementation.

• Collecting and actually walking through vendor-supplied recipes and checklists for highly available SAP installations (such as cluster-specific documents, or installing 9iRAC for SAP, and so on).

• Leveraging the hands-on experience of SAP, and SAP Global Partners, when it comes to designing and actually implementing HA/DR solutions.

• Working in a technical sandbox environment with demo or other such working copies of HA software, or the applicable hardware components.

By taking advantage of resources such as these, and gaining whatever limited hands-on or training exposure he can get, the solution architect will be well-positioned to further rule out solutions that fail to meet the organization’s needs.