Chapter 6. Selecting Your Testing Tools
With so many tools, utilities, and comprehensive application testing suites on the market today, it can be a pretty daunting task to sift through everything out there just to identify the handful of core tool sets you'll use in the name of SAP performance management. It's the goal of this chapter, therefore, to identify many of the alternatives, and then help you narrow down your list of options. I share the experiences my colleagues and I have gained over the last few years, and detail what I consider to be best-of-breed products for many of the core SAP Technology Stack layers. So sit back and enjoy—now that we have the fundamentals behind us, along with the hard work of nailing down requirements and staffing the T3, we can finally begin the fun of piloting and ultimately selecting the best suite of products useful in stress testing and performance tuning your SAP environment.
It's been said that end users are the final measure of success when it comes to testing and tuning. Although I agree with this statement in principle, waiting for an up-down nod of the head from your users after every implemented change is neither cost effective nor practical. Instead, I like to think of meeting my success criteria as the affirmative nod, and my test tools as my users—the tools will confirm how successful the testing is. The key is to use the right tools, though. Thus, throughout this chapter we will work our way up the SAP Technology Stack, identifying the pros and cons of various tool sets, utilities, test suite applications, and so on. And, for consistency, I'll note whether a tool falls into the three categories previously described:
Level One: tools that tend to be hardware- or software-specific, making it possible to characterize individual subsystems within the technology stack
Level Two: the standard SAP benchmark kit for a particular SAP product or component
Level Three: testing end-to-end customer-specific business processes and data, leveraging a somewhat customized method and any number of different SAP-aware testing suites
Finally, I'll identify whether a particular tool best facilitates delta testing (measuring the difference in performance between two alternatives), load testing (testing an expected daily load), stress or volume testing (reflecting peaks beyond the expected daily load, like those associated with quarter-end), and/or smoke testing (testing a system to the point at which a subsystem is saturated, thereby identifying the bottleneck that holds back greater throughput or better response times).
The term test tool implies a tool used in the active testing of a particular solution or solution component. As such, true test tools are not necessarily the same tools useful in verifying your solution's configuration or collecting performance data—these latter tools are what I call “monitoring tools” and will be discussed in Chapter 7. Instead, test tools do the work of stressing your system. They are the software utilities that emulate users, place a particular load on a hardware component, or in some other way push the system. In either case, though, how can you possibly know which tools to use? Outside of making a decision based on the various claims of test-tool vendors, or enlisting the assistance of a third-party expert (whether an internal IT colleague or external consultant), you'll be forced to make your choice based on some kind of research. In the name of due diligence, I actually prefer this latter method. But to do so intelligently, you will need a good test-tool evaluation process that goes beyond simple Internet-based research or installing a demonstration and evaluation version of the tool: you need a real test-tool piloting process. In the next section, I'll share the process I've used time and again, ideal when it comes to proving that a particular tool is up to the testing/tuning task at hand.