8.8. Next Steps
Before we move on to Chapter 9, which discusses characterizing workloads, nailing down test mixes, and more, it's important to set the stage in terms of basic input, processing, and output requirements—a kind of Data Processing 101 refresher. That is, let's step back and look at the big picture again relevant to testing before jumping back into the details of testing and performance tuning.
8.8.1. Input—Test Mixes and Workloads
An essential component of performing any kind of process regards input. Input in the context of stress testing includes the customer-specific data necessary to complete the “data entry” portion of SAPGUI or other frontend screens, and the master and other transactional data behind the scenes necessary to actually execute a business transaction. Thus, things like organization and customer numbers, material numbers, storage locations, plant codes, and the like represent key input parameters. Other things like user accounts, appropriate authorizations, other security parameters, and so on also represent input. So too does the number or range of items, like the number of materials ordered in an order entry transaction, or the range of purchase orders reviewed in a “PO display” transaction.
Of course, stress testing can be performed “beneath” the application layer. Disk subsystem testing might require batch input files, configuration or parameter files that indicate the mix of reads to writes, for example, and other similar data necessary to execute a test. All of these input data may then be combined into different collections of test mixes: readonly test mixes that do not make any net incremental changes to a mySAP database, mixes focused on exercising a disk subsystem's throughput capabilities, mixes designed to emulate daily loads or peak transactional loads, and so on.
Different randomization or pseudorandom algorithms for selecting the input to be used by a particular virtual user in a particular test run also represent critical stress-test input. This input, or workload, is critical to emulate as closely as possible and to repeat across different tests in as identical a manner as possible. Controlling when a user commences a test run, the type and number of business transactions he or she runs, and for how long are also important input parameters. The subject of input alone is so important to effective stress testing and performance tuning that it warrants its own chapter—Chapter 9.
8.8.2. Processing—Execution Checklists and More
Once all of the necessary input has been assembled, and its use has been determined, the actual execution of a stress-test run or TU can commence. The time spent in executing the run is generically referred to as processing, of course, and represents the initial goal of stress testing—to actually simulate a business process or some kind of load on either an entire system or simply a component of a particular SAP Technology Stack. How well this load is emulated—how well it maps to simulating the environment described by the goals of the test at hand—is measured afterwards. Because we need to create a repeatable and predictable load on the system, the use of automated test tools, automated processes, and easytofollow execution checklists is highly encouraged. Such tools help us to ensure that different test runs are, for example, the same in terms of the amount of time they execute, the number and mix of users simulated, the quantity of orders created and processed, and so on. You can read about all of this and more in Chapters 10, 11, and 12.
8.8.3. Output—Monitoring Considerations
Once a test run has executed for an appropriate amount of time or has otherwise completed its objective, its output must be collected, measured, analyzed, and brought together in a meaningful way to illustrate the results of the change that was tested. But remember that monitoring and testrun analysis involves much more than a postrun snapshot of a system's performance. A test run needs to be monitored throughout its execution. More to the point, the performance observed relevant to each layer of the SAP Technology Stack (or the significant layers, in the case of componentspecific or systemslevel testing) needs to be observed, noted, and later analyzed. Output will include the processing load borne by a server's CPUs and RAM, the network load created within the system, the disk queue lengths or total processed I/Os associated with the database, the number of orders or other posttransaction deliverables processed during the period, and so on.
Output also includes the following factors: any errors or warnings; the number of virtual or physical clients that “died” or otherwise became unavailable during the execution of the test run; whether a transaction completed successfully, partially, or at all; and much more. Even seemingly simple things like the stop and start times of a test run, or total executionversusthink time of a particular virtual user, represent key output data. The monitoring and analysis of test runs are covered in depth throughout Chapters 12 and 13.