13.4. Collecting and Cleaning Real-World Data
In the last few pages, we have seen how to produce or find the data necessary to quantify the performance of a system. Now we must bring together all of these data pulled from different sources both during and after a test run, and clean them in a way that makes for meaningful configuration changes and improved performance tuning.
13.4.1. Collection Preparation
If you've embraced my scripted approaches to collecting real-time data during a test run and to saving the results to an output log file, you'll be in good shape overall. Worst case, the output files simply will need to be delimited in some manner (if you failed to do this in scripting for output) so that the contents may then be poured into Excel. Depending on what you collected, too, you might want to build additional spaces before or after certain fields in your output data so that, for example, your columns line up nicely in Excel. Over the years, I've learned to keep my output logs pretty consistent in this regard, but I remember well what my first few pivot tables looked like, and the work necessary to clean up the input files behind them.
You might need additional preparation, though, prior to collecting historical data from CCMS. For example, in Basis releases prior to Web AS 6.20, where you're still using ST03 to analyze your workload and have a just-executed test run to analyze, you'll need to “build totals” first. From the main screen in ST03, simply click the Build Totals button, and select whether you prefer to conduct this operation in the foreground (real time) or background (through a batch process that will immediately execute without any further input). Once the operation has completed, refresh the ST03 display and you're in business.
For those with newer Basis releases or for those of you who prefer ST03N, a similar process is used—underneath “Today's workload” simply double-click “Total” as I've shown in Figure 13-18. A box entitled “Total data for today not available” will be displayed, like the one in Figure 13-19. Select Background or Dialog, and once the totals have been calculated, they will be available for display.
Figure 13-18. ST03N has no Build Totals button like ST03. Instead, double-click Total underneath “Today's workload” and follow the prompts.
Figure 13-19. To aggregate the most recent performance totals from CCMS, select either the Background or Dialog button, and the updated results will be quickly collected and displayed.
13.4.2. Loading SAPGUI Screen Output into Excel
Capturing SAPGUI screen contents and saving them out to a file does not always require a tool capable of SAP API–aware screen-scraping. Some transactions, like the Workload screen in STAD, give the end user the ability to natively save the screen's contents to a number of file formats simply by clicking the Download button—unconverted (text), spreadsheet, rich text format, and HTML format are all at your disposal instantly. But one of the coolest things about the SAPGUI is that these four formats are actually at your disposal whenever you need to save list-based output locally, pretty much regardless of what kind of list you happen to be looking at—ST07, ST10, AL08, and anything else that generates a list can be manipulated in this way. To save screen-based output to a local file or mapped network share, simply enter the transaction %pc in the same place you type in a T-code like /nAL08, and then press the Enter key. The same window you see in STAD after pressing the Download button (see Figure 13-20) pops up, and defaults to saving the screen's contents in unconverted file format. Choose the spreadsheet format (or any of the other three if you prefer, though XLS is surprisingly the most compact format, not to mention often the most useful). Then press Enter, browse to the desired directory path, type in the name of the output file you would like to create, and press the Save button to save the data to the file name you specified. It couldn't be much easier.
Figure 13-20. Use the %pc transaction to save a SAPGUI's screen-based output to a spreadsheet, text file, and more.
13.4.3. Cleaning Up the Output Logs
Before the data found in an output log are pulled into Excel, they need to be cleaned up in what amounts to a two-phased approach. First, the data are visually analyzed for obvious outliers—for data that appear incorrect, duplicate data (often the result of variables that failed to be reinitialized), and so on. I don't automatically toss all of these data, of course. I analyze them and in many cases correlate them to data points collected at totally different layers in the SAP Technology Stack, to determine with more certainty what stays and what goes. I keep copies of all of the original output logs, too, because you never know what you might need to review again one day soon. After this first phase of data cleansing, the data must be further cleaned. I'll explain this second phase in the next section using a fish analogy.
13.4.4. Chopping Off the Head and the Tail
I've never been a very good fisherman. Maybe it's because I'm not a morning person, and my grumbling, if I manage to get up that early at all, sends the fish off in search of a quieter spot. Or maybe it's because I just can't seem to get too excited about catching and eating something that takes hours to find and pull up, when it's so readily available at one of the corner grocery stores for less money than I'd pay in lost bait and Diet Cokes. Maybe it's because it's just fish. Give me a license to take down some Angus beef! Now that's something to get excited about. A large prime rib, an enormous porterhouse, a couple of New York strips—they're worth my time.
No matter, I get my share of fishing when it comes to cleansing data in preparation for pouring it all into Excel. Specifically, the performance data and other information found in OS logs, inline data collection output logs, CCMS postrun analysis screens, and so on all need to be pared down to reflect the steady-state or level-testing time period. As depicted in Figure 13-21, the raw data associated with startup and ramp-up, and then again with ramp-down and stopping, must be removed. Otherwise, the imprecise nature and timing associated with ramping up and down thousands of clients would affect the accuracy of any analysis.
Figure 13-21. Here, chopping off the head and the tail of a test run in terms of the total collected output data is clearly illustrated.
The only part of a test run that should be analyzed (unless you're expressly testing for something else, like how long it actually takes to log in 1,000 users) is the body of the run—the period of time after ramp-up has completed and before ramp-down has commenced. It's during this time that all users are logged in, all workloads are at their peak, and everything is generally running smoothly. In a nutshell, the system is performing the real work of the stress test during this time. It is this precise time period that is to be measured and compared against identical time periods (e.g., 30 minutes) from other test runs.
13.4.5. Dumping Other Output into Excel
With the raw data cleaned up, the bad data eliminated, and everything generally good to go, now is the time to import all remaining output and other performance data into Excel. If you were on top of things during scripting, you may have already set up many of your scripted CCMS monitoring transactions or inline data collection output-file routines for the XLS format in the first place. I typically create a single workbook with which to stage and measure all of my test runs for a particular stress test. If I'm running three very different kinds of tests, I'll create three different worksheets within my single Excel workbook, though, and stage multiple runs so that the same data occupy the same columns, thereby facilitating analysis. Note that because each worksheet gets its own tab (label), I take this opportunity to double-click the tab and rename the worksheet to something meaningful.
If you have some text-based output data that need to be imported into Excel, simply open or create your Excel test-run workbook, navigate to an empty (or appropriate, if you've used it before) worksheet, and click on the cell that will represent the upper left-hand corner of the location of the data you wish to import. Then from the tool bar, click on “Data” and then select the option “Import external data” followed by “Import data.” Navigate through the file system until you find your data source, and then double-click it. The initial screen of the Text Import Wizard opens up, as shown in Figure 13-22. Ensure you set the “Delimited” option (assuming this is true), and then press Next.
Figure 13-22. Be sure to select the proper file type that best describes your data, taking care to be as consistent as possible when importing multiple input files.
Screen two of the Text Import Wizard opens up, where you need to ensure that the appropriate delimiter is selected (I usually go with comma delimited, though space-and tab-delimited output can be useful as well—it all depends on how you separated your output data's fields, and whether some of those fields already contain commas). Click the Next button again, and the final Text Import Wizard screen is displayed, as shown in Figure 13-23. Here, you have a final opportunity to manipulate data columns, tweak formats, identify the decimal and thousands separators (by clicking the Advanced button—very useful in case the default German values were never changed in your SAP test system!), and so on. Click Finish when you're ready, take the default in response to the question “Where do you want to put the data,” and press OK. Your text-based output log will be poured into Excel, ready for further analysis.
Figure 13-23. Once you've indicated the proper column data format and addressed any “advanced” items, select Finish to initiate conversion of your test run's raw output data into Excel.
