How to Approach Business Process Testing

Planning for, executing, and analyzing the results of business process testing is a detail-oriented job, to say the least. And even at its best, it is still phenomenally time-consuming. Historically, much testing has been performed manually. Drawbacks to this approach, as you see in Figure 15.3, are fairly obvious though. For example:

• Each tester recruited or otherwise brought in from the business must be able to devote an adequate amount of time to testing. This in turn takes away from the tester’s other duties within his organization.

• Testers brought in from third-party consulting and integration partners are probably not experts in your particular business processes—relevant industry experience is therefore a must. Further, they are by no means cheap (though this is relative).

• Testers must be trained in the tools and methodology employed by your team for testing, not to mention the method of documenting output and other results such that the data collected is captured and presented in a clear, consistent manner.

• These testers need physical resources—office/lab space or a “war room” to easily facilitate coordinated testing, complete with enough physical client devices (desktops, laptops, PDAs, and any other access devices eventually to be used on the productive system) to actually perform the testing.

• Finally, because these expensive human resources operate in “real time” only, the ability to consistently and repeatedly execute business processes is impacted. That is, testers must be present to execute a test, and they must be lucid and otherwise at the top of their game all day long.

It is because of the drawbacks shown in the preceding list that automated testing tools for SAP have really grown in popularity over the last six or seven years. Tools like SAP CATT, SAP eCATT, AutoTester ONE, Compuware TestPartner, Mercury Interactive QuickTest, and quite a few others solve the dilemmas described in the preceding list in that they

• Require fewer expensive human resources

• Can execute the same tests over and over again without getting bored and making mistakes, or needing a coffee break, and so on

• Can consistently execute a test much more rapidly than their manual-approach counterparts

• Can be easily modified to include/test a set of variants or other values unique to similar test cases (such as testing the creation of a sales order for many different sales groups or distribution centers)

• Can act as a single repository for all test cases and related documentation, thus facilitating management of the testing process and key deliverables—CATT, AutoTester ONE, TestPartner, and QuickTest are more limited in this respect than eCATT, though, as you see in the next section.

Thus, compared to manual testing methods, the investment in automated testing tools to enable script development consistently pays for itself quickly. Even in the case of third-party tools (where license fees might range from tens of thousands of dollars to a hundred thousand dollars and more), the return on investment is still typically rapid—less than a few months.

Third-Party Tools and Other Resources

Although AutoTester’s products were the first to be certified with SAP, Compuware’s TestPartner has the special distinction of being the first (and as of this writing, only) product to be certified to run with eCATT. If you’re interested in TestPartner or eCATT’s interface for third-party products, please check out the Planning CD, especially the PDF published by SAP AG entitled “BC-eCATT Interface Description - Test Tool Integration into SAP eCATT.”

In my own experience, though, I’ve seen a lot of different testing tools outside of SAP-approved tools that have been used to drive business process testing. The most basic of these are simple scripting utilities adept at propelling the SAPGUI or WebGUI simply by virtue of being Windows products. More sophisticated products exist, too, as do custom scripting approaches. Tools or approaches not already mentioned include

• AutoTester Client/Server

• Mercury Interactive WinRunner

• HiddenSoft AutoIT

• Custom Perl scripts

• Custom Visual Test scripts

• Segue SilkPerformer

Most of these represent older offerings, or embody little in the way of SAP-specific capabilities. This is where SAP’s eCATT represents a quantum leap forward, in that it can handily address integration testing within the SAP framework.

SAP eCATT Differentiators

SAP eCATT’s ability to drive business process scripts on newer mySAP.com solutions that leverage the SAP Control Framework, as well as the ability to work through the legacy SAPGUI API and interface to external tools, makes it my favorite automated testing tool of choice for pure business process testing.

To access all of this value, Web Application Server (Web AS) version 6.20 is required at minimum, along with SAPGUI version 6.20, of course. The price is right, however, as eCATT ships “included” with Web AS. Additional key benefits of eCATT include

• Support for remote centralized testing, insofar as being able to drive tests and act as the single test platform for many mySAP products, thereby simplifying data and script management.

• Support for all mySAP.com components running on SAP Basis Release 4.6C or greater; the core testing tool must reside on Web AS 6.20, but it can test a greater suite of SAP products.

• It can be integrated with external testing tools (like those discussed in more detail in Chapter 16) that may already be used by a particular organization or necessary to support testing business processes that touch non-SAP systems.

• Full integration with the Test Workbench.

• Tight integration with transaction ST30, the Performance Analysis Tool.

• It supports the reuse of test data, by storing this data in separate container objects.

• Support for various GUI interfaces, as seen in Figure 15.4.

  Figure 15.4. Consistent with its wide enterprise reach, eCATT supports multiple SAP graphical user interfaces.

  

• The TCD command allows you to test business processes using the proven batch input technology found in earlier releases of CATT.

• The SAPGUI command provides a new flexible test option for transactions that cannot be tested via TCD; precisely, it’s the SAPGUI command that supports the SAP Control Framework discussed previously.

• Older CATT scripts (called test procedures) can be simply migrated to newer eCATT test scripts—note that a bit of script tweaking is still required, though, to reflect changes to RFC destinations (created with transaction SM59) and other details that enable eCATT to run tests across multiple SAP instances.

The first point in the preceding list is especially exciting, in that eCATT allows end-to-end mySAP.com solution testing to be executed and managed from a single Web AS 6.20 or newer instance. Figure 15.5 illustrates this well—in the past, test cases usually had to be built and run from each SAP component. That is, every test system within the BW landscape, CRM landscape, SRM landscape, and so on became the de facto testing platform/repository for its component. This complicated business process testing quite a bit, and forced a lot of duplication of effort when it came to maintaining test cases where business processes crossed over to other mySAP.com components or different enterprise applications altogether.

mySAP.com Landscape Considerations

As I indicated earlier, managing and executing test cases from each and every test instance within a particular mySAP.com component’s system landscape was the norm until the arrival of eCATT. Today, eCATT lets you simplify functional, integration, and regression testing. But eCATT alone does not allow you to minimize the number of servers or instances in your testing environment. As you see in Figure 15.6, the number of development, test, and production platforms in a typical mySAP.com system landscape remains unchanged.

Other SAP initiatives, like Multiple Components, One Database (MCOD), will serve to reduce the number of test or development database servers within a particular testing environment. As you can clearly see in Figure 15.7, all things being equal, the number of SAP instances will still remain unchanged, though—testing and the need for test instances exists regardless of how the database for a particular test system is accommodated.

Other test-related areas of importance are impacted too, and are examined next.

Additional People Considerations

With the kind of automated testing I have discussed thus far, there is a significant potential reduction in human testing resources—people—that can be realized:

• Test execution specialists will be reduced—as you saw earlier in this chapter, the number of testers is reduced when an automated tool approach is taken.

• Fewer test-case developers and maintainers are needed. However, the people tasked with these responsibilities must understand how their core business processes impact and otherwise touch other mySAP.com components. So, although fewer people may be required, they must be highly qualified across the board—in their business area, in testing, and in general.

• The need for testing coordinators is similarly reduced, given the reduction in people needed overall.

However, the number of folks responsible for identifying and fine-tuning the business processes remains pretty consistent regardless of the specific testing approach (manual or automated). Elsewhere in the SAP Technical Support Organization, headcount remains the same, too.

Process Constraints and Issues

Testing includes much more than just testing valid data or combinations of data. Rather, a good testing process also embraces what SAP refers to as negative testing, where test cases are created with invalid data. The goal of these particular cases is to determine how well the system recovers or handles incorrect data, and how well the system communicates this fact back to the end user. For example, test cases should be created and tested where invalid customer and material numbers are fed to sales order processes, to ensure that error handling and general feedback is both present and appropriate.

Sound testing also includes experimenting with additional boundary testing as well as all combinations of client and user interfaces—a comprehensive testing process requires true end-to-end execution and analysis. Thus, if the Java-based SAPGUI will exist in your environment, and a particular version of the classic SAPGUI is required to support long-time SAP R/3 users, testing must be performed that includes both user interfaces.

Other Areas of Impact

Another area impacted by automated testing regards the number of physical desktops (or other client access devices) needed to perform the testing. Because fewer people are required, fewer SAPGUI, WebGUI, and other access devices used to execute these interfaces are needed as well.

In addition, because scripted test cases are inherently easy to duplicate, fewer test runs are required. This is because every test run executes precisely the same steps, in precisely the same order, as every other test. Omissions, errors, timing inconsistencies, and so on are virtually eliminated with automated testing. By reducing the number of test runs required, we reduce the number of billable hours that consultants will charge a client, and free up valuable hardware resources for other tasks.

Finally, as I said previously, a good automated testing approach should not be limited to the tools you currently have on hand. On the contrary, it is always preferable to use a tool that can be “extended” to include third-party testing/scripting tools, to support testing cross-platform, or to allow for testing complex enterprise-wide business processes. As you can see in Figure 15.8, SAP’s eCATT offering is perfectly positioned to address this, too. It allows third-party vendors and software developers to integrate their test tools using the BC-eCATT interface—for more information on certified interfaces like BC-eCATT, see http://www.sap.com/partners/icc/scenarios/technology/.

Applications that cannot otherwise be tested exclusively with eCATT can leverage this extensibility, thereby opening the doors to testing applications written in HTML, Visual Basic, and so on. With eCATT handling the transport of these additional third-party scripts throughout your mySAP.com test landscape (actually, the SAP Change and Transport System performs the actual transports), your centrally managed eCATT scripts can call scripts created by your third-party tools, and then execute them anywhere in the system.