Three Types of Business Process Testing

Different goals exist in regard to testing business processes. In the most generic sense, functional testing represents any business-process–specific testing executed to ensure that the business process works. In other words, every applicable option, every drop-down menu selection, every radio button, and so on needs to be tested initially, and then retested after every change.

Integration testing takes functional testing to the next level, ensuring that an organization’s business processes work together with other business processes, both within and across multiple functional areas, such that the SAP system as a whole operates as expected. Thus, much of the work prior to Go-Live falls under the category of integration testing, to the point where it is quite common to maintain a dedicated system within the SAP system landscape for integration testing. Some organizations refer to this system as the QA, or Consolidation, or Test System. Others call it simply Integration; I like the term Test/QA. Regardless, the goal of this core landscape component is to allow for end-to-end mySAP solution testing outside of the primary development system, prior to deploying the business processes to be used productively by end users.

Finally, regression testing allows for quickly proving that a specific set of data and processes yields consistent and repeatable results, even when a subset of that data or process changes. Thus, regression testing is all about testing the integrity of a given business process, regardless of the minutiae related to specific changing business logic/coding and programming practices. And regression testing ensures that currently implemented stable business processes continue to work after changes to other business processes are made.

As you see in Figure 15.2, all three types of business process testing complement one another, and aid in creating and maintaining a functionally useful SAP environment.

In all three cases, business processes are typically scripted into repeatable business cases, and documented to some extent. Functional tests tend to create the most documentation in terms of volume, as every case must be covered. Integration tests are fewer, and therefore drive less raw documentation. Regression tests in my experience tend to be more pointed (very discrete or specific test cases, like creating a sales order for a specific sales organization), such that the documentation requirements are very detailed but leaner per test scenario than in functional testing. Thus, a huge number of specific regression and functional tests are created, perhaps many tens or hundreds of times more than those related to integration testing. Given this relationship, it is nearly always mandated that one set of scripted business scenarios be used for both functional and regression testing, and another set used for end-to-end complex integration scenarios.

When to Execute Business Process Testing

There is no general scheduling rule when it comes to testing the functionality of your mySAP solution, other than to say simply that testing never truly ends. True, much of it is performed after the initial configuration is drafted in the Development environment and subsequently promoted into a QA or Consolidation or Integration environment. But if you ask any ABAP or functional consultant engaged with deploying SAP whether their work ends on the day of Go-Live, you’ll get a quick “not likely.” Why? Because it’s not likely that a particular solution will NOT be further refined, or process extended, or feature added or otherwise modified over time. And preparation for full-fledged SAP upgrades demands even more functional and integration testing, especially if you’re unfortunate enough to have implemented custom routines or user exits. Bottom line, as long as changes are made to the SAP solution, some level of functional, integration, and regression testing will certainly be required as well.

I tend to see functional testing play a significant role in the entire life cycle of an SAP solution, therefore, until the solution is retired. And because the functional testing continues throughout the life cycle, so too must the integration and regression testing. Sure, the quantity of work tends to slow down somewhat after Go-Live. But it picks up just as quickly when a new business group is added to the SAP system, or when a new plant or distribution center is brought online, or a new product line is released. And it picks up noticeably as legacy and other integration points are brought in or changed, or new functional areas are added to the implementation. And significant changes arise from full-scale SAP upgrades, migrations, or corporate mergers, or when a new mySAP.com component is added to the SAP system landscape—these and other such activities potentially touch countless business processes, and therefore beg to be thoroughly tested prior to deploying updated processes to Production.

In addition to the really “big” changes just noted, other less obvious SAP business process changes like the following drive functional, integration, and regression testing:

• Adding or modifying custom reports

• Adding new output mechanisms, from electronic (via workflow and similar methods) to fax machines and printers

• Legal changes, especially those related to tax laws and so on

• System or business-process enhancements, and other similar development activities designed to refine a specific business process

• Various bug fixes and other system-wide updates designed to avoid known or potential issues

• Transitioning processes facilitated by older SAP transaction (like ME21) to newer equivalent transactions (like ME21N) that offer expanded functionality through the use of controls like drop-down list boxes and tree controls.

Again, any change to the system justifies retesting the affected business processes prior to redeploying the updated process to Production. Just how this is accomplished is covered next.

The Critical Nature of Functional Testing

Regardless of the enterprise application or software package, the attention given to functional testing directly impacts whether the end product—a productive SAP solution in this case—is successful or worthwhile. In the increasingly complex world of mySAP.com, this has never been more true. SAP continues to add new functionality to mature products like R/3 and BW, and in the last year alone has added completely new offerings like PLM and the SAP Exchange Infrastructure. Combined with the rest of the mySAP solution offerings and underpinning technology components, the opportunity to tightly integrate business processes across diverse functional and productivity areas has never been better. Similarly, the opportunity to really fail has never been as great, either. So it’s safe to say that the primary roles of functional and integration testing are more critical than ever.

The ASAP methodology and its newer counterparts ValueSAP/Global Roadmap and especially the SAP Solution Manager for Implementation go a long way toward generally guiding an SAP test team in their endeavors. But there are a few key areas that remain truly pivotal:

• The leadership of the team tasked with functional testing

• The general aptitude or ability of the programming and business process staff when it comes to testing

• The specific experience each tester has in their business area of focus, such as enterprise controlling or asset management or procurement and fulfillment

• The specific experience each tester brings to the table in terms of other mySAP.com projects in which he has participated from a testing perspective

Experience is huge, as should be evident in the preceding list. One reason for this is that very little formal education is offered that is centered on testing methodologies or building test cases. The training that SAP programming and functional folks receive most often amounts to the experience that each person gains during their paid SAP projects or engagements! Very few functional or SAP basis courses devote more than a few cursory minutes to functional, integration, or regression testing.

Short of complete failure, what is the risk of inadequately testing a mySAP solution? In the best of cases, it is simply that a much less refined system is ultimately offered up as “production.” Worst case, though, business processes fail, and key areas of customer satisfaction, manufacturing, human resource management, and so on are put at risk. To address this, consultants like myself and even more so my programming colleagues are then pulled into “post-production” support roles. In some cases, my programming colleagues seem to never even leave their client site after Go-Live, transitioning into post-Go-Live roles while our SAP customers continue to pay for further development and more testing of an end-product that in all reality fell short of everyone’s expectations. To net it out, then, ignoring or compressing the testing phase in an SAP project can be phenomenally expensive to everyone, affecting not only the project budget but the business’s ability to execute as well.

The Real Value in Integration Testing

When 95% of the coding and configuration (“programming” in the most general sense) has been completed within each mySAP component, work can begin in earnest testing how all components integrate together to create an enterprise solution. Integration testing must continue to focus inside each mySAP component, too, to ensure that business processes execute as expected, but outside is where most of the work lies. That is, SAP is integrated such that back in the functional testing phase, it was largely validated that SD interfaced well and worked as expected with MM, PP, and FI (for example). But in integration testing, what started as a suite of functional test cases was broadened to include an even larger scope of test cases. Thus, I contend that integration testing will not only always overlap functional testing, but add to it substantially.

During this phase in the project, all team members work together to run through the various scripted and otherwise recorded business processes, noting failures, successes, and unexpected variations or outcomes. Again and again, throughout the testing phase, each team comes together to test their configuration changes. The risks are many:

• In a perfect world, it would be ideal to start integration testing after the entire solution has been completely configured at a component level. Normally, though, the teams are fortunate to be 95% of the way there.

• After each set of failed integration tests, the team must go back to their respective drawing boards and address any shortcomings, variations, and errors in their code. The opportunity to introduce new errors into business processes that have performed flawlessly up to this point is possible, however.

• The teams must be kept in check against the project scope. It’s very easy for an ambitious developer to begin adding nice-to-haves or otherwise extra features. This feature-creep only adds complexity, though. And it further complicates integration testing as the Go-Live deadline looms closer and closer.

The next section covers business process testing that most often takes place after Go-Live, although a good bit of it can still occur prior to that big day—regression testing.

Regression Testing

Testing how well a business process works after a change to the system is performed is the goal of regression testing. This helps to ensure that your changes (or the addition of new functionality, which in itself represents a change, too) do not have a negative effect on business processes already in production or earmarked to be released to production soon. In my experience, I tend to see regression testing activities spike most during migration testing or when a new mySAP.com component is added to and integrated with an organization’s enterprise. And it’s not uncommon prior to Go-Live, too, when an organization is going through dreaded scope changes or succumbing to other alterations to the master plan. Overall, though, the bulk of regression testing occurs at the following times:

• After Go-Live, when new functionality is added to the currently implemented SAP components or products

• After Go-Live, when new components or products are integrated with the SAP enterprise

• After Go-Live, during the course of normal system maintenance after applying support packages, legal changes, and other patches/fixes

• After Go-Live, in support of planned migrations to new versions or releases of mySAP components

This type of testing, like functional and integration testing, takes place on the SAP instance implemented for testing—Test/QA, Integration, Consolidation, or whatever you call it in your environment. Keep in mind, of course, that the actual changes are made in Development, and then promoted to Test/QA. After these changes are tested and proven here, they are eventually promoted from Development to the Production instance.

Functional Versus Stress Testing

Whereas functional and other such testing is geared toward ensuring that a business process works, stress testing or load testing ensures that a business process works quickly. That is, functional testing focuses on the correctness of implementation, and stress testing assumes this correctness in order to focus on how the system behaves during daily and high-load periods. In the real world, after the bulk of the configuration work has been locked down, more attention is paid to functional and integration testing than is ever paid to load testing. A few months before Go-Live, however, a comprehensive systems and stress test should be executed. I take a closer look at this level of testing in the next chapter.