Framework-Driven (Structured) Test Scripts

Framework-driven testing isolates the AUT from the test scripts. It provides a set of functions in a shared function library. The functions are treated as if they were basic commands of the test tool's programming language. Framework-based test scripts can be programmed independently of the UI.

Framework-driven testing can occur at multiple levels (18):

• Menu/command level— executing simple commands

• Object level— performing actions on specific things

• Task level— taking care of specific, commonly repeated tasks

Developing Framework-Driven Test Scripts

The following hints will help you develop framework-based test scripts (26).

The Archer Group Framework

The Archer Group has spent countless hours refining this methodology based on the needs of multiple test projects and clients over the last five years. Like any software process, it has evolved to accommodate many testing scenarios and has been utilized in real-world situations.

The goal has always been to keep it simple, make it effective, and, above all, make it maintainable. In this chapter and in the next, we will discuss a testing framework designed for use with data-driven testing. The framework scripts can be found at www.phptr/mosley, which supports this book. Although originally developed for use with Rational's Robot product, the templates should be readily adaptable to other test tools.

The three primary focus areas of Archer Group's efforts have been:

We have adopted the term Control Synchronized Data-Driven Testing to refer to this type of automated testing. Why control synchronized? In addition to the data being utilized as inputs to the AUT, control information is added to the front of each data record to specify the where, how, and what-to-expect information. CSDDT specifies

• Where to go in the AUT; navigation to the window, tab, dialog, etc.; the target of the testing action

• What to do when you get there, the action to be performed

• What to expect after the action is performed—whether to expect an error or normal condition

• The data to be used for input or selection criteria

The input test data records that are read by the Main test script should first contain information that provides record-level identity/control or how the record is to be processed after it is read from the input data file. It is very beneficial to have the ability to skip records because they may not work initially, but we don't want to remove them from the input file. For the process of debugging newly added code, it might be necessary to step through the script after it is read in the record (perform a breakpoint after the record is read). This allows you to substitute data dynamically, such as using keyword data substitution (e.g., substitute text DATE with the current date in the format 01/25/2002). Based on this information, the input data record type can be categorized as:

• Good input record; this may or may not cause an expected error.

• Skip this record; record data is bad or incomplete or is being used to comment following records.

• Breakpoint at this record (in Rational Robot, this causes a software breakpoint used for debug purposes).

• Perform special processing such as keyword substitution.

• Terminate the test before end of file (EOF) is reached.

In general, CSDDT directs test script navigation of the AUT, indicating what window/tab, child window, dialog box, etc., is to be activated or selected. The data also specifies what action is to be performed, for example:

Select/Display

Insert/Add

Update/Modify

Delete

Save

OK

Cancel

Apply

Furthermore, the data also describes the expected results, whether or not an error condition is expected, and the type of error. The CSDDT data record format also allows comments to be included, which makes the data records self-documenting.

Last, but not least, the data records include the test data values that will be input to the AUT. CSDDT allows a variable number of input data fields. For example, one record could have 0 data fields (a command to execute only, such as CANCEL), and the next could have 50 or more data fields. The size should be dynamic.

Visualize that each data record to be read and acted upon by the Main test script is one line in a text file. The first seven sections or fields (separated by double quotes and commas, similar to a CSV file) contain the control information, and the remaining fields are the data to be used as input to AUT.

We have separated the general tests into several groups according to test type. Each type corresponds to a unique automated test type and a specific class of automated test script drives. Some of these tests may use strictly record/playback types of test scripts, while the Business Rules Test utilizes the CSDDT approach.

• The GUI Test's purpose is to test Windows/GUI component functionality.

• The properties test tests object properties of all windows, child windows, tabs, and objects.

• The Special Features Test (when needed) tests custom controls, features to be tested that do not readily fall into other categories, and may include special manual testing.

• The Data Base Content/Initialization Test/Program. his may not be a test but instead be utilized to either validate database content for testing or initialize a database with test-specific records.

• The Business Rules Test tests client and server business rules/edits via data input and data-controlled selections.

• Performance and/or Load Test (when needed).