“Test Plan Driven” Method Architecture

• Driver Script

  • Performs initialization (if required)

  • Calls the Application-Specific “Controller” Script, passing to it the file-names of the Test Cases (which have been saved from the spreadsheets as “tab-delimited” files)

• The “Controller” Script

  • Reads and processes the file-name received from Driver

  • Matches on “Key Words” contained in the input-file

  • Builds a parameter-list from the records that follow

  • Calls “Utility” scripts associated with the “Key Words,” passing the created parameter-list

• Utility Scripts

  • Process input parameter-list received from the “Controller” script

  • Perform specific tasks (e.g., press a key or button, enter data, verify data, etc.), calling “User Defined Functions” if required

  • Report any errors to a Test Report for the test case

  • Return to “Controller” script

• User Defined Functions

  • General and Application-Specific functions may be called by any of the above script-types in order to perform specific tasks (4)

Using TestDirector to Drive the Test Suite

If TestDirector is being used to drive the Test Suite, then the architecture changes somewhat, as TestDirector does not have the capacity to pass a parameter to a called script. Consequently, TestDirector cannot be used to call the Controller script. We handle this by creating a Test Case Driver script to act as an intermediary between Test Director and the Controller. The Test Case driver is called by Test Director, and it subsequently calls the Controller script passing to it the required tab-delimited file name. Note that this means that there must be one Test Case Driver per Test Case. The actual code contained within these is exactly the same, but each must have the name of the Test Case.

TestDirector

• Calls Initialization script if required

• Calls the “Test Case Driver” script for each Test Case

Test Case Driver

• Performs initialization (calls Initialization scripts) if required

• Calls the application-specific “Controller” script, passing to it the file-names of the Test Cases (which have been saved from the spreadsheet as “tab-delimited” files)

From this point, the architecture is the same as for using a Driver script to drive the Test Suite. Refer to Figure 7.2

The following is an example TSL driver script outline.

Having filled out the spreadsheet as in Table 7.2, the following would then be done:

When the Driver script is executed, the following would be done:

• Call Controller script, passing Test Case name: ret_code = call “Controller”(“TC0001A”)

• Controller:

  • Opens file TC0001A.txt and reads each record

  • On “Test_Description” writes the Test Description to the Test Report

  • On “Start_Test”, calls Application-Specific Utility script Start_Up (Screen~Main Screen)

  • Start_Up:

    • Writes data to the Test Report re: start of test, time, date, etc.

    • Verifies we are positioned at the Main Screen

    • Returns to Controller

• Controller:

  • On “Test_Step” writes the Test Step Number and Description to the Test Report

  • On “Enter”, calls Generic Utility script Enter (Selection~1)

  • Enter:

    • Verifies field “Selection” exists

    • Enters “1” in Selection

    • Returns to Controller

• Controller:

  • On “Action”, calls Generic Utility script Action (Press_Key~Enter)

  • Action:

    • Verifies “Enter” is a valid key name

    • Calls WinRunner function to press the “Enter” key

    • Returns to Controller

• Controller:

  • On “Verify”, calls Generic Utility script Verify (Screen~Payment Screen)

  • Verify:

    • Calls WinRunner function to set the Screen (logical name) in the GUI_file

    • Note: this verifies that the Screen is displayed and active

    • Returns to Controller

• Controller:

  • On “Test_Step” writes the Test Step Number and Description to the Test Report

  • On “Enter”, calls Utility script Enter (Payment Amount~125.87|Payment Method~Check)

  • Enter:

    • Verifies field “Payment Amount” exists

    • Enters “125.87” in Payment Amount

    • Verifies field “Payment Method” exists

    • Enters “Check” in Payment Method

    • Returns to Controller

• Controller:

  • On “Action”, calls Utility script Action (Press_Key~F9)

  • Action:

    • Verifies “F9” is a valid key name

    • Calls WinRunner function to press the “F9” key

    • Returns to Controller

• Controller:

  • On “Verify”, calls Utility script Verify (Screen~Payment Screen)

  • Verify:

    • Calls WinRunner function to set the Screen (logical name) in the GUI_file

    • Note: In this step, we are verifying that the screen has not changed

    • Returns to Controller

• Controller:

  • On “Test_Step” writes the Test Step Number and Description to the Test Report

  • On “Verify_Data”, calls Utility script Ver_data (Payment Amount~$ 125.87|Current Balance ~$1,309.77|Status Message~Payment processed.)

  • Ver_data:

    • Verifies field “Payment Amount” exists

    • Extracts the contents of Payment Amount & compares with data entered

    • If data doesn't match, reports an error-msg to the Test Report showing Expected versus Actual results

    • Verifies field “Current_Balance” exists

    • Extracts the contents of Current_Balance & compares with data entered

    • If data doesn't match, reports an error-msg to the Test Report

    • Verifies field “Status Message” exists

    • Extracts the contents of Status Message & compares with data entered

    • If data doesn't match, reports an error-msg to the Test Report

    • Returns to Controller

• Controller:

  • On “End_Test”, calls the Application-Specific Utility script End_Test (Press_Key~F3| Screen~Main Screen)

  • End_Test:

    • Performs End-Of-Test functions, and includes some of the same parameters as the Action and Verify Utilities. Will ordinarily contain additional functions as required by the Application Under Test.

    • Verifies “F3” is a valid key-name

    • Calls WinRunner function to press the “F3” key

    • Calls WinRunner function to set the Screen (logical name) in the GUI_file

    • Note: In this step, we must minimally verify that we have returned to the Starting Position (“Main Screen”).

    • Returns to Controller

• Controller:

  • Performs End-Of-Test functions:

  • Verifies that additional GUI_files opened during the test are closed

  • Verifies that we have returned to the Starting Position (“Main Screen”) in the event that the End_Test routine was not run (due to a fatal error encountered).

  • Calls an application-specific “ReturnToBaseState()” function to ensure we are at the Starting Position.

  • Returns to Driver

The Driver script tests the Return Code from the Controller and writes a “pass/fail” message to the Summary Report. It also determines what test to run next, and calls the Controller script, passing the next Test Case name to it. The Driver continues to the tests until there are no more to run except if an error forces the routine to end.

The different types of keyword files roughly correspond to the Excel tables that Nagle's DDE used to drive testing (see Tables 7.3, 7.4, and 7.5).

Zambelich's tool kit also provides a set of “generic” functions used by the controller script. They include functions in three Toolkit Function libraries (Main_TK1, Main_TK2, and Main_TK3). These compiled modules are extracted from “ToolKit.zip” and are then installed in WinRunner's Function Generator category ATS ToolKit Functions by script AUT_INIT in the APP_Init folder. The utility scripts fall into the categories below.

Table 7.3. Test Administration Keywords

Utility Keyword	Client/Server Usage	Web Usage	Note
START_TEST:	Start_Up	Start_Up (Application-Specific)
TEST_DESCRIPTION:	N/A - See 'Start_Test'
TEST_STEP:	N/A - See 'Start_Test'
END_TEST:	End_Test (Application-Specific)

Table 7.4. Data Manipulation Keywords

Utility Keyword	(Gen_Util)	(Web_Util)	Note
ACTION:	Action	Web_Action
DO_IF: / ELSE:
SELECT_LIST_ITEM:	Sel_List
ENTER:	Enter	Web_Enter
UPDATE_IF_NULL:	Enter
APPEND:	——
EXTRACT_DATA:	Sav_Data	Web_Save_Data
EXTRACT_TABLE_DATA:	——	Web_Save_Table
GENERATE_DATA:	Gen_Data
GENERATE_INPUT:
CALCULATE:	Calculate (Gen_Util)
CALCULATE_DATE:	Gen_Data	Calc_Date (Gen_Util)
SET_CONDITION:	Set_Cond	Web_Set_Cond (Gen_Util)
SET_DATE:	Set_Date	Web_Set_Date

Table 7.5. Verification Keywords

Utility Keyword	Client/Server Usage	Web Usage	Note
MATCH_DATA:	Ver_Data	Web_Verify_Data
VERIFY_DATA:
VERIFY_TEXT:	Ver_Text	Web_Verify_Text
VERIFY_TEXT_LINKS:	——	Verify_Text_Links
VERIFY_IMAGE	Ver_Bitmap	Web_Verify_Image
VERIFY_ATTRIBUTES:	Ver_Attr	Web_Verify_Prop
VERIFY_ENABLED:	Ver_Enbl	Web_Verify_Enbl?
VERIFY_VALUE:	Ver_Value	Web_Ver_Value
VERIFY_NUMERIC_VALUE:	Ver_Value	Ver_Value (Gen_Util)
VERIFY:	Verify	Web_Verify
VERIFY_LIST_ITEMS:	Ver_List (Gen_Util)
VERIFY_TREEVIEW_ITEM:	Ver_Tree	—-

The Just Enough Software Test Automation FTP site (www.phptr/mosley) contains documentation explaining Zambelich's data-driven automation framework. He has provided an overview document that fully describes the Functions and Utility scripts and his slide presentation, as well as some excellent example spreadsheets (see Figure 7.3).

Automated Testing Specialists offers the toolkit and an associated consuting program for a fee. If you are using WinRunner and want to obtain the tookkit and implement this framework, you should contact Keith Zambelich directly (at Automated Testing Specialists, Inc., P.O. Box 65564, Los Angeles, CA 90065, email: [email protected]) to obtain the TooKit for WinRunner.