Chapter 10. Test Management and Work Items
This chapter was meant to be a discussion of ClearQuest Test Management (CQTM), but during the writing of the book Rational announced the end of marking of CQTM. We decided instead to focus on the new product for test management that is based on the Jazz foundation, Rational Quality Manager (RQM). This chapter is neither about test management nor about using Rational Quality Manager but rather on the work item objects of Rational Quality Manager, the methods of customizing test work items, and how work items are used in the testing lifecycle.
10.1. What Is Rational Quality Manager?
IBM Rational Quality Manager is a collaborative, Web-based tool that offers comprehensive test planning, test construction, and test asset management throughout the software development lifecycle. Rational Quality Manager is designed as a full replacement for Rational Manual Tester, Rational ClearQuest Test Manager, and Rational TestManager.
Rational Quality Manager is based on the Jazz platform and inherits many characteristics from that platform. One part of the Jazz foundation is work item management, which is the essence of this book.
The Rational Test Lab Manager (RTLM) is a product, as its name implies, aimed at managing the test lab assets, managing and executing requests for test asset provisioning and test execution. RTLM is not a stand-alone product; it is built on top of Rational Quality Manager.
The Jazz-based work items are the main collaboration tool among the team members. For more information on the Jazz platform see http://jazz.net and www.ibm.com/software/rational/jazz/.
10.2. Understanding Test Entities and Work Items
All test objects are created in a project area, which is the base repository entity; currently test elements cannot be shared between projects but can only be copied.
Many of the test artifacts are independent entities in the Quality Manager repository. You can create relationships between these artifacts, or the artifacts can remain completely independent. Usually you create test cases and add them to a test plan, but you can also work with test cases in the Quality Manager outside the context of a test plan.
Figure 10.1 shows the relationships between these test artifacts.
Figure 10.1. Test artifact relationships in Rational Quality Manager
Full descriptions of the test artifacts and their relationships can be found in Rational Quality Manager online-Help > Reference > General reference for Rational Quality Manager > Test artifact relationships.
As mentioned in Chapter 1, “Work Items,” test objects are not always considered as work items. But some test objects have their own lifecycle, data, and user interaction forms and thus conform to our definition of a work item. Let’s examine each of the elements in Figure 10.1 to determine if they conform to that definition.
The Defect and the Requirement are defined in the Jazz foundation as work items. Indeed, these are the same work items that you can find in Rational Team Concert (RTC), and we can all agree that they conform to our definition.
Let’s continue to examine the other test artifacts to see if they conform to our definition of work item. A test case is created, assigned to a user by role, reviewed, implemented, and approved (for execution in a specific release) and thus has a workflow. A test plan has a similar workflow. So test plans and test cases do conform to our definition. Actually, in ClearQuest Test Management (CQTM) these objects are state-based record types just like defects and enhancement requests.
In Rational Quality Manager the test plan and test case objects also have a lifecycle, but they are of a different type from work items like defects, enhancements, and requirements. In the Jazz foundation, test plans and test cases are not considered work items and do not appear in the work item list, although they have all the elements that conform to the work item definition.
Other objects like test execution and test results do not have a workflow and cannot be considered as work items.
In the Rational Quality Manager screen shot in Figure 10.2 you can see in the left bar the Requirements icon to access the requirements work items, and the Defects icon to access the defects work items. Actually both icons will allow you to create any Jazz work item that is permitted in your system. After you have selected Create Defect and the defect form is loaded, you can change the value of the field Type to Task, Requirement, or any other available work item type, and the form will be adjusted to the newly selected type. Also note that the test plan is in state Under Review and the current selected action Approve will transition it to state Approved. The Summary section contains fields (Product, Release, and Description) that the user can select or enter values. These fields can be customized by the Rational Quality Manager administrator; we shall discuss this in section 10.4, “Customization.” Also note that you can create from each test plan section an additional work item; see on the right side of the Summary section the link Work Item: Create; these additional work items are used to assign some of the test plan work to other practitioners. This is also shown in Figures 10.3 and 10.4.
There is, however, a difference in the workflow between a test case and the Jazz work item. A test case is not closed; it can be reused again and again in each product release, as long as the product needs to be tested.
Figure 10.2. Rational Quality Manager test plan interface: user action in the Test Plan Overview section
10.3. Work Items in the Test Process
In the previous section we identified the Rational Quality Manager entities and their relationships. Some of those entities correspond to the definition of a work item. The following procedure describes a general test process and explains how work items are used.
Different organizations may adopt different processes. We deliberately ignore the operational and logistics steps in the test process as they are not the focus of this book.
1. Analyze the requirements.
The first stage in the test process is to analyze the requirements and to establish the test strategy. Requirements can be Jazz work items in Rational Quality Manager or external records in tools such as Rational RequisitePro or Rational Requirements Composer or Rational DOORS.
RequisitePro to Rational Quality Manager Integration
This integration creates links between RequisitePro requirements and Rational Quality Manager work items of type Requirement.
The next step is to create a test plan and in some cases more than one test plan. Each test plan will be linked to the list of requirements. The test plan goal is to ensure that the application is tested and verified to meet this list of requirements.
In Figure 10.3 you can see the Requirements section of the test plan. Note the warning in requirement ID 20; it means that the requirement was changed after it was included in the test plan. Also note the Create Work Item link; it is used to assign the completion of the Requirements section to a team member.
Figure 10.3. Requirements section in a test plan
3. Assign members to complete test plan tasks.
The test plan owner is responsible for filling in all the sections in the test plan. The owner can delegate some of the work by creating work items. For example, to request a team leader to fill in the Test Objectives section, the test plan owner navigates to the Test Objectives section and clicks on Work Item: Create. This will create a Jazz work item of type Task-Quality that includes the task description and a reference to the test plan. The owner will select the team leader name to complete the assignment. The same procedure can be performed on each and every section of the test plan.
Figure 10.4 shows the creation of the work item. Note that a simplified form is opened (not the full work item form) and a default summary test is included.
Figure 10.4. Creating a work item to assign tasks
The Jazz administrator can set rules that will prevent the test plan from transitioning to the Under Review state unless all the relevant Task-Quality work items are completed. In Figure 10.5 see the State Transition Constraints rules; the second rule is marked, meaning that all Task-Quality work items assigned to the test plan must be resolved in order to allow the transition to the next state.
Figure 10.5. Test Artifact State Transition Constraints
The test plan owner can set up a formal review process that is visible at all times to all members of the larger project team. The test plan includes a Formal Review section; within this section the test plan owner can select team members to review or approve the test plan or some sections of the test plan. The owner selects the type of task (Review or Approve) and the assigned user; this will create a Jazz work item of type Task-Review. Note that the work item of type Task-Review is created for both reviewing and approving the test plan. The assigned user can see the tasks in the Personal Tasks view in a dashboard. In Figure 10.6 you can see that two tasks were set, one user selected as reviewer and one as approver.
Figure 10.6. Using a work item for the review process
In the Formal Review section, it is possible to assign a person to a Quality-Task work item (by clicking on the work item link) to complete this section, that is, selecting reviewers. It is not the same as assigning a person to a Review-Task, which mean reviewing the test plan or parts of the test plan.
The Rational Quality Manager administrator can set rules that will prevent the test plan from transitioning to the Approved state unless all the relevant Task-Review work items are completed. In Figure 10.5 see the State Transition Constraints rules; the third rule is marked, meaning that all approvals in the Task-Review work items assigned to the test plan for approval must be resolved (approved) in order to allow the transition to the next state.
5. Create test cases and associate them with the test plan.
The test case contains the information about what needs to be tested, when, how, and by whom. There are additional fields for information such as preconditions, post-conditions, configuration, risk assessment, and possibly other customized fields. The test case is not a Jazz work item although it has similar workflow elements. As in the test plan, the test case contains sections; within each section there are fields with the test case data. The owner can assign other team members to fill in one or more sections of the test case, by creating a work item for each required section.
The test case also includes a formal review-approve process with features similar to those described for the test plan in the previous item. The administrator can set state transition constraints on test cases. In Figure 10.5 the line “Execute approved Test Cases only” is not checked, meaning that the tester can run a test case even before final approval.
Figure 10.7 shows how to create a test case from a test plan; this will associate the test case automatically with the current test plan. Selecting the + sign (next to the Create New Test Case icon) allows adding an existing test case to the test plan.
Figure 10.7. Creating a test case and associating it with the test plan
6. Associate test cases with requirements, and verify coverage.
Test cases are created so that we can test that application features correspond to the requirements definitions. To ensure that all the requirements are covered by test cases, we link each test case to one or more requirements. This allows us to easily identify uncovered requirements. We can also identify test cases that are not related to any requirement and are probably redundant. Figure 10.8 shows the Requirements section of a test case.
Figure 10.8. The Requirements section of a test case
7. Create test scripts and associate them with the test cases.
Test cases are meant to be executed. The artifact that contains the test steps that are executables is the test script. A test script can be manual as provided by Rational Quality Manager, or automatic, which will be created with a dedicated tool such as the Rational Functional Tester or other supported tools.
Automatic test tools such as RFT perform the same operations that a human tester will do, while interacting with the application GUI. It will type text, press on buttons, verify return values, and do these things faster and more reliably.
In Figure 10.9 you can see the Test Scripts section in a test case. You can assign more than one script to a test case, but when executing a test case a single script has to be selected for execution.
Figure 10.9. The Test Scripts section of a test case
8. Create a test suite.
The test team usually organizes the execution test cases in suites. A suite is a test artifact that contains a list of test cases in a specific order and can execute them according to the included instructions. For example, a regression test suite contains all the test cases that ran successfully in a previous release.
9. Execute the tests.
Execution of a manual test requires the tester to perform the instruction on the application under test (AUT) and verify that it behaves correctly.
10. Report defects.
If the testers find an error, they can report a defect immediately, while the test is still in execution, or they can report one or more defects at the end of the test from the test results log as shown in Figure 10.10. The defect is a Jazz work item, and it is linked to the test results, so that other team members working on resolving that defect will have easy access to the test result log.
Automatic tests are usually unattended; the testers examine the results at the end of the test run. They can report a defect and relate it to an error in the test results.
10.4. Customization
There are different types of customization. The Jazz work items are customized via the Rational Team Concert Eclipse client (in RTC 3.0 also with the Web client); we discussed this subject in Chapter 3, “Workflow,” and Chapter 4, “The Data.” But there are the test-specific work items or artifacts—test plan and test case—for which Rational Quality Manager provides the ability to customize certain aspects. In the next sections we shall explain what can be customized in each of the test work items and how this is done.
Figure 10.10. Defect creation from the Execution Results log
10.4.1. Customizing Jazz Work Items
The default Rational Quality Manager process template comes with the following Jazz-based work items:
• Defect
• Requirements
• Task
• Task-Quality
• Task-Review
Defect, Requirements, and Task type work items are well known and used in many systems. Task-Quality is a work item type used to assign some of the test plan and test case creation tasks to the relevant team members. Task-Review is a work item type used to assign the task of formal reviewing of the test plan or test case to the relevant team members.
You can open a project with the Rational Team Concert Eclipse client and use the Process Configuration to customize each work item. You can add fields, modify the workflow, organize the presentations, define roles, and set permissions. This is identical to customizing Rational Team Concert work items as described in Chapter 4, “The Data,” and section 3.6, “Jazz Workflow,” in Chapter 3, “Workflow.”
Also refer to section 1.4.1, “Which Work Item Elements Can Be Customized?,” in Chapter 1, “Work Items.”
10.4.2. Testing Specific Work Items
Rational Quality Manager provides the ability to perform some customization of test plans and test cases. The customizable elements are
• Sections: A section is part of the test plan or test case that has a tab and one or more fields depending on the section type. A customized section has only one rich text multiline field.
• Categories: A category is a collection of fields usually used to categorize the test plan. Each category has list of values that the user can pick from. The category structure and definition for test cases are similar to those of test plans.
The Manage Sections button, on top of the section list, allows you to select the sections that will appear in the test plan or the test case. In the Manage Sections wizard you can remove sections you do not need and add available sections that are not used. You can set the order of the sections. The Manage Sections screen is shown in Figure 10.11.
Figure 10.11. Manage Sections screen of a test case
You can also create a new section by clicking the + sign, which will open a dialog to define the new section. See Figure 10.12 for the Custom Section wizard.
Figure 10.12. Adding a new section to a test plan
After the test plan is saved, the new section is part of the test plan. See Figure 10.13 for a sample new section, Test Ideas, and the rich text field in the section. The new section includes the Work Item: Create capability, to create a new work item (of type Task-Quality) and assign a user as the owner of this section, just like any other section.
You can create additional sections for test plans and for test cases. Also note that marking the check box Show All Sections will change the user interface and all sections will be displayed one after the other on a single page. The user will also be able to access a section by scrolling.
Figure 10.13. A new section, Test Ideas
10.4.2.2. Organizing Information with Categories
Categories are fields that help organize the information in the test plan and the test case. They allow easy viewing of the information needed. Instead of building a rigidly structured test plan hierarchy, categories allow you a more powerful way to view and find data.
The definition of the category types (fields) and the categories (values) of each type can be performed in two ways. The first way is by the administrator via
Admin->system properties
In the interface that opens, select Test Plan Configuration or Test Case Configuration to define and modify the configuration types. See the example of Test Plan Configuration in Figure 10.14.
The second way to enter the Edit Category wizard is from the Summary section of the test plan or the test case. Click on the three bars icon, as indicated in Figure 10.15, to open the category wizard. The capabilities are the same as if you entered from the admin system properties, assuming you have the permission.
Figure 10.14. Test plan categories: administrator setting of category types and their values
Figure 10.15. Editing categories from the test plan
10.4.2.3. Setting Permissions for Customization
Permission to customize the categories and sections is set with the Rational Team Concert Eclipse client. This is configured in the following steps:
1. Open the project area.
2. Select the Process Configuration tab.
3. Expand Project Configurations.
4. Mark the permission.
5. Select the role to allow the permission.
6. Under the Permitted actions tree mark Save Category (server) and Save Category Type (server). The administrator can set the permission of roles to create, edit, or archive category or category type.
In Figure 10.16 you can see that the role Tester has the permission to create and edit categories but does not have the permission to create and edit category types.
Figure 10.16. Category and category type permissions: defining the permitted actions for a role
In a similar way the permission to create and edit sections of a test plan and test case is set with Rational Team Concert Process Configuration, in the Permitted actions under the Rational Quality Manager tree: Save Test Case (server) and Save Test Plan (server). In Figure 10.17 you can see that the role Tester has the permission to create and edit test cases but does not have the permission to create or remove a section of a test case.
10.4.2.4. Testing Execution States
Test execution states are not state machine states that describe a workflow but are the statuses of the test results. As there are more than just pass/fail values for a test, the organization can define and customize the test result states to be incomplete, stopped, and so on.
To customize the execution states select
Admin->system properties
In the form that opens you can select and enable the Execution States, as well as define new states. The Manage Execution States page is shown in Figure 10.18.
Figure 10.17. Test case permissions: defining the permitted actions for a role
Figure 10.18. Rational Quality Manager Execution States
10.5. Summary
This chapter is somewhat different from other chapters because it discusses work items in a different context: in the test management process. We all know that defect management is an inherent process of testing because the majority of defects are created by the testing team. In this chapter we demonstrated how other types of work items are involved in the test process, work items such as requirements, tasks for review, tasks for approval, and tasks for general quality work.
We discussed the characteristics of test plans and test cases and showed that test plans and especially test cases have workflow, data, and presentations that are the elements that define other work items, and therefore we can consider test plans and test cases as work items. We explained how test cases and test plans can be customized with Rational Quality Manager by adding and modifying sections and defining category types and categories. The Jazz administrator can give permission to customize categories and sections by roles and also set some rules that control the test process, including preventing state transition if all tasks are not complete.