Today is the progress meeting where news regarding the testing progress is communicated to the team. While you were thinking how adequately you are prepared to answer testing status questions, your turn comes to present how far the testing efforts have led to, with respect to the execution timeline, and are there any risks interfering with the estimated plan.
Do these questions frustrate you? It is surely the case when concrete testing figures are missing, and your answers are instantly calculated based on the following equation:
This chapter helps you remove the abstract factors when relaying testing information to the stakeholders. This is done by grouping tests and reporting numbers based on them through the acquirement of knowledge in:
- Creating automated and manual test suites
- Executing test suites
- Exploring test suite results and analyzing them
- Creating customized reports
A test suite is a subset of tests grouped together for a specific testing purpose at a specific time. The execution result of this group of tests delivers data and figures around a certain testing attribute significant in its time context. The composing tests can be either manual or automated.
Test suites come in variations depending on the type and size of the system. The following list shows common test suite examples along with their targeted purpose:
- Code-coverage test suites: They are used to measure the extent of code branches covered by a set of tests, where a higher coverage ensures a clearer vision of the system quality. Code coverage percentage is context dependent where a safety-critical system enforces a higher coverage than, for example, a bookstore application.
- Regression test suites: They are run after the modifications affecting the source code or design of the application under test. These test suites are mainly used to ascertain that the previously working features did not get affected by the additions of new application features or changes subsequent to bug fixing.
- Smoke test suites: They contain test cases verifying the high-level functionality of the system. They could also represent the entrance criteria to system acceptance in the testing process after each build.
Test suite development is not a one-time process; on the contrary, it is continuously maintained through the addition of new test cases and data as issues are discovered within the application even after the release.
Inside Test Studio, test suites are labelled as test lists. Their availability offers a great flexibility in assigning the previously developed automated tests to user-defined test suites. The following sections create and execute test lists based on the File Comparer application's features and tests.