Some of the reasons why you would want to achieve software quality are to make sure that the system does what it's supposed to do; uncover errors and/or to provide assurance for your software user. This chapter will explore some general terminologies and processes in software testing to shed light over some concepts used in this book, and briefly introduce the tool automation features that are covered in the next chapters.
If you are already familiar with the following testing concepts, you can jump to Test Studio uncovered in the later chapters of this book.
The following is a conceptual overview of some fundamental testing terminologies and principles. These are used in day-to-day testing activities and will be directly referred to in the chapters when explaining the business case for our examples.
A test case is a scenario that will be executed by the tester or by an automation tool, such as the Test Studio for any of the software testing purposes, such as uncovering potential errors in the system. It contains:
- Test case identifier: This identifier uniquely distinguishes a test case.
- Priority: The priority holds a value to indicate the importance of a test case so that the most important ones are executed first and so on.
- Preconditions: The preconditions describe the initial application state in which the test case is to be executed. It includes actions that need to be completed before starting the execution of the test case, such as performing certain configurations on the application, or other details about the application's state that are found relevant.
- Procedure: The procedure of a test case is the set of steps that the tester or automated testing tool needs to follow.
- Expected behavior: It is important to set an expected behavior resulting from the procedure. How else would you verify the functionality you are testing? The expected behavior of a test case is specified before running a test, and it describes a logical and friendly response to your input from the system. When you compare the actual response of the system to the preset expected behavior, you determine whether the test case was a success or a failure.
When executing a test case, you would add at least one field to your test case description. It is called the actual behavior and it logs the response of the system to the procedure. If the actual behavior deviates from the expected behavior, an incident report is created. This incident report is further analyzed and in case a flaw is identified in the system, a fix is provided to solve the issue. The information that an incident report would include are the details of the test case in addition to the actual behavior that describes the anomalous events. The following example demonstrates the basic fields found in a sample incident report. It describes a transaction carried out at a bank's ATM:
- Incident report identifier: ATM-398
- Preconditions: User account balance is $1000
- Procedure: It includes the following steps:
- User inserts a card.
- User enters the pin.
- Attempts to withdraw a sum of $500.
- Expected behavior: Operation is allowed
- Actual behavior: Operation is rejected, insufficient funds in account!
- Procedure results: Fail
The following definition appears in the ISTQB (International Software Testing Qualification Board) glossary:
"The set of generic and specific conditions, agreed upon with the stakeholders, for permitting a process to be officially completed. The purpose of exit criteria is to prevent a task from being considered completed when there are still outstanding parts of the task, which have not been finished. Exit criteria are used to report against and to plan when to stop testing. [After Gilb and Graham]"
Software testing is governed by a set of principles out of which we list the pesticide paradox. The following definition appears in the ISTQB glossary:
If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new defects. To overcome this, "pesticide paradox", the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.
Element recognition is a pillar of automated test execution as the tool used can't perform an action on an object unless it recognizes it and knows how to find it. Element identification is important in making the automated scripts less fragile during execution. This topic will be reflected in this book.