Practices listed in this section are focused on the best way to write tests. Write the simplest code to pass the test as it ensures cleaner and clearer design and avoids unnecessary features.
The idea is that the simpler the implementation, the better and easier it is to maintain the product. The idea adheres to the keep it simple, stupid (KISS) principle. This states that most systems work best if they are kept simple rather than made complex; therefore, simplicity should be a key goal in design, and unnecessary complexity should be avoided. Write assertions first, act later as it clarifies the purpose of the requirements and tests early.
Once the assertion is written, the purpose of the test is clear and the developer can concentrate on the code that will accomplish that assertion and, later on, on the actual implementation. Minimize assertions in each test as it avoids assertion roulette; it allows the execution of more asserts.
If multiple assertions are used within one test method, it might be hard to tell which of them caused a test failure. This is especially common when tests are executed as part of the CI process. If the problem cannot be reproduced on a developer's machine (as may be the case if the problem is caused by environmental issues), fixing the problem may be difficult and time consuming.
When one assert fails, execution of that test method stops. If there are other asserts in that method, they will not be run and information that can be used in debugging is lost.
Last but not least, having multiple asserts creates confusion about the objective of the test.
This practice does not mean that there should always be only one assert per test method. If there are other asserts that test the same logical condition or unit of functionality, they can be used within the same method.
Let's go through a few examples:
@Test
public final void whenOneNumberIsUsedThenReturnValueIsThatSameNumber() {
Assert.assertEquals(3, StringCalculator.add("3"));
}
@Test
public final void whenTwoNumbersAreUsedThenReturnValueIsTheirSum() {
Assert.assertEquals(3+6, StringCalculator.add("3,6"));
}
The preceding code contains two specifications that clearly define what the objective of the tests is. By reading the method names and looking at the assert, there should be clarity on what is being tested. Consider the following example:
@Test
public final void whenNegativeNumbersAreUsedThenRuntimeExceptionIsThrown() {
RuntimeException exception = null;
try {
StringCalculator.add("3,-6,15,-18,46,33");
} catch (RuntimeException e) {
exception = e;
}
Assert.assertNotNull("Exception was not thrown", exception);
Assert.assertEquals("Negatives not allowed: [-6, -18]",
exception.getMessage());
}
This specification has more than one assert, but they are testing the same logical unit of functionality. The first assert is confirming that the exception exists, and the second that its message is correct. When multiple asserts are used in one test method, they should all contain messages that explain the failure. This way, debugging the failed assert is easier. In the case of one assert per test method, messages are welcome but not necessary, since it should be clear from the method name what the objective of the test is:
@Test
public final void whenAddIsUsedThenItWorks() {
Assert.assertEquals(0, StringCalculator.add(""));
Assert.assertEquals(3, StringCalculator.add("3"));
Assert.assertEquals(3+6, StringCalculator.add("3,6"));
Assert.assertEquals(3+6+15+18+46+33,
StringCalculator.add("3,6,15,18,46,33"));
Assert.assertEquals(3+6+15, StringCalculator.add("3,6n15"));
Assert.assertEquals(3+6+15,
StringCalculator.add("//;n3;6;15")); Assert.assertEquals(3+1000+6,
StringCalculator.add("3,1000,1001,6,1234"));
}
This test has many asserts. It is unclear what the functionality is, and if one of them fails, it is not known whether the rest would work or not. It might be hard to understand the failure when this test is executed through some CI tools.
Benefits: The tests work in any order independently, whether all or only a subset is run.
Each test should be independent of the others. Developers should be able to execute any individual test, a set of tests, or all of them. Often, due to the test runner's design, there is no guarantee that tests will be executed in any particular order. If there are dependencies between tests, they might easily be broken with the introduction of new ones.
Benefits: These tests are used often.
If it takes a lot of time to run tests, developers will stop using them or run only a small subset related to the changes they are making. The benefit of fast tests, besides fostering their usage, is quick feedback. The sooner the problem is detected, the easier it is to fix it. Knowledge about the code that produced the problem is still fresh. If the developer already started working on the next feature while waiting for the completion of the execution of the tests, they might decide to postpone fixing the problem until that new feature is developed. On the other hand, if they drops their current work to fix the bug, time is lost in context switching.
Tests should be so quick that developers can run all of them after each change without getting bored or frustrated.
Benefits: This reduces code dependency and test execution will be faster.
Mocks are prerequisites for the fast execution of tests and the ability to concentrate on a single unit of functionality. By mocking dependencies external to the method that is being tested, the developer is able to focus on the task at hand without spending time in setting them up. In the case of bigger teams, those dependencies might not even be developed. Also, the execution of tests without mocks tends to be slow. Good candidates for mocks are databases, other products, services, and so on.
Benefits: This allows setup and teardown code to be executed before and after the class or each method.
In many cases, some code needs to be executed before the test class or before each method in a class. For this purpose, JUnit has @BeforeClass and @Before annotations that should be used as the setup phase. @BeforeClass executes the associated method before the class is loaded (before the first test method is run).
@Before executes the associated method before each test is run. Both should be used when there are certain preconditions required by tests. The most common example is setting up test data in the (hopefully in-memory) database.
At the opposite end are @After and @AfterClass annotations, which should be used as the teardown phase. Their main purpose is to destroy data or a state created during the setup phase or by the tests themselves. As stated in one of the previous practices, each test should be independent from the others. Moreover, no test should be affected by the others. The teardown phase helps to maintain the system as if no test was previously executed.
Benefits: It provides test clarity.
Developers often approach test code in the same way as implementation. One of the common mistakes is to create base classes that are extended by tests. This practice avoids code duplication at the expense of test clarity. When possible, base classes used for testing should be avoided or limited. Having to navigate from the test class to its parent, to the parent of the parent, and so on in order to understand the logic behind tests often introduces unnecessary confusion. Clarity in tests should be more important than avoiding code duplication.