Case Study: Testing running_sum
In Case Study: Testing above_freezing , we tested a program that involved only immutable types. In this section, you’ll learn how to test functions involving mutable types, like lists and dictionaries.
Suppose we need to write a function that modifies a list so that it contains a running sum of the values in it. For example, if the list is [1, 2, 3], the list should be mutated so that the first value is 1, the second value is the sum of the first two numbers, 1 + 2, and the third value is the sum of the first three numbers, 1 + 2 + 3, so we expect that the list [1, 2, 3] will be modified to be [1, 3, 6].
Following the function design recipe (see Designing New Functions: A Recipe), here is a file named sums.py that contains the completed function with one (passing) example test:
| | from typing import List |
| | |
| | def running_sum(L: List[float]) -> None: |
| | """Modify L so that it contains the running sums of its original items. |
| | |
| | >>> L = [4, 0, 2, -5, 0] |
| | >>> running_sum(L) |
| | >>> L |
| | [4, 4, 6, 1, 1] |
| | """ |
| | |
| | for i in range(len(L)): |
| | L[i] = L[i - 1] + L[i] |
The structure of the test in the docstring is different from what you’ve seen before. Because there is no return statement, running_sum returns None. Writing a test that checks whether None is returned isn’t enough to know whether the function call worked as expected. You also need to check whether the list passed to the function is mutated in the way you expect it to be. To do this, we follow these steps:
- Create a variable that refers to a list.
- Call the function, passing that variable as an argument to it.
- Check whether the list that the variable refers to was mutated correctly.
Following those steps, we created a variable, L, that refers to the list [4, 0, 2, -5, 0], called running_sum(L), and confirmed that L now refers to [4, 4, 6, 1, 1].
Although this test case passes, it doesn’t guarantee that the function will always work—and in fact there is a bug. In the next section, we’ll design a set of test cases to more thoroughly test this function and discover the bug.
Choosing Test Cases for running_sum
Function running_sum has one parameter, which is a List[float]. For our test cases, we need to decide both on the size of the list and the values of the items. For size, we should test with the empty list, a short list with one item and another with two items (the shortest case where two numbers interact), and a longer list with several items.
When passed either the empty list or a list of length one, the modified list should be the same as the original.
When passed a two-number list, the first number should be unchanged and the second number should be changed to be the sum of the two original numbers.
For longer lists, things get more interesting. The values can be negative, positive, or zero, so the resulting values might be bigger than, the same as, or less than they were originally. We’ll divide our test of longer lists into four cases: all negative values, all zero, all positive values, and a mix of negative, zero, and positive values. The resulting tests are shown in this table:
|
Test Case Description |
List Before |
List After |
|---|---|---|
|
Empty list |
[] |
[] |
|
One-item list |
[5] |
[5] |
|
Two-item list |
[2, 5] |
[2, 7] |
|
Multiple items, all negative |
[-1, -5, -3, -4] |
[-1, -6, -9, -13] |
|
Multiple items, all zero |
[0, 0, 0, 0] |
[0, 0, 0, 0] |
|
Multiple items, all positive |
[4, 2, 3, 6] |
[4, 6, 9, 15] |
|
Multiple items, mixed |
[4, 0, 2, -5, 0] |
[4, 4, 6, 1, 1] |
Now that we’ve decided on our test cases, the next step is to implement them using unittest.
Testing running_sum Using unittest
To test running_sum, we’ll use this subclass of unittest.TestCase named TestRunningSum:
| | import unittest |
| | import sums as sums |
| | |
| | class TestRunningSum(unittest.TestCase): |
| | """Tests for sums.running_sum.""" |
| | |
| | def test_running_sum_empty(self): |
| | """Test an empty list.""" |
| | |
| | argument = [] |
| | expected = [] |
| | sums.running_sum(argument) |
| | self.assertEqual(expected, argument, "The list is empty.") |
| | |
| | def test_running_sum_one_item(self): |
| | """Test a one-item list.""" |
| | |
| | argument = [5] |
| | expected = [5] |
| | sums.running_sum(argument) |
| | self.assertEqual(expected, argument, "The list contains one item.") |
| | |
| | def test_running_sum_two_items(self): |
| | """Test a two-item list.""" |
| | |
| | argument = [2, 5] |
| | expected = [2, 7] |
| | sums.running_sum(argument) |
| | self.assertEqual(expected, argument, "The list contains two items.") |
| | |
| | def test_running_sum_multi_negative(self): |
| | """Test a list of negative values.""" |
| | |
| | argument = [-1, -5, -3, -4] |
| | expected = [-1, -6, -9, -13] |
| | sums.running_sum(argument) |
| | self.assertEqual(expected, argument, |
| | "The list contains only negative values.") |
| | |
| | def test_running_sum_multi_zeros(self): |
| | """Test a list of zeros.""" |
| | |
| | argument = [0, 0, 0, 0] |
| | expected = [0, 0, 0, 0] |
| | sums.running_sum(argument) |
| | self.assertEqual(expected, argument, "The list contains only zeros.") |
| | |
| | def test_running_sum_multi_positive(self): |
| | """Test a list of positive values.""" |
| | |
| | argument = [4, 2, 3, 6] |
| | expected = [4, 6, 9, 15] |
| | sums.running_sum(argument) |
| | self.assertEqual(expected, argument, |
| | "The list contains only positive values.") |
| | |
| | def test_running_sum_multi_mix(self): |
| | """Test a list containing mixture of negative values, zeros and |
| | positive values.""" |
| | |
| | argument = [4, 0, 2, -5, 0] |
| | expected = [4, 4, 6, 1, 1] |
| | sums.running_sum(argument) |
| | self.assertEqual(expected, argument, |
| | "The list contains a mixture of negative values, zeros and" |
| | + "positive values.") |
| | |
| | unittest.main() |
Next we run the tests and see only three of them pass (the empty list, a list with several zeros, and a list with a mixture of negative values, zeros, and positive values):
| | ..FF.FF |
| | ====================================================================== |
| | FAIL: test_running_sum_multi_negative (__main__.TestRunningSum) |
| | Test a list of negative values. |
| | ---------------------------------------------------------------------- |
| | Traceback (most recent call last): |
| | File "test_running_sum.py", line 38, in test_running_sum_multi_negative |
| | "The list contains only negative values.") |
| | AssertionError: Lists differ: [-1, -6, -9, -13] != [-5, -10, -13, -17] |
| | |
| | First differing element 0: |
| | -1 |
| | -5 |
| | |
| | - [-1, -6, -9, -13] |
| | + [-5, -10, -13, -17] : The list contains only negative values. |
| | |
| | ====================================================================== |
| | FAIL: test_running_sum_multi_positive (__main__.TestRunningSum) |
| | Test a list of positive values. |
| | ---------------------------------------------------------------------- |
| | Traceback (most recent call last): |
| | File "test_running_sum.py", line 55, in test_running_sum_multi_positive |
| | "The list contains only positive values.") |
| | AssertionError: Lists differ: [4, 6, 9, 15] != [10, 12, 15, 21] |
| | |
| | First differing element 0: |
| | 4 |
| | 10 |
| | |
| | - [4, 6, 9, 15] |
| | + [10, 12, 15, 21] : The list contains only positive values. |
| | |
| | ====================================================================== |
| | FAIL: test_running_sum_one_item (__main__.TestRunningSum) |
| | Test a one-item list. |
| | ---------------------------------------------------------------------- |
| | Traceback (most recent call last): |
| | File "test_running_sum.py", line 21, in test_running_sum_one_item |
| | self.assertEqual(expected, argument, "The list contains one item.") |
| | AssertionError: Lists differ: [5] != [10] |
| | |
| | First differing element 0: |
| | 5 |
| | 10 |
| | |
| | - [5] |
| | + [10] : The list contains one item. |
| | |
| | ====================================================================== |
| | FAIL: test_running_sum_two_items (__main__.TestRunningSum) |
| | Test a two-item list. |
| | ---------------------------------------------------------------------- |
| | Traceback (most recent call last): |
| | File "test_running_sum.py", line 29, in test_running_sum_two_items |
| | self.assertEqual(expected, argument, "The list contains two items.") |
| | AssertionError: Lists differ: [2, 7] != [7, 12] |
| | |
| | First differing element 0: |
| | 2 |
| | 7 |
| | |
| | - [2, 7] |
| | + [7, 12] : The list contains two items. |
| | |
| | ---------------------------------------------------------------------- |
| | Ran 7 tests in 0.002s |
| | |
| | FAILED (failures=4) |
The four that failed were a list with one item, a list with two items, a list with all negative values, and a list with all positive values. To find the bug, let’s focus on the simplest test case, the single-item list:
| | ====================================================================== |
| | FAIL: test_running_sum_one_item (__main__.TestRunningSum) |
| | Test a one-item list. |
| | ---------------------------------------------------------------------- |
| | Traceback (most recent call last): |
| | File "/Users/campbell/pybook/gwpy2/Book/code/testdebug/test_running_sum. |
| | py", line 21, in test_running_sum_one_item |
| | self.assertEqual(expected, argument, "The list contains one item.") |
| | AssertionError: Lists differ: [5] != [10] |
| | First differing element 0: |
| | 5 |
| | 10 |
| | |
| | - [5] |
| | + [10] : The list contains one item. |
For this test, the list argument was [5]. After the function call, we expected the list to be [5], but the list was mutated to become [10]. Looking back at the function definition of running_sum, when i refers to 0, the for loop body executes the statement L[0] = L[-1] + L[0]. L[-1] refers to the last element of the list—the 5—and L[0] refers to that same value. Oops! L[0] shouldn’t be changed, since the running sum of L[0] is simply L[0].
Looking at the other three failing tests, the failure messages indicate that the first different elements are those at index 0. The same problem that we describe for the single-item list happened for these test cases as well.
So how did those other three tests pass? In those cases, L[-1] + L[0] produced the same value that L[0] originally referred to. For example, for the list containing a mixture of values, [4, 0, 2, -5, 0], the item at index -1 happened to be 0, so 0 + 4 evaluated to 4, and that matched L[0]’s original value. Interestingly, the simple single-item list test case revealed the problem, whereas the more complex test case that involved a list of multiple values hid it!
To fix the problem, we can adjust the for loop header to start the running sum from index 1 rather than from index 0:
| | from typing import List |
| | |
| | def running_sum(L: List[float]) -> None: |
| | """Modify L so that it contains the running sums of its original items. |
| | |
| | >>> L = [4, 0, 2, -5, 0] |
| | >>> running_sum(L) |
| | >>> L |
| | [4, 4, 6, 1, 1] |
| | """ |
| | |
| | for i in range(1, len(L)): |
| | L[i] = L[i - 1] + L[i] |
When the tests are rerun, all seven tests pass:
| | ....... |
| | ---------------------------------------------------------------------- |
| | Ran 7 tests in 0.000s |
| | |
| | OK |
In the next section, you’ll see some general guidelines for choosing test cases.