The doctest tool is flexible and extremely easy to use but, as we've noticed, it falls somewhat short when it comes to writing disciplined tests. That's not to say that it's impossible; we've seen that we can write well-behaved, isolated tests in doctest. The problem is that doctest doesn't do any of that work for us. Fortunately, we have another testing tool on hand, a tool that requires a bit more structure in our tests, and provides a bit more support: unittest.
The unittest module was designed based on the requirements of unit testing, but it's not actually limited to that. You can use unit test for integration and system testing, too.
Like doctest, unittest is a part of the Python standard library; thus, if you've got Python, you have unit test.
In this chapter, we're going to cover the following topics:
- Writing tests within the
unittestframework - Running our new tests
- Looking at the features that make
unittesta good choice for larger test suites
Before we start talking about new concepts and features, let's take a look at how to use unittest to express the ideas that we've already learned about. That way, we'll have something solid on which ground our new understanding.
We're going to revisit the PID class, or at least the tests for the PID class, from Chapter 3, Unit Testing with doctest. We're going to rewrite the tests so that they operate within the unittest framework.
Before moving on, take a moment to refer back to the final version of the pid.txt file from Chapter 3, Unit Testing with doctest. We'll be implementing the same tests using the unittest framework.
Create a new file called test_pid.py in the same directory as pid.py. Notice that this is a .py file: unittest tests are pure Python source code, rather than being plain text with source code embedded in it. This means that the tests will be less useful from a documentary point of view, but grants other benefits in exchange.
Insert the following code into your newly created test_pid.py file:
from unittest import TestCase, main
from unittest.mock import Mock, patch
import pid
class test_pid_constructor(TestCase):
def test_constructor_with_when_parameter(self):
controller = pid.PID(P = 0.5, I = 0.5, D = 0.5,
setpoint = 1, initial = 12,
when = 43)
self.assertEqual(controller.gains, (0.5, 0.5, 0.5))
self.assertAlmostEqual(controller.setpoint[0], 1.0)
self.assertEqual(len(controller.setpoint), 1)
self.assertAlmostEqual(controller.previous_time, 43.0)
self.assertAlmostEqual(controller.previous_error, -11.0)
self.assertAlmostEqual(controller.integrated_error, 0)It has been argued, sometimes with good reason, that unit tests should not contain more than one assertion. The idea is that each unit test should test one thing and one thing only, to further narrow down what the problem is, when the test fails. It's a good point but not something to be overly fanatic about, in my opinion. In cases like the preceding code, splitting each assertion out into its own test function will not produce any more informative error messages than we get in this way; it would just increase our overhead.
My rule of thumb is that a test function can have any number of trivial assertions, and at most one non-trivial assertion:
@patch('pid.time', Mock(side_effect = [1.0]))
def test_constructor_without_when_parameter(self):
controller = pid.PID(P = 0.5, I = 0.5, D = 0.5,
setpoint = 0, initial = 12)
self.assertEqual(controller.gains, (0.5, 0.5, 0.5))
self.assertAlmostEqual(controller.setpoint[0], 0.0)
self.assertEqual(len(controller.setpoint), 1)
self.assertAlmostEqual(controller.previous_time, 1.0)
self.assertAlmostEqual(controller.previous_error, -12.0)
self.assertAlmostEqual(controller.integrated_error, 0)
class test_pid_calculate_response(TestCase):
def test_with_when_parameter(self):
mock = Mock()
mock.gains = (0.5, 0.5, 0.5)
mock.setpoint = [0.0]
mock.previous_time = 1.0
mock.previous_error = -12.0
mock.integrated_error = 0.0
self.assertEqual(pid.PID.calculate_response(mock, 6, 2), -3)
self.assertEqual(pid.PID.calculate_response(mock, 3, 3), -4.5)
self.assertEqual(pid.PID.calculate_response(mock, -1.5, 4), -0.75)
self.assertEqual(pid.PID.calculate_response(mock, -2.25, 5), -1.125)
@patch('pid.time', Mock(side_effect = [2.0, 3.0, 4.0, 5.0]))
def test_without_when_parameter(self):
mock = Mock()
mock.gains = (0.5, 0.5, 0.5)
mock.setpoint = [0.0]
mock.previous_time = 1.0
mock.previous_error = -12.0
mock.integrated_error = 0.0
self.assertEqual(pid.PID.calculate_response(mock, 6), -3)
self.assertEqual(pid.PID.calculate_response(mock, 3), -4.5)
self.assertEqual(pid.PID.calculate_response(mock, -1.5), -0.75)
self.assertEqual(pid.PID.calculate_response(mock, -2.25), -1.125)Now, run the tests by typing the following on the command line:
python3 -m unittest discoverYou should see output similar to this:
So, what did we do there? There are several things to notice:
- First, all of the tests are their own methods of classes that inherit from
unittest.TestCase. - The tests are named
test_<something>, where<something>is a description to help you (and others who share the code) remember what the test is actually checking. This matters becauseunittest(and several other testing tools) use the name to differentiate tests from non-test methods. As a rule of thumb, your test method names and test module filenames should start with test. - Because each test is a method, each test naturally runs in its own variable scope. Right here, we gain a big advantage from keeping the tests isolated.
- We inherited a bunch of
assert<Something>methods fromTestCase. These give us more flexible ways of checking whether values match, and provide more useful error reports, than Python's basicassertstatement. - We used
unittest.mock.patchas a method decorator. In Chapter 4, Decoupling Units with unittest.mock, we used it as a context manager. Either way, it does the same thing: it replaces an object with a mock object, and then puts the original back. When used as a decorator, the replacement happens before the method runs, and the original is put back after the method is complete. That's exactly what we need when our test is a method, so we'll be doing it in this way quite a lot. - We didn't patch over
time.time, we patchedover pid.time. This is because we're not reimporting thepidmodule for each test here. Thepidmodule containsfrom time import time, which means that, when it is first loaded, thetimefunction is referenced directly into thepidmodule's scope. From then on, changingtime.timedoesn't have any effect onpid.time, unless we change it and then reimport thepidmodule. Instead of going to all that trouble, we just patchedpid.timedirectly. - We didn't tell
unittestwhich tests to run. Instead, we told it to discover them and it found the tests on its own and ran them automatically. This often works well and saves effort. We'll be looking at a more elaborate tool for test discovery and execution in Chapter 6, Running Your Tests with Nose. - The
unittestmodule prints out one dot for each successful test. It will give you more information for tests that fail, or raise an unexpected exception.
The actual tests we performed are the same ones that were written in doctest. So far, all we're seeing is a different way of expressing them.
Each test method embodies a single test of a single unit. This gives us a convenient way to structure our tests, grouping together related tests into the same class so that they're easier to find. You might have noticed that we used two test classes in the example. This was for organizational purposes in this case, although there can also be good practical reasons to separate your tests into multiple classes. We'll talk about that soon.
Putting each test into its own method means that each test executes in an isolated namespace, which makes it easier to keep unittest-style tests from interfering with each other, relative to doctest-style tests. This also means that unittest knows how many unit tests are in your test file, instead of simply knowing how many expressions there are (you might have noticed that doctest counts each >>> line as a separate test). Finally, putting each test in its own method means that each test has a name, which can be a valuable feature. When you run unittest, it will include the names of any failing tests in the error report.
Tests in unittest don't directly care about anything that isn't part of a call to one of the TestCase assert methods. This means that we don't have to be bothered about the return values of any functions we call or the results of any expressions we use, unless they're important to the test. This also means that we need to remember to write an assert describing every aspect of the test that we want to have checked. We'll go through the various assertion methods of TestCase shortly.