The general layout of a test case is to set certain variables to known values, run one or more functions, methods, or processes, and then "prove" that the correct results were returned or calculated by using TestCase assertion methods.

There are a few different assertion methods available to confirm that specific results have been achieved. We just saw assertEqual, which will cause a test failure if the two parameters do not pass an equality check. The inverse, assertNotEqual, will fail if the two parameters do compare as equal. The assertTrue and assertFalse methods each accept a single expression, and fail if the expression does not pass an if test. These tests are not checking for the Boolean values True or False; they return the same value that would be returned if an if statement were used: False, None, 0, or an empty list, dictionary, or tuple would pass an assertFalse, while nonzero numbers, containers with values in them, or the value True would pass an assertTrue.

The assertRaises method accepts an exception class, a callable object (function, method, or an object with a __call__ method), and arbitrary arguments and keyword arguments to be passed into the callable. The assertion method will invoke the function with the supplied arguments, and will fail or raise an error if the method does not raise the expected exception class.

In addition, each of these methods accepts an optional argument named msg; if supplied, it will be included in the error message if the assertion fails. This is useful for clarifying what was expected or explaining where a bug may have occurred to cause the assertion to fail.

	import unittest
	
	def average(seq):
		return sum(seq) / len(seq)

	class TestAverage(unittest.TestCase):
		def test_python30_zero(self):
			self.assertRaises(ZeroDivisionError,
					average,
					[])

		def test_python31_zero(self):
			with self.assertRaises(ZeroDivisionError):
				average([])

	if __name__ == "__main__":
		unittest.main()

After writing a few small tests, we often find that we have to do the same setup code for several related tests. For example, the following simple list subclass has three methods for simple statistical calculations:

	from collections import defaultdict

	class StatsList(list):
		def mean(self):
			return sum(self) / len(self)
		
		def median(self):
			if len(self) % 2:
				return self[int(len(self) / 2)]
			else:
				idx = int(len(self) / 2)
				return (self[idx] + self[idx-1]) / 2

		def mode(self):
			freqs = defaultdict(int)
			for item in self:
				freqs[item] += 1
			mode_freq = max(freqs.values())
			modes = []
			for item, value in freqs.items():
				if value == mode_freq:
					modes.append(item)
			return modes

Clearly, we're going to want to test situations with each of these three methods that have very similar inputs; we'll want to see what happens with empty lists or with lists containing non-numeric values or with lists containing a normal dataset. We can use the setUp method on the TestCase class to do initialization for each test. This method accepts no arguments, and allows us to do arbitrary setup before each test is run. For example, we can test all three methods on identical lists of integers as follows:

	from stats import StatsList
	import unittest

	class TestValidInputs(unittest.TestCase):
		def setUp(self):
			self.stats = StatsList([1,2,2,3,3,4])
			
		def test_mean(self):
			self.assertEqual(self.stats.mean(), 2.5)
			
		def test_median(self):
			self.assertEqual(self.stats.median(), 2.5)
			self.stats.append(4)
			self.assertEqual(self.stats.median(), 3)
			
		def test_mode(self):
			self.assertEqual(self.stats.mode(), [2,3])
			self.stats.remove(2)
			self.assertEqual(self.stats.mode(), [3])
			
	if __name__ == "__main__":
		unittest.main()

If we run this example, it indicates that all tests pass. Notice first that the setUp method is never explicitly called inside the three test_* methods. The test suite does this on our behalf. More importantly notice how test_median alters the list, by adding an additional 4 to it, yet when test_mode is called, the list has returned to the values specified in setUp (if it had not, there would be two fours in the list, and the mode method would have returned three values). This shows that setUp is called individually before each test, to ensure the test class has a clean slate for testing. Tests can be executed in any order, and the results of one test do not depend on results from another.

In addition to the setUp method, TestCase offers a no-argument tearDown method, which can be used for cleaning up after each and every test on the class has run. This is useful if cleanup entails more than just letting an object be garbage collected. For example, if we are testing code that does file IO, our tests may create new files as a side effect of testing; the tearDown method can be used to remove these files and ensure the system is in the same state it was before the tests ran. Test cases should never have side effects.

In general, we group test methods into separate TestCase subclasses depending on what setup code they have in common. Several tests that require the same or similar setup will be placed in one class, while tests that require unrelated setup go in their own class.

It doesn't take long for a collection of unit tests to grow very large and unwieldy. It quickly becomes complicated to load and run all the tests at once. This is a primary goal of unit testing; it should be trivial to run all tests on our program and get a quick, "yes or no", answer to the question, "Did my recent changes break any existing tests?"

It is possible to collect groups of TestCase objects or modules containing tests into collections called TestSuites, and to load specific tests at specific times. In older versions of Python, this resulted in a lot of boilerplate code just to load and execute all the tests on a project. If this much control is needed, the functionality is still available, but most programmers can use test discovery, which will automatically find and run tests in the current package or subpackages.

Test discovery is built into Python 3.2 (and the simultaneously developed Python 2.7) and later, but can also be used in Python 3.1 or older versions of Python by installing the discover module available from http://pypi.python.org/pypi/discover/.

The discover module basically looks for any modules in the current folder or subfolders with names that start with the characters test. If it finds any TestCase or TestSuite objects in these modules, the tests are executed. It's a painless way to ensure you don't miss running any tests. To use it, simply ensure your test modules are named test_<something>.py and then run one of the following two commands, depending on which version of Python you have installed:

Sometimes a test is known to fail, but we don't want the test suite to report a failure under those conditions. This may be because a broken or unfinished feature has had tests written, but we aren't currently focusing on improving it. More often, it happens because a feature is only available on a certain platform, Python version, or for advanced versions of a specific library. Python provides us with a few decorators to mark tests as expected to fail or to be skipped under known conditions.

These decorators are:

These are applied using the Python decorator syntax. The first one accepts no arguments, and simply tells the test runner not to record the test as a failure even if it does, in fact, fail. The skip method goes one step further and doesn't even bother to run the test. It expects a single string argument describing why the test was skipped. The other two decorators accept two arguments, one a Boolean expression that indicates whether or not the test should be run, and a similar description. In use, these three decorators might be applied like this:

	import unittest
	import sys
	
	class SkipTests(unittest.TestCase):
		@unittest.expectedFailure
		def test_fails(self):
			self.assertEqual(False, True)
			
		@unittest.skip("Test is useless")
		def test_skip(self):
			self.assertEqual(False, True)
		
		@unittest.skipIf(sys.version_info.minor == 1,
				"broken on 3.1")
		def test_skipif(self):
			self.assertEqual(False, True)
			
		@unittest.skipUnless(sys.platform.startswith('linux'),
				"broken on linux")
		def test_skipunless(self):
			self.assertEqual(False, True)
	if __name__ == "__main__":
		unittest.main()

The first test fails, but it is reported as an expected failure; the second test is never run. The other two tests may or may not be run depending on the current Python version and operating system. On my Linux system running Python 3.1 the output looks like this:

	xssF
	=============================================================
	FAIL: test_skipunless (__main__.SkipTests)
	--------------------------------------------------------------
	Traceback (most recent call last):
	File "skipping_tests.py", line 21, in test_skipunless
	self.assertEqual(False, True)
	AssertionError: False != True

	--------------------------------------------------------------
	Ran 4 tests in 0.001s
	
	FAILED (failures=1, skipped=2, expected failures=1)

The x on the first line indicates an expected failure; the two s characters represent skipped tests, and the F indicates a real failure, since the conditional to skipUnless was True on my system.