There are different kinds of tests. However, at the minimum, a programmers need to know unit tests since they have to be able to write them. Unit testing checks the smallest testable part of an application. Integration testing checks whether these parts work well with each other.
The word unit is the key term here. Just test one unit at a time. Let's take a look at a simple example of a test case:
# tests.py
from django.test import TestCase
from django.core.urlresolvers import resolve
from .views import HomeView
class HomePageOpenTestCase(TestCase):
def test_home_page_resolves(self):
view = resolve('/')
self.assertEqual(view.func.__name__,
HomeView.as_view().__name__)This is a simple test that checks whether, when a user visits the root of our website's domain, they are correctly taken to the home page view. Like most good tests, it has a long and self-descriptive name. The test simply uses Django's resolve() function to match the view callable mapped to the "/" root location to the known view function by their names.
It is more important to note what is not done in this test. We have not tried to retrieve the HTML contents of the page or check its status code. We have restricted ourselves to test just one unit, that is, the resolve() function, which maps the URL paths to view functions.
Assuming that this test resides in, say, app1 of your project, the test can be run with the following command:
$ ./manage.py test app1 Creating test database for alias 'default'... . ----------------------------------------------------------------- Ran 1 test in 0.088s OK Destroying test database for alias 'default'...
This command runs all the tests in the app1 application or package. The default test runner will look for tests in all modules in this package matching the pattern test*.py.
Django now uses the standard unittest module provided by Python rather than bundling its own. You can write a testcase class by subclassing from django.test.TestCase. This class typically has methods with the following naming convention:
test*: Any method whose name starts withtestwill be executed as a test method. It takes no parameters and returns no values. Tests will be run in an alphabetical order.setUp(optional): This method will be run before each test method. It can be used to create common objects or perform other initialization tasks that bring your test case to a known state.tearDown(optional): This method will be run after a test method, irrespective of whether the test passed or not. Clean-up tasks are usually performed here.
A test case is a way to logically group test methods, all of which test a scenario. When all the test methods pass (that is, do not raise any exception), then the test case is considered passed. If any of them fail, then the test case fails.
Each test method usually invokes an assert*() method to check some expected outcome of the test. In our first example, we used assertEqual() to check whether the function name matches with the expected function.
Similar to assertEqual(), the Python 3 unittest library provides more than 32 assert methods. It is further extended by Django by more than 19 framework-specific assert methods. You must choose the most appropriate method based on the end outcome that you are expecting so that you will get the most helpful error message.
Let's see why by looking at an example testcase that has the following setUp() method:
def setUp(self):
self.l1 = [1, 2]
self.l2 = [1, 0]Our test is to assert that l1 and l2 are equal (and it should fail, given their values). Let's take a look at several equivalent ways to accomplish this:
|
Test Assertion Statement |
What Test Output Looks Like (unimportant lines omitted) |
|---|---|
assert self.l1 == self.l2 |
assert self.l1 == self.l2 AssertionError |
self.assertEqual(self.l1, self.l2) |
AssertionError: Lists differ: [1, 2] != [1, 0] First differing element 1: 2 0 |
self.assertListEqual( self.l1, self.l2) |
AssertionError: Lists differ: [1, 2] != [1, 0] First differing element 1: 2 0 |
self.assertListEqual(self.l1, None) |
AssertionError: Second sequence is not a list: None |
The first statement uses Python's built- in assert keyword. Notice that it throws the least helpful error. You cannot infer what values or types are in the self.l1 and self.l2 variables. This is primarily the reason why we need to use the assert*() methods.
Next, the exception thrown by assertEqual() very helpfully tells you that you are comparing two lists and even tells you at which position they begin to differ. This is exactly similar to the exception thrown by the more specialized assertListEqual() function. This is because, as the documentation would tell you, if assertEqual() is given two lists for comparison, then it hands it over to assertListEqual().
Despite this, as the last example proves, it is always better to use the most specific assert* method for your tests. Since the second argument is not a list, the error clearly tells you that a list was expected.
Therefore, you need to familiarize yourself with all the assert methods, and choose the most specific one to evaluate the result you expect. This also applies to when you are checking whether your application does not do things it is not supposed to do, that is, a negative test case. You can check for exceptions or warnings using assertRaises and assertWarns respectively.
We have already seen that the best test cases test a small unit of code at a time. They also need to be fast. A programmer needs to run tests at least once before every commit to the source control. Even a delay of a few seconds can tempt a programmer to skip running tests (which is not a good thing).
Here are some qualities of a good test case (which is a subjective term, of course) in the form of an easy-to-remember mnemonic "F.I.R.S.T. class test case":
- Fast: the faster the tests, the more often they are run. Ideally, your tests should complete in a few seconds.
- Independent: Each test case must be independent of others and can be run in any order.
- Repeatable: The results must be the same every time a test is run. Ideally, all random and varying factors must be controlled or set to known values before a test is run.
- Small: Test cases must be as short as possible for speed and ease of understanding.
- Transparent: Avoid tricky implementations or ambiguous test cases.
Additionally, make sure that your tests are automatic. Eliminate any manual steps, no matter how small. Automated tests are more likely to be a part of your team's workflow and easier to use for tooling purposes.
Perhaps, even more important are the don'ts to remember while writing test cases:
- Do not (re)test the framework: Django is well tested. Don't check for URL lookup, template rendering, and other framework-related functionality.
- Do not test implementation details: Test the interface and leave the minor implementation details. It makes it easier to refactor this later without breaking the tests.
- Test models most, templates least: Templates should have the least business logic, and they change more often.
- Avoid HTML output validation: Test views use their context variable's output rather than its HTML-rendered output.
- Avoid using the web test client in unit tests: Web test clients invoke several components and are therefore, better suited for integration tests.
- Avoid interacting with external systems: Mock them if possible. Database is an exception since test database is in-memory and quite fast.
Of course, you can (and should) break the rules where you have a good reason to (just like I did in my first example). Ultimately, the more creative you are at writing tests, the earlier you can catch bugs, and the better your application will be.