Most skilled Python programmers agree that testing is one of the most important aspects of software development in Python. Even though this chapter is placed near the end of the book, it is not an afterthought; everything we have studied so far will help us when writing tests. We'll be studying:
- The importance of unit testing and test-driven development
- The standard
unittestmodule - The
py.testautomated testing suite - Code coverage
More and more programmers are learning how important testing their code is. If you're among them, feel free to skim this section. You'll find the next section, where we actually learn how to do the tests in Python, more scintillating. If you're not convinced of the importance of testing, I promise that your code is broken, you just don't know it. Read on!
Some people argue that testing is more important in Python code because of its dynamic nature; compiled languages such as Java and C++ are occasionally thought to be somehow 'safer' because they enforce type checking at compile time. The thing is, Python tests rarely, if ever, check types. They're checking values. They're making sure that the right attributes have been set at the right time or that the sequence has the right length, order, and values. They aren't checking to make absolutely sure that a list was returned if a tuple or custom container would suffice. These higher-level things need to be tested in any language. The real reason Python programmers test more than programmers of other languages is that it is so easy to test in Python!
But why test? Do we really need to test? What if we didn't test? To answer those questions, write a program from scratch without any testing at all. Don't run it until it is completely written, start to finish. Don't pick a large program, just a few classes interacting. If you can't come up with anything, try a two player tic-tac-toe game; it's fairly simple if you make both players human players (no artificial intelligence). You don't even have to try to calculate who the winner is.
Now run your program. And fix all the errors. How many were there? I recorded eight on my tic-tac-toe implementation, and I'm not sure I caught them all. Did you?
We need to test our code to make sure it works. Running the program, as we just did, and fixing the errors is one crude form of testing. Most programmers tend to write a few lines of code and run the program to make sure those lines are doing what they expect. But changing a few lines of code can affect parts of the program that the developer hadn't considered as being influenced by the changes. Because they weren't thinking of this influence, they won't test the other functionality. As we can see from the the program we just wrote, if they haven't tested it, it's almost certainly broken.
To handle this, we write automated tests. Written tests are simply programs that automatically run certain inputs through other programs (or, more often, parts of other programs, such as one function or class). We can run these test programs in seconds and cover more possible user inputs than one programmer would think to test every time they change something.
There are four main reasons to write tests:
- To ensure that code is working the way the developer thinks it should
- To ensure that code continues working when we make changes
- To ensure the developer understood the requirements
- To ensure that the code we are writing has a maintainable interface
The first point really doesn't justify the time it takes to write a test; we can simply test the code directly (perhaps through the interactive interpreter). But as soon as we have to perform the same sequence of test actions multiple times, it takes less time to automate those steps once and then run them whenever necessary. It is a good idea to run tests whenever we have to change code, whether it is during initial development or maintenance releases. When we have a comprehensive set of automated tests, we can run them after code changes and know that we didn't inadvertently break anything that was tested.
However, the most interesting points in the list are the last two. When we write tests for code, it can actually help us design the API, interface, or pattern that code takes. Thus, if we, as the programmers, misunderstood the requirements from management or the client, writing a test can help highlight that misunderstanding. On the other side, if we're not certain how we want to design a class, we can write a test that interacts with that class so we have an idea what the most natural way to test it would be. In fact, it is often beneficial to write the tests before we write the code we are testing!
Writing tests first is the mantra of test-driven development. Test-driven development takes the "untested code is broken code" concept one step further and suggests that only unwritten code should be untested. Do not write any code until you have written the tests for this code. So the first step is to write a test that proves the code would work. Obviously, the test is going to fail, since the code hasn't been written. Then write the code that ensures the test passes. Then write another test for the next segment of code.
Test-driven development is fun. It allows us to build little puzzles to solve; these are the tests. Then we implement the code to solve the puzzles. Then we make a more complicated puzzle, and we write code that solves the new puzzle without unsolving the previous one.
There are two goals to the test-driven methodology. The first is to ensure that tests really get written. It's so very easy, after we have written code, to say: "Hmm, it seems to work, I don't have to write any tests for this. It was just a small change, nothing could have broken." If the test is already written before we write the code, we will know exactly when it works (because the test will pass), and we'll know in the future if it is ever broken by a change we, or someone else has made.
Secondly, writing tests first forces us to consider exactly how the code is going to be interacted with. It tells us what methods we need objects to have and how attributes will be accessed. It helps us break up the initial problem into smaller, testable problems, and then to recombine the tested solutions into larger, also tested solutions. Writing tests can thus become a part of the design process. Often, if we're writing a test for a not-fully-specified object, we will discover anomalies in the design that force us to consider new aspects of the software.
As a concrete example, we may be writing code that uses an object-relational mapper to store object properties in a database. It is common to use an automatically assigned database id in such objects, and our code might be using this id for various purposes, say as a key in a dictionary. If we are writing a test for such code, before we write it, we may realize that our design is faulty because objects do not have these IDs until they have been saved to the database. If we want to manipulate an object without saving it in our test, it will highlight this problem before we have written any code.
Testing makes software better. Writing tests before we release the software makes it better before the end user sees or purchases the buggy version (I have worked for companies that thrive on the "the users can test it" philosophy. It's not a healthy business model!). Writing tests before we write the software makes it better the first time it is written.