Choosing Test Cases

Having a set of tests that pass is good; it shows that your code does what it should in the situations you’ve thought of. However, for any large project there will be situations that don’t occur to you. Tests can show the absence of many bugs, but it can’t show that a program is fully correct.

It’s important to make sure you have good test coverage: that your test cases cover important situations. In this section, we provide some heuristics that will help you come up with a fairly thorough set of test cases.

Now that you’ve seen two example sets of tests, we’ll give you an overview of things to think about while you’re developing tests for other functions. Some of them overlap and not all will apply in every situation, but they are all worth thinking about while you are figuring out what to test.

• Think about size. When a test involves a collection such as a list, string, dictionary, or file, you need to do the following:
  • Test the empty collection.
  • Test a collection with one item in it.
  • Test a general case with several items.
  • Test the smallest interesting case, such as sorting a list containing two values.

• Think about dichotomies. A dichotomy is a contrast between two things. Examples of dichotomies are empty/full, even/odd, positive/negative, and alphabetic/nonalphabetic. If a function deals with two or more different categories or situations, make sure you test all of them.

• Think about boundaries. If a function behaves differently around a particular boundary or threshold, test exactly that boundary case.

• Think about order. If a function behaves differently when values appear in different orders, identify those orders and test each one of them. For the sorting example mentioned earlier, you’ll want one test case where the items are in order and one where they are not.

If you carefully plan your test cases according to these ideas and your code passes the tests, there’s a very good chance that it will work for all other cases as well. Over time you’ll commit fewer and fewer errors. Whenever you find an error, figure out why it happened; as you mentally catalog them, you’ll subsequently become more conscious of them. And that’s really the whole point of focusing on quality. The more you do it, the less likely it is for problems to arise.