In this chapter, we will focus on two aspects of testing for scientific programming. The first aspect is the often difficult topic of what to test in scientific computing. The second aspect covers the question of how to test. We will distinguish between manual and automated testing. Manual testing is what is done by every programmer to quickly check that an implementation is working or not. Automated testing is the refined, automated variant of that idea. We will introduce some tools available for automatic testing in general, with a view on the particular case of scientific computing.
During the development of code, you do a lot of small tests in order to test its functionality. This could be called manual testing. Typically, you would test if a given function does what it is supposed to do, by manually testing the function in an interactive environment. For instance, suppose that you implement the bisection algorithm. It is an algorithm that finds a zero (root) of a scalar non-linear function. To start the algorithm, an interval has to be given with the property that the function takes different signs on the interval boundaries, see Exercise 4, Chapter 7, Functions, for more information.
You will then test an implementation of that algorithm, typically by checking that:
- A solution is found when the function has opposite signs at the interval boundaries
- An exception is raised when the function has the same sign at the interval boundaries
Manual testing, as necessary as it may seem to be, is unsatisfactory. Once you have convinced yourself that the code does what it is supposed to do, you formulate a relatively small number of demonstration examples to convince others of the quality of the code. At that stage, one often looses interest in the tests made during development and they are forgotten or even deleted. As soon as you change a detail and things no longer work correctly, you might regret that your earlier tests are no longer available.