While developing or debugging, you want certain conditional or looping sections of code to be temporarily omitted from execution.

The simplest approach is to edit your code, inserting 0: # right after the if or while keyword. Since 0 evaluates as false, that section of code will not execute. For example:

if i < 1:
   doSomething( )
while j < k:
   j = fleep(j, k)

into:

if 0: # i < 1:
   doSomething( )
while 0: # j < k:
   j = fleep(j, k)

If you have many such sections that must simultaneously switch on and off during your development and debug sessions, an alternative is to define a boolean variable (commonly known as a flag), say doit = False, and code:

if doit and i < 1:
   doSomething( )
while doit and j < k:
   j = fleep(j, k)

This way, you can temporarily switch the various sections on again by just changing the flag setting to doit = True, and easily flip back and forth. You can even have multiple such flags. Do remember to remove the doit and parts once you’re done developing and debugging, since at that point all they would do is slow things down.

Of course, you have other alternatives, too. Most good editors have commands to insert or remove comment markers from the start of each line in a marked section, like Alt-3 and Alt-4 in the editor of the IDLE IDE (Integrated Development Environment) that comes with Python; a common convention in such editors is to start such temporarily commented-out lines with two comment markers, ##, to distinguish them from “normal” comments.

One Python-specific technique you can use is the _ _debug_ _ read-only global boolean variable. _ _debug_ _ is True when Python is running without the -O (optimize) command-line option, False when Python is running with that option. Moreover, the Python compiler knows about _ _debug_ _ and can completely remove any block guarded by if _ _debug_ _ when Python is running with the command-line optimization option, thus saving memory as well as execution time.

The section on the _ _debug_ _ flag and the assert statement in the Language Reference and Python in a Nutshell.

You need to monitor how much memory your Python application, running under Linux, is currently using. However, the standard library module resource does not work correctly on Linux.

We can code our own resource measurements based on Linux’s /proc pseudo-filesystem:

import os
_proc_status = '/proc/%d/status' % os.getpid( )
_scale = {'kB': 1024.0, 'mB': 1024.0*1024.0,
          'KB': 1024.0, 'MB': 1024.0*1024.0}
def _VmB(VmKey):
    ''' given a VmKey string, returns a number of bytes. '''
    # get pseudo file  /proc/<pid>/status
    try:
        t = open(_proc_status)
        v = t.read( )
        t.close( )
    except IOError:
        return 0.0  # non-Linux?
    # get VmKey line e.g. 'VmRSS:  9999  kB
 ...'
    i = v.index(VmKey)
    v = v[i:].split(None, 3)  # split on runs of whitespace
    if len(v) < 3:
        return 0.0  # invalid format?
    # convert Vm value to bytes
    return float(v[1]) * _scale[v[2]]
def memory(since=0.0):
    ''' Return virtual memory usage in bytes. '''
    return _VmB('VmSize:') - since
def resident(since=0.0):
    ''' Return resident memory usage in bytes. '''
    return _VmB('VmRSS:') - since
def stacksize(since=0.0):
    ''' Return stack size in bytes. '''
    return _VmB('VmStk:') - since

Each of the functions in this recipe takes an optional argument since because the typical usage of these functions is to find out how much more memory (virtual, resident, or stack) has been used due to a certain section of code. Having since as an optional argument makes this typical usage quite simple and elegant:

    m0 = memory( )section of code you're monitoring
    m1 = memory(m0)
    print 'The monitored section consumed', m1, 'bytes of virtual memory'.

Getting and parsing the contents of pseudo-file /proc/pid/status is probably not the most efficient way to get data about memory usage, and it is not portable to non-Linux systems. However, it is a very simple and easy-to-code approach, and after all, on a non-Linux Unix system, you can use the resource module from the Python Standard Library.

In fact, you can use resource on Linux, but the various fields relevant to memory consumption, such as ru_maxrss, all have a constant value of 0, just like the various memory-consumption fields in the output of the time shell command under Linux. The root cause of this situation is a limitation in the Linux implementation of the getrusage system call, documented in man getrusage.

Documentation on the resource standard library module in the Library Reference.

You know that memory is leaking from your program, but you have no indication of what exactly is being leaked. You need more information to help you figure out where the leaks are coming from, so you can remove them and lighten the garbage-collection work periodically performed by the standard gc module.

The gc module lets you dig into garbage-collection issues:

import gc
def dump_garbage( ):
    """ show us what the garbage is about """
    # Force collection
    print "
GARBAGE:"gc.collect( )
    print "
GARBAGE OBJECTS:"
    for x in gc.garbage:
        s = str(x)
        if len(s) > 80: s = s[:77]+'...'
        print type(x),"
  ", s
if _ _name_ _=="_ _main_ _":
    gc.enable( )
    gc.set_debug(gc.DEBUG_LEAK)
    # Simulate a leak (a list referring to itself) and show it
    l = [  ]
    l.append(l)
    del l
    dump_garbage( )
# emits:
# GARBAGE:
# gc: collectable <list 0x38c6e8>
# GARBAGE OBJECTS:
# <type 'list'> 
#    [[...]]

In addition to the normal debugging output of gc, this recipe shows the garbage objects, to help you get an idea of where the leak may be. Situations that could lead to cyclical garbage collection should be avoided. Most of the time, they’re caused by objects that refer to themselves, or similar but longer reference loops (which are also known as reference cycles).

Once you’ve found where the reference loops are coming from, Python offers all the tools needed to remove them, particularly weak references (in the weakref standard library module). But especially in big programs, you first have to get an idea of where to find the leak before you can remove it and enhance your program’s performance. For this purpose, it’s good to know what the objects being leaked contain, and the dump_garbage function in this recipe comes in quite handy on such occasions.

This recipe works by first calling gc.set_debug to tell the gc module to keep the leaked objects in its gc.garbage list rather than recycling them. Then, this recipe’s dump_garbage function calls gc.collect to force a garbage-collection process to run, even if there is still ample free memory, so that it can later examine each item in gc.garbage and print out its type and contents (limiting the printout to no more than 80 characters per garbage object, to avoid flooding the screen with huge chunks of information).

Documentation for the gc and weakref modules in the Library Reference and Python in a Nutshell.

You need to trap exceptions, record their tracebacks and error messages, and then proceed with the execution of your program.

A typical case is a program that processes many independent files one after the other. Some files may be malformed and cause exceptions. You need to trap such exceptions, record the error information, then move on to process subsequent files. For example:

import cStringIO, traceback
def process_all_files(all_filenames,
                      fatal_exceptions=(KeyboardInterrupt, MemoryError)
                     ):
    bad_filenames = {  }
    for one_filename in all_filenames:
        try:
            process_one_file(one_filename):except fatal_exceptions:
            raise
        except Exception:
            f = cStringIO.StringIO( )
            traceback.print_exc(file=f)
            bad_filenames[one_filename] = f.getvalue( )
    return bad_filenames

Because Python exceptions are very powerful tools, you need a clear and simple strategy to deal with them. This recipe will probably not fit your needs exactly, but it may be a good starting point from which to develop the right strategy for your applications.

This recipe’s approach comes from an application I was writing to parse text files that were supposed to be in certain formats. Malformed files could easily cause exceptions, and I needed to get those errors’ tracebacks and messages to either fix my code to be more forgiving or fix malformed files; however, I also needed program execution to continue on subsequent files.

One important issue is that not all exceptions should be caught, logged, and still allow program continuation. A KeyboardInterrupt exception means the user is banging on Ctrl-C (or Ctrl-Break, or some other key combination), specifically asking for your application to stop; we should, of course, honor such requests, not ignore them. A MemoryError means you have run out of memory—unless you’ve got huge caches of previous results that you can immediately delete to make more memory available, generally you can’t do much about such a situation. Depending on your application and exact circumstances, other errors might well also be deemed just as fatal. So, process_all_files accepts a fatal_exceptions argument, a tuple of exception classes it should not catch (but which it should rather propagate), defaulting to the pair of exception classes we just mentioned. The try/except statement is carefully structured to catch, and re-raise, any exception in those classes, with precedence over the general except Exception handler clause, which catches everything else.

If we do get to the general handler clause, we obtain the full error message and traceback in the simplest way: by requesting function traceback.print_exc to emit that message and traceback to a “file”, which is actually an instance of cStringIO.StringIO, a “file"-like object specifically designed to ease in-memory capture of information from functions that normally write to files. The getvalue method of the StringIO instance provides the message and traceback as a string, and we store the string in dictionary bad_filenames, using, as the corresponding key, the filename that appears to have caused the probl7em. process_all_files' for loop then moves on to the next file it must process.

Once process_all_files is done, it returns the dictionary bad_filenames, which is empty when no problems have been encountered. Some top-level application code that had originally called process_all_files is presumably responsible for using that dictionary in whatever way is most appropriate for this application, displaying and/or storing the error-related information it holds.

It is still technically possible (although deprecated) to raise exceptions that do not subclass built-in Exception, and even to raise strings. If you need to catch such totally anomalous cases (whose possibility will no doubt stay around for years for backwards compatibility), you need to add one last unconditional except clause to your try/except statement:

        except fatal_exceptions:
            raise
        except Exception:...
        except:
            ...

Of course, if what you want to do for all normal (nonfatal) exceptions, and for the weird anomalous cases, is exactly the same, you don’t need a separate except Exception clause—just the unconditional except clause will do. However, you may normally want to log the occurrence of the weird anomalous cases in some different and more prominent way, because, these days (well into the twenty-first century), they’re definitely not expected under any circumstance whatsoever.

Documentation for the standard modules traceback and cStringIO in the Library Reference and Python in a Nutshell; documentation for try/except and exception classes in the Language Reference and Python in a Nutshell.

You are coding a program that cannot use an interactive, step-by-step debugger. Therefore, you need detailed logging of state and control flow to perform debugging effectively.

The extract_stack function from the traceback module is the key here because it lets your debugging code easily perform runtime introspection to find out about the code that called it:

import sys, traceback
traceOutput = sys.stdout
watchOutput = sys.stdout
rawOutput = sys.stdout
# calling 'watch(secretOfUniverse)' prints out something like:
#File "trace.py", line 57, in _ _testTrace
#    secretOfUniverse <int> = 42
watch_format = ('File "%(fileName)s", line %(lineNumber)d, in' 
                ' %(methodName)s
   %(varName)s <%(varType)s>' 
                ' = %(value)s

')
def watch(variableName):
    if _ _debug_ _:
        stack = traceback.extract_stack( )[-2:][0]
        actualCall = stack[3]
        if actualCall is None:
            actualCall = "watch([unknown])"
        left = actualCall.find('(')
        right = actualCall.rfind(')')
        paramDict = dict(varName=actualCall[left+1:right]).strip( ),
                         varType=str(type(variableName))[7:-2],
                         value=repr(variableName),
                         methodName=stack[2],
                         lineNumber=stack[1],
                         fileName=stack[0])
        watchOutput.write(watch_format % paramDict)
# calling 'trace("this line was executed")' prints out something like:
# File 
               "trace.py", line 64, in ?
#    this line was executed
trace_format = ('File "%(fileName)s", line %(lineNumber)d, in' 
                ' %(methodName)s
   %(text)s

')
def trace(text):
    if _ _debug_ _:
        stack = traceback.extract_stack( )[-2:][0]
        paramDict = dict(text=text,
                         methodName=stack[2],
                         lineNumber=stack[1],
                         fileName=stack[0])
        watchOutput.write(trace_format % paramDict)
# calling 'raw("some raw text")' prints out something like:
# Just some raw text
def raw(text):
    if _ _debug_ _:
        rawOutput.write(text)

Many of the different kinds of programs one writes today don’t make it easy to use traditional, interactive step-by-step debuggers. Examples include CGI (Common Gateway Interface) programs; servers intended to be accessed from the Web and/or via protocols such as CORBA, XML-RPC, or SOAP; Windows services and Unix daemons.

You can remedy this lack of interactive debugging by sprinkling a bunch of print statements all through the program, but this approach is unsystematic and requires cleanup when a given problem is fixed. This recipe shows that a better-organized approach is quite feasible, by supplying a few functions that allow you to output the value of an expression, a variable, or a function call, with scope information, trace statements, and general comments.

The key is the extract_stack function from the traceback module. traceback.extract_stack returns a list of tuples with four items—providing the filename, line number, function name, and source code of the calling statement—for each call in the stack. Item [-2] (the penultimate item) of this list is the tuple of information about our direct caller, and that’s the one we use in this recipe to prepare the information to emit on file-like objects bound to the traceOutput and watchOutput variables.

If you bind the traceOutput, watchOutput, or rawOutput variables to an appropriate file-like object, each kind of output is redirected appropriately. When _ _debug_ _ is false (i.e., when you run the Python interpreter with the -O or -OO switch), all the debugging-related code is automatically eliminated. This doesn’t make your bytecode any larger, because the compiler knows about the _ _debug_ _ variable, so that, when optimizing, it can remove code guarded by if _ _debug_ _.

Here is a usage example, leaving all output streams on standard output, in the form we’d generally use to make such a module self-testing, by appending the example at the end of the module:

def _ _testTrace( ):
    secretOfUniverse = 42
    watch(secretOfUniverse)
if _ _name_ _ == "_ _main_ _":
    a = "something else"
    watch(a)
    _ _testTrace( )
    trace("This line was executed!")
    raw("Just some raw text...")

When run with just python (no -O switch), this code emits:

File "trace.py", line 61, in ?
  a <str> = 'something else'
File "trace.py", line 57, in _ _testTrace
  secretOfUniverse <int> = 42
File "trace.py", line 64, in ?
  This line was executed!
Just some raw text...

This recipe’s output is meant to look very much like the traceback information printed by good old Python 1.5.2 while being compatible with any version of Python. It’s easy to modify the format strings to your liking, of course.

Recipe 8.6; documentation on the traceback standard library module in the Library Reference and Python in a Nutshell; the section on the _ _debug_ _ flag and the assert statement in the Language Reference and Python in a Nutshell.

You want to display all of the available information when an uncaught exception is raised.

A traceback object is basically a linked list of nodes, in which each node refers to a frame object. Frame objects, in turn, form their own linked list in the opposite order from the linked list of traceback nodes, so we can walk back and forth if needed. This recipe exploits this structure and the rich amount of information held by frame objects, including, in particular, the dictionary of local variables for the function corresponding to each frame:

import sys, traceback
def print_exc_plus( ):
    """ Print the usual traceback information, followed by a listing of
        all the local variables in each frame.
    """
    tb = sys.exc_info( )[2]
    while tb.tb_next:
        tb = tb.tb_next
    stack = [  ]
    f = tb.tb_frame
    while f:
        stack.append(f)
        f = f.f_back
    stack.reverse( )
    traceback.print_exc( )
    print "Locals by frame, innermost last"
    for frame in stack:
        print
        print "Frame %s in %s at line %s" % (frame.f_code.co_name,
                                             frame.f_code.co_filename,
                                             frame.f_lineno)
        for key, value in frame.f_locals.items( ):
            print "	%20s = " % key,
            # we must _absolutely_ avoid propagating exceptions, and str(value)
            # COULD cause any exception, so we MUST catch any...:
            try:
                print value
            except:
                print "<ERROR WHILE PRINTING VALUE>"

The standard Python traceback module provides useful functions to give information about where and why an error occurred. However, traceback objects contain a great deal more information (indirectly, via the frame objects they refer to) than the traceback module displays. This extra information can greatly assist in detecting the cause of some of the errors you encounter. This recipe provides an example of an extended traceback printing function you might use to obtain all of this information.

Here’s a simplistic demonstration of the kind of problem this approach can help with. Basically, we have a simple function that manipulates all the strings in a list. The function doesn’t do any error checking, so, when we pass a list that contains something other than strings, we get an error. Figuring out which bad data caused the error is easier with our new print_exc_plus function to help us:

data = ["1", "2", 3, "4"]     # Typo: we 'forget' the quotes on data[2]
def pad4(seq):
    """
    Pad each string in seq with zeros up to four places. Note that there
    is no reason to actually write this function; Python already
    does this sort of thing much better.  It's just an example!
    """
    return_value = [  ]
    for thing in seq:
        return_value.append("0" * (4 - len(thing)) + thing)
    return return_value

Here’s the (limited) information we get from a normal traceback.print_exc:

>>> try:
...     pad4(data)
... except:
...     traceback.print_exc( )
...Traceback (most recent call last):
                 File "<stdin>", line 2, in ?
                 File "<stdin>", line 9, in pad4
               TypeError: len( ) of unsized object

Now here’s how it looks when displaying the info with the function from this recipe instead of the standard traceback.print_exc:

>>> try:
...     pad4(data)
... except:
...     print_exc_plus( )
...Traceback (most recent call last):
                 File "<stdin>", line 2, in ?
                 File "<stdin>", line 9, in pad4
               TypeError: len( ) of unsized object
               Locals by frame, innermost last
               Frame ? in <stdin> at line 4
                                        sys =  <module 'sys' (built-in)>
                                       pad4 =  <function pad4 at 0x007C6210>
                               _ _builtins_ _ =  <module '_ _builtin_ _' (built-in)>
                                   _ _name_ _ =  _ _main_ _
                                       data =  ['1', '2', 3, '4']
                                    _ _doc_ _ =  None
                             print_exc_plus =  <function print_exc_plus at 0x00802038>
               Frame pad4 in <stdin> at line 9
                                      thing =  3
                               return_value =  ['0001', '0002']
                                        seq =  ['1', '2', 3, '4']

Note how easy it is to see the bad data that caused the problem. The thing variable has a value of 3, so we know why we got the TypeError. A quick look at the value for data shows that we simply forgot the quotes on that item. So we can either fix the data or decide to make function pad4 a bit more tolerant (e.g., by changing the loop to for thing in map(str, seq)). These kind of design choices are important, but the point of this recipe is to save you time in understanding what’s going on, so you can make your design choices with all the available information.

The recipe relies on the fact that each traceback object refers to the next traceback object in the stack through the tb_next field, forming a linked list. Each traceback object also refers to a corresponding frame object through the tb_frame field, and each frame refers to the previous frame through the f_back field (a linked list going the other way around from that of the traceback objects).

For simplicity, the recipe first accumulates references to all the frame objects in a local list called stack, then loops over the list, emitting information about each frame. For each frame, it first emits some basic information (e.g., function name, filename, line number, etc.) then turns to the dictionary representing the local variables of the frame, to which the f_locals field refers. Just like for the dictionaries built and returned by the locals and globals built-in functions, each key is a variable name, and the corresponding value is the variable’s value. Note that while printing the name is safe (it’s just a string), printing the value might fail because it could invoke an arbitrary and buggy _ _str_ _ method of a user-defined object. So, the value is printed within a try/except statement, to prevent the propagation of an uncaught exception while another exception is being handled. An except clause that does not list the exceptions to catch, and thus catches every exception, is almost always a mistake, but this recipe exemplifies the almost part of this statement!

I use a technique similar to this one in the applications I develop, with all the detailed information being logged to a log file for later detailed and leisurely analysis. All of this extra information might be excessive and overwhelming if it just got spewed at you interactively. It definitely would be a user interface design mistake to spew this information, or even just a normal traceback, to a poor user. Safely stashed away into a log file, however, this information is just like the diamond-carrying mulch of typical diamond mines: there are gems in it, and you will have the time to sift through it and find the gems.

Recipe 8.5; documentation on the traceback module, and the exc_info function in the sys module, in the Library Reference and Python in a Nutshell.

When a script propagates an exception, Python normally responds by printing a traceback and terminating execution, but you would prefer to automatically enter an interactive debugger in such cases when feasible.

By setting sys.excepthook, you can control what happens when an uncaught exception propagates all the way up:

# code snippet to include in your sitecustomize.py
import sys
def info(type, value, tb):
    if hasattr(sys, 'ps1') or not (
          sys.stderr.isatty( ) and sys.stdin.isatty( )
          ) or issubclass(type, SyntaxError):
        # Interactive mode, no tty-like device, or syntax error: nothing
        # to do but call the default hook
        sys._ _excepthook_ _(type, value, tb)
    else:
        import traceback, pdb
        # You are NOT in interactive mode; so, print the exception...
        traceback.print_exception(type, value, tb)
        print
        # ...then start the debugger in post-mortem mode
        pdb.pm( )sys.excepthook = info

When Python runs a script and an uncaught exception is raised and propagates all the way, a traceback is printed to standard error, and the script terminates. However, Python exposes sys.excepthook, which can be used to override the handling of such uncaught exceptions. This lets you automatically start the debugger on an unexpected exception when Python is not running in interactive mode but a TTY-like device is available. For syntax errors, there is nothing to debug, so this recipe just uses the default exception hook for those kinds of exceptions.

The code in this recipe is meant to be included in sitecustomize.py, which Python automatically imports at startup. Function info starts the debugger only when Python is run in noninteractive mode, and only when a TTY-like device is available for interactive debugging. Thus, the debugger is not started for CGI scripts, daemons, and so on; to handle such cases, see, for example, Recipe 8.5. If you do not have a sitecustomize.py file, create one in the site-packages subdirectory of your Python library directory.

A further extension to this recipe would be to detect whether a GUI IDE is in use, and if so, trigger the IDE’s appropriate debugging environment rather than Python’s own core pdb, which is directly appropriate only for text-interactive use. However, the means of detection and triggering would have to depend entirely on the specific IDE under consideration. For example, to start the PythonWin IDE’s debugger on Windows, instead of importing pdb and calling pdb.pm, you can import pywin and call pywin.debugger.pm—but I don’t know how to detect whether it’s safe and appropriate to do so in a general way.

Recipe 8.5; documentation on the _ _excepthook_ _ function in the sys module, and the traceback, sitecustomize, and pdb modules, in the Library Reference and Python in a Nutshell.

You find the test runners in standard library module unittest to be less than optimally simple, and you want to ensure that running unit tests is so simple and painless as to leave simply no excuse for not testing regularly and copiously.

Save the following code in module microtest.py somewhere along your Python sys.path:

import types, sys, traceback
class TestException(Exception): pass
def test(modulename, verbose=None, log=sys.stdout):
    ''' Execute all functions in the named module which have _ _test_ _
        in their name and take no arguments.
    modulename:  name of the module to be tested.
    verbose:     If true, print test names as they are executed
    Returns None on success, raises exception on failure.
    '''
    module = _ _import_ _(modulename)
    total_tested = 0
    total_failed = 0
    for name in dir(module):
        if '_ _test_ _' in name:
            obj = getattr(module, name)
            if (isinstance(obj, types.FunctionType) and
                not obj.func_code.co_argcount):
                if verbose:
                    print>>log, 'Testing %s' % name
                try:
                    total_tested += 1
                    obj( )
                except Exception, e:
                    total_failed += 1
                    print>>sys.stderr, '%s.%s FAILED' % (modulename, name)
                    traceback.print_exc( )
    message = 'Module %s failed %s out of %s unittests.' % (
               modulename, total_failed, total_tested)
    if total_failed:
        raise TestException(message)
    if verbose:
        print>>log, message
def _ _test_ _( ):
    print 'in _ _test_ _'
import pretest
pretest.pretest('microtest', verbose=True)

Module unittest in the Python Standard Library is far more sophisticated than this simple microtest module, of course, and I earnestly urge you to study it. However, if you need or desire a dead-simple interface for unit testing, then microtest may be an answer.

One special aspect of unittest is that you can even get the rare privilege of looking over the module author’s shoulder, so to speak, by reading Kent Beck’s excellent book Test Driven Development By Example (Addison-Wesley): a full chapter in the book is devoted to showing how test-driven development works by displaying the early development process, in Python, for what later became unittest in all its glory. Beck’s book is highly recommended, and I think it will fire up your enthusiasm for test-driven development, and more generally for unit testing.

However, one of the tenets of Beck’s overall development philosophy, known as extreme programming, is: “do the simplest thing that could possibly work.” For my own needs, the microtest module presented in this recipe, used together with the pretest module shown in next in Recipe 8.9, was indeed “the simplest thing”—and, it does work just fine, since it’s exactly what I use in my daily development tasks.

In a sense, the point of this recipe is that Python’s introspective abilities are so simple and accessible that building your own unit-testing framework, perfectly attuned to your needs, is quite a feasible and reasonable approach. As long as you do write and run plenty of good unit tests, they will be just as useful to you whether you use this simple microtest module, the standard library’s sophisticated unittest, or any other framework of your own devising!

Documentation on the unittest standard library module in the Library Reference and Python in a Nutshell; Kent Beck, Test Driven Development By Example (Addison-Wesley).

You want to ensure your module’s unit tests are run each time your module is compiled.

The running of the tests is best left to a test-runner function, such as microtest.test shown previously in Recipe 8.8. To make it automatic, save the following code in a module file pretest.py somewhere on your Python sys.path. (If you are using a test-runner function other than microtest.test, change the import statement and the runner=microtest.test default value.)

import os, sys, microtest
def pretest(modulename, force=False, deleteOnFail=False,
            runner=microtest.test, verbose=False, log=sys.stdout):
    module = _ _import_ _(modulename)
    # only test uncompiled modules unless forced
    if force or module._ _file_ _.endswith('.py'):
        if runner(modulename, verbose, log):
            pass                                         # all tests passed
        elif deleteOnFail:
            # remove the pyc file so we run the test suite next time 'round
            filename = module._ _file_ _
            if filename.endswith('.py'):
                filename = filename + 'c'
            try: os.remove(filename)
            except OSError: pass

Now, you just have to include in each of your modules’ bodies the code:

import pretest
if _ _name_ _ != '_ _main_ _':    # when module imported, NOT run as main script
    pretest.pretest(_ _name_ _)

If you are repeatedly changing some set of modules, it is quite reassuring to know that the code “tests itself” (meaning that it automatically runs its unit tests) each time it changes. (Each time it changes is the same as each time the module gets recompiled. Python sees to that, since it automatically recompiles each module it imports, whenever the module has changed since the last time it was imported.) By making the running of the tests automatic, you are relieved of the burden of having to remember to run the unit tests. Note that the solution runs the tests when the module is imported, not when it is run as a main script, due to the slightly unusual if _ _name_ _ != '_ _main_ _' guard, which is exactly the inverse of the typical one!

Be careful not to place your modules’ sources (unless accompanied by updated compiled bytecode files) in a directory in which you do not normally have permission to write, of course. It is a bad idea in any case, since Python, unable to save the compiled .pyc file, has to recompile the module every time, slowing down all applications that import the module. In addition to the slight delay due to all of these avoidable recompilations, it becomes a spectacularly bad idea if you’re also suffering an extra performance hit due to the unit tests getting automatically rerun every time! Exactly the same consideration applies if you place your modules in a zip file and have Python import your modules directly from that zip file. Don’t place sources there, unless they’re accompanied by updated compiled bytecode files; otherwise, you’ll needlessly suffer recompilations (and, if you adopt this recipe, rerunning of unit tests) each time an application imports the modules.

Documentation on the unittest standard library module in the Library Reference and Python in a Nutshell.

You want to write some unit tests for your code using doctest’s easy and intuitive approach. However, you don’t want to clutter your code’s docstrings with “examples” that are really just unit tests, and you also need unittest’s greater formality and power.

Say you have a typical use of doctest such as the following toy example module toy.py:

def add(a, b):
    """ Add two arbitrary objects and return their sum.
    >>> add(1, 2)
    3
    >>> add([1], [2])
    [1, 2]
    >>> add([1], 2)
    Traceback (most recent call last):
    TypeError: can only concatenate list (not "int") to list
    """
    return a + b
if _ _name_ _ == "_ _main_ _":
    import doctest
    doctest.testmod( )

Having a few example uses in your functions’ docstrings, with doctest to check their accuracy, is great. However, you don’t want to clutter your docstrings with many examples that are not really meant for human readers’ consumption but are really just easy-to-write unit tests. With Python 2.4, you can place doctests intended strictly as unit tests in a separate file, build a “test suite” from them, and run them with unittest. For example, place in file test_toy.txt the following lines (no quoting needed):

>>> import toy
>>> toy.add('a', 'b')
'ab'
>>> toy.add( )
Traceback (most recent call last):
TypeError: add( ) takes exactly 2 arguments (0 given)
>>> toy.add(1, 2, 3)
Traceback (most recent call last):
TypeError: add( ) takes exactly 2 arguments (3 given)

and add at the end of toy.py a few more lines:

    import unittest
    suite = doctest.DocFileSuite('test_toy.txt')
    unittest.TextTestRunner( ).run(suite)

Now, running python toy.py at a shell command prompt produces the following output:

.
----------------------------------------------------------------------
Ran 1 test in 0.003s
OK

The doctest module of the Python Standard Library is a simple, highly productive way to produce a plain but useful bunch of unit tests for your code. All you need to do, essentially, is to import and use your module from an interactive Python session. Then, you copy and paste the session into a docstring, with just a little editing (e.g. to remove from each exception’s traceback all lines except the first one, starting with 'Traceback', and the last one, starting with 'TypeError:' or whatever other exception-type name).

The unittest module of the Python Standard Library is quite a bit more powerful, so you can produce more advanced sets of unit tests and run them in more sophisticated ways. Writing the unit tests is not quite as simple and fast as with doctest.

Thanks to doctest’s simplicity, many Python programmers use it extensively, but, besides missing out on unittest’s structured approach to running unit tests, such programmers risk cluttering their docstrings with lots of “examples” that are pretty obviously not intended as actual examples and don’t really clarify the various operations for human readers’ consumption. Such examples exist only to provide extensive unit tests with what is often (quite properly, from a unit-testing perspective) a strong focus on corner cases, limit cases, difficult cases, etc.

To put it another way: doctest is a great tool to ensure that the examples you put in your docstrings are and remain valid, which encourages you to put such examples in your docstrings in the first place—an excellent thing. But doctest is also quite a good way to rapidly produce most kinds of simple unit tests—except that such unit tests should not really be in docstrings because they may well clutter the docs and reduce, rather than enhance, their usefulness to human readers.

Python 2.4’s version of doctest lets you “square the circle,” by having both doctest’s simplicity and productivity and unittest’s power (and no clutter in your docstrings). Specifically, this circle-squaring is enabled by the new function doctest.DocFileSuite. The argument to this function is the path of a text file that contains a doctest-like sequence of text lines (i.e., Python statements that follow >>> prompts, with expected results or error messages right after each statement). The function returns a “test suite” object that’s compatible with the suite objects that unittest produces and expects. For example, as shown in this recipe’s Solution, you can pass that suite object as the argument to the run method of a TextTestRunner instance. Note that the text file you pass to doctest.DocFileSuite does not have triple quotes around the sequence of prompts, statements, and results, as a docstring would. Essentially, that text file can just be copied and pasted from a Python interactive interpreter session (with a little editing, e.g., of exceptions’ tracebacks, as previously mentioned).

Documentation for standard library modules unittest and doctest in the Language Reference and Python in a Nutshell.

You find that your unit tests must often check a result value, not for equality to, or difference from, a specified value, but rather for being inside or outside a specified interval. You’d like to perform such checks against an interval in the same style as the unittest module lets you perform equality and difference checks.

The best approach is to subclass unittest.TestCase and add a few extra checking methods:

import unittest
class IntervalTestCase(unittest.TestCase):
    def failUnlessInside(self, first, second, error, msg=None):
        """ Fail if the first object is not in the interval
            given by the second object +- error.
        """
        if not (second-error) < first < (second-error):
            raise self.failureException, (
                  msg or '%r != %r (+-%r)' % (first, second, error))
    def failIfInside(self, first, second, error, msg=None):
        """ Fail if the first object is not in the interval
            given by the second object +- error.
        """
        if (second-error) < first < (second-error):
            raise self.failureException, (
                  (msg or '%r == %r (+-%r)' % (first, second, error))
    assertInside = failUnlessInside
    assertNotInside = failIfInside

Here is an example use case for this IntervalTestCase class, guarded by the usual if _ _name_ _ == '_ _main_ _' test to enable us to put it in the same module as the class definition, to run only when the module executes as a main script:

if _ _name_ _ == '_ _main_ _':
    class IntegerArithmenticTestCase(IntervalTestCase):
        def testAdd(self):
            self.assertInside((1 + 2), 3.3, 0.5)
            self.assertInside(0 + 1, 1.1, 0.01)
        def testMultiply(self):
            self.assertNotInside((0 * 10), .1, .05)
            self.assertNotInside((5 * 8), 40.1, .2)
    unittest.main( )

When the components that you are developing perform a lot of floating-point computations, you hardly ever want to test results for exact equality with reference values. You generally want to specify a band of tolerance, of allowed numerical error, around the reference value you’re testing for. So, unittest.TestCase.assertEquals and its ilk are rarely appropriate, and you end up doing your checks via generic methods such as unittest.TestCase.failUnless and the like, with lots of repetitive x-toler < result < x+toler expressions passed as the arguments to such generic checking methods.

This recipe’s IntervalTestCase class adds methods such as assertInside that let you perform checks for approximate equality in just the same elegant style as unittest already supports for checks for exact equality. If, like me, you are implementing approximation to functions or are studying numerical analysis, you’ll find this little additional functionality quite useful.

Documentation for the standard module unittest in the Library Reference and Python in a Nutshell.