In Python programs, exceptions are typically used for a variety of purposes. Here are some of their most common roles:

We’ll see such typical uses in action later in this part of the book. For now, let’s get started with a look at Python’s exception-processing tools.

When you write try statements, a variety of clauses can appear after the try statement block. Table 27-1 summarizes all the possible forms—you must use at least one. We’ve already met some of these: as you know, except clauses catch exceptions, finally clauses run on the way out, and else clauses run if no exceptions are encountered. Syntactically, there may be any number of except clauses, but there should be only one else. Through Python 2.4, the finally clause must appear alone (without else or except); it’s really a different statement. As of Python 2.5, however, a finally can appear in the same statement as except and else.

We’ll explore the entries with the extra value part when we meet the raise statement. The first and fourth entries in Table 27-1 are new here:

Because Python looks for a match within a given try by inspecting except clauses from top to bottom, the parenthesized version is like listing each exception in its own except clause, but the statement body needs to be coded only once. Here’s an example of multiple except clauses at work, which demonstrates just how specific your handlers can be:

try:
    action(  )
except NameError:
    ...
except IndexError
    ...
except KeyError:
    ...
except (AttributeError, TypeError, SyntaxError):
    ...
else:
    ...

In this example, if an exception is raised while the call to the action function is running, Python returns to the try, and searches for the first except that names the exception raised. It inspects except clauses from top to bottom and left to right, and runs the statements under the first one that matches. If none match, the exception is propagated past this try. Note that the else runs only when no exception occurs in action—it does not run when an exception without a matching except is raised.

If you really want a general “catch-all” clause, an empty except does the trick:

try:
    action(  )
except NameError:
    ...                   # Handle NameError
except IndexError:
    ...                   # Handle IndexError
except:
    ...                   # Handle all other exceptions
else:
    ...                   # Handle the no-exception case

The empty except clause is a sort of wildcard feature—because it catches everything, it allows your handlers to be as general or specific as you like. In some scenarios, this form may be more convenient than listing all possible exceptions in a try. For example, the following catches everything without listing anything:

try:
    action(  )
except:
    ...                   # Catch all possible exceptions

Empty excepts also raise some design issues, though. Although convenient, they may catch unexpected system exceptions unrelated to your code, and may inadvertently intercept exceptions meant for another handler. For example, even system exit calls in Python trigger exceptions, and you usually want these to pass. We’ll revisit this as a gotcha at the end of this part of the book. For now, I’ll just say: use with care.

At first glance, the purpose of the else clause is not always obvious to Python newcomers. Without it, though, there is no way to tell (without setting and checking Boolean flags) whether the flow of control has proceeded past a try statement because no exception was raised, or because an exception occurred and was handled:

try:
    ...run code...
except IndexError:
    ...handle exception...
# Did we get here because the try failed or not?

Much like the way else clauses in loops make the exit cause more apparent, the else clause provides syntax in a try that makes what has happened obvious and unambiguous:

try:
    ...run code...
except IndexError:
    ...handle exception...
else:
    ...no exception occurred...

You can almost emulate an else clause by moving its code into the try block:

try:
    ...run code...
    ...no exception occurred...
except IndexError:
    ...handle exception...

This can lead to incorrect exception classifications, though. If the “no exception occurred” action triggers an IndexError, it will register as a failure of the try block, and hence erroneously trigger the exception handler below the try (subtle, but true!). By using an explicit else clause instead, you make the logic more obvious, and guarantee that except handlers will run only for real failures in the code you’re wrapping in a try, not for failures in the else case’s action.

Because the control flow through a program is easier to capture in Python than in English, let’s run some examples that further illustrate exception basics. I’ve mentioned that exceptions not caught by try statements percolate up to the top level of the Python process, and run Python’s default exception-handling logic (i.e., Python terminates the running program, and prints a standard error message). Let’s look at an example. Running the following module, bad.py, generates a divide-by-zero exception:

def gobad(x, y):
    return x / y

def gosouth(x):
    print gobad(x, 0)

gosouth(1)

Because the program ignores the exception it triggers, Python kills the program, and prints a message:^[75]

% python bad.py
Traceback (most recent call last):
  File "bad.py", line 7, in <module>
    gosouth(1)
  File "bad.py", line 5, in gosouth
    print gobad(x, 0)
  File "bad.py", line 2, in gobad
    return x / y
ZeroDivisionError: integer division or modulo by zero

The message consists of a stack trace and the name of (and any extra data about) the exception that was raised. The stack trace lists all lines active when the exception occurred, from oldest to newest. Note that because we’re not working at the interactive prompt, in this case, the file and line number information is useful. For example, here we can see that the bad divide happens at the last entry in the trace—line 2 of the file bad.py, a return statement.

Because Python detects and reports all errors at runtime by raising exceptions, exceptions are intimately bound up with the ideas of error handling and debugging in general. If you’ve worked through this book’s examples, you’ve undoubtedly seen an exception or two along the way—even typos usually generate a SyntaxError or other exception when a file is imported or executed (that’s when the compiler is run). By default, you get a useful error display like the one above, which helps you track down the problem.

Often, this standard error message is all you need to resolve problems in your code. For more heavy-duty debugging jobs, you can catch exceptions with try statements, or use debugging tools I’ll introduce in Chapter 29, such as the pdb standard library module.

Python’s default exception handling is often exactly what you want—especially for code in a top-level script file, an error generally should terminate your program immediately. For many programs, there is no need to be more specific about errors in your code.

Sometimes, though, you’ll want to catch errors and recover from them instead. If you don’t want your program terminated when Python raises an exception, simply catch it by wrapping the program logic in a try. This is an important capability for programs such as network servers, which must keep running persistently. For example, the following code catches and recovers from the TypeError Python raises immediately when you try to concatenate a list and a string (the + operator wants the same sequence type on both sides):

def kaboom(x, y):
    print x + y                # Trigger TypeError

try:
    kaboom([0,1,2], "spam")
except TypeError:              # Catch and recover here
    print 'Hello world!'
print 'resuming here'          # Continue here if exception or not

When the exception occurs in the function kaboom, control jumps to the try statement’s except clause, which prints a message. Since an exception is “dead” after it’s been caught like this, the program continues executing below the try, rather than being terminated by Python. In effect, the code processes and clears the error.

Notice that once you’ve caught an error, control resumes at the place where you caught it (i.e., after the try); there is no direct way to go back to the place where the exception occurred (here, in the function kaboom). In a sense, this makes exceptions more like simple jumps than function calls—there is no way to return to the code that triggered the error.

We saw some simple try/finally examples earlier. Here’s a more realistic example that illustrates a typical role for this statement:

class MyError(Exception): pass

def stuff(file):
    raise MyError(  )

file = open('data', 'w')     # Open an output file
try:
    stuff(file)              # Raises exception
finally:
    file.close(  )           # Always close file to flush output buffers
...                          # Continue here only if no exception

In this code, we’ve wrapped a call to a file-processing function in a try with a finally clause to make sure that the file is always closed, and thus finalized, whether the function triggers an exception or not. This way, later code can be sure that the file’s output buffer’s content has been flush from memory to disk. A similar code structure can guarantee that server connections are closed, and so on.

This particular example’s function isn’t all that useful (it just raises an exception), but wrapping calls in try/finally statements is a good way to ensure that your closing-time (i.e., termination) activities always run. Again, Python always runs the code in your finally blocks, regardless of whether an exception happens in the try block.^[76]

When the function here raises its exception, the control flow jumps back, and runs the finally block to close the file. The exception is then propagated on to either another try or the default top-level handler, which prints the standard error message and shuts down the program; the statement after this try is never reached. If the function here did not raise an exception, the program would still execute the finally block to close your file, but it would then continue below the entire try statement.

Also, notice that the user-defined exception here is again defined with a class—as we’ll see in the next chapter, exceptions today should all be class instances.

Prior to Python 2.5, it is actually possible to combine finally and except clauses in a try by syntactically nesting a try/except in the try block of a try/finally statement (we’ll explore this technique more fully in Chapter 29). In fact, the following has the same effect as the new merged form shown in the previous section:

try:
    try:
        main-action
    except Exception1:
        handler1
    except Exception2:
        handler2
    ...
    else:
        no-error
finally:
    clean-up

Again, the finally block is always run on the way out, regardless of what happened in the main action, and regardless of any exception handlers run in the nested try (trace through the four cases listed previously to see how this works the same). However, this equivalent is more obscure, and requires more code than the new merged form. Mixing finally into the same statement is easier to write and read, and so is the preferred technique today.

Here’s a demonstration of the merged try statement form at work. The following codes four common scenarios, with print statements that describe the meaning of each:

print '-' * 30, '
EXCEPTION RAISED AND CAUGHT'
try:
    x = 'spam'[99]
except IndexError:
    print 'except run'
finally:
    print 'finally run'
print 'after run'

print '-' * 30, '
NO EXCEPTION RAISED'
try:
    x = 'spam'[3]
except IndexError:
    print 'except run'
finally:
    print 'finally run'
print 'after run'

print '-' * 30, '
NO EXCEPTION RAISED, ELSE RUN'
try:
    x = 'spam'[3]
except IndexError:
    print 'except run'
else:
    print 'else run'
finally:
    print 'finally run'
print 'after run'

print '-' * 30, '
EXCEPTION RAISED BUT NOT CAUGHT'
try:
    x = 1 / 0
except IndexError:
    print 'except run'
finally:
    print 'finally run'
print 'after run'

When this code is run, the following output is produced. Trace through the code to see how exception handling produces each of the test’s outputs here:

------------------------------
EXCEPTION RAISED AND CAUGHT
except run
finally run
after run
------------------------------
NO EXCEPTION RAISED
finally run
after run
------------------------------
NO EXCEPTION RAISED, ELSE RUN
else run
finally run
after run
------------------------------
EXCEPTION RAISED BUT NOT CAUGHT
finally run

Traceback (most recent call last):
  File "C:/Python25/mergedexc.py", line 32, in <module>
    x = 1 / 0
ZeroDivisionError: integer division or modulo by zero

This example uses built-in operations in the main action to trigger exceptions or not, and relies on the fact that Python always checks for errors as our code is running. The next section shows how to raise exceptions manually instead.

Python programs can trigger built-in and user-defined exceptions using the raise statement. User-defined exceptions should be class instance objects today, like the one that calling MyBad creates in the following code:

class MyBad: pass

def stuff(  ):
    raise MyBad(  )            # Trigger exception manually

try:
    stuff(  )                  # Raises exception
except MyBad:
    print 'got it'             # Handle exception here
...                            # Resume execution here

This time the raise occurs inside a function, but this makes no real difference—control jumps back to the except block immediately. Notice that user-defined exceptions are caught with try statements, just like built-in exceptions.

As stated earlier, a raise statement can pass an extra data item along with the exception for use in a handler. In general, the extra data allows you to send context information about the exception to a handler. If you’re writing a data file parser, for example, you might raise a syntax error exception on errors and also pass along an object that gives line and file information to the handler (we’ll meet such an example in Chapter 28).

This is useful because when an exception is raised, it may cross arbitrary file boundaries—the raise statement that triggers an exception, and the try statement that catches it may be in completely different files. It is not generally feasible to store extra details in global variables because the try statement might not know which file the globals reside in. Passing extra data along with the exception itself enables the try statement to access it more reliably. Strictly speaking, every exception has this extra data: as with function return values, it defaults to the special None object if nothing is passed explicitly. The following code, raisedata.py, illustrates this concept at work with simple string-based exceptions:

myException = 'Error'                 # String object

def raiser1(  ):
    raise myException, "hello"        # Raise, pass data

def raiser2(  ):
    raise myException                 # Raise, None implied

def tryer(func):
    try:
        func(  )
    except myException, extraInfo:    # Run func; catch exception + data
        print 'got this:', extraInfo

% python
>>> from raisedata import *
>>> tryer(raiser1)# Explicitly passed extra data
got this: hello
>>> tryer(raiser2)# Extra data is None by default
got this: None

Here, the tryer function always requests the extra data object; it comes back as an explicit string from raiser1, but defaults to None in raiser2’s raise statement.

In the next chapter, we’ll see that the same hook can also be used to access instances raised in conjunction with class-based exceptions—the variable in the except is then assigned to the raised instance, which gives access to attached state information, as well as callable class methods.

A raise statement that does not include an exception name or extra data value simply reraises the current exception. This form is typically used if you need to catch and handle an exception, but don’t want the exception to die in your code:

>>> try:
...     raise IndexError, 'spam'
... except IndexError:
...     print 'propagating'
...     raise
...
propagating
Traceback (most recent call last):
  File "<stdin>", line 2, in ?
IndexError: spam

Running a raise this way reraises the exception, and propagates it to a higher handler, or the default handler at the top, which stops the program with a standard error message.

Assertions are typically used to verify program conditions during development. When displayed, their error message text automatically includes source code line information, and the value listed in the assert statement. Consider the file asserter.py:

def f(x):
    assert x < 0, 'x must be negative'
    return x ** 2

% python
>>> import asserter
>>> asserter.f(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "asserter.py", line 2, in f
    assert x < 0, 'x must be negative'
AssertionError: x must be negative

It’s important to keep in mind that assert is mostly intended for trapping user-defined constraints, not for catching genuine programming errors. Because Python traps programming errors itself, there is usually no need to code asserts to catch things like out-of-bounds indexes, type mismatches, and zero divides:

def reciprocal(x):
    assert x != 0              # A useless assert!
    return 1 / x               # Python checks for zero automatically

Such asserts are generally superfluous—because Python raises exceptions on errors automatically, you might as well let it do the job for you.^[77] For another example of common assert usage, see the abstract superclass example in Chapter 24; there, we used assert to make calls to undefined methods fail with a message.

This feature will not become an official part of Python until version 2.6. In Python 2.5, it is not yet available by default; it must be enabled with the special future import statement form we met in the modules part of this book (because of the two new reserved words with and as, this feature is being introduced gradually, as usual):

from _  _future_  _ import with_statement

When you run this import statement in 2.5, you enable the new with statement, and its two reserved words. The basic format of the with statement looks like this:

with expression [as variable]:
    with-block

The expression here is assumed to return an object that supports the context management protocol (more on this protocol in a moment). This object may also return a value that will be assigned to the name variable if the optional as clause is present.

Note that the variable is not assigned the result of the expression; the result of the expression is the object that supports the context protocol, and the variable may be assigned something else. The object returned by the expression may then run startup code before the with-block is started, as well as termination code after the block is done, whether the block raised an exception or not.

Some built-in Python objects have been augmented to support the context management protocol, and so can be used with the with statement. For example, file objects have a context manager that automatically closes the file after the with block, regardless of whether an exception is raised:

with open(r'C:pythonscripts') as myfile:
    for line in myfile:
        print line
        line = line.replace('spam', 'SPAM')
        ...more code here...

Here, the call to open returns a simple file object that is assigned to the name myfile. We can use myfile with the usual file tools—in this case, the file iterator reads line by line in the for loop.

However, this object also supports the context management protocol used by the with statement. After this with statement has run, the context management machinery guarantees that the file object referenced by myfile is automatically closed, even if the for loop raised an exception while processing the file.

We won’t cover Python’s multithreading modules in this book (for more on that topic, see follow-up application-level texts such as Programming Python), but the lock and condition variable synchronization tools they define also support the with statement by supporting the context management protocol:

lock = threading.Lock(  )
with lock:
    # critical section of code
    ...access shared resources...

Here, the context management machinery guarantees that the lock is automatically acquired before the block is executed and released once the block is complete.

The decimal module (see Chapter 5 for more on decimals) also uses context managers to simplify saving and restoring the current decimal context, which specifies the precision and rounding characteristics for calculations.

The interface expected of objects used in with statements is somewhat complex, and many programmers only need to know how to use existing context managers. For tool builders who might want to write new ones, though, let’s take a quick look at what is involved. Here’s how the with statement actually works:

doStuff(  )
{                                   # C program
    if (doFirstThing(  ) == ERROR)  # Detect errors everywhere
        return ERROR;               # even if not handled here
    if (doNextThing(  ) == ERROR)
        return ERROR;
    ...
    return doLastThing(  );
}

main(  )

{
    if (doStuff(  ) == ERROR)
        badEnding(  );
    else
        goodEnding(  );
}

def doStuff(  ):       # Python code
    doFirstThing(  )   # We don't care about exceptions here,
    doNextThing(  )    # so we don't need to detect them
    ...
    doLastThing(  )

if_  _name_  _ == '_  _main_  _':
    try:
        doStuff(  )    # This is where we care about results,
    except:            # so it's the only place we must check
        badEnding(  )
    else:
        goodEnding(  )

Let’s look at a quick demo of the protocol in action. The following defines a context manager object that traces the entry and exit of the with block in any with statement it is used for:

from _  _future_  _ import with_statement         # Required in Python 2.5

class TraceBlock:
    def message(self, arg):
        print 'running', arg
    def _  _enter_  _(self):
        print 'starting with block'
        return self
    def _  _exit_  _(self, exc_type, exc_value, exc_tb):
        if exc_type is None:
            print 'exited normally
'
        else:
            print 'raise an exception!', exc_type
            return False  # propagate

with TraceBlock(  ) as action:
    action.message('test 1')
    print 'reached'

with TraceBlock(  ) as action:
    action.message('test 2')
    raise TypeError
    print 'not reached'

Notice that this class’ _ _exit_ _ method returns False to propagate the exception; deleting the return statement there would have the same effect, as the default None return value of functions is False by definition. Also, notice how the _ _enter_ _ method returns self as the object to assign to the as variable; in other use cases, this might return a completely different object instead.

When run, the context manager traces the entry and exit of the with statement block with its _ _enter_ _ and _ _exit_ _ methods:

% python withas.py
starting with block
running test 1
reached
exited normally

starting with block
running test 2
raise an exception! <type 'exceptions.TypeError'>

Traceback (most recent call last):
  File "C:/Python25/withas.py", line 22, in <module>
    raise TypeError
TypeError

Context managers are somewhat advanced devices that are not yet officially part of Python, so we’ll skip additional details here (see Python’s standard manuals for the full story—for example, a new contextlib standard module provides additional tools for coding context managers). For simpler purposes, the try/finally statement provides sufficient support for termination-time activities.