Before you started writing functions, all code was written at the top-level of a module (i.e., not nested in a def), so the names either lived in the module itself, or were built-ins that Python predefines (e.g., open).^[1] Functions provide a nested namespace (i.e., a scope), which localizes the names they use, such that names inside the function won’t clash with those outside (in a module or other function). Functions define a local scope, and modules define a global scope. The two scopes are related as follows:

Note that any type of assignment within a function classifies a name as local: = statements, imports, defs, argument passing, and so on. Also notice that in-place changes to objects do not classify names as locals; only actual name assignments do. For instance, if name L is assigned to a list at the top level of a module, a statement like L.append(X) within a function will not classify L as a local, whereas L = X will. In the former case, L will be found in the global scope as usual, and change the global list.

If the prior section sounds confusing, it really boils down to three simple rules:

In other words, all names assigned inside a function def statement (or lambda—an expression we’ll meet later) are locals by default; functions can use both names in lexically (i.e., physically) enclosing functions and the global scope, but they must declare globals to change them. Python’s name resolution is sometimes called the LEGB rule, after the scope names:

Figure 13-1 illustrates Python’s four scopes. Note that the second “E” scope lookup layer—enclosing defs or lambdas—technically can correspond to more than one lookup layer. It only comes into play when you nest functions within functions.^[2]

Figure 13-1. The LEGB scope lookup rule

Also, keep in mind that these rules only apply to simple variable names (such as spam). In Parts Part V and Part VI, we’ll see that the rules for qualified attribute names (such as object.spam) live in a particular object and follow a completely different set of lookup rules than the scope ideas covered here. Attribute references (names following periods) search one or more objects, not scopes, and may invoke something called inheritance discussed in Part VI.

Let’s look at an example that demonstrates scope ideas. Suppose we write the following code in a module file:

# Global scope
X = 99                # X and func assigned in module: global
          
def func(Y):          # Y and Z assigned in function: locals
    # local scope
    Z = X + Y         # X is a global.
    return Z

func(1)               # func in module: result=100

This module, and the function it contains, use a number of names to do their business. Using Python’s scope rules, we can classify the names as follows:

The whole point behind this name segregation scheme is that local variables serve as temporary names you need only while a function is running. For instance, the argument Y and the addition result Z exist only inside the function; these names don’t interfere with the enclosing module’s namespace (or any other function, for that matter).

The local/global distinction also makes a function easier to understand; most of the names it uses appear in the function itself, not at some arbitrary place in a module. Because you can be sure that local names are not changed by some remote function in your program, they also tend to make programs easier to debug.

We’ve been talking about the built-in scope in the abstract, but it’s a bit simpler than you may think. Really, the built-in scope is just a prebuilt standard library module called __builtin__, which you can import and inspect if you want to see which names are predefined:

>>> import __builtin__
>>> dir(__builtin__)
['ArithmeticError', 'AssertionError', 'AttributeError', 'DeprecationWarning', 
'EOFError', 'Ellipsis', 
    ...many more names ommitted...
'str', 'super', 'tuple', 'type', 'unichr', 'unicode', 
'vars', 'xrange', 'zip']

The names in this list are the built-in scope in Python; roughly the first half are built-in exceptions, and the second are built-in functions. Because Python automatically searches this module last in its LEGB lookup rule, you get all the names in this list for free—they can be used without importing any module. In fact, there are two ways to refer to a built-in function: by the LEGB rule, or by manually importing:

>>> zip                        # The normal way
<built-in function zip>

>>> import __builtin__      # The hard way
>>> __builtin__.zip
<built-in function zip>

The second of these is sometimes useful in advanced work. The careful reader might also notice that, because the LEGB lookup procedure takes the first occurrence of a name that it finds, names in the local scope may override variables of the same name in both the global and built-in scopes, and global names may override built-ins. A function can, for instance, create a local variable called open by assigning to it:

def hider(  ):
    open = 'spam'     # Local variable, hides built-in
    ...

However, this will hide the built-in function called open that lives in the built-in (outer) scope. It’s also usually a bug, and a nasty one at that, because Python will not issue a message about it—there are times in advanced programming where you may really want to replace a built-in name by redefining it in your code.

Functions can similary hide global variables of the same name with locals:

X = 88          # Global X

def func(  ):
    X = 99      # Local X: hides global

func(  )
print X         # Prints 88: unchanged

Here, the assignment within the function creates a local X that is a competely different variable than the global X in the module outside the function. Because of this, there is no way to change a name outside the function, without adding a global declaration to the def—as described in the next section.

With the addition of nested function scopes, variable lookup rules become slightly more complex. Within a function:

Notice that the global declaration still maps variables to the enclosing module. When nested functions are present, variables in enclosing functions may only be referenced, not changed. Let’s illustrate all this with some real code.

Here is an example of a nested scope:

def f1(  ):
    x = 88
    def f2(  ):
        print x
    f2(  )

f1(  )                  # Prints 88

First off, this is legal Python code: the def is simply an executable statement that can appear anywhere any other statement can—including nested in another def. Here, the nested def runs while a call to function f1 is running; it generates a function and assigns it to name f2, a local variable within f1’s local scope. In a sense, f2 is a temporary function, that only lives during the execution of (and is only visible to code in) the enclosing f1.

But notice what happens inside f2: when it prints variable x, it refers to the x that lives in the enclosing f1 function’s local scope. Because functions can access names in all physically enclosing def statements, the x in f2 is automatically mapped to the x in f1, by the LEGB lookup rule.

This enclosing scope lookup works even if the enclosing function has already returned. For example, the following code defines a function that makes and returns another:

def f1(  ):
    x = 88
    def f2(  ):
        print x
    return f2

action = f1(  )            # Make, return function.
action(  )                 # Call it now: prints 88

In this code, the call to action is really running the function we named f2 when f1 ran. f2 remembers the enclosing scope’s x in f1, even though f1 is no longer active. This sort of behavior is also sometimes called a closure—an object that remembers values in enclosing scopes, even though those scopes may not be around any more. Although classes (described in Part VI) are usually best at remembering state, such functions provide another alternative.

In earlier versions of Python, all this sort of code failed, because nested defs did not do anything about scopes—a reference to a variable within f2 would search local (f2), then global (the code outside f1), and then built-in. Because it skipped the scopes of enclosing functions, an error would result. To work around this, programmers typically used default argument values to pass-in (remember) the objects in an enclosing scope:

def f1(  ):
    x = 88
    def f2(x=x):
        print x
    f2(  )

f1(  )                  # Prints 88

This code works in all Python releases, and you’ll still see this pattern in much existing Python code. We’ll meet defaults in more detail later in this chapter; in short, the syntax arg=val in a def header means that argument arg will default to value val, if no real value is passed to arg in a call.

In the modified f2, the x=x means that argument x will default to the value of x in the enclosing scope—because the second x is evaluated before Python steps into the nested def, it still refers to the x in f1. In effect, the default remembers what x was in f1, the object 88.

That’s fairly complex, and depends entirely on the timing of default value evaluations. That’s also why the nested scope lookup rule was added to Python, to make defaults unnecessary for this role. Today, Python automatically remembers any values required in the enclosing scope, for use in nested defs.

Of course, the best prescription here is probably just don’t do that. Programs are much simpler if you do not nest defs within defs. Here’s an equivalent of the prior example, which banishes the notion of nesting; notice that it’s okay to call a function (see following) defined after the one that contains the call like this, as long as the second def runs before the call of the first function—code inside a def is never evaluated until the function is actually called:

>>> def f1(  ):
...     x = 88
...     f2(x)
...
>>> def f2(x):
...     print x
...
>>> f1(  )
88

If you avoid nesting this way, you can almost forget about the nested scopes concept in Python, at least for defs. However, you are even more likely to care about such things when you start coding lambda expressions. We also won’t meet lambda in depth until Chapter 14; in short, lambda is an expression that generates a new function to be called later, much like a def statement (because it’s an expression, it can be used in places that def cannot, such as within list and dictionary literals).

Also like a def, lambda expressions introduce a new local scope. With the enclosing scopes lookup layer, they can see all the variables that live in the function in which they are coded. The following works today, only because the nested scope rules are now applied:

def func(  ):
    x = 4
    action = (lambda n: x ** n)          # x in enclosing def
    return action

x = func(  )
print x(2) # Prints 16

Prior to the introduction of nested function scopes, programmers used defaults to pass values from an enclosing scope into lambdas, as for defs. For instance, the following works on all Python releases:

def func(  ):
    x = 4
    action = (lambda n, x=x: x ** n)     # Pass x in manually.

Because lambdas are expressions, they naturally (and even normally) nest inside enclosing defs. Hence, they are perhaps the biggest beneficiary of the addition of enclosing function scopes in the lookup rules; in most cases, it is no longer necesary to pass values into lambdas with defaults. We’ll have more to say about both defaults and lambdas later, so you may want to return and review this section later.

Before ending this discussion, note that scopes nest arbitrarily, but only enclosing functions (not classes, described in Part VI) are searched:

>>> def f1(  ):
...     x = 99
...     def f2(  ):
...         def f3(  ):
...             print x       # Found in f1's local scope!
...         f3(  )
...     f2(  )
...
>>> f1(  )
99

Python will search the local scopes of all enclosing defs, after the referencing function’s local scope, and before the module’s global scope. However, this sort of code seems less likely in practice.

Here’s an example that illustrates some of these properties at work:

>>> def changer(x, y):     # Function
...    x = 2               # Changes local name's value only
...    y[0] = 'spam'       # Changes shared object in place
...
>>> X = 1
>>> L = [1, 2]             # Caller
>>> changer(X, L)          # Pass immutable and mutable
>>> X, L                   # X unchanged, L is different
(1, ['spam', 2])

In this code, the changer function assigns to argument name x and a component in the object referenced by argument y. The two assignments within the function are only slightly different in syntax, but have radically different results:

Figure 13-2 illustrates the name/object bindings that exist immediately after the function has been called, and before its code has run.

Figure 13-2. References: arguments share objects with the caller

If you recall some of the discussion about shared mutable objects in Chapter 4 and Chapter 7, you’ll recognize that this is the exact same phenomenon at work: changing a mutable object in-place can impact other references to the object. Here, its effect is to make one of the arguments work like an output of the function.

If you don’t want in-place changes within functions to impact objects you pass to them, simply make explicit copies of mutable objects, as we learned in Chapter 7. For function arguments, we can copy the list at the point of call:

L = [1, 2]
changer(X, L[:])       # Pass a copy, so my L does not change.

We can also copy within the function itself, if we never want to change passed-in objects, regardless of how the function is called:

def changer(x, y):
   y = y[:]            # Copy input list so I don't impact caller.
   x = 2
   y[0] = 'spam'       # Changes my list copy only

Both of these copying schemes don’t stop the function from changing the object—they just prevent those changes from impacting the caller. To really prevent changes, we can always convert to immutable objects to force the issue. Tuples, for example, throw an exception when changes are attempted:

L = [1, 2]
changer(X, tuple(L))   # Pass a tuple, so changes are errors.

This scheme uses the built-in tuple function, which builds a new tuple out of all the items in a sequence. It’s also something of an extreme—because it forces the function to be written to never change passed in arguments, it might impose more limitation on the function than it should, and should often be avoided. You never know when changing arguments might come in handy for other calls in the future. The function will also lose the ability to call any list-specific methods on the argument, even methods that do not change the object in-place.

We’ve already discussed the return statement, and used it in a few examples. But here’s a trick: because return sends back any sort of object, it can return multiple values, by packaging them in a tuple. In fact, although Python doesn’t have what some languages label call-by-reference argument passing, we can usually simulate it by returning tuples, and assigning results back to the original argument names in the caller:

>>> def multiple(x, y):
...     x = 2                    # Changes local names only
...     y = [3, 4]
...     return x, y              # Return new values in a tuple.
...
>>> X = 1
>>> L = [1, 2]
>>> X, L = multiple(X, L)        # Assign results to caller's names.
>>> X, L
(2, [3, 4])

It looks like the code is returning two values here, but it’s just one—a two-item tuple, with the optional surrounding parentheses omitted. After the call returns, use tuple assignment to unpack the parts of the returned tuple. (If you’ve forgotten why, flip back to Section 7.1 in Chapter 7 and Section 8.1 in Chapter 8.)The net effect of this coding pattern is to simulate the output parameters of other languages, by explicit assignments. X and L change after the call, but only because the code said so.

Python matches names by position by default, like most other languages. For instance, if you define a function that requires three arguments, you must call it with three arguments:

>>> def f(a, b, c): print a, b, c
...

Here, we pass them by position: a is matched to 1, b is matched to 2, and so on:

>>> f(1, 2, 3)
1 2 3

In Python, though, you can be more specific about what goes where when you call a function. Keyword arguments allow us to match by name, instead of position:

>>> f(c=3, b=2, a=1)
1 2 3

Here, the c=3 in the call means match this against the argument named c in the header. The net effect of this call is the same as the prior, but notice that the left to right order no longer matters when keywords are used, because arguments are matched by name, not position. It’s even possible to combine positional and keyword arguments in a single call—all positionals are matched first from left to right in the header, before keywords are matched by name:

>>> f(1, c=3, b=2)
1 2 3

When most people see this the first time, they wonder why one would use such a tool. Keywords typically have two roles in Python. First of all, they make your calls a bit more self-documenting (assuming that you use better argument names than a, b, and c). For example, a call of this form:

func(name='Bob', age=40, job='dev')

is much more meaningful than a call with three naked values separated by commas—the keywords serve as labels for the data in the call. The second major use of keywords occurs in conjunction with defaults.

We introduced defaults earlier when discussing nested function scopes. In short, defaults allow us to make selected function arguments optional; if not passed a value, the argument is assigned its default before the function runs. For example, here is a function that requires one argument, and defaults two:

>>> def f(a, b=2, c=3): print a, b, c
...

When called, we must provide a value for a, either by position or keyword; if we don’t pass values to b and c, they default to 2 and 3, respectively:

>>> f(1)
1 2 3
>>> f(a=1)
1 2 3

If we pass two values, only c gets its default; with three values, no defaults are used:

>>> f(1, 4)
1 4 3
>>> f(1, 4, 5)
1 4 5

Finally, here is how the keyword and default features interact: because they subvert the normal left-to-right positional mapping, keywords allow us to essentially skip over arguments with defaults:

>>> f(1, c=6)
1 2 6

Here a gets 1 by position, c gets 6 by keyword, and b, in between, defaults to 2.

The last two matching extensions, * and **, are designed to support functions that take any number of arguments. The first collects unmatched positional arguments into a tuple:

>>> def f(*args): print args

When this function is called, Python collects all the positional arguments into a new tuple, and assigns the variable args to that tuple. Because it is a normal tuple object, it might be indexed, stepped through with a for loop, and so on:

>>> f(  )
(  )
>>> f(1)
(1,)
>>> f(1,2,3,4)
(1, 2, 3, 4)

The ** feature is similar, but only works for keyword arguments—it collects them into a new dictionary, which can then be processed with normal dictionary tools. In a sense, the ** form allows you to convert from keywords to dictionaries:

>>> def f(**args): print args
...
>>> f(  )
{  }
>>> f(a=1, b=2)
{'a': 1, 'b': 2}

Finally, function headers can combine normal arguments, the *, and the **, to implement wildly flexible call signatures:

>>> def f(a, *pargs, **kargs): print a, pargs, kargs
...
>>> f(1, 2, 3, x=1, y=2)
1 (2, 3) {'y': 2, 'x': 1}

In fact, these features can be combined in more complex ways that almost seem ambiguous at first glance—an idea we will revisit later in this chapter.

Here is a slightly larger example that demonstrates both keywords and defaults in action. In the following, the caller must always pass at least two arguments (to match spam and eggs), but the other two are optional; if omitted, Python assigns toast and ham to the defaults specified in the header:

def func(spam, eggs, toast=0, ham=0):   # First 2 required
    print (spam, eggs, toast, ham)

func(1, 2)                              # Output: (1, 2, 0, 0)
func(1, ham=1, eggs=0)                  # Output: (1, 0, 0, 1)
func(spam=1, eggs=0)                    # Output: (1, 0, 0, 0)
func(toast=1, eggs=2, spam=3)           # Output: (3, 2, 1, 0)
func(1, 2, 3, 4)                        # Output: (1, 2, 3, 4)

Notice that when keyword arguments are used in the call, the order in which arguments are listed doesn’t matter; Python matches by name, not position. The caller must supply values for spam and eggs, but they can be matched by position or name. Also notice that the form name=value means different things in the call and def: a keyword in the call, and a default in the header.

To make this more concrete, let’s work through an exercise that demonstrates a practical application for argument matching tools: suppose you want to code a function that is able to compute the minimum value from an arbitrary set of arguments, and an arbitrary set of object datatypes. That is, the function should accept zero or more arguments—as many as you wish to pass. Moreover, the function should work for all kinds of Python object types—numbers, strings, lists, lists of dictionaries, files, even None.

The first part of this provides a natural example of how the * feature can be put to good use—we can collect arguments into a tuple, and step over each in turn with a simple for loop. The second part of this problem definition is easy: because every object type supports comparisons, we don’t have to specialize the function per type (an application of polymorphism ); simply compare objects blindly, and let Python perform the correct sort of comparison.

def min1(*args):
    res = args[0]
    for arg in args[1:]:
        if arg < res:
            res = arg
    return res

def min2(first, *rest):
    for arg in rest:
        if arg < first:
            first = arg
    return first

def min3(*args):
    tmp = list(args)
    tmp.sort(  )
    return tmp[0]

print min1(3,4,1,2)
print min2("bb", "aa")
print min3([2,2], [1,1], [3,3])

C:Python22>python mins.py
2
aa
[1, 1]

def minmax(test, *args):
    res = args[0]
    for arg in args[1:]:
        if test(arg, res):
            res = arg
    return res

def lessthan(x, y): return x < y     # see also: lambda
def grtrthan(x, y): return x > y

print minmax(lessthan, 4, 2, 1, 5, 6, 3)
print minmax(grtrthan, 4, 2, 1, 5, 6, 3)

% python minmax.py
1
6

Here’s a more useful example of special argument-matching modes at work. Earlier in the chapter, we wrote a function that returned the intersection of two sequences (it picked out items that appeared in both). Here is a version that intersects an arbitrary number of sequences (1 or more), by using the varargs matching form *args to collect all arguments passed. Because the arguments come in as a tuple, we can process them in a simple for loop. Just for fun, we’ve also coded an arbitrary-number-arguments union function too; it collects items that appear in any of the operands:

def intersect(*args):
    res = [  ]
    for x in args[0]:                  # Scan first sequence
        for other in args[1:]:         # for all other args.
            if x not in other: break   # Item in each one?
        else:                          # No:  break out of loop
            res.append(x)              # Yes: add items to end
    return res

def union(*args):
    res = [  ]
    for seq in args:                   # For all args
        for x in seq:                  # For all nodes
            if not x in res:
                res.append(x)          # Add new items to result.
    return res

Since these are tools worth reusing (and are too big to retype interactively), we’ve stored the functions in a module file called inter2.py (more on modules in Part V). In both functions, the arguments passed in at the call come in as the args tuple. As in the original intersect, both work on any kind of sequence. Here they are processing strings, mixed types, and more than two sequences:

% python
>>> from inter2 import intersect, union
>>> s1, s2, s3 = "SPAM", "SCAM", "SLAM"

>>> intersect(s1, s2), union(s1, s2)           # Two operands
(['S', 'A', 'M'], ['S', 'P', 'A', 'M', 'C'])

>>> intersect([1,2,3], (1,4))                  # Mixed types 
[1]

>>> intersect(s1, s2, s3)                      # Three operands
['S', 'A', 'M']

>>> union(s1, s2, s3)
['S', 'P', 'A', 'M', 'C', 'L']

If you choose to use and combine the special argument matching modes, Python will ask you to follow two ordering rules:

Moreover, Python internally carries out the following steps to match arguments before assignment:

This is as complicated as it looks, but tracing Python’s matching algorithm helps to understand some cases, especially when modes are mixed. We’ll postpone additional examples of these special matching modes until we do the exercises at the end of Part IV.

from Tkinter import *
widget = Button(text="Press me", command=someFunction)

As you can see, advanced argument matching modes can be complex. They are also entirely optional; you can get by with just simple positional matching, and it’s probably a good idea to do so if you’re just starting out. However, because some Python tools make use of them, they’re important to know in general.