For example, in typical use, this automatically run code may be used to augment calls to functions and classes. It arranges this by installing wrapper objects to be invoked later:

Decorators achieve these effects by automatically rebinding function and class names to other callables, at the end of def and class statements. When later invoked, these callables can perform tasks such as tracing and timing function calls, managing access to class instance attributes, and so on.

Although most examples in this chapter deal with using wrappers to intercept later calls to functions and classes, this is not the only way decorators can be used:

In other words, function decorators can be used to manage both function calls and function objects, and class decorators can be used to manage both class instances and classes themselves. By returning the decorated object itself instead of a wrapper, decorators become a simple post-creation step for functions and classes.

Regardless of the role they play, decorators provide a convenient and explicit way to code tools useful both during program development and in live production systems.

Depending on your job description, you might encounter decorators as a user or a provider. As we’ve seen, Python itself comes with built-in decorators that have specialized roles—static method declaration, property creation, and more. In addition, many popular Python toolkits include decorators to perform tasks such as managing database or user-interface logic. In such cases, we can get by without knowing how the decorators are coded.

For more general tasks, programmers can code arbitrary decorators of their own. For example, function decorators may be used to augment functions with code that adds call tracing, performs argument validity testing during debugging, automatically acquires and releases thread locks, times calls made to function for optimization, and so on. Any behavior you can imagine adding to a function call is a candidate for custom function decorators.

On the other hand, function decorators are designed to augment only a specific function or method call, not an entire object interface. Class decorators fill the latter role better—because they can intercept instance creation calls, they can be used to implement arbitrary object interface augmentation or management tasks. For example, custom class decorators can trace or validate every attribute reference made for an object. They can also be used to implement proxy objects, singleton classes, and other common coding patterns. In fact, we’ll find that many class decorators bear a strong resemblance to the delegation coding pattern we met in Chapter 30.

Like many advanced Python tools, decorators are never strictly required from a purely technical perspective: their functionality can often be implemented instead using simple helper function calls or other techniques (and at a base level, we can always manually code the name rebinding that decorators perform automatically).

That said, decorators provide an explicit syntax for such tasks, which makes intent clearer, can minimize augmentation code redundancy, and may help ensure correct API usage:

In other words, beyond their technical model, decorators offer some advantages in terms of code maintenance and aesthetics. Moreover, as structuring tools, decorators naturally foster encapsulation of code, which reduces redundancy and makes future changes easier.

Decorators do have some potential drawbacks, too—when they insert wrapper logic, they can alter the types of the decorated objects, and they may incur extra calls. On the other hand, the same considerations apply to any technique that adds wrapping logic to objects.

We’ll explore these tradeoffs in the context of real code later in this chapter. Although the choice to use decorators is still somewhat subjective, their advantages are compelling enough that they are quickly becoming best practice in the Python world. To help you decide for yourself, let’s turn to the details.

Function decorators have been available in Python since version 2.5. As we saw earlier in this book, they are largely just syntactic sugar that runs one function through another at the end of a def statement, and rebinds the original function name to the result.

@decorator              # Decorate function
def F(arg):
    ...

F(99)                   # Call function

def F(arg):
    ...
F = decorator(F)        # Rebind function name to decorator result

F(99)                   # Essentially calls decorator(F)(99)

func(6, 7)
decorator(func)(6, 7)

class C:
    @staticmethod
    def meth(...): ...            # meth = staticmethod(meth)

class C:
    @property
    def name(self): ...           # name = property(name)

def decorator(F):
    # Process function F
    return F

@decorator
def func(): ...                  # func = decorator(func)

def decorator(F):
    # Save or use function F
    # Return a different callable: nested def, class with __call__, etc.

@decorator
def func(): ...                  # func = decorator(func)

def decorator(F):                     # On @ decoration
    def wrapper(*args):               # On wrapped function call
        # Use F and args
        # F(*args) calls original function
    return wrapper

@decorator                            # func = decorator(func)
def func(x, y):                       # func is passed to decorator's F
    ...

func(6, 7)                            # 6, 7 are passed to wrapper's *args

class decorator:
    def __init__(self, func):         # On @ decoration
        self.func = func
    def __call__(self, *args):        # On wrapped function call
        # Use self.func and args
        # self.func(*args) calls original function

@decorator
def func(x, y):                       # func = decorator(func)
    ...                               # func is passed to __init__

func(6, 7)                            # 6, 7 are passed to __call__'s *args

class decorator:
    def __init__(self, func):           # func is method without instance
        self.func = func
    def __call__(self, *args):          # self is decorator instance
        # self.func(*args) fails!  # C instance not in args!

class C:
    @decorator
    def method(self, x, y):             # method = decorator(method)
        ...                              # Rebound to decorator instance

def decorator(F):                       # F is func or method without instance
    def wrapper(*args):                 # class instance in args[0] for method
        # F(*args) runs func or method
    return wrapper

@decorator
def func(x, y):                         # func = decorator(func)
    ...
func(6, 7)                              # Really calls wrapper(6, 7)

class C:
    @decorator
    def method(self, x, y):             # method = decorator(method)
        ...                             # Rebound to simple function

X = C()
X.method(6, 7)                          # Really calls wrapper(X, 6, 7)

Function decorators proved so useful that the model was extended to allow class decoration in Python 2.6 and 3.0. Class decorators are strongly related to function decorators; in fact, they use the same syntax and very similar coding patterns. Rather than wrapping individual functions or methods, though, class decorators are a way to manage classes, or wrap up instance construction calls with extra logic that manages or augments instances created from a class.

@decorator                 # Decorate class
class C:
    ...

x = C(99)                  # Make an instance

class C:
    ...
C = decorator(C)           # Rebind class name to decorator result

x = C(99)                  # Essentially calls decorator(C)(99)

def decorator(C):
    # Process class C
    return C

@decorator
class C: ...                                    # C = decorator(C)

def decorator(C):
    # Save or use class C
    # Return a different callable: nested def, class with __call__, etc.

@decorator
class C: ...                                    # C = decorator(C)

def decorator(cls):                             # On @ decoration
    class Wrapper:
        def __init__(self, *args):              # On instance creation
            self.wrapped = cls(*args)
        def __getattr__(self, name):            # On attribute fetch
            return getattr(self.wrapped, name)
    return Wrapper

@decorator
class C:                             # C = decorator(C)
    def __init__(self, x, y):        # Run by Wrapper.__init__
        self.attr = 'spam'

x = C(6, 7)                          # Really calls Wrapper(6, 7)
print(x.attr)                        # Runs Wrapper.__getattr__, prints "spam"

class Decorator:
    def __init__(self, C):                    # On @ decoration
        self.C = C
    def __call__(self, *args):                # On instance creation
        self.wrapped = self.C(*args)
        return self
    def __getattr__(self, attrname):          # On atrribute fetch
        return getattr(self.wrapped, attrname)

@Decorator
class C: ...                                  # C = Decorator(C)

x = C()
y = C()                                       # Overwrites x!

def decorator(C):                             # On @ decoration
    class Wrapper:
        def __init__(self, *args):            # On instance creation
            self.wrapped = C(*args)
    return Wrapper

class Wrapper: ...
def decorator(C):                             # On @ decoration
    def onCall(*args):                        # On instance creation
        return Wrapper(C(*args))              # Embed instance in instance
    return onCall

Sometimes one decorator isn’t enough. To support multiple steps of augmentation, decorator syntax allows you to add multiple layers of wrapper logic to a decorated function or method. When this feature is used, each decorator must appear on a line of its own. Decorator syntax of this form:

@A
@B
@C
def f(...):
    ...

runs the same as the following:

def f(...):
    ...
f = A(B(C(f)))

Here, the original function is passed through three different decorators, and the resulting callable object is assigned back to the original name. Each decorator processes the result of the prior, which may be the original function or an inserted wrapper.

If all the decorators insert wrappers, the net effect is that when the original function name is called, three different layers of wrapping object logic will be invoked, to augment the original function in three different ways. The last decorator listed is the first applied, and the most deeply nested (insert joke about “interior decorators” here...).

Just as for functions, multiple class decorators result in multiple nested function calls, and possibly multiple levels of wrapper logic around instance creation calls. For example, the following code:

@spam
@eggs
class C:
    ...

X = C()

is equivalent to the following:

class C:
    ...
C = spam(eggs(C))

X = C()

Again, each decorator is free to return either the original class or an inserted wrapper object. With wrappers, when an instance of the original C class is finally requested, the call is redirected to the wrapping layer objects provided by both the spam and eggs decorators, which may have arbitrarily different roles.

For example, the following do-nothing decorators simply return the decorated function:

def d1(F): return F
def d2(F): return F
def d3(F): return F

@d1
@d2
@d3
def func():               # func = d1(d2(d3(func)))
    print('spam')

func()                    # Prints "spam"

The same syntax works on classes, as do these same do-nothing decorators.

When decorators insert wrapper function objects, though, they may augment the original function when called—the following concatenates to its result in the decorator layers, as it runs the layers from inner to outer:

def d1(F): return lambda: 'X' + F()
def d2(F): return lambda: 'Y' + F()
def d3(F): return lambda: 'Z' + F()

@d1
@d2
@d3
def func():               # func = d1(d2(d3(func)))
    return 'spam'

print(func())             # Prints "XYZspam"

We use lambda functions to implement wrapper layers here (each retains the wrapped function in an enclosing scope); in practice, wrappers can take the form of functions, callable classes, and more. When designed well, decorator nesting allows us to combine augmentation steps in a wide variety of ways.

Both function and class decorators can also seem to take arguments, although really these arguments are passed to a callable that in effect returns the decorator, which in turn returns a callable. The following, for instance:

@decorator(A, B)
def F(arg):
    ...

F(99)

is automatically mapped into this equivalent form, where decorator is a callable that returns the actual decorator. The returned decorator in turn returns the callable run later for calls to the original function name:

def F(arg):
    ...
F = decorator(A, B)(F)    # Rebind F to result of decorator's return value

F(99)                     # Essentially calls decorator(A, B)(F)(99)

Decorator arguments are resolved before decoration ever occurs, and they are usually used to retain state information for use in later calls. The decorator function in this example, for instance, might take a form like the following:

def decorator(A, B):
    # Save or use A, B
    def actualDecorator(F):
        # Save or use function F
        # Return a callable: nested def, class with __call__, etc.
        return callable
    return actualDecorator

The outer function in this structure generally saves the decorator arguments away as state information, for use in the actual decorator, the callable it returns, or both. This code snippet retains the state information argument in enclosing function scope references, but class attributes are commonly used as well.

In other words, decorator arguments often imply three levels of callables: a callable to accept decorator arguments, which returns a callable to serve as decorator, which returns a callable to handle calls to the original function or class. Each of the three levels may be a function or class and may retain state in the form of scopes or class attributes. We’ll see concrete examples of decorator arguments employed later in this chapter.

Although much of the rest of this chapter focuses on wrapping later calls to functions and classes, I should underscore that the decorator mechanism is more general than this—it is a protocol for passing functions and classes through a callable immediately after they are created. As such, it can also be used to invoke arbitrary post-creation processing:

def decorator(O):
    # Save or augment function or class O
    return O

@decorator
def F(): ...                 # F = decorator(F)

@decorator
class C: ...                 # C = decorator(C)

As long as we return the original decorated object this way instead of a wrapper, we can manage functions and classes themselves, not just later calls to them. We’ll see more realistic examples later in this chapter that use this idea to register callable objects to an API with decoration and assign attributes to functions when they are created.

To get started, let’s revive the call tracer example we met in Chapter 31. The following defines and applies a function decorator that counts the number of calls made to the decorated function and prints a trace message for each call:

class tracer:
    def __init__(self, func):             # On @ decoration: save original func
        self.calls = 0
        self.func = func
    def __call__(self, *args):            # On later calls: run original func
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        self.func(*args)

@tracer
def spam(a, b, c):           # spam = tracer(spam)
    print(a + b + c)         # Wraps spam in a decorator object

Notice how each function decorated with this class will create a new instance, with its own saved function object and calls counter. Also observe how the *args argument syntax is used to pack and unpack arbitrarily many passed-in arguments. This generality enables this decorator to be used to wrap any function with any number of arguments (this version doesn’t yet work on class methods, but we’ll fix that later in this section).

Now, if we import this module’s function and test it interactively, we get the following sort of behavior—each call generates a trace message initially, because the decorator class intercepts it. This code runs under both Python 2.6 and 3.0, as does all code in this chapter unless otherwise noted:

>>> from decorator1 import spam

>>> spam(1, 2, 3)            # Really calls the tracer wrapper object
call 1 to spam
6

>>> spam('a', 'b', 'c')      # Invokes __call__ in class
call 2 to spam
abc

>>> spam.calls               # Number calls in wrapper state information
2
>>> spam
<decorator1.tracer object at 0x02D9A730>

When run, the tracer class saves away the decorated function, and intercepts later calls to it, in order to add a layer of logic that counts and prints each call. Notice how the total number of calls shows up as an attribute of the decorated function—spam is really an instance of the tracer class when decorated (a finding that may have ramifications for programs that do type checking, but is generally benign).

For function calls, the @ decoration syntax can be more convenient than modifying each call to account for the extra logic level, and it avoids accidentally calling the original function directly. Consider a nondecorator equivalent such as the following:

calls = 0
def tracer(func, *args):
    global calls
    calls += 1
    print('call %s to %s' % (calls, func.__name__))
    func(*args)

def spam(a, b, c):
    print(a, b, c)

>>> spam(1, 2, 3)            # Normal non-traced call: accidental?
1 2 3

>>> tracer(spam, 1, 2, 3)    # Special traced call without decorators
call 1 to spam
1 2 3

This alternative can be used on any function without the special @ syntax, but unlike the decorator version, it requires extra syntax at every place where the function is called in your code; furthermore, its intent may not be as obvious, and it does not ensure that the extra layer will be invoked for normal calls. Although decorators are never required (we can always rebind names manually), they are often the most convenient option.

The last example of the prior section raises an important issue. Function decorators have a variety of options for retaining state information provided at decoration time, for use during the actual function call. They generally need to support multiple decorated objects and multiple calls, but there are a number of ways to implement these goals: instance attributes, global variables, nonlocal variables, and function attributes can all be used for retaining state.

class tracer:                                # State via instance attributes
    def __init__(self, func):                # On @ decorator
        self.calls = 0                       # Save func for later call
        self.func  = func
    def __call__(self, *args, **kwargs):     # On call to original function
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        return self.func(*args, **kwargs)

@tracer
def spam(a, b, c):          # Same as: spam = tracer(spam)
    print(a + b + c)        # Triggers tracer.__init__

@tracer
def eggs(x, y):             # Same as: eggs = tracer(eggs)
    print(x ** y)           # Wraps eggs in a tracer object

spam(1, 2, 3)               # Really calls tracer instance: runs tracer.__call__
spam(a=4, b=5, c=6)         # spam is an instance attribute

eggs(2, 16)                 # Really calls tracer instance, self.func is eggs
eggs(4, y=4)                # self.calls is per-function here (need 3.0 nonlocal)

call 1 to spam
6
call 2 to spam
15
call 1 to eggs
65536
call 2 to eggs
256

calls = 0
def tracer(func):                         # State via enclosing scope and global
    def wrapper(*args, **kwargs):         # Instead of class attributes
        global calls                      # calls is global, not per-function
        calls += 1
        print('call %s to %s' % (calls, func.__name__))
        return func(*args, **kwargs)
    return wrapper

@tracer
def spam(a, b, c):        # Same as: spam = tracer(spam)
    print(a + b + c)

@tracer
def eggs(x, y):           # Same as: eggs = tracer(eggs)
    print(x ** y)

spam(1, 2, 3)             # Really calls wrapper, bound to func
spam(a=4, b=5, c=6)       # wrapper calls spam

eggs(2, 16)               # Really calls wrapper, bound to eggs
eggs(4, y=4)              # Global calls is not per-function here!

call 1 to spam
6
call 2 to spam
15
call 3 to eggs
65536
call 4 to eggs
256

def tracer(func):                        # State via enclosing scope and nonlocal
    calls = 0                            # Instead of class attrs or global
    def wrapper(*args, **kwargs):        # calls is per-function, not global
        nonlocal calls
        calls += 1
        print('call %s to %s' % (calls, func.__name__))
        return func(*args, **kwargs)
    return wrapper

@tracer
def spam(a, b, c):        # Same as: spam = tracer(spam)
    print(a + b + c)

@tracer
def eggs(x, y):           # Same as: eggs = tracer(eggs)
    print(x ** y)

spam(1, 2, 3)             # Really calls wrapper, bound to func
spam(a=4, b=5, c=6)       # wrapper calls spam

eggs(2, 16)               # Really calls wrapper, bound to eggs
eggs(4, y=4)              # Nonlocal calls _is_ not per-function here

call 1 to spam
6
call 2 to spam
15
call 1 to eggs
65536
call 2 to eggs
256

def tracer(func):                        # State via enclosing scope and func attr
    def wrapper(*args, **kwargs):        # calls is per-function, not global
        wrapper.calls += 1
        print('call %s to %s' % (wrapper.calls, func.__name__))
        return func(*args, **kwargs)
    wrapper.calls = 0
    return wrapper

When I wrote the first tracer function decorator above, I naively assumed that it could also be applied to any method—decorated methods should work the same, but the automatic self instance argument would simply be included at the front of *args. Unfortunately, I was wrong: when applied to a class’s method, the first version of the tracer fails, because self is the instance of the decorator class and the instance of the decorated subject class is not included in *args. This is true in both Python 3.0 and 2.6.

I introduced this phenomenon earlier in this chapter, but now we can see it in the context of realistic working code. Given the class-based tracing decorator:

class tracer:
    def __init__(self, func):                # On @ decorator
        self.calls = 0                       # Save func for later call
        self.func  = func
    def __call__(self, *args, **kwargs):     # On call to original function
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        return self.func(*args, **kwargs)

decoration of simple functions works as advertised earlier:

@tracer
def spam(a, b, c):                           # spam = tracer(spam)
    print(a + b + c)                         # Triggers tracer.__init__

spam(1, 2, 3)                                # Runs tracer.__call__
spam(a=4, b=5, c=6)                          # spam is an instance attribute

However, decoration of class methods fails (more lucid readers might recognize this as our Person class resurrected from the object-oriented tutorial in Chapter 27):

class Person:
    def __init__(self, name, pay):
        self.name = name
        self.pay  = pay

    @tracer
    def giveRaise(self, percent):            # giveRaise = tracer(giveRaise)
        self.pay *= (1.0 + percent)

    @tracer
    def lastName(self):                      # lastName = tracer(lastName)
        return self.name.split()[-1]

bob = Person('Bob Smith', 50000)             # tracer remembers method funcs
bob.giveRaise(.25)                           # Runs tracer.__call__(???, .25)
print(bob.lastName())                        # Runs tracer.__call__(???)

The root of the problem here is in the self argument of the tracer class’s __call__ method—is it a tracer instance or a Person instance? We really need both as it’s coded: the tracer for decorator state, and the Person for routing on to the original method. Really, self must be the tracer object, to provide access to tracer’s state information; this is true whether decorating a simple function or a method.

Unfortunately, when our decorated method name is rebound to a class instance object with a __call__, Python passes only the tracer instance to self; it doesn’t pass along the Person subject in the arguments list at all. Moreover, because the tracer knows nothing about the Person instance we are trying to process with method calls, there’s no way to create a bound method with an instance, and thus no way to correctly dispatch the call.

In fact, the prior listing winds up passing too few arguments to the decorated method, and results in an error. Add a line to the decorator’s __call__ to print all its arguments to verify this; as you can see, self is the tracer, and the Person instance is entirely absent:

<__main__.tracer object at 0x02D6AD90> (0.25,) {}
call 1 to giveRaise
Traceback (most recent call last):
  File "C:/misc/tracer.py", line 56, in <module>
    bob.giveRaise(.25)
  File "C:/misc/tracer.py", line 9, in __call__
    return self.func(*args, **kwargs)
TypeError: giveRaise() takes exactly 2 positional arguments (1 given)

As mentioned earlier, this happens because Python passes the implied subject instance to self when a method name is bound to a simple function only; when it is an instance of a callable class, that class’s instance is passed instead. Technically, Python only makes a bound method object containing the subject instance when the method is a simple function.

# A decorator for both functions and methods

def tracer(func):                        # Use function, not class with __call__
    calls = 0                            # Else "self" is decorator instance only!
    def onCall(*args, **kwargs):
        nonlocal calls
        calls += 1
        print('call %s to %s' % (calls, func.__name__))
        return func(*args, **kwargs)
    return onCall


# Applies to simple functions

@tracer
def spam(a, b, c):                       # spam = tracer(spam)
    print(a + b + c)                     # onCall remembers spam

spam(1, 2, 3)                            # Runs onCall(1, 2, 3)
spam(a=4, b=5, c=6)


# Applies to class method functions too!

class Person:
    def __init__(self, name, pay):
        self.name = name
        self.pay  = pay

    @tracer
    def giveRaise(self, percent):        # giveRaise = tracer(giveRaise)
        self.pay *= (1.0 + percent)      # onCall remembers giveRaise

    @tracer
    def lastName(self):                  # lastName = tracer(lastName)
        return self.name.split()[-1]

print('methods...')
bob = Person('Bob Smith', 50000)
sue = Person('Sue Jones', 100000)
print(bob.name, sue.name)
sue.giveRaise(.10)                       # Runs onCall(sue, .10)
print(sue.pay)
print(bob.lastName(), sue.lastName())    # Runs onCall(bob), lastName in scopes

call 1 to spam
6
call 2 to spam
15
methods...
Bob Smith Sue Jones
call 1 to giveRaise
110000.0
call 1 to lastName
call 2 to lastName
Smith Jones

class Descriptor(object):
    def __get__(self, instance, owner): ...

class Subject:
    attr = Descriptor()

X = Subject()
X.attr         # Roughly runs Descriptor.__get__(Subject.attr, X, Subject)

class tracer(object):
    def __init__(self, func):                # On @ decorator
        self.calls = 0                       # Save func for later call
        self.func  = func
    def __call__(self, *args, **kwargs):     # On call to original func
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        return self.func(*args, **kwargs)
    def __get__(self, instance, owner):      # On method attribute fetch
        return wrapper(self, instance)

class wrapper:
    def __init__(self, desc, subj):          # Save both instances
        self.desc = desc                     # Route calls back to  decr
        self.subj = subj
    def __call__(self, *args, **kwargs):
        return self.desc(self.subj, *args, **kwargs)  # Runs tracer.__call__

@tracer
def spam(a, b, c):                           # spam = tracer(spam)
    ...same as prior...                      # Uses __call__ only

class Person:
    @tracer
    def giveRaise(self, percent):            # giveRaise = tracer(giverRaise)
        ...same as prior...                  # Makes giveRaise a descriptor

sue.giveRaise(.10)                           # Runs __get__ then __call__

class tracer(object):
    def __init__(self, func):                # On @ decorator
        self.calls = 0                       # Save func for later call
        self.func  = func
    def __call__(self, *args, **kwargs):     # On call to original func
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        return self.func(*args, **kwargs)
    def __get__(self, instance, owner):                # On method fetch
        def wrapper(*args, **kwargs):                  # Retain both inst
            return self(instance, *args, **kwargs)     # Runs __call__
        return wrapper

To sample the fuller flavor of what function decorators are capable of, let’s turn to a different use case. Our next decorator times calls made to a decorated function—both the time for one call, and the total time among all calls. The decorator is applied to two functions, in order to compare the time requirements of list comprehensions and the map built-in call (for comparison, also see Chapter 20 for another nondecorator example that times iteration alternatives like these):

import time

class timer:
    def __init__(self, func):
        self.func    = func
        self.alltime = 0
    def __call__(self, *args, **kargs):
        start   = time.clock()
        result  = self.func(*args, **kargs)
        elapsed = time.clock() - start
        self.alltime += elapsed
        print('%s: %.5f, %.5f' % (self.func.__name__, elapsed, self.alltime))
        return result

@timer
def listcomp(N):
    return [x * 2 for x in range(N)]

@timer
def mapcall(N):
    return map((lambda x: x * 2), range(N))

result = listcomp(5)                # Time for this call, all calls, return value
listcomp(50000)
listcomp(500000)
listcomp(1000000)
print(result)
print('allTime = %s' % listcomp.alltime)      # Total time for all listcomp calls

print('')
result = mapcall(5)
mapcall(50000)
mapcall(500000)
mapcall(1000000)
print(result)
print('allTime = %s' % mapcall.alltime)       # Total time for all mapcall calls

print('map/comp = %s' % round(mapcall.alltime / listcomp.alltime, 3))

In this case, a nondecorator approach would allow the subject functions to be used with or without timing, but it would also complicate the call signature when timing is desired (we’d need to add code at every call instead of once at the def), and there would be no direct way to guarantee that all list builder calls in a program are routed through timer logic, short of finding and potentially changing them all.

When run in Python 2.6, the output of this file’s self-test code is as follows:

listcomp: 0.00002, 0.00002
listcomp: 0.00910, 0.00912
listcomp: 0.09105, 0.10017
listcomp: 0.17605, 0.27622
[0, 2, 4, 6, 8]
allTime = 0.276223304917

mapcall: 0.00003, 0.00003
mapcall: 0.01363, 0.01366
mapcall: 0.13579, 0.14945
mapcall: 0.27648, 0.42593
[0, 2, 4, 6, 8]
allTime = 0.425933533452
map/comp = 1.542

Testing subtlety: I didn’t run this under Python 3.0 because, as described in Chapter 14, the map built-in returns an iterator in 3.0, instead of an actual list as in 2.6. Hence, 3.0’s map doesn’t quite compare directly to a list comprehension’s work (as is, the map test takes virtually no time at all in 3.0!).

If you wish to run this under 3.0, too, use list(map()) to force it to build a list like the list comprehension does, or else you’re not really comparing apples to apples. Don’t do so in 2.6, though—if you do, the map test will be charged for building two lists, not one.

The following sort of code would pick fairly for 2.6 and 3.0; note, though, that while this makes the comparison between list comprehensions and map more fair in either 2.6 or 3.0, because range is also an iterator in 3.0, the results for 2.6 and 3.0 won’t compare directly:

...
import sys

@timer
def listcomp(N):
    return [x * 2 for x in range(N)]

if sys.version_info[0] == 2:
    @timer
    def mapcall(N):
        return map((lambda x: x * 2), range(N))
else:
    @timer
    def mapcall(N):
        return list(map((lambda x: x * 2), range(N)))
    ...

Finally, as we learned in the modules part of this book if you want to be able to reuse this decorator in other modules, you should indent the self-test code at the bottom of the file under a __name__ == '__main__' test so it runs only when the file is run, not when it’s imported. We won’t do this, though, because we’re about to add another feature to our code.

The timer decorator of the prior section works, but it would be nice if it was more configurable—providing an output label and turning trace messages on and off, for instance, might be useful in a general-purpose tool like this. Decorator arguments come in handy here: when they’re coded properly, we can use them to specify configuration options that can vary for each decorated function. A label, for instance, might be added as follows:

def timer(label=''):
    def decorator(func):
        def onCall(*args):          # args passed to function
            ...                     # func retained in enclosing scope
            print(label, ...        # label retained in enclosing scope
        return onCall
    return decorator                # Returns that actual decorator

@timer('==>')                       # Like listcomp = timer('==>')(listcomp)
def listcomp(N): ...                # listcomp is rebound to decorator

listcomp(...)                       # Really calls decorator

This code adds an enclosing scope to retain a decorator argument for use on a later actual call. When the listcomp function is defined, it really invokes decorator (the result of timer, run before decoration actually occurs), with the label value available in its enclosing scope. That is, timer returns the decorator, which remembers both the decorator argument and the original function and returns a callable which invokes the original function on later calls.

We can put this structure to use in our timer to allow a label and a trace control flag to be passed in at decoration time. Here’s an example that does just that, coded in a module file named mytools.py so it can be imported as a general tool:

import time

def timer(label='', trace=True):                  # On decorator args: retain args
    class Timer:
        def __init__(self, func):                 # On @: retain decorated func
            self.func    = func
            self.alltime = 0
        def __call__(self, *args, **kargs):       # On calls: call original
            start   = time.clock()
            result  = self.func(*args, **kargs)
            elapsed = time.clock() - start
            self.alltime += elapsed
            if trace:
                format = '%s %s: %.5f, %.5f'
                values = (label, self.func.__name__, elapsed, self.alltime)
                print(format % values)
            return result
    return Timer

Mostly all we’ve done here is embed the original Timer class in an enclosing function, in order to create a scope that retains the decorator arguments. The outer timer function is called before decoration occurs, and it simply returns the Timer class to serve as the actual decorator. On decoration, an instance of Timer is made that remembers the decorated function itself, but also has access to the decorator arguments in the enclosing function scope.

This time, rather than embedding self-test code in this file, we’ll run the decorator in a different file. Here’s a client of our timer decorator, the module file testseqs.py, applying it to sequence iteration alternatives again:

from mytools import timer

@timer(label='[CCC]==>')
def listcomp(N):                           # Like listcomp = timer(...)(listcomp)
    return [x * 2 for x in range(N)]       # listcomp(...) triggers Timer.__call__

@timer(trace=True, label='[MMM]==>')
def mapcall(N):
    return map((lambda x: x * 2), range(N))

for func in (listcomp, mapcall):
    print('')
    result = func(5)        # Time for this call, all calls, return value
    func(50000)
    func(500000)
    func(1000000)
    print(result)
    print('allTime = %s' % func.alltime)   # Total time for all calls

print('map/comp = %s' % round(mapcall.alltime / listcomp.alltime, 3))

Again, if you wish to run this fairly in 3.0, wrap the map function in a list call. When run as-is in 2.6, this file prints the following output—each decorated function now has a label of its own, defined by decorator arguments:

[CCC]==> listcomp: 0.00003, 0.00003
[CCC]==> listcomp: 0.00640, 0.00643
[CCC]==> listcomp: 0.08687, 0.09330
[CCC]==> listcomp: 0.17911, 0.27241
[0, 2, 4, 6, 8]
allTime = 0.272407666337

[MMM]==> mapcall: 0.00004, 0.00004
[MMM]==> mapcall: 0.01340, 0.01343
[MMM]==> mapcall: 0.13907, 0.15250
[MMM]==> mapcall: 0.27907, 0.43157
[0, 2, 4, 6, 8]
allTime = 0.431572169089
map/comp = 1.584

As usual, we can also test this interactively to see how the configuration arguments come into play:

>>> from mytools import timer
>>> @timer(trace=False)                    # No tracing, collect total time
... def listcomp(N):
...     return [x * 2 for x in range(N)]
...
>>> x = listcomp(5000)
>>> x = listcomp(5000)
>>> x = listcomp(5000)
>>> listcomp
<mytools.Timer instance at 0x025C77B0>
>>> listcomp.alltime
0.0051938863738243413

>>> @timer(trace=True, label='	=>')       # Turn on tracing
... def listcomp(N):
...     return [x * 2 for x in range(N)]
...
>>> x = listcomp(5000)
        => listcomp: 0.00155, 0.00155
>>> x = listcomp(5000)
        => listcomp: 0.00156, 0.00311
>>> x = listcomp(5000)
        => listcomp: 0.00174, 0.00486
>>> listcomp.alltime
0.0048562736325408196

This timing function decorator can be used for any function, both in modules and interactively. In other words, it automatically qualifies as a general-purpose tool for timing code in our scripts. Watch for another example of decorator arguments in the section Implementing Private Attributes, and again in A Basic Range-Testing Decorator for Positional Arguments.

Because class decorators may intercept instance creation calls, they can be used to either manage all the instances of a class, or augment the interfaces of those instances. To demonstrate, here’s a first class decorator example that does the former—managing all instances of a class. This code implements the classic singleton coding pattern, where at most one instance of a class ever exists. Its singleton function defines and returns a function for managing instances, and the @ syntax automatically wraps up a subject class in this function:

instances = {}
def getInstance(aClass, *args):                 # Manage global table
    if aClass not in instances:                 # Add **kargs for keywords
        instances[aClass] = aClass(*args)       # One dict entry per class
    return instances[aClass]

def singleton(aClass):                          # On @ decoration
    def onCall(*args):                          # On instance creation
        return getInstance(aClass, *args)
    return onCall

To use this, decorate the classes for which you want to enforce a single-instance model:

@singleton                                      # Person = singleton(Person)
class Person:                                   # Rebinds Person to onCall
     def __init__(self, name, hours, rate):     # onCall remembers Person
        self.name = name
        self.hours = hours
        self.rate = rate
     def pay(self):
        return self.hours * self.rate

@singleton                                      # Spam = singleton(Spam)
class Spam:                                     # Rebinds Spam to onCall
    def __init__(self, val):                    # onCall remembers Spam
        self.attr = val

bob = Person('Bob', 40, 10)                     # Really calls onCall
print(bob.name, bob.pay())

sue = Person('Sue', 50, 20)                     # Same, single object
print(sue.name, sue.pay())

X = Spam(42)                                    # One Person, one Spam
Y = Spam(99)
print(X.attr, Y.attr)

Now, when the Person or Spam class is later used to create an instance, the wrapping logic layer provided by the decorator routes instance construction calls to onCall, which in turn calls getInstance to manage and share a single instance per class, regardless of how many construction calls are made. Here’s this code’s output:

Bob 400
Bob 400
42 42

Interestingly, you can code a more self-contained solution here if you’re able to use the nonlocal statement (available in Python 3.0 and later) to change enclosing scope names, as described earlier—the following alternative achieves an identical effect, by using one enclosing scope per class, instead of one global table entry per class:

def singleton(aClass):                          # On @ decoration
    instance = None
    def onCall(*args):                          # On instance creation
        nonlocal instance                       # 3.0 and later nonlocal
        if instance == None:
            instance = aClass(*args)            # One scope per class
        return instance
    return onCall

This version works the same, but it does not depend on names in the global scope outside the decorator. In either Python 2.6 or 3.0, you can also code a self-contained solution with a class instead—the following uses one instance per class, rather than an enclosing scope or global table, and works the same as the other two versions (in fact, it relies on the same coding pattern that we will later see is a common decorator class blunder; here we want just one instance, but that’s not always the case):

class singleton:
    def __init__(self, aClass):                 # On @ decoration
        self.aClass = aClass
        self.instance = None
    def __call__(self, *args):                  # On instance creation
        if self.instance == None:
            self.instance = self.aClass(*args)  # One instance per class
        return self.instance

To make this decorator a fully general-purpose tool, store it in an importable module file, indent the self-test code under a __name__ check, and add support for keyword arguments in construction calls with **kargs syntax (I’ll leave this as a suggested exercise).

The singleton example of the prior section illustrated using class decorators to manage all the instances of a class. Another common use case for class decorators augments the interface of each generated instance. Class decorators can essentially install on instances a wrapper logic layer that manages access to their interfaces in some way.

For example, in Chapter 30, the __getattr__ operator overloading method is shown as a way to wrap up entire object interfaces of embedded instances, in order to implement the delegation coding pattern. We saw similar examples in the managed attribute coverage of the prior chapter. Recall that __getattr__ is run when an undefined attribute name is fetched; we can use this hook to intercept method calls in a controller class and propagate them to an embedded object.

For reference, here’s the original nondecorator delegation example, working on two built-in type objects:

class Wrapper:
    def __init__(self, object):
        self.wrapped = object                    # Save object
    def __getattr__(self, attrname):
        print('Trace:', attrname)                # Trace fetch
        return getattr(self.wrapped, attrname)   # Delegate fetch

>>> x = Wrapper([1,2,3])                         # Wrap a list
>>> x.append(4)                                  # Delegate to list method
Trace: append
>>> x.wrapped                                    # Print my member
[1, 2, 3, 4]

>>> x = Wrapper({"a": 1, "b": 2})                # Wrap a dictionary
>>> list(x.keys())                               # Delegate to dictionary method
Trace: keys                                      # Use list() in 3.0
['a', 'b']

In this code, the Wrapper class intercepts access to any of the wrapped object’s attributes, prints a trace message, and uses the getattr built-in to pass off the request to the wrapped object. Specifically, it traces attribute accesses made outside the wrapped object’s class; accesses inside the wrapped object’s methods are not caught and run normally by design. This whole-interface model differs from the behavior of function decorators, which wrap up just one specific method.

Class decorators provide an alternative and convenient way to code this __getattr__ technique to wrap an entire interface. In 2.6 and 3.0, for example, the prior class example can be coded as a class decorator that triggers wrapped instance creation, instead of passing a pre-made instance into the wrapper’s constructor (also augmented here to support keyword arguments with **kargs and to count the number of accesses made):

def Tracer(aClass):                                   # On @ decorator
    class Wrapper:
        def __init__(self, *args, **kargs):           # On instance creation
            self.fetches = 0
            self.wrapped = aClass(*args, **kargs)     # Use enclosing scope name
        def __getattr__(self, attrname):
            print('Trace: ' + attrname)               # Catches all but own attrs
            self.fetches += 1
            return getattr(self.wrapped, attrname)    # Delegate to wrapped obj
    return Wrapper

@Tracer
class Spam:                                    # Spam = Tracer(Spam)
    def display(self):                         # Spam is rebound to Wrapper
        print('Spam!' * 8)

@Tracer
class Person:                                  # Person = Tracer(Person)
    def __init__(self, name, hours, rate):     # Wrapper remembers Person
        self.name = name
        self.hours = hours
        self.rate = rate
    def pay(self):                             # Accesses outside class traced
        return self.hours * self.rate          # In-method accesses not traced

food = Spam()                                  # Triggers Wrapper()
food.display()                                 # Triggers __getattr__
print([food.fetches])

bob = Person('Bob', 40, 50)                    # bob is really a Wrapper
print(bob.name)                                # Wrapper embeds a Person
print(bob.pay())

print('')
sue = Person('Sue', rate=100, hours=60)        # sue is a different Wrapper
print(sue.name)                                # with a different Person
print(sue.pay())

print(bob.name)                                # bob has different state
print(bob.pay())
print([bob.fetches, sue.fetches])              # Wrapper attrs not traced

It’s important to note that this is very different from the tracer decorator we met earlier. In Coding Function Decorators, we looked at decorators that enabled us to trace and time calls to a given function or method. In contrast, by intercepting instance creation calls, the class decorator here allows us to trace an entire object interface—i.e., accesses to any of its attributes.

The following is the output produced by this code under both 2.6 and 3.0: attribute fetches on instances of both the Spam and Person classes invoke the __getattr__ logic in the Wrapper class, because food and bob are really instances of Wrapper, thanks to the decorator’s redirection of instance creation calls:

Trace: display
Spam!Spam!Spam!Spam!Spam!Spam!Spam!Spam!
[1]
Trace: name
Bob
Trace: pay
2000

Trace: name
Sue
Trace: pay
6000
Trace: name
Bob
Trace: pay
2000
[4, 2]

Notice that the preceding code decorates a user-defined class. Just like in the original example in Chapter 30, we can also use the decorator to wrap up a built-in type such as a list, as long as we either subclass to allow decoration syntax or perform the decoration manually—decorator syntax requires a class statement for the @ line.

In the following, x is really a Wrapper again due to the indirection of decoration (I moved the decorator class to module file tracer.py in order to reuse it this way):

>>> from tracer import Tracer     # Decorator moved to a module file

>>> @Tracer
... class MyList(list): pass      # MyList = Tracer(MyList)

>>> x = MyList([1, 2, 3])         # Triggers Wrapper()
>>> x.append(4)                   # Triggers __getattr__, append
Trace: append
>>> x.wrapped
[1, 2, 3, 4]

>>> WrapList = Tracer(list)       # Or perform decoration manually
>>> x = WrapList([4, 5, 6])       # Else subclass statement required
>>> x.append(7)
Trace: append
>>> x.wrapped
[4, 5, 6, 7]

The decorator approach allows us to move instance creation into the decorator itself, instead of requiring a premade object to be passed in. Although this seems like a minor difference, it lets us retain normal instance creation syntax and realize all the benefits of decorators in general. Rather than requiring all instance creation calls to route objects through a wrapper manually, we need only augment classes with decorator syntax:

@Tracer                                          # Decorator approach
class Person: ...
bob = Person('Bob', 40, 50)
sue = Person('Sue', rate=100, hours=60)

class Person: ...                                # Non-decorator approach
bob = Wrapper(Person('Bob', 40, 50))
sue = Wrapper(Person('Sue', rate=100, hours=60))

Assuming you will make more than one instance of a class, decorators will generally be a net win in terms of both code size and code maintenance.

>>> x                                   # 2.6
Trace: __repr__
[4, 5, 6, 7]
>>> x                                   # 3.0
<tracer.Wrapper object at 0x026C07D0>

Curiously, the decorator function in this example can almost be coded as a class instead of a function, with the proper operator overloading protocol. The following slightly simplified alternative works similarly because its __init__ is triggered when the @ decorator is applied to the class, and its __call__ is triggered when a subject class instance is created. Our objects are really instances of Tracer this time, and we essentially just trade an enclosing scope reference for an instance attribute here:

class Tracer:
    def __init__(self, aClass):               # On @decorator
        self.aClass = aClass                  # Use instance attribute
    def __call__(self, *args):                # On instance creation
        self.wrapped = self.aClass(*args)     # ONE (LAST) INSTANCE PER CLASS!
        return self
    def __getattr__(self, attrname):
        print('Trace: ' + attrname)
        return getattr(self.wrapped, attrname)

@Tracer                                       # Triggers __init__
class Spam:                                   # Like: Spam = Tracer(Spam)
    def display(self):
        print('Spam!' * 8)

...
food = Spam()                                 # Triggers __call__
food.display()                                # Triggers __getattr__

As we saw in the abstract earlier, though, this class-only alternative handles multiple classes as before, but it won’t quite work for multiple instances of a given class: each instance construction call triggers __call__, which overwrites the prior instance. The net effect is that Tracer saves just one instance—the last one created. Experiment with this yourself to see how, but here’s an example of the problem:

@Tracer
class Person:                                 # Person = Tracer(Person)
    def __init__(self, name):                 # Wrapper bound to Person
        self.name = name

bob = Person('Bob')                           # bob is really a Wrapper
print(bob.name)                               # Wrapper embeds a Person
Sue = Person('Sue')
print(sue.name)                               # sue overwrites bob
print(bob.name)                               # OOPS: now bob's name is 'Sue'!

This code’s output follows—because this tracer only has a single shared instance, the second overwrites the first:

Trace: name
Bob
Trace: name
Sue
Trace: name
Sue

The problem here is bad state retention—we make one decorator instance per class, but not per class instance, such that only the last instance is retained. The solution, as in our prior class blunder for decorating methods, lies in abandoning class-based decorators.

The earlier function-based Tracer version does work for multiple instances, because each instance construction call makes a new Wrapper instance, instead of overwriting the state of a single shared Tracer instance; the original nondecorator version handles multiple instances correctly for the same reason. Decorators are not only arguably magical, they can also be incredibly subtle!

Regardless of such subtleties, the Tracer class decorator example ultimately still relies on __getattr__ to intercept fetches on a wrapped and embedded instance object. As we saw earlier, all we’ve really accomplished is moving the instance creation call inside a class, instead of passing the instance into a manager function. With the original nondecorator tracing example, we would simply code instance creation differently:

class Spam:                                   # Non-decorator version
    ...                                       # Any class will do
food = Wrapper(Spam())                        # Special creation syntax

@Tracer
class Spam:                                   # Decorator version
    ...                                       # Requires @ syntax at class
food = Spam()                                 # Normal creation syntax

Essentially, class decorators shift special syntax requirements from the instance creation call to the class statement itself. This is also true for the singleton example earlier in this section—rather than decorating a class and using normal instance creation calls, we could simply pass the class and its construction arguments into a manager function:

instances = {}
def getInstance(aClass, *args):
    if aClass not in instances:
        instances[aClass] = aClass(*args)
    return instances[aClass]

bob = getInstance(Person, 'Bob', 40, 10)    # Versus: bob = Person('Bob', 40, 10)

Alternatively, we could use Python’s introspection facilities to fetch the class from an already-created instance (assuming creating an initial instance is acceptable):

instances = {}
def getInstance(object):
    aClass = object.__class__
    if aClass not in instances:
        instances[aClass] = object
    return instances[aClass]

bob = getInstance(Person('Bob', 40, 10))    # Versus: bob = Person('Bob', 40, 10)

The same holds true for function decorators like the tracer we wrote earlier: rather than decorating a function with logic that intercepts later calls, we could simply pass the function and its arguments into a manager that dispatches the call:

def func(x, y):                   # Nondecorator version
    ...                           # def tracer(func, args): ... func(*args)
result = tracer(func, (1, 2))     # Special call syntax

@tracer
def func(x, y):                   # Decorator version
    ...                           # Rebinds name: func = tracer(func)
result = func(1, 2)               # Normal call syntax

Manager function approaches like this place the burden of using special syntax on calls, instead of expecting decoration syntax at function and class definitions.

So why did I just show you ways to not use decorators to implement singletons? As I mentioned at the start of this chapter, decorators present us with tradeoffs. Although syntax matters, we all too often forget to ask the “why” questions when confronted with new tools. Now that we’ve seen how decorators actually work, let’s step back for a minute to glimpse the big picture here.

Like most language features, decorators have both pros and cons. For example, in the negatives column, class decorators suffer from two potential drawbacks:

Similar concerns apply with function decorators: both decoration and manager functions incur extra calls, and type changes generally occur when decorating (but not otherwise).

That said, neither of these is a very serious issue. For most programs, the type difference issue is unlikely to matter and the speed hit of the extra calls will be insignificant; furthermore, the latter occurs only when wrappers are used, can often be negated by simply removing the decorator when optimal performance is required, and is also incurred by nondecorator solutions that add wrapping logic (including metaclasses, as we’ll see in Chapter 39).

Conversely, as we saw at the start of this chapter, decorators have three main advantages. Compared to the manager (a.k.a. “helper”) function solutions of the prior section, decorators offer:

Decorators also promote code encapsulation to reduce redundancy and minimize future maintenance effort; although other code structuring tools do too, decorators make this natural for augmentation tasks.

None of these benefits completely requires decorator syntax to be achieved, though, and decorator usage is ultimately a stylistic choice. That said, most programmers find them to be a net win, especially as a tool for using libraries and APIs correctly.

I can recall similar arguments being made both for and against constructor functions in classes—prior to the introduction of __init__ methods, the same effect was often achieved by running an instance through a method manually when creating it (e.g., X=Class().init()). Over time, though, despite being fundamentally a stylistic choice, the __init__ syntax came to be universally preferred because it was more explicit, consistent, and maintainable. Although you should be the judge, decorators seem to bring many of the same assets to the table.

The following class decorator implements a Private declaration for class instance attributes—that is, attributes stored on an instance, or inherited from one of its classes. It disallows fetch and change access to such attributes from outside the decorated class, but still allows the class itself to access those names freely within its methods. It’s not exactly C++ or Java, but it provides similar access control as an option in Python.

We saw an incomplete first-cut implementation of instance attribute privacy for changes only in Chapter 29. The version here extends this concept to validate attribute fetches too, and it uses delegation instead of inheritance to implement the model. In fact, in a sense this is just an extension to the attribute tracer class decorator we met earlier.

Although this example utilizes the new syntactic sugar of class decorators to code attribute privacy, its attribute interception is ultimately still based upon the __getattr__ and __setattr__ operator overloading methods we met in prior chapters. When a private attribute access is detected, this version uses the raise statement to raise an exception, along with an error message; the exception may be caught in a try or allowed to terminate the script.

Here is the code, along with a self test at the bottom of the file. It will work under both Python 2.6 and 3.0 because it employs 3.0 print and raise syntax, though it catches operator overloading method attributes in 2.6 only (more on this in a moment):

"""
Privacy for attributes fetched from class instances.
See self-test code at end of file for a usage example.
Decorator same as: Doubler = Private('data', 'size')(Doubler).
Private returns onDecorator, onDecorator returns onInstance,
and each onInstance instance embeds a Doubler instance.
"""

traceMe = False
def trace(*args):
    if traceMe: print('[' + ' '.join(map(str, args)) + ']')

def Private(*privates):                          # privates in enclosing scope
    def onDecorator(aClass):                     # aClass in enclosing scope
        class onInstance:                        # wrapped in instance attribute
            def __init__(self, *args, **kargs):
                self.wrapped = aClass(*args, **kargs)
            def __getattr__(self, attr):         # My attrs don't call getattr
                trace('get:', attr)              # Others assumed in wrapped
                if attr in privates:
                    raise TypeError('private attribute fetch: ' + attr)
                else:
                    return getattr(self.wrapped, attr)
            def __setattr__(self, attr, value):             # Outside accesses
                trace('set:', attr, value)                  # Others run normally
                if attr == 'wrapped':                       # Allow my attrs
                    self.__dict__[attr] = value             # Avoid looping
                elif attr in privates:
                    raise TypeError('private attribute change: ' + attr)
                else:
                    setattr(self.wrapped, attr, value)      # Wrapped obj attrs
        return onInstance                                   # Or use __dict__
    return onDecorator


if __name__ == '__main__':
    traceMe = True

    @Private('data', 'size')                   # Doubler = Private(...)(Doubler)
    class Doubler:
        def __init__(self, label, start):
            self.label = label                 # Accesses inside the subject class
            self.data  = start                 # Not intercepted: run normally
        def size(self):
            return len(self.data)              # Methods run with no checking
        def double(self):                      # Because privacy not inherited
            for i in range(self.size()):
                self.data[i] = self.data[i] * 2
        def display(self):
            print('%s => %s' % (self.label, self.data))

    X = Doubler('X is', [1, 2, 3])
    Y = Doubler('Y is', [−10, −20, −30])

    # The followng all succeed
    print(X.label)                             # Accesses outside subject class
    X.display(); X.double(); X.display()       # Intercepted: validated, delegated
    print(Y.label)
    Y.display(); Y.double()
    Y.label = 'Spam'
    Y.display()

    # The following all fail properly
    """
    print(X.size())          # prints "TypeError: private attribute fetch: size"
    print(X.data)
    X.data = [1, 1, 1]
    X.size = lambda S: 0
    print(Y.data)
    print(Y.size())
    """

When traceMe is True, the module file’s self-test code produces the following output. Notice how the decorator catches and validates both attribute fetches and assignments run outside of the wrapped class, but does not catch attribute accesses inside the class itself:

[set: wrapped <__main__.Doubler object at 0x02B2AAF0>]
[set: wrapped <__main__.Doubler object at 0x02B2AE70>]
[get: label]
X is
[get: display]
X is => [1, 2, 3]
[get: double]
[get: display]
X is => [2, 4, 6]
[get: label]
Y is
[get: display]
Y is => [−10, −20, −30]
[get: double]
[set: label Spam]
[get: display]
Spam => [−20, −40, −60]

This code is a bit complex, and you’re probably best off tracing through it on your own to see how it works. To help you study, though, here are a few highlights worth mentioning.

Now that we have a Private implementation, it’s straightforward to generalize the code to allow for Public declarations too—they are essentially the inverse of Private declarations, so we need only negate the inner test. The example listed in this section allows a class to use decorators to define a set of either Private or Public instance attributes (attributes stored on an instance or inherited from its classes), with the following semantics:

Private and Public declarations are intended to be mutually exclusive: when using Private, all undeclared names are considered Public, and when using Public, all undeclared names are considered Private. They are essentially inverses, though undeclared names not created by class methods behave slightly differently—they can be assigned and thus created outside the class under Private (all undeclared names are accessible), but not under Public (all undeclared names are inaccessible).

Again, study this code on your own to get a feel for how this works. Notice that this scheme adds an additional fourth level of state retention at the top, beyond that described in the preceding section: the test functions used by the lambdas are saved in an extra enclosing scope. This example is coded to run under either Python 2.6 or 3.0, though it comes with a caveat when run under 3.0 (explained briefly in the file’s docstring and expanded on after the code):

"""
Class decorator with Private and Public attribute declarations.
Controls access to attributes stored on an instance, or inherited
by it from its classes. Private declares attribute names that
cannot be fetched or assigned outside the decorated class, and
Public declares all the names that can. Caveat: this works in
3.0 for normally named attributes only: __X__ operator overloading
methods implicitly run for built-in operations do not trigger
either __getattr__ or __getattribute__ in new-style classes.
Add __X__ methods here to intercept and delegate built-ins.
"""

traceMe = False
def trace(*args):
    if traceMe: print('[' + ' '.join(map(str, args)) + ']')

def accessControl(failIf):
    def onDecorator(aClass):
        class onInstance:
            def __init__(self, *args, **kargs):
                self.__wrapped = aClass(*args, **kargs)
            def __getattr__(self, attr):
                trace('get:', attr)
                if failIf(attr):
                    raise TypeError('private attribute fetch: ' + attr)
                else:
                    return getattr(self.__wrapped, attr)
            def __setattr__(self, attr, value):
                trace('set:', attr, value)
                if attr == '_onInstance__wrapped':
                    self.__dict__[attr] = value
                elif failIf(attr):
                    raise TypeError('private attribute change: ' + attr)
                else:
                    setattr(self.__wrapped, attr, value)
        return onInstance
    return onDecorator

def Private(*attributes):
    return accessControl(failIf=(lambda attr: attr in attributes))

def Public(*attributes):
    return accessControl(failIf=(lambda attr: attr not in attributes))

See the prior example’s self-test code for a usage example. Here’s a quick look at these class decorators in action at the interactive prompt (they work the same in 2.6 and 3.0); as advertised, non-Private or Public names can be fetched and changed from outside the subject class, but Private or non-Public names cannot:

>>> from access import Private, Public

>>> @Private('age')                             # Person = Private('age')(Person)
... class Person:                               # Person = onInstance with state
...     def __init__(self, name, age):
...         self.name = name
...         self.age  = age                     # Inside accesses run normally
...
>>> X = Person('Bob', 40)
>>> X.name                                      # Outside accesses validated
'Bob'
>>> X.name = 'Sue'
>>> X.name
'Sue'
>>> X.age
TypeError: private attribute fetch: age
>>> X.age = 'Tom'
TypeError: private attribute change: age

>>> @Public('name')
... class Person:
...     def __init__(self, name, age):
...         self.name = name
...         self.age  = age
...
>>> X = Person('bob', 40)                       # X is an onInstance
>>> X.name                                      # onInstance embeds Person
'bob'
>>> X.name = 'Sue'
>>> X.name
'Sue'
>>> X.age
TypeError: private attribute fetch: age
>>> X.age = 'Tom'
TypeError: private attribute change: age

To help you analyze the code, here are a few final notes on this version. Since this is just a generalization of the preceding section’s example, most of the notes there apply here as well.

As is, this example works as planned under Python 2.6 and 3.0 (provided operator overloading methods to be delegated are redefined in the wrapper). As with most software, though, there is always room for improvement.

C:misc> c:python26python
>>> from access import Private
>>> @Private('age')
... class Person:
...     def __init__(self):
...         self.age = 42
...     def __str__(self):
...         return 'Person: ' + str(self.age)
...     def __add__(self, yrs):
...         self.age += yrs
...
>>> X = Person()
>>> X.age                                   # Name validations fail correctly
TypeError: private attribute fetch: age
>>> print(X)                                # __getattr__ => runs Person.__str__
Person: 42
>>> X + 10                                  # __getattr__ => runs Person.__add__
>>> print(X)                                # __getattr__ => runs Person.__str__
Person: 52

C:misc> c:python30python
>>> from access import Private
>>> @Private('age')
... class Person:
...     def __init__(self):
...         self.age = 42
...     def __str__(self):
...         return 'Person: ' + str(self.age)
...     def __add__(self, yrs):
...         self.age += yrs
...
>>> X = Person()                            # Name validations still work
>>> X.age                                   # But 3.0 fails to delegate built-ins!
TypeError: private attribute fetch: age
>>> print(X)
<access.onInstance object at 0x025E0790>
>>> X + 10
TypeError: unsupported operand type(s) for +: 'onInstance' and 'int'
>>> print(X)
<access.onInstance object at 0x025E0790>

def accessControl(failIf):
    def onDecorator(aClass):
        class onInstance:
            def __init__(self, *args, **kargs):
                self.__wrapped = aClass(*args, **kargs)

            # Intercept and delegate operator overloading methods
            def __str__(self):
                return str(self.__wrapped)
            def __add__(self, other):
                return self.__wrapped + other
            def __getitem__(self, index):
                return self.__wrapped[index]         # If needed
            def __call__(self, *args, **kargs):
                return self.__wrapped(*arg, *kargs)  # If needed
            ...plus any others needed...

            # Intercept and delegate named attributes
            def __getattr__(self, attr):
                ...
            def __setattr__(self, attr, value):
                ...
        return onInstance
    return onDecorator

# trace support as before

def accessControl(failIf):
    def onDecorator(aClass):
        def getattributes(self, attr):
            trace('get:', attr)
            if failIf(attr):
                raise TypeError('private attribute fetch: ' + attr)
            else:
                return object.__getattribute__(self, attr)
        aClass.__getattribute__ = getattributes
        return aClass
    return onDecorator

def Private(*attributes):
    return accessControl(failIf=(lambda attr: attr in attributes))

def Public(*attributes):
    return accessControl(failIf=(lambda attr: attr not in attributes))

Now that I’ve gone to such great lengths to add Private and Public attribute declarations for Python code, I must again remind you that it is not entirely Pythonic to add access controls to your classes like this. In fact, most Python programmers will probably find this example to be largely or totally irrelevant, apart from serving as a demonstration of decorators in action. Most large Python programs get by successfully without any such controls at all. If you do wish to regulate attribute access in order to eliminate coding mistakes, though, or happen to be a soon-to-be-ex-C++-or-Java programmer, most things are possible with Python’s operator overloading and introspection tools.

In the object-oriented tutorial of Chapter 27, we wrote a class that gave a raise to objects representing people based upon a passed-in percentage:

class Person:
     ...
     def giveRaise(self, percent):
        self.pay = int(self.pay * (1 + percent))

There, we noted that if we wanted the code to be robust it would be a good idea to check the percentage to make sure it’s not too large or too small. We could implement such a check with either if or assert statements in the method itself, using inline tests:

class Person:
    def giveRaise(self, percent):                # Validate with inline code
        if percent < 0.0 or percent > 1.0:
            raise TypeError, 'percent invalid'
        self.pay = int(self.pay * (1 + percent))

class Person:                                    # Validate with asserts
    def giveRaise(self, percent):
        assert percent >= 0.0 and percent <= 1.0, 'percent invalid'
        self.pay = int(self.pay * (1 + percent))

However, this approach clutters up the method with inline tests that will probably be useful only during development. For more complex cases, this can become tedious (imagine trying to inline the code needed to implement the attribute privacy provided by the last section’s decorator). Perhaps worse, if the validation logic ever needs to change, there may be arbitrarily many inline copies to find and update.

A more useful and interesting alternative would be to develop a general tool that can perform range tests for us automatically, for the arguments of any function or method we might code now or in the future. A decorator approach makes this explicit and convenient:

class Person:
    @rangetest(percent=(0.0, 1.0))               # Use decorator to validate
    def giveRaise(self, percent):
        self.pay = int(self.pay * (1 + percent))

Isolating validation logic in a decorator simplifies both clients and future maintenance.

Notice that our goal here is different than the attribute validations coded in the prior chapter’s final example. Here, we mean to validate the values of function arguments when passed, rather than attribute values when set. Python’s decorator and introspection tools allow us to code this new task just as easily.

Let’s start with a basic range test implementation. To keep things simple, we’ll begin by coding a decorator that works only for positional arguments and assumes they always appear at the same position in every call; they cannot be passed by keyword name, and we don’t support additional **args keywords in calls because this can invalidate the positions declared in the decorator. Code the following in a file called devtools.py:

def rangetest(*argchecks):                  # Validate positional arg ranges
    def onDecorator(func):
        if not __debug__:                   # True if "python -O main.py args..."
            return func                     # No-op: call original directly
        else:                               # Else wrapper while debugging
            def onCall(*args):
                for (ix, low, high) in argchecks:
                    if args[ix] < low or args[ix] > high:
                        errmsg = 'Argument %s not in %s..%s' % (ix, low, high)
                        raise TypeError(errmsg)
                return func(*args)
            return onCall
    return onDecorator

As is, this code is mostly a rehash of the coding patterns we explored earlier: we use decorator arguments, nested scopes for state retention, and so on.

We also use nested def statements to ensure that this works for both simple functions and methods, as we learned earlier. When used for a class method, onCall receives the subject class’s instance in the first item in *args and passes this along to self in the original method function; argument numbers in range tests start at 1 in this case, not 0.

Also notice this code’s use of the __debug__ built-in variable, though—Python sets this to True, unless it’s being run with the –O optimize command-line flag (e.g., python –O main.py). When __debug__ is False, the decorator returns the origin function unchanged, to avoid extra calls and their associated performance penalty.

This first iteration solution is used as follows:

# File devtools_test.py

from devtools import rangetest
print(__debug__)                           # False if "python –O main.py"

@rangetest((1, 0, 120))                    # persinfo = rangetest(...)(persinfo)
def persinfo(name, age):                   # age must be in 0..120
    print('%s is %s years old' % (name, age))

@rangetest([0, 1, 12], [1, 1, 31], [2, 0, 2009])
def birthday(M, D, Y):
    print('birthday = {0}/{1}/{2}'.format(M, D, Y))

class Person:
    def __init__(self, name, job, pay):
        self.job  = job
        self.pay  = pay

    @rangetest([1, 0.0, 1.0])              # giveRaise = rangetest(...)(giveRaise)
    def giveRaise(self, percent):          # Arg 0 is the self instance here
        self.pay = int(self.pay * (1 + percent))

# Comment lines raise TypeError unless "python -O" used on shell command line

persinfo('Bob Smith', 45)                  # Really runs onCall(...) with state
#persinfo('Bob Smith', 200)                    # Or person if –O cmd line argument

birthday(5, 31, 1963)
#birthday(5, 32, 1963)

sue = Person('Sue Jones', 'dev', 100000)
sue.giveRaise(.10)                         # Really runs onCall(self, .10)
print(sue.pay)                             # Or giveRaise(self, .10) if –O
#sue.giveRaise(1.10)
#print(sue.pay)

When run, valid calls in this code produce the following output (all the code in this section works the same under Python 2.6 and 3.0, because function decorators are supported in both, we’re not using attribute delegation, and we use 3.0-style print calls and exception construction syntax):

C:misc> C:python30python devtools_test.py
True
Bob Smith is 45 years old
birthday = 5/31/1963
110000

Uncommenting any of the invalid calls causes a TypeError to be raised by the decorator. Here’s the result when the last two lines are allowed to run (as usual, I’ve omitted some of the error message text here to save space):

C:misc> C:python30python devtools_test.py
True
Bob Smith is 45 years old
birthday = 5/31/1963
110000
TypeError: Argument 1 not in 0.0..1.0

Running Python with its –O flag at a system command line will disable range testing, but also avoid the performance overhead of the wrapping layer—we wind up calling the original undecorated function directly. Assuming this is a debugging tool only, you can use this flag to optimize your program for production use:

C:misc> C:python30python –O devtools_test.py
False
Bob Smith is 45 years old
birthday = 5/31/1963
110000
231000

The prior version illustrates the basics we need to employ, but it’s fairly limited—it supports validating arguments passed by position only, and it does not validate keyword arguments (in fact, it assumes that no keywords are passed in a way that makes argument position numbers incorrect). Additionally, it does nothing about arguments with defaults that may be omitted in a given call. That’s fine if all your arguments are passed by position and never defaulted, but less than ideal in a general tool. Python supports much more flexible argument-passing modes, which we’re not yet addressing.

The mutation of our example shown next does better. By matching the wrapped function’s expected arguments against the actual arguments passed in a call, it supports range validations for arguments passed by either position or keyword name, and it skips testing for default arguments omitted in the call. In short, arguments to be validated are specified by keyword arguments to the decorator, which later steps through both the *pargs positionals tuple and the **kargs keywords dictionary to validate.

"""
File devtools.py: function decorator that performs range-test
validation for passed arguments. Arguments are specified by
keyword to the decorator. In the actual call, arguments may
be passed by position or keyword, and defaults may be omitted.
See devtools_test.py for example use cases.
"""

trace = True

def rangetest(**argchecks):                 # Validate ranges for both+defaults
    def onDecorator(func):                  # onCall remembers func and argchecks
        if not __debug__:                   # True if "python –O main.py args..."
            return func                     # Wrap if debugging; else use original
        else:
            import sys
            code     = func.__code__
            allargs  = code.co_varnames[:code.co_argcount]
            funcname = func.__name__

            def onCall(*pargs, **kargs):
                # All pargs match first N expected args by position
                # The rest must be in kargs or be omitted defaults
                positionals = list(allargs)
                positionals = positionals[:len(pargs)]

                for (argname, (low, high)) in argchecks.items():
                    # For all args to be checked
                    if argname in kargs:
                        # Was passed by name
                        if kargs[argname] < low or kargs[argname] > high:
                            errmsg = '{0} argument "{1}" not in {2}..{3}'
                            errmsg = errmsg.format(funcname, argname, low, high)
                            raise TypeError(errmsg)

                    elif argname in positionals:
                        # Was passed by position
                        position = positionals.index(argname)
                        if pargs[position] < low or pargs[position] > high:
                            errmsg = '{0} argument "{1}" not in {2}..{3}'
                            errmsg = errmsg.format(funcname, argname, low, high)
                            raise TypeError(errmsg)
                    else:
                        # Assume not passed: default
                        if trace:
                            print('Argument "{0}" defaulted'.format(argname))

                return func(*pargs, **kargs)    # OK: run original call
            return onCall
    return onDecorator

The following test script shows how the decorator is used—arguments to be validated are given by keyword decorator arguments, and at actual calls we can pass by name or position and omit arguments with defaults even if they are to be validated otherwise:

# File devtools_test.py
# Comment lines raise TypeError unless "python –O" used on shell command line
from devtools import rangetest


# Test functions, positional and keyword

@rangetest(age=(0, 120))                  # persinfo = rangetest(...)(persinfo)
def persinfo(name, age):
    print('%s is %s years old' % (name, age))

@rangetest(M=(1, 12), D=(1, 31), Y=(0, 2009))
def birthday(M, D, Y):
    print('birthday = {0}/{1}/{2}'.format(M, D, Y))

persinfo('Bob', 40)
persinfo(age=40, name='Bob')
birthday(5, D=1, Y=1963)
#persinfo('Bob', 150)
#persinfo(age=150, name='Bob')
#birthday(5, D=40, Y=1963)


# Test methods, positional and keyword

class Person:
    def __init__(self, name, job, pay):
        self.job  = job
        self.pay  = pay
                                          # giveRaise = rangetest(...)(giveRaise)
    @rangetest(percent=(0.0, 1.0))        # percent passed by name or position
    def giveRaise(self, percent):
        self.pay = int(self.pay * (1 + percent))

bob = Person('Bob Smith', 'dev', 100000)
sue = Person('Sue Jones', 'dev', 100000)
bob.giveRaise(.10)
sue.giveRaise(percent=.20)
print(bob.pay, sue.pay)
#bob.giveRaise(1.10)
#bob.giveRaise(percent=1.20)


# Test omitted defaults: skipped

@rangetest(a=(1, 10), b=(1, 10), c=(1, 10), d=(1, 10))
def omitargs(a, b=7, c=8, d=9):
    print(a, b, c, d)

omitargs(1, 2, 3, 4)
omitargs(1, 2, 3)
omitargs(1, 2, 3, d=4)
omitargs(1, d=4)
omitargs(d=4, a=1)
omitargs(1, b=2, d=4)
omitargs(d=8, c=7, a=1)

#omitargs(1, 2, 3, 11)         # Bad d
#omitargs(1, 2, 11)           # Bad c
#omitargs(1, 2, 3, d=11)       # Bad d
#omitargs(11, d=4)           # Bad a
#omitargs(d=4, a=11)         # Bad a
#omitargs(1, b=11, d=4)       # Bad b
#omitargs(d=8, c=7, a=11)     # Bad a

When this script is run, out-of-range arguments raise an exception as before, but arguments may be passed by either name or position, and omitted defaults are not validated. This code runs on both 2.6 and 3.0, but extra tuple parentheses print in 2.6. Trace its output and test this further on your own to experiment; it works as before, but its scope has been broadened:

C:misc> C:python30python devtools_test.py
Bob is 40 years old
Bob is 40 years old
birthday = 5/1/1963
110000 120000
1 2 3 4
Argument "d" defaulted
1 2 3 9
1 2 3 4
Argument "c" defaulted
Argument "b" defaulted
1 7 8 4
Argument "c" defaulted
Argument "b" defaulted
1 7 8 4
Argument "c" defaulted
1 2 8 4
Argument "b" defaulted
1 7 7 8

On validation errors, we get an exception as before (unless the –O command-line argument is passed to Python) when one of the method test lines is uncommented:

TypeError: giveRaise argument "percent" not in 0.0..1.0

This decorator’s code relies on both introspection APIs and subtle constraints of argument passing. To be fully general we could in principle try to mimic Python’s argument matching logic in its entirety to see which names have been passed in which modes, but that’s far too much complexity for our tool. It would be better if we could somehow match arguments passed by name against the set of all expected arguments’ names, in order to determine which position arguments actually appear in during a given call.

# In Python 3.0 (and 2.6 for compatibility):
>>> def func(a, b, c, d):
...     x = 1
...     y = 2
...
>>> code = func.__code__                     # Code object of function object
>>> code.co_nlocals
6
>>> code.co_varnames                         # All local var names
('a', 'b', 'c', 'd', 'x', 'y')
>>> code.co_varnames[:code.co_argcount]      # First N locals are expected args
('a', 'b', 'c', 'd')

>>> import sys                               # For backward compatibility
>>> sys.version_info                         # [0] is major release number
(3, 0, 0, 'final', 0)
>>> code = func.__code__ if sys.version_info[0] == 3 else func.func_code

Although our range-testing tool works as planned, two caveats remain. First, as mentioned earlier, calls to the original function that are not valid still fail in our final decorator. The following both trigger exceptions, for example:

omitargs()
omitargs(d=8, c=7, b=6)

These only fail, though, where we try to invoke the original function, at the end of the wrapper. While we could try to imitate Python’s argument matching to avoid this, there’s not much reason to do so—since the call would fail at this point anyhow, we might as well let Python’s own argument-matching logic detect the problem for us.

Lastly, although our final version handles positional arguments, keyword arguments, and omitted defaults, it still doesn’t do anything explicit about *args and **args that may be used in a decorated function that accepts arbitrarily many arguments. We probably don’t need to care for our purposes, though:

In other words, as it is the code supports testing arbitrary keyword arguments by name, but not arbitrary positionals that are unnamed and hence have no set position in the function’s argument signature.

In principle, we could extend the decorator’s interface to support *args in the decorated function, too, for the rare cases where this might be useful (e.g., a special argument name with a test to apply to all arguments in the wrapper’s *pargs beyond the length of the expected arguments list). Since we’ve already exhausted the space allocation for this example, though, if you care about such improvements you’ve officially crossed over into the realm of suggested exercises.

Interestingly, the function annotation feature introduced in Python 3.0 could provide an alternative to the decorator arguments used by our example to specify range tests. As we learned in Chapter 19, annotations allow us to associate expressions with arguments and return values, by coding them in the def header line itself; Python collects annotations in a dictionary and attaches it to the annotated function.

We could use this in our example to code range limits in the header line, instead of in decorator arguments. We would still need a function decorator to wrap the function in order to intercept later calls, but we would essentially trade decorator argument syntax:

@rangetest(a=(1, 5), c=(0.0, 1.0))
def func(a, b, c):                         # func = rangetest(...)(func)
    print(a + b + c)

for annotation syntax like this:

@rangetest
def func(a:(1, 5), b, c:(0.0, 1.0)):
    print(a + b + c)

That is, the range constraints would be moved into the function itself, instead of being coded externally. The following script illustrates the structure of the resulting decorators under both schemes, in incomplete skeleton code. The decorator arguments code pattern is that of our complete solution shown earlier; the annotation alternative requires one less level of nesting, because it doesn’t need to retain decorator arguments:

# Using decorator arguments

def rangetest(**argchecks):
    def onDecorator(func):
        def onCall(*pargs, **kargs):
            print(argchecks)
            for check in argchecks: pass     # Add validation code here
            return func(*pargs, **kargs)
        return onCall
    return onDecorator

@rangetest(a=(1, 5), c=(0.0, 1.0))
def func(a, b, c):                           # func = rangetest(...)(func)
    print(a + b + c)

func(1, 2, c=3)                              # Runs onCall, argchecks in scope


# Using function annotations

def rangetest(func):
    def onCall(*pargs, **kargs):
        argchecks = func.__annotations__
        print(argchecks)
        for check in argchecks: pass         # Add validation code here
        return func(*pargs, **kargs)
    return onCall

@rangetest
def func(a:(1, 5), b, c:(0.0, 1.0)):         # func = rangetest(func)
    print(a + b + c)

func(1, 2, c=3)                              # Runs onCall, annotations on func

When run, both schemes have access to the same validation test information, but in different forms—the decorator argument version’s information is retained in an argument in an enclosing scope, and the annotation version’s information is retained in an attribute of the function itself:

{'a': (1, 5), 'c': (0.0, 1.0)}
6
{'a': (1, 5), 'c': (0.0, 1.0)}
6

I’ll leave fleshing out the rest of the annotation-based version as a suggested exercise; its code would be identical to that of our complete solution shown earlier, because range-test information is simply on the function instead of in an enclosing scope. Really, all this buys us is a different user interface for our tool—it will still need to match argument names against expected argument names to obtain relative positions as before.

In fact, using annotation instead of decorator arguments in this example actually limits its utility. For one thing, annotation only works under Python 3.0, so 2.6 is no longer supported; function decorators with arguments, on the other hand, work in both versions.

More importantly, by moving the validation specifications into the def header, we essentially commit the function to a single role—since annotation allows us to code only one expression per argument, it can have only one purpose. For instance, we cannot use range-test annotations for any other role.

By contrast, because decorator arguments are coded outside the function itself, they are both easier to remove and more general—the code of the function itself does not imply a single decoration purpose. In fact, by nesting decorators with arguments, we can apply multiple augmentation steps to the same function; annotation directly supports only one. With decorator arguments, the function itself also retains a simpler, normal appearance.

Still, if you have a single purpose in mind, and you can commit to supporting 3.X only, the choice between annotation and decorator arguments is largely stylistic and subjective. As is so often true in life, one person’s annotation may well be another’s syntactic clutter....

The coding pattern we’ve arrived at for processing arguments in decorators could be applied in other contexts. Checking argument data types at development time, for example, is a straightforward extension:

def typetest(**argchecks):
    def onDecorator(func):
           ....
           def onCall(*pargs, **kargs):
                positionals = list(allargs)[:len(pargs)]
                for (argname, type) in argchecks.items():
                    if argname in kargs:
                        if not isinstance(kargs[argname], type):
                            ...
                            raise TypeError(errmsg)
                    elif argname in positionals:
                        position = positionals.index(argname)
                        if not isinstance(pargs[position], type):
                            ...
                            raise TypeError(errmsg)
                    else:
                        # Assume not passed: default
                return func(*pargs, **kargs)
            return onCall
    return onDecorator

@typetest(a=int, c=float)
def func(a, b, c, d):                    # func = typetest(...)(func)
    ...

func(1, 2, 3.0, 4)                       # Okay
func('spam', 2, 99, 4)                   # Triggers exception correctly

In fact, we might even generalize further by passing in a test function, much as we did to add Public decorations earlier; a single copy of this sort of code would suffice for both range and type testing. Using function annotations instead of decorator arguments for such a decorator, as described in the prior section, would make this look even more like type declarations in other languages:

@typetest
def func(a: int, b, c: float, d):        # func = typetest(func)
    ...                                  # Gasp!...

As you should have learned in this book, though, this particular role is generally a bad idea in working code, and not at all Pythonic (in fact, it’s often a symptom of an ex-C++ programmer’s first attempts to use Python).

Type testing restricts your function to work on specific types only, instead of allowing it to operate on any types with compatible interfaces. In effect, it limits your code and breaks its flexibility. On the other hand, every rule has exceptions; type checking may come in handy in isolated cases while debugging and when interfacing with code written in more restrictive languages, such as C++. This general pattern of argument processing might also be applicable in a variety of less controversial roles.