7
METHODS AND DECORATORS

Python’s decorators are a handy way to modify functions. Decorators were first introduced in Python 2.2, with the classmethod() and staticmethod() decorators, but were overhauled to become more flexible and readable. Along with these two original decorators, Python now provides a few right out of the box and supports the simple creation of custom decorators. But it seems as though most developers do not understand how they work behind the scenes.

This chapter aims to change that—we’ll cover what a decorator is and how to use it, as well as how to create your own decorators. Then we’ll look at using decorators to create static, class, and abstract methods and take a close look at the super() function, which allows you to place implementable code inside an abstract method.

Decorators and When to Use Them

A decorator is a function that takes another function as an argument and replaces it with a new, modified function. The primary use case for decorators is in factoring common code that needs to be called before, after, or around multiple functions. If you’ve ever written Emacs Lisp code, you may have used the defadvice decorator, which allows you to define code called around a function. If you’ve used method combinations in the Common Lisp Object System (CLOS), Python decorators follow the same concepts. We’ll look at some simple decorator definitions, and then we’ll examine some common situations in which you’d use decorators.

Creating Decorators

The odds are good that you’ve already used decorators to make your own wrapper functions. The dullest possible decorator, and the simplest example, is the identity() function, which does nothing except return the original function. Here is its definition:

def identity(f):
    return f

You would then use your decorator like this:

@identity
def foo():
    return 'bar'

You enter the name of the decorator preceded by an @ symbol and then enter the function you want to use it on. This is the same as writing the following:

def foo():
    return 'bar'
foo = identity(foo)

This decorator is useless, but it works. Let’s look at another, more useful example in Listing 7-1.

_functions = {}
def register(f):
    global _functions
    _functions[f.__name__] = f
    return f
@register
def foo():
    return 'bar'

Listing 7-1: A decorator to organize functions in a dictionary

In Listing 7-1, the register decorator stores the decorated function name into a dictionary. The _functions dictionary can then be used and accessed using the function name to retrieve a function: _functions['foo'] points to the foo() function.

In the following sections, I will explain how to write your own decorators. Then I’ll cover how the built-in decorators provided by Python work and explain how (and when) to use them.

Writing Decorators

As mentioned, decorators are often used when refactoring repeated code around functions. Consider the following set of functions that need to check whether the username they receive as an argument is the admin or not and, if the user is not an admin, raise an exception:

class Store(object):
    def get_food(self, username, food):
        if username != 'admin':
            raise Exception("This user is not allowed to get food")
        return self.storage.get(food)

    def put_food(self, username, food):
        if username != 'admin':
            raise Exception("This user is not allowed to put food")
        self.storage.put(food)

We can see there’s some repeated code here. The obvious first step to making this code more efficient is to factor the code that checks for admin status:

➊ def check_is_admin(username):
    if username != 'admin':
        raise Exception("This user is not allowed to get or put food")

class Store(object):
    def get_food(self, username, food):
        check_is_admin(username)
        return self.storage.get(food)

    def put_food(self, username, food):
        check_is_admin(username)
        self.storage.put(food)

We’ve moved the checking code into its own function ➊. Now our code looks a bit cleaner, but we can do even better if we use a decorator, as shown in Listing 7-2.

def check_is_admin(f):
 ➊ def wrapper(*args, **kwargs):
        if kwargs.get('username') != 'admin':
            raise Exception("This user is not allowed to get or put food")
        return f(*args, **kwargs)
    return wrapper

class Store(object):
    @check_is_admin
    def get_food(self, username, food):
        return self.storage.get(food)

    @check_is_admin
    def put_food(self, username, food):
        self.storage.put(food)

Listing 7-2: Adding a decorator to the factored code

We define our check_is_admin decorator ➊ and then call it whenever we need to check for access rights. The decorator inspects the arguments passed to the function using the kwargs variable and retrieves the username argument, performing the username check before calling the actual function. Using decorators like this makes it easier to manage common functionality. To anyone with much Python experience, this is probably old hat, but what you might not realize is that this naive approach to implementing decorators has some major drawbacks.

Stacking Decorators

You can also use several decorators on top of a single function or method, as shown in Listing 7-3.

def check_user_is_not(username):
    def user_check_decorator(f):
        def wrapper(*args, **kwargs):
            if kwargs.get('username') == username:
                raise Exception("This user is not allowed to get food")
            return f(*args, **kwargs)
        return wrapper
    return user_check_decorator

class Store(object):
    @check_user_is_not("admin")
    @check_user_is_not("user123")
    def get_food(self, username, food):
        return self.storage.get(food)

Listing 7-3: Using more than one decorator with a single function

Here, check_user_is_not() is a factory function for our decorator user_check_decorator(). It creates a function decorator that depends on the username variable and then returns that variable. The function user_check_decorator() will serve as a function decorator for get_food().

The function get_food() gets decorated twice using check_user_is_not(). The question here is which username should be checked first—admin or user123? The answer is in the following code, where I translated Listing 7-3 into equivalent code without using a decorator.

class Store(object):
    def get_food(self, username, food):
        return self.storage.get(food)

Store.get_food = check_user_is_not("user123")(Store.get_food)
Store.get_food = check_user_is_not("admin")(Store.get_food)

The decorator list is applied from top to bottom, so the decorators closest to the def keyword will be applied first and executed last. In the example above, the program will check for admin first and then for user123.

Writing Class Decorators

It’s also possible to implement class decorators, though these are less often used in the wild. Class decorators work in the same way as function decorators, but they act on classes rather than functions. The following is an example of a class decorator that sets attributes for two classes:

import uuid

def set_class_name_and_id(klass):
    klass.name = str(klass)
    klass.random_id = uuid.uuid4()
    return klass

@set_class_name_and_id
class SomeClass(object):
    pass

When the class is loaded and defined, it will set the name and random_id attributes, like so:

>>> SomeClass.name
"<class '__main__.SomeClass'>"
>>> SomeClass.random_id
UUID('d244dc42-f0ca-451c-9670-732dc32417cd')

As with function decorators, this can be handy for factorizing common code that manipulates classes.

Another possible use for class decorators is to wrap a function or class with classes. For example, class decorators are often used for wrapping a function that’s storing a state. The following example wraps the print() function to check how many times it has been called in a session:

class CountCalls(object):
    def __init__(self, f):
        self.f = f
        self.called = 0
    def __call__(self, *args, **kwargs):
        self.called += 1
        return self.f(*args, **kwargs)

@CountCalls
def print_hello():
    print("hello")

We can then use this to check how many times the function print_hello() has been called:

>>> print_hello.called
0
>>> print_hello()
hello
>>> print_hello.called
1

Retrieving Original Attributes with the update_wrapper Decorator

As mentioned, a decorator replaces the original function with a new one built on the fly. However, this new function lacks many of the attributes of the original function, such as its docstring and its name. Listing 7-4 shows how the function foobar() loses its docstring and its name attribute once it is decorated with the is_admin decorator.

>>> def is_admin(f):
...     def wrapper(*args, **kwargs):
...         if kwargs.get('username') != 'admin':
...             raise Exception("This user is not allowed to get food")
...         return f(*args, **kwargs)
...     return wrapper
...
>>> def foobar(username="someone"):
...     """Do crazy stuff."""
...     pass
...
>>> foobar.func_doc
'Do crazy stuff.'
>>> foobar.__name__
'foobar'
>>> @is_admin
... def foobar(username="someone"):
...     """Do crazy stuff."""
...     pass
...
>>> foobar.__doc__
>>> foobar.__name__
'wrapper'

Listing 7-4: A decorated function loses its docstring and name attributes.

Not having the correct docstring and name attribute for a function can be problematic in various situations, such as when generating the source code documentation.

Fortunately, the functools module in the Python Standard Library solves this problem with the update_wrapper() function, which copies the attributes from the original function that were lost to the wrapper itself. The source code of update_wrapper() is shown in Listing 7-5.

WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__qualname__', '__doc__',
                       '__annotations__')
WRAPPER_UPDATES = ('__dict__',)
def update_wrapper(wrapper,
                   wrapped,
                   assigned = WRAPPER_ASSIGNMENTS,
                   updated = WRAPPER_UPDATES):
    for attr in assigned:
        try:
            value = getattr(wrapped, attr)
        except AttributeError:
            pass
        else:
            setattr(wrapper, attr, value)
    for attr in updated:
        getattr(wrapper, attr).update(getattr(wrapped, attr, {}))
    # Issue #17482: set __wrapped__ last so we don't inadvertently copy it
    # from the wrapped function when updating __dict__
    wrapper.__wrapped__ = wrapped
    # Return the wrapper so this can be used as a decorator via partial()
    return wrapper

Listing 7-5: The update_wrapper() source code

In Listing 7-5, the update_wrapper() source code highlights which attributes are worth saving when wrapping a function with a decorator. By default, the __name__ attribute, __doc__ attribute, and some other attributes are copied. You can also personalize which attributes of a function are copied to the decorated function. When we use update_wrapper() to rewrite our example from Listing 7-4, things are much nicer:

>>> def foobar(username="someone"):
...     """Do crazy stuff."""
...     pass
...
>>> foobar = functools.update_wrapper(is_admin, foobar)
>>> foobar.__name__
'foobar'
>>> foobar.__doc__
'Do crazy stuff.'

Now the foobar() function has the correct name and docstring even when decorated by is_admin.

wraps: A Decorator for Decorators

It can get tedious to use update_wrapper() manually when creating decorators, so functools provides a decorator for decorators called wraps. Listing 7-6 shows the wraps decorator in use.

import functools

def check_is_admin(f):
    @functools.wraps(f)
    def wrapper(*args, **kwargs):
        if kwargs.get('username') != 'admin':
            raise Exception("This user is not allowed to get food")
        return f(*args, **kwargs)
    return wrapper

class Store(object):
    @check_is_admin
    def get_food(self, username, food):
        """Get food from storage."""
        return self.storage.get(food)

Listing 7-6: Updating our decorator with wraps from functools

With functools.wrap, the decorator function check_is_admin() that returns the wrapper() function takes care of copying the docstring, name function, and other information from the function f passed as argument. Thus, the decorated function (get_food(), in this case) still sees its unchanged signature.

Extracting Relevant Information with inspect

In our examples so far, we have assumed that the decorated function will always have a username passed to it as a keyword argument, but that might not be the case. It might instead have a bunch of information from which we need to extract the username to check. With this in mind, we’ll build a smarter version of our decorator that can look at the decorated function’s arguments and pull out what it needs.

For this, Python has the inspect module, which allows us to retrieve a function’s signature and operate on it, as shown in Listing 7-7.

import functools
import inspect

def check_is_admin(f):
    @functools.wraps(f)
    def wrapper(*args, **kwargs):
        func_args = inspect.getcallargs(f, *args, **kwargs)
        if func_args.get('username') != 'admin':
            raise Exception("This user is not allowed to get food")
        return f(*args, **kwargs)
    return wrapper
@check_is_admin
def get_food(username, type='chocolate'):
    return type + " nom nom nom!"

Listing 7-7: Using tools from the inspect module to extract information

The function that does the heavy lifting here is inspect.getcallargs(), which returns a dictionary containing the names and values of the arguments as key-value pairs. In our example, this function returns {'username': 'admin','type': 'chocolate'}. That means that our decorator does not have to check whether the username parameter is a positional or a keyword argument; all the decorator has to do is look for username in the dictionary.

Using functools.wraps and the inspect module, you should be able to write any custom decorator that you would ever need. However, do not abuse the inspect module: while being able to guess what the function will accept as an argument sounds handy, this capability can be fragile, breaking easily when function signatures change. Decorators are a terrific way to implement the Don’t Repeat Yourself mantra so cherished by developers.

How Methods Work in Python

Methods are pretty simple to use and understand, and you’ve likely just used them correctly without delving in much deeper than you needed to. But to understand what certain decorators do, you need to know how methods work behind the scenes.

A method is a function that is stored as a class attribute. Let’s have a look at what happens when we try to access such an attribute directly:

>>> class Pizza(object):
...     def __init__(self, size):
...         self.size = size
...     def get_size(self):
...         return self.size
...
>>> Pizza.get_size
<function Pizza.get_size at 0x7fdbfd1a8b90>

We are told that get_size() is a function—but why is that? The reason is that at this stage, get_size() is not tied to any particular object. Therefore, it is treated as a normal function. Python will raise an error if we try to call it directly, like so:

>>> Pizza.get_size()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: get_size() missing 1 required positional argument: 'self'

Python complains that we have not provided the necessary self argument. Indeed, as it is not bound to any object, the self argument cannot be set automatically. However, we are able to use the get_size() function not only by passing an arbitrary instance of the class to the method if we want to but also by passing any object, as long as it has the properties that the method expects to find. Here’s an example:

>>> Pizza.get_size(Pizza(42))
42

This call works, just as promised. It is, however, not very convenient: we have to refer to the class every time we want to call one of its methods.

So Python goes the extra mile for us by binding a class’s methods to its instances. In other words, we can access get_size() from any Pizza instance, and, better still, Python will automatically pass the object itself to the method’s self parameter, like so:

>>> Pizza(42).get_size
<bound method Pizza.get_size of <__main__.Pizza object at 0x7f3138827910>>
>>> Pizza(42).get_size()
42

As expected, we do not have to provide any argument to get_size(), since it’s a bound method: its self argument is automatically set to our Pizza instance. Here is an even clearer example:

>>> m = Pizza(42).get_size
>>> m()
42

As long as you have a reference to the bound method, you do not even have to keep a reference to your Pizza object. Moreover, if you have a reference to a method but you want to find out which object it is bound to, you can just check the method’s __self__ property, like so:

>>> m = Pizza(42).get_size
>>> m.__self__
<__main__.Pizza object at 0x7f3138827910>
>>> m == m.__self__.get_size
True

Obviously, we still have a reference to our object, and we can find it if we want.

Static Methods

Static methods belong to a class, rather than an instance of a class, so they don’t actually operate on or affect class instances. Instead, a static method operates on the parameters it takes. Static methods are generally used to create utility functions, because they do not depend on the state of the class or its objects.

For example, in Listing 7-8, the static mix_ingredients() method belongs to the Pizza class but could actually be used to mix ingredients for any other food.

class Pizza(object):
    @staticmethod
    def mix_ingredients(x, y):
        return x + y

    def cook(self):
        return self.mix_ingredients(self.cheese, self.vegetables)

Listing 7-8: Creating a static method as part of a class

You could write mix_ingredients() as a non-static method if you wanted to, but it would take a self argument that would never actually be used. Using the @staticmethod decorator gives us several things.

The first is speed: Python does not have to instantiate a bound method for each Pizza object we create. Bound methods are objects, too, and creating them has a CPU and memory cost—even if it’s low. Using a static method lets us avoid that, like so:

>>> Pizza().cook is Pizza().cook
False
>>> Pizza().mix_ingredients is Pizza.mix_ingredients
True
>>> Pizza().mix_ingredients is Pizza().mix_ingredients
True

Second, static methods improve the readability of the code. When we see @staticmethod, we know that the method does not depend on the state of the object.

Third, static methods can be overridden in subclasses. If instead of a static method, we used a mix_ingredients() function defined at the top level of our module, a class inheriting from Pizza wouldn’t be able to change the way we mix ingredients for our pizza without overriding the cook() method itself. With static methods, the subclasses can override the method for their own purposes.

Unfortunately, Python is not always able to detect for itself whether a method is static or not—I call that a defect of the language design. One possible approach is to add a check that detects such pattern and emits a warning using flake8. We will look into how to do this in “Extending flake8 with AST Checks” on page 140.

Class Methods

Class methods are bound to a class rather than its instances. That means that those methods cannot access the state of the object but only the state and methods of the class. Listing 7-9 shows how to write a class method.

>>> class Pizza(object):
...     radius = 42
...     @classmethod
...     def get_radius(cls):
...         return cls.radius
...
>>> Pizza.get_radius
<bound method type.get_radius of <class '__main__.Pizza'>>
>>> Pizza().get_radius
<bound method type.get_radius of <class '__main__.Pizza'>>
>>> Pizza.get_radius is Pizza().get_radius
True
>>> Pizza.get_radius()
42

Listing 7-9: Binding a class method to its class

As you can see, there are various ways to access the get_radius() class method, but however you choose to access it, the method is always bound to the class it is attached to. Also, its first argument must be the class itself. Remember: classes are objects too!

Class methods are principally useful for creating factory methods, which instantiate objects using a different signature than __init__:

class Pizza(object):
    def __init__(self, ingredients):
        self.ingredients = ingredients

    @classmethod
    def from_fridge(cls, fridge):
        return cls(fridge.get_cheese() + fridge.get_vegetables())

If we used a @staticmethod here instead of a @classmethod, we would have to hardcode the Pizza class name in our method, making any class inheriting from Pizza unable to use our factory for its own purposes. In this case, however, we provide a from_fridge() factory method that we can pass a Fridge object to. If we call this method with something like Pizza.from_fridge(myfridge), it returns a brand-new Pizza with ingredients taken from what’s available in myfridge.

Any time you write a method that cares only about the class of the object and not about the object’s state, it should be declared as a class method.

Abstract Methods

An abstract method is defined in an abstract base class that may not itself provide any implementation. When a class has an abstract method, it cannot be instantiated. As a consequence, an abstract class (defined as a class that has at least one abstract method) must be used as a parent class by another class. This subclass will be in charge of implementing the abstract method, making it possible to instantiate the parent class.

We can use abstract base classes to make clear the relationships between other, connected classes derived from the base class but make the abstract base class itself impossible to instantiate. By using abstract base classes, you can ensure the classes derived from the base class implement particular methods from the base class, or an exception will be raised. The following example shows the simplest way to write an abstract method in Python:

class Pizza(object):
    @staticmethod
    def get_radius():
        raise NotImplementedError

With this definition, any class inheriting from Pizza must implement and override the get_radius() method; otherwise, calling the method raises the exception shown here. This is handy for making sure that each subclass of Pizza implements its own way of computing and returning its radius.

This way of implementing abstract methods has a drawback: if you write a class that inherits from Pizza but forget to implement get_radius(), the error is raised only if you try to use that method at runtime. Here’s an example:

>>> Pizza()
<__main__.Pizza object at 0x7fb747353d90>
>>> Pizza().get_radius()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in get_radius
NotImplementedError

As Pizza is directly instantiable, there’s no way to prevent this from happening. One way to make sure you get an early warning about forgetting to implement and override the method, or trying to instantiate an object with abstract methods, is to use Python’s built-in abc (abstract base classes) module instead, like so:

import abc

class BasePizza(object, metaclass=abc.ABCMeta):

    @abc.abstractmethod
    def get_radius(self):
         """Method that should do something."""

The abc module provides a set of decorators to use on top of methods that will be defined as abstracts and a metaclass to enable this. When you use abc and its special metaclass, as shown above, instantiating a BasePizza or a class inheriting from it that doesn’t override get_radius() causes a TypeError:

>>> BasePizza()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class BasePizza with abstract methods
get_radius

We try to instantiate the abstract BasePizza class and are immediately told it can’t be done!

While using abstract methods doesn’t guarantee that the method is implemented by the user, this decorator helps you catch the error earlier. This is especially handy when you are providing interfaces that must be implemented by other developers; it’s a good documentation hint.

Mixing Static, Class, and Abstract Methods

Each of these decorators is useful on its own, but the time may come when you’ll have to use them together.

For example, you could define a factory method as a class method while forcing the implementation to be made in a subclass. In that case, you’d need to have a class method defined as both an abstract method and a class method. This section gives some tips that will help you with that.

First, an abstract method’s prototype is not set in stone. When you implement the method, there is nothing stopping you from extending the argument list as you see fit. Listing 7-10 is an example of code in which a subclass extends the signature of the abstract method of its parent.

import abc

class BasePizza(object, metaclass=abc.ABCMeta):

    @abc.abstractmethod
    def get_ingredients(self):
         """Returns the ingredient list."""

class Calzone(BasePizza):
    def get_ingredients(self, with_egg=False):
        egg = Egg() if with_egg else None
        return self.ingredients + [egg]

Listing 7-10: Using a subclass to extend the signature of the abstract method of its parent

We define the Calzone subclass to inherit from the BasePizza class. We can define the Calzone subclass’s methods any way we like, as long as they support the interface we define in BasePizza. This includes implementing the methods as either class or static methods. The following code defines an abstract get_ingredients() method in the base class and a static get_ingredients() method in the DietPizza subclass:

import abc

class BasePizza(object, metaclass=abc.ABCMeta):

    @abc.abstractmethod
    def get_ingredients(self):
         """Returns the ingredient list."""

class DietPizza(BasePizza):
    @staticmethod
    def get_ingredients():
        return None

Even though our static get_ingredients() method doesn’t return a result based on the object’s state, it supports our abstract BasePizza class’s interface, so it’s still valid.

It is also possible to use the @staticmethod and @classmethod decorators on top of @abstractmethod in order to indicate that a method is, for example, both static and abstract, as shown in Listing 7-11.

import abc

class BasePizza(object, metaclass=abc.ABCMeta):

    ingredients = ['cheese']

    @classmethod
    @abc.abstractmethod
    def get_ingredients(cls):
         """Returns the ingredient list."""
         return cls.ingredients

Listing 7-11: Using a class method decorator with abstract methods

The abstract method get_ingredients() needs to be implemented by a subclass, but it’s also a class method, meaning the first argument it will receive will be a class (not an object).

Note that by defining get_ingredients() as a class method in BasePizza like this, you are not forcing any subclasses to define get_ingredients() as a class method—it could be a regular method. The same would apply if we had defined it as a static method: there’s no way to force subclasses to implement abstract methods as a specific kind of method. As we have seen, you can change the signature of an abstract method when implementing it in a subclass in any way you like.

Putting Implementations in Abstract Methods

Hold the phone: in Listing 7-12, we have an implementation in an abstract method. Can we do that? The answer is yes. Python does not have a problem with it! You can put code in your abstract methods and call it using super(), as demonstrated in Listing 7-12.

import abc

class BasePizza(object, metaclass=abc.ABCMeta):

    default_ingredients = ['cheese']

    @classmethod
    @abc.abstractmethod
    def get_ingredients(cls):
         """Returns the default ingredient list."""
         return cls.default_ingredients

class DietPizza(BasePizza):
    def get_ingredients(self):
        return [Egg()] + super(DietPizza, self).get_ingredients()

Listing 7-12: Using an implementation in an abstract method

In this example, every Pizza you make that inherits from BasePizza has to override the get_ingredients() method, but every Pizza also has access to the base class’s default mechanism for getting the ingredients list. This mechanism is especially useful when providing an interface to implement while also providing base code that might be useful to all inheriting classes.

The Truth About super

Python has always allowed developers to use both single and multiple inheritances to extend their classes, but even today, many developers do not seem to understand how these mechanisms, and the super() method that is associated with them, work. To fully understand your code, you need to understand the trade-offs.

Multiple inheritances are used in many places, particularly in code involving a mixin pattern. A mixin is a class that inherits from two or more other classes, combining their features.

As you should know by now, classes are objects in Python. The construct used to create a class is a special statement that you should be well familiar with: class classname(expression of inheritance).

The code in parentheses is a Python expression that returns the list of class objects to be used as the class’s parents. Ordinarily, you would specify them directly, but you could also write something like this to specify the list of parent objects:

>>> def parent():
...     return object
...
>>> class A(parent()):
...     pass
...
>>> A.mro()
[<class '__main__.A'>, <type 'object'>]

This code works as expected: we declare class A with object as its parent class. The class method mro() returns the method resolution order used to resolve attributes—it defines how the next method to call is found via the tree of inheritance between classes. The current MRO system was first implemented in Python 2.3, and its internal workings are described in the Python 2.3 release notes. It defines how the system browses the tree of inheritance between classes to find the method to call.

We already saw that the canonical way to call a method in a parent class is to use the super() function, but what you probably don’t know is that super() is actually a constructor and you instantiate a super object each time you call it. It takes either one or two arguments: the first argument is a class, and the second, optional argument is either a subclass or an instance of the first argument.

The object returned by the constructor functions as a proxy for the parent classes of the first argument. It has its own __getattribute__ method that iterates over the classes in the MRO list and returns the first matching attribute it finds. The __getattribute__ method is called when an attribute of the super() object is retrieved, as shown in Listing 7-13.

>>> class A(object):
...     bar = 42
...     def foo(self):
...             pass
...
>>> class B(object):
...     bar = 0
...
>>> class C(A, B):
...     xyz = 'abc'
...
>>> C.mro()
[<class '__main__.C'>, <class '__main__.A'>, <class '__main__.B'>, <type 'object'>]
>>> super(C, C()).bar
42
>>> super(C, C()).foo
<bound method C.foo of <__main__.C object at 0x7f0299255a90>>
>>> super(B).__self__
>>> super(B, B()).__self__
<__main__.B object at 0x1096717f0>

Listing 7-13: The super() function is a constructor that instantiates a super object.

When requesting an attribute of the super object of an instance of C, the __getattribute__ method of the super() object walks through the MRO list and returns the attribute from the first class it finds that has the super attribute.

In Listing 7-13, we called super() with two arguments, meaning we used a bound super object. If we call super() with only one argument, it returns an unbound super object instead:

>>> super(C)
<super: <class 'C'>, NULL>

Since no instance has been provided as the second argument, the super object cannot be bound to any instance. Therefore, you cannot use this unbound object to access class attributes. If you try, you’ll get the following errors:

>>> super(C).foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'super' object has no attribute 'foo'
>>> super(C).bar
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'super' object has no attribute 'bar'
>>> super(C).xyz
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'super' object has no attribute 'xyz'

At first glance, it might seem like this unbound kind of super object is useless, but actually the way the super class implements the descriptor protocol __get__ makes unbound super objects useful as class attributes:

>>> class D(C):
...     sup = super(C)
...
>>> D().sup
<super: <class 'C'>, <D object>>
>>> D().sup.foo
<bound method D.foo of <__main__.D object at 0x7f0299255bd0>>
>>> D().sup.bar
42

The unbound super object’s __get__ method is called using the instance super(C).__get__(D()) and the attribute name 'foo' as arguments, allowing it to find and resolve foo.

There are plenty of situations in which using super() can be tricky, such as when handling different method signatures along the inheritance chain. Unfortunately, there’s no silver bullet for all occasions. The best precaution is to use tricks such as having all your methods accept their arguments using *args, **kwargs.

Since Python 3, super() has picked up a bit of magic: it can now be called from within a method without any arguments. When no arguments are passed to super(), it automatically searches the stack frame for arguments:

class B(A):
      def foo(self):
          super().foo()

The standard way of accessing parent attributes in subclasses is super(), and you should always use it. It allows cooperative calls of parent methods without any surprises, such as parent methods not being called or being called twice when multiple inheritances are used.

Summary

Equipped with what you learned in this chapter, you should be unbeatable on everything that concerns methods definition in Python. Decorators are essential when it comes to code factorization, and proper use of the built-in decorators provided by Python can vastly improve the neatness of your Python code. Abstract classes are especially useful when providing an API to other developers and services.

Class inheritance is not often fully understood, and having an overview of the internal machinery of the language is a good way to fully apprehend how this works. There should be no secrets left on this topic for you now!