6
Functions
In our lust for measurement, we frequently measure that which we can rather than that which we wish to measure…and forget that there is a difference.
George Udny Yule
In This Chapter
The last and perhaps most powerful compound statement that we discuss is the function. Functions give you a way to name a code block wrapped as an object. That code can then be invoked by use of that name, allowing the same code to be called multiple times and in multiple places.
Defining Functions
A function definition defines a function object, which wraps the executable block. The definition does not run the code block but just defines the function. The definition describes how the function can be called, what it is named, what parameters can be passed to it, and what will be executed when it is invoked. The building blocks of a function are the controlling statement, an optional docstring, the controlled code block, and a return statement.
Control Statement
The first line of a function definition is the control statement, which takes the following form:
def <Function Name> (<Parameters>):
The def keyword indicates a function definition, <Function Name> is where the name that will be used to call the function is defined, and <Parameters> is where any arguments that can be passed to the function are defined. For example, the following function is defined with the name do_nothing and a single parameter named not_used:
def do_nothing(not_used): pass
The code block in this case consists of a single pass statement, which does nothing. The Python style guide, PEP8, has conventions for naming functions (see https://www.python.org/dev/peps/pep-0008/#function-and-variable-names).
Docstrings
The next part of a function definition is the documentation string, or docstring, which contains documentation for the function. It can be omitted, and the Python compiler will not object. However, it is highly recommended to supply a docstring for all but the most obvious methods. The docstring communicates your intentions in writing a function, what the function does, and how it should be called. PEP8 provides guidance regarding the content of docstrings (see https://www.python.org/dev/peps/pep-0008/#documentation-strings). The docstring consists of a single-line string or a multiline string surrounded in three pairs of double quotes that immediately follows the control statement:
def do_nothing(not_used): """This function does nothing.""" pass
For a single-line docstring, the quotes are on the same line as the text. For a multiline docstring, the quotes are generally above and below the text, as in Listing 6.1.
Listing 6.1 Multiline Docstring
def do_nothing(not_used):
"""
This function does nothing.
This function uses a pass statement to
avoid doing anything.
Parameters:
not_used - a parameter of any type,
which is not used.
"""
pass
The first line of the docstring should be a statement summarizing what the function does. With a more detailed explanation, a blank line is left after the first statement. There are many different possible conventions for what is contained after the first line of a docstring, but generally you want to offer an explanation of what the function does, what parameters it takes, and what it is expected to return. The docstring is useful both for someone reading your code and for various utilities that read and display either the first line or the whole docstring. For example, if you call the help() function on the function do_nothing(), the docstring is displayed as shown in Listing 6.2.
Listing 6.2 Docstring from help
help(do_nothing)
Help on function do_nothing in module __main__:
do_nothing(not_used)
This function does nothing.
This function uses a pass statement to avoid doing anything.
Parameters:
not_used - a parameter of any type,
which is not used.
Parameters
Parameters allow you to pass values into a function, which can be used in the function’s code block. A parameter is like a variable given to a function when it is called, where the parameter can be different every time you call the function. A function does not have to accept any parameters. For a function that should not accept parameters, you leave the parentheses after the function name empty:
def no_params():
print("I don't listen to nobody")
When you call a function, you pass the values for the parameters within the parentheses following the function name. Parameter values can be set based on the position at which they are passed or based on keywords. Functions can be defined to require their parameters be passed in either or a combination of these ways. The values passed to a function are attached to variables with the names defined in the function definition. Listing 6.3 defines three parameters: first, second, and third. These variables are then available to the code block that follows, which prints out the values for each parameter.
Listing 6.3 Parameters by Position or Keyword
def does_order(first, second, third):
'''Prints parameters.'''
print(f'First: {first}')
print(f'Second: {second}')
print(f'Third: {third}')
does_order(1, 2, 3)
First: 1
Second: 2
Third: 3
does_order(first=1, second=2, third=3)
First: 1
Second: 2
Third: 3
does_order(1, third=3, second=2)
First: 1
Second: 2
Third: 3
Listing 6.3 defines the function does_order() and then calls it three times. The first time, it uses the position of the arguments, (1, 2, 3), to assign the variable values. It assigns the first value to the first parameter, first, the second value to the second parameter, second, and the third value to the third parameter, third.
The second time the listing calls the function does_order(), it uses keyword assignment, explicitly assigning the values using the parameter names, (first=1, second=2, third=3). In the third call, the first parameter is assigned by position, and the other two are assigned using keyword assignment. Notice that in all three cases, the parameters are assigned the same values.
Keyword assignments do not rely on the position of the keywords. For example, you can assign third=3 in the position before second=2 without issue. You cannot use a keyword assignment to the left of a positional assignment, however:
does_order(second=2, 1, 3)
File "<ipython-input-9-eed80203e699>", line 1
does_order(second=2, 1, 3)
^
SyntaxError: positional argument follows keyword argument
You can require that a parameter be called only using the keyword method by putting a * to its left in the function definition. All parameters to the right of the star can only be called using keywords. Listing 6.4 shows how to make the parameter third a required keyword parameter and then call it using the keyword syntax.
Listing 6.4 Parameters Requiring Keywords
def does_keyword(first, second, *, third):
'''Prints parameters.'''
print(f'First: {first}')
print(f'Second: {second}')
print(f'Third: {third}')
does_keyword(1, 2, third=3)
First: 1
Second: 2
Third: 3
If you try to call a required keyword parameter using positional syntax, you get an error:
does_keyword(1, 2, 3)
-------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-15-88b97f8a6c32> in <module>
----> 1 does_keyword(1, 2, 3)
TypeError: does_keyword() takes 2 positional arguments but 3 were given
You can make a parameter optional by assigning to it a default value in the function definition. This value will be used if no value is provided for the parameter during the function call. Listing 6.5 defines a function, does_defaults(), whose third parameter has the default value 3. The function is then called twice: first using positional assignment for all three parameters and then using the default value for the third.
Listing 6.5 Parameters with Defaults
def does_defaults(first, second, third=3):
'''Prints parameters.'''
print(f'First: {first}')
print(f'Second: {second}')
print(f'Third: {third}')
does_defaults(1, 2, 3)
First: 1
Second: 2
Third: 3
does_defaults(1, 2)
First: 1
Second: 2
Third: 3
Much as with the restriction regarding the order of keyword and position arguments during a function call, you cannot define a function with a default value parameter to the left of a non-default value parameter:
def does_defaults(first=1, second, third=3):
'''Prints parameters.'''
print(f'First: {first}')
print(f'Second: {second}')
print(f'Third: {third}')
File "<ipython-input-19-a015eaeb01be>", line 1
def does_defaults(first=1, second, third=3):
^
SyntaxError: non-default argument follows default argument
Default values are defined in the function definition, not in the function call. This means that if you use a mutable object, such as a list or dictionary, as a default value, it will be created once for the function. Every time you call that function using that default, the same list or dictionary object will be used. This can lead to subtle problems if it is not expected. Listing 6.6 defines a function with a list as the default argument. The code block appends 1 to the list. Notice that every time the function is called, the list retains the values from previous calls.
Listing 6.6 Mutable Defaults
def does_list_default(my_list=[]):
'''Uses list as default.'''
my_list.append(1)
print(my_list)
does_list_default()
[1]
does_list_default()
[1, 1]
does_list_default()
[1, 1, 1]
Generally, it’s a good practice to avoid using mutable objects as default parameters to avoid difficult-to-trace bugs and confusion. Listing 6.7 demonstrates a common pattern to handle default values for mutable parameter types. The default value in the function definition is set to None. The code block tests whether the parameter has an assigned value. If it does not, a new list is created and assigned to the variable. Because the list is created in the code block, a new list is created every time the function is called without a value supplied for the parameter.
Listing 6.7 Default Pattern in a Code Block
def does_list_param(my_list=None):
'''Assigns default in code to avoid confusion.'''
my_list = my_list or []
my_list.append(1)
print(my_list)
does_list_param()
[1]
does_list_param()
[1]
does_list_param()
[1]
As of Python 3.8, you can restrict parameters to positional assignment only. A parameter to the left of a forward slash (/) in a function definition is restricted to positional assignment. Listing 6.8 defines the function does_positional so that its first parameter, first, is positional only.
Listing 6.8 Positional-Only Parameters (Python 3.8 and Later)
def does_positional(first, /, second, third):
'''Demonstrates a positional parameter.'''
print(f'First: {first}')
print(f'Second: {second}')
print(f'Third: {third}')
does_positional(1, 2, 3)
First: 1
Second: 2
Third: 3
If you try to call does_positional by using keyword assignment for first, you get an error:
does_positional(first=1, second=2, third=3) ----------------------------------------------------------------------- TypeError Traceback (most recent call las t) <ipython-input-24-7b1f45f64358> in <module> ----> 1 does_positional(first=1, second=2, third=3) TypeError: does_positional() got some positional-only arguments passed as keyword arguments: 'first'
Listing 6.9 modifies does_positional to use positional-only and keyword-only parameters. The parameter first is positional only, the parameter second can be set using positional or keyword assignment, and the last, third, is keyword only.
Listing 6.9 Positional-Only and Keyword-Only Parameters
def does_positional(first, /, second, *, third):
'''Demonstrates a positional and keyword parameters.'''
print(f'First: {first}')
print(f'Second: {second}')
print(f'Third: {third}')
does_positional(1, 2, third=3)
First: 1
Second: 2
Third: 3
You can use wildcards in function definitions to accept an undefined number of positional or keyword arguments. This is often done when a function calls a function from an outside API. The function can pass the arguments through without requiring that all of the outside API’s parameters be defined.
To use a wildcard for positional parameters, you use the * character. Listing 6.10 demonstrates the definition of a function with the positional wildcard parameter *args. The code block receives any positional arguments given in a function call as items in a list named args. This function goes through the list and prints each item. The function is then called with the arguments 'Donkey', 3, and ['a'], each of which is accessed from the list and printed.
Listing 6.10 Positional Wildcard Parameters
def does_wildcard_positions(*args):
'''Demonstrates wildcard for positional parameters.'''
for item in args:
print(item)
does_wildcard_positions('Donkey', 3, ['a'])
Donkey
3
['a']
To define a function with keyword wildcard parameters, you define a parameter that starts with **. For example, Listing 6.11 defines the function does_wildcard_keywords with the parameter **kwargs. In the code block, the keyword parameters are available as keys and values in the dictionary kwargs.
Listing 6.11 Keyword Wildcard Parameters
def does_wildcard_keywords(**kwargs):
'''Demonstrates wildcard for keyword parameters.'''
for key, value in kwargs.items():
print(f'{key} : {value}')
does_wildcard_keywords(one=1, name='Martha')
one : 1
name : Martha
You can use both positional and keyword wildcard parameters in the same function: Just define the positional parameters first and the keyword parameters second. Listing 6.12 demonstrates a function using both positional and keyword parameters.
Listing 6.12 Positional and Keyword Wildcard Parameters
def does_wildcards(*args, **kwargs):
'''Demonstrates wildcard parameters.'''
print(f'Positional: {args}')
print(f'Keyword: {kwargs}')
does_wildcards(1, 2, a='a', b=3)
Positional: (1, 2)
Keyword: {'a': 'a', 'b': 3}
Return Statements
Return statements define what value a function evaluates to when called. A return statement consists of the keyword return followed by an expression. The expression can be a simple value, a more complicated calculation, or a call to another function. Listing 6.13 defines a function that takes a number as an argument and returns that number plus 1.
Listing 6.13 Return Value
def adds_one(some_number):
'''Demonstrates return statement.'''
return some_number + 1
adds_one(1)
2
Every Python function has a return value. If you do not define a return statement explicitly, the function returns the special value None:
def returns_none():
'''Demonstrates default return value.'''
pass
returns_none() == None
True
This example omits a return statement and then tests that the value returned is equal to None.
Scope in Functions
Scope refers to the availability of objects defined in code. A variable defined in the global scope is available throughout your code, whereas a variable defined in a local scope is available only in that scope. Listing 6.14 defines a variable outer and a variable inner. Both variables are available in the code block of the function shows_scope, where you print them both.
Listing 6.14 Local and Global Scope
outer = 'Global scope'
def shows_scope():
'''Demonstrates local variable.'''
inner = 'Local scope'
print(outer)
print(inner)
shows_scope()
Global scope
Local scope
The variable inner is local to the function, as it is defined in the function’s code block. If you try to call inner from outside the function, it is not defined:
print(inner) ------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-39-9504624e1153> in <module> ----> 1 print(inner) NameError: name 'inner' is not defined
Understanding scope is useful when you use decorators, as described in the next section.
Decorators
A decorator enables you to design functions that modify other functions. Decorators are commonly used to set up logging using a set convention or by third-party libraries. While you may not need to write your own decorators, it is useful to understand how they work. This section walks through the concepts involved.
In Python, everything is an object, including functions. This means you can point a variable to a function. Listing 6.15 defines the function add_one(n), which takes a number and adds 1 to it. Next, it creates the variable my_func, which has the function add_one() as its value.
Note
When you are not calling a function, you do not use parentheses in the variable assignment. By omitting the parentheses, you are referring to the function object and not to a return value. You can see this where Listing 6.15 prints my_func, which is indeed a function object. You can then call the function by adding the parentheses and argument to my_func, which returns the argument plus 1.
Listing 6.15 A Function as Variable Value
def add_one(n):
'''Adds one to a number.'''
return n + 1
my_func = add_one
print(my_func)
<function add_one at 0x1075953a0>
my_func(2)
3
Because functions are objects, you can use them with data structures such as dictionaries or lists. Listing 6.16 defines two functions and puts them in a list pointed to by the variable my_functions. It then iterates through the list, assigning each function to the variable my_func during its iteration and calling the function during the for loop’s code block.
Listing 6.16 Calling a List of Functions
def add_one(n):
'''Adds one to a number.'''
return n + 1
def add_two(n):
'''Adds two to a number.'''
return n + 2
my_functions = [add_one, add_two]
for my_func in my_functions:
print(my_func(1))
2
3
Python allows you to define a function as part of another function’s code block. A function defined in this way is called a nested function. Listing 6.17 defines the function nested() in the code block of the function called_nested(). This nested function is then used as a return value for the outer function.
Listing 6.17 Nested Functions
def call_nested():
'''Calls a nested function.'''
print('outer')
def nested():
'''Prints a message.'''
print('nested')
return nested
my_func = call_nested()
outer
my_func()
nested
You can also wrap one function with another, adding functionality before or after. Listing 6.18 wraps the function add_one(number) with the function wrapper(number). The wrapping function takes a parameter, number, which it then passes to the wrapped function. It also has statements before and after calling add_one(number). You can see the order of the print statements when you call wrapper(1) and see that it returns the expected values from add_one: 1 and 2.
Listing 6.18 Wrapping Functions
def add_one(number):
'''Adds to a number.'''
print('Adding 1')
return number + 1
def wrapper(number):
'''Wraps another function.'''
print('Before calling function')
retval = add_one(number)
print('After calling function')
return retval
wrapper(1)
Before calling function
Adding 1
After calling function
2
It is also possible to go a step further and use a function as a parameter. You can pass a function as a value to a function that has a nested function definition wrapping the function that was passed. For example, Listing 6.19 first defines the function add_one(number) as before. But now it defines the function wrapper(number) nested in the code block of a new function, do_wrapping(some_func). This new function takes a function as an argument and then uses that function in the definition of wrapper(number). It then returns the newly defined version of wrapper(number). By assigning this result to a variable and calling it, you can see the wrapped results.
Listing 6.19 Nested Wrapping Function
def add_one(number):
'''Adds to a number.'''
print('Adding 1')
return number + 1
def do_wrapping(some_func):
'''Returns a wrapped function.'''
print('wrapping function')
def wrapper(number):
'''Wraps another function.'''
print('Before calling function')
retval = some_func(number)
print('After calling function')
return retval
return wrapper
my_func = do_wrapping(add_one)
wrapping function
my_func(1)
Before calling function
Adding 1
After calling function
2
You can use do_wrapping(some_func) to wrap any function that you like. For example, if you have the function add_two(number), you can pass it as an argument just as you did add_one(number):
my_func = do_wrapping(add_two) my_func(1) wrapping function Before calling function Adding 2 After calling function 3
Decorators provide syntax that can simplify this type of function wrapping. Instead of calling do_wrapping(some_func), assigning it to a variable, and then invoking the function from the variable, you can simply put @do_wrapping at the top of the function definition. Then the function add_one(number) can be called directly, and the wrapping happens behind the scenes.
You can see in Listing 6.20 that add_one(number) is wrapped in a similar fashion as in Listing 6.18, but with the simpler decorator syntax.
Listing 6.20 Decorator Syntax
def do_wrapping(some_func):
'''Returns a wrapped function.'''
print('wrapping function')
def wrapper(number):
'''Wraps another function.'''
print('Before calling function')
retval = some_func(number)
print('After calling function')
return retval
return wrapper
@do_wrapping
def add_one(number):
'''Adds to a number.'''
print('Adding 1')
return number + 1
wrapping function
add_one(1)
Before calling function
Adding 1
After calling function
2
Anonymous Functions
The vast majority of the time you define functions, you will want to use the syntax for named functions. This is what you have seen up to this point. There is an alternative, however: the unnamed, anonymous function. In Python, anonymous functions are known as lambda functions, and they have the following syntax:
lambda <Parameter>: <Statement>
where lambda is the keyword designating a lambda function, <Parameter> is an input parameter, and <Statement> is the statement to execute using the parameter. The result of <Statement> is the return value. This is how you define a lambda function that adds one to an input value:
lambda x: x +1
In general, your code will be easier to read, use, and debug if you avoid lambda functions, but one useful place for them is when a simple function is applied as an argument to another. Listing 6.21 defines the function apply_to_list(data, my_func), which takes a list and a function as arguments. When you call this function with the intention of adding 1 to each member of the list, the lambda function is an elegant solution.
Listing 6.21 Lambda Function
def apply_to_list(data, my_func):
'''Applies a function to items in a list.'''
for item in data:
print(f'{my_func(item)}')
apply_to_list([1, 2, 3], lambda x: x + 1)
2
3
4
Summary
Functions, which are important building blocks in constructing complex programs, are reusable named blocks of code. Functions are documented with docstrings. Functions can accept parameters in a number of ways. A function uses a return statement to pass a value at the end of its execution. Decorators are special functions that wrap other functions. Anonymous, or lambda, functions are unnamed.
Questions
For Questions 1–3, refer to Listing 6.22.
Listing 6.22 Functions for Questions 1–3
def add_prefix(word, prefix='before-'):
'''Prepend a word.'''
return f'{prefix}{word}'3
def return_one():
return 1
def wrapper():
print('a')
retval = return_one()
print('b')
print(retval)
1. What would be the output of the following call:
add_prefix('nighttime', 'after-')
2. What would be the output of the following call:
add_prefix('nighttime')
3. What would be the output of the following call:
add_prefix()
4. Which line should you put above a function definition to decorate it with the function standard_logging ?
*standard_logging **standard_logging @standard_logging [standard_logging]
5. What would be printed by the following call:
wrapper()