Considering the sources of errors

You might be able to divine the potential sources of error in your application by reading tea leaves, but that's hardly an efficient way to do things. Errors actually fall into well-defined categories that help you predict (to some degree) when and where they’ll occur. By thinking about these categories as you work through your application, you’re far more likely to discover potential errors' sources before they occur and cause potential damage. The two principal categories are

• Errors that occur at a specific time
• Errors that are of a specific type

The following sections discuss these two categories in greater detail. The overall concept is that you need to think about error classifications in order to start finding and fixing potential errors in your application before they become a problem.

Considering version differences

Python is one of the few languages around today that has active support for two major language versions. Even though Python 2.x support will officially end in 2020 (see https://pythonclock.org/ for details), you can bet that many developers will continue to use it until they're certain that the libraries they need come in a fully compatible Python 3.x form. However, the problem isn’t just with libraries but also with processes, documentation, existing code, and all sorts of other things that could affect someone who is using functional programming techniques in Python.

Although the Python community has worked hard to make the transition easier, you can see significant functional programming differences by reviewing the Python 2.x material at https://docs.python.org/2/howto/functional.html and comparing it to the Python 3.x material at https://docs.python.org/3/howto/functional.html. The transition will introduce bugs into your applications, some of them quite hard to find and others that the compiler will let you know about. Articles, such as the one at http://sebastianraschka.com/Articles/2014_python_2_3_key_diff.html can help you locate and potentially fix these issues. (Note especially the integer division differences stated by the article because they really can throw your functional code off in a manner that is particularly hard to find.)

Dealing with late binding closures

You need to realize that Python is late binding, which means that Python looks up the values of variables when it calls an inner function that is part of a loop only when the loop is completed. Consequently, rather than use individual values within a loop, what you see is the final value. For a demonstration of this issue, consider the following code:

def create_values(numValues):
return [lambda x : i * x for i in range(numValues)]

for indValue in create_values(5):
print(indValue(2))

This code creates the specified number of functions, one for each value in range(numValues), which is create_values(5) (five) in this case. The idea is to create an output of five values using a particular multiplier (which is indValue(2) in this case). You might assume that the first function call will be 0 (the value of i) * 2 (the value of x supplied as an input). However, the first function is never called while i is equal to 0. In fact, it gets called the first time only when its value is 4 — at the end of the loop. As a result, the output you see when you call this function is a series of 8s. To fix this code, you need to use the following create_values() code instead:

def create_values(numValues):
return [lambda x, i=i : i * x for i in
range(numValues)]

This version of the code uses a trick to force the value of i to reflect the actual value produced by each of the values output by range(numValues). Instead of being part of the inner function, i is now provided as an input. You call the function in the same manner as before, but now the output is correct. Oddly enough, this particular problem isn't specific to lambda expressions; it can happen in any Python code. However, developers see it more often in this situation because the tendency is to use a lambda expression in this case.

You can find another example of this late-binding closure issue in the posting at https://bugs.python.org/issue27738 (with another fix like the one shown in this section). The discussion at https://stackoverflow.com/questions/1107210/python-lambda-problems provides another solution to this problem using functools.partial(). The point is that you must remember that Python is late binding.

Using a variable

In some situations, you can't use a lambda expression inline. Fortunately, Python will generally find these errors and tell you about them, as in the following code:

garbled = "IXX aXXmX sXeXcXrXeXt mXXeXsXsXaXXXXXXgXeX!XX"
print filter(lambda x: x != "X", garbled)

Obviously, this example is incredible simple, and you likely wouldn’t use it in the real world. However, it shows that you can't use the lambda inline in this case; you must first assign it to a variable and then loop through the values. The following code shows the correct alternative code:

garbled = "IXX aXXmX sXeXcXrXeXt mXXeXsXsXaXXXXXXgXeX!XX"
ungarble = filter(lambda x: x != "X", garbled)
for x in ungarble:
print(x, end='')

Working with third-party libraries

Your Python functional programming experience will include third-party libraries that may not always benefit from the functional programming approach. Before you assume that a particular approach will work, you should review potential sources of error online. For example, the following message thread discusses potential problems with using lambda expressions to perform an aggregation with Pandas: https://github.com/pandas-dev/pandas/issues/7186. In many cases, the community of developers will have alternatives for you to try, as happened in this case.

Understanding the built-in exceptions

Python comes with a host of built-in exceptions — far more than you might think possible. You can see a list of these exceptions at https://docs.python.org/3.6/library/exceptions.html. The documentation breaks the exception list down into categories. Here is a brief overview of the Python exception categories that you work with regularly:

• Base classes: The base classes provide the essential building blocks (such as the Exception exception) for other exceptions. However, you might actually see some of these exceptions, such as the ArithmeticError exception, when working with an application.
• Concrete exceptions: Applications can experience hard errors — errors that are hard to overcome because no good way to handle them exists or they signal an event that the application must handle. For example, when a system runs out of memory, Python generates a MemoryError exception. Recovering from this error is hard because it releasing memory from other uses isn't always possible. When the user presses an interrupt key (such as Ctrl+C or Delete), Python generates a KeyboardInterrupt exception. The application must handle this exception before proceeding with any other tasks.
• OS exceptions: The operating system can generate errors that Python then passes along to your application. For example, if your application tries to open a file that doesn't exist, the operating system generates a FileNotFoundError exception.
• Warnings: Python tries to warn you about unexpected events or actions that could result in errors later. For example, if you try to inappropriately use a resource, such as an icon, Python generates a ResourceWarning exception. You want to remember that this particular category is a warning and not an actual error: Ignoring it can cause you woe later, but you can ignore it.

Obtaining a list of exception arguments

The list of arguments supplied with exceptions varies by exception and by what the sender provides. You can't always easily figure out what you can hope to obtain in the way of additional information. One way to handle the problem is to simply print everything by using code like this:

import sys
try:
File = open('myfile.txt')
except IOError as e:
for Arg in e.args:
print(Arg)
else:
print("File opened as expected.")
File.close();

The args property always contains a list of the exception arguments in string format. You can use a simple for loop to print each of the arguments. The only problem with this approach is that you're missing the argument names, so you know the output information (which is obvious in this case), but you don’t know what to call it.

A more complex method of dealing with the issue is to print both the names and the contents of the arguments. The following code displays both the names and the values of each of the arguments:

import sys
try:
File = open('myfile.txt')
except IOError as e:
for Entry in dir(e):
if (not Entry.startswith("_")):
try:
print(Entry, " = ", e.__getattribute__(Entry))
except AttributeError:
print("Attribute ", Entry, " not accessible.")
else:
print("File opened as expected.")
File.close();

In this case, you begin by getting a listing of the attributes associated with the error argument object using the dir() function. The output of the dir() function is a list of strings containing the names of the attributes that you can print. Only those arguments that don't start with an underscore (_) contain useful information about the exception. However, even some of those entries are inaccessible, so you must encase the output code in a second try…except block.

The attribute name is easy because it’s contained in Entry. To obtain the value associated with that attribute, you must use the __getattribute() function and supply the name of the attribute you want. When you run this code, you see both the name and the value of each of the attributes supplied with a particular error argument object. In this case, the actual output is as follows:

args = (2, 'No such file or directory')
Attribute characters_written not accessible.
errno = 2
filename = myfile.txt
filename2 = None
strerror = No such file or directory
winerror = None
with_traceback = <built-in method with_traceback of
FileNotFoundError object at 0x0000000003416DC8>

Considering functional style exception handling

The previous sections of this chapter have discussed using exceptions, but as presented in previous chapters, Haskell actually discourages the use of exceptions, partly because they're indicative of state, and many functional programming aficionados discourage this use as well. The fact that Haskell does present exceptions as needed is proof that they’re not absolutely forbidden, which is a good thing considering that in some situations, you really do need to use exceptions when working with Python.

However, when working in a functional programming environment with Python, you have some alternatives to using exceptions that are more in line with the functional programming paradigm. For example, instead of raising an exception as the result of certain events, you could always use a base value, as discussed at https://softwareengineering.stackexchange.com/questions/334769/functional-style-exception-handling.

Haskell also offers some specialized numeric handling that you might also want to incorporate as part of using Python. For example, as shown in Chapter 16, the Fractional type allows statements such as 5 / 0 in Haskell. The same statement produces an error in Python. Fortunately, you have access to the fractions package in Python, as described at https://docs.python.org/3/library/fractions.html.

Although the fractions package addresses some issues and you get a full fractional type, that package doesn't address the 5 / 0 problem; you still get a ZeroDivisionError exception. To avoid this final issue, you can use specialized techniques such as those found in the message thread at https://stackoverflow.com/questions/27317517/make-division-by-zero-equal-to-zero. The point is that you have ways around exceptions in some cases if you want to use a more functional style of reporting. If you really want some of the advantages of using Haskell in your Python application, the hyphen module at https://github.com/tbarnetlamb/hyphen makes it possible.