Hour 23. Fixing Problem Code
What You’ll Learn in This Hour:
How to use a traceback to locate errors
How to find errors with the pdb debugger
How to search the Internet for answers
What to do when you try a fix
How to find outside support
One of the most important skills you’ll cultivate as a developer is the ability to figure out what is wrong with your code. This is a skill you gain over time, but there are still a few tricks you can learn now that will make fixing bugs so much easier. In this hour, you will learn some basic tricks for troubleshooting your code as well as where to look for help.
When Your Code Has a Bug
Don’t panic. No matter how careful you are when writing code, bugs happen. You might have a typo. You may have something installed incorrectly. You may be missing a library. These things happen.
Also, there’s a misconception that the longer you’ve been coding, the less likely you are to get a bug. This couldn’t be further from the truth. The longer you’ve been coding, the more interesting your bugs get. They never go away.
So, relax. Bugs happen to everyone, and sometimes you even come away from a bug with new knowledge. For example, you may learn that two libraries don’t work well together, or that a certain data type has some unexpected behavior. Or, maybe you finally learn how to spell “achievement” without a spell checker.
After you’ve remembered not to panic, you’ll want to gather some information. You’ll need this not only for yourself, but for anyone else who wants to help you:
Check to see if you got an error, or if something didn’t act like expected.
If you got an error, save the text.
If you didn’t get an error, take a moment to describe what you thought should happen as well as what did happen.
Check your imports.
Check your last commit.
Remove .pyc files.
Locating Errors with a Traceback
A traceback is the output from certain kinds of errors that shows not only what went wrong, but where it went wrong and how deep the error was within your program. It’s also called a stack trace. Think of your program like a stack of classes and functions. You start in one function. That function calls another function, adding it to the stack. That function calls another function, so that one ends up on the stack. When an error occurs, Python knows about every class and function that happens to be on the stack, and it prints those into a traceback.
Let’s create a traceback so we can dissect it. First, see if you can pick out the error in our file:
class Alpha(object):
def __init__(self):
self.one = "One"
self.two = "Two"
def __str__(self):
return self.one, self.two
def print_object(obj):
print obj
def main():
object_a = Alpha()
print_object(object_a)
if __name__ == '__main__':
main()
Now, let’s run it and see what the traceback says:
$ python stack_error.py
Traceback (most recent call last):
File "stack_error.py", line 17, in <module>
main()
File "stack_error.py", line 15, in main
print_object(object_a)
File "stack_error.py", line 11, in print_object
print obj
TypeError: __str__ returned non-string (type tuple)
Let’s take this stack line by line. Although you can read tracebacks from beginning to end, it’s usually more useful to read them from end to beginning. Let’s start with the actual error we got:
TypeError: __str__ returned non-string (type tuple)
Okay, we have a TypeError, which means we have a data type mismatch. We were expecting a string, and got a tuple. The following will tell us where we called this function:
File "stack_error.py", line 11, in print_object
print obj
We apparently called this function when we tried to print obj. This call is done on line 11. What is __str__() returning in the traceback? Let’s look at our code:
def __str__(self):
return self.one, self.two
It is returning a tuple, rather than a string. We can fix that by making sure that __str__() returns the two values as a string. Here’s our fix:
def __str__(self):
return self.one + " " + self.two
Now, when we run the script, we don’t get a traceback.
Finding Errors with the pdb Debugger
Things get interesting when there’s no traceback or error message. First, make sure you have a clear idea of what you were expecting and what you got. It can help to write this down. Sometimes the very act of writing down what you see happening inspires a solution to your problem. I can’t count the number of times when the solution popped into my head in the middle of describing the problem to someone else.
Once you know what you expected, it’s time to break out a Python module called pdb, which is an interactive debugger that comes with Python. It runs your code, line by line, allowing you to see the path the logic takes through your code, as well as any variables that have been created. You can use it in three ways: from the command line, in the shell, or within your program.
If you don’t know quite where the error is, the easiest way to use pdb is on the command line. Let’s say we have this script:
from mylib import get_list
def main():
mylist = get_list()
for item in mylist:
print item
if __name__ == '__main__':
main()
We import a function from another library we wrote a while back. It’s supposed to give us a list of items. We then print those items, and this is what happens when we run it:
$ python usingpdb.py
t
h
r
e
e
That wasn’t what we expected. Why is it printing out one letter on each line? What’s going on here? Let’s run the script again, but this time using pdb:
$ python -m pdb usingpdb.py
> /Users/kcunningham/scripts/troubleshoot/usingpdb.py(1)<module>()
-> from mylib import get_list
(Pdb)
The (Pdb) line is a prompt where we can do a few things: We can step through a program, look at what variables have been declared, change variables, and run the program for a certain amount of time. With each line, the line about to be executed is shown, and you can get information about the program, what has been stored, change some variables, or execute the line (or many lines).
Table 23.1 covers some of the actions you can enter.
What we should be most interested in is what’s being printed out, so let’s keep moving through the code, line by line, using step. We finally hit the line where we started the loop, so let’s check what is stored in mylist:
-> for item in mylist:
(Pdb) mylist
'three'
It’s a string. The value returned to mylist was supposed to be a list. What happens if we change mylist to an actual list, as follows:
(Pdb) mylist = ['one', 'two', 'three']
(Pdb) step
>
/Users/kcunningham/Dropbox/Pearson/TYPython/scripts/troubleshoot/usingpdb.py(6)
main()
-> print item
(Pdb) step
one
>
/Users/kcunningham/Dropbox/Pearson/TYPython/scripts/troubleshoot/usingpdb.py(5)
main()
-> for item in mylist:
(Pdb) step
>
/Users/kcunningham/Dropbox/Pearson/TYPython/scripts/troubleshoot/usingpdb.py(6)
main()
-> print item
(Pdb) step
two
>
/Users/kcunningham/Dropbox/Pearson/TYPython/scripts/troubleshoot/usingpdb.py(5)
main()
-> for item in mylist:
(Pdb) c
three
The program finished and will be restarted
This time, three items were printed out. We know the issue is in the function where a list is supposed to be created.
Searching the Internet for Solutions
If you can’t figure out your problem by stepping through your program, it’s probably time for a quick search on the Internet. This seems like it would be intuitive, but there are a few tricks to it.
If you got an error, that error message is likely going to be your key to finding a solution. However, you can’t just cut and paste your error message into a search engine and expect an answer. It’s too unique. What you need to do is look for the parts of the error message that are more generic, that other users may have encountered.
Let’s say we get this traceback:
Traceback (most recent call last):
File "conc_error.py", line 13, in <module>
main()
File "conc_error.py", line 10, in main
print one + two
TypeError: cannot concatenate 'str' and 'int' objects
The information in the traceback is too specific. No one else is going to have this same filename, nor is a line like print one + two likely to involve the same error as we’re getting. The last line is helpful, though. What does a search on that line return? The first page of results links to ten articles about just this error (see Figure 23.1).
If you get results from another language, try adding “Python” to your search. This should exclude many of the results that won’t help you.
Here are some items you want to exclude from your search:
Filenames
Paths
Variable names
You’ll want to include the following items, which may not be in the error line:
Libraries
Version of Python
Operating system
Trying a Fix
Sometimes, a fix is obvious. Sometimes, you want to try something new, such as organizing your code differently or using a different library. Diving straight in can cause issues later, if you decide that the fix wasn’t really working for you.
The first step when trying a fix is to make a new branch so that you can keep your changes separate from your main code. If you decide that your idea isn’t going to work out, all you have to do is delete the branch. If your new code fixes the problem, you can merge down to master.
Now that you have a new branch, you should be methodical about what you change. Don’t just dive in and change random items. At best, you’ll waste time; at worst, you’ll introduce new bugs.
If you have more than one idea about what is wrong with your code, or you know you’ll have to change several items, change one thing at a time, and make sure the code is working as you expect. Sure, the code may not be fixed yet, but you should be able to predict what one section of code does.
If your code is complex, try breaking it out a bit more. Do you have a ton of functionality pushed onto one line? Break it out onto several lines, so you can see what each component is doing.
Also, break out just the section of code that is giving you trouble. Does a certain module seem to be having issues? Write up a quick script that uses just that module rather than trying to debug in the main program, where ten modules are being used.
As you verify that modules are working as they should, you can start recombining modules to see if the issue lies in two not working well together. Perhaps they have functions that override each other, or maybe they’re just too memory intensive to use together on the system you have.
Finding Outside Support
There comes a point in every developer’s life where she can’t solve a problem with her code on her own, and it appears that no one on the Internet has ever had a problem like it. Now is the time to get some outside support.
One of the most valuable things you have as a programmer is other programmers. Having other people you can talk to and show your code to can help fix a problem in a matter of minutes that would have taken you hours, or even days, to solve. Some people like to paint programming as a loner’s task, but in reality, programmers tend to thrive most when they work together.
So, where do you find these people? Chat protocols and user groups are a good place to start.
Internet Relay Chat
Internet Relay Chat (IRC) is a chat protocol. Users can create and join rooms and talk (via text) to a large number of people. They can also message each other privately. Remember, though, that IRC isn’t a program in and of itself. Others take that protocol and write programs that use it. There are programs hosted on servers that take care of creating rooms and dealing with people logging in, and there are programs that deal just with the individual user interfaces.
You’ll need a client program to use IRC. You can download and install one on your computer, or you can use a web-based client. Table 23.2 shows some popular options.
There are many IRC networks, but the one most programmers flock to is Freenode. Freenode (http://freenode.net/) is global and free to use, and it hosts not only channels about Python, but ones about many other languages and topics.
In order to use Freenode’s IRC network, you’ll need to register an account. Though you can technically use Freenode without registering, many channels only permit registered users to join.
To register a nick (the IRC term for username), go to https://webchat.freenode.net/. Enter the nickname you want, fill out the CAPTCHA, and click Connect. If your nickname is available, text will scroll across the screen.
In the text box at the bottom of the screen, enter the following text (with your details swapped in):
/msg NickServ REGISTER password [email protected]
This will register your account with Freenode. Now, feel free to start joining channels! To join a channel, type /join #channel. Table 23.3 has some useful channels on the Freenode network.
You can follow some general guidelines that will help you be more successful in the wilds of IRC:
If you have a problem, state it. Don’t ask if you can ask a question.
Be understanding if no one can solve your problem. Sometimes, no one is on. Sometimes, everyone is busy. Try again in an hour.
Never, ever, ever post code into the channel. Use dpaste.com.
In general, the Python community is filled with great people. If someone seems abrupt, they might just be fitting your problem into an already busy schedule. Don’t be offended! Many of us are in IRC so we can help others.
If you have a problem with someone in IRC, please talk to one of the mods. Mods have an @ symbol by their name. Mods can’t fix what they don’t know about.
Respect the rules of the channel. Each channel is different, so check the topic before you dive in. To see what the current topic is, type /topic.
Finally, the Freenode IRC channels abide by the PSF’s Code of Conduct. In short, be open to suggestions, be considerate of others, and be respectful. You can read the full Code of Conduct here: http://www.python.org/psf/codeofconduct/.
Local User Groups
If you want more face-to-face interaction, a great way to meet people is to join a local user group. Most Python user groups are organized through Meetup (http://meetup.com). Though it costs money to start a group on Meetup, creating an account and joining groups is free.
To find groups in your area, simply search for Python. This will bring up groups that talk about Python in general, as well as frameworks that use Python or groups that include Python under a larger umbrella.
You can also find a listing of all official Python meet-ups at http://python.meetup.com/. Note that this listing may not include some groups in your area, so searching is still a good idea.
Mailing Lists
One last place to look for help is mailing lists. Nearly every project or library has a mailing list, and Python has two that are often used by people seeking help with general Python programming.
The Python Tutor mailing list is set up specifically for beginners. It is staffed by volunteers from the Python community, and encourages both public and private discussion. You can sign up for the mailing list here: http://www.python.org/mailman/listinfo/tutor.
If your question is more advanced, you may be directed to python-list. This mailing list is a more general list for the Python community, and includes both beginners and experts in the field. You can sign up for that mailing list here: http://mail.python.org/mailman/listinfo/python-list.
Summary
During this hour, you learned how to prepare yourself for debugging a program. You learned about tracebacks and pdb, and also learned how to meet other Python users.
Q&A
Q. Where can I learn more about IRC?
A. There’s so much more to IRC than what we covered in this hour. Freenode’s FAQ covers much of what you can do with IRC in depth, but for a faster overview, check out IRCHelp at http://www.irchelp.org/.
Q. There’s no user group in my area. Can I start one?
A. Certainly! The Python community has put together a guide for creating your own user group at http://wiki.python.org/moin/StartingYourUsersGroup.
Workshop
The Workshop contains quiz questions and exercises to help you solidify your understanding of the material covered. Try to answer all questions before looking at the answers that follow.
Quiz
1. What are the three ways you can use pdb?
2. Why should you create a new branch to try a fix?
3. What should you remove from an error’s text before searching for it?
Answers
1. pdb can be used on the command line, in the shell, or inserted into a program.
2. Creating a new branch allows you to see exactly what you changed, and it allows you to keep your fixes separate from your main code, just in case you decide that the fix will not work after all.
3. You should remove anything that is unique to your program, such as variable names, filenames, and lines of code.
Exercise
Using the number-guessing game you wrote in Hour 13, “Using Python’s Modules to Add Functionality” (or another script you have handy), take some time to get used to using pdb. Run your script with the pdb flag and then do the following:
Add a breakpoint.
Trigger the breakpoint.
Remove the breakpoint.
Add a temporary breakpoint.
Step through the program, line by line, stopping when you enter functions.
Run each line of the program, not stopping when you enter functions.
Change a variable so that the program runs differently.