8.6. Troubleshooting
Regardless of the specific debugging techniques you use (and there are about as many methods for debugging as there are programmers), there are a few general principles to keep in mind as you debug your scripts.
The first task in any debugging effort is to learn how to consistently reproduce the bug. If it takes more than a few steps to manually trigger the buggy behavior, consider writing a script to trigger it. You will be able to debug much more quickly this way.
As you are debugging, you will want to progressively narrow your scope. In many cases, this involves eliminating half the possibilities at each stage of troubleshooting. Analysis is the thoughtful consideration of a bug's likely point of origin, based on detailed knowledge of the code base. In practice, you will probably use a combination of analysis and sheer brute force. A preliminary analysis will isolate the area of your code that is most likely to contain a given bug and then reviewing all of the code within that area will often help to locate it precisely.
Use debuggers, but don't spend an extended period of time getting the debuggers to work. Often, you step through a piece of code, statement by statement, only to find that you accidentally fixed the problem. Stepping through the code is invaluable as the more times you go through it, the more streamlined and commented it tends to become. Becoming more in tune with your code in this way can help to make you a better programmer.
If you are attempting to write scripts just for simple admin purposes and don't wish to spend a lot of time debugging, use a search engine and see if the specific portion of your script has been written before. In the course of writing this book, we found many of our scripts in an almost identical state on the web. In some cases, there are a finite number of ways of writing a script and if someone else has found the way to get the script to work, then learn from their work and build on it.
When trying to isolate a bug, you often want to change only one thing at a time. Debugging is a process where you make changes to code and then test to see if you've fixed a bug. Then you make another change, test again, and so on until the bug is fixed. At each iteration, make sure to change only one thing so that when the bug is fixed, you will know exactly what caused it. If you change several things at once, you risk including unnecessary changes in your fix, which may in some cases cause bugs themselves.
A trace statement is a console or log message that is inserted into a piece of code suspected of containing a bug, then generally removed once the bug has been found. Trace statements not only trace the path of execution through code, but the changing state of program variables as execution progresses. Once you have found the bug, you may find it helpful to leave a few of the trace statements in the code, perhaps converting console messages into file-based logging messages to assist in future debugging.
If you're using a third party server, database, or script, check all of the components and you will often find a good amount of useful information about errors in the log files for each application or operating system. You may have to configure the component to log the sort of information you're interested in.
Sometimes, after you've been hunting a bug for long enough, you begin to despair of ever finding it. When this happens, it can be useful to start from scratch. Create a new script, and bring each function from your old script over one at a time, checking each portion thoroughly before integrating it into the new script. At times, it is also a good idea to break each portion of a script into a separate scripts of its own.
Research shows that bugs tend to cluster. When you encounter a new bug, think of the parts of the code where you have found bugs in the past, and whether they could be involved with the current bug. At times, this is just that the functions you are working with may not be your strongest, or that the code in general is just buggy, but experience tells that where there is one pesky bug, there are likely to be others.
One of the most obscure sources of bugs is from using incompatible versions of third-party libraries. It is also one of the last things to check when you've exhausted other debugging strategies. For example, if version 5.1 of some library has a dependency on version 1.4g of SSL or some other library but you install 1.4b instead, the results may be issues that are difficult or impossible to diagnose. Checking your documentation can help with this.
If all else fails, read the instructions. It's remarkable how often this simple step is . In their rush to start programming with some class library or utility, some developers will adopt a trial-and-error approach to using, for example, a new Perl mod. If there is little or no documentation, this may be appropriate. It's possible that your bug results from misuse of the mod and the underlying code is failing to check that you have obeyed all the necessary preconditions for its use.
When a bug suddenly appears in functionality that has been working for some time, you should immediately wonder what has recently changed in the scripts or software that calls the scripts that might have caused the bug. This is where a version control system can be helpful, providing you with the ability to look at the change history of your code, or re-creating successively older versions of the code base until you get one in which the bug disappears. CVS and Subversion are both great examples of version control systems.
What may be multi-causal problems are often troubleshot as a single-cause bug. When troubleshooting network issues and buggy scripts, it is often hardest to isolate issues that contain multiple errors. In fact, we often do not consider this until trying everything else. But they do happen and if nothing else explains an issue, look for multiple bugs.
Normally you scrutinize the error messages you get very carefully, hoping for a clue as to where to start your debugging efforts. But if you're not having any luck with that approach, remember that error messages can sometimes be misleading. Sometimes programmers don't put as much thought into the handling and reporting of error conditions as one would like, so it may be wise to avoid interpreting the error message too literally, and to consider possibilities other than the ones that are specifically identified.
When you're really stuck on a bug, it can be helpful to grab another programmer and explain the bug to them. Also tell them the efforts you've made so far to hunt down its source. They may offer some helpful advice, but this is not what the technique is really about. It sometimes happens that in the course of explaining the problem to another person, you realize something about the bug you didn't think of before.
Many have noted that solutions come much easier after a period of intense concentration on the problem, followed by a period of rest. Another way to get a fresh look at a piece of code you've been staring at for too long is to print it out and review it. We read faster on paper than on the screen, so this may be why it's slightly easier to spot an error in printed code than displayed code.
After a time you may notice that you are prone to writing particular kinds of bugs. If you can identify a consistent weakness like this, you can take preventative steps. If you have a code-review checklist, augment the checklist to include a check specifically for the type of bug you favor. Simply maintaining an awareness of your "favorite" defects can help reduce your tendency to inject them.