Profiling involves generating detailed statistics to show how often and for how long various routines of a program execute. This helps you isolate the offending parts of a program, and those parts probably need redesigning.

Python provides a built-in module named cProfile, which enables the generation of detailed statistics pertaining to a program. The module gives details such as the total program running time, the time taken to run each function, and the number of times each function is called. These statistics make it easy to determine the parts of the code that need optimization.

In particular, cProfile provides the following data for a function or script:

• ncalls: This indicates the number of times a function is called 
• tottime: This indicates the time spent on a function, which excludes the time spent on calling other functions
•  percall: This is tottime divided by ncalls
• cumtime: This indicates the time spent on a function, including calls to other functions
• percall: This is cumtime divided by tottime 

To profile a function named spam(), use the following code:

import cProfile
cProfile.run('spam()','spam.profile')

You can then view the results of profiling by using another module called pstats, as follows:

import pstats
stats = pstats.Stats('spam.profile')
stats.strip_dirs().sort_stats('time').print_stats()

More importantly, you can profile an entire script. Let's assume that you want to profile a script named myscript.py. You can simply navigate to the directory of the script using a command-line tool and then type and run the following command:

python -m cProfile myscript.py

Here's a partial output from running the preceding command on the 8.08_vornoi_diagram code from Chapter 8, Fun With Canvas:

57607465 function calls (57607420 primitive calls) in 110.040 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
 1 50.100 50.100 95.452 95.452 8.09_vornoi_diagram.py:15(create_voronoi_diagram)
 1 0.000 0.000 110.040 110.040 8.09_vornoi_diagram.py:5(<module>)
 400125 2.423 0.000 14.616 0.000 __init__.py:2313(_create)
 400125 0.661 0.000 15.277 0.000 __init__.py:2342(create_rectangle)
 400128 1.849 0.000 2.743 0.000 __init__.py:95(_cnfmerge)
 625 0.001 0.000 0.003 0.000 random.py:170(randrange)
 625 0.002 0.000 0.002 0.000 random.py:220(_randbelow)
 50400000 30.072 0.000 30.072 0.000 {built-in method math.hypot}
 1 14.202 14.202 14.358 14.358 {method 'mainloop' of '_tkinter.tkapp' objects}

I specifically chose to profile this program because it takes a long time to execute. In this case, it took ~110 seconds to run and most of the time was spent running the create_vornoi_diagram function (~95 seconds). So now this function is a perfect candidate for optimization.

In addition to the cProfile module, there are other modules, such as PyCallGraph and objgraph, and they provide visual graphs for the profile data.