13. Python Runtime Services
This chapter describes modules that are related to the Python interpreter runtime. Topics include garbage collection, basic management of objects (copying, marshalling, and so on), weak references, and interpreter environment.
atexit
The atexit module is used to register functions to execute when the Python interpreter exits. A single function is provided:
register(func [,args [,kwargs]])
Adds function func to a list of functions that will execute when the interpreter exits. args is a tuple of arguments to pass to the function. kwargs is a dictionary of keyword arguments. The function is invoked as func(*args,**kwargs). Upon exit, functions are invoked in reverse order of registration (the most recently added exit function is invoked first). If an error occurs, an exception message will be printed to standard error but will otherwise be ignored.
copy
The copy module provides functions for making shallow and deep copies of compound objects, including lists, tuples, dictionaries, and instances of user-defined objects.
copy(x)
Makes a shallow copy of x by creating a new compound object and duplicating the members of x by reference. For built-in types, it is somewhat uncommon to use this function. Instead, you use calls such as list(x), dict(x), set(x), and so forth to create a shallow copy of x (it should be noted that using the type name directly like this is also significantly faster than using copy()).
deepcopy(x [, visit])
Makes a deep copy of x by creating a new compound object and recursively duplicating all the members of x. visit is an optional dictionary that’s used to keep track of visited objects in order to detect and avoid cycles in recursively defined data structures. This argument is typically only supplied if deepcopy() is being called recursively as described later in this chapter.
Although it is not usually necessary, a class can implement customized copy methods by implementing the methods _ _copy_ _(self) and _ _deepcopy_ _(self, visit), which implement the shallow and deep copy operations respectively. The _ _deepcopy_ _() method must accept a dictionary, visit, which is used to keep track of previously encountered objects during the copy process. It’s not necessary for _ _deepcopy_ _() to do anything with visit other than pass it to other deepcopy() operations carried out in the implementation (if any).
If a class implements the methods _ _getstate_ _() and _ _setstate_ _() that are used by the pickle module, they will be used by the copy module to create copies.
Notes
• This module can be used with simple types such as integers and strings, but there’s little need to do so.
• The copy functions don’t work with modules, class objects, functions, methods, tracebacks, stack frames, files, sockets, and other similar types. When an object can’t be copied, the copy.error exception is raised.
gc
The gc module provides an interface for controlling the garbage collector used to collect cycles in objects such as lists, tuples, dictionaries, and instances. As various types of container objects are created, they’re placed on a list that’s internal to the interpreter. Whenever container objects are deallocated, they’re removed from this list. If the number of allocations exceeds the number of deallocations by a user-definable threshold value, the garbage collector is invoked. The garbage collector works by scanning this list and identifying collections of objects that are no longer being used but haven’t been deallocated due to circular dependencies. In addition, the garbage collector uses a three-level generational scheme in which objects that survive the initial garbage-collection step are placed onto lists of objects that are checked less frequently. This provides better performance for programs that have a large number of long-lived objects.
collect([generation])
Runs a full garbage collection. This function checks all generations and returns the number of unreachable objects found. generation is an optional integer in the range 0 - 2 that specifies the generation to collect.
disable()
Disables garbage collection.
enable()
Enables garbage collection.
garbage
A variable containing a read-only list of user-defined instances that are no longer in use, but which cannot be garbage collected because they are involved in a reference cycle and they define a _ _del_ _() method. Such objects cannot be garbage-collected because in order to break the reference cycle, the interpreter must arbitrarily destroy one of the objects first. However, there is no way to know if the _ _del_ _() method of the remaining objects in the cycle needs to perform critical operations on the object that was just destroyed.
get_count()
Returns a tuple (count0, count1, count2) containing the number of objects currently in each generation.
get_debug()
Returns the debugging flags currently set.
get_objects()
Returns a list of all objects being tracked by the garbage collector. Does not include the returned list.
get_referrers(obj1, obj2, ...)
Returns a list of all objects that directly refer to the objects obj1, obj2, and so on. The returned list may include objects that have not yet been garbage-collected as well as partially constructed objects.
get_referents(obj1, obj2, ...)
Returns a list of objects that the objects obj1, obj2, and so on refer to. For example, if obj1 is a container, this would return a list of the objects in the container.
get_threshold()
Returns the current collection threshold as a tuple.
isenabled()
Returns True if garbage collection is enabled.
set_debug(flags)
Sets the garbage-collection debugging flags, which can be used to debug the behavior of the garbage collector. flags is the bitwise OR of the constants DEBUG_STATS, DEBUG_COLLECTABLE, DEBUG_UNCOLLECTABLE, DEBUG_INSTANCES, DEBUG_OBJECTS, DEBUG_SAVEALL, and DEBUG_LEAK. The DEBUG_LEAK flag is probably the most useful because it will have the collector print information useful for debugging programs with memory leaks.
set_threshold(threshold0 [, threshold1[, threshold2]])
Sets the collection frequency of garbage collection. Objects are classified into three generations, where generation 0 contains the youngest objects and generation 2 contains the oldest objects. Objects that survive a garbage-collection step are moved to the next-oldest generation. Once an object reaches generation 2, it stays in that generation. threshold0 is the difference between the number of allocations and deallocations that must be reached before garbage collection occurs in generation 0. threshold1 is the number of collections of generation 0 that must occur before generation 1 is scanned. threshold2 is the number of collections that must occur in generation 1 before generation 2 is collected. The default threshold is currently set to (700,10,10). Setting threshold0 to 0 disables garbage collection.
Notes
• Circular references involving objects with a _ _del_ _() method are not garbage-collected and are placed on the list gc.garbage (uncollectable objects). These objects are not collected due to difficulties related to object finalization.
• The functions get_referrers() and get_referents() only apply to objects that support garbage collection. In addition, these functions are only intended for debugging. They should not be used for other purposes.
inspect
The inspect module is used to gather information about live Python objects such as attributes, documentation strings, source code, stack frames, and so on.
cleandoc(doc)
Cleans up a documentation string doc by changing all tabs into whitespace and removing indentation that might have been inserted to make the docstring line up with other statements inside a function or method.
currentframe()
Returns the frame object corresponding to the caller’s stack frame.
formatargspec(args [, varags [, varkw [, defaults]]])
Produces a nicely formatted string representing the values returned by getargspec().
formatargvalues(args [, varargs [, varkw [, locals]]])
Produces a nicely formatted string representing the values returned by getargvalues().
getargspec(func)
Given a function, func, returns a named tuple ArgSpec(args, varargs, varkw, defaults). args is a list of argument names, and varargs is the name of the * argument (if any). varkw is the name of the ** argument (if any), and defaults is a tuple of default argument values or None if there are no default argument values. If there are default argument values, the defaults tuple represents the values of the last n arguments in args, where n is the len(defaults).
getargvalues(frame)
Returns the values of arguments supplied to a function with execution frame frame. Returns a tuple ArgInfo(args, varargs, varkw, locals). args is a list of argument names, varargs is the name of the * argument (if any), and varkw is the name of the ** argument (if any). locals is the local dictionary of the frame.
getclasstree(classes [, unique])
Given a list of related classes, classes, this function organizes the classes into a hierarchy based on inheritance. The hierarchy is represented as a collection of nested lists, where each entry in the list is a list of classes that inherit from the class that immediately precedes the list. Each entry in the list is a 2-tuple (cls, bases), where cls is the class object and bases is a tuple of base classes. If unique is True, each class only appears once in the returned list. Otherwise, a class may appear multiple times if multiple inheritance is being used.
getcomments(object)
Returns a string consisting of comments that immediately precede the definition of object in Python source code. If object is a module, comments defined at the top of the module are returned. Returns None if no comments are found.
getdoc(object)
Returns the documentation string for object. The documentation string is first processed using the cleandoc() function before being returned.
getfile(object)
Returns the name of the file in which object was defined. May return TypeError if this information is not applicable or available (for example, for built-in functions).
getframeinfo(frame [, context])
Returns a named tuple Traceback(filename, lineno, function, code_context, index) containing information about the frame object frame. filename and line specify a source code location. The context parameter specifies the number of lines of context from the source code to retrieve. The contextlist field in the returned tuple contains a list of source lines corresponding to this context. The index field is a numerical index within this list for the line corresponding to frame.
getinnerframes(traceback [, context])
Returns a list of frame records for the frame of a traceback and all inner frames. Each frame-record is a 6-tuple consisting of (frame, filename, line, funcname, contextlist, index). filename, line, context, contextlist, and index have the same meaning as with getframeinfo().
getmembers(object [, predicate])
Returns all of the members of object. Typically, the members are obtained by looking in the _ _dict_ _ attribute of an object, but this function may return attributes of object stored elsewhere (for example, docstrings in _ _doc_ _, objects’ names in _ _name_ _, and so on). The members are returned a list of (name, value) pairs. predicate is an optional function that accepts a member object as an argument and returns True or False. Only members for which predicate returns True are returned. Functions such as isfunction() and isclass() can be used as predicate functions.
getmodule(object)
Returns the module in which object was defined (if possible).
getmoduleinfo(path)
Returns information about how Python would interpret the file path. If path is not a Python module, None is returned. Otherwise, a named tuple ModuleInfo(name, suffix, mode, module_type) is returned where name is the name of the module, suffix is the filename suffix, mode is the file mode that would be used to open the module, and module_type is an integer code specifying the module type. Module type codes are defined in the imp module as follows:
getmodulename(path)
Returns the name of the module that would be used for the file path. If path does not look like a Python module, None is returned.
getmro(cls)
Returns a tuple of classes that represent the method-resolution ordering used to resolve methods in class cls. Refer to Chapter 7, “Classes and Object-Oriented Programming,” for further details.
getouterframes(frame [, context])
Returns a list of frame records for frame and all outer frames. This list represents the calling sequence where the first entry contains information for frame. Each frame record is a 6-tuple (frame, filename, line, funcname, contextlist, index) where the fields have the same meaning as for getinnerframes() The context argument has the same meaning as for getframeinfo().
getsourcefile(object)
Returns the name of the Python source file in which object was defined.
getsourcelines(object)
Returns a tuple (sourcelines, firstline) corresponding to the definition of object. sourcelines is a list of source code lines, and firstline is the line number of the first source code line. Raises IOError if source code can’t be found.
getsource(object)
Returns source code of object as a single string. Raises IOError if the source code can’t be found.
isabstract(object)
Returns True if object is an abstract base class.
isbuiltin(object)
Returns True if object is a built-in function.
isclass(object)
Returns True if object is a class.
Returns True if object is a code object.
isdatadescriptor(object)
Returns True if object is a data descriptor object. This is the case if object defines both a _ _get_ _() and _ _set_ _() method.
isframe(object)
Returns True if object is a frame object.
isfunction(object)
Returns True if object is a function object.
isgenerator(object)
Returns True if object is a generator object.
isgeneratorfunction(object)
Returns True if object is a generator function. This is different than isgenerator() in that it tests if object is a function that creates a generator when called. It is not used to check if object is an actively running generator.
ismethod(object)
Returns True if object is a method.
ismethoddescriptor(object)
Returns True if object is a method descriptor object. This is the case if object is not a method, class, or function and it defines a _ _get_ _() method but does not define _ _set_ _().
ismodule(object)
Returns True if object is a module object.
isroutine(object)
Returns True if object is a user-defined or built-in function or method.
istraceback(object)
Returns True if object is a traceback object.
stack([context])
Returns a list of frame records corresponding to the stack of the caller. Each frame record is a 6-tuple (frame, filename, line, funcname, contextlist, index), which contains the same information as returned by getinnerframes(). context specifies the number of lines of source context to return in each frame record.
trace([context])
Returns a list of frame records for the stack between the current frame and the frame in which the current exception was raised. The first frame record is the caller, and the last frame record is the frame where the exception occurred. context specifies the number of lines of source context to return in each frame record.
marshal
The marshal module is used to serialize Python objects in an “undocumented” Python-specific data format. marshal is similar to the pickle and shelve modules, but it is less powerful and intended for use only with simple objects. It shouldn’t be used to implement persistent objects in general (use pickle instead). However, for simple built-in types, the marshal module is a very fast approach for saving and loading data.
dump(value, file [, version])
Writes the object value to the open file object file. If value is an unsupported type, a ValueError exception is raised. version is an integer that specifies the data format to use. The default output format is found in marshal.version and is currently set to 2. Version 0 is an older format used by earlier versions of Python.
dumps(value [,version])
Returns the string written by the dump() function. If value is an unsupported type, a ValueError exception is raised. version is the same as described previously.
load(file)
Reads and returns the next value from the open file object file. If no valid value is read, an EOFError, ValueError, or TypeError exception will be raised. The format of the input data is automatically detected.
loads(string)
Reads and returns the next value from the string string.
Notes
• Data is stored in a binary architecture-independent format.
• Only None, integers, long integers, floats, complex numbers, strings, Unicode strings, tuples, lists, dictionaries, and code objects are supported. Lists, tuples, and dictionaries can only contain supported objects. Class instances and recursive references in lists, tuples, and dictionaries are not supported.
• Integers may be promoted to long integers if the built-in integer type doesn’t have enough precision—for example, if the marshalled data contains a 64-bit integer, but the data is being read on a 32-bit machine.
• marshal is not intended to be secure against erroneous or maliciously constructed data and should not be used to unmarshal data from untrusted sources.
• marshal is significantly faster than pickle, but it isn’t as flexible.
pickle
The pickle module is used to serialize Python objects into a stream of bytes suitable for storing in a file, transferring across a network, or placing in a database. This process is variously called pickling, serializing, marshalling, or flattening. The resulting byte stream can also be converted back into a series of Python objects using an unpickling process.
The following functions are used to turn an object into a byte-stream.
dump(object, file [, protocol ])
Dumps a pickled representation of object to the file object file. protocol specifies the output format of the data. Protocol 0 (the default) is a text-based format that is backwards-compatible with earlier versions of Python. Protocol 1 is a binary protocol that is also compatible with most earlier Python versions. Protocol 2 is a newer protocol that provides more efficient pickling of classes and instances. Protocol 3 is used by Python 3 and is not backwards-compatible. If protocol is negative, the most modern protocol will be selected. The variable pickle.HIGHEST_PROTOCOL contains the highest protocol available. If object doesn’t support pickling, a pickle.PicklingError exception is raised.
dumps(object [, protocol])
Same as dump(), but returns a string containing the pickled data.
The following example shows how you use these functions to save objects to a file:
The following functions are used to restore a pickled object.
load(file)
Loads and returns a pickled representation of an object from the file object file. It is not necessary to specify the input protocol as it is automatically detected. A pickle.UnpicklingError exception is raised if the file contains corrupted data that can’t be decoded. If an end-of-file is detected, an EOFError exception is raised.
loads(string)
Same as load(), but reads the pickled representation of an object from a string.
The following example shows how you use these functions to load data:
When loading, it is not necessary to specify the protocol or any information about the type of object being loaded. That information is saved as part of the pickle data format itself.
If you are pickling more than one Python object, you can simply make repeated calls to dump() and load() as shown in the previous examples. When making multiple calls, you simply have to make sure the sequence of load() calls matches the sequence of dump() calls that were used to write the file.
When working with complicated data structures involving cycles or shared references, using dump() and load() can be problematic because they don’t maintain any internal state about objects that have already been pickled or restored. This can result in output files that are excessively large and that don’t properly restore the relationship between objects when loaded. An alternative approach is to use Pickler and Unpickler objects.
Pickler(file [, protocol ])
Creates a pickling object that writes data to the file object file with the specified pickle protocol. An instance p of Pickler has a method p.dump(x) that dumps an object x to file. Once x has been dumped, its identity is remembered. If a subsequent p.dump() operation is used to write the same object, a reference to the previously dumped object is saved instead of writing a new copy. The method p.clear_memo() clears the internal dictionary used to track previously dumped objects. You would use this if you wanted to write a fresh copy of a previously dumped object (that is, if its value changed since the last dump() operation).
Unpickler(file)
Creates an unpickling object that reads data from the file object file. An instance u of Unpickler has a method u.load() that loads and returns a new object from file. An Unpickler keeps track of objects it has returned because the input source might contain an object reference created by the Pickler object. In this case, u.load() returns a reference to the previously loaded object.
The pickle module works with most kinds of normal Python objects. This includes:
• None
• Numbers and strings
• Tuples, lists, and dictionaries containing only pickleable objects
• Instances of user-defined classes defined at the top level of a module
When instances of a user-defined class are pickled, the instance data is the only part that gets pickled. The corresponding class definition is not saved—instead, the pickled data merely contains the name of the associated class and module. When instances are unpickled, the module in which the class is defined is automatically imported in order to access the class definition when re-creating instances. It should also be noted that when restoring an instance, the _ _init_ _() method of a class is not invoked. Instead, the instance is re-created through other means and the instance data restored.
One restriction on instances is that the corresponding class definition must appear at the top level of a module (that is, no nested classes). In addition, if the instance’s class definition was originally defined in _ _main_ _, that class definition must be manually reloaded prior to unpickling a saved object (because there’s no way for the interpreter to know how to automatically load the necessary class definitions back into _ _main_ _ when unpickling).
It is not normally necessary to do anything to make a user-defined class work with pickle. However, a class can define customized methods for saving and restoring its state by implementing the special methods _ _getstate_ _() and _ _setstate_ _(). The _ _getstate_ _() method must return a pickleable object (such as a string or tuple) representing the state of the object. The _ _setstate_ _() method accepts the pickled object and restores its state. If these methods are undefined, the default behavior is to pickle an instance’s underlying _ _dict_ _ attribute. It should be noted that if these methods are defined, they will also be used by the copy module to implement the shallow and deep copy operations.
Notes
• In Python 2, a module called cPickle contains a C implementation of functions in the pickle module. It is significantly faster than pickle, but is restricted in that it doesn’t allow subclassing of the Pickler and Unpickler objects. Python 3 has a support module that also contains C implementation, but it is used more transparently (pickle takes advantage of it automatically as appropriate).
• The data format used by pickle is Python-specific and shouldn’t be assumed to be compatible with any external standards such as XML.
• Whenever possible, the pickle module should be used instead of the marshal module because pickle is more flexible, the data encoding is documented, and additional error-checking is performed.
• Due to security concerns, programs should not unpickle data received from untrusted sources.
• Use of the pickle module with types defined in extension modules is much more involved than what is described here. Implementers of extension types should consult the online documentation for details concerning the low-level protocol required to make these objects work with pickle—in particular, details on how to implement the _ _reduce_ _() and _ _reduce_ex_ _() special methods that pickle uses to create the serialized byte sequences.
sys
The sys module contains variables and functions that pertain to the operation of the interpreter and its environment.
Variables
The following variables are defined.
api_version
An integer representing the C API version of the Python interpreter. Used when working with extension modules.
argv
List of command-line options passed to a program. argv[0] is the name of the program.
builtin_module_names
Tuple containing names of modules built into the Python executable.
byteorder
Native byte-ordering of the machine—'little' for little-endian or 'big' for big-endian.
String containing copyright message.
_ _displayhook_ _
Original value of the displayhook() function.
dont_write_bytecode
Boolean flag that determines whether or not Python writes bytecode (.pyc or .pyo files) when importing modules. The initial value is True unless the -B option to the interpreter is given. The setting can be changed as needed in your own program.
dllhandle
Integer handle for the Python DLL (Windows).
_ _excepthook_ _
Original value of the excepthook() function.
exec_prefix
Directory where platform-dependent Python files are installed.
executable
String containing the name of the interpreter executable.
flags
An object representing the settings of different command-line options supplied to the Python interpreter itself. The following table lists the attributes of flags along with the corresponding command-line option that turns the flag on. These attributes are read-only.
An object that holds information about internal representation of floating-point numbers. The values of these attributes are taken from the float.h C header file.
hexversion
Integer whose hexadecimal representation encodes the version information contained in sys.version_info. The value of this integer is always guaranteed to increase with newer versions of the interpreter.
last_type, last_value, last_traceback
These variables are set when an unhandled exception is encountered and the interpreter prints an error message. last_type is the last exception type, last_value is the last exception value, and last_traceback is a stack trace. Note that the use of these variables is not thread-safe. sys.exc_info() should be used instead.
maxint
Largest integer supported by the integer type (Python 2 only).
maxsize
Largest integer value supported by the C size_t datatype on the system. This value determines the largest possible length for strings, lists, dicts, and other built-in types.
maxunicode
Integer that indicates the largest Unicode code point that can be represented. The default value is 65535 for the 16-bit UCS-2 encoding. A larger value will be found if Python has been configured to use UCS-4.
modules
Dictionary that maps module names to module objects.
List of strings specifying the search path for modules. The first entry is always set to the directory in which the script used to start Python is located (if available). Refer to Chapter 8, “Iterators and Generators.”
platform
Platform identifier string, such as 'linux-i386'.
prefix
Directory where platform-independent Python files are installed.
ps1, ps2
Strings containing the text for the primary and secondary prompts of the interpreter. Initially, ps1 is set to '>>> ' and ps2 is set to '... '. The str() method of whatever object is assigned to these values is evaluated to generate the prompt text.
py3kwarning
Flag set to True in Python 2 when the interpreter is run with the -3 option.
stdin, stdout, stderr
File objects corresponding to standard input, standard output, and standard error. stdin is used for the raw_input() and input() functions. stdout is used for print and the prompts of raw_input() and input(). stderr is used for the interpreter’s prompts and error messages. These variables can be assigned to any object that supports a write() method operating on a single string argument.
_ _stdin_ _, _ _stdout_ _, _ _stderr_ _
File objects containing the values of stdin, stdout, and stderr at the start of the interpreter.
tracebacklimit
Maximum number of levels of traceback information printed when an unhandled exception occurs. The default value is 1000. A value of 0 suppresses all traceback information and causes only the exception type and value to be printed.
version
Version string.
version_info
Version information represented as a tuple (major, minor, micro, releaselevel, serial). All values are integers except releaselevel, which is the string 'alpha', 'beta', 'candidate', or 'final'.
warnoptions
List of warning options supplied to the interpreter with the –W command-line option.
winver
The version number used to form registry keys on Windows.
Functions
The following functions are available:
_clear_type_cache()
Clears the internal type cache. To optimize method lookups, a small 1024-entry cache of recently used methods is maintained inside the interpreter. This cache speeds up repeated method lookups—especially in code that has deep inheritance hierarchies. Normally, you don’t need to clear this cache, but you might do so if you are trying to track down a really tricky memory reference counting issue. For example, if a method in the cache was holding a reference to an object that you were expecting to be destroyed.
_current_frames()
Returns a dictionary mapping thread identifiers to the topmost stack frame of the executing thread at the time of call. This information can be useful in writing tools related to thread debugging (that is, tracking down deadlock). Keep in mind that the values returned by this function only represent a snapshot of the interpreter at the time of call. Threads may be executing elsewhere by the time you look at the returned data.
displayhook([value])
This function is called to print the result of an expression when the interpreter is running in interactive mode. By default, the value of repr(value) is printed to standard output and value is saved in the variable _ _builtin_ _._. displayhook can be redefined to provide different behavior if desired.
excepthook(type,value,traceback)
This function is called when an uncaught exception occurs. type is the exception class, value is the value supplied by the raise statement, and traceback is a traceback object. The default behavior is to print the exception and traceback to standard error. However, this function can be redefined to provide alternative handling of uncaught exceptions (which may be useful in specialized applications such as debuggers or CGI scripts).
exc_clear()
Clears all information related to the last exception that occurred. It only clears information specific to the calling thread.
exc_info()
Returns a tuple (type, value, traceback) containing information about the exception that’s currently being handled. type is the exception type, value is the exception parameter passed to raise, and traceback is a traceback object containing the call stack at the point where the exception occurred. Returns None if no exception is currently being handled.
exit([n])
Exits Python by raising the SystemExit exception. n is an integer exit code indicating a status code. A value of 0 is considered normal (the default); nonzero values are considered abnormal. If a noninteger value is given to n, it’s printed to sys.stderr and an exit code of 1 is used.
Returns the value of the check interval, which specifies how often the interpreter checks for signals, thread switches, and other periodic events.
getdefaultencoding()
Gets the default string encoding in Unicode conversions. Returns a value such as 'ascii' or 'utf-8'. The default encoding is set by the site module.
getdlopenflags()
Returns the flags parameter that is supplied to the C function dlopen() when loading extension modules on UNIX. See dl module.
getfilesystemencoding()
Returns the character encoding used to map Unicode filenames to filenames used by the underlying operating system. Returns 'mbcs' on Windows or 'utf-8' on Macintosh OS X. On UNIX systems, the encoding depends on locale settings and will return the value of the locale CODESET parameter. May return None, in which case the system default encoding is used.
_getframe([depth])
Returns a frame object from the call stack. If depth is omitted or zero, the topmost frame is returned. Otherwise, the frame for that many calls below the current frame is returned. For example, _getframe(1) returns the caller’s frame. Raises ValueError if depth is invalid.
getprofile()
Returns the profile function set by the setprofile() function.
getrecursionlimit()
Returns the recursion limit for functions.
getrefcount(object)
Returns the reference count of object.
getsizeof(object [, default])
Returns the size of object in bytes. This calculation is made by calling the _ _sizeof_ _() special method of object. If undefined, a TypeError will be generated unless a default value has been specified with the default argument. Because objects are free to define _ _sizeof_ _() however they wish, there is no guarantee that the result of this function is a true measure of memory use. However, for built-in types such as lists or string, it is correct.
gettrace()
Returns the trace function set by the settrace() function.
getwindowsversion()
Returns a tuple (major,minor,build,platform,text) that describes the version of Windows being used. major is the major version number. For example, a value of 4 indicates Windows NT 4.0, and a value of 5 indicates Windows 2000 and Windows XP variants. minor is the minor version number. For example, 0 indicates Windows 2000, whereas 1 indicates Windows XP. build is the Windows build number. platform identifies the platform and is an integer with one of the following common values: 0 (Win32s on Windows 3.1), 1 (Windows 95,98, or Me), 2 (Windows NT, 2000, XP), or 3 (Windows CE). text is a string containing additional information such as "Service Pack 3".
setcheckinterval(n)
Sets the number of Python virtual machine instructions that must be executed by the interpreter before it checks for periodic events such as signals and thread context switches. The default value is 10.
setdefaultencoding(enc)
Sets the default encoding. enc is a string such as 'ascii' or 'utf-8'. This function is only defined inside the site module. It can be called from user-definable sitecustomize modules.
setdlopenflags(flags)
Sets the flags passed to the C dlopen() function, which is used to load extension modules on UNIX. This will affect the way in which symbols are resolved between libraries and other extension modules. flags is the bitwise OR of values that can be found in the dl module (see Chapter 19, “Network Programming”)—for example, sys.setdlopenflags(dl.RTLD_NOW | dl.RTLD_GLOBAL).
setprofile(pfunc)
Sets the system profile function that can be used to implement a source code profiler.
setrecursionlimit(n)
Changes the recursion limit for functions. The default value is 1000. Note that the operating system may impose a hard limit on the stack size, so setting this too high may cause the Python interpreter process to crash with a Segmentation Fault or Access Violation.
settrace(tfunc)
Sets the system trace function, which can be used to implement a debugger. Refer to Chapter 11 for information about the Python debugger.
traceback
The traceback module is used to gather and print stack traces of a program after an exception has occurred. The functions in this module operate on traceback objects such as the third item returned by the sys.exc_info() function. The main use of this module is in code that needs to report errors in a non-standard way—for example, if you were running Python programs deeply embedded within a network server and you wanted to redirect tracebacks to a log file.
print_tb(traceback [, limit [, file]])
Prints up to limit stack trace entries from traceback to the file file. If limit is omitted, all the entries are printed. If file is omitted, the output is sent to sys.stderr.
print_exception(type, value, traceback [, limit [, file]])
Prints exception information and a stack trace to file. type is the exception type, and value is the exception value. limit and file are the same as in print_tb().
print_exc([limit [, file]])
Same as print_exception() applied to the information returned by the sys.exc_info() function.
format_exc([limit [, file]])
Returns a string containing the same information printed by print_exc().
print_last([limit [, file]])
Same as print_exception (sys.last_type, sys.last_value, sys.last_traceback, limit, file).
print_stack([frame [, limit [, file]]])
Prints a stack trace from the point at which it’s invoked. frame specifies an optional stack frame from which to start. limit and file have the same meaning as for print_tb().
extract_tb(traceback [, limit])
Extracts the stack trace information used by print_tb(). The return value is a list of tuples of the form (filename, line, funcname, text) containing the same information that normally appears in a stack trace. limit is the number of entries to return.
extract_stack([frame [, limit]])
Extracts the same stack trace information used by print_stack(), but obtained from the stack frame frame. If frame is omitted, the current stack frame of the caller is used and limit is the number of entries to return.
format_list(list)
Formats stack trace information for printing. list is a list of tuples as returned by extract_tb() or extract_stack().
format_exception_only(type, value)
Formats exception information for printing.
format_exception(type, value, traceback [, limit])
Formats an exception and stack trace for printing.
format_tb(traceback [, limit])
Same as format_list(extract_tb(traceback, limit)).
format_stack([frame [, limit]])
Same as format_list(extract_stack(frame, limit)).
tb_lineno(traceback)
Returns the line number set in a traceback object.
types
The types module defines names for the built-in types that correspond to functions, modules, generators, stack frames, and other program elements. The contents of this module are often used in conjunction with the built-in isinstance() function and other type-related operations.
Most of the preceding type objects serve as constructors that can be used to create an object of that type. The following descriptions provide the parameters used to create functions, modules, code objects, and methods. Chapter 3 contains detailed information about the attributes of the objects created and the arguments that need to be supplied to the functions described next.
FunctionType(code, globals [, name [, defarags [, closure]]])
Creates a new function object.
Creates a new code object.
MethodType(function, instance, class)
Creates a new bound instance method.
ModuleType(name [, doc])
Creates a new module object.
Notes
• The types module should not be used to refer the type of built-in objects such as integers, lists, or dictionaries. In Python 2, types contains other names such as IntType and DictType. However, these names are just aliases for the built-in type names of int and dict. In modern code, you should just use the built-in type names because the types module only contains the names listed previously in Python 3.
warnings
The warnings module provides functions to issue and filter warning messages. Unlike exceptions, warnings are intended to alert the user to potential problems, but without generating an exception or causing execution to stop. One of the primary uses of the warnings module is to inform users about deprecated language features that may not be supported in future versions of Python. For example:
Like exceptions, warnings are organized into a class hierarchy that describes general categories of warnings. The following lists the currently supported categories:
Each of these classes is available in the _ _builtin_ _ module as well as the exceptions module. In addition, they are also instances of Exception. This makes it possible to easily convert warnings into errors.
Warnings are issued using the warn() function. For example:
If desired, warnings can be filtered. The filtering process can be used to alter the output behavior of warning messages, to ignore warnings, or to turn warnings into exceptions. The filterwarnings() function is used to add a filter for a specific type of warning. For example:
Limited forms of filtering can also be specified using the –W option to the interpreter. For example:
% python –Wignore:the regex:DeprecationWarning
The following functions are defined in the warnings module:
warn(message[, category[, stacklevel]])
Issues a warning. message is a string containing the warning message, category is the warning class (such as DeprecationWarning), and stacklevel is an integer that specifies the stack frame from which the warning message should originate. By default, category is UserWarning and stacklevel is 1.
warn_explicit(message, category, filename, lineno[, module[, registry]])
This is a low-level version of the warn() function. message and category have the same meaning as for warn(). filename, lineno, and module explicitly specify the location of the warning. registry is an object representing all the currently active filters. If registry is omitted, the warning message is not suppressed.
showwarning(message, category, filename, lineno[, file])
Writes a warning to a file. If file is omitted, the warning is printed to sys.stderr.
formatwarning(message, category, filename, lineno)
Creates the formatted string that is printed when a warning is issued.
filterwarnings(action[, message[, category[, module[, lineno[, append]]]]])
Adds an entry to the list of warning filters. action is one of 'error', 'ignore', 'always', 'default', 'once', or 'module'. The following list provides an explanation of each:
message is a regular expression string that is used to match against the warning message. category is a warning class such as DeprecationError. module is a regular expression string that is matched against the module name. lineno is a specific line number or 0 to match against all lines. append specifies that the filter should be appended to the list of all filters (checked last). By default, new filters are added to the beginning of the filter list. If any argument is omitted, it defaults to a value that matches all warnings.
resetwarnings()
Resets all the warning filters. This discards all previous calls to filterwarnings() as well as options specified with –W.
Notes
• The list of currently active filters is found in the warnings.filters variable.
• When warnings are converted to exceptions, the warning category becomes the exception type. For instance, an error on DeprecationWarning will raise a DeprecationWarning exception.
• The –W option can be used to specify a warning filter on the command line. The general format of this option is
-Waction:message:category:module:lineno
where each part has the same meaning as for the filterwarning() function. However, in this case, the message and module fields specify substrings (instead of regular expressions) for the first part of the warning message and module name to be filtered, respectively.
weakref
The weakref module is used to provide support for weak references. Normally, a reference to an object causes its reference count to increase—effectively keeping the object alive until the reference goes away. A weak reference, on the other hand, provides a way of referring to an object without increasing its reference count. This can be useful in certain kinds of applications that must manage objects in unusual ways. For example, in an object-oriented program, where you might implement a relationship such as the Observer pattern, a weak reference can be used to avoid the creation of reference cycles. An example of this is shown in the “Object Memory Management” section of Chapter 7.
A weak reference is created using the weakref.ref() function as follows:
Once a weak reference is created, the original object can be obtained from the weak reference by simply calling it as a function with no arguments. If the underlying object still exists, it will be returned. Otherwise, None is returned to indicate that the original object no longer exists. For example:
The following functions are defined by the weakref module:
ref(object[, callback])
Creates a weak reference to object. callback is an optional function that will be called when object is about to be destroyed. If supplied, this function should accept a single argument, which is the corresponding weak reference object. More than one weak reference may refer to the same object. In this case, the callback functions will be called in order from the most recently applied reference to the oldest reference. object can be obtained from a weak reference by calling the returned weak reference object as a function with no arguments. If the original object no longer exists, None will be returned. ref() actually defines a type, ReferenceType, that can be used for type-checking and subclasses.
proxy(object[, callback])
Creates a proxy using a weak reference to object. The returned proxy object is really a wrapper around the original object that provides access to its attributes and methods. As long as the original object exists, manipulation of the proxy object will transparently mimic the behavior of the underlying object. On the other hand, if the original object has been destroyed, operations on the proxy will raise a weakref.ReferenceError to indicate that the object no longer exists. callback is a callback function with the same meaning as for the ref() function. The type of a proxy object is either ProxyType or CallableProxyType, depending on whether or not the original object is callable.
getweakrefcount(object)
Returns the number of weak references and proxies that refer to object.
getweakrefs(object)
Returns a list of all weak reference and proxy objects that refer to object.
WeakKeyDictionary([dict])
Creates a dictionary in which the keys are referenced weakly. When there are no more strong references to a key, the corresponding entry in the dictionary is automatically removed. If supplied, the items in dict are initially added to the returned WeakKeyDictionary object. Because only certain types of objects can be weakly referenced, there are numerous restrictions on acceptable key values. In particular, built-in strings cannot be used as weak keys. However, instances of user-defined classes that define a _ _hash_ _() method can be used as keys. An instance of WeakKeyDictionary has two methods, iterkeyrefs() and keyrefs(), that return the weak key references.
WeakValueDictionary([dict])
Creates a dictionary in which the values are referenced weakly. When there are no more strong references to a value, corresponding entries in the dictionary will be discarded. If supplied, the entries in dict are added to the returned WeakValueDictionary. An instance of WeakValueDictionary has two methods, itervaluerefs() and valuerefs(), that return the weak value references.
ProxyTypes
This is a tuple (ProxyType, CallableProxyType) that can be used for testing if an object is one of the two kinds of proxy objects created by the proxy() function—for example, isinstance(object, ProxyTypes).
Example
One application of weak references is to create caches of recently computed results. For instance, if a function takes a long time to compute a result, it might make sense to cache these results and to reuse them as long as they are still in use someplace in the application. For example:
Notes
• Only class instances, functions, methods, sets, frozen sets, files, generators, type objects, and certain object types defined in library modules (for example, sockets, arrays, and regular expression patterns) support weak references. Built-in functions and most built-in types such as lists, dictionaries, strings, and numbers cannot be used.
• If iteration is ever used on a WeakKeyDictionary or WeakValueDictionary, great care should be taken to ensure that the dictionary does not change size because this may produce bizarre side effects such as items mysteriously disappearing from the dictionary for no apparent reason.
• If an exception occurs during the execution of a callback registered with ref() or proxy(), the exception is printed to standard error and ignored.
• Weak references are hashable as long as the original object is hashable. Moreover, the weak reference will maintain its hash value after the original object has been deleted, provided that the original hash value is computed while the object still exists.
• Weak references can be tested for equality but not for ordering. If the objects are still alive, references are equal if the underlying objects have the same value. Otherwise, references are equal if they are the same reference.