Developer’s Library: Essential References for Programming Professionals
Developer’s Library books are designed to provide practicing programmers with unique, high-quality references and tutorials on the programming languages and technologies they use in their daily work.
All books in the Developer’s Library are written by expert technology practitioners who are especially skilled at organizing and presenting information in a way that’s useful for other programmers.
Key titles include some of the best, most widely acclaimed books within their topic areas:
PHP and MySQL Web Development
Luke Welling & Laura Thomson
ISBN 978-0-672-32916-6
MySQL
Paul DuBois
ISBN-13: 978-0-672-32938-8
Linux Kernel Development
Robert Love
ISBN-13: 978-0-672-32946-3
Python Essential Reference
David Beazley
ISBN-13: 978-0-672-32862-6
Programming in Objective-C
Stephen G. Kochan
ISBN-13: 978-0-321-56615-7
PostgreSQL
Korry Douglas
ISBN-13: 978-0-672-33015-5
Developer’s Library books are available at most retail and online bookstores, as well as by subscription from Safari Books Online at safari.informit.com
Developer’s Library
informit.com/devlibrary
Python
Essential Reference
Fourth Edition
Upper Saddle River, NJ • Boston • Indianapolis • San Francisco
New York • Toronto • Montreal • London • Munich • Paris • Madrid
Cape Town • Sydney • Tokyo • Singapore • Mexico City
Fourth Edition
Copyright © 2009 by Pearson Education, Inc.
All rights reserved. No part of this book shall be reproduced, stored in a retrieval system, or transmitted by any means, electronic, mechanical, photocopying, recording, or otherwise, without written permission from the publisher. No patent liability is assumed with respect to the use of the information contained herein. Although every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions. Nor is any liability assumed for damages resulting from the use of the information contained herein.
ISBN-13: 978-0-672-32978-4
ISBN-10: 0-672-32978-6
Printed in the United States of America
First Printing June 2009
Acquisitions Editor
Mark Taber
Development Editor
Michael Thurston
Managing Editor
Patrick Kanouse
Project Editor
Seth Kerney
Copy Editor
Lisa Thibault
Indexer
David Beazley
Proofreader
Megan Wade
Technical Editors
Noah Gift
Kurt Grandis
Publishing Coordinator
Vanessa Evans
Book Designer
Gary Adair
Compositor
Bronkella Publishing
Library of Congress Cataloging-in-Publication data is on file.
Trademarks
All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Addison-Wesley cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark.
Warning and Disclaimer
Every effort has been made to make this book as complete and as accurate as possible, but no warranty or fitness is implied. The information provided is on an “as is” basis. The author and the publisher shall have neither liability nor responsibility to any person or entity with respect to any loss or damages arising from the information contained in this book.
Bulk Sales
Addison-Wesley offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales. For more information, please contact
U.S. Corporate and Government Sales
1-800-382-3419
[email protected]
For sales outside of the U.S., please contact
International Sales
[email protected]
To register this product and gain access to bonus content, go to www.informit.com/register to sign in and enter the ISBN. After you register the product, a link to the additional content will be listed on your Account page, under Registered Products.
Contents at a Glance
2 Lexical Conventions and Syntax
5 Program Structure and Control Flow
6 Functions and Functional Programming
7 Classes and Object-Oriented Programming
8 Modules, Packages, and Distribution
11 Testing, Debugging, Profiling, and Tuning
15 Data Structures, Algorithms, and Code Simplification
18 File and Directory Handling
21 Network Programming and Sockets
22 Internet Application Programming
24 Internet Data Handling and Encoding
25 Miscellaneous Library Modules
Part III: Extending and Embedding
26 Extending and Embedding Python
Table of Contents
Variables and Arithmetic Expressions
2 Lexical Conventions and Syntax
Line Structure and Indentation
Identifiers and Reserved Words
Operators, Delimiters, and Special Symbols
Reference Counting and Garbage Collection
Built-in Types for Representing Data
Built-in Types for Representing Program Structure
Built-in Types for Interpreter Internals
Object Behavior and Special Methods
Object Creation and Destruction
Object Comparison and Ordering
Attribute Wrapping and Descriptors
Boolean Expressions and Truth Values
5 Program Structure and Control Flow
Program Structure and Execution
Context Managers and the with Statement
6 Functions and Functional Programming
Parameter Passing and Return Values
Functions as Objects and Closures
Coroutines and yield Expressions
Using Generators and Coroutines
7 Classes and Object-Oriented Programming
Polymorphism Dynamic Binding and Duck Typing
Static Methods and Class Methods
Data Encapsulation and Private Attributes
Object Representation and Attribute Binding
Types and Class Membership Tests
8 Modules, Packages, and Distribution
Modules and the import Statement
Importing Selected Symbols from a Module
Module Loading and Compilation
Module Reloading and Unloading
Distributing Python Programs and Libraries
Installing Third-Party Libraries
Standard Input, Output, and Error
Variable Interpolation in Text Output
Object Persistence and the pickle Module
Interpreter Options and Environment
11 Testing, Debugging, Profiling, and Tuning
Documentation Strings and the doctest Module
Unit Testing and the unittest Module
The Python Debugger and the pdb Module
Debugging from the Command Line
12 Built-In Functions and Exceptions
Real-Valued Random Distributions
15 Data Structures, Algorithms, and Code Simplification
Packing and Unpacking Functions
Relational Database API Specification
Mapping Results into Dictionaries
18 File and Directory Handling
Mathematical Operations Involving Dates
Miscellaneous Utility Functions
File Creation and File Descriptors
Concurrent Programming and Python
Shared Data and Synchronization
Miscellaneous Utility Functions
General Advice on Multiprocessing
Semaphore and Bounded Semaphore
Thread Termination and Suspension
21 Network Programming and Sockets
Advanced Asynchronous I/O Example
When to Consider Asynchronous Networking
Customization of Application Servers
22 Internet Application Programming
http.server (BaseHTTPServer, CGIHTTPServer, SimpleHTTPServer)
urllib.robotparser (robotparser)
xmlrpc.server (SimpleXMLRPCServer, DocXMLRPCServer)
24 Internet Data Handling and Encoding
25 Miscellaneous Library Modules
26 Extending and Embedding Python
Compiling and Packaging Extensions
Type Conversion from Python to C
Type Conversion from C to Python
Embedding the Python Interpreter
Basic Interpreter Operation and Setup
Converting Python Objects to C
Alternative Type Construction Methods
Advanced Extending and Embedding
Source Code Encoding and Identifiers
Set and Dictionary Comprehensions
File Names, Arguments, and Environment Variables
Simultaneous Python 2 and Python 3 Support
About the Author
David M. Beazley is a long-time Python enthusiast, having been involved with the Python community since 1996. He is probably best known for his work on SWIG, a popular software package for integrating C/C++ programs with other programming languages, including Python, Perl, Ruby, Tcl, and Java. He has also written a number of other programming tools, including PLY, a Python implementation of lex and yacc. Dave spent seven years working in the Theoretical Physics Division at Los Alamos National Laboratory, where he helped pioneer the use of Python with massively parallel supercomputers. After that, Dave went off to work as an evil professor, where he enjoyed tormenting college students with a variety of insane programming projects. However, he has since seen the error of his ways and is now working as an independent software developer, consultant, Python trainer, and occasional jazz musician living in Chicago. He can be contacted at http://www.dabeaz.com.
About the Technical Editor
Noah Gift is the co-author of Python For UNIX and Linux System Administration (O’Reilly) and is also working on Google App Engine In Action (Manning). He is an author, speaker, consultant, and community leader, writing for publications such as IBM developerWorks, Red Hat Magazine, O’Reilly, and MacTech. His consulting company’s website is http://www.giftcs.com, and much of his writing can be found at http://noahgift.com. You can also follow Noah on Twitter.
Noah has a master’s degree in CIS from Cal State, Los Angeles, a B.S. in nutritional science from Cal Poly San Luis Obispo, is an Apple and LPI-certified SysAdmin, and has worked at companies such as Caltech, Disney Feature Animation, Sony Imageworks, and Turner Studios. He is currently working at Weta Digital in New Zealand. In his free time he enjoys spending time with his wife Leah and their son Liam, composing for the piano, running marathons, and exercising religiously.
Acknowledgments
This book would not be possible without the support of many people. First and foremost, I’d like to thank Noah Gift for jumping into the project and providing his amazing feedback on the fourth edition. Kurt Grandis also provided useful comments for many chapters. I’d also like to acknowledge past technical reviewers Timothy Boronczyk, Paul DuBois, Mats Wichmann, David Ascher, and Tim Bell for their valuable comments and advice that made earlier editions a success. Guido van Rossum, Jeremy Hylton, Fred Drake, Roger Masse, and Barry Warsaw also provided tremendous assistance with the first edition while hosting me for a few weeks back in the hot summer of 1999. Last, but not least, this book would not be possible without all of the feedback I received from readers. There are far too many people to list individually, but I have done my best to incorporate your suggestions for making the book even better. I’d also like to thank all the folks at Addison-Wesley and Pearson Education for their continued commitment to the project and assistance. Mark Taber, Michael Thurston, Seth Kerney, and Lisa Thibault all helped out to get this edition out the door in good shape. A special thanks is in order for Robin Drake, whose tremendous effort in editing previous editions made the third edition possible. Finally, I’d like to acknowledge my amazing wife and partner Paula Kamen for all of her encouragement, diabolical humor, and love.
We Want to Hear from You!
As the reader of this book, you are our most important critic and commentator. We value your opinion and want to know what we’re doing right, what we could do better, what areas you’d like to see us publish in, and any other words of wisdom you’re willing to pass our way.
You can email or write me directly to let me know what you did or didn’t like about this book—as well as what we can do to make our books stronger.
Please note that I cannot help you with technical problems related to the topic of this book, and that due to the high volume of mail I receive, I might not be able to reply to every message.
When you write, please be sure to include this book’s title and author as well as your name and phone or email address. I will carefully review your comments and share them with the author and editors who worked on the book.
Email: [email protected]
Mail: Mark Taber
Associate Publisher
Pearson Education
800 East 96th Street
Indianapolis, IN 46240 USA
Reader Services
Visit our website and register this book at informit.com/register for convenient access to any updates, downloads, or errata that might be available for this book.