Unlike many language communities, the Python world has a very clear center. This center has grown over the years, with Guido van Rossum as the permanent core, surrounded physically by a trusty cohort called “Pythonlabs,” and surrounded virtually by the “python-dev group.” Pythonlabs consists of a few key Python developers(Tim Peters, Barry Warsaw, Jeremy Hylton, and Fred Drake) who were recruited by Guido to work with him on Python and Python-related projects, first at BeOpen.com, and then at Zope Corporation. Along with Guido, they have generally been the ones making the most radical changes to the Python internals, although there have been some notable exceptions from other contributors.

Python is too big for so few people to manage and grow as fast as its users would like (especially as Python is only a part-time job, even for Guido). A supporting cast, generally referred to as the python-dev crowd after the mailing list that anchors the discussions, is available to help in design discussions, implementing, testing, and, most of all, arguing (all in good faith, though).

To most Python users, however, the work of Pythonlabs and Python-dev is gratefully acknowledged but somewhat mysterious. Many more people live in one of the outer layers of Pythondom, either virtual or physical (or, hopefully, both). We’ll get back to the technically “deep” layers later—it’s far more reasonable to learn about a community from the tourism bureau than from the city planning committee, however.

While most Python-related communications occur on the Internet, it’s nice to ground the names with faces and accents, and to get a feel for the real-world personalities behind the online personas. There are two great ways to do that, each with their own benefits: user groups and conferences.

You may have a local Python or Zope user group in your neighborhood. There is a list at http://www.python.org/UserGroups.html, but check with your local computer society, Linux users group or other such organization for possibly more up-to-date information. If there isn’t one yet, chances are quite good that there is pent-up demand for it, and they’re easy to organize. It’s a great occasion to learn more about Python, find out who else is using Python locally (which can be helpful at resume-passing time), and meet people with whom you share at least one interest. Both of the authors visit local user groups on a regular basis.

Whether or not you partake in an occasional beer with local Pythoneers, we encourage you to go to Python conferences. There are several regular conferences, all worth attending.

The oldest conference, the International Python Conference (IPC), is now part of the O’Reilly Open Source Convention (OSCON) (see http://conferences.oreillynet.com for information). IPC has grown over the years from a very small informal gathering to a large, very well attended meeting where Python experts, enthusiasts and novices from around the world come to mingle. For many, it is the only occasion to talk to their Python acquaintances in real life, and it’s well worth it.

A new conference, PyCon, has recently been started. Meant to complement the track at OSCON, it is a low-cost, hacker’s conference, emphasizing technical discussions and foregoing the more commercial aspects of trade shows such as an exhibit hall and fancy lunches. When not writing this chapter, I (David) am helping to plan the second PyCon event, which promises to have very high-quality presentations.

Both of the two conferences mentioned are based in the U.S. An alternative for some is to head to Europe for some of the Python conferences there: EuroPython (http://www.europython.org), and the Python UK Conference (http://www.python-uk.org). More conferences may have sprung up to accommodate the crowds. See http://www.python.org/workshops/ for updates.

As easy as Python is to learn, no book or website can give all the answers. If you have a Python-related question, there are a few avenues for free and paid support. Information about all of the mailing lists mentioned below, including how to subscribe, is available at http://www.python.org/psa/MailingLists.html.

All of the resources mentioned thus far have been generalist resources. Two kinds of more focused discussion groups also need to be mentioned. The first and most essential one is the discussion around the implementation of the language itself, on a list called python-dev. Additionally, some specialized topics naturally lead to long-term discussions, which can get quite technical but still remain within a well-defined domain. Typical examples of this kind of special interest group are the discussions aimed at defining specialized libraries that should be added to Python, such as those for scientific processing, database interfaces, XML or Unicode support. People interested in working on such areas typically form a special interest group(s) mailing list to coordinate their activities, get access to an area of http://www.python.org to publicize their results, and, when they have reached internal agreement, lobby the greater Python community (and Guido in particular) for inclusion of a particular functionality (as the Numeric SIG did for some syntactic changes) or just to report an agreement (as the Database SIG did to announce the definition of the DB-API standard). These days, many satellite discussions occur in other mailing lists that aren’t “official” SIGs, although those tend to not feed back to the language core quite as much. Information about SIGs is always available at http://www.python.org/sigs.

Apart from those few “physical” conversations that happen in the Reston, VA area (where Pythonlabs team members are located), daily discussion about the development of Python itself happens on a mailing list called python-dev. The number of people participating in this list is considerably larger (dozens of active contributors, probably thousands of regular readers), and provides a very good forum for people who already know most of what there is to know about Python today, and want to participate in shaping tomorrow’s Python. If you’re curious, you can subscribe to the mailing list (http://mail.python.org/mailman/listinfo/python-dev), read it through a mail/web interface (e.g., http://aspn.activestate.com/ASPN/Mail/Browse/Threaded/python-dev/) or through a mail/news interfaces (see e.g. http://www.gmane.org). However, most readers of this book will find the discussions on python-dev to be much too detailed and technical. To get a feel for what’s keeping the core Python developers busy while staying awake, you’re better off reading the edited summaries that get posted to python-dev and python-list on a periodic basis and that are archived at http://www.python.org/dev/summary/ (you can also ask to receive them by email). It’s important to keep in mind that Python-dev is a working group. If you are a Python novice, you’re welcome to read, but you probably shouldn’t post except in very rare circumstances. You should definitely not ask for help on python-dev—as we’ve seen, there are many other, more appropriate places to go for help. Similarly, requests for new syntax or new features are in large part inappropriate on python-dev.

If you’re too busy to follow python-list on a regular basis, there are several options to make sure that you learn of all new developments in the Python world.

Major announcements typically get sent both to python-list and to python-announce, which is a moderated list. That’s the primary forum to announce new Python packages or modules, new conferences, and other major announcements of broad interest to the community.

Several people report on Python developments, such as Fredrik Lundh, a longstanding web log of Python news at http://www.pythonware.com/daily/.

A rotating volunteer also monitors python-list and summarizes it on a weekly basis. Sponsored by Doctor Dobb’s Journal, archives and information on how to subscribe are available at http://www.ddj.com/topics/pythonurl/.

It used to be hard to find things on the Internet. Some of us remember days before the Web, when word of mouth and secret handshakes seemed to be required to find particular pieces of software. These days, search engines like Google do 95% of the hard work. Regardless of the topic, searches on Google are very likely to get you what you want.

Software that’s available on the Web has typically been announced in public, or at the very least discussed in public. You can search the various mailing lists mentioned above with specialized search engines, such as Google’s Groups interface (although that doesn’t cover all of the Python mailing lists, only those mirrored as newsgroups), http://python.org/search, or the mailing list archives at http://aspn.activestate.com/ASPN.

“The Vaults of Parnassus” is a fairly old (in Internet years) and well-established directory of Python software. It uses a library-style directory of Python software and Python-related tools. The vaults are at: http://py.vaults.ca/. Note that the vaults archive only metadata finding something on the vaults is no guarantee that the pages it refers you to are still around, or that the information on the vaults are necessarily up to date.

A new project which, unlike some of its predecessors, seems likely to succeed is called PyPI (Python Package Index). Hosted at http://www.python.org/pypi, the current prototype lets people register their python package either manually, or, preferably, using the package description software distutils, which is part of the standard library. As of this writing there are only a few dozen packages listed, but by the time you read this the catalog is likely to be much larger.

The Python Cookbook is a joint project combining the efforts of ActiveState, O’Reilly and Associates, and the Python community. ActiveState hosts a web site (aspn.ActiveState.com/Python/Cookbook), which lets anyone post their favorite recipe of Python code, and solicit feedback from readers of the site. O’Reilly published a book from selected, edited, and expanded recipes called the Python Cookbook, coedited by Alex Martelli and David Ascher. The book contains hundreds of well-motivated recipes explained in detail, and has become a favorite even of long-time Python programmers. The online site contains hundreds more recipes, and is constantly being updated. Both are excellent resources for that smallest kind of software package, the snippets and idioms that define the fluent speaker of a language.

While each operating system provides a wide variety of interfaces, Unix and related operating systems like Linux tend to provide that interface through command-line tools and special-purpose files, both of which tend to vary too much across versions to allow for useful programmatic interfaces. Windows and Macintosh use a more API-oriented approach, and as a result make the operating system more naturally accessible from a programming language like Python. There are Python interfaces to pretty much every corner of Windows and the Macintosh APIs.

There are some domains of computing that lend themselves to well-defined libraries. These typically involve specialized data types (e.g., arrays of numbers, dates) or special-purpose algorithms (e.g., three-dimensional graphics engines). In this section we survey some of the most popular such libraries.

Scientific computing libraries

Python has a significant following in the scientific and engineering fields. Its fairly mathematically reasonable syntax and ease of extensibility make it a good match for scientists who need to combine easy to use syntax with the ability to attach high-power computational engines.

The granddaddy of scientific libraries in Python is called Numeric Python, NumPy, or Numeric for short. Numeric consists of a set of Python and C extension modules that provide very powerful and high performance array operations. With Numeric, one can do “en masse” operations on very large multidimensional arrays of numbers much faster than one can do in pure Python. Numeric is the current standard solution for doing such operations in Python, although many are looking to its proposed replacement, numarray.

Here’s an example of typical NumPy code, numpytest.py, and one representation of the data in generates:

from Numeric import *              
coords = arange(-6, 6, .02)               # Create a range of coordinates.
xs = sin(coords)                          # Take the sine of all of the x's.
ys = cos(coords)*exp(-coords*coords/18.0) # Take a complex function of the y's.
zx = xs * ys[:,NewAxis]                   # Multiply the x row with the y column.

If you remember your math, you might figure out that xs is an array of the sines of the numbers between -6 and 6, and ys is an array of the cosines of those same numbers scaled by an exponential function centered at 0. zs is simply the outer product of those two arrays of numbers. If you’re curious as to what that might look like, you could convert the array zs into an image and obtain the image shown in Figure 29-1.

Figure 29-1. Graphical representation of the array zs in numpytest.py

NumPy lets you manipulate very large arrays of numbers very efficiently. The preceding code runs orders of magnitude faster than comparable code using large lists of numbers and uses a fraction of the memory. Many Python users never have to deal with these kinds of issues, but many scientists and engineers require such capabilities daily.

Numarray (http://stsdas.stsci.edu/numarray/) is a reimplementation of the basic functionality of Numeric, while aiming to fix some of the problems with the original having to do with type conversions, extensibility, memory efficiency and other factors. It’s too early to tell, but our sources indicate that numarray is well positioned to take over Numeric’s market share in a nice, progressive, and orchestrated transition.

For those interested in doing more with numbers, the SciPy project is worth looking at. SciPy supplements the core array-crunching modules with a variety of high-level scientific and engineering modules, such as solvers, optimizers, statistical routines, and a very interesting inlining tool that compiles expressions using C++ in a semi-transparent way, called weave. SciPy also includes the most Pythonic scientific graphing software, called Chaco. Information about SciPy can be found at its web site, http://www.scipy.org.

Finally, there are other scientific libraries for Python not listed above—some for dealing with Fortran libraries, some for doing specialized processing in the fields of chemistry, genetics, etc. Use the resources mentioned to look for software in your particular field.

import pygame
pygame.cdrom.init(  )
cd_object = pygame.cdrom.CD(pygame.cdrom.get_count(  )-1)
cd_object.init(  )
if cd_object.get_track_audio(i):
    audio_track_found = 1
    cd_object.play(i)
    pygame.time.delay(5000)
cd_object.eject(  )
cd_object.quit(  )

Working with a GUI toolkit is a requirement for many people who write end-user- software. The decision of which GUI toolkit to use is typically quite a complex one, involving such factors as portability, performance, look and feel, licensing and cost, threading models, etc. Suffice it to say that Python has interfaces to all of the popular GUI toolkits on the major platforms.

There are several popular cross-platform toolkits with Python interfaces:

Another cross-platform solution is:

For the most native Windows look and feel, you can use either low-level Win32 calls through the win32ui and win32gui modules, or the higher-level Microsoft Foundation Classes (MFC). Both interfaces are available with win32all, either standalone or bundled with ActivePython.

Jython users can use the native Java toolkits (AWT and Swing).

Unix/Linux users can also use:

Several other toolkit interfaces are available, but have been omitted here because they are not as popular or battle-tested.

Many of the libraries mentioned are wrappers around third-party libraries written in a compiled language like C, C++ or Fortran. There are several options available to those wishing to expose such libraries to Python.

def primes(int kmax):
    cdef int n, k, i
    cdef int p[1000]
    result = [  ]
    if kmax > 1000:
        kmax = 1000
    k = 0
    n = 2
    while k < kmax:
        i = 0
        while i < k and n % p[i] <> 0:
            i = i + 1
        if i == k:
            p[k] = n
            k = k + 1
            result.append(n)
        n = n + 1
    return result

def binary_search(seq, t):
        min = 0; max = len(seq) - 1
        while 1:
            if max < min:
                return -1
            m = (min  + max)  / 2
            if seq[m] < t: 
                min = m  + 1 
            elif seq[m] > t: 
                max = m  - 1 
            else:
                return m

def c_int_binary_search(seq,t):
    # Do a little type checking in Python.
    assert(type(t) == type(1))
    assert(type(seq) == type([  ]))
    
    # Now the C code
    code = """
           # Line 29 "binary_search.py"
           int val, m, min = 0;  
           int max = seq.length(  ) - 1;
           PyObject *return_val; 
           for(;;) {
               if (max < min) { 
                   return_val =  Py::new_reference_to(Py::Int(-1)); 
                   break;
               } 
               m =  (min + max) /2;
               val = py_to_int(PyList_GetItem(seq.ptr(  ),m),"val"); 
               if (val  < t) 
                   min = m  + 1;
               else if (val >  t)
                   max = m - 1;
               else {
                    return_val = Py::new_reference_to(Py::Int(m));
                    break;
                }
           }
           """
    return inline(code,['seq','t'])

>>> from ctypes import cdll, c_double
>>> printf = cdll.msvcrt.printf
>>> printf("An int %d, a double %f
", 1234, c_double(3.14))
An int 1234, a double 3.140000

In this section, we’ll list some modules that, while not necessarily deeply complex or major efforts, provide nice solutions to common problems.

The platform module (part of the standard library in Python 2.3 but available for earlier python version at http://www.egenix.com/files/python/platform.py) is a module that lets you find out many more subtle variations of what your current platform, compared to sys.platform. Instead of just saying win32, it lets you know that you’re running on e.g. "Windows-2000-5.0.2195-SP3“. Sometimes this sort of information is crucial, and it’s hard to identify without code like that in platform.py.

turtle.py, which is part of the standard distribution but unknown to many, provides a simple LOGO-like system for interactive drawing and beginning programmers. Built on Tkinter, it provides a nice first step for teaching programming.

Trent Mick’s go (at http://starship.python.net/crew/tmick/) is a simple command line tool that makes it trivial to build bookmarks of often-used directories. Instead of having to remember long path names, you can teach go which directories you use often and then use two-word commands to go there (go home, go python23, etc.).

Python developers who wish to share their Python code with other Python developers should learn how to use the distutils package, which lets people build distributions of their programs that are easy for other developers to install. distutils is documented as part of the standard Python library (http://www.python.org/doc/current/dist/dist.html).

Python developers who are building standalone applications for use by non-Python developers have a variety of approaches available to them. To make it easy for end users, it is generally a good idea to ship a single file that installs everything the users need—this may include all or a part of the Python distribution. While you can do it the hard way, by shipping the hundreds of files that make up this distribution, it is much easier to use a tool that packages the program and the part of the standard distribution that the program needs.

The Python distribution includes such a program, called freeze, which, while it should work, has far fewer features than the alternatives.

Zope is the grand-daddy of Python web application frameworks. While Zope is open source, it is very much the product of a company called Zope Corp., in collaboration with a vast user community. Zope is a very powerful tool for building content management systems, including such advanced features as replication, transactional support, sophisticated security models and workflow. Zope often stumps people who expect it to be a simple system. While efforts are under way to redesign part of Zope to make learning Zope easier, those who find Zope most useful are typically those with very large or complex websites to build. Information about the Zope software is available at http://www.zope.org, and information about Zope Corp. is at http://www.zope.com.

If Zope is the web application server for sites with sophisticated workflow, Twisted is more of a swiss army knife for networked application development. Not strictly focused on the web and the HTTP/HTML standards that anchor the web, Twisted is a framework for building networked applications. Using Twisted, it is relatively easy to build high-performance clients and servers for any protocol, from instant messaging to IRC to HTTP to NNTP. The Twisted framework, like all frameworks, requires a certain learning curve. Those who do learn it do tend to be passionate about it, and it seems to perform admirably. Twisted’s home is http://www.twistedmatrix.com.

Quixote is a dynamic web application framework built by Python programmers for Python programmers. Unlike many of the alternatives such as Zope, Quixote deliberately does not try to cater to web designers. To use Quixote means to program in Python, even for HTML generation. To those of us who are more comfortable with Python modules and classes than with HTTP redirects, however, that’s a great benefit. Quixote has wonderfully clear documentation (at least if you are comfortable with Python) at http://www.mems-exchange.org/software/quixote/.

The three systems, Zope, Twisted, and Quixote, while probably among the most popular, do not cover the extent of Python/web frameworks. If embarking on a new project, you owe it to yourself to compare the various frameworks and figure out which one is best for you. Programming on the Web is a complex task with lots of variables—it’s very possible that in your particular case, you’d want to use something more like a PHP-style system such as Spyce (http://spyce.sourceforge.net), or more of a component-oriented system like WebWare for Python (http://webware.sourceforge.net/).