Before we get to any code, I want to start out by defining what we mean by “integration” here. Although that term can be interpreted almost as widely as “object,” our focus in this chapter is on tight integration—where control is transferred between languages by a simple, direct, and fast in-process function call. Although it is also possible to link components of an application less directly using IPC and networking tools such as sockets and pipes that we explored earlier in the book, we are interested in this part of the book in more direct and efficient techniques.

When you mix Python with components written in C (or other compiled languages), either Python or C can be “on top.” Because of that, there are two distinct integration modes and two distinct APIs:

Extending generally has three main roles: to optimize programs—recoding parts of a program in C is a last-resort performance boost; to leverage existing libraries—opening them up for use in Python code extends their reach; and to allow Python programs to do things not directly supported by the language—Python code cannot normally access devices at absolute memory addresses, for instance, but can call C functions that do. For example, the NumPy package for Python is largely an instance of extending at work: by integrating optimized numeric libraries, it turns Python into a flexible and efficient system for numeric programming that some compare to Matlab.

Embedding typically takes the role of customization—by running user-configurable Python code, a system can be modified without shipping or building its full source code. For instance, some programs provide a Python customization layer that can be used to modify the program on site by modifying Python code. Embedding is also sometimes used to route events to Python-coded callback handlers. Python GUI toolkits, for example, usually employ embedding in some fashion to dispatch user events.

Figure 20-1 sketches this traditional dual-mode integration model. In extending, control passes from Python through a glue layer on its way to C code. In embedding, C code processes Python objects and runs Python code by calling Python C API functions. Because Python is “on top” in extending, it defines a fixed integration structure, which can be automated with tools such as SWIG—a code generator we’ll meet in this chapter, which produces glue code required to wrap C and C++ libraries. Because Python is subordinate in embedding, it instead provides a set of API tools which C programs employ as needed.

Figure 20-1. Traditional integration model

In some models, things are not as clear-cut. For example, under the ctypes module discussed later, Python scripts make library calls rather than employing C glue code. In systems such as Cython (and its Pyrex predecessor), things are more different still—C libraries are produced from combinations of Python and C code. And in Jython and IronPython, the model is similar, but Java and C# components replace the C language, and the integration is largely automated. We will meet such alternative systems later in this chapter. For now, our focus is on traditional Python/C integration models.

This chapter introduces extending first, and then moves on to explore the basics of embedding. Although we will study these topics in isolation, keep in mind that many systems combine the two techniques. For instance, embedded Python code run from C can also import and call linked-in C extensions to interface with the enclosing application. And in callback-based systems, C libraries initially accessed through extending interfaces may later use embedding techniques to run Python callback handlers on events.

For example, when we created buttons with Python’s tkinter GUI library earlier in the book, we called out to a C library through the extending API. When our GUI’s user later clicked those buttons, the GUI C library caught the event and routed it to our Python functions with embedding. Although most of the details are hidden to Python code, control jumps often and freely between languages in such systems. Python has an open and reentrant architecture that lets you mix languages arbitrarily.

To use SWIG, instead of writing the C code in the prior section, write the C function you want to use from Python without any Python integration logic at all, as though it is to be used from C alone. For instance, Example 20-4 is a recoding of Example 20-1 as a straight C function.

/*********************************************************************
 * A simple C library file, with a single function, "message",
 * which is to be made available for use in Python programs.
 * There is nothing about Python here--this C function can be
 * called from a C program, as well as Python (with glue code).
 *********************************************************************/

#include <string.h>
#include <hellolib.h>

static char result[1024];                /* this isn't exported */

char *
message(char *label)                     /* this is exported */
{
    strcpy(result, "Hello, ");           /* build up C string */
    strcat(result, label);               /* add passed-in label */
    return result;                       /* return a temporary */
}

While you’re at it, define the usual C header file to declare the function externally, as shown in Example 20-5. This is probably overkill for such a small example, but it will prove a point.

/********************************************************************
 * Define hellolib.c exports to the C namespace, not to Python
 * programs--the latter is defined by a method registration
 * table in a Python extension module's code, not by this .h;
 ********************************************************************/

extern char *message(char *label);

Now, instead of all the Python extension glue code shown in the prior sections, simply write a SWIG type declarations input file, as in Example 20-6.

/******************************************************
 * Swig module description file, for a C lib file.
 * Generate by saying "swig -python hellolib.i".
 ******************************************************/

%module hellowrap

%{
#include <hellolib.h>
%}

extern char *message(char*);    /* or: %include "../HelloLib/hellolib.h"   */
                                /* or: %include hellolib.h, and use -I arg */

This file spells out the C function’s type signature. In general, SWIG scans files containing ANSI C and C++ declarations. Its input file can take the form of an interface description file (usually with a .i suffix) or a C/C++ header or source file. Interface files like this one are the most common input form; they can contain comments in C or C++ format, type declarations just like standard header files, and SWIG directives that all start with %. For example:

In this example, SWIG could also be made to read the hellolib.h header file of Example 20-5 directly. But one of the advantages of writing special SWIG input files like hellolib.i is that you can pick and choose which functions are wrapped and exported to Python, and you may use directives to gain more control over the generation process.

SWIG is a utility program that you run from your build scripts; it is not a programming language, so there is not much more to show here. Simply add a step to your makefile that runs SWIG and compile its output to be linked with Python. Example 20-7 shows one way to do it on Cygwin.

##################################################################
# Use SWIG to integrate hellolib.c for use in Python programs on
# Cygwin.  The DLL must have a leading "_" in its name in current
# SWIG (>1.3.13) because also makes a .py without "_" in its name.
##################################################################

PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1
CLIB  = ../HelloLib
SWIG  = /cygdrive/c/temp/swigwin-2.0.0/swig

# the library plus its wrapper
_hellowrap.dll: hellolib_wrap.o $(CLIB)/hellolib.o
        gcc -shared hellolib_wrap.o $(CLIB)/hellolib.o 
                       -L$(PYLIB) -lpython3.1 -o $@

# generated wrapper module code
hellolib_wrap.o: hellolib_wrap.c $(CLIB)/hellolib.h
        gcc hellolib_wrap.c -g -I$(CLIB) -I$(PYINC) -c -o $@

hellolib_wrap.c: hellolib.i
        $(SWIG) -python -I$(CLIB) hellolib.i

# C library code (in another directory)
$(CLIB)/hellolib.o: $(CLIB)/hellolib.c $(CLIB)/hellolib.h
        gcc $(CLIB)/hellolib.c -g -I$(CLIB) -c -o $(CLIB)/hellolib.o

clean:
        rm -f *.dll *.o *.pyc core
force:
        rm -f *.dll *.o *.pyc core hellolib_wrap.c hellowrap.py

When run on the hellolib.i input file by this makefile, SWIG generates two files:

The former is named for the input file, and the latter per the %module directive. Really, SWIG generates two modules today: it uses a combination of Python and C code to achieve the integration. Scripts ultimately import the generated Python module file, which internally imports the generated and compiled C module. You can wade through this generated code in the book’s examples distribution if you are so inclined, but it is prone to change over time and is too generalized to be simple.

To build the C module, the makefile runs SWIG to generate the glue code; compiles its output; compiles the original C library code if needed; and then combines the result with the compiled wrapper to produce _hellowrap.dll, the DLL which hellowrap.py will expect to find when imported by a Python script:

.../PP4E/Integrate/Extend/Swig$ dir
hellolib.i  makefile.hellolib-swig

.../PP4E/Integrate/Extend/Swig$ make -f makefile.hellolib-swig
/cygdrive/c/temp/swigwin-2.0.0/swig -python -I../HelloLib hellolib.i
gcc hellolib_wrap.c -g -I../HelloLib -I/usr/local/include/python3.1
                 -c -o hellolib_wrap.o
gcc ../HelloLib/hellolib.c -g -I../HelloLib -c -o ../HelloLib/hellolib.o
gcc -shared hellolib_wrap.o ../HelloLib/hellolib.o 
                -L/usr/local/bin -lpython3.1 -o _hellowrap.dll

.../PP4E/Integrate/Extend/Swig$ dir
_hellowrap.dll  hellolib_wrap.c  hellowrap.py
hellolib.i      hellolib_wrap.o  makefile.hellolib-swig

The result is a dynamically loaded C extension module file ready to be imported by Python code. Like all modules, _hellowrap.dll must, along with hellowrap.py, be placed in a directory on your Python module search path (the directory where you compile will suffice if you run Python there too). Notice that the .dll file must be built with a leading underscore in its name; this is required because SWIG also created the .py file of the same name without the underscore—if named the same, only one could be imported, and we need both (scripts import the .py which in turn imports the .dll internally).

As usual in C development, you may have to barter with the makefile to get it to work on your system. Once you’ve run the makefile, though, you are finished. The generated C module is used exactly like the manually coded version shown before, except that SWIG has taken care of the complicated parts automatically. Function calls in our Python code are routed through the generated SWIG layer, to the C code in Example 20-4, and back again; with SWIG, this all “just works”:

.../PP4E/Integrate/Extend/Swig$ python
 >>> import hellowrap                       # import glue + library file
>>> hellowrap.message('swig world')         # cwd always searched on imports
'Hello, swig world'

>>> hellowrap.__file__
'hellowrap.py'
>>> dir(hellowrap)
['__builtins__', '__doc__', '__file__', '__name__', '_hellowrap', ... 'message']

>>> hellowrap._hellowrap
<module '_hellowrap' from '_hellowrap.dll'>

In other words, once you learn how to use SWIG, you can often largely forget the details behind integration coding. In fact, SWIG is so adept at generating Python glue code that it’s usually easier and less error prone to code C extensions for Python as purely C- or C++-based libraries first, and later add them to Python by running their header files through SWIG, as demonstrated here.

We’ve mostly just scratched the SWIG surface here, and there’s more for you to learn about it from its Python-specific manual—available with SWIG at http://www.swig.org. Although its examples in this book are simple, SWIG is powerful enough to integrate libraries as complex as Windows extensions and commonly used graphics APIs such as OpenGL. We’ll apply it again later in this chapter, and explore its “shadow class” model for wrapping C++ classes too. For now, let’s move on to a more useful extension example.

As is, the C extension module exports a function-based interface, but it’s easy to wrap its functions in Python code that makes the interface look any way you like. For instance, Example 20-10 makes the functions accessible by dictionary indexing and integrates with the os.environ object—it guarantees that the object will stay in sync with fetches and changes made by calling our C extension functions.

import os
from cenviron import getenv, putenv       # get C module's methods

class EnvMapping:                         # wrap in a Python class
    def __setitem__(self, key, value):
        os.environ[key] = value           # on writes: Env[key]=value
        putenv(key, value)                # put in os.environ too

    def __getitem__(self, key):
        value = getenv(key)               # on reads: Env[key]
        os.environ[key] = value           # integrity check
        return value

Env = EnvMapping()                        # make one instance

To use this module, clients may import its Env object using Env['var'] dictionary syntax to refer to environment variables. Example 20-11 goes a step further and exports the functions as qualified attribute names rather than as calls or keys—variables are referenced with Env.var attribute syntax.

import os
from cenviron import getenv, putenv       # get C module's methods

class EnvWrapper:                         # wrap in a Python class
    def __setattr__(self, name, value):
        os.environ[name] = value          # on writes: Env.name=value
        putenv(name, value)               # put in os.environ too

    def __getattr__(self, name):
        value = getenv(name)              # on reads: Env.name
        os.environ[name] = value          # integrity check
        return value

Env = EnvWrapper()                        # make one instance

The following shows our Python wrappers running atop our C extension module’s functions to access environment variables. The main point to notice here is that you can graft many different sorts of interface models on top of extension functions by providing Python wrappers in addition to C extensions:

>>> from envmap import Env
>>> Env['USER']
'skipper'
>>> Env['USER'] = 'professor'
>>> Env['USER']
'professor'
>>>
>>> from envattr import Env
>>> Env.USER
'professor'
>>> Env.USER = 'gilligan'
>>> Env.USER
'gilligan'

You can manually code extension modules like we just did, but you don’t necessarily have to. Because this example really just wraps functions that already exist in standard C libraries, the entire cenviron.c C code file in Example 20-8 can be replaced with a simple SWIG input file that looks like Example 20-12.

/***************************************************************
 * Swig module description file, to generate all Python wrapper
 * code for C lib getenv/putenv calls: "swig -python environ.i".
 ***************************************************************/

%module environ

extern char * getenv(const char *varname);
extern int    putenv(char *assignment);

And you’re done. Well, almost; you still need to run this file through SWIG and compile its output. As before, simply add a SWIG step to your makefile and compile its output file into a shareable object for dynamic linking, and you’re in business. Example 20-13 is a Cygwin makefile that does the job.

# build environ extension from SWIG generated code

PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1
SWIG  = /cygdrive/c/temp/swigwin-2.0.0/swig

_environ.dll: environ_wrap.c
        gcc environ_wrap.c -g -I$(PYINC) -L$(PYLIB) -lpython3.1 -shared -o $@

environ_wrap.c: environ.i
        $(SWIG) -python environ.i

clean:
        rm -f *.o *.dll *.pyc core environ_wrap.c environ.py

When run on environ.i, SWIG generates two files and two modules—environ.py (the Python interface module we import) and environ_wrap.c (the lower-level glue code module file we compile into _environ.dll to be imported by the .py). Because the functions being wrapped here live in standard linked-in C libraries, there is nothing to combine with the generated code; this makefile simply runs SWIG and compiles the wrapper file into a C extension module, ready to be imported:

.../PP4E/Integrate/Extend/Swig/Environ$ make -f makefile.environ-swig
/cygdrive/c/temp/swigwin-2.0.0/swig -python environ.i
gcc environ_wrap.c -g -I/usr/local/include/python3.1 -L/usr/local/bin -lpython3.1
    -shared -o _environ.dll

And now you’re really done. The resulting C extension module is linked when imported, and it’s used as before (except that SWIG handled all the gory bits):

.../PP4E/Integrate/Extend/Swig/Environ$ ls
_environ.dll  environ.i  environ.py  environ_wrap.c  makefile.environ-swig

.../PP4E/Integrate/Extend/Swig/Environ$ python
>>> import environ
>>> environ.getenv('USER')
'gilligan'
>>> environ.__name__, environ.__file__, environ
('environ', 'environ.py', <module 'environ' from 'environ.py'>)
>>> dir(environ)
[ ... '_environ', 'getenv', 'putenv' ... ]

To see how this works, we need a C++ class. To illustrate, let’s code one to be used in Python scripts. You have to understand C++ to make sense of this section, of course, and SWIG supports advanced C++ tools (including templates and overloaded functions and operators), but I’ll keep this example simple for illustration. The following C++ files define a Number class with four methods (add, sub, square, and display), a data member (data), and a constructor and destructor. Example 20-14 shows the header file.

class Number
{
public:
    Number(int start);             // constructor
    ~Number();                     // destructor
    void add(int value);           // update data member
    void sub(int value);
    int  square();                 // return a value
    void display();                // print data member
    int data;
};

Example 20-15 is the C++ class’s implementation file; most methods print a message when called to trace class operations. Notice how this uses printf instead of C++’s cout; this once resolved an output overlap issue when mixing C++ cout with Python 2.X standard output streams on Cygwin. It’s probably a moot point today—because Python 3.X’s output system and buffering might mix with C++’s arbitrarily, C++ should generally flush the output stream (with fflush(stdout) or cout<<flush) if it prints intermixed text that doesn’t end in a newline. Obscure but true when disparate language systems are mixed.

///////////////////////////////////////////////////////////////
// implement a C++ class, to be used from Python code or not;
// caveat: cout and print usually both work, but I ran into a
// c++/py output overlap issue on Cygwin that prompted printf
///////////////////////////////////////////////////////////////

#include "number.h"
#include "stdio.h"                       // versus #include "iostream.h"

Number::Number(int start) {
    data = start;                        // python print goes to stdout
    printf("Number: %d
", data);        // or: cout << "Number: " << data << endl;
}

Number::~Number() {
    printf("~Number: %d
", data);
}

void Number::add(int value) {
    data += value;
    printf("add %d
", value);
}

void Number::sub(int value) {
    data -= value;
    printf("sub %d
", value);
}

int Number::square() {
    return data * data;       // if print label, fflush(stdout) or cout << flush
}

void Number::display() {
    printf("Number=%d
", data);
}

So that you can compare languages, the following is how this class is used in a C++ program. Example 20-16 makes a Number object, calls its methods, and fetches and sets its data attribute directly (C++ distinguishes between “members” and “methods,” while they’re usually both called “attributes” in Python).

#include "iostream.h"
#include "number.h"

main()
{
    Number *num;
    int res, val;

    num = new Number(1);            // make a C++ class instance
    num->add(4);                    // call its methods
    num->display();
    num->sub(2);
    num->display();

    res = num->square();                     // method return value
    cout << "square: " << res << endl;

    num->data = 99;                          // set C++ data member
    val = num->data;                         // fetch C++ data member
    cout << "data:   " << val << endl;
    cout << "data+1: " << val + 1 << endl;

    num->display();
    cout << num << endl;            // print raw instance ptr
    delete num;                     // run destructor
}

You can use the g++ command-line C++ compiler program to compile and run this code on Cygwin (it’s the same on Linux). If you don’t use a similar system, you’ll have to extrapolate; there are far too many C++ compiler differences to list here. Type the compile command directly or use the cxxtest target in this example directory’s makefile shown ahead, and then run the purely C++ program created:

.../PP4E/Integrate/Extend/Swig/Shadow$ make -f makefile.number-swig cxxtest
g++ main.cxx number.cxx -Wno-deprecated

.../PP4E/Integrate/Extend/Swig/Shadow$ ./a.exe
Number: 1
add 4
Number=5
sub 2
Number=3
square: 9
data:   99
data+1: 100
Number=99
0xe502c0
~Number: 99

But enough C++: let’s get back to Python. To use the C++ Number class of the preceding section in Python scripts, you need to code or generate a glue logic layer between the two languages, just as in prior C extension examples. To generate that layer automatically, write a SWIG input file like the one shown in Example 20-17.

/********************************************************
 * Swig module description file for wrapping a C++ class.
 * Generate by running "swig -c++ -python number.i".
 * The C++ module is generated in file number_wrap.cxx;
 * module 'number' refers to the number.py shadow class.
 ********************************************************/

%module number

%{
#include "number.h"
%}

%include number.h

This interface file simply directs SWIG to read the C++ class’s type signature information from the %-included number.h header file. SWIG uses the class declaration to generate two different Python modules again:

The former must be compiled into a binary library. The latter imports and uses the former’s compiled form and is the file that Python scripts ultimately import. As for simple functions, SWIG achieves the integration with a combination of Python and C++ code.

After running SWIG, the Cygwin makefile shown in Example 20-18 combines the generated number_wrap.cxx C++ wrapper code module with the C++ class implementation file to create a _number.dll—a dynamically loaded extension module that must be in a directory on your Python module search path when imported from a Python script, along with the generated number.py (all files are in the same current working directory here).

As before, the compiled C extension module must be named with a leading underscore in SWIG today: _number.dll, following a Python convention, rather than the other formats used by earlier releases. The shadow class module number.py internally imports _number.dll. Be sure to use a -c++ command-line argument for SWIG; an older -shadow argument is no longer needed to create the wrapper class in addition to the lower-level functional interface module, as this is enabled by default.

###########################################################################
# Use SWIG to integrate the number.h C++ class for use in Python programs.
# Update: name "_number.dll" matters, because shadow class imports _number.
# Update: the "-shadow" swig command line arg is deprecated (on by default).
# Update: swig no longer creates a .doc file to rm here (ancient history).
###########################################################################

PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1
SWIG  = /cygdrive/c/temp/swigwin-2.0.0/swig

all: _number.dll number.py

# wrapper + real class
_number.dll: number_wrap.o number.o
        g++ -shared number_wrap.o number.o -L$(PYLIB) -lpython3.1 -o $@

# generated class wrapper module(s)
number_wrap.o: number_wrap.cxx number.h
        g++ number_wrap.cxx -c -g -I$(PYINC)

number_wrap.cxx: number.i
        $(SWIG) -c++ -python number.i

number.py: number.i
        $(SWIG) -c++ -python number.i

# wrapped C++ class code
number.o: number.cxx number.h
        g++ number.cxx -c -g -Wno-deprecated

# non Python test
cxxtest:
        g++ main.cxx number.cxx -Wno-deprecated

clean:
        rm -f *.pyc *.o *.dll core a.exe
force:
        rm -f *.pyc *.o *.dll core a.exe number_wrap.cxx number.py

As usual, run this makefile to generate and compile the necessary glue code into an extension module that can be imported by Python programs:

.../PP4E/Integrate/Extend/Swig/Shadow$ make -f makefile.number-swig
/cygdrive/c/temp/swigwin-2.0.0/swig -c++ -python number.i
g++ number_wrap.cxx -c -g -I/usr/local/include/python3.1
g++ number.cxx -c -g -Wno-deprecated
g++ -shared number_wrap.o number.o -L/usr/local/bin -lpython3.1 -o _number.dll

.../PP4E/Integrate/Extend/Swig/Shadow$ ls
_number.dll  makefile.number-swig  number.i   number_wrap.cxx
a.exe        number.cxx            number.o   number_wrap.o
main.cxx     number.h              number.py

Once the glue code is generated and compiled, Python scripts can access the C++ class as though it were coded in Python. In fact, it is—the imported number.py shadow class which runs on top of the extension module is generated Python code. Example 20-19 repeats the main.cxx file’s class tests. Here, though, the C++ class is being utilized from the Python programming language—an arguably amazing feat, but the code is remarkably natural on the Python side of the fence.

"""
use C++ class in Python code (c++ module + py shadow class)
this script runs the same tests as the main.cxx C++ file
"""

from number import Number         # imports .py C++ shadow class module

num = Number(1)                   # make a C++ class object in Python
num.add(4)                        # call its methods from Python
num.display()                     # num saves the C++ 'this' pointer
num.sub(2)
num.display()

res = num.square()                # converted C++ int return value
print('square: ', res)

num.data = 99                     # set C++ data member, generated __setattr__
val = num.data                    # get C++ data member, generated __getattr__
print('data:   ', val)            # returns a normal Python integer object
print('data+1: ', val + 1)

num.display()
print(num)                        # runs repr in shadow/proxy class
del num                           # runs C++ destructor automatically

Because the C++ class and its wrappers are automatically loaded when imported by the number.py shadow class module, you run this script like any other:

.../PP4E/Integrate/Extend/Swig/Shadow$ python main.py
Number: 1
add 4
Number=5
sub 2
Number=3
square:  9
data:    99
data+1:  100
Number=99
<number.Number; proxy of <Swig Object of type 'Number *' at 0x7ff4bb48> >
~Number: 99

Much of this output is coming from the C++ class’s methods and is largely the same as the main.cxx results shown in Example 20-16 (less the instance output format—it’s a Python shadow class instance now).

"""
run similar tests to main.cxx and main.py
but use low-level C accessor function interface
"""

from _number import *           # c++ extension module wrapper

num = new_Number(1)
Number_add(num, 4)              # pass C++ 'this' pointer explicitly
Number_display(num)             # use accessor functions in the C module
Number_sub(num, 2)
Number_display(num)
print(Number_square(num))

Number_data_set(num, 99)
print(Number_data_get(num))
Number_display(num)
print(num)
delete_Number(num)

.../PP4E/Integrate/Extend/Swig/Shadow$ python main_low.py
Number: 1
add 4
Number=5
sub 2
Number=3
9
99
Number=99
_6025aa00_p_Number
~Number: 99

"sublass C++ class in Python (generated shadow class)"

from number import Number                # import shadow class

class MyNumber(Number):
    def add(self, other):                # extend method
        print('in Python add...')
        Number.add(self, other)
    def mul(self, other):                # add new method
        print('in Python mul...')
        self.data = self.data * other

num = MyNumber(1)               # same tests as main.cxx, main.py
num.add(4)                      # using Python subclass of shadow class
num.display()                   # add() is specialized in Python
num.sub(2)
num.display()
print(num.square())

num.data = 99
print(num.data)
num.display()

num.mul(2)                      # mul() is implemented in Python
num.display()
print(num)                      # repr from shadow superclass
del num

.../PP4E/Integrate/Extend/Swig/Shadow$ python main_subclass.py
Number: 1
in Python add...
add 4
Number=5
sub 2
Number=3
9
99
Number=99
in Python mul...
Number=198
<__main__.MyNumber; proxy of <Swig Object of type 'Number *' at 0x7ff4baa0> >
~Number: 198

.../PP4E/Integrate/Extend/Swig/Shadow$ python
>>> import _number
>>> _number.__file__              # the C++ class plus generated glue module
'_number.dll'
>>> import number                 # the generated Python shadow class module
>>> number.__file__
'number.py'

>>> x = number.Number(2)          # make a C++ class instance in Python
Number: 2
>>> y = number.Number(4)          # make another C++ object
Number: 4
>>> x, y
(<number.Number; proxy of <Swig Object of type 'Number *' at 0x7ff4bcf8> >,
 <number.Number; proxy of <Swig Object of type 'Number *' at 0x7ff4b998> >)

>>> x.display()                   # call C++ method (like C++ x->display())
Number=2
>>> x.add(y.data)                 # fetch C++ data member, call C++ method
add 4
>>> x.display()
Number=6

>>> y.data = x.data + y.data + 32         # set C++ data member
>>> y.display()                           # y records the C++ this pointer
Number=42

>>> y.square()                            # method with return value
1764
>>> t = y.square()
>>> t, type(t)                            # type is class in Python 3.X
(1764, <class 'int'>)

The first thing you should know about Python’s embedded-call API is that it is less structured than the extension interfaces. Embedding Python in C may require a bit more creativity on your part than extending: you must pick tools from a general collection of calls to implement the Python integration instead of coding to a boilerplate structure. The upside of this loose structure is that programs can combine embedding calls and strategies to build up arbitrary integration architectures.

The lack of a more rigid model for embedding is largely the result of a less clear-cut goal. When extending Python, there is a distinct separation for Python and C responsibilities and a clear structure for the integration. C modules and types are required to fit the Python module/type model by conforming to standard extension structures. This makes the integration seamless for Python clients: C extensions look like Python objects and handle most of the work. It also supports automation tools such as SWIG.

But when Python is embedded, the structure isn’t as obvious; because C is the enclosing level, there is no clear way to know what model the embedded Python code should fit. C may want to run objects fetched from modules, strings fetched from files or parsed out of documents, and so on. Instead of deciding what C can and cannot do, Python provides a collection of general embedding interface tools, which you use and structure according to your embedding goals.

Most of these tools correspond to tools available to Python programs. Table 20-1 lists some of the more common API calls used for embedding, as well as their Python equivalents. In general, if you can figure out how to accomplish your embedding goals in pure Python code, you can probably find C API tools that achieve the same results.

Because embedding relies on API call selection, becoming familiar with the Python C API is fundamental to the embedding task. This chapter presents a handful of representative embedding examples and discusses common API calls, but it does not provide a comprehensive list of all tools in the API. Once you’ve mastered the examples here, you’ll probably need to consult Python’s integration manuals for more details on available calls in this domain. As mentioned previously, Python offers two standard manuals for C/C++ integration programmers: Extending and Embedding, an integration tutorial; and Python/C API, the Python runtime library reference.

You can find the most recent releases of these manuals at http://www.python.org, and possibly installed on your computer alongside Python itself. Beyond this chapter, these manuals are likely to be your best resource for up-to-date and complete Python API tool information.

Before we jump into details, let’s get a handle on some of the core ideas in the embedding domain. When this book speaks of “embedded” Python code, it simply means any Python program structure that can be executed from C with a direct in-process function call interface. Generally speaking, embedded Python code can take a variety of forms:

The Python binary library is usually what is physically embedded and linked in the C program. The actual Python code run from C can come from a wide variety of sources:

Registration is a technique commonly used in callback scenarios that we will explore in more detail later in this chapter. But especially for strings of code, there are as many possible sources as there are for C character strings in general. For example, C programs can construct arbitrary Python code dynamically by building and running strings.

Finally, once you have some Python code to run, you need a way to communicate with it: the Python code may need to use inputs passed in from the C layer and may want to generate outputs to communicate results back to C. In fact, embedding generally becomes interesting only when the embedded code has access to the enclosing C layer. Usually, the form of the embedded code suggests its communication media:

Naturally, all embedded code forms can also communicate with C using general system-level tools: files, sockets, pipes, and so on. These techniques are generally less direct and slower, though. Here, we are still interested in in-process function call integration.

Perhaps the simplest way to run Python code from C is by calling the PyRun_SimpleString API function. With it, C programs can execute Python programs represented as C character string arrays. This call is also very limited: all code runs in the same namespace (the module __main__), the code strings must be Python statements (not expressions), and there is no direct way to communicate inputs or outputs with the Python code run.

Still, it’s a simple place to start. Moreover, when augmented with an imported C extension module that the embedded Python code can use to communicate with the enclosing C layer, this technique can satisfy many embedding goals. To demonstrate the basics, the C program in Example 20-23 runs Python code to accomplish the same results as the Python interactive session listed in the prior section.

/*******************************************************
 * simple code strings: C acts like the interactive
 * prompt, code runs in __main__, no output sent to C;
 *******************************************************/

#include <Python.h>    /* standard API def */

main() {
    printf("embed-simple
");
    Py_Initialize();
    PyRun_SimpleString("import usermod");                /* load .py file */
    PyRun_SimpleString("print(usermod.message)");        /* on Python path */
    PyRun_SimpleString("x = usermod.message");           /* compile and run */
    PyRun_SimpleString("print(usermod.transform(x))");
    Py_Finalize();
}

The first thing you should notice here is that when Python is embedded, C programs always call Py_Initialize to initialize linked-in Python libraries before using any other API functions and normally call Py_Finalize to shut the interpreter down.

The rest of this code is straightforward—C submits hardcoded strings to Python that are roughly what we typed interactively. In fact, we could concatenate all the Python code strings here with characters between, and submit it once as a single string. Internally, PyRun_SimpleString invokes the Python compiler and interpreter to run the strings sent from C; as usual, the Python compiler is always available in systems that contain Python.

# a Cygwin makefile that builds a C executable that embeds
# Python, assuming no external module libs must be linked in;
# uses Python header files, links in the Python lib file;
# both may be in other dirs (e.g., /usr) in your install;

PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1

embed-simple: embed-simple.o
        gcc embed-simple.o -L$(PYLIB) -lpython3.1 -g -o embed-simple

embed-simple.o: embed-simple.c
        gcc embed-simple.c -c -g -I$(PYINC)

.../PP4E/Integrate/Embed/Basics$ make -f makefile.1
gcc embed-simple.c -c -g -I/usr/local/include/python3.1
gcc embed-simple.o -L/usr/local/bin -lpython3.1 -g -o embed-simple

# cygwin makefile to build all 5 basic embedding examples at once

PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1

BASICS = embed-simple.exe   
         embed-string.exe   
         embed-object.exe   
         embed-dict.exe     
         embed-bytecode.exe

all:    $(BASICS)

embed%.exe: embed%.o
        gcc embed$*.o -L$(PYLIB) -lpython3.1 -g -o $@

embed%.o: embed%.c
        gcc embed$*.c -c -g -I$(PYINC)

clean:
        rm -f *.o *.pyc $(BASICS) core

.../PP4E/Integrate/Embed/Basics$ make -f makefile.basics clean
rm -f *.o *.pyc embed-simple.exe embed-string.exe embed-object.exe
embed-dict.exe embed-bytecode.exe core

.../PP4E/Integrate/Embed/Basics$ make -f makefile.basics
gcc embed-simple.c -c -g -I/usr/local/include/python3.1
gcc embed-simple.o -L/usr/local/bin -lpython3.1 -g -o embed-simple.exe
gcc embed-string.c -c -g -I/usr/local/include/python3.1
gcc embed-string.o -L/usr/local/bin -lpython3.1 -g -o embed-string.exe
gcc embed-object.c -c -g -I/usr/local/include/python3.1
gcc embed-object.o -L/usr/local/bin -lpython3.1 -g -o embed-object.exe
gcc embed-dict.c -c -g -I/usr/local/include/python3.1
gcc embed-dict.o -L/usr/local/bin -lpython3.1 -g -o embed-dict.exe
gcc embed-bytecode.c -c -g -I/usr/local/include/python3.1
gcc embed-bytecode.o -L/usr/local/bin -lpython3.1 -g -o embed-bytecode.exe
rm embed-dict.o embed-object.o embed-simple.o embed-bytecode.o embed-string.o

.../PP4E/Integrate/Embed/Basics$ ./embed-simple
embed-simple
The meaning of life...
THE MEANING OF PYTHON...

Example 20-26 uses the following API calls to run code strings that return expression results back to C:

The import calls are used to fetch the namespace of the usermod module listed in Example 20-22 so that code strings can be run there directly and will have access to names defined in that module without qualifications. Py_Import_ImportModule is like a Python import statement, but the imported module object is returned to C; it is not assigned to a Python variable name. As a result, it’s probably more similar to the Python __import__ built-in function.

The PyRun_String call is the one that actually runs code here, though. It takes a code string, a parser mode flag, and dictionary object pointers to serve as the global and local namespaces for running the code string. The mode flag can be Py_eval_input to run an expression or Py_file_input to run a statement; when running an expression, the result of evaluating the expression is returned from this call (it comes back as a PyObject* object pointer). The two namespace dictionary pointer arguments allow you to distinguish global and local scopes, but they are typically passed the same dictionary such that code runs in a single namespace.

/* code-strings with results and namespaces */

#include <Python.h>

main() {
    char *cstr;
    PyObject *pstr, *pmod, *pdict;
    printf("embed-string
");
    Py_Initialize();

    /* get usermod.message */
    pmod  = PyImport_ImportModule("usermod");
    pdict = PyModule_GetDict(pmod);
    pstr  = PyRun_String("message", Py_eval_input, pdict, pdict);

    /* convert to C */
    PyArg_Parse(pstr, "s", &cstr);
    printf("%s
", cstr);

    /* assign usermod.X */
    PyObject_SetAttrString(pmod, "X", pstr);

    /* print usermod.transform(X) */
    (void) PyRun_String("print(transform(X))", Py_file_input, pdict, pdict);
    Py_DECREF(pmod);
    Py_DECREF(pstr);
    Py_Finalize();
}

When compiled and run, this file produces the same result as its predecessor:

.../PP4E/Integrate/Embed/Basics$ ./embed-string
embed-string
The meaning of life...
THE MEANING OF PYTHON...

However, very different work goes into producing this output. This time, C fetches, converts, and prints the value of the Python module’s message attribute directly by running a string expression and assigning a global variable (X) within the module’s namespace to serve as input for a Python print statement string.

Because the string execution call in this version lets you specify namespaces, you can better partition the embedded code your system runs—each grouping can have a distinct namespace to avoid overwriting other groups’ variables. And because this call returns a result, you can better communicate with the embedded code; expression results are outputs, and assignments to globals in the namespace in which code runs can serve as inputs.

Before we move on, I need to explain three coding issues here. First, this program also decrements the reference count on objects passed to it from Python, using the Py_DECREF call described in Python’s C API manuals. These calls are not strictly needed here (the objects’ space is reclaimed when the programs exits anyhow), but they demonstrate how embedding interfaces must manage reference counts when Python passes object ownership to C. If this was a function called from a larger system, for instance, you would generally want to decrement the count to allow Python to reclaim the objects.

Second, in a realistic program, you should generally test the return values of all the API calls in this program immediately to detect errors (e.g., import failure). Error tests are omitted in this section’s example to keep the code simple, but they should be included in your programs to make them more robust.

And third, there is a related function that lets you run entire files of code, but it is not demonstrated in this chapter: PyRun_File. Because you can always load a file’s text and run it as a single code string with PyRun_String, the PyRun_File call’s main advantage is to avoid allocating memory for file content. In such multiline code strings, the character terminates lines and indentation group blocks as usual.

The last two sections dealt with running strings of code, but it’s easy for C programs to deal in terms of Python objects, too. Example 20-27 accomplishes the same task as Examples 20-23 and 20-26, but it uses other API tools to interact with objects in the Python module directly:

We used both of the data conversion functions earlier in this chapter in extension modules. The PyEval_CallObject call in this version of the example is the key point here: it runs the imported function with a tuple of arguments, much like the Python func(*args) call syntax. The Python function’s return value comes back to C as a PyObject*, a generic Python object pointer.

/* fetch and call objects in modules */

#include <Python.h>

main() {
    char *cstr;
    PyObject *pstr, *pmod, *pfunc, *pargs;
    printf("embed-object
");
    Py_Initialize();

    /* get usermod.message */
    pmod = PyImport_ImportModule("usermod");
    pstr = PyObject_GetAttrString(pmod, "message");

    /* convert string to C */
    PyArg_Parse(pstr, "s", &cstr);
    printf("%s
", cstr);
    Py_DECREF(pstr);

    /* call usermod.transform(usermod.message) */
    pfunc = PyObject_GetAttrString(pmod, "transform");
    pargs = Py_BuildValue("(s)", cstr);
    pstr  = PyEval_CallObject(pfunc, pargs);
    PyArg_Parse(pstr, "s", &cstr);
    printf("%s
", cstr);

    /* free owned objects */
    Py_DECREF(pmod);
    Py_DECREF(pstr);
    Py_DECREF(pfunc);        /* not really needed in main() */
    Py_DECREF(pargs);        /* since all memory goes away  */
    Py_Finalize();
}

When compiled and run, the result is the same again:

.../PP4E/Integrate/Embed/Basics$ ./embed-object
embed-object
The meaning of life...
THE MEANING OF PYTHON...

However, this output is generated by C this time—first, by fetching the Python module’s message attribute value, and then by fetching and calling the module’s transform function object directly and printing its return value that is sent back to C. Input to the transform function is a function argument here, not a preset global variable. Notice that message is fetched as a module attribute this time, instead of by running its name as a code string; as this shows, there is often more than one way to accomplish the same goals with different API calls.

Running functions in modules like this is a simple way to structure embedding; code in the module file can be changed arbitrarily without having to recompile the C program that runs it. It also provides a direct communication model: inputs and outputs to Python code can take the form of function arguments and return values.

When we used PyRun_String earlier to run expressions with results, code was executed in the namespace of an existing Python module. Sometimes, though, it’s more convenient to create a brand-new namespace for running code strings that is independent of any existing module files. The C file in Example 20-28 shows how; the new namespace is created as a new Python dictionary object, and a handful of new API calls are employed in the process:

The main trick here is the new dictionary. Inputs and outputs for the embedded code strings are mapped to this dictionary by passing it as the code’s namespace dictionaries in the PyRun_String call. The net effect is that the C program in Example 20-28 works just like this Python code:

 >>> d = {}
 >>> d['Y'] = 2
 >>> exec('X = 99', d, d)
 >>> exec('X = X + Y', d, d)
 >>> print(d['X'])
 101

But here, each Python operation is replaced by a C API call.

/* make a new dictionary for code string namespace */

#include <Python.h>

main() {
    int cval;
    PyObject *pdict, *pval;
    printf("embed-dict
");
    Py_Initialize();

    /* make a new namespace */
    pdict = PyDict_New();
    PyDict_SetItemString(pdict, "__builtins__", PyEval_GetBuiltins());

    PyDict_SetItemString(pdict, "Y", PyLong_FromLong(2));  /* dict['Y'] = 2   */
    PyRun_String("X = 99",  Py_file_input, pdict, pdict);  /* run statements  */
    PyRun_String("X = X+Y", Py_file_input, pdict, pdict);  /* same X and Y    */
    pval = PyDict_GetItemString(pdict, "X");               /* fetch dict['X'] */

    PyArg_Parse(pval, "i", &cval);                         /* convert to C */
    printf("%d
", cval);                                  /* result=101 */
    Py_DECREF(pdict);
    Py_Finalize();
}

When compiled and run, this C program creates this sort of output, tailored for this use case:

.../PP4E/Integrate/Embed/Basics$ ./embed-dict
embed-dict
101

The output is different this time: it reflects the value of the Python variable X assigned by the embedded Python code strings and fetched by C. In general, C can fetch module attributes either by calling PyObject_GetAttrString with the module or by using PyDict_GetItemString to index the module’s attribute dictionary (expression strings work, too, but they are less direct). Here, there is no module at all, so dictionary indexing is used to access the code’s namespace in C.

Besides allowing you to partition code string namespaces independent of any Python module files on the underlying system, this scheme provides a natural communication mechanism. Values that are stored in the new dictionary before code is run serve as inputs, and names assigned by the embedded code can later be fetched out of the dictionary to serve as code outputs. For instance, the variable Y in the second string run refers to a name set to 2 by C; X is assigned by the Python code and fetched later by C code as the printed result.

There is one subtlety in this embedding mode: dictionaries that serve as namespaces for running code are generally required to have a __builtins__ link to the built-in scope searched last for name lookups, set with code of this form:

PyDict_SetItemString(pdict, "__builtins__", PyEval_GetBuiltins());

This is esoteric, and it is normally handled by Python internally for modules and built-ins like the exec function. For raw dictionaries used as namespaces, though, we are responsible for setting the link manually if we expect to reference built-in names. This still holds true in Python 3.X.

Finally, when you call Python function objects from C, you are actually running the already compiled bytecode associated with the object (e.g., a function body), normally created when the enclosing module is imported. When running strings, Python must compile the string before running it. Because compilation is a slow process, this can be a substantial overhead if you run a code string more than once. Instead, precompile the string to a bytecode object to be run later, using the API calls illustrated in Example 20-29:

The first of these takes the mode flag that is normally passed to PyRun_String, as well as a second string argument that is used only in error messages. The second takes two namespace dictionaries. These two API calls are used in Example 20-29 to compile and execute three strings of Python code in turn.

/* precompile code strings to bytecode objects */

#include <Python.h>
#include <compile.h>
#include <eval.h>

main() {
    int i;
    char *cval;
    PyObject *pcode1, *pcode2, *pcode3, *presult, *pdict;
    char *codestr1, *codestr2, *codestr3;
    printf("embed-bytecode
");

    Py_Initialize();
    codestr1 = "import usermod
print(usermod.message)";    /* statements */
    codestr2 = "usermod.transform(usermod.message)";        /* expression */
    codestr3 = "print('%d:%d' % (X, X ** 2), end=' ')";     /* use input X */

    /* make new namespace dictionary */
    pdict = PyDict_New();
    if (pdict == NULL) return −1;
    PyDict_SetItemString(pdict, "__builtins__", PyEval_GetBuiltins());

    /* precompile strings of code to bytecode objects */
    pcode1 = Py_CompileString(codestr1, "<embed>", Py_file_input);
    pcode2 = Py_CompileString(codestr2, "<embed>", Py_eval_input);
    pcode3 = Py_CompileString(codestr3, "<embed>", Py_file_input);

    /* run compiled bytecode in namespace dict */
    if (pcode1 && pcode2 && pcode3) {
        (void)    PyEval_EvalCode((PyCodeObject *)pcode1, pdict, pdict);
        presult = PyEval_EvalCode((PyCodeObject *)pcode2, pdict, pdict);
        PyArg_Parse(presult, "s", &cval);
        printf("%s
", cval);
        Py_DECREF(presult);

        /* rerun code object repeatedly */
        for (i = 0; i <= 10; i++) {
            PyDict_SetItemString(pdict, "X", PyLong_FromLong(i));
            (void) PyEval_EvalCode((PyCodeObject *)pcode3, pdict, pdict);
        }
        printf("
");
    }
    /* free referenced objects */
    Py_XDECREF(pdict);
    Py_XDECREF(pcode1);
    Py_XDECREF(pcode2);
    Py_XDECREF(pcode3);
    Py_Finalize();
}

This program combines a variety of techniques that we’ve already seen. The namespace in which the compiled code strings run, for instance, is a newly created dictionary (not an existing module object), and inputs for code strings are passed as preset variables in the namespace. When built and executed, the first part of the output is similar to previous examples in this section, but the last line represents running the same precompiled code string 11 times:

.../PP4E/Integrate/Embed/Basics$ embed-bytecode
embed-bytecode
The meaning of life...
THE MEANING OF PYTHON...

0:0 1:1 2:4 3:9 4:16 5:25 6:36 7:49 8:64 9:81 10:100

If your system executes Python code strings multiple times, it is a major speedup to precompile to bytecode in this fashion. This step is not required in other contexts that invoke callable Python objects—including the common embedding use case presented in the next section.

This section’s C and Python files demonstrate the coding techniques used to implement explicitly registered callback handlers. First, the C file in Example 20-30 implements interfaces for registering Python handlers, as well as code to run those handlers in response to later events:

In other words, this example uses both the embedding and the extending interfaces we’ve already met to register and invoke Python event handler code. Study Example 20-30 for more on its operation.

#include <Python.h>
#include <stdlib.h>

/***********************************************/
/* 1) code to route events to Python object    */
/* note that we could run strings here instead */
/***********************************************/

static PyObject *Handler = NULL;     /* keep Python object in C */

void Route_Event(char *label, int count)
{
    char *cres;
    PyObject *args, *pres;

    /* call Python handler */
    args = Py_BuildValue("(si)", label, count);   /* make arg-list */
    pres = PyEval_CallObject(Handler, args);      /* apply: run a call */
    Py_DECREF(args);                              /* add error checks */

    if (pres != NULL) {
        /* use and decref handler result */
        PyArg_Parse(pres, "s", &cres);
        printf("%s
", cres);
        Py_DECREF(pres);
    }
}

/*****************************************************/
/* 2) python extension module to register handlers   */
/* python imports this module to set handler objects */
/*****************************************************/

static PyObject *
Register_Handler(PyObject *self, PyObject *args)
{
    /* save Python callable object */
    Py_XDECREF(Handler);                 /* called before? */
    PyArg_Parse(args, "(O)", &Handler);  /* one argument */
    Py_XINCREF(Handler);                 /* add a reference */
    Py_INCREF(Py_None);                  /* return 'None': success */
    return Py_None;
}

static PyObject *
Trigger_Event(PyObject *self, PyObject *args)
{
    /* let Python simulate event caught by C */
    static count = 0;
    Route_Event("spam", count++);
    Py_INCREF(Py_None);
    return Py_None;
}

static PyMethodDef cregister_methods[] = {
    {"setHandler",    Register_Handler, METH_VARARGS, ""},  /* name, &func,... */
    {"triggerEvent",  Trigger_Event,    METH_VARARGS, ""},
    {NULL, NULL, 0, NULL}                                   /* end of table */
};

static struct PyModuleDef cregistermodule = {
   PyModuleDef_HEAD_INIT,
   "cregister",       /* name of module */
   "cregister mod",   /* module documentation, may be NULL */
   −1,                /* size of per-interpreter module state, −1=in global vars */
   cregister_methods  /* link to methods table */
};

PyMODINIT_FUNC
PyInit_cregister()                      /* called on first import */
{
    return PyModule_Create(&cregistermodule);
}

Ultimately, this C file is an extension module for Python, not a standalone C program that embeds Python (though C could just as well be on top). To compile it into a dynamically loaded module file, run the makefile in Example 20-31 on Cygwin (and use something similar on other platforms). As we learned earlier in this chapter, the resulting cregister.dll file will be loaded when first imported by a Python script if it is placed in a directory on Python’s module search path (e.g., in . or PYTHONPATH settings).

######################################################################
# Cygwin makefile that builds cregister.dll. a dynamically loaded
# C extension module (shareable), which is imported by register.py
######################################################################

PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1

CMODS = cregister.dll

all: $(CMODS)

cregister.dll: cregister.c
        gcc cregister.c -g -I$(PYINC) -shared -L$(PYLIB) -lpython3.1 -o $@

clean:
        rm -f *.pyc $(CMODS)

Now that we have a C extension module set to register and dispatch Python handlers, all we need are some Python handlers. The Python module shown in Example 20-32 defines two callback handler functions and imports the C extension module to register handlers and trigger events.

"""
#########################################################################
in Python, register for and handle event callbacks from the C language;
compile and link the C code, and launch this with 'python register.py'
#########################################################################
"""

####################################
# C calls these Python functions;
# handle an event, return a result
####################################

def callback1(label, count):
    return 'callback1 => %s number %i' % (label, count)

def callback2(label, count):
    return 'callback2 => ' +  label * count

#######################################
# Python calls a C extension module
# to register handlers, trigger events
#######################################

import cregister

print('
Test1:')
cregister.setHandler(callback1)      # register callback function
for i in range(3):
    cregister.triggerEvent()         # simulate events caught by C layer

print('
Test2:')
cregister.setHandler(callback2)
for i in range(3):
    cregister.triggerEvent()         # routes these events to callback2

That’s it—the Python/C callback integration is set to go. To kick off the system, run the Python script; it registers one handler function, forces three events to be triggered, and then changes the event handler and does it again:

.../PP4E/Integrate/Embed/Regist$ make -f makefile.regist
gcc cregister.c -g -I/usr/local/include/python3.1 -shared -L/usr/local/bin
-lpython3.1 -o cregister.dll

.../PP4E/Integrate/Embed/Regist$ python register.py

Test1:
callback1 => spam number 0
callback1 => spam number 1
callback1 => spam number 2

Test2:
callback2 => spamspamspam
callback2 => spamspamspamspam
callback2 => spamspamspamspamspam

This output is printed by the C event router function, but its content is the return values of the handler functions in the Python module. Actually, something pretty wild is going on under the hood. When Python forces an event to trigger, control flows between languages like this:

That is, we jump from Python to C to Python and back again. Along the way, control passes through both extending and embedding interfaces. When the Python callback handler is running, two Python levels are active, and one C level in the middle. Luckily, this just works; Python’s API is reentrant, so you don’t need to be concerned about having multiple Python interpreter levels active at the same time. Each level runs different code and operates independently.

Trace through this example’s output and code for more illumination. Here, we’re moving on to the last quick example we have time and space to explore—in the name of symmetry, using Python classes from C.