The internal layout of an extension source module typically follows a pattern like the one shown in Figure 5-2.

Figure 5-2. Python extension module internal layout

The primary sections shown in Figure 5-2 are described in Table 5-1.

The skeleton example shown in Figure 5-2 utilizes the following types from the Python API:

It also uses the following API functions:

The method table defines the entry points for accessing the internal functions in the extension, and it deserves a bit more explanation. The full syntax of an entry in the table is as follows:

{method_name, method_function, method_flags, method_docstring}

The fields are defined in detail in Table 5-2.

The method_flags entry is a bit field. Its value depends on what flags are specified, and each flag is defined to be a value that is the equivalent of setting a unique bit within an integer variable. Although this implies that the flags can be combined without overwriting each other, in actuality only the calling convention flags METH_VARARGS and METH_KEYWORDS can be used at the same time, although they can be combined with a binding flag. For a detailed discussion of the method flags, refer to the Python documentation (you can view it online here: http://docs.python.org/c-api/structures.html).

We will be using only a few of the available method flags. The rest are more useful when dealing with complex interfaces that we won’t be using. Table 5-3 lists the three most common method flags. In reality, if you are writing your own extension it is usually possible to design the extension functions such that the more esoteric flags are not necessary.

static PyObject *ex_varargs(PyObject *self, PyObject *args)
{
    int param1;
    double param2;
    char param3[80];

    PyArg_ParseTuple(args, "ids", &dev_handle, &param2, param3);

    return PyString_FromFormat("Received %d, %f, %s" % (param1, param2, param3));
}

int PyArg_ParseTupleAndKeywords(PyObject *arg, PyObject *kwdict,
                                char *format, char **kwlist, ...);

static PyObject *ex_keywords(PyObject *self, PyObject *args, PyObject *kw)
{
    int param1;
    double param2;
    char param3[80];

    static char *kwlist[] = {"param1", "param2", "param3", NULL}

    if (!PyArg_ParseTupleAndKeywords(*args, *kw, "ids", kwlist,
                                     &dev_handle, &param2, param3);
        return NULL;

    return PyString_FromFormat("Received %d, %f, %s" % (param1, param2, param3));
}

static PyObject *ex_keywords(PyObject *self, PyObject *noargs)
{
    return PyString_FromFormat("No arguments received or expected");
}

Passing integers between an application program and a wrapper is fine, but what if we want to work with things like strings, floating-point values, lists, or other Python objects? The methods PyArg_ParseTuple(), PyArg_ParseTupleAndKeywords(), and Py_BuildValue() can handle the common Python data types, and there are also other methods available, such as PyArg_VaParse(), PyArg_Parse(), PyArg_UnpackTuple(), and Py_VaBuildValue(). Refer to Python’s C API documentation for details.

The key to PyArg_ParseTuple() and PyArg_ParseTupleAndKeywords() lies in understanding the various format codes that are used. Table 5-4 lists some of the more commonly used codes.

For example, if we had an extension function that expected a floating-point value as an input parameter, like, say, an analog output function, we could write something like this:

static PyObject *WriteAnalog(PyObject *self, PyObject *args)
{
    int     dev_handle;
    int     out_port;
    float   out_value;

    PyArg_ParseTuple(args, "iif", &dev_handle, &out_port, &out_pin, &out_value);
    rc = AIOWriteData(dev_handle, out_port, out_pin, out_value);

    return Py_BuildValue("i", rc);
}

In this example, the function AIOWriteData() would be supplied by the vendor’s DLL. In reality, analog I/O tends to be somewhat more complex than this, because there are often parameters for things like the output range (+/− V), the reference source, a conversion clock, and so on.

Let’s take a look at what an extension written in C for a hypothetical discrete digital I/O card would look like. First off, the device uses a DLL to access the hardware via the PCI bus, so we need to know what API functions the DLL exposes. These are defined in the header file supplied with the DLL. This is a very simple piece of hardware, so just eight basic functions are needed. Here is the hypothetical header file (PDev.h) we’ll be using:

/* PDev.h - API for a simple discrete digital I/O card */

typedef int dev_handle;

#define PDEV_OK  1
#define PDEV_ERR 0

/*
 * Open Device Channel
 *
 * Opens a channel to a particular I/O device. The device is specified
 * by passing its unit number, which is assigned to the device by a
 * setup utility. dev_handle is an int type.
 *
 * Returns:   If dev_handle is > 0 then handle is valid
 *            If dev_handle is = 0 then an error has occurred
 */
dev_handle PDevOpen(int unit_num);

/* Close Device Channel
 * Closes a channel to a particular I/O device. Once a channel is
 * closed it must be explicitly re-opened by using the PDevOpen API
 * function. Closing a channel does not reset the DIO configuration.
 *
 * Returns:   If return is 1 (true) then channel closed OK
 *            If return is = 0 then an error has occurred
 */
int PDevClose(dev_handle handle);

/*
 * Reset Device Configuration
 *
 * Forces the device to perform an internal reset. All
 * configuration is returned to the factory default
 * setting (All DIO is defined as inputs).
 *
 * Returns:   1 (true) if no error occurred
 *            0 (false) if error encountered
 */
int PDevCfgReset(dev_handle handle);

/*
 * Configure Device Discrete I/O
 *
 * Defines the pins to be assigned as outputs. By default all
 * pins are in the read mode initially, and they must be
 * specifically set to the write mode. All 16 I/O pins are
 * assigned at once. For any pin where the corresponding binary
 * value of the assignment parameter is 1, then the pin will
 * be an output.
 *
 * Returns:   1 (true) if no error occurred
 *            0 (false) if error encountered
 */
int PDevDIOCfg(dev_handle handle, int out_pins);

/*
 * Read Discrete Input Pin
 *
 * Reads the data present on a particular pin. The pin may be
 * either an input or an output. If the pin is an output the
 * value returned will be the value last assigned to the pin.
 *
 * Returns:    The input value for the specified pin, which
 *             will be either 0 or 1. An error is indicated
 *             by a return value of −1.
 */
int PDevDIOReadBit(dev_handle handle, int port, int pin);

/*
 * Read Discrete Input Port
 *
 * Reads the data present on an entire port and returns it as a
 * byte value. The individual pins may be either inputs or outputs.
 * If the pins have been defined as inputs then the value read
 * will be the value last assigned to the pins.
 *
 * Returns:   An integer value representing the input states
 *            of all pins in the specified port. Only the
 *            least significant byte of the return value is
 *            valid. An error is indicated by a return value
 *            of −1.
 */
int PDevDIOReadPort(dev_handle handle, int port);

/*
 * Write Discrete Output Pin
 *
 * Sets a particular pin of a specified port to a value of either
 * 0 (off) or 1 (on, typically +5V). The pin must be defined as
 * an output or the call will return an error code.
 *
 * Returns:   1 (true) if no error occurred
 *            0 (false) if error encountered
 */
int PDevDIOWriteBit(dev_handle handle, int port, int pin, int value);

/*
 * Write Discrete Output Port
 *
 * Sets all of the pins of a specified port to the unsigned value
 * passed in port_value parameter. Only the lower eight bits of the
 * parameter are used. If any pin in the port is configured as an
 * input then an error code will be returned.
 *
 * Returns:   1 (true) if no error occurred
 *            0 (false) if error encountered
 */
int PDevDIOWritePort(dev_handle handle, int port, int value);

Our hypothetical device has two 8-bit ports, and the pins (bits) of each port may be set to be either inputs or outputs. Pins may be read or written either individually or as a group of eight bits (a port).

Let’s create a simple wrapper for the device’s DLL. We won’t try to be clever here, although there are some things that could be implemented if we wanted to give the wrapper extended capabilities. I’ll talk about those a little later.

The first step is to determine what we will need to import, and define any global variables the wrapper will need:

#include    <Python.h>
#include    <stdio.h>
#include    <PDev.h>

Notice that in this example we don’t really need any static global variables.

So, now that we have the top of the file, let’s do the bottom. The second step is to create the initialization function, which is what Python will execute when the wrapper DLL (a .pyd file) is loaded, and define the function mapping table:

static PyMethodDef PDevapiMethods[] = {
    {"OpenDevice",      OpenDev,    METH_VARARGS,
                        "Open specific DIO device."},
    {"CloseDevice",     CloseDev,   METH_VARARGS,
                        "Close an open DIO device."},
    {"ConfigOutputs",   ConfigDev,  METH_VARARGS,
                        "Config DIO pins as outputs."},
    {"ConfigReset",     ConfigRst,  METH_VARARGS,
                        "Reset all DIO pins to input mode."},
    {"ReadInputPin",    ReadPin,    METH_VARARGS,
                        "Read a single specific pin."},
    {"ReadInputPort",   ReadPort,   METH_VARARGS,
                        "Read an entire 8-bit port."},
    {"WriteOutputPin",  WritePin,   METH_VARARGS,
                        "Write to a specific pin."},
    {"WriteOutputPort", WritePort,  METH_VARARGS,
                        "Write to an entire port."},
    {NULL, NULL}
};


/*******************************************************************************/
/*  initPDevapi
 *
 *  Initialize this module when loaded by Python. Instantiates the methods table.
 *
 *  No input parameters.
 *
 *  Returns nothing.
 *******************************************************************************/
void initPDevAPI(void)
{
    Py_InitModule("PDevapi", PDevapiMethods);

    dev_handle = −1;
}

There’s nothing fancy here; I’ve basically just mapped the functions in the API to wrapper functions on a one-to-one basis. This does mean, however, that the Python code that calls this module will need to be responsible for keeping track of the device ID and the device handle, at a minimum.

Lastly, we define the interface functions. Here’s the entire extension, which I’ve called PDevAPI.c:

#include    <Python.h>
#include    <stdio.h>
#include    <PDev.h>

static PyObject *OpenDev(PyObject *self, PyObject *args)
{
    int     dev_num;
    int     dev_handle;

    PyArg_ParseTuple(args, "i", &dev_num);
    dev_handle = PDevOpen(dev_num);

    return Py_BuildValue("i", dev_handle);
}

static PyObject *CloseDev(PyObject *self, PyObject *args)
{
    int     dev_handle;
    int     rc;

    PyArg_ParseTuple(args, "i", &dev_handle);
    rc = PDevClose(dev_handle);

    return Py_BuildValue("i", rc);
}

static PyObject *ConfigDev(PyObject *self, PyObject *args)
{
    int     dev_handle;
    char    cfg_str[32];
    int     rc;

    memset((char *) cfg_str, '', 32);    /* clear config string */

    PyArg_ParseTuple(args, "is", &dev_handle, cfg_str);
    rc = PDevDIOCfg(dev_handle, cfg_str);

    return Py_BuildValue("i", rc);
}

static PyObject *ConfigRst(PyObject *self, PyObject *args)
{
    int     dev_handle;
    int     rc;

    PyArg_ParseTuple(args, "i", &dev_handle);
    rc = PDevCfgReset(dev_handle);

    return Py_BuildValue("i", rc);
}

static PyObject *ReadPin(PyObject *self, PyObject *args)
{
    int     dev_handle;
    int     in_port;
    int     in_pin;
    int     in_value;

    PyArg_ParseTuple(args, "iii", &dev_handle, &in_port, &in_pin);
    in_value = PDevDIOReadBit(dev_handle, in_port, in_pin);

    return Py_BuildValue("i", in_value);
}

static PyObject *ReadPort(PyObject *self, PyObject *args)
{
    int     dev_handle;
    int     in_port;
    int     in_value;

    PyArg_ParseTuple(args, "iii", &dev_handle, &in_port);
    in_value = PDevDIOReadPort(dev_handle, in_port);

    return Py_BuildValue("i", in_value);
}

static PyObject *WritePin(PyObject *self, PyObject *args)
{
    int     dev_handle;
    int     out_port;
    int     out_pin;
    int     out_value;

    PyArg_ParseTuple(args, "iii", &dev_handle, &out_port, &out_pin, &out_value);
    rc = PDevDIOWriteBit(dev_handle, out_port, out_pin, out_value);

    return Py_BuildValue("i", rc);
}

static PyObject *WritePort(PyObject *self, PyObject *args)
    int     dev_handle;
    int     out_port;
    int     out_value;

    PyArg_ParseTuple(args, "iii", &dev_handle, &out_port, &out_value);
    rc = PDevDIOWritePort(dev_handle, out_port, out_value);

    return Py_BuildValue("i", rc);
}

static PyMethodDef PDevapiMethods[] = {
    {"OpenDevice",      OpenDev,    METH_VARARGS,
                        "Open specific DIO device."},
    {"CloseDevice",     CloseDev,   METH_VARARGS,
                        "Close an open DIO device."},
    {"ConfigOutputs",   ConfigDev,  METH_VARARGS,
                        "Config DIO pins as outputs."},
    {"ConfigReset",     ConfigRst,  METH_VARARGS,
                        "Reset all DIO pins to input mode."},
    {"ReadInputPin",    ReadPin,    METH_VARARGS,
                        "Read a single specific pin."},
    {"ReadInputPort",   ReadPort,   METH_VARARGS,
                        "Read an entire 8-bit port."},
    {"WriteOutputPin",  WritePin,   METH_VARARGS,
                        "Write to a specific pin."},
    {"WriteOutputPort", WritePort,  METH_VARARGS,
                        "Write to an entire port."},
    {NULL, NULL}
};

/*******************************************************************************/
/*  initPDevapi
 *
 *  Initialize this module when loaded by Python. Instantiates the methods table.
 *
 *  No input parameters.
 *
 *  Returns nothing.
 *******************************************************************************/
void initPDevAPI(void)
{
    Py_InitModule("PDevapi", PDevapiMethods);

And that’s it.

I should point out that this API uses integers and strings only as parameter values, and the functions return only integers. The extension code shown is just a translation wrapper between the device API and Python, nothing more. In many cases, however, that’s all a wrapper really needs to be.

In a real-life situation, you would probably want to create a Python module with functions for opening the device channel and saving the handle the API returns in a persistent variable object, writing to specific bits and then verifying that the bits changed state as expected, and even setting up an output pattern playback capability, among other things.

Let’s assume that we want to get the value of a particular input pin and write an 8-bit value to a port to set the status of some external hardware based on the true or false state of that particular input pin. Here’s a simple example:

import PDevAPI

PDdevID = 1
PDev = 0

# open device, check for error return
PDev_handle = PDevAPI.OpenDevice(PDevID)

if PDev_handle:
    # define port 0 as inputs, port 1 as outputs
    PDevAPI.ConfigOutputs(PDev_handle, 0xFF00)

    pinval = PDevAPI.ReadInputPin(PDev_handle, 0, 2)
    if pinval:
        portval = 49
    else:
        portval = 0
    PDevAPI.WriteOutputPort(PDev_handle, 1, 42)
    PDevAPI.CloseDevice(PDev_handle)
    return 1
else:
    print "Could not open device: %d" % PdevID
    return 0

The example starts by attempting to open the interface to the hardware. If it succeeds, the handle value contains some positive integer, which is equivalent to True in Python. Should it fail and return 0, we can treat it as False.

Next, we want to define the inputs and outputs. The call to PDev.ConfigOutputs does this. The value of 0xFF00 indicates that we want the uppermost 8 bits of the 16 bits available in the hardware to be outputs:

0xFF0016 = 11111111000000002

Now that we have a valid device handle and the I/O has been configured, we can read a specific bit and take action based on its returned value. In this case we will write the decimal value 42 to the output port if the input bit is true; otherwise, we write all 0 bits to the output port.

The number 42 is interesting in its own right, but in this case it is the decimal equivalent of 00101010₂. These might be control signals for a bank of heater elements, a set of valves, perhaps some lights, or just about anything else that could be controlled effectively in an on/off fashion.

Lastly, let’s look at some of things I didn’t put into this code.

For starters, there’s no error return checking for the ConfigOutputs(), ReadInputPin(), WriteOutputPort(), and CloseDevice() calls. I left this out just to keep things simple, but in reality you would probably want to include this, especially if whatever this interface is controlling could possibly do something that could cause damage in the real world. In fact, it’s not uncommon for high-reliability software to have as many, or even more, lines of code dedicated to error checking and fault handling than lines of code that deal directly with the control logic. It’s easy to set an output value, but it can take some effort to detect a problem after the value has been set, when something doesn’t work correctly. The software must then take action to try to recover from the fault, or at least put the system into a safe state to minimize any possible damage. In a later chapter I’ll show you how to do a simple fault analysis so you can get an idea of what might go wrong, and the possible actions the software could take to deal with the problem.

The example also doesn’t provide any support for dealing directly with bits. We could easily write a couple of utility functions that would allow us to enable or disable a specific bit without disturbing any other bits in the byte that makes up a port. A time-honored way to do this is called read-modify-write. Here’s how to set a particular bit:

def SetPortBit(PDev_handle, portnum, bitpos):
    # read the current port state
    portval = PDevAPI.ReadInputPort(PDev_handle, portnum)

    if portval >= 0:
        insval = 1 << bitpos
        # change the designated bit
        newval = portval | insval
        # write it back out
        rc = PDevAPI.WriteOutputPort(PDev_handle, portnum, newval)
    else:
        rc = 0
return rc

The shift operation generates a value that is a power of two (2ⁿ). So, a bit at position 2⁰ is 1, 2⁴ is 16, and so on. The resulting value, or mask, is then ORed into the value read from the port (most discrete digital I/O devices will allow you to read from any port or pin, even those designated as outputs), and the mask value is applied. It doesn’t matter what is already at the designated bit position, it will be set to 1. The rest of the bits in the mask value are 0s, so a 1 in the original data in portval stays 1, and a 0 stays 0. (Refer back to Chapter 3 for an overview of how Python’s bitwise operators work.)

Clearing a bit (setting it to 0) involves creating a mask with all the bits except the one we want to clear set to 1, and then using an AND instead of an OR. Here’s an example:

def ClearPortBit(PDev_handle, bitpos):
    # read the current port state
    portval = PDevAPI.ReadInputPort(PDev_handle, portnum)

    if portval >= 0:
        insval = 1 << bitpos
        newval = ~insval & portval
        rc = PDevAPI.WriteOutputPort(PDev_handle, portnum, newval)
    else:
        rc = 0
return rc

The function ClearPortBit() reads the current output port state, creates a bit value using a left shift, and then applies the one’s complement of the bit value to the current port value using Python’s bitwise AND operator. If bitpos is 2 and the resulting value of insval is 4, the bit at position 2² is 1, or 00000100 in binary. Inverting this results in 11111011, or 251. When this is ANDed with the port bit values in portval, any port bit that is a 1 will remain 1, 0s will remain 0s, and the bit we want to clear at 2² will always end up as 0.

You might be concerned about the output port glitching, or outputting a short spike when the new value is written back, but not to worry. A correctly designed discrete digital I/O port will not do this, so you can safely assume that the hardware will do what you expect. If you do happen run across some hardware that behaves this way, well, you might want to consider using something else instead. The reason you don’t have to worry is that almost every discrete digital I/O device made utilizes latches. The data is loaded into the latch, and the latch drives the output pins. The result is that any latched output that is already a 1 will remain a 1. (See Chapter 2 for a schematic of a latched discrete digital I/O port.)

You could, of course, write utility functions to handle bit manipulation in C and put them into the extension, rather than writing them in Python, as I’ve done. The resulting code would execute faster than native Python, but it might not be worth the trouble. It is often easier to keep the core low-level I/O functions in the wrapper module, and then create a module specifically to interface with the low-level wrapper. The Python interface module could also implement error checking and parameter verification, and provide additional utilities that aren’t speed-critical. In effect, the wrapper would have two parts: one part in C to interface with the hardware, and one part in Python to provide extended capabilities to the application that uses it. Figure 5-3 shows this concept in block diagram form.

ctypes exports three primary interface classes: cdll, windll, and oledll. cdll is used for both Linux and Windows and supports library modules that use the cdecl calling convention. The windll class supports Windows libraries that use the stdcall convention. oledll also uses the stdcall convention, but it assumes that the library functions return an HRESULT error code. cdecl is the default for C programs, so most library objects on Linux will use this convention. On a Windows system you should check the appropriate Windows technical documentation to see what the various library objects use as their calling convention.

Using ctypes is straightforward. Here is what happens when the Windows C runtime library, msvcrt (Microsoft Visual C Run-Time), is accessed using ctypes:

>>> from ctypes import *
>>> msvcrt = cdll.msvcrt
>>> msvcrt
<CDLL 'msvcrt', handle 78000000 at 97b0f0>
>>> msvcrt.printf("Testing...
")
Testing...
11
>>> rc = msvcrt.printf("Testing...
")
Testing...

Figure 5-3. DLL wrapper extension and support module

This calls the trusty printf() function, which behaves exactly like you might expect. Note that the call returns the number of characters passed to printf(). If you look up the documentation for the printf() function you will see that this is exactly what it is supposed to do, although in most cases the return value is simply ignored. In order to catch the return value and prevent it from showing up on the console, I used the variable rc.

On a Linux system, you will need to specify the full name of the shared library object:

libc = cdll.LoadLibrary("libc.so.6")

Alternatively, you can pass the library name into the class constructor method, like this:

libc = CDLL("libc.so.6")

This technique also works with Windows DLLs:

libc = CDLL("msvcrt.dll")

Notice that with Windows the .dll extension is supplied automatically, but with Linux the full name must be used.

Once a DLL has been loaded using one of the ctypes classes, its internal functions are accessed as methods of the resulting ctypes object.

When calling a function in an external DLL or capturing a return value, one needs to be aware of how ctypes translates between Python’s native data types and those used in C. Table 5-5 shows a comparison of ctypes, C, and Python data types.

Understanding how ctypes handles the conversion between Python and C is essential. For example, if we want to pass a floating-point value to a DLL function from Python, it must be converted into the appropriate type. Just handing off a float value from Python won’t work.

Let’s assume that a DLL has been loaded and we now have an object called someDLL to reference it. If it contains a function called someFunc() that we want to use, and this function accepts an integer parameter and a floating-point parameter, we might think that the following would work:

int_var = 5
float_var = 22.47
someDLL.someFunc(int_var, float_var)

Unfortunately, this won’t work, and Python will issue an exception traceback. The reason for this is that all Python types except for integers and strings have to be wrapped with the appropriate ctypes type, like this:

someDLL.someFunc(int_var, c_double(float_var))

This will now work as expected.

The native Python object types None, integer, long, and string are the only types that can be used directly as parameters via ctypes to functions in a DLL. A None object is passed as a NULL pointer. A string object is passed as a pointer to the location in memory where the data resides. Integer types (both integer and long) are passed as whatever the platform’s default C int type happens to be. Note that a long data object will be masked to fit into the default C int type.

Although ctypes may appear to be simple, it can quickly become rather complex if you want to do things like pass pointers, define a callback function, or get the value of a variable exported from a DLL. We will stick with the basic capabilities in this book, but I would encourage you to take a look at the ctypes documentation located at http://docs.python.org/library/ctypes.html. Also, the various data types in ctypes are useful for more than just accessing an external DLL. In Chapter 12 I’ll show you how to use ctypes to handle binary data directly in a Python program, including C structures, and in Chapter 14 we’ll see how ctypes is used in a commercial API for a USB interface device.