You need to make a method call to an XML-RPC server.

The xmlrpclib module makes writing XML-RPC clients very easy. For example, we can use XML-RPC to access O’Reilly’s Meerkat server and get the five most recent items about Python:

from xmlrpclib import Server
server = Server("http://www.oreillynet.com/meerkat/xml-rpc/server.php")
print server.meerkat.getItems(
    {'search': '[Pp]ython', 'num_items': 5, 'descriptions': 0}
)

XML-RPC is a simple, lightweight approach to distributed processing. xmlrpclib, which makes it easy to write XML-RPC clients in Python, is part of the Python Standard Library.

To use xmlrpclib, you first instantiate a proxy to the server, calling the ServerProxy class (also known by the name Server) and passing in the URL to which you want to connect. Then, on that proxy instance, you can access and call whatever methods the remote XML-RPC server supplies. In this case, you know that Meerkat supplies a getItems method, so you call the method of the same name on the server proxy instance. The proxy relays the call to the server, waits for the server to respond, and finally returns the call’s results.

This recipe uses O’Reilly’s Meerkat service, intended for syndication of contents such as news and product announcements. Specifically, the recipe queries Meerkat for the five most recent items mentioning either “Python” or “python”. If you try this recipe, be warned that response times from Meerkat are variable, depending on the quality of your Internet connection, the time of day, and the level of traffic on the Internet. If the script takes a long time to answer, it doesn’t mean you did something wrong—it just means you have to be patient!

Using xmlrpclib by passing raw dictionaries, as in this recipe’s code, is quite workable but somewhat unPythonic. Here’s an easy alternative that looks nicer:

from xmlrpclib import Server
server = Server("http://www.oreillynet.com/meerkat/xml-rpc/server.php")
class MeerkatQuery(object):
    def _ _init_ _(self, search, num_items=5, descriptions=0):
        self.search = search
        self.num_items = num_items
        self.descriptions = descriptions
q = MeerkatQuery("[Pp]ython")
print server.meerkat.getItems(q)

You can package the instance attributes and their default values in several different ways, but the main point of this variant is that, as the argument to the getItems method, an instance object with the right attributes works just as well as a dictionary object with the same information packaged as dictionary items.

The xmlrpclib module is part of the Python Standard Library and is well documented in its chapter of the Library Reference portion of Python’s online documentation. Meerkat is at http://www.oreillynet.com/meerkat/.

You need to implement an XML-RPC server.

Module SimpleXMLRPCServer, which is part of the Python Standard Library, makes writing XML-RPC servers reasonably easy. Here’s how you can write an XML-RPC server:

# Server code sxr_server.py
import SimpleXMLRPCServer
class StringFunctions(object):
    def _ _init_ _(self):
        # Make all the functions in Python's standard string module
        # available as 'python_string.func_name' for each func_name
        import string
        self.python_string = string
    def _privateFunction(self):
        # This function cannot be called directly through XML-RPC because
        # it starts with an underscore character '_', i.e., it's "private"
        return "you'll never get this result on the client"
    def chop_in_half(self, astr):
        return astr[:len(astr)/2]
    def repeat(self, astr, times):
        return astr * times
if _ _name_ _=='_ _main_ _':
    server = SimpleXMLRPCServer.SimpleXMLRPCServer(("localhost", 8000))
    server.register_instance(StringFunctions( ))
    server.register_function(lambda astr: '_' + astr, '_string')
    server.serve_forever( )

And here is a client script that accesses the server you just wrote:

# Client code sxr_client.py
import xmlrpclib
server = xmlrpclib.Server('http://localhost:8000')
print server.chop_in_half('I am a confident guy')
# emits:I an a con
print server.repeat('Repetition is the key to learning!
', 5)
# emits 5 lines, all Repetition is the key to learning!
print server._string('<= underscore')
# emits _<= underscore
print server.python_string.join(['I', 'like it!'], " don't ")
# emits I don't like it!
print server._privateFunction( )    # this will throw an exception
# terminates client script with traceback for xmlrpclib.Fault

This recipe demonstrates the creation of a simple XML-RPC server using the SimpleXMLRPCServer module of the standard Python library. The module contains a class of the same name that listens for HTTP requests on a specified port and dispatches any XML-RPC calls to registered instances or registered functions. This recipe demonstrates both usages. To create a server, we instantiate the SimpleXMLRPCServer class, supplying the hostname and port for the server. Then, on that instance, we call register_instance as many times as needed to make other instances available as services. In addition, or as an alternative, we call register_function to make functions similarly available as services. Once we have registered all the instances and/or all the functions we want to expose, we call the serve_forever method of the server instance, and our XML-RPC server is active. Yes, it is really that simple. The only output on the shell prompt window on which you run the server is one line of log information each time a client accesses the server; the only way to terminate the server is to send it an interrupt, for example with a Ctrl-C keystroke.

Registering functions (as opposed to an instance) is necessary when a function name begins with an underscore (_) or contains characters not allowed in Python identifiers (e.g., accented letters, punctuation marks, etc.). Dotted names (e.g., python_string.join) are correctly resolved for registered instances.

The SimpleXMLRPCServer module is part of the Python Standard Library and is documented in its chapter of the Library Reference portion of Python’s online documentation.

You need to establish a lightweight, highly scalable, distributed processing system and want to use the XML-RPC protocol.

Package medusa lets you implement lightweight, highly scalable, asynchronous (event-driven) network servers. An XML-RPC handler is included in the Medusa distribution. Here is how you can code an XML-RPC server with Medusa:

# xmlrpc_server.py
from socket import gethostname
from medusa.xmlrpc_handler import xmlrpc_handler
from medusa.http_server import http_server
from medusa import asyncore
class xmlrpc_server(xmlrpc_handler):
    # initialize and run the server
    def _ _init_ _(self, host=None, port=8182):
        if host is None:
            host = gethostname( )
        hs = http_server(host, port)
        hs.install_handler(self)
        asyncore.loop( )
    # an example of a method to be exposed via the XML-RPC protocol
    def add(self, op1, op2):
        return op1 + op2
    # the infrastructure ("plumbing") to expose methods
    def call(self, method, params):
        print "calling method: %s, params: %s" % (method, params)
        if method == 'add':
            return self.add(*params)
        return "method not found: %s" % method
if _ _name_ _ == '_ _main_ _':
    server = xmlrpc_server( )

And here is a client script that accesses the server you just wrote:

# xmlrpc_client.py
from socket import gethostname
from xmlrpclib import Transport, dumps
class xmlrpc_connection(object):
    def _ _init_ _(self, host=None, port=8182):
        if host is None:
            host = gethostname( )
        self.host = "%s:%s" % (host, port)
        self.transport = Transport( )
    def remote(self, method, params=( )):
        """ Invoke the server with the given method name and parameters.
            The return value is always a tuple. """
        return self.transport.request(self.host, '/RPC2',
                                      dumps(params, method))
if _ _name_ _ == '_ _main_ _':
    connection = xmlrpc_connection( )
    answer, = connection.remote("add", (40, 2))
    print "The answer is:", answer

This recipe demonstrates remote method calls between two machines (or two processes, even on the same machine) using the XML-RPC protocol and provides a complete example of working client/server code.

XML-RPC is one of the easiest ways to handle distributed processing tasks. There’s no messing around with the low-level socket details, nor is it necessary to write an interface definition. The protocol is platform and language neutral. The XML-RPC specification can be found at http://www.xml-rpc.com and is well worth studying. It’s nowhere as functionally rich as heavyweight stuff like CORBA, but, to compensate, it is much simpler!

To run this recipe’s Solution, you must download the Medusa library from http://www.nightmare.com (the Python Standard Library includes the asyncore and asynchat modules, originally from Medusa, but not the other parts of Medusa required for this recipe). With Medusa, you implement an XML-RPC server by subclassing the xmlrpc_handler class and passing an instance of your class to the install_handler method of an instance of http_server. HTTP is the transport-level protocol used by the XML-RPC standard, and http_server handles all transport-level issues on your behalf. You need to provide only the handler part, by customizing xmlrpc_handler through subclassing and method overriding. Specifically, you must override the call method, which the Medusa framework calls on your instance with the name of the XML-RPC method being called, along with its parameters, as arguments. This is exactly what we do in this recipe, in which we expose a single XML-RPC method named add which accepts two numeric parameters and returns their sum as the method’s result.

This recipe’s XML-RPC client uses xmlrpclib in a more sophisticated way than Recipe 15.1, by accessing the Transport class explicitly. In theory, this approach allows finer-grained control. However, this recipe does not exert that kind of control, and it’s rarely required in XML-RPC clients that you actually deploy, anyway.

The xmlrpclib module is part of the Python Standard Library and is documented in a chapter of the Library Reference portion of Python’s online documentation. Medusa is at http://www.nightmare.com.

You are coding an XML-RPC server, using the Python Standard Library’s SimpleXMLRPCServer module, and you want to make it possible for a remote client to cause the XML-RPC server to exit cleanly.

You have to use your own request-handling loop (instead of the serve_forever method of SimpleXMLRPCServer) so that you can stop looping when appropriate. For example:

import SimpleXMLRPCServer
running = True
def finis( ):
    global running
    running = False
    return 1
server = SimpleXMLRPCServer.SimpleXMLRPCServer(('127.0.0.1', 8000))
server.register_function(finis)
while running:
    server.handle_request( )

SimpleXMLRPCServer ’s serve_forever method, as its name implies, attempts to keep serving “forever”—that is, it keeps serving until the whole server process is killed. Sometimes, it’s useful to allow remote clients to request a clean termination of a service by remotely calling a server-exposed function, and this recipe demonstrates the simplest way to allow this functionality.

The finis function (which gets exposed to remote clients via the register_function call) sets the global variable running to False (and then returns something that is not None because the XML-RPC protocol cannot deal with the None object). Using the while running loop, instead of a serve_forever call, then ensures that the server stops serving and terminates when the variable running becomes false.

If you prefer to subclass SimpleXMLRPCServer, you can obtain a similar effect by overriding the serve_forever method: that is, instead of placing the simple while running: server.handle_request loop inline, you can code, for example (with the same function finis as in the recipe’s Solution):

class MyServer(SimpleXMLRPCServer.SimpleXMLRPCServer):
    def serve_forever(self):
        while running:
            self.handle_request( )
server = MyServer(('127.0.0.1', 8000))
server.register_function(finis)
server.serve_forever( )

However, this alternative approach offers no special advantage (unless you have a fetish for being object oriented for no particular purpose), and, since this alternative approach is telling a little white lie (by using the name serve_forever for a method that does not keep serving “forever”!), the simpler approach in the recipe’s Solution can definitely be recommended.

The SimpleXMLRPCServer module is part of the Python Standard Library and is documented in a chapter of the Library Reference portion of Python’s online documentation.

You are coding XML-RPC servers with the Python Standard Library SimpleXMLRPCServer class and want to ensure you’re using the simple but useful idioms that can ease your coding, or give your servers more flexibility at no substantial cost to you.

Here are a few tweaks I generally use, to enhance my servers’ usability, when I’m developing servers based on SimpleXMLRPCServer:

# give the base class a short, readable nickname
from SimpleXMLRPCServer import SimpleXMLRPCServer as BaseServer
class Server(BaseServer):
    def _ _init_ _(self, host, port):
        # accept separate hostname and portnumber and group them
        BaseServer._ _init_ _(self, (host, port))
    def server_bind(self):
        # allow fast restart of the server after it's killed
        import socket
        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        BaseServer.server_bind(self)
    allowedClientHosts = '127.0.0.1', '192.168.0.15',
    def verify_request(self, request, client_address):
        # forbid requests except from specific client hosts
        return client_address[0] in self.allowedClientHosts

The recipe begins with a statement of the form from module import name as nickname, a Python idiom that is often handy for importing something under a short and usable nickname. It’s certainly miles better than having to repeatedly write SimpleXMLRPCServer.SimpleXMLRPCServer after a simple import statement, or using the ill-advised construct from module import *, which mixes up all the namespaces and can often cause subtle and hard-to-find bugs.

The sole purpose of the _ _init_ _ statement of class Server is to accept host and port as separate parameters and group them into the required tuple. I find myself often writing such statements with the many Python functions and classes that require this address tuple grouping (your tastes, of course, may be different).

By default, a server socket belonging to a process that dies is kept busy for quite a while. Particularly during development, it is handy to kill such a process, edit the script, and restart immediately. For such an immediate restart to work, you must ensure the code of your server sets the SO_REUSEADDR option on the relevant socket, as the recipe’s code does in its overridden method server_bind.

Last but not least, the recipe overrides verify_request in order to apply a simple check that refuses service except to requests coming from client hosts on a predefined list. This approach doesn’t provide rock-solid security, but nevertheless, it is potentially useful. Again, it’s particularly useful during development, to help avoid those cases where some other developer on the same LAN accidentally connects his client to the server I’m just developing, and we both experience puzzling problems until we figure out what’s happened!

The SimpleXMLRPCServer module is part of the Python Standard Library and is documented in a chapter of the Library Reference portion of Python’s online documentation.

You are writing an XML-RPC server and want to add a GUI to it, or you’re writing a GUI application that you want to be able to interact as an XML-RPC server too.

As long as you use Twisted for the network interaction, and wxPython for the GUI, this task is reasonably easy, since these packages can cooperate through the twisted.internet.wxreactor module. You do need to have specific incantations at the start of your program, as follows:

# To use wxPython and Twisted together, do the following, in exact order:
import wx
from twisted.internet import wxreactor
wxreactor.install( )
from twisted.internet import reactor
# Then, have whatever other imports as may be necessary to your program
from twisted.web import xmlrpc, server
class MyFrame(wx.Frame):
    ''' Main window for this wx application. '''
    def _ _init_ _(self, parent, ID, title, pos=wx.DefaultPosition,
                 size=(200, 100), style=wx.DEFAULT_FRAME_STYLE):
        wx.Frame._ _init_ _(self, parent, ID, title, pos, size, style)
        wx.EVT_CLOSE(self, self.OnCloseWindow)
    def OnCloseWindow(self, event):
        self.Destroy( )
        reactor.stop( )
class MyXMLRPCApp(wx.App, xmlrpc.XMLRPC):
    ''' We're a wx Application _AND_ an XML-RPC server too. '''
    def OnInit(self):
        ''' wx-related startup code: builds the GUI. '''
        self.frame = MyFrame(None, -1, 'Hello')
        self.frame.Show(True)
        self.SetTopWindow(self.frame)
        return True
    # methods exposed to XML-RPC clients:
    def xmlrpc_stop(self):
        """ Closes the wx application. """
        self.frame.Close( )
        return 'Shutdown initiated'
    def xmlrpc_title(self, x):
        """ Change the wx application's window's caption. """
        self.frame.SetTitle(x)
        return 'Title set to %r' % x
    def xmlrpc_add(self, x, y):
        """ Provide some computational services to clients. """
        return x + y
if _ _name_ _ == '_ _main_ _':
    # pass False to emit stdout/stderr to shell, not an additional wx window
    app = MyXMLRPCApp(False)
    # Make the wx application twisted-aware
    reactor.registerWxApp(app)
    # Make a XML-RPC Server listening to port 7080
    reactor.listenTCP(7080, server.Site(app))
    # Start both reactor parts (wx MainLoop and XML-RPC server)
    reactor.run( )

It is often useful to give an XML-RPC server a GUI, for example, to display the current status to an operator or administrator. Conversely, it is often useful to give a GUI application the ability to accept remote requests from other programs, and making the application an XML-RPC server is an excellent, simple way to accomplish that purpose.

Either way, if you use Twisted for the networking part, you’re off to a good start, because Twisted offers specialized reactor implementations to ease cooperation with several GUI toolkits. In particular, this recipe shows how a Twisted-based XML-RPC server can sport a wxPython GUI thanks to the twisted.internet.wxreactor module.

To try this recipe, save the code from the “Solution” as a Python script and start it from a shell. If you run some kind of “personal firewall” that’s normally set to impede TCP/IP communication between programs running on your machine, ensure it’s set to let such communication happen on TCP port 7080. Then, from any interactive Python interpreter session on the same machine, do:

>>> import xmlrpclib
>>> s = xmlrpclib.ServerProxy('http://localhost:7080')
>>> s.add(23, 42)65
>>> s.title('Changed Title')
Title set to 'Changed Title'

Observe that the title of the wx application’s window has changed. Now, you can close the application, either by whatever GUI means you normally use on your platform (it is a totally cross-platform application, after all), or by calling s.stop( ) from the same Python interpreter interactive session that we just showed. You can also run such a client on any other machine, as long as it has open TCP/IP connectivity on port 7080 with the machine running the server. (In particular, make sure you open port 7080 on any firewall that would normally block that port, whether the firewall is on either of the machines, or on any other network apparatus that may lie between them.)

Both Twisted and wxPython, while already rich and solid frameworks, are still growing and changing, so it may be important to ensure you have the right releases installed properly on your machine. This recipe should run on any platform that is equipped with Python 2.3 or better, wxPython 2.4.2.4 or better, and Twisted 1.3.0 or better. Of course, we don’t have access to every platform in the world, nor to all future releases of these tools, so we tested the recipe only under Windows/XP, Mac OS X 10.3.6, and Linux, with Python 2.3 and 2.4, wxPython 2.4.2.4, and some 2.5.x.y releases, and Twisted 1.3.0 specifically.

Since the recipe relies only on published, supported aspects of the various tools, one can hope that the recipe will also work elsewhere, and will work with future releases of the tools. However, if this recipe’s approach does not prove satisfactory for your purposes, you may want to try a different approach based on threads, shown at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286201.

Twisted’s home page is http://www.twistedmatrix.com; documentation on Twisted XML-RPC support is at http://www.twistedmatrix.com/documents/current/howto/xmlrpc; wxPython’s home page is http://www.wxpython.org.

You want to implement Python clients and servers for some distributed processing task, without repetitious “boilerplate” code, and with excellent performance and scalability characteristics.

Use the Perspective Broker (PB) subsystem of the Twisted framework. A PB server just subclasses the PB’s Root class and adds remotely callable methods. Here is an example of a server script which adds just one remotely callable method, named Pong:

from twisted.spread import pb
from twisted.internet import reactor
PORT = 8992
class Ponger(pb.Root):
    def remote_Pong(self, ball):
        print 'CATCH', ball,
        ball += 1
        print 'THROW', ball
        return ball
reactor.listenTCP(PORT, pb.BrokerFactory(Ponger( )))
reactor.run( )

We could write an equally trivial script for the client side of the interaction, but let’s instead have a rather feature-rich PB client, which deals with important issues often ignored in introductory examples of distributed programming, such as error handling:

from twisted.spread import pb
from twisted.internet import reactor
import sys
PORT = 8992
DELAY         = 1
DOG_DELAY     = 2
RESTART_DELAY = 5
class Pinger(object):
    def _ _init_ _(self, host):
        self.ping = None
        self.host = host
        self.ball = 0
        self._start( )
    def _start(self):
        print 'Waiting for Server', self.host
        dfr = pb.getObjectAt(self.host, PORT, 30)
        dfr.addCallbacks(self._gotRemote, self._remoteFail)
    def _gotRemote(self, remote):
        remote.notifyOnDisconnect(self._remoteFail)
        self.remote = remote
        self._ping( )
    def _remoteFail(self, _ _):
        if self.ping:
            print 'ping failed, canceling and restarting'
            self.ping.cancel( )
            self.ping = None
        self.restart = reactor.callLater(RESTART_DELAY, self._start)
    def _watchdog(self):
        print 'ping timed out, canceling and restarting'
        self._start( )
    def _ping(self):
        self.dog = reactor.callLater(DOG_DELAY, self._watchdog)
        self.ball += 1
        print 'THROW', self.ball,
        dfr = self.remote.callRemote('Pong', self.ball)
        dfr.addCallbacks(self._pong, self._remoteFail)
    def _pong(self, ball):
        self.dog.cancel( )
        print 'CATCH',  ball
        self.ball = ball
        self.ping = reactor.callLater(DELAY, self._ping)
if _ _name_ _ == '_ _main_ _':
    if len(sys.argv) != 2:
        print 'Usage: %s serverhost' % sys.argv[0]
        sys.exit(1)
    host = sys.argv[1]
    print 'Ping-pong client to host', host
    Pinger(host)
    reactor.run( )

Twisted is a framework for asynchronous (also known as event-driven) programming of network clients, servers, proxies, and so on. The asynchronous programming model (which Twisted implements through the Reactor Design Pattern embodied in the twisted.internet.reactor module) provides excellent performance and scalability characteristics for Twisted-based programs.

Twisted also includes many subsystems that offer your programs ready-to-go networking functionality. One of these subsystems, Perspective Broker (PB), is implemented in the twisted.spread.pb module. PB lets you code distributed-programming clients and servers, with an ease of use that’s most clearly displayed in the server program at the start of this recipe’s Solution. In just a few lines of code, the server class is able to expose remotely callable methods: all it takes is subclassing the Root class of the pb module and naming each remotely callable method with a prefix of remote_.

Most of the client code in this recipe is concerned with diagnosing and handling possible problems and errors with the connection to the server. Specifically, if the connection fails for any reason, including a timeout diagnosed by the watchdog timer that the client sets up each time it pings, the client attempts to reconnect to the server. If you kill the server, the client keeps trying to reconnect, periodically, until you restart the server.

Error-handling apart, the client is essentially as simple as the server. In the method _start, the client calls function getObjectAt of module twisted.spread.pb, which takes as its arguments the server’s host, a port number, and a “time-out” delay in seconds. As usual in Python networking, the host can be either a network name, such as localhost, or a string representing an IP address, such as 127.0.0.1.

If no problems arise, getObjectAt returns an object that proxies for the remote PB server. The proxy object, in turn, has a callRemote method, which takes as its arguments the method name as a string, followed by any arguments you are passing to the remote method. callRemote returns a Twisted deferred object, the lynchpin of Twisted’s style of asynchronous (event-driven) programming. Learning to use deferreds effectively is the fundamental step in learning to program with Twisted.

A deferred object represents an event that may occur in the future (the success-case) or may end in failure. Given a deferred, you can add callbacks to it for both success and failure cases. (You can also chain callbacks, a possibility that this recipe does not exploit.) When the deferred’s event occurs, Twisted calls your “success-case” callback, passing as its argument the “result” of the deferred. Alternatively, if the deferred ends in failure, Twisted calls your failure-case callback, passing as its argument a failure object that wraps a Python exception object.

As you see in this recipe, despite deferreds’ potentially rich and vast functionality, their use is really quite simple in most cases. For example, in the failure cases, the client in this recipe wants to retry connecting: therefore, method _remoteFail accepts the failure-object argument with an argument name of “two underscores” (_ _), a common Python convention that indicates the argument will be ignored.

The Twisted web site, at http://www.twistedmatrix.com, has abundant documentation about all of Twisted’s elements and subsystems, including Perspective Broker and deferred objects.

You need to implement a CORBA server and client to distribute a processing task, such as the all-important network-centralized fortune-cookie distribution.

CORBA is a solid, rich, mature object-oriented RPC protocol, and several CORBA ORBs offer excellent Python support. This recipe requires multiple files. Here is the interface definition file, fortune.idl, coded in CORBA’s own IDL (Interface Definition Language):

module Fortune {
    interface CookieServer {
        string get_cookie( );
    };
};

This code is quite readable even if you’ve never seen CORBA’s IDL before: it defines a module named Fortune, whose only contents is an interface named CookieServer, whose only contents is a function (method) named get_cookie, which takes no arguments and returns a string. This code says nothing at all about the implementation: IDL is a language for defining interfaces.

The server script is a simple Python program:

import sys, os
import CORBA, Fortune, Fortune_ _POA
FORTUNE_PATH = "/usr/games/fortune"
class CookieServer_i(Fortune_ _POA.CookieServer):
    def get_cookie(self):
        pipe   = os.popen(FORTUNE_PATH)
        cookie = pipe.read( )
        if pipe.close( ):
            # An error occurred with the pipe
            cookie = "Oh dear, couldn't get a fortune
"
        return cookie
orb = CORBA.ORB_init(sys.argv)
poa = orb.resolve_initial_references("RootPOA")
servant = CookieServer_i( )
poa.activate_object(servant)
print orb.object_to_string(servant._this( ))
# see the Discussion session about what this print statement emits
poa._get_the_POAManager( ).activate( )
orb.run( )

And here’s a demonstration of client code for this server, using a Python interactive command shell:

>>> import CORBA, Fortune
>>> orb = CORBA.ORB_init( )
>>> o = orb.string_to_object(
...   "corbaloc::host.example.com/fortune")
>>> o = o._narrow(Fortune.CookieServer)
>>> print o.get_cookie( )

CORBA has a reputation for being hard to use, but it is really very easy, especially with Python. This example shows the complete CORBA implementation of a fortune-cookie server and its client. To run this example, you need a Python-compatible CORBA implementation (i.e., an ORB)—or, if you wish, two such ORBs, since you can use two different CORBA implementations, one for the client and one for the server, and let them interoperate with the CORBA IIOP inter-ORB protocol. Several free CORBA implementations, which fully support Python, are available for you to download and install. The Python language support is part of the CORBA standards, so, if a certain ORB supports Python at all, you can code your Python source for it in just the same way as you can code it for any other compliant ORB, be it free or commercial. In this recipe, we use the free ORB known as omniORB. With omniORB, you can use omniORBpy, which lets you develop CORBA applications from Python.

With most ORBs, you must convert the interface definition coded in IDL into Python declarations with an IDL compiler. For example, with omniORBpy:

omniidl -bpython fortune.idl

This creates Python modules named Fortune and Fortune_ _POA, in files Fortune.py and Fortune_POA.py, to be used by clients and servers, respectively.

In the server, we implement the CookieServer CORBA interface by importing Fortune_ _POA and subclassing the CookieServer class that the module exposes. Specifically, in our own subclass, we need to override the get_cookie method (i.e., implement the methods that the interface asserts we’re implementing). Then, we start CORBA to get an orb instance, ask the ORB for a POA (Portable Object Adaptor), instantiate our own interface-implementing object, and pass it to the POA instance’s activate_object method. Finally, we call the activate method on the POA manager and the run method on the ORB to start our service.

When you run the server, it prints out a long hex string, such as:

IOR:010000001d00000049444c3a466f7274756e652f436f6f6b69655365727665723
a312e300000000001000000000000005c000000010102000d0000003135382e313234
2e36342e330000f90a07000000666f7274756e6500020000000000000008000000010
0000000545441010000001c0000000100000001000100010000000100010509010100
0100000009010100

Printing this string is the purpose of the object_to_string call that our recipe’s server performs just before it activates and runs.

You have to pass this string value as the argument of the client’s orb.string_to_object( ) call to contact your server. Such long hex strings may not be convenient to communicate to clients. To remedy this, it’s easy to make your server support a simple corbaloc URL string, like the one used in the client example, but doing so involves omniORB-specific code that is not necessarily portable to other ORBs. (See the omniORBpy manual for details of corbaloc URL support.)

You can download omniORBpy, including its documentation, from http://www.omniorb.org/omniORBpy/.

You need to send commands to one or more logins that can be on a local machine, or a remote machine, and the Telnet protocol is acceptable.

Telnet is one of the oldest protocols in the TCP/IP stack, but it may still be serviceable (at least within an intranet that is well protected against sniffing and spoofing attacks). In any case, Python’s standard module telnetlib supports Telnet quite well:

# auto_telnet.py - remote control via telnet
import os, sys, telnetlib
from getpass import getpass
class AutoTelnet(object):
    def _ _init_ _(self, user_list, cmd_list, **kw):
        # optional parameters are host, timeout in seconds, command
        # prompt to expect from the host on successful logins:
        self.host = kw.get('host', 'localhost')
        self.timeout = kw.get('timeout', 600)
        self.command_prompt = kw.get('command_prompt', "$ ")
        # collect passwords for each user, interactively
        self.passwd = {  }
        for user in user_list:
            self.passwd[user] = getpass("Enter user '%s' password: " % user)
        # instantiate Telnet proxy
        self.telnet = telnetlib.Telnet( )
        for user in user_list:
            # login with given host and user, and act appropriately
            self.telnet.open(self.host)
            ok = self.action(user, cmd_list)
            if not ok:
                print "Unable to process:", user
            self.telnet.close( )
    def action(self, user, cmd_list):
        # wait for a login prompt
        t = self.telnet
        t.write("
")
        login_prompt = "login: "
        response = t.read_until(login_prompt, 5)
        if login_prompt in response:
            print response
        else:
            return 0
        # supply user and password for login
        t.write("%s
" % user)
        password_prompt = "Password:"
        response = t.read_until(password_prompt, 3)
        if password_prompt in response:
            print response
        else:
            return 0
        t.write("%s
" % self.passwd[user])
        # wait for command prompt to indicate successful login
        response = t.read_until(self.command_prompt, 5)
        if self.command_prompt not in response:
            return 0
        # send each command and wait for command prompt after each
        for cmd in cmd_list:
            t.write("%s
" % cmd)
            response = t.read_until(self.command_prompt, self.timeout)
            if self.command_prompt not in response:
                return 0
            print response
        return 1
if _ _name_ _ == '_ _main_ _':
    # code which runs as a main script, only
    basename = os.path.splitext(os.path.basename(sys.argv[0]))[0]
    logname = os.environ.get("LOGNAME", os.environ.get("USERNAME"))
    host = 'localhost'
    import getopt
    optlist, user_list = getopt.getopt(sys.argv[1:], 'c:f:h:')
    usage = """
usage: %s [-h host] [-f cmdfile] [-c "command"] user1 user2 ...
    -c  command
    -f  command file
    -h  host  (default: '%s')
Example:  %s -c "echo $HOME" %s
""" % (basename, host, basename, logname)
    if len(sys.argv) < 2:
        print usage
        sys.exit(1)
    cmd_list = [  ]
    for opt, optarg in optlist:
        if opt == '-f':
            for r in open(optarg):
                if r.rstrip( ):
                    cmd_list.append(r)
        elif opt == '-c':
            command = optarg
            if command[0] == '"' and command[-1] == '"':
                command = command[1:-1]
            cmd_list.append(command)
        elif opt == '-h':
            host = optarg
    autoTelnet = AutoTelnet(user_list, cmd_list, host=host)

Python’s telnetlib lets you easily automate access to Telnet servers, even from non-Unix machines. As a flexible alternative to the popen functions, which only run commands locally as the user that’s running the script, telnetlib, which can work across an intranet and can login and run commands as different users, is a handy technique to have in your system administration toolbox.

Production code generally has to be made more robust, but this recipe should be enough to get you started in the right direction. The recipe’s AutoTelnet class instantiates a single telnetlib.Telnet object and uses that single object in a loop over a list of users. For each user, the recipe calls the open method of the Telnet instance to open the connection to the specified host, runs a series of commands in AutoTelnet’s action method, and finally calls the close method of the Telnet instance to terminate the connection.

AutoTelnet’s action method is where the action is. All operations depend on two methods of the Telnet instance. The write method takes a single string argument and writes it to the connection. The read_until method takes two arguments, a string to wait for and a timeout in seconds, and returns a string with all the characters received from the connection until the timeout elapsed or the waited-for string occurred. action’s code uses these two methods to wait for a login prompt and send the username; wait for a password prompt and send the password; and then, repeatedly, wait for a command prompt (typically from a Unix shell at the other end of the connection) and send the commands in the list sequentially (waiting for a command prompt again after sending each one).

One warning (which applies to any use of Telnet and some other old protocols): except when transmitting completely public data, not protected by passwords that might be of interest to intruders of ill will, do not run Telnet (or non-anonymous FTP, for that matter) on networks on which you are not completely sure that nobody is packet-sniffing, since these protocols date from an older, more trusting age. These protocols let passwords and everything else travel in the clear, open to any snooper. This issue is not Python specific; it applies to any implementation of these protocols, since it depends on the definition of the protocols themselves. Whether or not you use Python, be advised: if there is any risk that someone might be packet-sniffing, use SSH instead, as shown next in Recipe 15.10 so that no password ever travels on the network in the clear, and so that the connection stream itself gets encrypted.

Documentation on the standard library module telnetlib in the Library Reference; Recipe 15.10.

You need to send commands, using the SSH protocol, to one or more logins that can be on a local machine or a remote machine.

SSH is a secure replacement for the old Telnet protocol. One way to use SSH from a Python program is with the third-party paramiko package:

# auto_ssh.py - remote control via ssh
import os, sys, paramiko
from getpass import getpass
paramiko.util.log_to_file('auto_ssh.log', 0)
def parse_user(user, default_host, default_port):
    ''' given name[@host[:port]], returns name, host, int(port),
        applying defaults for hose and/or port if necessary
    '''
    if '@' not in user:
        return user, default_host, default_port
    user, host = user.split('@', 1)
    if ':' in host:
        host, port = host.split(':', 1)
    else:
        port = default_port
    return user, host, int(port)
def autoSsh(users, cmds, host='localhost', port=22, timeout=5.0,
            maxsize=2000, passwords=None):
    ''' run commands for given users, w/default host, port, and timeout,
        emitting to standard output all given commands and their
        responses (no more than 'maxsize' characters of each response).
    '''
    if passwords is None:
        passwords = {  }
    for user in users:
        if user not in passwords:
            passwords[user] = getpass("Enter user '%s' password: " % user)
    for user in users:
        user, host, port = parse_user(user, default_host, default_port)
        try:
            transport = paramiko.Transport((host, port))
            transport.connect(username=user, password=passwords[user])
            channel = transport.open_session( )
            if timeout: channel.settimeout(timeout)
            for cmd in cmd_list:
                channel.exec_command(cmd)
                response = channel.recv(max_size)
                print 'CMD %r(%r) -> %s' % (cmd, user, response)
        except Exception, err:
            print "ERR: unable to process %r: %s" % (user, err)
if _ _name_ _ == '_ _main_ _':
    logname = os.environ.get("LOGNAME", os.environ.get("USERNAME"))
    host = 'localhost'
    port = 22
    usage = """
usage: %s [-h host] [-p port] [-f cmdfile] [-c "command"] user1 user2 ...
    -c  command
    -f  command file
    -h  default host  (default: localhost)
    -p  default host  (default: 22)
Example:  %s -c "echo $HOME" %s
same as:  %s -c "echo $HOME" %s@localhost:22
""" % (sys.argv[0], sys.argv[0], logname, sys.argv[0], logname)
    import getopt
    optlist, user_list = getopt.getopt(sys.argv[1:], 'c:f:h:p:')
    if not user_list:
        print usage
        sys.exit(1)
    cmd_list = [  ]
    for opt, optarg in optlist:
        if opt == '-f':
            for r in open(optarg, 'rU'):
                if r.rstrip( ):
                    cmd_list.append(r)
        elif opt == '-c':
            command = optarg
            if command[0] == '"' and command[-1] == '"':
                command = command[1:-1]
            cmd_list.append(command)
        elif opt == '-h':
            host = optarg
        elif opt == '-p':
            port = optarg
        else:
            print 'unknown option %r' % opt
            print usage
            sys.exit(1)
    autoSsh(user_list, cmd_list, host=host, port=port)

The third-party extension paramiko package lets you easily automate access to all sorts of SSH services, even from non-Unix machines. paramiko even lets you write your own SSH servers in Python. In this recipe, however, we use paramiko on the client side, as a more secure alternative to the similar use of telnetlib shown previously in Recipe 15.9.

Production code generally has to be made more robust, but this recipe should be enough to get you started in the right direction. The recipe’s autoSsh function first ensures it knows passwords for all the users (asking interactively for the passwords of users it doesn’t know about). Then, it loops over all the users, parsing strings such as foo@bar:2222 to mean user foo at host bar, port 2222, and defaulting the host and port values, if necessary.

The loop body relies on two types of objects supplied by paramiko, Transport and Channel. The transport is constructed by giving it the ( host, port ) pair and then a connection is made with a username and password. (Alternatively, depending on the SSH server, one might connect using a private key, but this recipe uses just a password.) The channel is obtained from the transport, and the recipe then sets a timeout (by default, 6 seconds) to ensure that no long-term hanging occurs in case of problems with an SSH server or the network path to it. Finally, an inner loop over all commands sends each command, receives a response (up to a maximum length in bytes, 2000 by default), and prints the command and response.

paramiko’s home page at http://www.lag.net/~robey/paramiko/; paramiko requires another third-party extension to Python, the Python Cryptography Toolkit, whose home page is at http://www.amk.ca/python/code/crypto; docs on SSH at http://www.openssh.com/, http://www.ucolick.org/~sla/ssh/, http://kimmo.suominen.com/docs/ssh/; Richard Silverman and Daniel J. Barrett, SSH: The Secure Shell, The Definitive Guide (O’Reilly); Recipe 15.9.

You want your Python application to check SSL client authentication, by delegating, over HTTPS, to an Apache server that is running mod_ssl.

The Apache web server has good support for SSL, and we can write a Python script to exploit that support to authenticate a client. For example:

import httplib
CERT_FILE = '/home/robr/mycert'
PKEY_FILE = '/home/robr/mycert'
HOSTNAME = 'localhost'
conn = httplib.HTTPSConnection(HOSTNAME,
           key_file = PKEY_FILE, cert_file = CERT_FILE)
conn.putrequest('GET', '/ssltest/')
conn.endheaders( )
response = conn.getresponse( )
print response.read( )

The Solution code assumes that mycert is a certificate file formatted by PEM (Privacy-enhanced Electronic Mail), which includes both the public certificate and the private key. You can keep the public and private keys in separate files: you need to pass the names of the files in question as the values for the key_file and cert_file arguments of HTTPSConnection.

To safely perform SSL authentication, you will generally set up your own certification authority (CA). You do not want to enable a third-party organization to hand out all the “keys” to the locks that you put up to protect your security.

The Apache server installation that you use for this authentication needs to be configured to require SSL client authentication with the appropriate CA. My httpd.conf file contains the stanza:

SSLCACertificatePath /etc/httpd/conf/ssl.crt
SSLCACertificateFile /etc/httpd/conf/ssl.crt/myCA.crt
SSLVerifyClient      require
SSLVerifyDepth       2
SSLRequireSSL

The configuration of an Apache server cannot refer to more than one SSLCACertificateFile. You can put more than one CA certificate in that file, but doing so grants authentication to any client who has a certificate from any one of the certificate authorities you accept, which is unlikely to be what you want. Therefore, this recipe is fully applicable only when you can reasonably set up an Apache server to accept your own CA as the sole recognized one. In exchange for this modest inconvenience, however, you do get a handy and robust approach to client authentication between web-enabled applications, particularly good for SOAP or XML-RPC implementations, or custom applications that communicate via HTTP/HTTPS.

Descriptions of SSL and its use with Apache can be found at http://httpd.apache.org/docs-2.0/ssl/ssl_howto.html and http://www.pseudonym.org/ssl/ssl_cook.html. The httplib module is part of the Python Standard Library and is documented in a chapter of the Library Reference portion of Python’s online documentation.