The echo
client and
server programs shown previously serve to illustrate socket
fundamentals. But the server model suffers from a fairly major flaw:
if multiple clients try to connect to the server, and it takes a long time
to process a given client’s request, the server will fail. More
accurately, if the cost of handling a given request prevents the
server from returning to the code that checks for new clients in a
timely manner, it won’t be able to keep up with all the requests, and
some clients will eventually be denied connections.
In real-world client/server programs, it’s far more typical to code a server so as to avoid blocking new requests while handling a current client’s request. Perhaps the easiest way to do so is to service each client’s request in parallel—in a new process, in a new thread, or by manually switching (multiplexing) between clients in an event loop. This isn’t a socket issue per se, and we already learned how to start processes and threads in Chapter 5. But since these schemes are so typical of socket server programming, let’s explore all three ways to handle client requests in parallel here.
The script in Example 13-4 works like the
original echo
server, but instead
forks a new process to handle each new client connection. Because
the handleClient
function runs in
a new process, the dispatcher
function can immediately resume its main loop in order to detect and
service a new incoming request.
Example 13-4. PP3EInternetSocketsfork-server.py
############################################################################# # Server side: open a socket on a port, listen for a message from a client, # and send an echo reply; forks a process to handle each client connection; # child processes share parent's socket descriptors; fork is less portable # than threads--not yet on Windows, unless Cygwin or similar installed; ############################################################################# import os, time, sys from socket import * # get socket constructor and constants myHost = '' # server machine, '' means local host myPort = 50007 # listen on a non-reserved port number sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object sockobj.bind((myHost, myPort)) # bind it to server port number sockobj.listen(5) # allow 5 pending connects def now( ): # current time on server return time.ctime(time.time( )) activeChildren = [] def reapChildren( ): # reap any dead child processes while activeChildren: # else may fill up system table pid,stat = os.waitpid(0, os.WNOHANG) # don't hang if no child exited if not pid: break activeChildren.remove(pid) def handleClient(connection): # child process: reply, exit time.sleep(5) # simulate a blocking activity while True: # read, write a client socket data = connection.recv(1024) # till eof when socket closed if not data: break connection.send('Echo=>%s at %s' % (data, now( ))) connection.close( ) os._exit(0) def dispatcher( ): # listen until process killed while True: # wait for next connection, connection, address = sockobj.accept( ) # pass to process for service print 'Server connected by', address, print 'at', now( ) reapChildren( ) # clean up exited children now childPid = os.fork( ) # copy this process if childPid == 0: # if in child process: handle handleClient(connection) else: # else: go accept next connect activeChildren.append(childPid) # add to active child pid list dispatcher( )
Parts of this script are a bit tricky, and most of
its library calls work only on Unix-like platforms (not Windows).
But before we get into too many details, let’s start up our server
and handle a few client requests. First, notice that to simulate a long-running
operation (e.g., database updates, other network traffic), this
server adds a five-second time.sleep
delay in its client handler
function, handleClient
. After
the delay, the original echo reply action is performed. That means
that when we run a server and clients this time, clients won’t
receive the echo reply until five seconds after they’ve sent their
requests to the server.
To help keep track of requests and replies, the server prints its system time each time a client connect request is received, and adds its system time to the reply. Clients print the reply time sent back from the server, not their own—clocks on the server and client may differ radically, so to compare apples to apples, all times are server times. Because of the simulated delays, we also must usually start each client in its own console window on Windows (on some platforms, clients will hang in a blocked state while waiting for their reply).
But the grander story here is that this script runs one main
parent process on the server machine, which does nothing but watch
for connections (in dispatcher
), plus one child process per
active client connection, running in parallel with both the main
parent process and the other client processes (in handleClient
). In principle, the server
can handle any number of clients without bogging down. To test,
let’s start the server remotely in a Telnet window, and start
three clients locally in three distinct console windows:
[server telnet window] [lutz@starship uploads]$uname -a
Linux starship ... [lutz@starship uploads]$python fork-server.py
Server connected by ('38.28.162.194', 1063) at Sun Jun 18 19:37:49 2000 Server connected by ('38.28.162.194', 1064) at Sun Jun 18 19:37:49 2000 Server connected by ('38.28.162.194', 1067) at Sun Jun 18 19:37:50 2000 [client window 1] C:...PP3EInternetSockets>python echo-client.py starship.python.net
Client received: 'Echo=>Hello network world at Sun Jun 18 19:37:54 2000' [client window 2] C:...PP3EInternetSockets>python echo-client.py starship.python.net Bruce
Client received: 'Echo=>Bruce at Sun Jun 18 19:37:54 2000' [client window 3] C:...PP3EInternetSockets>python echo-client.py starship.python.net The
Meaning of Life
Client received: 'Echo=>The at Sun Jun 18 19:37:55 2000' Client received: 'Echo=>Meaning at Sun Jun 18 19:37:56 2000' Client received: 'Echo=>of at Sun Jun 18 19:37:56 2000' Client received: 'Echo=>Life at Sun Jun 18 19:37:56 2000'
Again, all times here are on the server machine. This may be a little confusing because four windows are involved. In English, the test proceeds as follows:
The server starts running remotely.
All three clients are started and connect to the server at roughly the same time.
On the server, the client requests trigger three forked child processes, which all immediately go to sleep for five seconds (to simulate being busy doing something useful).
Each client waits until the server replies, which eventually happens five seconds after their initial requests.
In other words, all three clients are serviced at the same time by forked processes, while the main parent process continues listening for new client requests. If clients were not handled in parallel like this, no client could connect until the currently connected client’s five-second delay expired.
In a more realistic application, that delay could be fatal
if many clients were trying to connect at once—the server would be
stuck in the action we’re simulating with time.sleep
, and not get back to the main
loop to accept
new client
requests. With process forks per request, all clients can be
serviced in parallel.
Notice that we’re using the same client script here
(echo-client.py), just a different server;
clients simply send and receive data to a machine and port and
don’t care how their requests are handled on the server. Also note
that the server is running remotely on a Linux machine. (As we
learned in Chapter 5, the
fork
call is not supported on
Windows in standard Python at the time this book was written.) We
can also run this test on a Linux server entirely, with two Telnet
windows. It works about the same as when clients are started
locally, in a DOS console window, but here “local” means a remote
machine you’re telnetting to locally:
[one Telnet window] [lutz@starship uploads]$python fork-server.py &
[1] 3379 Server connected by ('127.0.0.1', 2928) at Sun Jun 18 22:44:50 2000 Server connected by ('127.0.0.1', 2929) at Sun Jun 18 22:45:08 2000 Server connected by ('208.185.174.112', 2930) at Sun Jun 18 22:45:50 2000 [another Telnet window, same machine] [lutz@starship uploads]$python echo-client.py
Client received: 'Echo=>Hello network world at Sun Jun 18 22:44:55 2000' [lutz@starship uploads]$python echo-client.py localhost niNiNI
Client received: 'Echo=>niNiNI at Sun Jun 18 22:45:13 2000' [lutz@starship uploads]$python echo-client.py starship.python.net Say no More!
Client received: 'Echo=>Say at Sun Jun 18 22:45:55 2000' Client received: 'Echo=>no at Sun Jun 18 22:45:55 2000' Client received: 'Echo=>More! at Sun Jun 18 22:45:55 2000'
Now let’s move on to the tricky bits. This server script is fairly straightforward as forking code goes, but a few comments about some of the library tools it employs are in order.
We met os.fork
in
Chapter 5, but recall that
forked processes are essentially a copy of the process that forks
them, and so they inherit file and socket descriptors from their
parent process. As a result, the new child process that runs the
handleClient
function has
access to the connection socket created in the parent process.
Programs know they are in a forked child process if the fork call
returns 0; otherwise, the original parent process gets back the
new child’s ID.
In earlier fork examples, child processes usually call one
of the exec
variants to start a
new program in the child process. Here, instead, the child process
simply calls a function in the same program and exits with
os._exit
. It’s imperative to
call os._exit
here—if we did
not, each child would live on after handleClient
returns, and compete for
accepting new client requests.
In fact, without the exit call, we’d wind up with as many
perpetual server processes as requests served—remove the exit call
and do a ps
shell command after
running a few clients, and you’ll see what I mean. With the call,
only the single parent process listens for new requests. os._exit
is like sys.exit
, but it exits the calling
process immediately without cleanup actions. It’s normally used
only in child processes, and sys.exit
is used everywhere else.
Note, however, that it’s not quite enough to make
sure that child processes exit and die. On systems like Linux,
parents must also be sure to issue a wait
system call to remove the entries
for dead child processes from the system’s process table. If we
don’t do this, the child processes will no longer run, but they
will consume an entry in the system process table. For
long-running servers, these bogus entries may become
problematic.
It’s common to call such dead-but-listed child processes
zombies: they continue to use system
resources even though they’ve already passed over to the great
operating system beyond. To clean up after child processes are
gone, this server keeps a list, activeChildren
, of the process IDs of
all child processes it spawns. Whenever a new incoming client
request is received, the server runs its reapChildren
to issue a wait
for any dead children by issuing
the standard Python os.waitpid(0,os.WNOHANG)
call.
The os.waitpid
call
attempts to wait for a child process to exit and returns its
process ID and exit status. With a 0
for its first argument, it waits for
any child process. With the WNOHANG
parameter for its second, it
does nothing if no child process has exited (i.e., it does not
block or pause the caller). The net effect is that this call
simply asks the operating system for the process ID of any child
that has exited. If any have, the process ID returned is removed
both from the system process table and from this script’s activeChildren
list.
To see why all this complexity is needed, comment out the
reapChildren
call in this
script, run it on a server, and then run a few clients. On my
Linux server, a ps -f
full
process listing command shows that all the dead child processes
stay in the system process table (show as <defunct>
):
[lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD
lutz 3270 3264 0 22:33 pts/1 00:00:00 -bash
lutz 3311 3270 0 22:37 pts/1 00:00:00 python fork-server.py
lutz 3312 3311 0 22:37 pts/1 00:00:00 [python <defunct>]
lutz 3313 3311 0 22:37 pts/1 00:00:00 [python <defunct>]
lutz 3314 3311 0 22:37 pts/1 00:00:00 [python <defunct>]
lutz 3316 3311 0 22:37 pts/1 00:00:00 [python <defunct>]
lutz 3317 3311 0 22:37 pts/1 00:00:00 [python <defunct>]
lutz 3318 3311 0 22:37 pts/1 00:00:00 [python <defunct>]
lutz 3322 3270 0 22:38 pts/1 00:00:00 ps -f
When the reapChildren
command is reactivated, dead child zombie entries are cleaned up
each time the server gets a new client connection request, by
calling the Python os.waitpid
function. A few zombies may accumulate if the server is heavily
loaded, but they will remain only until the next client connection
is received:
[lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 3270 3264 0 22:33 pts/1 00:00:00 -bash lutz 3340 3270 0 22:41 pts/1 00:00:00 python fork-server.py lutz 3341 3340 0 22:41 pts/1 00:00:00 [python <defunct>] lutz 3342 3340 0 22:41 pts/1 00:00:00 [python <defunct>] lutz 3343 3340 0 22:41 pts/1 00:00:00 [python <defunct>] lutz 3344 3270 6 22:41 pts/1 00:00:00 ps -f [lutz@starship uploads]$ Server connected by ('38.28.131.174', 1170) at Sun Jun 18 22:41:43 2000 [lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 3270 3264 0 22:33 pts/1 00:00:00 -bash lutz 3340 3270 0 22:41 pts/1 00:00:00 python fork-server.py lutz 3345 3340 0 22:41 pts/1 00:00:00 [python <defunct>] lutz 3346 3270 0 22:41 pts/1 00:00:00 ps -f
If you type fast enough, you can actually see a child
process morph from a real running program into a zombie. Here, for
example, a child spawned to handle a new request (process ID
11785) changes to <defunct>
on exit. Its process
entry will be removed completely when the next request is
received:
[lutz@starship uploads]$ Server connected by ('38.28.57.160', 1106) at Mon Jun 19 22:34:39 2000 [lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11780 11089 0 22:34 pts/2 00:00:00 python fork-server.py lutz 11785 11780 0 22:34 pts/2 00:00:00 python fork-server.py lutz 11786 11089 0 22:34 pts/2 00:00:00 ps -f [lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11780 11089 0 22:34 pts/2 00:00:00 python fork-server.py lutz 11785 11780 0 22:34 pts/2 00:00:00 [python <defunct>] lutz 11787 11089 0 22:34 pts/2 00:00:00 ps -f
On some systems, it’s also possible to clean up
zombie child processes by resetting the signal handler for the
SIGCHLD
signal raised by the
operating system when a child process exits. If a Python script
assigns the SIG_IGN
(ignore)
action as the SIGCHLD
signal
handler, zombies will be removed automatically and immediately as
child processes exit; the parent need not issue wait calls to
clean up after them. Because of that, this scheme is a simpler
alternative to manually reaping zombies (on platforms where it is
supported).
If you’ve already read Chapter
5, you know that Python’s standard signal
module lets scripts install
handlers for signals—software-generated events. If you haven’t
read that chapter, here is a brief bit of background to show how
this pans out for zombies. The program in Example 13-5 installs a
Python-coded signal handler function to respond to whatever signal
number you type on the command line.
Example 13-5. PP3EInternetSocketssignal-demo.py
############################################################################# # Demo Python's signal module; pass signal number as a command-line arg, # use a "kill -N pid" shell command to send this process a signal; e.g., # on my Linux machine, SIGUSR1=10, SIGUSR2=12, SIGCHLD=17, and SIGCHLD # handler stays in effect even if not restored: all other handlers restored # by Python after caught, but SIGCHLD behavior is left to the platform's # implementation; signal works on Windows but defines only a few signal # types; signals are not very portable in general; ############################################################################# import sys, signal, time def now( ): return time.ctime(time.time( )) def onSignal(signum, stackframe): # Python signal handler print 'Got signal', signum, 'at', now( ) # most handlers stay in effect if signum == signal.SIGCHLD: # but sigchld handler is not print 'sigchld caught' #signal.signal(signal.SIGCHLD, onSignal) signum = int(sys.argv[1]) signal.signal(signum, onSignal) # install signal handler while 1: signal.pause( ) # sleep waiting for signals
To run this script, simply put it in the background and send
it signals by typing the kill
-signal-number process-id
shell command line. Process
IDs are listed in the PID column of ps
command results. Here is this script
in action catching signal numbers 10 (reserved for general use)
and 9 (the unavoidable terminate signal):
[lutz@starship uploads]$python signal-demo.py 10 &
[1] 11297 [lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11297 11089 0 21:49 pts/2 00:00:00 python signal-demo.py 10 lutz 11298 11089 0 21:49 pts/2 00:00:00 ps -f [lutz@starship uploads]$kill -10 11297
Got signal 10 at Mon Jun 19 21:49:27 2000 [lutz@starship uploads]$kill -10 11297
Got signal 10 at Mon Jun 19 21:49:29 2000 [lutz@starship uploads]$kill -10 11297
Got signal 10 at Mon Jun 19 21:49:32 2000 [lutz@starship uploads]$kill -9 11297
[1]+ Killed python signal-demo.py 10
And here the script catches signal 17, which happens to be
SIGCHLD
on my Linux server.
Signal numbers vary from machine to machine, so you should
normally use their names, not their numbers. SIGCHLD
behavior may vary per platform
as well (see the signal
module’s library manual entry for more details):
[lutz@starship uploads]$python signal-demo.py 17 &
[1] 11320 [lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11320 11089 0 21:52 pts/2 00:00:00 python signal-demo.py 17 lutz 11321 11089 0 21:52 pts/2 00:00:00 ps -f [lutz@starship uploads]$kill -17 11320
Got signal 17 at Mon Jun 19 21:52:24 2000 [lutz@starship uploads] sigchld caught [lutz@starship uploads]$kill -17 11320
Got signal 17 at Mon Jun 19 21:52:27 2000 [lutz@starship uploads]$ sigchld caught
Now, to apply all of this to kill zombies, simply set the
SIGCHLD
signal handler to the
SIG_IGN
ignore handler action;
on systems where this assignment is supported, child processes
will be cleaned up when they exit. The forking server variant
shown in Example 13-6
uses this trick to manage its children.
Example 13-6. PP3EInternetSocketsfork-server-signal.py
#############################################################################
# Same as fork-server.py, but use the Python signal module to avoid keeping
# child zombie processes after they terminate, not an explicit loop before
# each new connection; SIG_IGN means ignore, and may not work with SIG_CHLD
# child exit signal on all platforms; on Linux, socket.accept cannot be
# interrupted with a signal;
#############################################################################
import os, time, sys, signal, signal
from socket import * # get socket constructor and constants
myHost = '' # server machine, '' means local host
myPort = 50007 # listen on a non-reserved port number
sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object
sockobj.bind((myHost, myPort)) # bind it to server port number
sockobj.listen(5) # up to 5 pending connectssignal.signal(signal.SIGCHLD, signal.SIG_IGN)
# avoid child zombie processes
def now( ): # time on server machine
return time.ctime(time.time( ))
def handleClient(connection): # child process replies, exits
time.sleep(5) # simulate a blocking activity
while True: # read, write a client socket
data = connection.recv(1024)
if not data: break
connection.send('Echo=>%s at %s' % (data, now( )))
connection.close( )
os._exit(0)
def dispatcher( ): # listen until process killed
while True: # wait for next connection,
connection, address = sockobj.accept( ) # pass to process for service
print 'Server connected by', address,
print 'at', now( )
childPid = os.fork( ) # copy this process
if childPid == 0: # if in child process: handle
handleClient(connection) # else: go accept next connect
dispatcher( )
Where applicable, this technique is:
Much simpler; we don’t need to manually track or reap child processes.
More accurate; it leaves no zombies temporarily between client requests.
In fact, only one line is dedicated to handling zombies
here: the signal.signal
call
near the top, to set the handler. Unfortunately, this version is
also even less portable than using os.fork
in the first place, because
signals may work slightly differently from platform to platform.
For instance, some platforms may not allow SIG_IGN
to be used as the SIGCHLD
action at all. On Linux systems,
though, this simpler forking server variant works like a
charm:
[lutz@starship uploads]$ Server connected by ('38.28.57.160', 1166) at Mon Jun 19 22:38:29 2000 [lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11827 11089 0 22:37 pts/2 00:00:00 python fork-server-signal.py lutz 11835 11827 0 22:38 pts/2 00:00:00 python fork-server-signal.py lutz 11836 11089 0 22:38 pts/2 00:00:00 ps -f [lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11827 11089 0 22:37 pts/2 00:00:00 python fork-server-signal.py lutz 11837 11089 0 22:38 pts/2 00:00:00 ps -f
Notice that in this version, the child process’s entry goes away as soon as it exits, even before a new client request is received; no “defunct” zombie ever appears. More dramatically, if we now start up the script we wrote earlier that spawns eight clients in parallel (testecho.py) to talk to this server, all appear on the server while running, but are removed immediately as they exit:
[lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11827 11089 0 22:37 pts/2 00:00:00 python fork-server-signal.py lutz 11839 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11840 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11841 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11842 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11843 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11844 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11845 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11846 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11848 11089 0 22:39 pts/2 00:00:00 ps -f [lutz@starship uploads]$ps -f
UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11827 11089 0 22:37 pts/2 00:00:00 python fork-server-signal.py lutz 11849 11089 0 22:39 pts/2 00:00:00 ps -f
The forking model just described works well on Unix-like platforms in general, but it suffers from some potentially significant limitations:
On some machines, starting a new process can be fairly expensive in terms of time and space resources.
Forking processes is a Unix technique; as we just noted, the fork call currently doesn’t work on non-Unix platforms such as Windows under standard Python. As we learned in Chapter 5, forks can be used in the Cygwin version of Python on Windows, but they may be inefficient and not exactly the same as Unix forks.
If you think that forking servers can be complicated, you’re not alone. As we just saw, forking also brings with it all the shenanigans of managing zombies—cleaning up after child processes that live shorter lives than their parents.
If you read Chapter 5, you know that one solution to all of these dilemmas is to use threads rather than processes. Threads run in parallel and share global (i.e., module and interpreter) memory, but they are usually less expensive to start, and work on both Unix-like machines and Microsoft Windows under standard Python today. Furthermore, some see threads as simpler to program—child threads die silently on exit, without leaving behind zombies to haunt the server.
Example 13-7 is another mutation of the echo server that handles client requests in parallel by running them in threads rather than in processes.
Example 13-7. PP3EInternetSockets hread-server.py
############################################################################# # Server side: open a socket on a port, listen for a message from a client, # and send an echo reply; echoes lines until eof when client closes socket; # spawns a thread to handle each client connection; threads share global # memory space with main thread; this is more portable than fork: threads # work on standard Windows systems, but process forks do not; ############################################################################# import thread, time from socket import * # get socket constructor and constants myHost = '' # server machine, '' means local host myPort = 50007 # listen on a non-reserved port number sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object sockobj.bind((myHost, myPort)) # bind it to server port number sockobj.listen(5) # allow up to 5 pending connects def now( ): return time.ctime(time.time( )) # current time on the server def handleClient(connection): # in spawned thread: reply time.sleep(5) # simulate a blocking activity while True: # read, write a client socket data = connection.recv(1024) if not data: break connection.send('Echo=>%s at %s' % (data, now( ))) connection.close( ) def dispatcher( ): # listen until process killed while True: # wait for next connection, connection, address = sockobj.accept( ) # pass to thread for service print 'Server connected by', address, print 'at', now( ) thread.start_new(handleClient, (connection,)) dispatcher( )
This dispatcher
delegates
each incoming client connection request to a newly spawned thread
running the handleClient
function. As a result, this server can process multiple clients at
once, and the main dispatcher loop can get quickly back to the top
to check for newly arrived requests. The net effect is that new
clients won’t be denied service due to a busy server.
Functionally, this version is similar to the fork
solution (clients are handled in
parallel), but it will work on any machine that supports threads,
including Windows and Linux. Let’s test it on both. First, start the
server on a Linux machine and run clients on both Linux and
Windows:
[window 1: thread-based server process, server keeps accepting client connections while threads are servicing prior requests] [lutz@starship uploads]$/usr/bin/python thread-server.py
Server connected by ('127.0.0.1', 2934) at Sun Jun 18 22:52:52 2000 Server connected by ('38.28.131.174', 1179) at Sun Jun 18 22:53:31 2000 Server connected by ('38.28.131.174', 1182) at Sun Jun 18 22:53:35 2000 Server connected by ('38.28.131.174', 1185) at Sun Jun 18 22:53:37 2000 [window 2: client, but on same server machine] [lutz@starship uploads]$python echo-client.py
Client received: 'Echo=>Hello network world at Sun Jun 18 22:52:57 2000' [window 3: remote client, PC] C:...PP3EInternetSockets>python echo-client.py starship.python.net
Client received: 'Echo=>Hello network world at Sun Jun 18 22:53:36 2000' [window 4: client PC] C:...PP3EInternetSockets>python echo-client.py starship.python.net Bruce
Client received: 'Echo=>Bruce at Sun Jun 18 22:53:40 2000' [window 5: client PC] C:...PP3EInternetSockets>python echo-client.py starship.python.net The
Meaning of Life
Client received: 'Echo=>The at Sun Jun 18 22:53:42 2000' Client received: 'Echo=>Meaning at Sun Jun 18 22:53:42 2000' Client received: 'Echo=>of at Sun Jun 18 22:53:42 2000' Client received: 'Echo=>Life at Sun Jun 18 22:53:42 2000'
Because this server uses threads rather than forked processes, we can run it portably on both Linux and a Windows PC. Here it is at work again, running on the same local Windows PC as its clients; again, the main point to notice is that new clients are accepted while prior clients are being processed in parallel with other clients and the main thread (in the five-second sleep delay):
[window 1: server, on local PC] C:...PP3EInternetSockets>python thread-server.py
Server connected by ('127.0.0.1', 1186) at Sun Jun 18 23:46:31 2000 Server connected by ('127.0.0.1', 1187) at Sun Jun 18 23:46:33 2000 Server connected by ('127.0.0.1', 1188) at Sun Jun 18 23:46:34 2000 [window 2: client, on local PC] C:...PP3EInternetSockets>python echo-client.py
Client received: 'Echo=>Hello network world at Sun Jun 18 23:46:36 2000' [window 3: client] C:...PP3EInternetSockets>python echo-client.py localhost Brian
Client received: 'Echo=>Brian at Sun Jun 18 23:46:38 2000' [window 4: client] C:...PP3EInternetSockets>python echo-client.py localhost Bright side of Life
Client received: 'Echo=>Bright at Sun Jun 18 23:46:39 2000' Client received: 'Echo=>side at Sun Jun 18 23:46:39 2000' Client received: 'Echo=>of at Sun Jun 18 23:46:39 2000' Client received: 'Echo=>Life at Sun Jun 18 23:46:39 2000'
Remember that a thread silently exits when the function it is
running returns; unlike the process fork
version, we don’t call anything like
os
._exit
in the client handler function (and
we shouldn’t—it may kill all threads in the process!). Because of
this, the thread version is not only more portable, but also
simpler.
Now that I’ve shown you how to write forking and
threading servers to process clients without blocking incoming
requests, I should also tell you that there are standard tools in
the Python library to make this process easier. In particular, the
SocketServer
module defines
classes that implement all flavors of forking and threading servers
that you are likely to be interested in. Simply create the desired
kind of imported server object, passing in a handler object with a
callback method of your own, as shown in Example 13-8.
Example 13-8. PP3EInternetSocketsclass-server.py
############################################################################# # Server side: open a socket on a port, listen for a message from a client, # and send an echo reply; this version uses the standard library module # SocketServer to do its work; SocketServer allows us to make a simple # TCPServer, a ThreadingTCPServer, a ForkingTCPServer, and more, and # routes each client connect request to a new instance of a passed-in # request handler object's handle method; SocketServer also supports # UDP and Unix domain sockets; see the library manual for other usage. ############################################################################# import SocketServer, time # get socket server, handler objects myHost = '' # server machine, '' means local host myPort = 50007 # listen on a non-reserved port number def now( ): return time.ctime(time.time( )) class MyClientHandler(SocketServer.BaseRequestHandler): def handle(self): # on each client connect print self.client_address, now( ) # show this client's address time.sleep(5) # simulate a blocking activity while True: # self.request is client socket data = self.request.recv(1024) # read, write a client socket if not data: break self.request.send('Echo=>%s at %s' % (data, now( ))) self.request.close( ) # make a threaded server, listen/handle clients forever myaddr = (myHost, myPort) server = SocketServer.ThreadingTCPServer(myaddr, MyClientHandler) server.serve_forever( )
This server works the same as the threading server we wrote by
hand in the previous section, but instead focuses on service
implementation (the customized handle
method), not on threading details.
It’s run the same way, too—here it is processing three clients
started by hand, plus eight spawned by the testecho
script shown in Example 13-3:
[window1: server, serverHost='localhost' in echo-client.py] C:...PP3EInternetSockets>python class-server.py
('127.0.0.1', 1189) Sun Jun 18 23:49:18 2000 ('127.0.0.1', 1190) Sun Jun 18 23:49:20 2000 ('127.0.0.1', 1191) Sun Jun 18 23:49:22 2000 ('127.0.0.1', 1192) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1193) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1194) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1195) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1196) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1197) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1198) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1199) Sun Jun 18 23:49:50 2000 [window2: client] C:...PP3EInternetSockets>python echo-client.py
Client received: 'Echo=>Hello network world at Sun Jun 18 23:49:23 2000' [window3: client] C:...PP3EInternetSockets>python echo-client.py localhost Robin
Client received: 'Echo=>Robin at Sun Jun 18 23:49:25 2000' [window4: client] C:...PP3EInternetSockets>python echo-client.py localhost Brave Sir Robin
Client received: 'Echo=>Brave at Sun Jun 18 23:49:27 2000' Client received: 'Echo=>Sir at Sun Jun 18 23:49:27 2000' Client received: 'Echo=>Robin at Sun Jun 18 23:49:27 2000' C:...PP3EInternetSockets>python testecho.py
[window4: contact remote server instead--times skewed] C:...PP3EInternetSockets>python echo-client.py starship.python.net Brave
Sir Robin Client received: 'Echo=>Brave at Sun Jun 18 23:03:28 2000' Client received: 'Echo=>Sir at Sun Jun 18 23:03:28 2000' Client received: 'Echo=>Robin at Sun Jun 18 23:03:29 2000'
To build a forking server instead, just use the class name
ForkingTCPServer
when creating
the server object. The SocketServer
module has more power than
shown by this example; it also supports synchronous (nonparallel)
servers, UDP and Unix sockets, and so on. See Python’s library
manual for more details. Also see the end of Chapter 18 for more on Python server
implementation tools.
For more advanced server needs, Python also comes with standard library tools that allow you to implement a full-blown HTTP (web) server that knows how to run server-side CGI scripts in a few lines of Python code. We’ll explore those tools in Chapter 16.
For other server options, see the open source Twisted system (http://twistedmatrix.com). Twisted is an asynchronous networking framework written in Python that supports TCP, UDP, multicast, SSL/TLS, serial communication, and more. It supports both clients and servers and includes implementations of a number of commonly used network services such as a web server, an IRC chat server, a mail server, a relational database interface, and an object broker.
Although Twisted supports processes and threads for longer-running actions, it also uses an asynchronous, event-driven model to handle clients, which is similar to the event loop of GUI libraries like Tkinter. In fact, it abstracts an event loop, which multiplexes among open socket connections—coincidentally, the topic of the next section.
So far we’ve seen how to handle multiple clients at once with both forked processes and spawned threads, and we’ve looked at a library class that encapsulates both schemes. Under both approaches, all client handlers seem to run in parallel with each other and with the main dispatch loop that continues watching for new incoming requests. Because all of these tasks run in parallel (i.e., at the same time), the server doesn’t get blocked when accepting new requests or when processing a long-running client handler.
Technically, though, threads and processes don’t really run in parallel, unless you’re lucky enough to have a machine with many CPUs. Instead, your operating system performs a juggling act—it divides the computer’s processing power among all active tasks. It runs part of one, then part of another, and so on. All the tasks appear to run in parallel, but only because the operating system switches focus between tasks so fast that you don’t usually notice. This process of switching between tasks is sometimes called time-slicing when done by an operating system; it is more generally known as multiplexing.
When we spawn threads and processes, we rely on the operating system to juggle the active tasks, but there’s no reason that a Python script can’t do so as well. For instance, a script might divide tasks into multiple steps—do a step of one task, then one of another, and so on, until all are completed. The script need only know how to divide its attention among the multiple active tasks to multiplex on its own.
Servers can apply this technique to yield yet another way to
handle multiple clients at once, a way that requires neither threads
nor forks. By multiplexing client connections and the main
dispatcher with the select
system
call, a single event loop can process clients and accept new ones in
parallel (or at least close enough to avoid stalling). Such servers
are sometimes called asynchronous, because they
service clients in spurts, as each becomes ready to communicate. In
asynchronous servers, a single main loop run in a single process and
thread decides which clients should get a bit of attention each time
through. Client requests and the main dispatcher are each given a
small slice of the server’s attention if they are ready to
converse.
Most of the magic behind this server structure is the
operating system select
call,
available in Python’s standard select
module. Roughly, select
is asked to monitor a list of input
sources, output sources, and exceptional condition sources and tells
us which sources are ready for processing. It can be made to simply
poll all the sources to see which are ready; wait for a maximum time
period for sources to become ready; or wait indefinitely until one
or more sources are ready for processing.
However used, select
lets
us direct attention to sockets ready to communicate, so as to avoid
blocking on calls to ones that are not. That is, when the sources
passed to select
are sockets, we
can be sure that socket calls like accept
, recv
, and send
will not block (pause) the server
when applied to objects returned by select
. Because of that, a single-loop
server that uses select
need not
get stuck communicating with one client or waiting for new ones
while other clients are starved for the server’s attention.
Let’s see how all of this translates into code. The
script in Example 13-9
implements another echo
server,
one that can handle multiple clients without ever starting new
processes or threads.
Example 13-9. PP3EInternetSocketsselect-server.py
############################################################################# # Server: handle multiple clients in parallel with select. use the select # module to manually multiplex among a set of sockets: main sockets which # accept new client connections, and input sockets connected to accepted # clients; select can take an optional 4th arg--0 to poll, n.m to wait n.m # seconds, or ommitted to wait till any socket is ready for processing. ############################################################################# import sys, time from select import select from socket import socket, AF_INET, SOCK_STREAM def now(): return time.ctime(time.time( )) myHost = '' # server machine, '' means local host myPort = 50007 # listen on a non-reserved port number if len(sys.argv) == 3: # allow host/port as cmdline args too myHost, myPort = sys.argv[1:] numPortSocks = 2 # number of ports for client connects # make main sockets for accepting new client requests mainsocks, readsocks, writesocks = [], [], [] for i in range(numPortSocks): portsock = socket(AF_INET, SOCK_STREAM) # make a TCP/IP spocket object portsock.bind((myHost, myPort)) # bind it to server port number portsock.listen(5) # listen, allow 5 pending connects mainsocks.append(portsock) # add to main list to identify readsocks.append(portsock) # add to select inputs list myPort += 1 # bind on consecutive ports # event loop: listen and multiplex until server process killed print 'select-server loop starting' while True: #print readsocks readables, writeables, exceptions = select(readsocks, writesocks, []) for sockobj in readables: if sockobj in mainsocks: # for ready input sockets # port socket: accept new client newsock, address = sockobj.accept( ) # accept should not block print 'Connect:', address, id(newsock) # newsock is a new socket readsocks.append(newsock) # add to select list, wait else: # client socket: read next line data = sockobj.recv(1024) # recv should not block print ' got', data, 'on', id(sockobj) if not data: # if closed by the clients sockobj.close( ) # close here and remv from readsocks.remove(sockobj) # del list else reselected else: # this may block: should really select for writes too sockobj.send('Echo=>%s at %s' % (data, now( )))
The bulk of this script is the big while
event loop at the end that calls
select
to find out which
sockets are ready for processing (these include main port sockets
on which clients can connect, and open client connections). It
then loops over all such ready sockets, accepting connections on
main port sockets and reading and echoing input on any client
sockets ready for input. Both the accept
and recv
calls in this code are guaranteed
to not block the server process after select
returns; as a result, this server
can quickly get back to the top of the loop to process newly
arrived client requests and already connected clients’ inputs. The
net effect is that all new requests and clients are serviced in
pseudoparallel fashion.
To make this process work, the server appends the connected
socket for each client to the readables
list passed to select
, and simply waits for the socket
to show up in the selected inputs list. For illustration purposes,
this server also listens for new clients on more than one port—on
ports 50007 and 50008, in our examples. Because these main port
sockets are also interrogated with select
, connection requests on either
port can be accepted without blocking either already connected
clients or new connection requests appearing on the other port.
The select
call returns
whatever sockets in readables
are ready for processing—both main port sockets and sockets
connected to clients currently being processed.
Let’s run this script locally to see how it does its
stuff (the client and server can also be run on different
machines, as in prior socket examples). First, we’ll assume we’ve
already started this server script in one window, and run a few
clients to talk to it. The following code is the interaction in
two such client windows running on Windows (MS-DOS consoles). The
first client simply runs the echo-client
script twice to contact the
server, and the second also kicks off the testecho
script to spawn eight echo-client
programs running in
parallel. As before, the server simply echoes back whatever text
that client sends. Notice that the second client window really
runs a script called echo-client-50008
so as to connect to
the second port socket in the server; it’s the same as echo-client
, with a different port
number (alas, the original script wasn’t designed to input a
port):
[client window 1] C:...PP3EInternetSockets>python echo-client.py
Client received: 'Echo=>Hello network world at Sun Aug 13 22:52:01 2000' C:...PP3EInternetSockets>python echo-client.py
Client received: 'Echo=>Hello network world at Sun Aug 13 22:52:03 2000' [client window 2] C:...PP3EInternetSockets>python echo-client-50008.py localhost Sir Lancelot
Client received: 'Echo=>Sir at Sun Aug 13 22:52:57 2000' Client received: 'Echo=>Lancelot at Sun Aug 13 22:52:57 2000' C:...PP3EInternetSockets>python testecho.py
The next code section is the sort of interaction and output
that show up in the window where the server has been started. The
first three connections come from echo-client
runs; the rest is the result
of the eight programs spawned by testecho
in the second client window.
Notice that for testecho
, new
client connections and client inputs are multiplexed together. If
you study the output closely, you’ll see that they overlap in
time, because all activity is dispatched by the single event loop
in the server.[*] Also note that the server gets an empty string when
the client has closed its socket. We take care to close and delete
these sockets at the server right away, or else they would be
needlessly reselected again and again, each time through the main
loop:
[server window]
C:...PP3EInternetSockets>python select-server.py
select-server loop starting
Connect: ('127.0.0.1', 1175) 7965520
got Hello network world on 7965520
got on 7965520
Connect: ('127.0.0.1', 1176) 7964288
got Hello network world on 7964288
got on 7964288
Connect: ('127.0.0.1', 1177) 7963920
got Sir on 7963920
got Lancelot on 7963920
got on 7963920
[testecho results]
Connect: ('127.0.0.1', 1178) 7965216
got Hello network world on 7965216
got on 7965216
Connect: ('127.0.0.1', 1179) 7963968
Connect: ('127.0.0.1', 1180) 7965424
got Hello network world on 7963968
Connect: ('127.0.0.1', 1181) 7962976
got Hello network world on 7965424
got on 7963968
got Hello network world on 7962976
got on 7965424
got on 7962976
Connect: ('127.0.0.1', 1182) 7963648
got Hello network world on 7963648
got on 7963648
Connect: ('127.0.0.1', 1183) 7966640
got Hello network world on 7966640
got on 7966640
Connect: ('127.0.0.1', 1184) 7966496
got Hello network world on 7966496
got on 7966496
Connect: ('127.0.0.1', 1185) 7965888
got Hello network world on 7965888
got on 7965888
A subtle but crucial point: a time.sleep
call to simulate a
long-running task doesn’t make sense in the server here—because
all clients are handled by the same single loop, sleeping would
pause everything (and defeat the whole point of a multiplexing
server). Here are a few additional notes before we move on:
select
call
detailsFormally, select
is
called with three lists of selectable objects (input
sources, output sources, and exceptional condition sources),
plus an optional timeout. The timeout argument may be a real
wait expiration value in seconds (use floating-point numbers
to express fractions of a second), a zero value to mean
simply poll and return immediately, or omitted to mean wait
until at least one object is ready (as done in our server
script earlier). The call returns a triple of ready
objects—subsets of the first three arguments—any or all of
which may be empty if the timeout expired before sources
became ready.
select
portabilityThe select
call
works only for sockets on Windows, but also works for things
like files and pipes on Unix and Macintosh. For servers
running over the Internet, of course, the primary devices we
are interested in are sockets.
select
lets us be
sure that socket calls like accept
and recv
won’t block (pause) the
caller, but it’s also possible to make Python sockets
nonblocking in general. Call the setblocking
method of socket
objects to set the socket to blocking or nonblocking mode.
For example, given a call like sock.setblocking(flag)
, the socket
sock
is set to
nonblocking mode if the flag is zero, and to blocking mode
otherwise. All sockets start out in blocking mode initially,
so socket calls may always make the caller wait.
But when in nonblocking mode, a socket.error
exception is raised
if a recv
socket call
doesn’t find any data, or if a send
call can’t immediately
transfer data. A script can catch this exception to
determine whether the socket is ready for processing. In
blocking mode, these calls always block until they can
proceed. Of course, there may be much more to processing
client requests than data transfers (requests may also
require long-running computations), so nonblocking sockets
don’t guarantee that servers won’t stall in general. They
are simply another way to code multiplexing servers. Like
select
, they are better
suited when client requests can be serviced quickly.
asyncore
module
frameworkIf you’re interested in using select
, you will probably also be
interested in checking out the asyncore.py
module in the standard
Python library. It implements a class-based callback model,
where input and output callbacks are dispatched to class
methods by a precoded select
event loop. As such, it
allows servers to be constructed without threads or forks,
and is a select
-based
alternative to the SocketServer
threading and forking
module we met in the prior sections. We’ll discuss this tool
again at the end of Chapter
18; see the Python library manual for details and a
usage example.
The Twisted system, described in the prior section,
abstracts away many of the details inherent in an
asynchronous server and provides an event-driven model and
framework. Twisted’s internal event engine is similar in
spirit to our select
-based server and the
asyncore
module, but it
is regarded as more advanced. Twisted is a third-party
system, not a standard library tool.
So when should you use select
to build a server, instead of
threads or forks? Needs vary per application, of course, but servers
based on the select
call
generally perform very well when client transactions are relatively
short and are not CPU-bound. If they are not short, threads or forks
may be a better way to split processing among multiple clients.
Threads and forks are especially useful if clients require
long-running processing above and beyond socket calls. However,
combinations are possible too—nothing is stopping a select-based
polling loop from using threads, too.
It’s important to remember that schemes based on select
(and nonblocking sockets) are not
completely immune to blocking. In the example earlier, for instance,
the send
call that echoes text
back to a client might block too, and hence stall the entire server.
We could work around that blocking potential by using select
to make sure that the output
operation is ready before we attempt it (e.g., use the writesocks
list and add another loop to
send replies to ready output sockets), albeit at a noticeable cost
in program clarity.
In general, though, if we cannot split up the processing of a
client’s request in such a way that it can be multiplexed with other
requests and not block the server’s loop, select
may not be the best way to
construct the server.
Moreover, select
also seems
more complex than spawning either processes or threads, because we
need to manually transfer control among all tasks (for instance,
compare the threaded and select
versions of this server, even without write selects). As usual,
though, the degree of that complexity varies per application. The
asyncore standard library module mentioned
earlier simplifies some of the tasks of implementing a select
-based event-loop socket
server.
[*] And the trace output on the server will probably look a bit different every time it runs. Clients and new connections are interleaved almost at random due to timing differences on the host machines.
3.141.38.121