Launching a separate thread or process per client may not be viable in any larger network server application where several hundred or thousand clients are concurrently connected to the server. Due to the limited available memory and host CPU power, we need a better technique to deal with large number of clients. Fortunately, Python provides the select
module to overcome this problem.
We need to write an efficient chat server that can handle several hundred or a large number of client connections. We will use the select()
method from the select
module that will enable our chat server and client to do any task without blocking a send or receive call all the time.
Let us design this recipe such that a single script can launch both client and server with an additional --name
argument. Only if --name=server
is passed from the command line, the script will launch the chat server. Any other value passed to the --name
argument, for example, client1
, client2
, will launch a chat client. Let's specify our char server port number from the command line using --port
argument. For a larger application, it may be preferable to write separate modules for the server and client.
Listing 2.3 shows an example of chat application using select.select
as follows:
#!/usr/bin/env python # Python Network Programming Cookbook -- Chapter - 2 # This program is optimized for Python 2.7 # It may run on any other version with/without modifications import select import socket import sys import signal import cPickle import struct import argparse SERVER_HOST = 'localhost' CHAT_SERVER_NAME = 'server' # Some utilities def send(channel, *args): buffer = cPickle.dumps(args) value = socket.htonl(len(buffer)) size = struct.pack("L",value) channel.send(size) channel.send(buffer) def receive(channel): size = struct.calcsize("L") size = channel.recv(size) try: size = socket.ntohl(struct.unpack("L", size)[0]) except struct.error, e: return '' buf = "" while len(buf) < size: buf = channel.recv(size - len(buf)) return cPickle.loads(buf)[0]
The send()
method takes one named argument channel and positional argument *args
. It serializes the data using the dumps()
method from the cPickle
module. It determines the size of the data using the struct
module. Similarly, receive()
takes one named argument channel
.
Now we can code the ChatServer
class as follows:
class ChatServer(object): """ An example chat server using select """ def __init__(self, port, backlog=5): self.clients = 0 self.clientmap = {} self.outputs = [] # list output sockets self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # Enable re-using socket address self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) self.server.bind((SERVER_HOST, port)) print 'Server listening to port: %s ...' %port self.server.listen(backlog) # Catch keyboard interrupts signal.signal(signal.SIGINT, self.sighandler) def sighandler(self, signum, frame): """ Clean up client outputs""" # Close the server print 'Shutting down server...' # Close existing client sockets for output in self.outputs: output.close() self.server.close() def get_client_name(self, client): """ Return the name of the client """ info = self.clientmap[client] host, name = info[0][0], info[1] return '@'.join((name, host))
Now the main executable method of the ChatServer
class should look like the following code:
def run(self): inputs = [self.server, sys.stdin] self.outputs = [] running = True while running: try: readable, writeable, exceptional = select.select(inputs, self.outputs, []) except select.error, e: break for sock in readable: if sock == self.server: # handle the server socket client, address = self.server.accept() print "Chat server: got connection %d from %s" % (client.fileno(), address) # Read the login name cname = receive(client).split('NAME: ')[1] # Compute client name and send back self.clients += 1 send(client, 'CLIENT: ' + str(address[0])) inputs.append(client) self.clientmap[client] = (address, cname) # Send joining information to other clients msg = " (Connected: New client (%d) from %s)" % (self.clients, self.get_client_name(client)) for output in self.outputs: send(output, msg) self.outputs.append(client) elif sock == sys.stdin: # handle standard input junk = sys.stdin.readline() running = False else: # handle all other sockets try: data = receive(sock) if data: # Send as new client's message... msg = ' #[' + self.get_client_name(sock) + ']>>' + data # Send data to all except ourself for output in self.outputs: if output != sock: send(output, msg) else: print "Chat server: %d hung up" % sock.fileno() self.clients -= 1 sock.close() inputs.remove(sock) self.outputs.remove(sock) # Sending client leaving info to others msg = " (Now hung up: Client from %s)" % self.get_client_name(sock) for output in self.outputs: send(output, msg) except socket.error, e: # Remove inputs.remove(sock) self.outputs.remove(sock) self.server.close()
The chat server initializes with a few data attributes. It stores the count of clients, map of each client, and output sockets. The usual server socket creation also sets the option to reuse an address so that there is no problem restarting the server again using the same port. An optional backlog argument to the chat server constructor sets the maximum number of queued connections to listen by the server.
An interesting aspect of this chat server is to catch the user interrupt, usually via keyboard, using the signal
module. So a signal handler sighandler
is registered for the interrupt signal (SIGINT
). This signal handler catches the keyboard interrupt signal and closes all output sockets where data may be waiting to be sent.
The main executive method of our chat server run()
performs its operation inside a while
loop. This method registers with a select interface where the input argument is the chat server socket, stdin
. The output argument is specified by the server's output socket list. In return, select
provides three lists: readable, writable, and exceptional sockets. The chat server is only interested in readable sockets where some data is ready to be read. If that socket indicates to itself, then that will mean a new client connection has been established. So the server retrieves the client's name and broadcasts this information to other clients. In another case, if anything comes from the input arguments, the chat server exits. Similarly, the chat server deals with the other client's socket inputs. It relays the data received from any client to others and also shares their joining/leaving information.
The chat client code class should contain the following code:
class ChatClient(object): """ A command line chat client using select """ def __init__(self, name, port, host=SERVER_HOST): self.name = name self.connected = False self.host = host self.port = port # Initial prompt self.prompt='[' + '@'.join((name, socket.gethostname().split('.')[0])) + ']> ' # Connect to server at port try: self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.sock.connect((host, self.port)) print "Now connected to chat server@ port %d" % self.port self.connected = True # Send my name... send(self.sock,'NAME: ' + self.name) data = receive(self.sock) # Contains client address, set it addr = data.split('CLIENT: ')[1] self.prompt = '[' + '@'.join((self.name, addr)) + ']> ' except socket.error, e: print "Failed to connect to chat server @ port %d" % self.port sys.exit(1) def run(self): """ Chat client main loop """ while self.connected: try: sys.stdout.write(self.prompt) sys.stdout.flush() # Wait for input from stdin and socket readable, writeable,exceptional = select.select([0, self.sock], [],[]) for sock in readable: if sock == 0: data = sys.stdin.readline().strip() if data: send(self.sock, data) elif sock == self.sock: data = receive(self.sock) if not data: print 'Client shutting down.' self.connected = False break else: sys.stdout.write(data + ' ') sys.stdout.flush() except KeyboardInterrupt: print " Client interrupted. """ self.sock.close() break
The chat client initializes with a name argument and sends this name to the chat server upon connecting. It also sets up a custom prompt [ name@host ]>
. The executive method of this client run()
continues its operation as long as the connection to the server is active. In a manner similar to the chat server, the chat client also registers with select()
. If anything in readable sockets is ready, it enables the client to receive data. If the sock value is 0
and there's any data available then the data can be sent. The same information is also shown in stdout or, in our case, the command-line console. Our main method should now get command-line arguments and call either the server or client as follows:
if __name__ == "__main__": parser = argparse.ArgumentParser(description='Socket Server Example with Select') parser.add_argument('--name', action="store", dest="name", required=True) parser.add_argument('--port', action="store", dest="port", type=int, required=True) given_args = parser.parse_args() port = given_args.port name = given_args.name if name == CHAT_SERVER_NAME: server = ChatServer(port) server.run() else: client = ChatClient(name=name, port=port) client.run()
We would like to run this script thrice: once for the chat server and twice for two chat clients. For the server, we pass –name=server
and port=8800
. For client1
, we change the name argument --name=client1
and for client2
, we put --name=client2
. Then from the client1
value prompt we send the message "Hello from client 1"
, which is printed in the prompt of the client2
. Similarly, we send "hello from client 2
" from the prompt of the client2
, which is shown in the prompt of the client1
.
The output for the server is as follows:
$ python 2_3_chat_server_with_select.py --name=server --port=8800 Server listening to port: 8800 ... Chat server: got connection 4 from ('127.0.0.1', 56565) Chat server: got connection 5 from ('127.0.0.1', 56566)
The output for client1
is as follows:
$ python 2_3_chat_server_with_select.py --name=client1 --port=8800 Now connected to chat server@ port 8800 [[email protected]]> (Connected: New client (2) from [email protected]) [[email protected]]> Hello from client 1 [[email protected]]> #[[email protected]]>>hello from client 2
The output for client2
is as follows:
$ python 2_3_chat_server_with_select.py --name=client2 --port=8800 Now connected to chat server@ port 8800 [[email protected]]> #[[email protected]]>>Hello from client 1 [[email protected]]> hello from client 2 [[email protected]]
The whole interaction is shown in the following screenshot:
3.145.55.198