Launching a separate thread or process per client may not be viable in any larger network server application where several hundred or thousand clients are concurrently connected to the server. Due to the limited available memory and host CPU power, we need a better technique to deal with large number of clients. Fortunately, Python provides the select module to overcome this problem.
We need to write an efficient chat server that can handle several hundred or a large number of client connections. We will use the select() method from the select module that will enable our chat server and client to do any task without blocking a send or receive call all the time.
Let us design this recipe such that a single script can launch both client and server with an additional --name argument. Only if --name=server is passed from the command line, the script will launch the chat server. Any other value passed to the --name argument, for example, client1, client2, will launch a chat client. Let's specify our char server port number from the command line using --port argument. For a larger application, it may be preferable to write separate modules for the server and client.
Listing 2.3 shows an example of chat application using select.select as follows:
#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter - 2
# This program is optimized for Python 2.7
# It may run on any other version with/without modifications
import select
import socket
import sys
import signal
import cPickle
import struct
import argparse
SERVER_HOST = 'localhost'
CHAT_SERVER_NAME = 'server'
# Some utilities
def send(channel, *args):
buffer = cPickle.dumps(args)
value = socket.htonl(len(buffer))
size = struct.pack("L",value)
channel.send(size)
channel.send(buffer)
def receive(channel):
size = struct.calcsize("L")
size = channel.recv(size)
try:
size = socket.ntohl(struct.unpack("L", size)[0])
except struct.error, e:
return ''
buf = ""
while len(buf) < size:
buf = channel.recv(size - len(buf))
return cPickle.loads(buf)[0]The send() method takes one named argument channel and positional argument *args. It serializes the data using the dumps() method from the cPickle module. It determines the size of the data using the struct module. Similarly, receive() takes one named argument channel.
Now we can code the ChatServer class as follows:
class ChatServer(object):
""" An example chat server using select """
def __init__(self, port, backlog=5):
self.clients = 0
self.clientmap = {}
self.outputs = [] # list output sockets
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Enable re-using socket address
self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.server.bind((SERVER_HOST, port))
print 'Server listening to port: %s ...' %port
self.server.listen(backlog)
# Catch keyboard interrupts
signal.signal(signal.SIGINT, self.sighandler)
def sighandler(self, signum, frame):
""" Clean up client outputs"""
# Close the server
print 'Shutting down server...'
# Close existing client sockets
for output in self.outputs:
output.close()
self.server.close()
def get_client_name(self, client):
""" Return the name of the client """
info = self.clientmap[client]
host, name = info[0][0], info[1]
return '@'.join((name, host))Now the main executable method of the ChatServer class should look like the following code:
def run(self):
inputs = [self.server, sys.stdin]
self.outputs = []
running = True
while running:
try:
readable, writeable, exceptional =
select.select(inputs, self.outputs, [])
except select.error, e:
break
for sock in readable:
if sock == self.server:
# handle the server socket
client, address = self.server.accept()
print "Chat server: got connection %d from %s" % (client.fileno(), address)
# Read the login name
cname = receive(client).split('NAME: ')[1]
# Compute client name and send back
self.clients += 1
send(client, 'CLIENT: ' + str(address[0]))
inputs.append(client)
self.clientmap[client] = (address, cname)
# Send joining information to other clients
msg = "
(Connected: New client (%d) from %s)" % (self.clients, self.get_client_name(client))
for output in self.outputs:
send(output, msg)
self.outputs.append(client)
elif sock == sys.stdin:
# handle standard input
junk = sys.stdin.readline()
running = False
else:
# handle all other sockets
try:
data = receive(sock)
if data:
# Send as new client's message...
msg = '
#[' + self.get_client_name(sock)
+ ']>>' + data
# Send data to all except ourself
for output in self.outputs:
if output != sock:
send(output, msg)
else:
print "Chat server: %d hung up" %
sock.fileno()
self.clients -= 1
sock.close()
inputs.remove(sock)
self.outputs.remove(sock)
# Sending client leaving info to others
msg = "
(Now hung up: Client from %s)" % self.get_client_name(sock)
for output in self.outputs:
send(output, msg)
except socket.error, e:
# Remove
inputs.remove(sock)
self.outputs.remove(sock)
self.server.close()The chat server initializes with a few data attributes. It stores the count of clients, map of each client, and output sockets. The usual server socket creation also sets the option to reuse an address so that there is no problem restarting the server again using the same port. An optional backlog argument to the chat server constructor sets the maximum number of queued connections to listen by the server.
An interesting aspect of this chat server is to catch the user interrupt, usually via keyboard, using the signal module. So a signal handler sighandler is registered for the interrupt signal (SIGINT). This signal handler catches the keyboard interrupt signal and closes all output sockets where data may be waiting to be sent.
The main executive method of our chat server run() performs its operation inside a while loop. This method registers with a select interface where the input argument is the chat server socket, stdin. The output argument is specified by the server's output socket list. In return, select provides three lists: readable, writable, and exceptional sockets. The chat server is only interested in readable sockets where some data is ready to be read. If that socket indicates to itself, then that will mean a new client connection has been established. So the server retrieves the client's name and broadcasts this information to other clients. In another case, if anything comes from the input arguments, the chat server exits. Similarly, the chat server deals with the other client's socket inputs. It relays the data received from any client to others and also shares their joining/leaving information.
The chat client code class should contain the following code:
class ChatClient(object):
""" A command line chat client using select """
def __init__(self, name, port, host=SERVER_HOST):
self.name = name
self.connected = False
self.host = host
self.port = port
# Initial prompt
self.prompt='[' + '@'.join((name, socket.gethostname().split('.')[0])) + ']> '
# Connect to server at port
try:
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.connect((host, self.port))
print "Now connected to chat server@ port %d" % self.port
self.connected = True
# Send my name...
send(self.sock,'NAME: ' + self.name)
data = receive(self.sock)
# Contains client address, set it
addr = data.split('CLIENT: ')[1]
self.prompt = '[' + '@'.join((self.name, addr)) + ']> '
except socket.error, e:
print "Failed to connect to chat server @ port %d" % self.port
sys.exit(1)
def run(self):
""" Chat client main loop """
while self.connected:
try:
sys.stdout.write(self.prompt)
sys.stdout.flush()
# Wait for input from stdin and socket
readable, writeable,exceptional = select.select([0, self.sock], [],[])
for sock in readable:
if sock == 0:
data = sys.stdin.readline().strip()
if data: send(self.sock, data)
elif sock == self.sock:
data = receive(self.sock)
if not data:
print 'Client shutting down.'
self.connected = False
break
else:
sys.stdout.write(data + '
')
sys.stdout.flush()
except KeyboardInterrupt:
print " Client interrupted. """
self.sock.close()
breakThe chat client initializes with a name argument and sends this name to the chat server upon connecting. It also sets up a custom prompt [ name@host ]>. The executive method of this client run() continues its operation as long as the connection to the server is active. In a manner similar to the chat server, the chat client also registers with select(). If anything in readable sockets is ready, it enables the client to receive data. If the sock value is 0 and there's any data available then the data can be sent. The same information is also shown in stdout or, in our case, the command-line console. Our main method should now get command-line arguments and call either the server or client as follows:
if __name__ == "__main__":
parser = argparse.ArgumentParser(description='Socket Server Example with Select')
parser.add_argument('--name', action="store", dest="name", required=True)
parser.add_argument('--port', action="store", dest="port", type=int, required=True)
given_args = parser.parse_args()
port = given_args.port
name = given_args.name
if name == CHAT_SERVER_NAME:
server = ChatServer(port)
server.run()
else:
client = ChatClient(name=name, port=port)
client.run()We would like to run this script thrice: once for the chat server and twice for two chat clients. For the server, we pass –name=server and port=8800. For client1, we change the name argument --name=client1 and for client2, we put --name=client2. Then from the client1 value prompt we send the message "Hello from client 1", which is printed in the prompt of the client2. Similarly, we send "hello from client 2" from the prompt of the client2, which is shown in the prompt of the client1.
The output for the server is as follows:
$ python 2_3_chat_server_with_select.py --name=server --port=8800 Server listening to port: 8800 ... Chat server: got connection 4 from ('127.0.0.1', 56565) Chat server: got connection 5 from ('127.0.0.1', 56566)
The output for client1 is as follows:
$ python 2_3_chat_server_with_select.py --name=client1 --port=8800 Now connected to chat server@ port 8800 [[email protected]]> (Connected: New client (2) from [email protected]) [[email protected]]> Hello from client 1 [[email protected]]> #[[email protected]]>>hello from client 2
The output for client2 is as follows:
$ python 2_3_chat_server_with_select.py --name=client2 --port=8800 Now connected to chat server@ port 8800 [[email protected]]> #[[email protected]]>>Hello from client 1 [[email protected]]> hello from client 2 [[email protected]]
The whole interaction is shown in the following screenshot:
