Once we know that reading commands will not wait for data, there must be another way to cause the client to react once the data is available. The fileevent command is useful for achieving this. By using this command, you can have your code executed only when some event related to the channel occurs. The command accepts the following as parameters:

For example, the preceding code causes the readData command to be executed always when the channel becomes readable, that is, when there is some data to be read from it.

Handling the writable event is important when creating the socket with the -async option. As mentioned before, the socket command will end immediately, and the socket creation may be delayed (due to some network operations). Therefore, once it is ready to transmit your data, you will be notified with that event.

The readData command reads the data using the read command, calculates the time spent on this task, and if the channel was closed, also correctly closes this side of the channel and sets the end variable to 1, which causes the client code to leave the event loop and end as a result.

The closure of the channel is detected using the eof command—eof $serverChannel returns 1 if the end of file event occurred during the most recently read operation on the channel.

The output from this client-side code is:

c:	cl_bookchapter6TCPsocketsfirstExample>tclsh85 client_async1.tcl
read: 12345
execution has been blocked for 0 seconds
read: 6789
execution has been blocked for 0 seconds
read: thank you for connecting to our server!
execution has been blocked for 0 seconds

What we have achieved is that there is no single line in the code which would block waiting for any network connection related event. The output clearly illustrates that read does not block, but returns any data that came from the server.

More interesting is the behavior of the gets command. As we know, this command reads the entire line from the channel or the blocks. In nonblocking mode, when there is data available for reading, but it does not constitute the full line, gets will return an empty string without removing this data from the input buffer. Let's modify the readData procedure slightly:

proc readData {serverChannel} {
global end
puts "noOfChars: [gets $serverChannel data]"
puts "fblocked: [fblocked $serverChannel]"
puts "eof: [eof $serverChannel]"
if {![fblocked $serverChannel]} {
if {[eof $serverChannel]} {
close $serverChannel
set end 1
break
}
puts "read: $data"
}
}

We have removed the time calculations and replaced read with gets. The procedure also introduces us to the fblocked command. The command returns 1 in case there is some data to read from the channel, but it is not sufficient for the reading command. The typical example is just gets, which attempts to read the entire line, and when only the characters 12345 without newline character at the end are available, the gets command will read an empty string and return -1 as the read character count, leaving the data in the buffer, and causing fblocked to report properly.

c:	cl_bookchapter6TCPsocketsfirstExample>tclsh85 client_async2.tcl
noOfChars: -1
fblocked: 1
eof: 0
noOfChars: 9
fblocked: 0
eof: 0
read: 123456789
noOfChars: 39
fblocked: 0
eof: 0
read: thank you for connecting to our server!
noOfChars: -1
fblocked: 0
eof: 1

Detecting the end of transmission may be a little tricky. In the current example, the server at the end sends the entire line (with the newline character at the end), so gets is able to read it properly. Following that, the channel is closed, resulting in another execution of the readData procedure on the client's side. This time gets also returns -1 as the number of characters read, but the channel is not blocked, and eof indicates that the connection is terminated.

So far so good, but what if the server sends the last line without the newline marker:

puts -nonewline $channelId "thank you for connecting to our server!"
flush $channelId
close $channelId

In this case, the client would get the following:

c:	cl_bookchapter6TCPsocketsfirstExample>tclsh85 client_async2.tcl
noOfChars: -1
fblocked: 1
eof: 0
noOfChars: 9
fblocked: 0
eof: 0
read: 123456789
noOfChars: -1
fblocked: 1
eof: 0
noOfChars: 39
fblocked: 0
eof: 1

As you can see, the last call to gets resulted in 39 characters of data being read. However, in parallel, it caused eof to report 1, effectively causing the read data to be discarded. This allows a mistake in the logic of the readData function, which checks for the end of file first. A better design would include checking if gets returns a value other than -1:

proc readData {serverChannel} {
global end
set noOfChars [gets $serverChannel data]
puts "noOfChars: $noOfChars"
puts "fblocked: [fblocked $serverChannel]"
puts "eof: [eof $serverChannel]"
if {![fblocked $serverChannel]} {
if {$noOfChars!=-1} {
puts "read: $data"
}
if {[eof $serverChannel]} {
close $serverChannel
set end 1
break
}
}
}

This makes the usage of fblocked redundant, because gets returns -1 also when it was unable to read the entire line. Finally, the procedure would look like:

proc readData {serverChannel} {
global end
set noOfChars [gets $serverChannel data]
if {$noOfChars!=-1} {
puts "read: $data"
}
if {[eof $serverChannel]} {
close $serverChannel
set end 1
break
}
}

To copy the content of the file, a handy command called fcopy will be used, which copies data from an input channel to an output channel. The format of the command follows:

fcopy inputChannel outputChannel ?-size numberOfBytes? ?-command commandName?

By default, the command will copy the data until end of file occurs, but this behavior can be altered by specifying the -size parameter—in this case, the command will copy only a numberOfBytes.

Normally, the command would block until the copying operation is completed, but in case the -command parameter is used, it will end immediately and the commandName command will be called once the data is copied.

In our example, we are using both options: we copy only the exact amount of bytes, that is, the size of the file, and on completion, the fileUploaded procedure will be called. The input channel allows us to read data from the file (returned by the open command). The output is the TCP channel to the server. Both channels are configured as binary at the moment.

During the fcopy operation, no other input/output operations are allowed on the involved channels, which is why we disabled the writable events earlier.

Once the file content is sent to the server using the TCP socket channel, the fileUploaded procedure closes the file channel and restores the handling of the writable events, allowing the repetition of the entire operation for the next filename in the list.

The server code that is able to understand the protocol described is:

socket -server serverProc 9876
proc serverProc {channelId clientAddress clientPort} {
while {true} {
fconfigure $channelId -encoding utf-8
fconfigure $channelId -translation lf
set fileName [gets $channelId]
set fileSize [gets $channelId]
if {[eof $channelId]} {
puts "end of transmission"
break
}
puts "incoming file $fileName of size $fileSize"
set fileOutput [open "uploads/$fileName" w]
fconfigure $fileOutput -translation binary
fconfigure $channelId -translation binary
fcopy $channelId $fileOutput -size $fileSize
puts "file copied"
close $fileOutput
}
close $channelId
}
vwait forever

All server logic is contained in the serverProc procedure. The server is designed to accept transmission of multiple files, so it enters an "infinitive" loop (while {true}). In each iteration, the channel is first configured to read text data from it. Then the name and size of file that is about to be uploaded are read. If any of these operations encounter the end of the file, the loop will be ended by calling the break command.

Following that, the file output channel is opened (all the incoming files will be stored in the uploads directory), and both this and the network channel are set to binary mode. Then the job of copying the data is done by the fcopy command, this time working in the blocking mode, because the -command parameter was not specified. Finally, the iteration ends with closure of the file.

If you exit the loop, the network channel will be closed and the serverProc procedure terminated.

The result of running the described example is illustrated in the following screenshot:

For the purpose of this book, we'll create an example that uses threads and Tcl's thread pool. We'll use the thread pool to process requests from clients. Thread pools allow us to run commands within separated threads. It passes the specified job to the first available thread or spawns a new thread if it is needed. Jobs are specified as Tcl scripts to be run, so it is intuitive to use them. We also provide initialization and clean-up scripts for the thread.

The requests that the clients send will be read from the main thread and once read, the request itself along with the channel is passed to child thread, which handles the request and sends back the response. This optimizes how child threads are used—we do not need to make a thread wait until all data is readable, but instead use a single thread for reading all the incoming data.

The request is sent in such a way that a line indicating the number of bytes to read, as plaintext, is sent first and is followed by the specified number of bytes of the request itself.

Let's start with creating the server. We'll need to set up our namespace and load the Thread package first:

package require Thread
namespace eval tpserver {}

Next we'll set up a new thread pool. We use the command tpool::create for this purpose, which returns the new thread pool's identifier. We store it in tpserver::tpool variable for later use.

set tpserver::tpool [tpool::create 
-initcmd [list source server-threads.tcl]]

Several options can be passed to the tpool::create command. The options -initcmd and -exitcmd specify commands that will be run in the new thread after it is created and before it is deleted accordingly. The options -minworkers and -maxworkers define how many threads should be running, and their values default to 0 and 4 accordingly. This means that, at most, four threads will be used and no thread will be created if not required. The -idletime option specifies a time (in seconds) after which a thread is terminated if it is not used. If not specified, threads are deleted as soon as they have no jobs to perform.

For this example, we specify a command to run to initialize the thread and use the default values for the remaining options. Child threads that will be used for operations will need to have their own code, which is responsible for handling requests. We have the thread source file called server-threads.tcl, which contains the code to run and respond to requests.

We now need to write the code for handling incoming connections. This code will be running in the main thread and will read the request using events. After reading the request, we'll add a job to the thread pool.

We start by setting the channel to the nonblocking mode and set up automatic translation of newline characters. We also set up a readable event so that whenever anything is sent, the command is run:

proc tpserver::accept {chan args} {
fconfigure $chan -buffering none -blocking 0 
-translation auto
fileevent $chan readable 
[list tpserver::readInfo $chan]
}

We'll then execute the command for reading the first line that specifies number of bytes. We check if the end of file has been received first. If so, we will close the channel and exit.

proc tpserver::readInfo {chan} {
if {[eof $chan]} {
catch {close $chan}
return
}

Following that, we try to read the line. If a complete line is not available, the gets command will return -1 and we'll wait for next readable event.

if {[gets $chan line] < 0} {
return
}

In the next step we try to remove all the whitespace around the number of bytes and check if it is a valid integer. If not, we close the channel immediately and return.

set bytes [string trim $line " 
"]
if {![string is integer -strict $bytes]} {
catch {close $chan}
return
}

After getting a valid number of bytes to read, we convert the channel into binary mode and set up a new readable event, which will read the specified amount of bytes.

fconfigure $chan -translation binary
fileevent $chan readable 
[list tpserver::readData $chan $bytes ""]
}

The command for reading data accepts the number of bytes left to be read and the data read so far. For the initial event, we'll set up the specified number of bytes and empty the buffer. The subsequent events will be set up with data read so far until the specified number of bytes has been read.

The command to read data also begins by checking if the end of the file has been reached:

proc tpserver::readData {chan numBytes data} {
if {[eof $chan]} {
catch {close $chan}
return
}

Next we try to read data from the channel. If that fails, we close the channel immediately and return.

if {[catch {
set d [read $chan $numBytes]
}]} {
catch {close $chan}
return
}

Following that, we append the data that was read to the buffer and calculate the new number of bytes remaining—by taking the difference between the number of bytes read now and the previous value.

append data $d
set numBytes [expr {$numBytes - [string length $d]}]

If we have read the entire request, we remove the readable event and invoke the command to handle the request:

if {$numBytes <= 0} {
fileevent $chan readable [list]
tpserver::handleRequest $chan $data
} else {

Otherwise, we set up a new readable event with the current data buffer and the remaining number of bytes to read:

fileevent $chan readable 
[list tpserver::readData $chan $numBytes $data]
}
}

When a complete request has been read, we will use the thread pool to deploy it. We first invoke the tls::detach command, which detaches a channel from the current thread. After this command, the current thread can no longer access the channel.

proc tpserver::handleRequest {chan data} {
variable tpool
thread::detach $chan

We then post a new job to be performed in the thread pool—the command to handle the response to a request along with request already read. We specify the channel to use and the request.

tpool::post -nowait -detached $tpool 
[list tpserver::respond $chan $data]
}

After that, this connection is handled by one of the threads in the thread pool.

Finally, we need to set up a listening connection to accept the requests:

socket -server tpserver::accept 12345

We now need to set up the server-threads.tcl file that handles responding to the requests.

We'll start by creating the respond command. It needs to attach to the channel using the tls::attach command. After running this command, the specified channel can be used like any other channel in this thread.

proc tpserver::respond {chan data} {
thread::attach $chan

We will then run the command to handle the response and catch any potential errors. If the command does not fail, the result variable will store the value returned by the respondHandle command. If there are any errors, result will store the error information.

set c [catch {
respondHandle $data
} result]

In the next step, we try to send the results as a list of two elements—the result from the catch command and the result or error, depending on value of the c variable.

catch {
puts -nonewline $chan [list $c $result]
flush $chan
}

We wrap writing output in a catch command because the write might fail, either because the client has already closed the channel or because of network issues.

Finally, we try to close the channel:

catch {close $chan}
}

We then need to write the command that handles the request itself.

proc tpserver::respondHandle {data} { return [eval $data]
}

While the respondHandle command can perform any action, for the purpose of this example, it simply evaluates the Tcl code sent by a remote peer.

Finally, we need to create a client that will allow us to send a command to any host and port, and will allow us to specify a callback to be run whenever results are available.

We'll start by creating a namespace for our client.

namespace eval tpclient {}

Following that, we'll create the command to send a request to the server. It accepts the host and the port, the request and callback data to invoke when results are retrieved, or if sending the request fails.

The callback will be run with two additional arguments specified—whether an error was encountered (true indicating an error and false indicating a successful request), followed by either error information or the result from the server.

We start by trying to open a TCP connection to a remote system and setup a channel for sending binary data:

proc tpclient::sendRequest {host port data callback} {
if {[catch {
set chan [socket $host $port]
fconfigure $chan -blocking 0 -buffering none 
-translation binary

Then we send the request by sending the length of the request followed by the request itself:

puts -nonewline $chan 
"[string length $data]
$data"

Finally, we set up a readable event to read results as soon as they are available:

fileevent $chan readable [list 
tpclient::readResponse $chan "" $callback]

If any of these operations fail, we close the channel if it was open and invoke the callback, specifying an error:

}]} {
catch {close $chan}
{*}$callback true "Unable to connect"
}
}

The command to read the response is written in a similar way to the command to read a request on the server—it reads the data and passes the current buffer to the next readable event. We also pass the callback command as the argument.

We start by trying to read data from the channel:

proc tpclient::readResponse {chan data callback} {
if {[catch {
set d [read $chan]
}]} {

If the read fails, the callback is invoked with an error indication.

catch {close $chan}
{*}$callback true "Unable to read data"
return
}

Otherwise, we append the newly read data to the buffer and check if the end of file message has been received:

append data $d
if {[eof $chan]} {

If the end of file has been reached, we close the channel. We then take the first index of the resulting list as an indication of an error and the actual data as the second element of the list and invoke the callback with this data:

catch {close $chan}
set error [lindex $data 0]
set data [lindex $data 1]
{*}$callback $error $data
} else {

Otherwise, we set up a new readable event with the current buffer:

fileevent $chan readable [list 
tpclient::readResponse $chan $data $callback]
}
}

Further, we can invoke the command to send a request to the server, specifying the expr {1+1} command to be run and the requestCallback command to be run as the callback:

tpclient::sendRequest 127.0.0.1 12345 
{expr {1+1}} 
requestCallback

We used 127.0.0.1, but if the server is located on any other machine, its IP address or hostname should be specified.

The callback itself simply prints either an error indication or a result from the command:

proc requestCallback {error data} {
if {$error} {
puts "Error: $data"
} else {
puts "Result: $data"
}
exit 0
}

Depending on whether the server (saved in server.tcl in the source code examples for this chapter) is running or not, the result should be either one of the following commands:

c:	cl_bookchapter6TCPsockets	hreads>tclsh85 client.tcl
Error: Unable to connect

or:

c:	cl_bookchapter6TCPsockets	hreads> tclsh85 client.tcl
Result: 2

Even though this example only does 1+1, using a thread pool to perform tasks can be useful if you need to offer functions that either use blocking functions (such as getting data from a database) or use CPU intensive operations (such as calculating an MD5 from a large file). Imposing limits on the number of threads itself prevents us from using up all available resources on one type of functionality.

Documentation for the Thread package as well as thread pools can be found on Tcl's wiki page - http://wiki.tcl.tk/thread in the Documentation section.