The Transmission Control Protocol (TCP) provides the backbone of HTTP communications. With TCP, we can open up interfaces between processes running on separate server hosts and remotely communicate between processes with less overhead and fewer complexities than HTTP.
Node provides us with the net module for creating TCP interfaces. When it comes to scaling, reliability, load balancing, synchronization, or real-time social communications, TCP is a fundamental element.
In this recipe, we're going to demonstrate the sort of foundation needed to communicate between processes over a network by setting up a TCP connection that allows us to remotely monitor and filter HTTP headers of website hits in real time.
First, let's create our first TCP server in server.js as follows:
var net = require('net'),
var fauxHttp = net.createServer(function(socket) {
socket.write('Hello, this is TCP
'),
socket.end();
socket.on('data', function (data) {
console.log(data.toString());
});
}).listen(8080);
We can use the nc (netcat) command-line program to test this out in another terminal as follows:
echo "testing 1 2 3" | nc localhost 8080
Note
If we're using Windows, we can download netcat from http://www.joncraton.org/blog/netcat-for-windows.
The response should be Hello, this is TCP
and the server.js console should output testing 1 2 3.
Remember, HTTP sits on top of TCP, so we can make an HTTP request of a TCP server. If we navigate our browser to http://localhost:8080 and watch the console, we'll see all the headers from our browser's HTTP request appear in the console, with the browser displaying Hello this is TCP.
We've given the TCP server the name of fauxHttp. We're going to use it to record HTTP headers from browser clients (with some adjustments we could easily adapt our code to work with an actual HTTP server).
Still inside server.js, we're going to create another TCP server that opens a second port for monitor.js to communicate with our server. Before we do though, we'll make a new EventEmitter object as a bridge between our two server.js TCP servers:
var net = require('net'),
stats = new (require('events').EventEmitter),
filter = 'User-Agent';
var fauxHttp = net.createServer(function(socket) {
socket.write('Hello, this is TCP
'),
socket.end();
socket.on('data', function (data) {
stats.emit('stats', data.toString());
});
}).listen(8080);
We've replaced console.log in the data listener of socket with new stats EventEmitter which will emit a custom stats event upon receiving TCP data.
We also included a filter variable to be used in our second TCP interface in server.js as shown in the following code:
var monitorInterface = net.createServer(function(socket) {
stats.on('stats', function (stats) {
var header = stats.match(filter + ':') || stats.match(''),
header = header.input.substr(header.index).split('
')[0];
socket.write(header);
});
socket.write('Specify a filter [e.g. User-Agent]'),
socket.on('data', function(data) {
filter = data.toString().replace('
',''),
socket.write('Attempting to filter by: ' + filter);
});
}).listen(8081);
Our monitorInterface server listens to our stats emitter to determine when the first TCP server has received information, sending this information (after it has been filtered) to a client connected on port 8081.
All we have to do now is create this client. Inside monitor.js we write the following code:
var net = require('net'),
var client = net.connect(8081, 'localhost', function () {
process.stdin.resume();
process.stdin.pipe(client);
}).on('data', function (data) {
console.log(data.toString());
}).on('end', function () {
console.log('session ended'),
});
When we open two terminals, running server.js in one and monitor.js in the other, and navigate to http://localhost:8080 in our browser server.js transmits the User-Agent string from the HTTP headers of each request to monitor.js.
We can apply a different filter, such as Accept. By simply typing it into a running monitor.js process, any non-matching filters will default to returning the preliminary request line (GET /, POST /route/here, and so on).
To run across separate systems, we simply place server.js on a remote host and then update the second parameter of net.connect from localhost to the name of our server, for example:
var client = net.connect(8081, 'nodecookbook.com', function () {
The HTTP layer works on top of TCP. So when our browser makes a request, the TCP server receives all the HTTP header information. http.createServer would handle these headers and other HTTP protocol interactions. However, net.createServer simply receives TCP data.
socket.write and socket.end are similar to response.write and response.end within an HTTP server callback function, but without reference to the requirements of the HTTP protocol. Nevertheless, the similarities are enough for our browser to be able to receive data from socket.write.
All our fauxHttp TCP server does is receive a request via port 8080, outputs Hello, this is TCP
to the client, and reads any data from the client directly through to our stats event emitter.
Our monitorInterface TCP server listens on a separate port (8081), essentially giving us a sort of (completely insecure) admin interface. In the monitorInterface callback, we listen to the stats emitter which is triggered whenever a browser (or any TCP client) hits localhost:8080.
Inside the listener callback of stats, we retrieve the desired header, using the filter variable to search the HTTP headers with the match objects index and input properties, enabling us to extract the specified header. If there are no matches, we match an empty string, thereby returning a match object containing an index of 0 resulting in the extraction of the first line of the HTTP headers (the requested path and method).
The last part of the monitorInterface TCP server callback listens on the socket for a data event and sets the filter variable to whatever the client sends. This enables the monitor.js client to alter the filter by piping the process.stdin stream directly into the TCP client. Meaning we can type directly into monitor.js' running process and the data event from the socket of monitorInterface will trigger in server.js receiving whatever is typed in monitor.js' STDIN.
monitor.js avails of this functionality by piping the process.stdin stream directly into the TCP client. This means we can type directly into the running process and the data event from socket of monitorInterface in server.js, and this will trigger passing anything typed from STDIN of monitor.js.
Let's look at some ways we can further harness the power of TCP.
There can be various reasons to forward a port. As an example, if we wish to SSH into our server over a mobile connection, we may find that port 22 has been blocked. The same can apply with corporate firewalls (this could be because a blanket block is applied to all privileged ports except the most common such as 80 and 443).
We can use the net module to forward TCP traffic from one port to another, essentially circumventing a firewall. So naturally this should be used only for legitimate cases and with any necessary permission.
First, we'll require net and define ports to forward from and to:
var net = require('net'),
var fromPort = process.argv[2] || 9000;
var toPort = process.argv[3] || 22;
So we can either define ports via command line or default to forwarding the arbitrary port 9000 to the SSH port.
Now we create a TCP server that receives connections via fromPort, creating a TCP client connection to toPort, passing all data between these connections as follows:
net.createServer(function (socket) {
var client;
socket.on('connect', function () {
client = net.connect(toPort);
client.on('data', function (data) {
socket.write(data);
});
})
.on('data', function (data) {
client.write(data);
})
.on('end', function() {
client.end();
});
}).listen(fromPort, function () {
console.log('Forwarding ' + this.address().port + ' to ' + toPort);
});
We use the data events to receive and push data between client (our bridge connection) and socket (the incoming connection).
If we now run our script on the remote server (with no arguments), we can log in to a secure shell from our local computer using port 9000 like so:
ssh -l username domain -p 9000
With the third-party pcap module, we can also observe TCP packets as they travel in and out of our system. This can be useful for analysis and optimization of expected behaviors, performance, and integrity.
On the command line:
npm install pcap
For our code:
var pcap = require('pcap'),
var pcapSession = pcap.createSession("","tcp"); //may need to put wlan0,
//eth0, etc. as 1st arg.
var tcpTracker = new pcap.TCP_tracker();
tcpTracker.on('end', function (session) {
console.log(session);
});
pcapSession.on('packet', function (packet) {
tcpTracker.track_packet(pcap.decode.packet(packet));
});
Tip
If pcap fails to choose the correct device, there will be no output (or maybe unrelated output). In this case, we need to know which device to sniff. If we are connected wirelessly, it may well be wlan0 or wlan1and if we are wired, in it could be eth0/eth1. We can find out by typing ifconfig (Linux, Mac OS X) or ipconfig (Windows) on the command line to see which device has an inet address matching the network part of our router's IP address (for example, 192.168.1.xxx).
If we save this as tcp_stats.js, we can run it with the following:
sudo node tcp_stats.js
The pcap module interfaces with privileged ports and therefore must be run as root (for operating systems such as Linux and Mac OS x that enforce privileged ports).
If we navigate to any website and then refresh the page, the tcpTracker end event of pcap is triggered in which we output the session object.
To initialize the tcpTracker, we create a pcap session and attach a listener for the packet event where we pass each decoded packet into tcpTracker.
Upon creating the pcap session we pass an empty string followed by tcp to the createSession method. The empty string causes pcap to automatically choose an interface (if this doesn't work we can specify the appropriate interface, for example, eth0, wlan1 or lo if we want to analyze localhost TCP packets). The second parameter, tcp, instructs pcap to only listen for TCP packets.