The transport layer—layer four in the OSI Reference Model—provides data delivery, flow control, and error recovery services to end hosts on the Internet. The two primary transport layer protocols we are concerned with are the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP).
TCP is a connection-oriented protocol. This means that the client and server negotiate a set of parameters that define how data is transferred before any data is exchanged, and that there is a clear demarcation of the start and end of a connection. TCP transfers data between two nodes in a reliable, in-order fashion, which frees application layer protocols from having to build in this functionality themselves.[21]
In contrast, UDP is a connectionless protocol. As a connectionless protocol, there is no guarantee that data ever reaches its intended destination, and there is also no guarantee about the shape of the data that does make it through (even the calculation of the checksum in the UDP header is optional unlike in TCP). Applications that transmit data over UDP sockets can choose to implement additional mechanisms to transmit data reliably, but such functionality must be built in to the application layer when UDP sockets are used.
We'll focus first in this chapter on how iptables represents transport layer information within log message output. We'll then see how these logs can catch suspicious transport layer activity.
The iptables LOG target has extensive machinery for logging TCP and UDP headers. The TCP header is far more complex than the UDP header, and some TCP header fields are logged only if specific command-line arguments are supplied to iptables when a LOG rule is added to the iptables policy.
The TCP header is defined in RFC 793, and the length of the header for any particular TCP segment[22] varies depending on the number of options that are included. The length of the header, excluding the options (which is the only variable-length field), is always 20 bytes. In an iptables log message, each field in the TCP header is prefixed with an identifying string, as shown in Figure 3-1.
All dark gray boxes in Figure 3-1 are always included within an iptables log message of a TCP packet; the fields shaded in lighter gray are included only if the specified command-line argument is given to iptables. The white boxes are never logged by iptables.
The LOG rule in the INPUT, OUTPUT, and FORWARD chains included in the default iptables policy in Chapter 1 are all built with the --log-tcp-options argument, so each log message contains a blob of hexadecimal codes whenever a TCP segment contains options. This chapter assumes that the default iptables policy implemented by the iptables.sh script from Chapter 1 is running on the iptablesfw system depicted in Figure 3-2. (This diagram is identical to Figure 1-2 and is duplicated here for convenience.)
To illustrate TCP options included within an iptables log message, we attempt to initiate a TCP connection to port 15104 from the ext_scanner system to the iptablesfw system.
Because the default policy does not allow communications with port 15104, the initial SYN packet is intercepted by the default iptables LOG and DROP rules. The tags iptables associates with each field of the TCP header are shown in bold below, starting with the source port (SPT) and ending with the options portion of the header (OPT):
[ext_scanner]$ nc -v 71.157.X.X15104 [iptablesfw]# tail /var/log/messages | grep 15104 Jul 12 15:10:22 iptablesfw kernel: DROP IN=eth0 OUT= MAC=00:13:d3:38:b6:e4:00:30:48:80:4e:37:08:00 SRC=144.202.X.XDST=71.157.X.XLEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=18723 DF PROTO=TCPSPT=47454DPT=15104WINDOW=5840RES=0x00SYN URGP=0OPT (020405B40402080A30 82048C0000000001030306)
To have iptables include TCP sequence and acknowledgment values, use the --log-tcp-sequence argument (see the sections in bold below):
[iptablesfw]# iptables -I INPUT 1 -p tcp --dport 15104 -j LOG --log-tcp-options--log-tcp-sequence[ext_scanner]$ nc -v 71.157.X.X15104 [iptablesfw]# tail /var/log/messages | grep 15104 Jul 12 15:33:53 iptablesfw kernel: IN=eth0 OUT= MAC=00:13:d3:38:b6:e4:00:30:48:80:4e:37:08:00 SRC=144.202.X.XDST=71.157.X.XLEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=62378 DF PROTO=TCP SPT=54133 DPT=15104SEQ=3180893451 ACK=0WINDOW=5840 RES=0x00 SYN URGP=0 OPT (020405B40402080A308766A10000000001030306)
The UDP header is defined in RFC 768. It is only eight bytes long and has no variable length fields (see Figure 3-3).
Since there are no special command-line arguments to influence how a UDP header is represented by the LOG target, iptables always logs UDP headers in the same way.
Even though the default LOG rules in the iptables policy discussed in Chapter 1 use the --log-tcp-options argument, if a UDP packet hits one of these rules, iptables does the right thing and only logs information that is actually in the packet; it won't attempt to log the options portion of a TCP header that does not exist. The UDP checksum is never logged, but the remaining three fields (SPT, DPT, and LEN) are all included:
[ext_scanner]$ echo -n "aaaa" | nc -u 71.157.X.X5001 [iptablesfw]# tail /var/log/messages | grep 5001 Jul 12 16:27:08 iptablesfw kernel: DROP IN=eth0 OUT= MAC=00:13:d3:38:b6:e4:00:30:48:80:4e:37:08:00 SRC=144.202.X.XDST=71.157.X.XLEN=33 TOS=0x00 PREC=0x00 TTL=64 ID=38817 DF PROTO=UDPSPT=44595DPT=5001LEN=12
Note
The UDP LEN field in the iptables log message above includes the length of the UDP header plus the length of the application layer data. In this case, the application layer data consists of the four bytes "aaaa", so adding this to the length of the UDP header (eight bytes) yields a total of 12 bytes. The -n command-line argument to the echo command instructs it not to add a trailing newline character. Had this argument not been used, the value of the LEN field would have been 13 to accommodate the additional byte.
[21] 1 Technically, the transport layer interacts with the session layer above and network layer below in the OSI Reference Model, but it is usually more useful to think of the session layer as subsumed within the application layer (along with the presentation layer).
[22] 2 Although the technical term for a unit of TCP information is a TCP segment, many people informally refer to TCP packets instead (packets is technically a term reserved for the network layer), and I use this colloquialism also. The same logic applies to UDP datagrams—it is more convenient to refer to UDP packets.