Remedial TCP/IP
Now that you have a simplified overview of how the IP system works, let’s look at a real network protocol in some depth. The dominant transport protocol on the Internet is the Transmission Control Protocol over Internet Protocol, or TCP/IP. Although TCP is a transport protocol and IP is a network protocol, the two are so tightly intertwined that they’re generally referred to as a single entity.
We’ll start with ICMP, and proceed to UDP and TCP.
ICMP
ICMP is used to transmit routing and availability messages across the network. Tools such as ping(8) and traceroute(8) use ICMP. ICMP includes all sorts of different protocols and tools.
While some people claim that you need to block ICMP for security purposes, those people don’t understand that ICMP is just as diverse as the better-understood transport protocols TCP and UDP. Proper IPv4 network performance requires large chunks of ICMPv4. If you must block ICMPv4 for security reasons, do so selectively. For example, blocking source quench messages breaks path maximum transmission unit (MTU) discovery, which will steer you directly into a world of hurt. If you don’t understand that last sentence, don’t block ICMPv4.
IPv6 dies without ICMPv6, as IPv6 doesn’t support packet fragmentation, so never block ICMPv6. If you don’t know what packet fragmentation is, just trust me on this.[30]
UDP
UDP is the most bare-bones data-transfer protocol that runs over IP. It offers no error handling, minimal integrity verification, and no defense whatsoever against data loss. The transport protocol considers each packet of UDP completely self-contained; there are no data-coherence checks at the protocol layer. Despite these drawbacks, UDP can be a good choice for particular sorts of data transfer, and many vital Internet services rely on it.
Note
This discussion covers both UDPv4 and UDPv6. While each runs over only the corresponding network protocol, they behave identically otherwise.
UDP is also a datagram protocol, meaning that each network transmission is complete and self-contained, and received as a single integral unit. While the application might not consider a single UDP packet a complete request, the network does.
When a host transmits data via UDP, it has no way of knowing if the data ever reaches its destination. Programs that receive UDP data just listen to the network and accept whatever happens to arrive. When a program receives data via UDP, it cannot verify the source of that data. Although each UDP packet does include a source address, this address is easily faked. Each UDP packet includes a checksum for the packet, but there’s no integrity checking for the data stream as a whole. This is why UDP is called connectionless, or stateless.
No integrity checking, no guard against data loss, the potential for faked packets—all this sounds pretty unreliable. So why use UDP at all?
UDP-based applications often have their own error-correction methods or otherwise don’t mesh well with more reliable protocols, such as TCP. For example, simple client DNS queries must time out within just a few seconds or users will whine uncontrollably. TCP connections time out only after two minutes. DNS requires quick failures and only a single packet per transaction, which makes UDP a better choice than TCP for simple DNS queries. Real-time streaming services, such as video conferencing applications, also use UDP. (After all, if a few pixels go missing during a video conference, you don’t want those pixels a minute later.) Most other UDP-based applications use UDP for similar reasons.
Because the UDP protocol itself doesn’t return anything when you connect to a port, there’s no reliable way to remotely test if a UDP port is reachable (although tools such as nmap try to do so).
If you want a protocol that responds at the network layer, look at TCP.
TCP
TCP includes nifty features, such as error correction and recovery. The receiver must acknowledge every packet it gets; otherwise, the sender retransmits any unacknowledged packets. Unlike UDP, applications that use TCP can expect reliable data transmission. This makes TCP a connected, or stateful, protocol.
Note
This discussion covers both TCPv6 and TCPv4. While they differ because of their underlying transport protocol, they behave in the same way.
TCP is also a streaming protocol, which means that a single request can be split among several network packets. While the sender might transmit several chunks of data one after the other, that data might arrive out of order or fragmented. The recipient must track these chunks and assemble them properly to complete the network transaction.
For hosts to exchange TCP data, they must set up a channel for that data to flow across. One host requests a connection, the other host responds to the request, and then the first host starts transmitting. This setup process is known as the three-way handshake. Similarly, once transmission is complete, the systems must do a certain amount of work to tear down the connections.
To test if a TCP port is open, you can use telnet(1) or nc(1) to connect to the port. Here, I see if I can connect to port 22 on the host caddis.
$ telnet caddis 22 Trying 192.0.2.35… Connected to caddis. Escape character is '^]'. SSH-2.0-OpenSSH_6.0 1 ^] 2 telnet> c Connection closed.
I connect to the remote port and see information displayed by the port, use the telnet escape character ^] (CTRL-]) to disconnect 1, and enter c 2 to close telnet.
TCP is commonly used by applications most suited to its fairly generic set of timeouts and transmission features, such as email programs, FTP clients, and web browsers.
How Protocols Fit Together
You can compare the network stack to sitting with your family at a holiday dinner. The datalink layer (ARP, in the case of Ethernet) lets you see everyone else at the table. IP gives every person at the table their own unique chair (except for the twins using piano bench NAT). ICMP provides basic, lower-layer information such as “The quickest way to the baked sweet potatoes is to get Uncle Mike to pass them”?[31] or “Aunt Liz can’t lift the ham platter.” TCP is where you hand someone the butter and the other person must say “thanks” before you let it go. UDP is like tossing a roll at Grandma Lucas; she might catch it or it might bounce off her forehead.
Transport Protocol Ports
Transport protocol ports permit one server to serve many different services over a single transport protocol, multiplexing connections between machines. When a network server starts, it attaches, or binds, to one or more logical ports. A logical port is just an arbitrary number ranging from 0 to 65536, although nothing uses port 0. For example, Internet mail servers often bind to port 25.
Each TCP or UDP packet arriving at a system carries a field containing its desired destination port number. If an incoming packet asks for port 25, it is connected to the mail server running on that port. This means that other programs can run on other ports, clients can talk to those different ports, and no one gets confused except you.
Note that port assignments are not some sort of physical constant, but rather are mutually agreed upon. There’s no reason that email services should run on port 25 other than the fact that everyone agrees that they should. If someone tries to send you email, their mail server will automatically connect to port 25 on your server. If you run email on port 80 and have a web server on port 25, you’ll never get your email, and your web server won’t get much traffic.
The file /etc/services contains a list of port numbers and the associated services. The file has a very simple, five-column format, as shown in these two sample lines:
www 80/tcp http # WorldWideWeb www 80/udp # HyperText Transfer Protocol
The first field is the name of the service assigned to this port. This entry is for the service www. Port 80 is assigned to www, both TCP and UDP. Then there’s a list of any other names assigned to this port. Port 80 is also known as http. Finally, there’s a comment that gives more detail about the service.
The HTTP protocol used on the Web runs over TCP, so why is UDP port 80 also reserved for HTTP? The answer is pretty simple: Computer people are easily confused. Having two services share the same port number but run on different protocols confuses people—for example, the syslog service runs on port 514 via UDP, and the lpr printer protocol runs on port 514 over TCP.[32]
Some server programs read /etc/services to learn which port to bind to on startup, and many client programs read /etc/services to learn which port they should try to connect to. If you run servers on unusual ports, you might need to edit this file to get these programs to attach where needed.
As with all standards, there are times you will want to break the rules. The SSH daemon sshd normally binds to port 22/TCP, but I’ve run it on ports 23 (telnet), 80 (www), 443 (https), and others to evade naïve packet-filtering firewalls. You will find your own reasons to break the standards. That’s fine, as long as you understand what you’re doing and how it affects others.
Reserved Ports
Ports below 1024 in both TCP and UDP can be opened only by the root user. These ports are assigned (mostly) to core Internet infrastructure protocols, such as DNS, SSH, HTTP, LDAP, and so on—services that only a few select hosts on each network should offer. Only programs with root-level privileges can bind to reserved ports. For example, a user can run a game server on a high-numbered port if the system policy allows, but that’s different from setting up a web page visible to the whole world that claims the machine’s official purpose is a game server. The port assignment for these core protocols is generally permanent, and if you want to interoperate with other sites, you won’t change them.
OpenBSD software usually binds to a reserved port as root and then drops privileges, performing the rest of its functions as an unprivileged user. These unprivileged users, discussed in Chapter 6, have even fewer privileges than a normal user account.
If you must run a service that binds to a reserved port, and it can run only as root, consider carefully whether you actually need it. Try to find an alternative server that does privilege separation. If you can’t, at least install that service on a dedicated machine to reduce its threat to other services on your network.
Which Ports Are Open?
So, network services are made available via TCP or UDP ports. Programs bind to ports to offer network services. This brings up two obvious questions:
-
Which ports are open?
-
What programs are listening to each port?
You can answer these questions with netstat(1) and fstat(1).
Using netstat
The netstat(1) program provides general visibility into the network stack. Use netstat to check your routing table, examine open sockets, see how many packets are traversing your interfaces, and so on. (I could write an entire book about netstat, but no one would buy it. Instead, I’ll sprinkle bits of netstat magic throughout this book.)
When looking at network information, I recommend turning off DNS lookups by using the -n flag. You can always rerun a check with DNS turned on, but adding DNS queries to the network sockets can sometimes skew the information you’re viewing, and almost always slows the command.
The -f argument lets you choose a protocol family to display. Use -f inet to see only IPv4 sockets, or -f inet6 to see only IPv6. Read netstat(1) for the full protocol list.
Finally, -a tells netstat to show all sockets opened by any process, rather than just sockets owned by the user.
Let’s put all those options together and have a look at the output. Here, I show the open IPv4 sockets on my system:
$ netstat -na -f inet Active Internet connections (including servers) Proto Recv-Q Send-Q ?Local Address Foreign Address (state) 1tcp 20 0 3192.0.2.135.22 4192.0.2.8.49997 5ESTABLISHED tcp 0 0 127.0.0.1.587 6*.* 7LISTEN tcp 0 0 127.0.0.1.25 *.* LISTEN tcp 0 0 8*.22 *.* LISTEN Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) 9udp 0 0 127.0.0.1.512 *.* udp 0 0 *.514 *.*
The list starts with open TCP ports 1. The Recv-Q and Send-Q columns 2 show the number of bytes that the system is in the process of receiving or trying to send.
The Local Address column shows the IP address attached to the local machine where this socket is listening. It’s possible—common, even—for a service to bind to a port on only a single address on a machine. If the port is part of an actual connection, as the first example 3 shows, the IP address is followed by the port number. This particular TCP connection is attached to port 22 at the address 192.0.2.135. Port 22 is reserved for SSH, so this is probably an SSH connection.
If the local address is an asterisk followed by a port number 8, this is a wildcard bind. A program has bound to this port, and has asked the kernel to figure out the IP address. It’s probably (but not necessarily) a listening socket.
The Foreign Address column 4 shows the IP address and port of the remote host involved in a connection. If there’s a foreign address shown, it always includes the port. If this column shows two asterisks 6, that means the service is waiting for a connection on the local port.
The (state) column applies only to TCP connections. A live and working TCP connection is in the ESTABLISHED state 5. Other states (SYN_RCVD, ACK, and SYN+ACK) are all normal parts of connection creation, while LAST_ACK, FIN_WAIT_1, and FIN_WAIT_2 mean that the connection is closing. A state of LISTEN 7 means that this socket is waiting for an incoming connection.
UDP ports are given their own section 9. You might see remote hosts in the UDP section, especially for long-running protocols such as NFS and NTP, but remember that UDP is stateless, so you’ll never see state on a UDP connection.
If you’re interested in only TCP or UDP sockets, you can use the -p flag to show only a particular protocol. Here, I look at TCP sockets:
$ netstat -na -p tcp
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 52 192.0.2.135.22 192.0.2.8.49997 ESTABLISHED
tcp6 0 0 ::1.587 *.* LISTEN
tcp 0 0 127.0.0.1.587 *.* LISTEN
tcp6 0 0 ::1.25 *.* LISTEN
tcp 0 0 127.0.0.1.25 *.* LISTEN
tcp 0 0 *.22 *.* LISTEN
tcp6 0 0 *.22 *.* LISTEN
While this looks similar to the first output example, note that we see both IPv4 and IPv6 TCP connections and services. TCP runs over both IPv4 and IPv6, so choosing it shows both address families. It’s entirely possible to have a service running on one address family and not the other. Many of my systems listen for incoming SSH connections only on IPv6; doing so hides me from port scanners and worms (for now, anyway).
Rather than listing every service waiting for an incoming connection, you can show only live connections by dropping the -a flag:
$ netstat -np tcp
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 52 192.0.2.135.22 192.0.2.8.49997 ESTABLISHED
Using fstat
Now that you know which TCP and UDP ports are open, how can you tell which programs are listening on them? OpenBSD includes fstat(1), a program that displays all open files and sockets on the system. Network connections are open sockets. Running fstat on an idle system can generate hundreds of lines of output—one entry for each file opened by any process. While that’s educational and useful, it’s not what we’re looking for. Specifically, we want to see which programs are holding network sockets open. The string internet indicates network sockets.
$ fstat | grep internet
mwlucas sshd 21403 3* internet stream tcp 0xd5365994 192.0.2.235:22 <-- 192.0.2.8:49997
root sendmail 19063 4* internet stream tcp 0xd537e330 127.0.0.1:25
root sendmail 19063 5* internet6 stream tcp 0xd537e4c8 [::1]:25
root sendmail 19063 6* internet stream tcp 0xd537e660 127.0.0.1:587
root sendmail 19063 7* internet6 stream tcp 0xd537e7f8 [::1]:587
root sshd 29561 3* internet6 stream tcp 0xd537e000 *:22
root sshd 29561 4* internet stream tcp 0xd537e198 *:22
_syslogd syslogd 12885 4* internet dgram udp *:514
First, you see an sshd process owned by the user mwlucas. That’s an unprivileged process, tied to a particular SSH session. Further down the list, you see an SSH daemon owned by root listening to the network. When a connection request arrives, the root-owned SSH daemon will hand it off to an unprivileged child process. You can also see that we have a variety of sendmail processes listening to the network.
This system runs the expected SSH and email servers, and no one has bound anything to odd ports. My nasty paranoid suspicions were unfounded (this time, anyway).
Between netstat and fstat, you should be able to get a good idea of what your system is doing on your network at any given time.