Understanding the TCP/IP model
Transmission Control Protocol
User Datagram Protocol
Understanding the TCP/IP Model
Transmission Control Protocol (TCP)
User Datagram Protocol (UDP)
Stream Control Transmission Protocol (SCTP)
Reliable User Datagram Protocol (RUDP)
In this chapter, we focus primarily on the TCP and UDP modes of transmission.
In the 1970s, two Defense Advanced Research Projects Agency (DARPA) scientists, Vint Cerf and Bob Kahn, often known as the fathers of the Internet, started researching reliable data communications across packet radio networks. From the lessons learned from the Networking Control Protocol (NCP), a set of protocols forming a part of Point-to-Point Protocol (PPP), Cerf and Kahn created the Transmission Control Program. Transmission Control Program was a huge success, and in 1974 it was initially standardized in RFC 675, Specification of Internet Transmission Control Program.
Application layer: This layer allows for process-to-process communication on the same host or different hosts. This layer leverages the lower layer protocols to transmit the information. The Application layer introduces different communication models such as the client/server model or peer-to-peer networking model. Some of the examples of aHTTP, FTP, and SSH.
Transport layer: This layer takes care of performing host-to-host communication that is either in a local LAN segment or remote network segments separated by routers. Two of the primary protocols in the Transport layer are TCP and UDP.
Internet layer: This layer defines the addressing and routing structures used by the TCP/IP protocols. IP defines the addressing that will be used by the hosts and network elements in the same or different segments and provides a function for hop-by-hop routing by sending datagrams to the next hop that holds the information to the next network segment.
Link layer: This layer is a combination of the Data Link layer and Physical layer of the OSI model. This layer provides information about hardware addresses (MAC address) and ensures physical transmission of data.
OSI Model vs. TCP/IP Model
OSI Model | TCP/IP Model |
---|---|
Transport layer is only connection-oriented. | Transport layer is both connection-oriented and connectionless. |
Allows users to standardize router, switch, motherboard, and other hardware. | Focuses only on establishing connection between different types of computers. |
Provides clear distinction between interfaces, services, and protocols. | Doesn’t provide clear distinction points. |
Problem of Ownership
Today, almost every IT-enabled organization depends on mission-critical applications to successfully run its business. Sometimes these applications are an important aspect of a company’s revenue generation model (e.g., an ecommerce portal). When those mission-critical applications stop working and the software developers are sure that it is not because of their code, they escalate the problem to network engineers or network administrators. The software developers usually own the Application layer, Presentation layer, and Session layer of the OSI model or just the Application layer of the TCP/IP model.
In 90 percent of the cases, if not more, network engineers verify the routing and reachability, which falls under the Link layer and Internet layer of the TCP/IP model (Physical layer, Data Link layer, and Network layer of the OSI model) and escalate it back to the application team, saying it is not their problem. This is where the finger-pointing game starts between the application developers and network engineers. If we take a close look at both the OSI model and the TCP/IP model, you will realize that both the application developers and the network engineers do not demonstrate ownership for an important layer—the Transport layer. Nobody wants to go the extra mile and check the information presented at the Transport layer. Instead of posing this as a challenge, we should be seeing this as an opportunity. Knowing how the data are transmitted and how host-to-host communication will happen can quickly help isolate the problem and help both the application developers and network engineers solve the problem quickly.
Transmission Control Protocol
When the initial research was being done by DARPA in 1973, the focus was on developing a protocol that would ensure secure transmission between two hosts while maintaining the integrity of the data, regardless of the amount of data being sent. DARPA and the University of Southern California collaborated and standardized the protocol specification of TCP, a protocol that provided a connection-oriented data transmission mechanism while ensuring data integrity, in RFC 793. RFC 793 was later updated by RFC 1122, RFC 3168, RFC 6093, and RFC 6528.
The current version of TCP allows two nodes or endpoints to establish a connection that enables two-way transmission of data; that is, a device can send and receive data at the same time. Each connection in TCP works in a client/server model, irrespective of which node assumes the role of the server or client, and each endpoint connection is uniquely identified using an ordered pair of IP address and port number. This ordered pair is known as a tuple or a socket. Thus, a TCP connection is often referred to as a socket connection. Now, before moving onto understanding how a TCP connection is established, let’s take a closer look at the TCP header, displayed in Figure 4-3.
Source port (16-bit): Specifies the port number of the sender.
Destination port (16-bit): Specifies the port number of the receiver.
Sequence number (32-bit): Used to keep track of the data (in bytes) sent out by the host during a TCP session. During a new TCP connection, the initial sequence number sent is a random 32-bit value. The receiver will use the sequence number and reply with an acknowledgment. When it comes to troubleshooting TCP issues, protocol analyzers often use a relative sequence number of 0 as it is easier to remember than some high-value random number. The sequence number is also used for validating the segments after transmission.
Acknowledgment number (32-bit): Used to keep track of every byte received by the receiver. An acknowledgment is sent in response to every packet that is received by the host.
Offset (4-bit): Specifies the length of the TCP header, which allows us to know where the actual data begins.
Reserved (6-bit): Reserved for further use as per RFC 793.
- Flags (6-bit): Enables various TCP actions for data processing and communication. The TCP software will perform specific actions when one or more flags are set in the TCP header. The following are the various flags that are set in TCP:
Urgent Pointer (URG)
Acknowledgment (ACK)
Push (PSH)
Reset (RST)
Synchronization (SYN)
Finish (FIN)
Window size (16-bit): Specifies the number of bytes the receiver is willing to receive. Using this field, the receiver tells the sender the amount of data that it is willing to receive.
Checksum (16-bit): Used to verify if the TCP header is okay or not. Using this field, TCP is able to reliably detect any transmission issues.
Urgent pointer (16-bit): This field is used to indicate how many bytes of packet data starting from the first byte are considered to be the urgent data by the sender. This field is only used when the URG flag is set under the Flags field.
- Options (0–320 bits): The TCP Options field is at the end of the header and is always specified in multiples of 8 bits. If any of the bits are not filled, they are padded with zeros. This field is used to include various TCP functions that do not belong to the general TCP header. The following is a list of various TCP functions available as part of the TCP Options field:
Maximum Segment Size (MSS)
Window Scaling
Selective Acknowledgments (SACK)
Timestamps
Nop
HTTP: Port 80
HTTPS: Port 443
Telnet: Port 23
SSH: Port 22
FTP: Port 21
DNS: Port 53
IMAP: Port 143
POP3: Port 110
Well-known ports: These port numbers range from 0 to 1023.
Registered ports: These port numbers range from 1024 to 49151. They are not assigned or controlled but can be registered to avoid any duplication.
Dynamic ports: These ports range from 49152 to 65535 and cannot be assigned, controlled, or registered.
When troubleshooting, TCP-related issues such as HTTP/TCP-based IP Service Level Agreement (SLA) probes that are deployed on network devices might stop working. As network engineer you might focus on validating the configuration or checking why the remote end has stopped responding, but it is equally important to keep an eye on the ports that are locally open on the box. It could be that even though the TCP session gets established, the connection does not terminate from time to time, and the device might run out of available ports to establish further TCP sessions.
TCP Flags
URG: The Urgent flag is set to signal the TCP application that the payload data must be processed urgently up to the set pointer in the Urgent Pointer field. Note that the Urgent Pointer field is relevant only when the URG flag is set in the TCP header.
ACK: The Acknowledgment flag is set in combination with the Acknowledgment Number field. This flag indicates the acknowledgment by the receiver that it has received the TCP packets that were previously sent.
PSH: The Transport layer, by default, stores the application data in a buffer for some time so that it can transmit data equal to the MSS size to ensure faster convergence and better performance in the network for TCP applications. Such behavior is not desirable, though, for certain applications, such as chat applications. Similar issues apply on the receiving end as well. The PSH flag in the TCP header solves this problem by telling the TCP software to immediately send the payload to the Network layer as soon as it receives the payload from the Application layer. In simple words, it tells the receiver and sender to immediately process the packets instead of buffering them.
RST: If the TCP software identifies an error during transmission, it sends an RST flag to reset the connection.
SYN: The SYN flag is the first step to initiate a TCP connection via the three-way handshake process.
FIN: – The FIN flag signals the receiver that the sender is ending the transmission.
TCP Three-Way Handshake
- 1.
SYN: In the first step, the client initiates a TCP connection toward the remote server. When it does that, the SYN flag is set to 1 in the TCP header and a random sequence number is used for this TCP connection. In this case, it is 926587467, as shown in Figure 4-6. Because this is the first packet, the ACK flag is set to 0. There are other fields that are also set in the TCP header such as Window size and MSS TCP options.
- 2.
SYN-ACK: When the server receives the SYN packet for the TCP connection, it responds back with an acknowledgment by setting the ACK flag bit to 1. Also, when the ACK flag is set, the Acknowledgment Number field is set to the value of one more than the received SYN packet. So, in this case, the Acknowledgment number field will have the value 926587468. Also, because TCP allows for two-way communication, the server also sets the SYN flag to 1 and sets a random sequence number in the TCP header. Note that the sequence number used in the SYN-ACK packet will be different than the one received from the client. In this case, it is set to 4227540456. Figure 4-7 displays the TCP SYN-ACK packet from server to client.
- 3.
ACK: On receiving the SYN from the server, the client now must respond back with an acknowledgment. For that, the client sends another TCP packet to the server with the ACK flag set, and the Acknowledgment Number field value set to the value of the sequence number plus 1. In this packet the SYN flag is set to 0. Figure 4-8 displays the ACK from the client to the server with the Acknowledgment Number field set to 4227540457. Note that after the ACK is received by the server, the minimum of the client or server’s MSS value is taken into consideration for data transmission.
FIN: When the client wishes to terminate the connection, it sends a TCP packet with the FIN flag set to 1 and sends it with a random sequence number. Note that at this point, the SYN flag will be set to 0 and the ACK is set to 0. If the client is supposed to send an acknowledgment to the server for the previously received TCP packet, the ACK flag can be set to 1 along with the acknowledgment number, but this ACK has no relation to the FIN flag that is set on the packet. TCP does this to reduce the number of packets being exchanged. Figure 4-10 shows the Wireshark capture of SSH connection being terminated when an ‘exit ’ command is typed on the terminal by the client. Figure 4-10 shows the Wireshark capture of TCP FIN packet when the client wishes to terminate the connection. Notice that in this packet, there is an ACK flag set as well, but this acknowledgment is for another sequence number that the client received.
FIN-ACK: On receiving the client’s termination request with TCP FIN, the server acknowledges the request by replying to the client with ACK. The server also sets the FIN flag to 1 and sends it to the client with a random sequence number different than that of the received FIN. Once this step is completed, the connection is terminated from the client to the server side. Figure 4-11 shows the Wireshark capture of a TCP FIN-ACK packet sent by the server toward the client in response to the FIN request received from the client.
ACK: This is the last step where the client sends the acknowledgment back to the server for the received FIN from the server. It sets the ACK flag to 1 and sets the Acknowledgment Number value to the Sequence Number of FIN plus 1. After this step is completed, the connection is terminated from the server to the client side. Figure 4-12 displays the Wireshark capture of the TCP ACK packet sent by the client to the server in response to the FIN packet it received from the server in the previous step. Note that the Sequence Number and Acknowledgment Number fields work in the same manner as they did during the session initiation process.
LISTEN: A TCP application is awaiting an inbound connection request.
SYN-SENT: A connection request has been sent but no acknowledgment has been received from the remote end.
SYN-RECEIVED: A connection request has been received and an acknowledgment has been sent to the remote host, but the host is awaiting an acknowledgment of the connection request sent out as a response to the original connection request.
ESTABLISHED: All SYN and ACK have been received and the connection has now been established. Both end hosts can start sharing the data.
FIN_WAIT_1: A session or connection termination request has been sent, but no acknowledgment has been received.
FIN_WAIT_2: An acknowledgment has been received from the remote host, but no corresponding termination request has been received from the remote host.
CLOSING: A session termination request has been sent and a corresponding session termination request has been received and acknowledged but no acknowledgment has been received from the remote host for the original session termination request.
CLOSE_WAIT: A session termination request was received and acknowledged but no corresponding session termination request has been sent out yet.
TIME_WAIT: The host waits for a reasonable amount of time to ensure the remote host receives the final acknowledgment of a session termination request.
LAST_ACK: Host awaits a final acknowledgment after sending an end of connection message in response to having received a session termination request.
Port Scanning
Many network and security analysts perform port scanning to find out about network and host vulnerabilities and services running on the network that can be exploited. Port scanning is a technique of determining which ports in a network or host are open to send or receive traffic. An open port indicates that a service such as HTTP/HTTPS or FTP is offered on the destination network or host. If attackers know what services are offered, they might be able to use other tools to identify security vulnerabilities to exploit those services.
Connect requests: In this technique an active connection is attempted using the three-way handshake. If the port is open, the three-way handshake is completed, and the scanner gracefully closes the connection by sending an active close request. If the port is closed, the destination responds back with an RST flag set. Note that this is not a safe scanning method, as these connection attempts are logged on the target host.
Half-open scan: In this technique, the three-way handshake is not completed and thus the name half-open scan. A SYN is sent by the scanner and it waits for a response. If the target port is open, it returns a SYN-ACK and the connection will be immediately torn down by the scanning host because it did not issue the connection request. Because the handshake never completed, the target host might not log these TCP SYN packet scans.
Non-SYN-ACK-RST scans: As per RFC 793, segments containing an RST flag are always discarded and segments containing an ACK always generate an RST flag. So, non-SYN packets that do not contain an RST or ACK could be used for port scanning. Note that this method of port scanning is only useful if the target host or network follows the RFC specifications. OSs that do not follow the RFC send RSTs from both open as well as closed ports, thus making it difficult for scanners to return accurate results.
SYN RST
SYN FIN
RST FIN
FIN
No flags
SYN Flag set: tcp && tcp.flags == 0x02
ACK flag set: tcp && tcp.flags == 0x10
RST flag set: tcp && tcp.flags == 0x04
FIN flag set: tcp && tcp.flags == 0x01
No flags set: tcp && tcp.flags == 0x3f
Investigating Packet Loss
Link errors/Layer 2 errors
Network congestion
Most of the time, once a network is set up, it runs smoothly. It could demonstrate transient or complete packet loss only when the hardware fails or the link has issues. Detecting hardware failures is not very complex, as multiple links and protocols running on the network hardware or host will start showing symptoms of hardware failure and can easily be fixed by replacing the complete hardware or a particular part that is causing the symptoms. When it comes to link issues, there could be several things to troubleshoot, some within our control and some outside. With a link, the issue could be with the unidirectional failures, Small Form-Factor Pluggables (SFPs), fiber or Ethernet cables, duplex settings, telco provider in the middle, and so on. The challenge with link issues is that even though the link might have errors, it will still forward some traffic and drop the remaining traffic, so network operators might not even know unless there is a notification of an event or a complaint from an end customer. With link issues, the data transmitted may also get corrupted and get eventually dropped. In most cases, an error counter on the network or host interface will increment to indicate an issue with the link, which then helps to identify and resolve the problem.
Traffic congestion, on the other hand, can cause a great deal of service disruption and is seen especially when transitioning between link speeds within the network (from 10 Gbps to 1 Gbps). If the higher speed link sends traffic at a rate the egress interface might not be able to keep up with, then it will start dropping the packets. In such cases, with TCP, the sender determines that the loss occurred in transit and will retransmit the packets. This scenario is also known as discards. Because TCP is a reliable connection-oriented protocol, it provides a mechanism to track data that have been sent and receive an acknowledgment of what has been received. If for any given packet the mapping ACK is not received, the TCP software resends the data assuming the packet has gone missing and ensuring reliable transmission of data. You might wonder why, after so much progress and innovation in the field of networking and development of 100 Gbps fiber links, we still have to deal with issues such as network congestion.
TCP Retransmission
TCP Out-of-Order Packets
Multiple paths: If the TCP segments are following multiple paths (ECMP paths to the destination) or via parallel processing paths within a router or a network equipment (e.g., per-packet load balancing), and either of the systems are not designed to ensure the ordering of the packets, this could lead to OOO packets in the network. Note that it is TCP’s job to deliver the packets in the right order.
QoS: Poorly configured QoS, especially a queueing mechanism, can cause OOO packets in the network. If the QoS settings do not forward the packet in a first in, first out (FIFO) manner or if the QoS settings drop the TCP packets along the path, this could lead to retransmission of those dropped TCP segments and eventually to OOO packets.
Oversubscription: Oversubscribed links in the network can cause OOO packets. The traffic will end up getting dropped, causing retransmission, slowdowns, and OOO packets.
Microbursts: A microburst is a behavior seen in networks when rapid bursts of data packets are seen in quick succession, leading to time intervals of full line-rate transmission. This can cause packets to get dropped due to buffer overflows on the interface. When such bursts occur in the network, links would end up dropping packets, causing retransmissions, slowness, and OOO packets.
When OOO segments are received by the TCP software, one of the main functions that it performs is reassembling packets in order or requesting retransmission of OOO packets. If the Wireshark capture shows that there are OOO packets, then as part of the troubleshooting process you might want to look at the possible causes listed earlier.
If you see a lot of TCP OOO packets, there is packet loss between the capture point and the sender. If you see a lot of TCP retransmissions, though, there is packet loss happening between the capture point and the receiver.
Troubleshooting with Wireshark Graphs
When troubleshooting TCP or any network issues in a large-scale environment, where there is large amount of data to be analyzed, it becomes challenging and time consuming to identify the root cause. In such a scenario, a quick peek at graphical data can give us a better understanding of what is happening in the network. Wireshark provides you with numerous graph options that can be used for investigating various types of issues. There are graphs in Wireshark that are specific for TCP and can come in handy for day-to-day network analysis and troubleshooting tasks. This section focuses on some of the various graphs that can be used.
TCP Stream Graphs
Time Sequence (Stevens)
Time Sequence (tcptrace)
Throughput
Round Trip Time
Window Scaling
Time Sequence (Stevens)
Time Sequence (tcptrace)
- 1.
The bytes in flight (blue lines or dots) touching the receive window (graph lines).
- 2.
Steps (these denote that the sender is not sending the data fast enough or it could be related to a receive window size issue).
Throughput Graph
In computer networks, goodput means the application-level throughput of a communication. It simply indicates good throughput of an application.
If the graph shows sporadic segments (dotted lines), it indicates that the device is sending sporadically as shown in Figure 4-25, and it usually indicates that there is packet loss in the path. If users click those sporadic segments, they might be able to see TCP retransmissions or OOO packets.
Window Scaling Graph
RTT Graph
I/O Graphs
Flow Graphs
Tracking any malicious host or application that is trying to access multiple servers on the network.
Tracking TCP retransmissions
Tracking connection resets
TCP Expert
TCP Display Filters and Their Functions
Display Filter | Function |
---|---|
tcp.flags == 0x2tcp.flags.syn == 1 | Capture all TCP SYN packets |
tcp.flags.reset == 1 | Capture TCP Resets |
(tcp.flags == 0x10) && (tcp.seq == 1) && (tcp.ack == 1) | Capture only third packet of the TCP three-way handshake |
tcp.time_delta > 1 | Filter TCP delays greater than t seconds; in this example, t = 1 |
tcp.time_delta > 1 && tcp.flags.fin == 0 && tcp.flags.reset == 0 | Identifying TCP delays but ignoring delays from the TCP connection termination process (during the connection termination process, TCP FIN is sent to the remote end, or the TCP reset flag is set) |
tcp.window_size >= 0 && tcp.window_size < 500 | Identifying small TCP window sizes |
tcp.analysis.out_of_order | Filtering TCP OOO packets |
(tcp.flags.syn == 1) && (!tcp.len == 0) | Filtering TCP SYN or SYN-ACK packets that contain data |
Wireshark Profile for TCP
No.
Time
Source
Destination
Protocol
Length
Info
- 1.
Right-click Profile: Default at the bottom right corner of the Wireshark application (Figure 4-35).
- 2.
Select the New option. This will open the profile modally.
- 3.
Create a profile named TCP and click OK.
- 4.
Right-click again at the bottom right corner of the Wireshark application and select the TCP profile from the Switch To submenu.
- 5.
Once selected, the new TCP profile will become your active profile. Note that at this point, this new profile will have the same columns and settings as the Default profile.
No.
Time
Delta
Source
Destination
TCP Delta
SEQ
ACK
Window
Bytes in flight
Info
SYN packets: There might be a delay before the initial SYN packet. For instance, once the Wireshark capture is started, you might ask the user to connect to a web server. There will be a delay in such a case before the first packet is seen on the wire.
Connection termination packets: TCP connection termination packets are basically FIN, FIN-ACK, RST, and RST-ACK. These packets are explicitly sent to close or terminate a connection. These packets could be sent when a user opens a new tab on the browser, or the session gets automatically closed after a page is loaded.
GET requests: GET requests can be generated in HTTP when a user clicks a link to open a new page or to request new data from the back end of the web application. Some GET requests are instant, but there might be GET requests initiated by background processes that might not have any priority, for instance, a GET request for .ico files.
DNS queries: DNS queries during a web browsing session are common and could lead to unexpected delays in response.
Image files: When a browser application requests image files or .ico files, there might be delays for such requests based on the web server settings or file size of the image.
User Datagram Protocol
No handshake mechanism
No session teardown
Smaller header size
Unreliable data delivery
No mechanism to manage OOO packets
No protection from data duplication
Source port: Identifies the sender’s port number.
Destination port: Identifies the receiver’s port number.
Length: Specifies the length in bytes of the UDP header and payload; minimum length is 8 bytes.
Checksum: This field is used to ensure the integrity of the data. This field stores the 16-bit words summed using 1’s complement arithmetic, which is calculated based on the IP header, the UDP header, and the payload.
Netstat Command for Verifying UDP
- 1.
Start the Wireshark application and start a capture on your computer’s NIC.
- 2.
Open a command prompt.
- 3.
Clear your DNS cache using the ipconfig /flushdns command.
- 4.
Try initiating a ping to a remote server or website from the command prompt.
- 5.
Close the command prompt.
- 6.
Stop the Wireshark capture.
Summary
Any network engineer or security analyst should have a deep and solid understanding of protocols working at different layers, but one of the layers that most engineers least on least is the Transport layer. The Transport layer protocols are crucial for ensuring end-to-end communication and transporting data between sender and receiver. To transport the data, the Transport layer has protocols that follow either connectionless or connection-oriented architecture with both having their respective use cases. In this chapter, we focused on the two key Transport layer protocols, TCP and UDP.
In this chapter, we explored the working of TCP and how it helps solves several problems such as data reliability, data integrity, and so on. We learned about the TCP connection process using three-way handshake, how port scanning is done by attackers, and how network engineers can investigate packet loss issues in the network. We learned that packet loss in the network can lead to issues such as TCP retransmissions and TCP OOO packets. We then saw how quick analysis and troubleshooting can be performed for network traffic using Wireshark graphs, including TCP Stream graphs, I/O graphs, and Flow graphs. As a network engineer or security analyst it is important to have custom profiles in Wireshark to analyze different types of traffic. We covered how users can create custom profiles for TCP and can quickly identify issues such as ZeroWindow by simply looking at the capture.
At the end of the chapter, we looked at UDP and in which scenarios UDP is used by different applications. We then learned how to filter UDP traffic and how we can leverage I/O graphs to learn about the UDP traffic pattern.
References in This Chapter
RFC 793: Transmission Control Protocol, DARPA, Information Sciences Institute University of Southern California, IETF, September 1981. http://tools.ietf.org/html/rfc793.
RFC 768: User Datagram Protocol, J. Postel, IETF, August 1980. http://tools.ietf.org/html/rfc768.