Four-Layer (“DoD”) IPv4 Architectural Model

Unlike the OSI network stack, which really does have seven layers, the DoD network model has four layers, as shown in the following.

A four-layer D o D model for I P V 4 has, from top to bottom, the application layer, transport layer, internet layer v 4, and the link layer.

An illustrated block diagram depicts how the data flows through the four layers of the D o D model. It contains a source, an I P packet, and a destination. Arrows mark the data flow direction.

It just confuses the issue to try to figure out which of the seven OSI layers the various protocols of TCP/IP fit into. It is simply not applicable. It’s like trying to figure out what color “sweet” is. The OSI seven-layer model did not even exist when TCP/IP was defined. Unfortunately, many people use terms like “layer 2” switches vs. “layer 3” switches. These refer to the OSI model. Books from Cisco Press and the Cisco certification exams are particularly adamant about using OSI terminology. I would be surprised if there is even a single actual OSI network running today. In this book we will try to consistently use the four-layer model terminology while referring to the OSI terminology when necessary for you to relate the topic to actual products or other books.

Note: Outgoing data begins in the application and is passed down the layers of the stack (adding headers at each layer) until it is written to the wire. Incoming data is read off the wire and travels up the layers of the stack (processing and removing headers at each layer) until it is accepted by the application. In the following discussion, for simplicity, I describe only the outgoing direction.

The Application Layer ²⁷ implements the protocols most people are familiar with (e.g., HTTP, SMTP, FTP). The software routines for these are typically contained in application programs such as browsers or web servers that make “system calls” to subroutines (or “functions” in C terminology) in the “socket API”²⁸ (an API is an Application Program Interface, or a collection of related subroutines, typically supplied with the operating system or C programming language compiler). The application code creates outgoing data streams and then calls routines in the socket API to actually send the data via TCP (Transmission Control Protocol) or UDP (User Datagram Protocol). Output to the Transport Layer is [DATA] using IP addresses.

The Transport Layer ²⁹ implements TCP ³⁰ (the Transmission Control Protocol) and UDP ³¹ (the User Datagram Protocol). These routines are internal to the socket API (hence live in Kernel Space ³²). In the case of TCP, packet sequencing, plus error detection and retransmission, is handled. The Transport Layer prepends a TCP or UDP packet header to the data passed down from the Application Layer and then passes the resulting packet down to the Internet Layer for further processing. Output to the Internet Layer is [TCP HDR[DATA]], using IP addresses.

The Internet Layer ³³ implements IP ³⁴ (the Internet Protocol) and various other related protocols such as ICMP ³⁵ (which includes the “ping” function among other things). The IP routine takes the data passed down from the Transport Layer routines, adds an IP packet header onto it, and then passes the now complete IPv4 packet down to routines in the Link Layer. Output to the Link Layer is [IP HDR[TCP HDR[DATA]]] using IP addresses.

The Link Layer ³⁶ implements protocols such as ARP ³⁷ (Address Resolution Protocol ) that map IP addresses to MAC addresses for transmission between nodes in a single network link. It contains protocols such as Ethernet, Wi-Fi, and MPLS. It also contains routines that actually read and write data (as fed down to it by routines in the Internet Layer) onto the network wire, in compliance with Ethernet or other standards. Output to wire: Ethernet frame containing the IP packet, using MAC addresses (or other Link Layer addresses for non-Ethernet networks).

Each layer “hides” the details (and/or hardware dependencies) from the higher layers. This is called “levels of abstraction.” An architect thinks in terms of abstractions such as roofs, walls, windows, etc. The next layer down (the builder) thinks in terms of abstractions such as bricks, glass, mortar, etc. Below the level of the builder, an industrial chemist thinks in terms of formulations of clay or silicon dioxide to create bricks and glass. If the architect tried to think at the chemical or atomic level, it would be very difficult to design a house. Their job is made possible by using levels of abstraction. Network programming is analogous. If application programmers had to think in terms of writing bits to the actual hardware, applications such as web browsers would be almost impossible. Each Network Layer is created by specialists who understand the details at their level, and lower layers can be treated as “black boxes” by people working at the higher layers.

Another important thing about Network Layers is that you can make major changes to one layer, without impacting the other layers much at all. The connections between layers are well defined and don’t change (much). This provides a great deal of separation between the layers. In the case of IPv6, the Internet Layer is almost completely redesigned internally, while the Link Layer and Transport Layer are not affected much at all (other than providing more bytes to store the larger IPv6 addresses). If your product is “IPv6-only,” that’s about the only change you would need to make to your application software (unless you display or allow entry of IP addresses). If your application is “dual stack” (can send and receive data over IPv4 or IPv6), then a few more changes are required in the Application Layer (e.g., to accept multiple IPv4 and IPv6 addresses from DNS and try connecting to one or more of them based on various factors or to accept incoming connections over both IPv4 and IPv6). This makes it possible to migrate (or “port”) network software (created for IPv4) to IPv6 or even dual stack with a fairly minor effort. In comparison, changing network code written for TCP/IP to use OSI instead would probably involve a complete redesign and major recoding effort.

Relevant Standards for IPv4

IPv4 Packet Header Structure

So what are these packet headers mentioned previously? In IPv4 packets, there is an IPv4 packet header,³⁸ then a TCP (or UDP) packet header, and then the packet data. Each header is a structured collection of data, including things such as the IPv4 address of the sending node and the IPv4 address of the destination node. Why are we getting down to this level of detail? Because some of the big changes from IPv4 to IPv6 have to do with the new and improved IP packet header architecture in IPv6. In this chapter, we’ll cover the IPv4 packet header. Here it is.

An illustration depicts the I P v 4 packet header. The packet consists of 7 layers. The first part of the packet is the I P version, and the last part consists of the data.

The IP Version field (4 bits) contains the value 4, which in binary is “0100” (you’ll never guess what goes in the first 4 bits of an IPv6 packet header!).

The Header Length field (4 bits) indicates how long the header is, in 32-bit “words.” The minimum value is “5,” which would be 160 bits, or 20 bytes. The maximum length is 15, which would be 480 bits, or 60 bytes. If you skip that number of words from the start of the packet, that is where the data starts (this is called the “offset” to the data). This will only ever be greater than 5 if there are options before the data part (which is not common).

The Type of Service field (8 bits) is defined in RFC 2474,³⁹ “Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 headers,” December 1998. This is used to implement a fairly simple QoS (Quality of Service ). QoS involves management of bandwidth by protocol, by sender, or by recipient. For example, you might want to give your VoIP connections a higher priority than your video downloads or the traffic from your boss higher priority than your co-worker’s traffic. Without QoS, bandwidth is on a first come–first served basis. 8 bits are not really enough to do a good job on QoS, and DiffServ is not widely implemented in current IPv4 networks. QoS is greatly improved in IPv6.

The Total Length field (16 bits) contains the total length of the packet (including the packet header) in bytes. The minimum length is 20 (20 bytes of header plus 0 bytes of data), and the maximum is 65,535 bytes (since only 16 bits are available to specify this). All network systems must handle packets of at least 576 bytes, but a more typical packet size is 1508 bytes. With IPv4, it is possible for some devices (like routers) to fragment packets ⁴⁰ (break them apart into multiple smaller packets) if required to get them through a part of the network that can’t handle packets that big. Packets that are fragmented must be reassembled at the other end. Fragmentation and reassembly is one of the messy parts of IPv4 that got cleaned up a lot in IPv6. A lot of hacking attacks exploit the messy scheme in IPv4.

The Identification (Fragment ID) field (16 bits) identifies which fragment of a once larger packet this one is, to help in reassembling the fragmented packet later. In IPv6 packet fragmentation is not done by intermediate nodes, so all the header fields related to fragmentation are no longer needed.

The next three bits are flags related to fragmentation. The first is reserved and must be zero (an April Fool’s RFC ⁴¹ once defined this as the “evil” bit, which the sender should set if they are doing something malicious). The next bit is the DF (Don’t Fragment) flag. If DF is set, the packet cannot be fragmented (so if such a packet reaches a part of the network that can’t handle one that big, that packet is dropped). The third bit is the MF (More Fragments) flag. If MF is set, there are more fragments to come. Unfragmented packets of course have the MF flag set to zero.

The Fragment Offset field (13 bits) is used in reassembly of fragmented packets. It is measured in 8-byte blocks. The first fragment of a set has an offset of 0. If you had a 2500-byte packet, and were fragmenting it into chunks of 1020 bytes, you would have three fragments as follows:

A table with five columns and four rows. The column headers are fragment I D, M F flag, total length, data size, and offset.

The Time-To-Live (TTL) field (8 bits) is to prevent packets from being shuttled around indefinitely on a network. It was originally intended to be lifetime in seconds (hence the name), but it has come to be implemented as “hop count.” This means that every time a packet crosses a switch or router, the hop count is decremented by one. If that count reaches zero, the packet is dropped. Typically, if this happens, an ICMPv4 message (“Time Exceeded”) is returned to the packet sender. This mechanism is how the traceroute command works. The primary purpose of TTL is to prevent looping (packets running around in circles).

The Protocol field (8 bits) defines the type of data found in the data portion of the packet. Protocol numbers are not to be confused with ports. Some common protocol numbers are

A table with 7 rows and 3 columns contains protocol details. The first column has the protocol numbers, and the next has the protocol acronyms, followed by their corresponding abbreviations.

The Header Checksum field (16 bits) is the 16-bit one’s complement of the one’s complement sum of all 16-bit words in the header. When computing, the checksum field itself is taken as zero. To validate the checksum, add all 16-bit words in the header together including the transmitted checksum. The result should be 0. If you get any other value, then at least 1 bit in the packet was corrupted. There are certain multiple bit errors that can cancel out, and hence bad packets can go undetected. Note that since the hop count (TTL) is decremented by one on each hop, the IP header checksum must be recalculated at each hop. The IP header Checksum was eliminated in IPv6.

The Source IP Address field (32 bits) contains the IPv4 address of the sender (may be modified by NAT).

The Destination IP Address field (32 bits) contains the IPv4 address of the recipient (may be modified by NAT in a reply packet).

Options (0–40 bytes) is not often used. These are not relevant to this book. If you want the details, read the RFCs.

Data (variable number of bytes) is the data part of the packet – not really part of the header. This is not included in the IP header checksum. The number of bytes in the Data field is the value of “Total Length” minus the value of “Header Length.”

An illustration depicts the I P v 4 and I P v 6 packet headers side by side, showcasing their difference.

A text box represents ten changes between the I P v 4 and I P v 6 headers.

IPv4 Addressing Model

In IPv4, addresses are 32 bits in length. They are really just integer numbers from 0 to 4,294,967,295. For the convenience of humans, these numbers are typically represented in dotted decimal notation . This splits the 32-bit addresses into four 8-bit fields and then represents each 8-bit field with a decimal number from 0 to 255. These decimal numbers cover all possible 8-bit binary patterns from 0000 0000 to 1111 1111. The decimal numbers are separated by “dots” (periods). Leading zeros can be eliminated. The following are all valid IPv4 addresses represented in dotted decimal:

Four I P v 4 addresses consist of a globally routable address, a private address, a broadcast address, and a loopback address.

Originally there were five classes of IPv4 addresses, as defined in RFC 791,⁴² “Internet Protocol,” September 1981:

Class A: First bit 0 (0.0.0.0–127.255.255.255), 8-bit network number, 24-bit node within network number, subnet mask 255.0.0.0. There are 128 class A networks, each containing 16.7M addresses.

Class B: First 2 bits “10” (128.0.0.0–191.255.255.255), 16-bit network number, 16-bit node within network number, subnet mask 255.255.0.0. There are 16,384 class B networks, each containing 65,536 addresses.

Class C: First 3 bits “110” (192.0.0.0–223.255.255.255), 24-bit network number, 8-bit node within network number, subnet mask 255.255.255.0. There are 2M class C networks, each containing 256 addresses.

Class D: First 4 bits “1110” (224.0.0.0–239.255.255.255), used for multicast.

Class E: First 4 bits “1111” (240.0.0.0–255.255.255.255), experimental/reserved (not forwarded by most routers).

Network Ports

Each IP address on a network node has 65,536 ports associated with it (the port number is a 16-bit value, and 2 to the 16th is 65,536). Any of those ports can either be used to make an outgoing connection or to accept incoming connections. There is a list of well-known ports ⁴³ that associates particular ports with certain protocols. For example, port 25 is associated with SMTP. There is nothing magical (or email-ish) about port 25. SMTP will work just as well on any other port, for example, 10025. Use of port 25 for SMTP is simply a convention that many people adopt. Such conventions make it easier to locate the SMTP server on a node you might not be familiar with. To be specific, ports are a Transport Layer thing, and there are really 65,536 TCP ports and another 65,536 UDP ports for each address. IP and ICMP, which are Internet Layer things, do not have any port(s) associated with them.

Anyone can reserve port numbers ⁴⁴ with IANA. I happen to have been awarded two port numbers, 4604 for my Identity Registration Protocol (IRP) and 4605 for my SixChat protocol. IANA reviewed both requests and determined that they were innovative (did not duplicate any other protocols) and viable (did something useful) and met all requirements for a modern protocol (e.g., support for Explicit TLS).

A screenshot depicts a table containing the details of an I R P port number registration process. It has eight columns and two rows.

When you deploy an Internet server (e.g., an SMTP server for sending and receiving email), the software opens a socket (a programming abstraction) in listen mode on a particular port (in the case of SMTP, port 25). An email client that wants to connect to it creates its own socket in connect mode and tells it to connect to a particular IP address (that of the SMTP server) using a particular port (in this case 25). When the connection attempt reaches the server, the server detects the attempt and accepts the connection (actually the port on the server that the connection is accepted on will be any available port, typically higher than 1024). A well-written server would then make a clone of itself (this is called forking in UNIX speak) and then go back to listening for further connections, while its clone went ahead and processed the connection. When the processing is complete on a given connection, the sockets used would be closed (on both server and client), and the clone of the server will quietly commit suicide. In theory you could have thousands of clones of the server all simultaneously handling email connections on a single server (given sufficient memory and other resources). Busy web servers (like those at Google) often have many thousands of connections being processed at any given time (but never more than 65,000 on a given interface – each connection uses up one port).

If threads are used instead of processes, the scheme is similar but has far less overhead.

In UNIX, ports with numbers under 1024 are special, and only software that has root privilege can use them. Most common Internet services use ports in that range. There are many well-known ports , but here are a few of the more common ones:

An image displays eight port numbers, along with their acronyms and abbreviations.

IPv4 Subnetting

This leads us naturally into the topic of IPv4 subnetting.⁴⁵ This is one of the more difficult areas of networking for people learning to work with IPv4. All addresses have two “parts,” the first part being the address of the network (e.g., 192.168.0.0) and the second being the node within that network (e.g., 0.0.2.5). These two parts can be split apart at some “bit boundary.” In this case, the address of the network is in the first 16 bits, and the node within the network is in the last 16 bits. The addresses of all nodes in such a network share the same first 16 bits, but each has a unique last 16 bits. So such a network might have nodes with addresses 192.168.2.5, 192.168.3.7, and 192.168.200.12, but not one with the address 192.169.2.1 (that address is in network 192.169.0.0, not network 192.168.0.0).

A subnet ⁴⁶ mask is a 32-bit value in which the first n bits (n=1–32) have the value 1 and the remaining 32-n bits have the value 0. It is used to split an IPv4 address into its two parts (the first n bits and the last 32-n bits). In the network just described, the subnet mask is 255.255.0.0 (the first 16 bits have the value 1; the last 16 bits have the value 0). You do a Boolean “AND” function of the address with the subnet mask to get the address of the network and a Boolean “AND” of the address with the one’s complement of the subnet mask (in this case 0.0.255.255) to get the node within subnet. This is difficult to visualize in dotted decimal. It is rather more obvious in binary. The Boolean “AND” function produces a 1 if both inputs are 1; else, it produces a 0. The “one’s complement” (Boolean “NOT”) function changes each 0 to a 1 and each 1 to a 0. With the “AND” function, where there is a 1 in the mask, the corresponding bit of the address “flows through” to the result. Where there is a 0 in the mask, the corresponding bit of the address is blocked (forced to the value 0). The following example (with addresses and mask shown in both dotted decimal and binary) should make this clear.

An illustration of an I P address using a subnet mask and without it. It presents the I P address in both decimal and binary forms.

For subnet mask 255.0.0.0 (class A), the first 8 bits are the network address, and the last 24 are the node within subnet.

For subnet mask 255.255.0.0 (class B), the first 16 bits are the network address, and the last 16 are the node within subnet.

For subnet mask 255.255.255.0 (class C), the first 24 bits are the network address, and the last 8 are the node within subnet.

Subnetting was easy when the three IP address classes (A, B, and C) were used. The first few bits of the address determined the subnet mask. If the first bit of the 32-bit address was “0,” then the address was class A, and the subnet mask was 255.0.0.0. If the first 2 bits of the address were “10,” then the address was class B, and the subnet mask was 255.255.0.0. If the first 3 bits of the address were “110,” then the address was class C, and the subnet mask was 255.255.255.0. This could actually be done automatically, so no one worried about subnet masks.

One of the changes made in the IPv4 addressing model in the mid-1990s was to introduce Classless Inter-domain Routing,⁴⁷ in RFC 1519,⁴⁸ “Classless Inter-domain Routing (CIDR),” September 1993. It was later replaced by RFC 4632 ⁴⁹, “Classless Inter-domain Routing (CIDR) ,” August 2006.

When CIDR was introduced, there were two consequences. First, the split between the two parts of the address could come at any bit boundary, not just after 8, 16, and 24 bits. Second, several small blocks (e.g., /28 blocks) could be carved out of a bigger block anywhere in the address space (perhaps from an old class A block, such as 7.x.x.x), so you could no longer determine the correct subnet mask by looking at the first few bits of an address. For example, a “/8” (class A) block might be carved up into 65,536 “/24” (class C) subnets, which could be allocated to different organizations.

Let’s say your ISP, instead of giving you a class C block, only gives you a “/28” block of real (routable) IPv4 addresses, which would be 16 real IPv4 addresses, for example, 123.45.67.0 through 123.45.67.15. First, two of these addresses are not usable (may not be assigned to nodes). 123.45.67.0 is the “network address,” and 123.45.67.15 is the “broadcast address.” That leaves 14 usable addresses (123.45.67.1 through 12.45.67.14). So what is your subnet mask? If you check the preceding table of useful CIDR block sizes, a /28 subnet has a subnet mask of 255.255.255.240. In binary that is 1111 1111 1111 1111 1111 1111 1111 0000 (first 28 bits are 1; last 4 bits are 0). However, by the old rules (first bit is a 0), these are really from a “class A” block, so the automatically generated subnet mask would have been 255.0.0.0, which is not correct.

Now, what if your organization really has 100 nodes that need IP addresses? How do you give each of them unique addresses if you only have 14 usable addresses to work with? That’s where Network Address Translation (NAT) comes in (covered in the next chapter). If you think CIDR made your life more “interesting,” wait until you see what NAT does to it! Getting rid of CIDR and NAT is one of the big wins in IPv6. In fact, you will find that the entire subject of subnets has become totally trivial.

MAC Addresses

IPv4 addresses are not actually used at the lowest layer of the IPv4 network stack (the Link Layer). Each network hardware interface actually has a 48-bit “MAC address ” burned into it by the manufacturer. The first 24 bits of this (called the “Organizationally Unique Identifier ⁵⁰” or OUI) specify the manufacturer and are purchased by vendors from the IEEE ⁵¹ (Institute of Electrical and Electronics Engineers). A given vendor may have multiple OUIs, but a given OUI is associated only with one vendor. The last 24 bits of this (called the “Network Interface Controller–Specific” part) are assigned by each manufacturer, to be unique within a given OUI. This means that the entire 48-bit value is globally unique. For example, Dell Computer has a number of OUIs assigned to them by the IEEE, including 00-06-5B, 00-08-74, and 00-18-8B. If you encounter a NIC with a MAC address that has one of those sets of 24 bits, it was made by Dell Computer. When you use the command “ipconfig /all” in Windows, you get a list of network configuration information for all your interfaces (some of which are “virtual”). If you look for “Local Area Connection,” that is information about your main (or only) network connection to your LAN. Under that, you will see an item labeled “Physical Connection,” followed by six pairs of hex digits, separated by dashes. That is the MAC address of your Network Interface Controller (NIC) . Mine is 00-18-8B-78-DA-1A. This means my NIC was made by Dell (my whole computer was, but the MAC address doesn’t tell you that). Actually, since the NIC I’m using is on the motherboard (not an add-on PCI card), this does tell me the motherboard was made by Dell.

You can look up the vendor of any device based on its OUI (or MAC-48 address). See

www.whatsmyip.org/mac-address-lookup/

This site tells me that the Ethernet adapter in my desktop computer (MAC address 9C-5C-8E-8F-2F-B0) was created by ASUSTek Computer Inc.

Network switches come in two varieties. “Layer 2” switches (which I would call Link Layer switches) only work with MAC addresses. They don’t even “see” IP addresses. Hence, “Layer 2” switches are IP version agnostic; they work equally well with IPv4 or IPv6 or a mixture of the two (dual stack). “Layer 3” switches (sometimes called “smart” switches) work with MAC addresses, but they also understand and can see IP addresses (these work at both the Link Layer and the Internet Layer, in terms of the four-layer Model). They can do things like create VLANs (Virtual LANs) to segregate traffic based on IP addresses. An IPv4-only “layer 3 switch” cannot work with IPv6 traffic (or at least none of its “higher-level” functions will affect IPv6 traffic). There are now a few dual-stack “layer 3” switches on the market, such as the SMC 8848M, which I happen to be running in my home network. I can even manage it over IPv6 (via Web and SNMP) and create VLANs ⁵² based on IPv6 addresses.

Mapping from IPv4 Addresses to Link Layer Addresses

The software in the Application Layer, the Transport Layer, and the Internet Layer of the IPv4 stack work with IP addresses. But the Link Layer (and the hardware) works with MAC addresses (or other Link Layer addresses). How do IPv4 addresses get mapped onto Link Layer addresses?

Address Resolution Protocol (ARP)

There are two protocols in IPv4 (that don’t even exist in IPv6) called ARP ⁵³ (Address Resolution Protocol ) and InARP ⁵⁴ (Inverse Address Resolution Protocol ). These protocols live in the Link Layer. ARP maps IP addresses onto Link Layer addresses. This is kind of like the mapping between FQDNs and IP addresses done in the Application Layer by DNS, but down in the Link Layer. InARP maps Link Layer addresses onto IP addresses (kind of like a reverse DNS lookup).

ARP is defined in RFC 826,⁵⁵ “An Ethernet Address Resolution Protocol,” November 1982. ARP operates only within the network segment (routing domain) that a host is connected to. It does not cross routers. It is used to determine the necessary Link Layer addresses to get a packet from one node in a subnet to another node in the same subnet (which could be a “default gateway” node that knows how to relay it on further). But for the hop from the sender to the default gateway, it is the same problem as getting the packet to any other local node. When the sender goes to send a packet, if the recipient’s address is on the local link, an ARP request is done for the recipient’s address, and the packet is sent to the recipient. If the recipient’s address is not on the local link, an ARP request is done instead for the sender’s default gateway address, and the packet is sent to the default gateway node, which will then worry about forwarding it on toward the real recipient.

Say Alice (one IPv4 node) wants to send a packet to Bob (another IPv4 node, on the same Ethernet network segment). Assume Alice does not currently know Bob’s MAC address. Each machine has a table of IP addresses and MAC addresses (called an ARP table). At this time, there is no entry in Alice’s ARP table with Bob’s IP address and MAC address. So Alice first sends an Ethernet ARP request to all machines on the network segment (using the Ethernet broadcast address), with the following info:

A few lines of code contain information that is part of an Ethernet A R P request sent over I P v 4.

All machines on the network segment will receive the packet, but everyone other than Bob will ignore it (“Not for me – IGNORE!”). Bob understands Ethernet protocol and IPv4. He recognizes his own IPv4 address (“It’s for ME!”). He adds Alice’s IPv4 address and Alice’s MAC address into his ARP table (for future reference) and then sends a response Ethernet ARP packet back to Alice, using her MAC address (which he now knows) instead of the broadcast address , with the following info:

A few lines of code contain information that is part of an Ethernet A R P response sent over I P v 4.

Only Alice gets the response (this was not a broadcast). Alice sees that this is a RESPONSE, and the sender’s address tells her whom the response was from. Alice then adds Bob’s IP address and MAC address into her ARP table. Now that she knows how to send things to Bob, she goes ahead with sending the packet that she originally was trying to send. This process is called address resolution , hence the name Address Resolution Protocol.

The ARP table has expiration times (TTL), and when an entry becomes “stale,” it will be discarded, and the next time a packet is sent to that address, a new fresh entry will be added to the ARP table.

In Windows, you can view your ARP table at any time, in a DOS window , with the command “arp –a”. The results might look something like the following.

The output table of a r p - a command in D O S windows. It contains three columns, internet address, physical address, and type.

Inverse ARP (InARP)

There is another protocol called Inverse ARP (InARP) that maps Link Layer addresses onto IP addresses. InARP is defined in RFC 2390,⁵⁶ “Inverse Address Resolution Protocol,” September 1998.

InARP is needed only by a few network hardware devices (like ATM). It works almost exactly like ARP, except different opcodes are used and the sender sends the recipient’s MAC address (which it knows), but zero fills the recipient’s IP address (which it wants to know). The recipient recognizes its own MAC address and responds with the same information that it does to an ARP. The older RARP (Reverse ARP) protocol is now deprecated.

IPv4 Broadcast

Any node can send a packet to a special IPv4 address (255.255.255.255), and all nodes on the local link will receive it. Any destination address that has all ones in the “node within subnet” field is broadcast (e.g., 172.16.255.255 in 172.16/16). Usually, there is some kind of information in the packet that allows most nodes to realize that packet does not concern them (e.g., if a broadcast packet contains a DHCPv4 request, all nodes that don’t have a DHCPv4 server will ignore it). This mechanism can help locate servers or solve other problems (like not yet having a valid IP address), but it can put unnecessary loads on all nodes that aren’t involved. It can also lead to broadcast storms, which involve massive amounts of useless traffic clogging or totally shutting down an IPv4 network. As an example, a “smurf attack”⁶¹ sends zillions of pings to the broadcast address with the source address containing the spoofed address of the node under attack (not the address of the actual sender). All nodes on the local link “respond” to the poor node under attack, which amplifies the attack. There are certain kinds of misconfigurations or hardware failures in network switches that can cause broadcast storms as well.

Packets sent to the broadcast address do not cross routers (or VLAN boundaries), so appropriate use of these can limit the extent of disruption due to excessive broadcasts or storms. The set of nodes that a broadcast will reach is called a broadcast domain. Switches do not block broadcasts – they relay packets with a broadcast destination address out all ports (unlike packets with a unicast destination address).

Broadcast is used in the DHCPv4, to allow a node to find and communicate with the DHCPv4 server before it even gets an address.

Broadcast does not exist in IPv6, because it can be so trouble-prone. Other mechanisms (e.g., multicast or solicited node multicast) are used to locate DHCPv6 servers or solve other problems for which broadcast may be used in IPv4.

IPv4 Multicast

Multicast allows a node to transmit a stream of data to one of a number of special “multicast ” addresses. Multicast supports only UDP, not TCP. Any number of other nodes can subscribe to that address and receive the datagrams. As one example, this could be used to send “broadcast” (in the media sense) radio or television programs. Multicast packet transmission differs from broadcast packet transmission in that only nodes that have subscribed to that multicast address receive the packets.

Sites like YouTube, and services like “on-demand” television, use traditional unicast (one sender connecting to one recipient) transmissions to each user. This requires a great deal of bandwidth and a powerful network infrastructure at the transmission site, especially if there are a large number of recipients (potentially millions). Multicast is necessary to bring costs and network bandwidth requirements low enough to make it competitive with media “broadcast” over satellite or cable systems.

Assuming there is a multicast program available on a particular multicast address (e.g., 239.1.2.3), a consumer can use a multicast client application to extend the distribution tree associated with that address to reach their computer. This corresponds to selecting a channel on a television. There may be multiple routers between the sender and this subscriber. All those routers must support multicast and be informed to replicate packets from the sender to that recipient. IGMP ⁶² is used to subscribe to a specific multicast address, and PIM ⁶³ is used to inform all intervening routers to extend the distribution tree to this client. The multicast server does not need to know anything about the recipients and does not get any response from them. The creation of the distribution tree and subscriptions to particular multicast addresses are handled by the clients and intervening multicast routers, not by the multicast server.

Unlike unicast routers, a multicast router does not need to know how to reach all possible distribution trees, only those for which it is passing traffic from a sender to a recipient. If there is no recipient subscribed to a given channel “downstream” from a router (from the sender to recipient), there is no need for it to replicate packets and forward them downstream . If a recipient downstream from that router subscribes to a particular address, then that router will start replicating incoming upstream packets from the multicast address and relay them downstream toward that recipient (or recipients). This is called adding a “graft” onto the tree. If there are recipients downstream on a particular path from a multicast router and the last one “tunes out,” then the last router in the path between the server and that node is informed to stop replicating packets along that path. This is called “pruning” the distribution tree. It is possible that one subscriber “tuning out” could result in an entire chain of multicast routers being pruned if there are no other subscribers down that path.

Multicast is often used for services such as IPTV, including applications such as distance learning. Not all IPv4 routers support multicast and the related protocols, so IPv4 multicast works best in “walled garden” networks, for example, within a single ISP’s network (e.g., Comcast subscriber accessing multicast content from Comcast). In such a situation, it is possible to ensure that all intervening routers support the necessary protocols (which are optional in IPv4).

It is possible to build a fully IPv4 multicast-compliant router using open source operating systems and an open source package called XORP ⁶⁴ (eXtensible Open Router Platform , at www.xorp.org ). XORP was first developed for FreeBSD, but is available on Linux, OpenBSD, NetBSD, and Mac OS X. The XORP technology and team was transferred to a commercial startup backed by VCs (called XORP Inc.⁶⁵). Many modern enterprise-class routers support Ipv4 multicast, but not all do. Not many small office/home office (SOHO)–class routers do. In IPv6, multicast is an integral part of the standard, and support is mandatory in all IPv6-compliant devices. It also works in a very different way and is much more scalable.

Internet Relay Chat ⁶⁶ (IRC) uses a different approach to multicast (not the standard multicast protocols) and creates a spanning tree across its overlay network to all nodes that subscribe to a given chat channel. Unlike multicast-delivered media content, IRC is a two-way channel.

Relevant Standards for IPv4 Multicast

Internet Group Management Protocol (IGMP)

IGMP ⁶⁷ is an Internet Layer protocol that supports IPv4 multicast. It manages the membership of IPv4 multicast groups and is used by network hosts and adjacent multicast routers to establish multicast group membership. There are three versions of it so far. IGMPv1 is defined in RFC 1112, “Host Extensions for IP Multicasting,” August 1989. IGMPv2 is defined in RFC 2236, “Internet Group Management Protocol, Version 2,” November 1997. IGMPv3 is defined in RFC 3376,⁶⁸ “Internet Group Management Protocol , Version 3,” October 2002.

Some “layer 2” switches have a feature called “IGMP snooping,” which allows them to look at the “layer 3” packet content, to enable multicast traffic to go only to those ports that have subscribers on them while blocking it (and thereby reducing unnecessary traffic) on ports with no subscribers. A switch without IGMP snooping will flood all connected nodes in the broadcast domain with all multicast traffic. This can be used by hackers to “deny service” to clients who are too busy receiving and ignoring multicast traffic to handle useful traffic. This is called a Denial of Service, or DoS, attack. Active IGMP snooping is described in RFC 4541,⁶⁹ “Considerations for Internet Group Management Protocol (IGMP) and Multicast Listener Discovery Protocol (MLD) Snooping Switches,” May 2006.

Relevant Standard for IPv4 Routing

In Windows, you can view all currently known routes with the “route print” command.

The output of route print command in windows. The first part of the output contains the interface list, and the next part includes the I P v 4 route table, which has active and persistent routes.

There are several routing protocols for IPv4 that are typically handled only in the core or where a customer network meets the core, the edge router. These include RIP, RIPv2, EIGRP, IS-IS, OSPF, and BGP.

TCP/IP routing is a very deep, complex subject, and we will be touching only on the most obvious aspects in this book, to give a rough idea of the differences in routing between IPv4 and IPv6.

RIP: Routing Information Protocol ,⁷⁹ version 1. Defined in RFC 1058,⁸⁰ “Routing Information Protocol,” June 1988. This protocol is very old and of primarily historic interest, since it does not support address blocks based on CIDR ⁸¹ (it is a classful routing protocol). It is used to exchange routing information with gateways and other hosts. It is based on the distance vector algorithm,⁸² which was first used in the ARPANET, circa 1967. RIP is a UDP-based protocol, using port 520.

RIPv2: Routing Information Protocol, version 2.⁸³ Defined in RFC 2453,⁸⁴ “RIP Version 2,” November 1998. Although OSPF and IS-IS are superior, there were so many implementations of RIP in use it was decided to try to improve on it. Extensions were made to incorporate the concepts of autonomous systems (ASs), IGP/EGP interactions, subnetting and authentication, as well as address blocks based on CIDR (it is a “classless” routing protocol). The lack of subnet masks in RIPv1 was a particular problem. RIPv2 is limited to networks whose longest routing path is 15 hops. It also uses fixed “metrics” to compare alternative routes, which is an oversimplification. However, RIPv2 becomes unstable if you try to account for different metrics. See RFC for details.

EIGRP: Enhanced Interior Gateway Routing Protocol .⁸⁵ This is not an IETF protocol, but a Cisco proprietary routing protocol based on their earlier IGRP.⁸⁶ EIGRP is able to deal with addresses allocated via CIDR (it is a classless routing protocol), including use of variable-length subnet masks. It can run separate routing processes for IPv4, IPv6, IPX, and AppleTalk protocols, but does not support translation between protocols. For details, see Cisco documentation. There is an RFC that covers a subset of the full Cisco EIGRP, RFC 7868,⁸⁷ “Cisco’s Enhanced Interior Gateway Routing Protocol (EIGRP),” May 2016.

IS-IS: Intermediate System to Intermediate System routing protocol .⁸⁸ IS-IS (pronounced “eye-sys”) was originally developed by Digital Equipment Corporation (DEC) as part of DECnet Phase V and formally defined as part of ISO/IEC 10589:2002 for the Open System Interconnection reference design. It is not an Internet standard, although the details are published as Informational RFC 1142,⁸⁹ “OSI IS-IS Intra-domain Routing Protocol,” February 1990 (since reclassified as historic by RFC 7142 ⁹⁰ in 2014). Another RFC specifies how to use IS-IS for routing in TCP/IP and/or OSI environments: RFC 1195,⁹¹ “Use of OSI IS-IS for Routing in TCP/IP and Dual Environments,” December 1990. IS-IS is an Interior Gateway Protocol, for use within an administrative domain or network. It is not intended for routing between autonomous systems, which is the role of BGP. It is not a distance vector algorithm; it is a link-state protocol.⁹² It operates by reliably flooding network topology information through a network of routers, allowing each router to build its own picture of the complete network. OSPF (developed by the IETF about the same time) is more widely used, although it appears that IS-IS has certain characteristics that make it superior in large ISPs.

OSPFv2: Open Shortest Path First ,⁹³ version 2. Unlike EIGRP and IS-IS, OSPF is an IETF standard. OSPFv2 is defined in RFC 2328,⁹⁴ “OSPF Version 2,” April 1998. OSPF is the most widely used Interior Gateway Protocol today (as opposed to BGP, which is an Exterior Gateway Protocol). Like IS-IS, OSPF is a link-state protocol.⁹⁵ It gathers link-state information from available routers and builds a topology map of the network. It was designed to support variable-length subnet masking (VLSM) or CIDR addressing models. Changes to the network topology are rapidly detected, and it converges on a new optimal routing structure within seconds. It allows specification of different metrics (“cost of transmission” in some sense) for various links to allow better modeling of the real world (where some links are fast and some slow). OSPF does not layer over UDP or TCP but uses IP datagrams with a protocol number of 89. This is very different from RIP or BGP. OSPF uses multicast, including the special addresses:

An image displays two special I P addresses that are part of O S P F multicasting.

For routing IPv4 multicast traffic, there is MOSPF (Multicast Open Short Path First), defined in RFC 1584,⁹⁶ “Multicast Extensions to OSPF,” March 1994. However, this is not widely used. Instead, most people use PIM ⁹⁷ in conjunction with OSPF or other Interior Gateway Protocols.

BGP-4: Border Gateway Protocol 4 .⁹⁸ Defined in RFC 4271,⁹⁹ “A Border Gateway Protocol 4 (BGP-4),” January 2006. This version supports routing only IPv4. There are defined multiprotocol extensions (BGP4+) that support IPv6 and other protocols, which will be described in Chapter 5.

BGP is an Exterior Gateway Protocol ¹⁰⁰ (compare with IS-IS and OSPFv2, which are Interior Gateway Protocols ¹⁰¹). It is not used within networks, but only between autonomous systems.¹⁰² Its primary function is to exchange AS network reachability information with other AS networks. This includes information on the list of autonomous systems (ASs) that reachability information traverses. This is sufficient for BGP to construct a graph of AS connectivity from which routing loops can be pruned, and, at the AS level, certain policy decisions may be enforced.

BGP-4 includes mechanisms for supporting CIDR. They can advertise a set of destinations as an IP prefix, eliminating the concept of network “class,” which was present in early BGP implementations. BGP-4 also has mechanisms that allow aggregation of routes and AS paths. Most networks that obtain service from ISPs never deploy BGP themselves. It is mostly for exchange of information between ISPs, especially if they are multihomed (obtain upstream service from more than one source). This would be referred to as Exterior Border Gateway Protocol or EBGP. Enormous networks that are too large for OSPF could deploy BGP themselves as a top level linking multiple OSPF routing domains (this would normally be referred to as Interior Border Gateway Protocol or IBGP).

BGP is a path vector protocol.¹⁰³ It does not use IGP metrics, but makes routing decisions based on path, network policies, and/or rulesets. It replaces the now defunct Exterior Gateway Protocol (EGP), which was formally specified in RFC 904,¹⁰⁴ “Exterior Gateway Protocol Formal Specification,” April 1984.

Relevant Standard for IPv4 NAT

It should be obvious from the number of RFCs that explain how NAT affects other things that NAT has a heavy impact on almost every aspect of networks. There are also a lot of “Informational” RFCs required to explain exactly how it impacts these things. Removing NAT has no downside (given sufficient public addresses) and vastly simplifies network architecture and management in addition to lowering costs. It also vastly simplifies application design and implementation. The removal of NAT and restoration of the flat address space is one of the main benefits of moving to IPv6. Unfortunately, we have an entire generation of network engineers who have assumed that NAT is “the way networks are done” and don’t realize it was created only as a temporary crutch to extend the life of the IPv4 address space until IPv6 could be completed and deployed. Before NAT, the IPv4 Internet was “flat,” and firewalls had very effective security without NAT (I call this “classic firewall architecture ”). In IPv6, we are simply returning to the original concept of “any node to any node connectivity” that characterized the pre-NAT IPv4 Internet. Protocols like SIP, IPsec, IKE, and Mobile IP will work far better without NAT in the way. DNS is also greatly simplified in the absence of NAT (no internal vs. external “views” are required).

Unfortunately, there is no possible way to remove NAT from the current Internet. There are far too many users to handle with the possible public addresses, and essentially all the routable addresses are already in use. The only possible way now to remove NAT is by migrating to IPv6.

Most routers and firewalls typically include NAT for IPv4 as part of their functionality, although it would be possible to have a NAT gateway without any filtering or routing capabilities that does only NAT.

In general, any gateway that modifies the source and/or destination addresses in a packet (possibly also the source port number) is doing NAT. There are several forms of it, the most popular being address masquerading (hide-mode NAT) and one-to-one (BINAT, or static NAT).

Most IPv4 networks today make use of private addresses as defined in RFC 1918,¹⁰⁶ “Address Allocation for Private Internets,” February 1996. Basically, three blocks of addresses (10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16) were permanently removed from the available Internet allocation pool, marked as “unroutable” on the Internet, and reserved for use as something similar to telephone extension numbers in an office (hiding behind a single company phone number, via a Private Branch Exchange). It is possible for any company to use addresses from any or all of these ranges to number the nodes inside their networks. However, these addresses cannot be routed on the Internet from anyone, since they are no longer globally unique. Hence, if the users of nodes with those addresses want to use the Internet, there must be address translation to and from “real” (globally unique) addresses at the gateway that connects them to the Internet, which is what NAT does.

One thing that confuses people is that internal telephone extensions don’t look like public telephone numbers (e.g., 100, 101, 1125 vs. 9472-4173). However, private IP addresses look just like public addresses (except for the address ranges) and in fact used to be public addresses that were repurposed.

The RFC 1918 private addresses are in the following ranges:

The three private address ranges of R F C 1918.

More recently another range was reserved for CGN (Carrier-Grade NAT ¹⁰⁷). These addresses cannot be used by end users, only ISPs deploying CGN:

A new private address range that is reserved for the use of C G N and which the I S P s solely use.

Note that the most popular form of NAT is more properly called NAPT (“Network Address Port Translation ”), which involves the translation of both IP addresses and port numbers.

NAT is defined in RFC 3022,¹⁰⁸ “Traditional IP Network Address Translation (Traditional NAT ),” January 2001. Some aspects of NAT are defined in RFC 2663,¹⁰⁹ “IP Network Address Translation (NAT) Terminology and Considerations,” August 1999.

One form of NAT traversal (STUN) is defined in RFC 5389,¹¹⁰ “Session Traversal Utilities for NAT (STUN),” October 2008. STUN is a protocol that serves as a tool for other protocols in dealing with Network Address Translation (NAT) traversal. It can be used by an endpoint to determine the IP address and port allocated to it by a NAT. It can also be used to check connectivity between two endpoints and as a keepalive protocol to maintain NAT bindings. STUN works with many existing NATs and does not require any special behavior from them.

Connection Without NAT (Inside the LAN)

Say you have two nodes (Alice and Bob) on your LAN. Alice has the address 10.50.3.12, and Bob has the address 10.50.3.75 (both private addresses). They can make connections within their LAN (to any address in the 10.0.0.0/8 network) with no problem. Say there is a web server (port 80) at 10.1.20.30.

In the following, we will specify the port number appended to the IP address, separated by a colon (e.g., 10.50.3.12:12345). When Alice makes a connection to the web server, the destination port is 80, but her source port is a randomly chosen value greater than 1024 that is not already in use (e.g., 12345 or 54321). The same source port would be used for the duration of the connection. Replies from the server would be sent using Alice’s source address and port as the destination address and port in the reply packets. See the following for example.

A table illustrates the port mapping for I P v 4 N A T. The source and destination I P address have port numbers.

Note: The preceding behavior is somewhat simplified. Such a server could accept only one connection at a time, which would have to complete before anyone else could connect. This is because a given address:port can only handle one connection at any given time. A real-world server would have a parent process listening for connections on a well-known port (e.g., 80). When some client connects to the well-known port, the parent process would create a child process (or thread), which would accept the connection (using yet another unused port number) and process it. Meanwhile, the main process would go back to listening for further connections on the well-known port. If ten users were connected at a time, there would be 11 processes running, one main process and ten child processes (one for each connection). From the viewpoint of the client (e.g., with “netstat –na”), it would appear that the remote port (the one on the server) was the original well-known port (e.g., 80).

Connection Through Hide-Mode NAT

But how do Alice and Bob connect to www.ipv6.org ? That node happens to have an IPv4 address of 194.63.248.52, and we’re still in Chapter 3 (about Ipv4), so they don’t have IPv6 yet! Let’s say there is a NAT gateway where their LAN (or ISP) connects to the Internet. It has an “outside” address (which must be a valid, routable Ipv4 address) of 12.34.56.137. If either Alice or Bob connects to www.ipv6.org (over Ipv4), the web page there will indicate to both of them that they are connecting over IPv4, from the address 12.34.56.137, not from their respective private addresses, even if the connections are made at the same instant. The web server log will show that both are connected from that one public address. How can www.ipv6.org reply with the correct web page to each of them?

With hide-mode NAT, the gateway is translating the source address in Alice’s packets from 10.50.3.12 to 12.34.56.137. It is also translating the source address in Bob’s packets from 10.50.3.75 to 12.34.56.137. The destination address is 194.63.248.52:80 for both Alice and Bob. Their browsers would each choose a random source port. Let’s say Alice’s chose 10123 and Bob’s chose 20321. The NAT gateway would not only translate the source address from both Alice and Bob; it would also shift the source ports and keep track of that shift in a table, which contains the source address, the original source port, and the shifted source port (for each connection). Let’s say Alice’s port is shifted to 30567 and Bob’s to 40765. The new source address for outgoing connections and the old destination address for incoming connections will always be the same (the outside address of the NAT gateway), so it does not need to keep those in the table. The resulting NAT table would look like the following.

A table illustrates an example of the outgoing and incoming port mapping for I P v 4 N A T. The source and destination I P address have port numbers.

Alice’s connection to www.ipv6.org appears to be coming from 12.34.56.137:30567. When www.ipv6.org replies to Alice, it is sent from 194.63.248.52:80 to 12.34.56.137:30567. The NAT gateway looks up that port in its table and sees that it was used for an outgoing connection from 10.50.3.12:10123, so it translates the destination address and port to Alice’s private address and port, thereby forwarding the packets correctly to Alice.

Bob’s connection to www.ipv6.org appears to be coming from 12.34.56.137:40765. When www.ipv6.org replies to Bob, it is sent from 194.63.248.52:80 to 12.34.56.137:40765. The NAT gateway looks that port up in its table and sees that it was used from a connection from 10.50.3.75:20321, so it translates the destination address and port to Bob’s private address and port, thereby forwarding those packets correctly to Bob.

BINAT (One-to-One NAT)

If you are doing NAT at your gateway, most routers or firewalls support another form of NAT, which is known as BINAT (bidirectional NAT ) or one-to-one NAT (sometimes also static NAT). This works much the same as regular (hide-mode) NAT, except there is no port shifting involved. This means there can only be one internal node associated with each globally routable external address. This is used only for servers that must be accessible from the outside world.

Typically, a server has both an internal (private) address (e.g., 10.0.0.13) and an external (unique, globally routable) address (e.g., 12.34.56.131). With outgoing connections, the gateway rewrites the source address of each packet to be the external address for that node (but does not shift the port). For incoming connections, the gateway rewrites the destination address to be the internal address for that node. Internally, the node will have only the internal address. However, if you connected to www.ipv6.org from such a node, the resulting web page would show a connection not from the internal address of the server, but from the unique external address associated with that node. This is similar to hide-mode NAT except that there is exactly one internal node per external address (rather than many), there is no port shifting, and the mapping can be done in both directions (incoming and outgoing connections).

There is a minor problem of the “missing ARP” that must be solved in some way for this to work (there is no physical node at the external address, so no node will respond to ARP requests concerning that address). One approach is to configure a static ARP on the gateway that can supply that response. Every operating system or router has some way to do this. Without that, connections from the outside will not work. It is also possible in most cases to assign the external address as an alias to the outside interface of the gateway (in addition to its real address). Solving the missing ARP problem is one of the most difficult things for firewall administrators to master. This problem only exists in IPv4. As no NAT is needed or done in IPv6, there is no missing ARP (actually in IPv6, it would be a missing ND response).

BINAT at least allows incoming connections but uses up one globally routable IPv4 address for each server node. Most SOHO gateways do not support BINAT. Many do have a simpler mechanism called port redirection, which allows incoming connections to the hide-mode external address. At most one internal server can be configured as the target for any given port. So you could configure an internal mail server and redirect ports 25 (SMTP), 110 (POP3), and 143 (IMAP) to it. However, if you have two internal web servers both configured for port 80, you could not redirect port 80 on the gateway to both servers.

Ramifications of Using NAT

When Network Address Translation happens, the NAT gateway is actually rewriting new values into the address and port number fields in the IP and TCP (or UDP) packet headers of all packets flowing through the NAT gateway, according to the rule just specified. For outgoing packets, it is rewriting the source address and source port. For incoming packets, it is rewriting the destination address and destination port. Obviously, this would invalidate the IP and TCP header checksums (the IP header contains source and destination addresses; the TCP header contains the source and destination port numbers). Therefore, the NAT gateway also has to recalculate both IP and TCP header checksums and rewrite those as well.

Packet fragmentation is a real complication for TCP and UDP via NAT. A NAT gateway must reassemble an entire packet, in order to be able to recalculate the TCP checksum (which covers all bytes in the payload, plus the pseudo header, which contains the source and destination addresses). It typically must then re-fragment the packet for further transmission.

What about the IPsec Authentication header (AH) ? (Note: IPsec will be discussed in detail in Chapter 6.) The IPsec AH algorithm works like a checksum, but there is a key that only the sender has, required to generate the cryptographic checksum. All this address and port rewriting invalidates the existing AH cryptographic checksum, and the NAT gateway does not have the necessary key to regenerate a correct new AH for the modified packet headers. Because of this IPsec does not work through a NAT gateway. Actually, AH is performing its function very effectively; it is detecting tampering with the contents of the packet header! It just happens that this tampering is done by a NAT gateway, not a hacker. It’s kind of like getting hit by “friendly fire” in a war zone (getting shot by your own side). If any node other than the original sender could generate a new valid AH checksum, then AH would not be very useful! IPsec and NAT are mutually exclusive (although IPsec VPNs can be made to work in conjunction with NAT traversal).

Another ramification involves FTP (File Transfer Protocol ). FTP is a very old protocol (RFC 765 ¹¹¹ is from 1980, back in the days of the First Internet). In active mode, FTP uses separate connections for control traffic (commands) and for data traffic. The initiating host identifies the corresponding data connection with its Network Layer and Transport Layer addresses. Unfortunately, NAT invalidates this. Fortunately, here, it is possible to create a reverse FTP proxy (included on most firewalls) that solves this problem. Without such a proxy though, FTP will not work if NAT is in place, even for outgoing connections. My company early on ported a popular one for IPv4 to IPv6. That allowed FTP connections to dual-stack networks such as freebsd.org to work from our own dual-stack network.

“Peer-to-peer” (like Kazaa, not “real” peer-to-peer) applications have the same kinds of problems with NAT. You must somehow provide a way for your peers to connect to you for these applications to work. All participants really need a real, globally routable IP address. This is not easy to arrange on the Second Internet. All such “fake” peer-to-peer applications must use NAT traversal.

SIP (Session Initiation Protocol ¹¹²) is used with many things, including VoIP and video conferencing. It also has major problems with NAT. SIP may use multiple ports to set up a connection and transmit the analog stream over RTP (Real-Time Transport Protocol ). IP addresses and port numbers are encoded in the payload and must be known prior to the traversal of NAT gateways (this was bad protocol design, but now we are stuck with it). Again, a SIP proxy on the gateway can help resolve this problem. Another solution is to use NAT traversal, such as STUN. Unfortunately, in these days of widespread NAT, both the caller and the callee are typically behind NAT, so VoIP must overcome problems with NAT both going out from the caller and coming in to the callee. If this sounds like an ugly mess, it is.

Another problem with NAT is the limit of 65,536 ports on the NAT gateway. When NAT was first deployed, most network applications used only one or two ports. Some recent applications (Apple’s iTunes and Google Maps) use 200–300 ports at a time for better performance. If each node is using 300 ports, then there can be at most 200 nodes behind a given external IPv4 address. If the NAT gateway runs out of ports, there can be very mysterious failures in network applications. For example, in Google Maps, some areas of the map never get drawn. There is no way for end users (or typically even the network administrator) to determine that this has happened other than by seeing mysterious failures in some applications. This means that a larger number of NAT gateways (and valid external IPv4 addresses) are required today than in the past, for a given number of users behind NAT. Just as we are running out of public IPv4 addresses!

Some legacy applications (like web surfing and email) work okay through one layer of NAT. Even with chat, today there must be an intermediary system that two or more chatters connect to via outgoing connections from their nodes (e.g., AOL Instant Messenger). In a flat address space (especially with working multicast), much better connectivity models are possible that may require little or no central facilities.

As the IPv4 addresses run out, it will become more common to have multiple layers of NAT (CGN). This can happen today, if you deploy a Wi-Fi access point with NAT behind a DSL modem that also has NAT. If you think a single layer of NAT causes problems, you should try dealing with multiple layers of it!

With the wide-scale deployment of NAT, we have lost the original end-to-end model of the early Second Internet, which was a core feature. We’ve also broken one of the fundamental rules of protocol design: never tamper with source or destination addresses or ports in an IP packet.

Today users are either content producers who can publish information or videos (e.g., cnn.com, youtube.com) or content consumers who can view the content published by the producers. It is much more complicated and expensive to be a producer in the current Second Internet (with NAT in the way) than to be a consumer. There are relatively few producers and millions of consumers. This was not that much of a problem when most people were running mainly web browsers and email clients on their nodes. As newer applications emerge (VoIP, IPTV, multiplayer games, peer-to-peer), this new “digital divide” between producers and consumers is becoming more of a problem. Today, many people would like to be prosumers (both producers and consumers of content). With IPv6 that is simple.

Another problem is that since the first implementation of networking on smartphones (WAP), there were not enough public IPv4 addresses for phones, so historically there have never been public IP addresses on phones. Phones could only be used to make outgoing connections – you could not deploy a server on your phone, and Alice’s phone could not connect directly to Bob’s phone. With IPv6 for the first time, phones have public addresses and hence can run servers or do end-to-end connections.

All these problems go away with a flat address space (no NAT). Unfortunately, there is no way to restore the flat address space of the early (pre-NAT) Second Internet. The Second Internet is now permanently broken (there are not enough addresses to allow even the existing users to have access without NAT, even if we use all the remaining unallocated addresses today). The only real solution is to switch to IPv6 (at least for protocols such as VoIP, P2P, multiplayer games, IPTV, and IPsec VPNs).

Standards Relevant to TCP

TCP Packet Header

An illustration of the T C P packet header containing six layers. The source and destination port details are present in the first layer.

Source Port (16 bits): Specifies the port that the data was written to on the sending node.

Destination Port (16 bits): Specifies the port that the data will be read from on the receiving node.

Data Offset (4 bits): Specifies the size of the TCP header in 32-bit words. The minimum value is 5 words (20 bytes), and the maximum value is 15 words (60 bytes), allowing for up to 40 bytes of options.

Reserved (4 bits): Not currently used and must be zeros.

Window Size (16 bits): Size of the receive window, which is the number of bytes that the receiver is willing to receive.

Checksum (16 bits): Used for error checking of the TCP header and data.

Urgent Pointer (16 bits): If the URG flag is set, this is the offset from the sequence number indicating the last urgent data byte.

Options (from zero to ten 32-bit words): Optional, not commonly used – see RFC for details.

A flow chart depicts the T C P state transition diagram. It has numerous blocks connected by arrows.

Notes

TCP uses sequence numbers to detect lost packets and/or reorder packets that arrive out of order. The cumulative acknowledgment scheme informs the sender that all packets up to the acknowledged sequence number have been received. Selective acknowledgment (RFC 2018 ¹¹⁷) allows for optimization of this feature. Lost data is automatically retransmitted by the sender. End-to-end flow control provides for a mismatch in performance between sender and receiver. A sliding window algorithm allows multiple packets to be in progress, which increases efficiency. Recently, congestion control has been added into TCP to avoid network congestion.

TCP is very complicated. The good news is that when used over IPv6, TCP works essentially the same way. The very minor changes will be covered later.

Standards Relevant to UDP

UDP Packet Header

A U D P header packet has three layers. The first layer has the source and destination port details. The next layer has length and checksum, and the final layer contains the data.

The Source Port field (16 bits) specifies which port number the data is being written to on the sending computer. This field is optional (if not used, fill with zeros).

The Destination Port field (16 bits) specifies which port number the data is being read from on the receiving computer.

The Length field (16 bits) is the number of bytes in the datagram, including the UDP header and the data. Therefore, the minimum value is 8 (the length of the UDP header). The maximum value in theory is 65,536 bytes, but this value is limited by the maximum packet size, typically 1508.

The Checksum field is optional (if not used, fill with zeros).

The Data field begins immediately after the Checksum field. It is not really part of the header, but it is factored into the checksum.

The DHCPv4

Let’s say our network uses 192.168.0.0/16. That means the subnet mask is 255.255.0.0. Our DNS servers are at 192.168.0.11 and 192.168.0.12. The DHCPv4 server is also running on 192.168.0.11. The default gateway is 192.168.0.1. We have created a pool of addresses from 192.168.1.0 to 192.168.1.255.

In this case, the node is requesting its last known IP address. Assuming it is still connected to the same network and the address is not already leased to someone else, the server may grant the request. Otherwise, the client will have to negotiate for a new address.

At this point, the client actually configures those values for its network interface and can begin using the network.

Useful Commands Related to DHCPv4

This is an example of the output from “ipconfig /all”.

A windows screen presents the output of an all ipconfig command. It has 11 lines.

IPv4 Network Configuration

Furthermore, assume the DHCPv4 server is correctly configured with this information and is managing the address range 192.168.1.0–192.168.1.255 (and that some leases have already been granted).