Most of the networking errors we have discussed so far involve scenarios where we clearly know when packets are not reaching their destination. However, what if packets are dropped without the source ever being informed that they have not reached their destination? Errors such as these can only be detected by monitoring network traffic, and we refer to such situations in which packets seem to just disappear as black holes.

When do such scenarios occur? One case is where a host tries to connect to another host that is down, or tries to connect to an IP address that has never been assigned to a host. One would think that using TCP would provide the ability to be informed that the packets never reached their destination, but often the packets are simply dropped. It also happens in cases where you are using a connectionless protocol such as UDP; thus, there is no way of communicating back to the original sender that the IP address is dead.

Another possibility is that you have a Maximum Transmission Unit (MTU) black hole. This happens when MTU exceeds the maximum MTU size allowed on a network (this can happen, for example, if VLAN tags are added to a packet), and the Don't Fragment (DP) flag is set in the IP header. In this case, any device whose MTU is smaller than the packet's size will drop the packet. The solution, in this case, is to run Path MTU Discovery (PMTUD) on all network devices. PTMUD solves the issue of packets being too big by sending a fragmentation needed ICMP message back to the device sending the large packets. This causes the offending device to reduce its MTU size. The problem with this fix is that some network devices block ICMP messages for security reasons. If this is the case, network devices will complete the TCP three-way handshake, but when data is transferred, the connection will hang because of the MTU size mismatch, and you will still have a black hole connection.

If you have network devices that block ICMP traffic, one possible solution is to use the RFC 4821 version of PTMUD. This version circumvents the problem using TCP or another protocol to probe the device with progressively larger packets. Another solution, which may not be practical in your case, but is still worth mentioning, is to change the maximum segment size (MSS) of all TCP connections lower than the Ethernet default of 1,500 bytes.