The common mail protocols for mail client to server and server to server communications are Post Office Protocol version 3 (POP3), Simple Mail Transfer Protocol (SMTP) and Internet Message Access Protocol version 4 (IMAP4).
Another common method for accessing e-mails is web access to mail, in which you have common mail servers such as Gmail, Yahoo!, and Hotmail. Some examples include Outlook Web Access (OWA) and RPC over HTTPS for the Outlook web client from Microsoft and others.
In this recipe, we will talk about the most common client-server and server-server protocols: POP3 and SMTP. We will also look at some typical problems by using the other methods.
When users are complaining about mail problems, first check if there are any obvious problems such as wrong username, bad password, and authentication protocols that are not configured. If none, connect Wireshark with port mirror to the complaining client; and if there are many of them, configure port mirror to the common server or the communications line connecting to it (when there is a remote server).
POP3 will usually be used for client to server communications, while SMTP will usually be used for server to server communications.
POP3 is usually used for mail client to mail server communications. When a client cannot access the mail server, perform the following checks:
- First, check if the correct username and password have been configured.
- Then, check if the authentication has passed correctly. In the following screenshot, you can see a session opened with a username that starts with
doronn@(all IDs were deleted) and a password that starts withu6F. - To see the TCP stream shown in the following screenshot, right-click on one of the packets in the stream and choose Follow TCP Stream from the dropdown menu:
- Any error messages in the authentication stage will prevent the communications from being established. You can see an example of this in the following screenshot where user authentication failed. In this case, we see that when the client gets the Logon failure, it closes the TCP connection.
- During the mail transfer, be aware that mail clients can easily fill a narrow-band communications line. You can check this by simply configuring the IO graphs with a filter on POP.
- Always check for common TCP indications: retransmissions, zero-window, window-full, and others. They can indicate a busy communication line, slow server, and other problems coming from the communications lines or end nodes and servers. These problems will mostly cause slow connectivity.
SMTP is commonly used for the following purposes:
- Server to server communications, in which SMTP is the mail protocol that runs between the servers
- In some clients, POP3 or IMAP4 are configured for incoming messages (messages from the server to the client); while SMTP is configured for outgoing messages (messages from the client to the server)
When you suspect slow server-to-server communications, follow these steps to resolve the problems.
- Check if the servers are located on the same site:
- If they are located on the same site, you probably have slow servers or another application problem. In most of the cases, the LAN will not cause any problems—especially when both servers are in the same data centre.
- If they are not located on the same site (when the servers are located in a remote site through WAN connections), check the load on the WAN connections. When sending large mails, they can easily block these lines—especially when they are narrow band (several Mbps).
- First, look for TCP problems; and check if you see them only on SMTP or on all other applications. For example, in the following screenshot, you can see many TCP retransmissions:
- Check if they are because of a slow SMTP server. Is it a mail problem? When you look at the following screenshot, you see that I've used the TCP Conversation statistics. After checking the Limit to display filter checkbox and clicking on Packets at the top of the window (to get the list from the higher amount of packets), we can see that only 793 packets are SMTP from the retransmitted packets. There are 9014 packets retransmitted between
172.16.30.247and172.16.30.2on port445(Microsoft DS), 2319 packets are retransmitted between172.16.30.180and192.5.11.198on port80(HTTP), and so on.
- In this case, SMTP is influenced only by bad communications. It is not an SMTP problem.
- Check for SMTP errors. In the following screenshot, you see an error
code 451, which is also called thelocal error in processingserver error. Also, a list of errors is listed.
You can also find a list of SMTP status codes in RFC 1893 (http://www.ietf.org/rfc/rfc1893.txt).
- When you want to know which errors have been sent by the two sides, configure a filter as shown in the following screenshot:
- Here you can see various events:
- Code 421: This indicates that the mail service is probably unavailable (1).
- Code 452: This indicates that the server cannot respond, and tells you to try again later. This happens due to load on the server or a server problem (2).
- Code 451: (code 250 is shown in the screenshot, see the following note) This indicates the user over quota (3).
- Code 452: This indicates that the mailbox size limit has been exceeded (4).
- Code 450: (code 250 is shown in the screenshot, see the following note) This indicates that the host was not found (5).
Tip
In SMTP (like in many other protocols), you can get several error codes in the same message. What you see in the packet list in Wireshark can be the first one, or a partial list of it. To see the full list of errors in the SMTP message, go to the packet details and open the specific packet, as in the following screenshot.
When you see too many codes, it indicates unavailability of the server. check with the server administrator.
Some other common methods that I mentioned earlier are web mail and RPC over HTTP:
- In web mail, we connect to the server with HTTPS; therefore this is exactly like working with HTTPS, as described in Chapter 10, HTTP and DNS. After logging in to the server, if any problems occur, they will be HTTPS problems.
- RPC over HTTPS will be same. Since RPC is a protocol which usually loads the network, it is considered to be sensitive to high delays and jitter. Microsoft came up with a solution to work with their Outlook client over HTTPS and not with the standard RPC. Again, since communication runs over HTTPS, problems will be HTTPS problems.
Mail clients will mostly use POP3 for communications with the server. In some cases, they will use SMTP as well. IMAP4 is used when server manipulation is required, for example, when you need to see messages that exist on a remote server without downloading them to the client. Server to server communications are usually implemented by SMTP.
In general, SMTP status codes are divided into three categories, which are structured in a way that helps you understand what exactly went wrong. The method and details of SMTP status codes is discussed in the following section.
POP3 is an application layer protocol used by mail clients to retrieve e-mail messages from the server. A typical POP3 session will look like the following screenshot:
- The client opens a TCP connection to the server.
- The server sends an OK message to the client (OK Messaging Multiplexor).
- The user sends the username and password.
- The protocol operations begin. NOOP (no operation) is a message sent to keep the connection open, STAT (status) is sent from the client to the server to query the message status. The server answers with the number of messages and their total size (in packet
1042, OK 0 0 means no messages and it has total size zero). - When there are no mail messages on the server, the client sends a QUIT message (
1048), the server confirms it (packet1136) and the TCP connection is closed (packets1137,1138, and1227).
In the case of encrypted connection, it will look nearly the same (see the following screenshot). After the connection establishment (1), there are several POP messages (2), TLS connection establishment (3), and then the encrypted application data.
The structure of SMTP status codes is as follows:
class . subject . detail
For example, when you see status code 450, it means the following:
- Class 4 indicates that it is a temporary problem
- Subject 5 indicates that it is a mail delivery status
- Detail 0 indicates an undefined error (RFC 3463, Paragraph 3.6)
The following table lists the various classes:
|
Status code |
Meaning |
Reason |
|---|---|---|
|
|
Success |
Operation succeeded |
|
|
Persistent transient failure |
A temporary condition has prevented the server from sending the message. It can be due to server load or network bottleneck. Usually, sending the message again will succeed. |
|
|
Permanent failure |
A permanent problem prevented the server from sending the message. Usually server or compatibility errors. |
The following table lists the various subjects:
|
Status code |
What is it |
What can be the reason |
|---|---|---|
|
|
Other or undefined status |
- |
|
|
Addressing status |
- |
|
|
Mailbox status |
- |
|
|
Mail system status |
- |
|
|
Network and routing status |
- |
|
|
Mail delivery protocol status |
- |
|
|
Message content or media status |
- |
|
|
Security or policy status |
- |
The list of status details are too long to be listed here. A full list can be found in the standard pages at http://tools.ietf.org/html/rfc3463.
Some common status codes are listed in the following table:
E-mails are sometimes referred to as one of the "silent killers" of networks, especially in small enterprises that use asymmetric lines to the Internet. When sending text messages, they will not consume anything from the network; but when you send a large file of several megabytes or even tens of megabytes over a narrow-band uplink to the ISP, the rest of the users in your office will suffer from network slowdown for many seconds, even minutes. I've seen this problem in many small offices.
Another issue with mail clients is that in some cases (configurable), mail clients are configured to download all new data from the server when they start to work. If you have a customer that complains of a network slowdown at the time when all employees start their day in the office, it might be due to the tens or hundreds of clients who opened their mail clients simultaneously and the mail server is located over a WAN.