Although there are dozens of standards that describe details of internet email, most build upon other standards. Underlying all email formatting standards is one called “RFC 822,” which is the 822nd Request for Comments published by the ARPANET/internet community. Internet protocol standards begin as “Internet Drafts” and become “RFCs” when they are ready for serious consideration by the internet technical community at large. The RFC becomes a standard after it is accepted by the internet community.

RFC 822 was published in 1982, which illustrates how old internet email systems are. Subsequent changes to email format have been extensions to RFC 822. A major extension is the Multipurpose Internet Message Extensions (MIME), which describe how to include email attachments in different formats.

FIGURE 14.1 contains an email message sent from “rsmith6” to “kev1111.” An email message consists of two parts: the headers and the body. The headers start at the beginning, of course, and continue until the first blank line appears. The message body follows: a single line of text in this case.

As with individual packets, the real data is at the end with technically elaborate headers attached to the beginning. When we first compose a message, we identify the recipient and subject, and we type in the body. Our email client constructs four standard headers:

1. From:

2. To: (also Cc: and Bcc:)

3. Date:

4. Subject:

Modern email clients often insert additional headers for technical reasons. Many include headers describing the format and character set used, so the recipient’s email client can distinguish between “richly formatted” email and plain text in different languages. Many also include a “Message ID” so that every email message contains a unique identifier.

When we write an email, the client software constructs those initial headers. Our client software inserts the date and formats the email addresses of the recipients. It also adds the other formatting and identification headers.

MIME formatting allows us to include non-ASCII text and files in email messages. Email messages originally were limited to 7-bit ASCII characters. MIME formatting transforms 8-bit data files into a 7-bit format that travels safely through email software. MIME supports graphically rich emails that include boldface text, italics, bulleted lists, international character sets, and graphics. FIGURE 14.2 shows a MIME email from Bob’s boss.

We can’t embed such text directly into an ASCII email. Moreover, we can’t transmit 8-bit data files through plain ASCII email. If an email software component only supports 7-bit ASCII characters, it might change the 8th bit. Most application files, like spreadsheets or word-processing documents, contain 8-bit data. MIME encoding allows us to send 8-bit application files and graphics by converting the data to a 7-bit ASCII format.

The email text shown in Figure 14.2 uses a large typeface, a non-ASCII accent character, boldface, and a bulleted list. FIGURE 14.3 shows the raw 7-bit ASCII text for this message. We have omitted Received headers and others that aren’t relevant here.

The Content-Type header signals the presence of MIME parts. The header first describes the type of MIME formatting being used. In this case, the message is a multipart message, indicating that it has two or more separate sections, each with its own MIME formatting.

The header also defines a special string, the boundary marker, to mark the boundary between different sections of the message. In this case, the text string starting with “Apple-Mail” serves as the boundary marker. It divides the message body into two separate sections: the plain text section and the formatted text section.

The message’s plain text section begins immediately after the first boundary marker. The section begins with headers that describe the encoding and character set used in that section. Following the headers, the section contains a “plain text” version of the message. The email client constructs this as a convenience in case the recipient’s email client can’t display the formatted message in the second section. However, even this “plain text” is more than ASCII; it uses the ISO 8859-1 character set, which is essentially ASCII extended with common accented characters.

The second boundary marker indicates the start of the second section. Figure 14.3 shows only the beginning lines of that section. The format is “text/html,” and HTML (Hypertext Markup Language) refers to the formatting used on web pages. We describe this further in Section 15.1.

In some systems, the email client uses a web browser—or parts of it—to display html-style email. This allows such emails to incorporate any formatting that a web page might use. These include variations in type size, weight, and style, as well as special formats like bulleted lists, as we see in the previous figure. However, it may also retrieve and display images from websites and, in some cases, execute scripts or other programs. We discuss those issues further in Section 14.3.

Many email users retrieve their email by visiting their email service using a web browser. Others rely on email client software residing on their computer. Client software uses a mailbox protocol to retrieve the email. The protocol logs in to the server containing the user’s mailbox. It then examines the mailbox and tells the user which messages are available. Some protocols automatically copy all messages to the user’s client. Others leave the messages on the server until the user deletes them.

There are several well-known mailbox protocols, including the Post Office Protocol (POP3) and the Internet Message Access Protocol (IMAP). Other users rely on Microsoft’s proprietary email protocol, the “Message API” or MAPI, to access its Exchange Server product. POP3 and IMAP are supported by most email client software, including Mozilla’s Thunderbird, Qualcomm’s Eudora, Apple Mail on MacOS, and Microsoft’s email products.

POP3: An Example

As in all internet protocols, we find the official description for POP3 in an RFC; this one is RFC 1939, “Post Office Protocol—Version 3.” The POP3 protocol includes a strategy to incorporate new features, and these are documented in a few later RFCs. However, RFC 1939 describes the fundamentals of the POP3 operation.

POP3 is a text-oriented protocol that uses a single connection. The client opens the connection, and the server replies with “+OK.” The client sends commands and the server responds to each one. Some commands yield a single line response, like those for authenticating. Others yield multiple lines, like those that list available messages or retrieve messages. The server uses a special text marker to indicate the end of the message.

Here’s a transcript of a simple POP3 exchange in which Bob’s email client finds and retrieves a single email message. We have omitted the email text.

The client’s first real action is to provide the user ID and password in Steps 2 and 4. This immediately indicates a vulnerability: If attackers sniff this connection, they can retrieve Bob’s user ID and password. This is why most ISPs today allow users to protect POP3 connections with SSL.

The “STAT” command in Step 6 asks for the amount of mail in the mailbox. The “+OK” response in Step 7 includes the number of messages (1) and the size of the mailbox in bytes (1227). The next two commands, “UIDL” and “LIST,” retrieve information about the message to construct the list of available messages.

The “RETR” message in Step 14 includes a message number as an argument and retrieves the single message (Steps 15 and 16). Once Bob’s client has retrieved messages, it sends a QUIT command; then the client and server close the connection.

When we create an email message for delivery, we don’t place it directly in the recipient’s mailbox. Instead, we contact an email server, transmit the message to the server, and let the server deliver it. The email delivery protocol is called the Simple Mail Transfer Protocol (SMTP), and it has been the workhorse of internet email since 1982. These SMTP servers, also called message transfer agents (MTAs), form the backbone of internet email delivery.

The SMTP protocol is extremely simple. We open a connection to the SMTP server and send a series of simple text commands to identify the recipients of the email, then we transmit the email message and close the connection. The server takes responsibility for delivering the message to its recipients.

In simple cases, we connect directly to an MTA, and that MTA places the email in the recipient’s mailbox. This often happens when sending email between users in the same site. When email must travel across the internet to a different site, it often passes through a series of MTAs. Section 14.2.2 provides a detailed example of this.

When two MTAs exchange email, they operate in a peer-to-peer relationship. Either MTA can initiate the connection, and either can respond. SMTP is a client/server protocol; the MTAs use it as peers because either host can play the role of client or server.

TABLE 14.1 lists TCP port numbers used by internet email protocols. Email client protocols like POP3 and IMAP may choose between a plaintext or SSL-protected connection by selecting the appropriate port number. When using SMTP, some sites associate port 465 with SSL protection, though this port is also used for a different protocol.

SMTP does not in fact require a separate port number to support SSL. Recent SMTP servers incorporate a “STARTTLS” command that transforms a plaintext SMTP connection into an SSL-protected one.

The only reliable way to authenticate email is to apply a cryptographic technique to the message itself. The most common approach is for the author to digitally sign the email. FIGURE 14.8 shows an excerpt from an email that was digitally signed using PGP software. We introduced PGP and other email security protocols in Section 13.3.

The signed portion of the email is carefully marked with the “BEGIN PGP SIGNED MESSAGE” and ends with the signature itself in a special ASCII encoding. The process for signing email is the same one described earlier in Section 8.5. Note that any reformatting of the email text will invalidate the signature. Email servers must be sure to preserve the exact contents of email they transfer, or signature checks will fail even on legitimate messages.

PGP is a relatively old program; the simple format shown in Figure 14.8 is peculiar to PGP. There is an internet standard for secure email based on a MIME extension, called Secure MIME, or S/MIME, and there is also a MIME extension called “PGP MIME.” Most modern email clients support these extensions either directly or through plug-in software. It can, however, be challenging to ensure that all—or most—email recipients have the right software to verify a digital signature.

Secure email poses another challenge because all recipients must have copies of the author’s public-key certificate. Many authors include the certificate in the email along with the signature, or they provide a weblink to a site that contains the certificate. Then the recipients can validate the certificate and the signature.

Unfortunately, not all secure email users understand this shortcoming. The email in Figure 14.8 did not include a certificate, nor did it provide any way of locating the certificate. Thus, most recipients had no way to validate the email.