Chapter 3. ESA Email Pipeline

In this chapter, you will learn the following:

• The full email-processing pipeline for all accepted messages

• How the system performs connection handling and rate limiting

• Operations performed in the ESA work queue

• How to configure options for acceptance and delivery of messages

Simple Mail Transfer Protocol (SMTP) connections and email messages handled by the Email Security Appliance (ESA) move from connection acceptance to SMTP conversation, to message acceptance, and then to filtering and delivery in a step-by-step process known as the email security pipeline. The pipeline is important because the fixed order of certain steps dictates configuration and architecture, and the flexibility of other steps means we have a great deal of leeway in designing and deploying an email environment based on ESA.

ESA Pipeline

The ESA pipeline has some stages that are fixed in order, apply to all messages, and other components that are either optional based on the message or completely optional for all messages. For example, spam filtering may not be performed on mail from certain senders, and virus scanning need not be performed at all on the pipeline. Direction of messages also dictates the order and stages that messages travel through. Messages that arrive at the ESA can pass through either an incoming pipeline or an outgoing pipeline, although we’ll see that this distinction is not always clear. Incoming and outgoing are not really separate pipelines or queues; in fact, they don’t refer to any distinct features, as all messages arriving at an ESA are treated more or less the same. They really only distinguish between which filtering steps occur and which don’t, and how the filtering affects the messages.

We can broadly group the pipeline stages into three: First, the connection and SMTP conversation stages, that control how and whether connections are accepted, and if accepted, whether the messages being transmitted and their sender and recipient addresses should be accepted. Second, after the message data has been accepted, filtering of the message occurs in steps, with each table and engine examining senders, recipients, headers, and body, as appropriate. Finally, if a message continues past all filtering, intact or modified, it proceeds to delivery, where a number of features can affect the final destination and rate at which messages are delivered.

Some messages do not proceed through the entire pipeline. There are a number of places where connections and messages from remote senders can be rejected or messages dropped. Messages generated on the ESA, such as notifications, reporting, alerts, and administrative copies of messages, do not pass through the acceptance and filtering stages and move directly to delivery.

We’ll closely examine each stage and the steps within each stage. First, we need to define and discuss one of the most important concepts on an ESA, that of a listener. The listener is the primary controller for the first stage of the email pipeline, as shown in Figure 3-1.

Figure 3-1. First Portion of the ESA Email Pipeline

Listeners

The ESA listens for connections on the interface and port specified in one or more SMTP listeners. You can think of a listener as an SMTP server daemon bound to a specific interface and port. A listener controls all of the aspects of the first third of the pipeline, the connection and SMTP conversation. These settings are global for a specific listener, and major features like Host Access Table (HAT) and Recipient Access Table (RAT) are tied to each specific listener. It’s certainly valid to have more than one listener on an ESA. In fact, there are good reasons to have multiple listeners on a single appliance, but for this chapter, we work with just a single listener. In addition to deciding whether to accept or reject connections, and at what rates, the listener also provides:

• Sender and recipient checks: The envelope sender and all envelope recipients can be validated during SMTP conversation, and this allows the ESA to reject undesirable senders and invalid recipients.

• The envelope sender’s domain can be tested against DNS, and senders from nonexistent domains can be rejected.

• The envelope sender address can be checked against the exception table to override a reject or accept decision.

• The envelope recipients are checked against the RAT and can be tested against an LDAP query for validation. Usually, this is just a test to see that the email address exists. The ESA can reject, drop, or bounce messages that fail an LDAP acceptance test.

• Partial recipient addresses (those that do not contain a fully qualified domain name after the @ sign) can have a default domain added. This is typically only used for internal email traffic. For example, some application servers might send messages to webmaster@host1, but the proper delivery address would be [email protected]. Default domain adds the example.com domain to addresses that don’t have it.

• Sender and recipient transformations: The envelope sender, some or all envelope recipients, and even message headers that contain email addresses can be changed during the SMTP conversation.

• Domain map: A one-to-one transformation of the domain portion of recipient email addresses. For example, if I have a domain map entry of @ironport.com with a target of @cisco.com, any RCPT TO address with form [email protected] is transformed into [email protected]. No change is made to the user portion of the email address.

• Masquerading: The transformation of envelope and visible email addresses, done either with static entries in a table or via LDAP query. This is most often used on outgoing mail to change internal email addresses into external email addresses.

• LDAP queries: All the LDAP queries that will be used on inbound messages must be assigned on the listener. We’ll look at LDAP queries and using them in policies later in this chapter and in Chapter 8, “Security Filtering.”

• Address and message content restrictions: Each listener has settings to control whether the system accepts 8-bit data, whether certain characters are allowed in recipient addresses, and whether to fix issues with illegal CR and LF characters.

• Miscellaneous other settings: There are a number of other message actions that are configured on the listener settings. Most of these can be overridden later in the pipeline.

• A disclaimer, which is text added to the top or bottom of a message, can be chosen for the messages arriving at the listener. These are also referred to as headers or footers, respectively. These can also be assigned using filters.

• A bounce profile can be chosen for messages. A bounce profile really should be called a retry and bounce profile, because it controls how often, and for how long, a delayed message should be retried. If a message exceeds that schedule, the message is bounced, and that behavior is controlled here. These can also be assigned using filters, or in the destination controls table during the delivery stage of the pipeline.

• Connection timeouts. To deal with stuck senders, we can limit the maximum time allowed for an inbound SMTP conversation. There are separate limits for SMTP connections that have actually attempted to send email, versus unproductive connections that haven’t.

Host Access Table (HAT) and Reputation Filters

The pipeline for any message begins with the standard TCP connection handshake. Based on the source IP in the TCP packets, the ESA takes a number of actions even before presenting an SMTP welcome (or rejection) banner. These actions include the following:

• A DNS PTR (reverse) lookup is performed to find the fully qualified hostname of the connecting IP. If no PTR is available, or the lookup times out, that fact is recorded. Note that all allocated IP addresses in use on the Internet need to have PTRs defined; we’ll examine this requirement later.

• A DNS A (forward) lookup is performed to determine the IP addresses mapped to the hostname returned in the first step, if any. If at least one of the IPs returned in this lookup matches the connecting IP, the DNS derived host and domain information is considered verified.

• A SenderBase Reputation Score (SBRS) lookup is performed, using DNS as the transport mechanism. The value returned is a real number in the range of –10.0 to +10.0, or the special value of None. Although this is a real number, in practice, the value returned has only one digit after the decimal point. SBRS reflects the likelihood that a given IP is sending spam, viruses, or other unwanted email. The lower the score, the more likely it is to be junk. Scores below –3.0 are almost always junk.

• Other DNS lookups are performed. If you’ve configured such features as DNS Blacklists (DNSBL) to third-party lookup services, these will be performed.

As you can see, an ESA depends heavily on DNS for its lookups. Once all these lookups are complete, the ESA now has the source IP, hostname (if any), SBRS, and DNSBL results. It also knows if a PTR exists, and if it does, whether there’s a forward and reverse DNS match. These attributes are used to classify the connection in a sender group in the HAT. Any single criteria match suffices for a sender group match. A default HAT includes some predefined sender groups, and you can define your own or modify the default groups. The default HAT includes a WHITELIST group, which starts empty, a BLACKLIST group for SBRS ranges of –10.0 to –3.0, a SUSPECTLIST group for SBRS from –3.0 to –1.0, UNKNOWNLIST for –1.0 to +10.0, and a default group ALL. Matches are performed from top to bottom and any connecting IP that doesn’t match a specific group fall into the ALL default group.

Essentially, the fate of a connection is determined entirely based on source IP, or information about the IP or host derived from these lookups. It’s not based on any information from the message—in fact, we haven’t even started the SMTP conversation yet. Once a HAT group is determined, it is fixed, and no later stage of the pipeline will alter that group. Because the first match wins, you should put the groups with more specific criteria first. For example, because WHITELIST lists specific IP addresses and hostnames, it should come before groups that use DNS conditions or SBRS. Because the WHITELIST is first, you can place a particular sender there to avoid false positives, overriding the classification based on SBRS.

The result of a connecting IP being classified in a group is the application of a Mail Flow Policy that controls how, and whether, connections are accepted. Every sender group has a corresponding policy, like the BLOCKED policy for the BLACKLIST sender group. The Mail Flow Policy encapsulates all the connection controls and rate limits at SMTP conversation time and controls many of the security filters that will apply to messages accepted under the policy. Policies are defined separately from groups to keep classification of senders distinct from actions taken. You can even use the same mail flow policy with more than one group, allowing for reporting and filtering based on group membership while still applying consistent policy. Mail flow policies are flexible, and you can create your own in addition to the built-in policies. All mail flow policies boil down to just four fundamental actions:

• Accept: The connection will be accepted and the SMTP conversation will start. The sender is limited to recipients in the domains specified in the RAT and may be subject to further checks on the recipient addresses. Messages received over an accepted connection are treated as incoming for policy purposes. This is the default action for most policies.

• Relay: The connection will be accepted, and the SMTP conversation will start. The sender is not limited to recipients in the RAT—in fact, the RAT is not checked at all. The sending IP is allowed to issue RCPT TO commands to any email address. The ability to send to any email address is a high level of privilege to grant any sender, so use it carefully by limiting it to very specific sending IPs (typically only your internal mail servers). Limiting access to relay policies to absolutely trustworthy hosts is a cornerstone of HAT security.

• TCP Refuse: The TCP connection is closed. The ESA issues a FIN to the sender to indicate that the connection is over before it even starts. Because this is usually confusing to the sender and is not RFC compliant, I strongly recommend against using this setting, as satisfying as it may be to refuse senders. Most senders will treat the ESA as down and reattempt connections for hours or days.

• Reject: The TCP connection is accepted, but the ESA greets the sender with an SMTP 554 banner to indicate that they are not welcome here. That’s immediately followed by a TCP FIN to close the connection. This banner code and text is customizable, so you can give the sender some information about why he has been rejected. This is the default action for the BLOCKED policy.

Any connection that continues into the SMTP conversation proper must have been greeted with either an Accept or Relay behavior. This means that all messages making it into the pipeline must have arrived on one of those two behaviors. For policy purposes, all messages that have been accepted are classified as incoming messages, and relayed are considered outgoing messages.

For accepted and relayed connections, the SMTP conversation starts with the ESA issuing a 250 <hostname> banner (by default). The client proceeds at this point with EHLO. The EHLO hostname string is recorded, and may be used in sender authentication tests, a topic covered in Chapter 15, “Advanced Topics.” You can’t reject connections in the HAT based on the hostname string or write filters or invoke anti-spam policies against it. There’s not much reason to: Although the RFC requires the client to send a resolvable host or domain name, and allows servers to verify the hostname/IP match, it prohibits servers from refusing messages on a mismatch.

Rate Limiting with Mail Flow Policies

If a mail flow policy has a behavior of accept, you can further specify exactly how much mail is accepted. This rate-limiting capability is exceptionally important for managing the email injection rates from individual senders so that no system can overwhelm the ESA. It’s also effective at slowing down possible spam senders that use a hit-and-run technique to deliver new spam content quickly, before anti-spam rules can be updated. This process of rate-limiting suspicious senders is called throttling, and the built-in HAT group SUSPECTLIST and its policy THROTTLED make use of these limits to control possible spam senders.

Figure 3-2 shows the mail flow policy connection and message limits.

Figure 3-2. Mail Flow Policy Connection and Message Limits

Figure 3-3 shows the mail flow policy rate-limiting options.

Figure 3-3. Mail Flow Policy Rate Limiting

DNS and Envelope Checks

After greetings are exchanged, the sender begins his message with the MAIL FROM command. There are a couple of features we can use to stop the sender right at this point. The mail flow policy has a feature called Envelope Sender DNS verification, which checks the domain of the MAIL FROM address to ensure that it exists in DNS. The closely related Exception Table (more properly, the Sender Verification Exception Table) defines sender addresses that should always be allowed or always rejected. Here’s what it looks like to the client to be rejected at this stage, because the domain ironport0.com doesn’t exist:

EHLO cisco.com
250-esa02.cisco.com
250-8BITMIME
250-SIZE 20971520
250 STARTTLS
MAIL FROM: <[email protected]>
553 Envelope sender <[email protected]> rejected

Why would you want to do this? Well, a sender whose domain can’t be found in DNS can’t have replies or bounces sent back to it, for one. Some groupware servers won’t accept these irresolvable sender addresses, and it’s pointless to accept a message on the ESA that ultimately won’t be delivered to the groupware server. Another use is to explicitly reject any sender connecting to your public MXs but attempting to use an internal domain or local address as the sender—akin to rejecting internal networks on the outside interface of a firewall. It all sounds like an ideal way to stop some junk, but there’s a false positive problem because some legitimate senders don’t have their DNS house in order. As we examine techniques to stop unwanted mail, I highlight the cost-benefit balance of each feature. Sender DNS verification will stop some junk, but be warned that it also occasionally stops some legitimate traffic that may require you to investigate and correct.

Sender Authentication

Sender authentication lookups are performed. If you’re configured to check Sender Policy Framework (SPF) records, new in AsyncOS 7.1 is the ability to perform such checks in the SMTP conversation and reject senders that fail the SPF test. DNS lookups for SPF verification are done here. Sender authentication is thoroughly covered in Chapter 15.

Recipient Access Table and LDAP Accept

In response to each RCPT TO command issued by the sender, the ESA checks the Recipient Access Table (RAT) to determine if the recipient, or the domain portion of the recipient, is allowed. If configured, an LDAP lookup for the recipient is also performed here. The goal in both cases is to avoid accepting messages for recipients that don’t exist locally and to prevent the abuse of the ESA as a relay for other domains. You must list in the RAT all local domains that you plan to accept messages for, even if you’re using LDAP recipient checks. To be accepted, a recipient must pass both checks. The RAT also acts as a backup if LDAP becomes unavailable.

The RAT takes entries in a few different forms, and can include usernames or just the host or domain portion after the @ symbol. Each entry has a basic action—Accept or Reject—and a few options. Table 3-1 describes the types of RAT entries.

Table 3-1. RAT Table Entries

Every RAT entry has a few options associated with it:

• Accept or Reject: Basic action to be taken. You will mostly use Accept here because the RAT default is Reject, but it is possible to selectively reject individual usernames within a domain that you would normally accept.

• Bypass LDAP: For recipients to be accepted, you can decide whether the address should further be checked by LDAP. This is perfect if some domains have addresses stored in a directory but others don’t or to make an exception to LDAP validation for a specific recipient. I often see this used to explicitly allow [email protected] even though the address may not be stored in a directory.

• Custom SMTP Response: Tells the ESA to use a custom response when rejecting or accepting a particular recipient domain or address. There’s very little call for this, but some companies put a specific rejection message on specific user addresses. For example, imagine a PR disaster that makes an executive’s email address a popular target. You might reject that recipient address, with a polite but firm message that indicates that the recipient isn’t available anymore.

• Bypass Receiving Control: When turned on, this option overrides the various rate limits that count recipients. The only case where you’d ever want to do this would be for abuse-reporting addresses, like abuse@ or postmaster@, so that these addresses can be used by senders that are trying to resolve problems with abuse reports. Even with this, it’s not going to allow a BLACKLIST sender past a connection rejection.

LDAP operations are normally performed later in the email pipeline, but LDAP Accept queries are an exception to that rule. This is so that the ESA can check, validate, and respond to the sender during the RCPT TO command. This recipient validation during conversation is extremely valuable and, if directory information is available, strongly recommended. If you have multiple Internet domains, but not all have LDAP available, deploy it for the ones that do.

Validating the recipient reduces the volume of messages that must be accepted, filtered, and attempted for delivery. In some environments, invalid recipient addresses can be more than 50% of the messages accepted, requiring much more work and potentially more equipment to handle. Further, validating recipients allows the ESAs to keep track of senders that have suspiciously high rates of invalid recipient accounts. These directory harvesters are up to no good, and the ESA can dynamically block this behavior. Lastly, validating the recipient address notifies the sender client immediately, avoiding the process of generating a bounce message later.

Recipient and Sender Manipulation

At this point, a sender whose message has passed all the HAT rules, and has at least one valid recipient, continues with the DATA command to deliver the message body. At this point, the ESA accepts the message, as no checks will be done on message content during the SMTP conversation. After all the data is sent, as indicated by the client concluding with <CRLF>.<CRLF>, the ESA writes the message contents to disk, and responds to the sender with a 250 code indicating message success. That response means that the ESA is taking responsibility for the message and the sender can consider it delivered.

At this point, message processing continues down the ESA pipeline independent of the SMTP connection that sent it. If the ESA doesn’t close the connection because of policy, the sender can either quit the conversation and close the connection or start from the beginning and deliver another message by issuing another MAIL FROM command.

Default Domain, Domain Map, and Aliases

Most MTAs have historically supported the ability to rewrite portions of recipient addresses, and the ESA provides this through several features. Note that the rewrite changes I’m talking about are happening to the envelope recipient address, and not the visible To: header that users see. For example, a message addressed to [email protected] but aliased to [email protected] still shows me [email protected] in the To: field when I receive it at my Cisco mailbox. The envelope recipient is used to route the message to the right place.

The recipient transform features are

• Default Domain: A setting on the listener that applies a domain name to recipient addresses that are injected without a fully qualified domain name (FQDN).

• Domain Map: A table on the listener that provides a one-to-one mapping of the domain portion of recipient addresses. For example, an entry of @ironport.com → @cisco.com changes all recipients of the form [email protected] into [email protected]. No change is made to the username portion.

• Aliases: A global table whose entries are managed through the CLI command aliasconfig. Aliases can change recipient addresses in one-to-one, one-to-many, and many-to-one mappings. One-to-one mappings are used to rewrite a single email address to another, such as [email protected] → [email protected]. One-to-many mappings are like distribution lists, allowing a message to a single address to be delivered to many recipients, such as [email protected]. Many-to-one mappings allow a single recipient to receive all messages for a given domain. You may want to use this mapping on Internet domains that are used only for websites, for example, routing all the mail for the domain to a PR or marketing department employee.

Note that aliasing is also available through LDAP by using the LDAP Routing feature described later in this chapter.

Masquerading

The complement to recipient address rewrites is called sender masquerading. If you need to change or hide addresses sent from within your organization, masquerading is the right tool. The masquerade transforms are typically performed on envelope sender addresses, but can also affect the addresses that users will see in the To, From, CC, and Reply-To headers. Masquerading address rewrites are performed only on outgoing email messages.

Each listener has a masquerade table that performs static mappings, and which lists the headers that should be examined for rewrites. Although individual user entries can be made, the most common use of masquerading is to flatten email address domain hierarchies or to hide internal or acquired domains that shouldn’t be used on the Internet.

For example, suppose your organization uses the domain recycling.org on the Internet but has separate departments for plastics, aluminum, and paper. Users in each department might use addresses such as [email protected] and [email protected]. These addresses are useful for maintaining separate email environments, but might not be valid addresses on the Internet. Assuming that there’s no clash of usernames, a masquerade entry of

@.recycling.org  @recycling.org

transforms all addresses of the form [email protected] to the top-level [email protected].

Note that masquerading is also available through LDAP, described next.

LDAP Operations

ESA supports LDAP queries for a wide variety of tasks, from validating and routing mail to authenticating users and clients. Thorough coverage of LDAP is provided in Chapter 7, “Directories and Policies,” but it’s instructive to know which queries are applied in this part of the pipeline.

LDAP Accept

We covered LDAP Accept for recipient validation in a previous section, but it makes another appearance here because LDAP Accept queries have two possible places in the pipeline: during the SMTP conversation immediately after the RAT check or after message acceptance and address transformations. When you add an LDAP Accept query to a listener, you have the choice of where in the pipeline the action should occur.

It’s not really much of a choice: The SMTP conversation check is definitely the right place to do it. LDAP Accept in the work queue means that the message must be accepted by the ESA; whether the recipient is valid or not, the sender has delivered the message and disconnected. You only have two options at this point if the recipient is invalid: bounce, which means composing a new message back to the purported sending address, or drop, which silently throws the message away. Because of bounce storms and forged sender addresses, bouncing is not a good substitute for in-conversation rejection.

The only possible justification in using the work queue would be for safety in the event of an LDAP outage: The work queue check allows the ESA to queue the messages while it waits for LDAP to become available. The SMTP conversation check is time-sensitive and cannot wait for LDAP to be available. However, this still isn’t a good reason to move to work-queue validation, as you can also set the SMTP conversation policy to allow mail in if LDAP is unavailable.

I recommend using SMTP conversation checks for LDAP Accept in all cases.

LDAP Routing and Masquerading

The operations just described under aliases and masquerading based on listener tables are also possible to perform on a per-user basis with LDAP. Masquerading is similar to the masquerade table, but the individual destination addresses are retrieved via an LDAP query rather than from the table.

LDAP routing incorporates the same recipient address rewriting that aliases can do, but adds another feature. The LDAP routing configuration must specify the source address in order to find the recipient, and then specifies either the rewrite address, the target hostname for the user, or both. A recipient can be rewritten and routed based on the new target address, or the recipient can simply be routed to another host. If your LDAP directory includes alias information for users, and hostname for mailbox servers, doing the lookup on the ESA and routing messages appropriately is ideal.

Unlike Accept queries, a failed routing or masquerading lookup doesn’t stop the message. No rewrite is performed, and the message continues on unchanged. For this reason, it’s safe to perform routing and masquerading even on users that don’t have the appropriate attributes in the directory.

Groups

An LDAP group query poses this simple question to the directory: Does this recipient (or sender) address have an attribute called X with value Y? It’s referred to as grouping because the most common form of the query is actually “does this user have a memberOf field with a value Y?” You can use it for any kind of user grouping you like: organizational groups or departments, physical location, spoken language, privilege level... anything that’s stored in an LDAP attribute associated with the email address.

The most common use of LDAP group queries is organizing users into different processing policies. This is commonly based on organizational information, like department, location, or job function. For example, you can create separate filtering policies for members of your legal department or create aggressive anti-spam policies for company executives.

The group queries are performed before any of the security policies or content filtering. The results of the group lookups are available to filters and policies in the work queue. A failed lookup is not harmful, so it’s safe to perform group queries on users that don’t have the appropriate attributes in the directory.

Work Queue and Filtering Engines

The second major stage of the email pipeline is organized into what’s called the ESA work queue. This section is shown in Figure 3-4.

Figure 3-4. Second Portion of the Email Pipeline: the Work Queue

The processing steps taken on a given message in the work queue is mostly configured through Incoming Mail Policies and Outgoing Mail Policies in the Mail Policies menu of the WUI.

Work Queue Overview

The work queue is the place where, not surprisingly, most of the work on the appliance is done. All the custom filtering and security filtering is done here. What’s not so apparent is that, at this point, the ESA is also moving to multiprocessing to take advantage of modern multicore CPUs. The individual filtering engines run in their own process, and multiple instances are run to take advantage of the hardware.

The work queue is really a place where messages line up until they can be picked up by an available process for filtering. When message throughput exceeds the throughput of all the filtering engines, the messages wait in the work queue until they can be processed.

Incoming and Outgoing Mail Policies

Every message that is accepted by the ESA is classified as either incoming or outgoing. The determination is made by the connection behavior of the mail flow policy that was applied at the HAT. Every message that is accepted is considered incoming, and every message that is relayed is considered outgoing.

Within incoming and outgoing, messages are further broken down into one or more incoming or outgoing mail policies. These policies are determined by sender or recipient addresses. Each policy lists sender and/or recipient addresses, either individually, by domain or subdomain, or by LDAP grouping. After a policy is defined, you can customize the settings and action of each stage of the work queue.

Effectively, the single ESA pipeline now splits into separate pipelines depending on the policy that applies. Messages continue down separate paths until they exit the work queue and are put into the last section of the pipeline: delivery. When a recipient matches more than one policy, the ordering of policies is important: first match wins.

One important feature of policies is message splintering. When the ESA examines a message with multiple recipients, it’s possible that the various recipients each match on different policies. For example, you might have users in the IT department that are allowed to receive binary executable files, while the rest of the employees are not. What happens to a message addressed to two users, one in the IT department and one in marketing? The ESA actually separates the two recipients, breaking up the message and simultaneously sending it down both policy pipelines. In this way, the appropriate action is taken for each user.

Message Filters

Message filters are the first module that runs on messages after all the address transformation work is done. Message filters provide a script-like interface for custom rules and can compare any part of a message with regular expression patterns. Because they run before most other processing, they also have some actions that can control work queue processing, such as skipping anti-spam or antivirus scanning.

Message filters are similar to the content filters engine that’s a few steps later in the pipeline—in fact, they’re really just two interfaces into the same engine. Because content filters are available in the WUI, and message filters only in the CLI, it’s far more typical to use content filters for custom rules. We cover the differences in depth in Chapter 11, “Message and Content Filters.”

Here are some tasks where you might use message filters:

• Rerouting messages based on sender, recipient, message size, or subject

• Bypassing anti-spam or antivirus filtering based on some message attribute

• Using advanced filter conditions that are only available here, like checking S/MIME signatures

• Conditionally adding or removing headers from messages

For general filtering tasks, like stopping profanity, dropping attachments, or adding disclaimers, it’s better to use content filters, because these run after the anti-spam and antivirus engines can drop the junk messages. There’s not much reason to filter content that will only be dropped or quarantined later in the pipeline.

Anti-Spam Engine

IronPort Anti-Spam (IPAS) runs in multiple instances, each in a separate AsyncOS process, to spread the scanning load across the available CPU cores. AsyncOS handles this automatically, running an appropriate number of processes for the given hardware and message load. This is why ESA models with 8 CPU cores have peak throughput of about twice that of models that have 4 CPU cores when running IPAS. IPAS only runs with a valid feature key installed on the system.

Under the hood, the engine that powers both IPAS and Virus Outbreak Filters (VOF) is called the Context-Adaptive Scanning Engine (CASE). CASE handles all parts of the message-scanning process: breaking down the message into component parts, decoding attachments, opening archive and compressed formats, and then applying the rules contained in the latest rule definition update. For IPAS, CASE applies the rules for spam messages.

Several actions can be taken with messages found to be spam positive or spam suspect, and some affect further processing in the pipeline:

• Drop: The message is immediately thrown away. Message processing does not continue in the work queue. It is possible to archive a copy of the message to disk in case recovery is required.

• Deliver: The message continues normal processing and moves on to antivirus scanning. The message can be modified, with changes to subject or header, or sent to an alternate recipient, but the message will go through the other scanning engines.

• Spam Quarantine: The message is marked for delivery to the local or external spam quarantine configured in the Security Services menu. The message continues through the work queue to the remaining filtering engines. This is important, because antivirus and any configured content filter rules will be run on these messages before the message is delivered to the quarantine. If a filter drops a message later in the pipeline, it will not appear in the quarantine.

• Bounce: The original message is immediately dropped, and a bounce message is generated and sent to the envelope sender of the original. Because of the widespread fraud of sender addresses, this is never a good idea. You should never bounce spam or virus messages.

Antivirus Engines

Like IPAS, the available antivirus engines run in a separate process and handle their own message breakdown, archive opening, and rules processing. If a message is found to be viral by one engine, or both, the message is considered viral. When a final verdict is reached, the ESA can take several actions:

• Drop: The message is immediately thrown away. This is the default for virus-infected messages. Message processing does NOT continue in the work queue. It is possible to archive a copy of the message to disk in case recovery is required and to optionally send a notification about the event.

• Deliver: The message continues normal processing and moves on to content filtering. This is the default action for Repaired, Encrypted, and Unscannable verdicts. It’s possible to modify the message with a custom header or a new subject or to deliver the original message as an attachment to a new message. Whatever method is chosen, message processing continues to content filters.

• Deliver with modification: For messages that have malicious attachments, it is possible to drop the attachments and deliver the rest of the message intact. The message data that is delivered will continue processing through the work queue.

• Quarantine: The message is marked for delivery to the local virus quarantine that’s part of the default ESA configuration. The message continues through the work queue to the remaining filtering engines.

Content Filtering

Content filters are the primary policy feature for writing custom rules. It’s a Swiss army knife of conditions and actions, allowing you to write rules that test almost any aspect of the message and its parts, the senders, and the recipients. For every incoming and outgoing policy, a selection of content filters is applied to all of the messages passing through that policy. Because of the depth of features, we cover this topic in Chapter 11.

Some common tasks for content filters include the following:

• Filtering attachments: Content filters can quarantine or drop messages with certain attachment types or drop the attachments and leave the rest of the message intact.

• Adding disclaimers: Additional text can be appended to messages, either above or below the message body.

• Rerouting messages to other systems: Filters can identify specific messages by header, sender, recipient, or other data and reroute a message to another host or recipient. You can integrate external encryption, archiving, or DLP systems by using filters to direct messages.

• Filtering for message content: Although I don’t recommend using content filter rules to stop spam or other junk messages, your organization may prohibit certain message content, like profanity. You can also use content filters to stop some sensitive information, such as U.S. Social Security numbers, credit-card numbers, or ABA routing numbers.

Content filters are created globally, but then are selectively applied on a per-policy basis. This allows you to create custom filtering rules for different groups of users. Because content filters run after the anti-spam and antivirus engines, they cannot skip those engines. It is possible to skip outbreak filters, however, and create message tags that can affect processing of DLP. Content filters are run in a well-defined order and offer an action to skip remaining content filters. Unless a content filter uses a final action like drop or bounce, or explicitly skips remaining filters, processing simply moves on to the next filter in the list for that policy.

Virus Outbreak Filters

Virus Outbreak Filters (VOF) actions are applied after content filtering. The purpose is to examine messages that were found to be nonviral by the binary engines, but may be part of a new virus outbreak. The rule examination actually happens with the CASE engine, in one pass, so if IPAS is running the additional work required to run the VOF engine is minimal. Actions are applied at this stage in the pipeline, after content filters, so that VOF doesn’t quarantine messages that your custom rules dictate be dropped. For example, if your organization doesn’t allow, as a blanket rule, commonly exploited file types, such as .pif and .exe, content filters can drop these attachments and VOF won’t act on them.

Every outbreak rule identifies an attachment type or other message attribute, and a severity level from 1–5. The more certain Cisco is about an outbreak, and how widespread it is, the higher the security level will be. Most rules top out at level 3. To date, the false positive rate (where messages were quarantined by rules but turned out to be nonviral) has been vanishingly low—zero for most customers.

Although it’s not a “black box,” VOF really only has one action that can be taken: quarantine. You can examine the rules associated with a given outbreak and, when a message is quarantined, you’ll have visibility into the reason for that action. There are three main configurable options. The first is the global severity level at which to start quarantining, which can be lowered for more aggressive filtering. I generally recommend the default, 3, because most rules are published at 3 and stay there. A rule published with severity 1 or 2 means that there’s a lot of uncertainty about the threat, and that any threat is likely contained within a few customers. The second option is the per-policy list of attachment types to be excluded from VOF rules. This list is empty by default, and I recommend keeping it that way—meaning not skipping VOF for any attachment types. One of the greatest features of VOF is the ability for Cisco to publish rules for any file type, and if a common file type, like XLS or PPT, suddenly has a security exploit in the wild, VOF can react. The third option is to enable Adaptive rules, which can be described as local VOF rules, based on message history and attachment types seen locally on this ESA. We examine VOF options in Chapter 9, “Automating Tasks.”

DLP and Encryption

Data Loss Prevention (DLP) is provided on the ESA with technology from RSA, a division of EMC Corporation. It requires a valid feature key to enable and use. The DLP engine examines content passed to it by the ESA, which extracts text from any attachments.

The primary purpose of DLP is to take action on messages containing sensitive information that are being sent from local senders to Internet destinations. Because of this purpose, DLP is only available on outgoing messages, and so it only appears as a pipeline stage for outgoing mail messages.

DLP rules are organized into policies, and because the ESA uses the word “policy” for so many different things, we use the term DLP policy to refer specifically to those created in the DLP Policy Manager for use by the RSA engine. The featureset includes a large number of built-in policies and the ability to create custom policies. Chapter 15 covers these policies.

When a message is found to have one or more matches according to a policy, the message is said to contain a DLP violation and has a severity setting from 0–100. Based on the DLP policy, violations are classified as Low, Medium, High, or Critical severities, and each classification can have a different action assigned. The actions that DLP supports are

• Deliver: Let the message continue to delivery, potentially with modifications to recipient or destination.

• Deliver with encryption: Let the message continue, but use one of the encryption methods available to secure the content. Encryption is supported gateway to gateway using Transport Layer Security (TLS) or through encryption of the actual message data through the Cisco Envelope Encryption technology.

• Quarantine: Hold the message in one of the ESA’s system quarantines. Each quarantine space specifies how much, and for how long, messages should be held, and what action is to be taken after they expire. The system quarantines are storage areas particular to each appliance.

• Drop: The message is discarded. No further processing is done.

Other actions like sending a copy of the message to another recipient (BCC) or sending a notification about the violation can be combined with these actions. DLP notifications can be composed with a custom message, including variables that refer to attributes of the message and the violation, such as sender, subject, recipient, and the policy name and the content that matched.

Encryption through Cisco IronPort’s Envelope Encryption engine is an optional stage of the email pipeline, usually triggered by content filters or DLP actions. The approach is called envelope encryption because the engine wraps the original message in an encrypted HTML file that adds security and authentication to the message. Authentication of users is handled by the Cisco Registered Email Service (CRES) and, because the technology is based on standard web technologies, is usable by anyone with a web browser or web-enabled mail client.

ESA also supports TLS for delivering email over a secure connection. TLS is covered in Chapter 12, “Advanced Networking.”

Delivery of Messages

Any messages that have not been dropped or bounced now proceed into the delivery stage. I often use the term “delivery queue” to refer to this stage, but in reality, the ESA uses more of a tree structure to organize messages. There is no single disk or memory queue where all messages line up to be delivered in order. The ESA creates a node in its tree for every delivery destination and can simultaneously connect and deliver to all active destinations. Figure 3-5 illustrates the steps taken during the delivery stages.

Figure 3-5. Delivery Stages of the ESA Pipeline

Most of these tables and settings in this section are optional. The default ESA settings are adequate in the vast majority of environments, with the major exception of the SMTP Routes table. If you used the System Setup Wizard to specify the delivery destination for your local domains, you shouldn’t have much reason to change it.

Selecting the Delivery Interface (Virtual Gateways)

When a single ESA has more than one interface, it must choose which interface should be the source of the connection. There are a couple of rules regarding interface selection. On an ESA with no other rules to override it, source interface for SMTP connections follows the source rules for any other kind of connection:

1. If the destination IP is on the same subnet as an ESA interface, use that source interface to send the packet.

2. If the destination IP is not on the same subnet as any ESA interface, send the packet to the default gateway, using the interface that’s on the same subnet as the gateway.

3. If the destination IP is on a network listed in a static route, send the packet to the gateway listed in that static route entry.

4. If the ESA has virtual gateways (that is, it has more than one interface on the same subnet as the gateway), then multiple source IPs would be eligible based on these three rules. The ESA chooses the numerically lowest IP based on an alphanumeric sort.

The strict alphanumeric sort can be a bit confusing. For example, if your ESA has these IPs in a DMZ:

192.168.1.28
192.168.1.9
192.168.1.100

Then a strict numeric sort puts them in this order because 1 is less than 9 when looking only at the first digit in the last octet:

192.168.1.100
192.168.1.28
192.168.1.9

As a result, 192.168.1.100 will be used as the source interface. This last rule on selecting delivery interfaces can be overridden in a couple of ways: using the altsrchost table or using the message or content filter action that selects a delivery interface. (Chapter 12 reviews both of these.)

Destination Controls

The destination controls table provides settings for connection and message rates when ESA is delivering to remote destinations. It also provides settings for attempting or enforcing the use of TLS to these destinations. You can think of it as the corollary to the HAT: instead of rate limiting remote senders, destination controls rate limits the ESA when delivering to remote servers.

Even without hard limits in this table, the ESA follows what is called the good neighbor algorithm when delivering mail to a remote destination. Instead of opening numerous connections all at once, it starts with a single connection and only makes another if the message volume warrants it. By slowly increasing connection counts, the ESA can avoid being forcibly rate limited by remote servers that see a large volume of connections as an attack.

In most environments, the connection and message rates in the table by default, combined with the good neighbor approach, are sufficient. Generally, you’ll be creating entries in this table to set TLS preferences and not rate limits.

Global Unsubscribe

Global unsubscribe is a rarely used table that comes in handy for certain tasks. It is only available in the CLI through the unsubscribe command. It tells the ESA that it should never send messages to certain recipients or domains. For any listed recipient or recipient domain, you have two actions: drop or bounce.

This can be useful if you have machine-generated messaging, like bills, statements, or marketing, and need to prevent it from being sent to sensitive domains or individuals. Normally, automated message subscription lists should be updated to exclude recipients that don’t want to receive the mailings, but those updates can often take days to process. Global unsubscribe is an immediate change that ensures that delivery is never made to that recipient or domain.

SMTP Routes

The determination of the host used to deliver a given recipient is primarily done through the global SMTP routes table, available through both the CLI and the WUI. Every SMTP route table specifies a domain and one or more destinations for the domain. Each destination for a domain is given a priority, or cost, that determines the order that destinations are tried in. Lower cost destinations are preferred. Only full domains and subdomains are allowed as entries; per-recipient destinations must be done with aliases, LDAP routing, or filters. For every entry, there are a few types of destinations supported, as listed in Table 3-2.

Table 3-2. SMTP Routes Destination Types

The table also has a default route that matches any destination, if no specific route entry exists. The default SMTP route works like other entries and uses the same destination types. The ESA typically has an empty default route, and you would normally only provide an entry for ESAs that handle internal traffic only. If no specific entry for the destination exists in the SMTP Routes, ESA uses the default route. With an empty default route, ESA uses the normal DNS process for determining destination host: Look up an MX record and the A record for any host returned in the MX result. If no MX record is found, ESA then looks up an A record for the domain. If no records are found at all, the message is immediately bounced as an unrecoverable error.

Each SMTP route entry can have more than one destination. Providing multiple destinations allows for load balancing or failover across the available destinations. As of AsyncOS 7.1, each destination can have a priority value, with lower numbers being higher priority as with MX records. Zero is the highest priority. Connections to equal-priority hosts are load-balanced in a round-robin fashion. Connections to lower-priority hosts are only made if all of the higher-priority hosts are unavailable.

Lastly, if delivery to a specific destination requires that the SMTP client (in this case, the ESA), authenticate in order to be able to deliver email, you must configure SMTP Auth profiles and specify them in the appropriate destination entries.

Selecting Bounce Profiles

Bounce profiles aren’t really a separate stage of the pipeline, but rather a small policy attached to each message that determines what to do if the message can’t be delivered. Bounce profiles decide how many times, and how often, a message should be retried before the ESA gives up and declares it permanently undeliverable. For permanently undeliverable messages, the bounce profile also determines whether a delivery status notification (DSN) or bounce is generated and where it is sent. Bounce messages are also called non-delivery records (NDR) and usually go to the envelope sender of the message.

Bounce profiles can be assigned to a message in several places in the pipeline, and assigning a different profile later in the pipeline overrides earlier settings. The three places where bounce profiles can be assigned are

1. The listener that the message was received on. Every listener assigns a bounce profile as the message is accepted.

2. Message filters can assign a bounce profile using the bounce-profile action.

3. Destination controls can assign a bounce profile to recipients within a specific destination domain.

These assignments occur in this order and the last match wins, so any bounce profile chosen in a filter overrides the listener setting. Choosing a bounce profile in destination controls overrides both the listener and any filter settings, except if the value Default is chosen, which can be interpreted more accurately as “use the Default if no other bounce profile has been chosen.” If you set a custom bounce profile in a filter, it will not be overridden by a destination control that chooses Default.

Handling Delivery Errors with Bounce Profiles

Not every recipient on every message is always going to be deliverable. Users mistype addresses, employees leave companies, and mail servers don’t always accept messages.

Delivery errors boil down to two general classes: temporary and permanent. Permanent errors include DNS errors, like a nonexistent domain, or email address problems, like a nonexistent or permanently disabled account.

Temporary errors include being unable to make a connection to a remote server, not being able to negotiate connection parameters (such as TLS), or any kind of 4yz SMTP error code returned from the remote server. Eventually, all temporary errors can become permanent errors if the message exceeds the retry limits defined in the bounce profile.

Retry limits for messages are controlled in the individual settings for each bounce profile created through Bounce Profiles in the Network menu of the WUI or the CLI command bounceconfig. Each profile encapsulates settings like the maximum number of retries to make, the maximum time a message may stay in queue, and what action to take if the message exceeds those limits. For example, you can create a bounce profile with a short lifetime like 4 hours and apply it to outgoing messages, while another profile applies a 72-hour limit to incoming messages. You can even create a no bounce profile to prevent some messages from generating a bounce, which can be useful for automated systems that send email.

Retry limits for unreachable hosts are controlled globally, not per-profile. All retry settings for both hosts and messages are expressed in a minimum and maximum time between attempts. The ESA uses a retreating schedule of attempts, starting from the minimum. For example, if the minimum retry is 60s, the first attempt is one minute later, the next two minutes after that, the next four minutes, and so on until the maximum one hour limit is reached.

When a temporary error exceeds the retry limits, the error becomes permanent and ESA can generate a DSN. The default setting is to generate a DSN or bounce message and send it back to the sender, and include information about the error in the message. You can send bounces to another address—for example, when the sender is an automated system. You can also customize the DSN text using a bounce notification template. Customized DSNs can include additional information about the delivery error or about the original message that caused the bounce. If you want the end-user text in a non-English language, you can use customized DSNs to create them.

One last option with bounce profiles is to generate a DSN when a message is undeliverable for some period. The DSN simply states that the message has not been delivered and that the system will continue to attempt to deliver it. Because most users are usually just confused by these status messages, they are off by default, but you can enable them and provide a single or multiple delay warnings at any interval.

Final Disposition

The ESA can never be the final destination for a message because it does not host a mail store. As a result, there are only three possible final states for a message: dropped, bounced, or delivered. There are several non-final states that a message can be in: queued, in process, quarantined, or active, but eventually these will all have a final state. Active messages and recipients are those awaiting delivery and are further broken down into attempted and unattempted.

Messages that are not in a final state have an active MID on the system, and logging for the message continues until it reaches a final state. Because the final state may be determined days after the initial injection, the MID value may span across log files.

There are several places where the final disposition of a message can be determined. Message tracking on the ESA directly, or centralized on an SMA, is probably the most convenient, because it provides a thorough search capability and fast search times. Message disposition is also recorded directly in the IronPort Text Mail logs or in an optional log format called delivery logs. Delivery logs are stored in binary and require additional processing, but the text mail logs are, of course, in plain ASCII text and can be searched directly.

Once a message reaches final disposition, all of its contents are removed from memory and disk and only the record in the logs and tracking is kept on the ESA. It is possible to archive message contents locally on the ESA using a filter action, but it’s not possible to directly reinject messages from those archives.

For delivered messages, ESA waits until the message has been acknowledged with a 2yz response code from the SMTP server. Any text accompanying the response code is logged.

Summary

This chapter describes the entirety of the message lifecycle in the ESA pipeline, from initial SMTP connection to final delivery. We covered basic message transport capabilities, LDAP features, and security filtering. You should have a good understanding of the stages of the ESA email pipeline and the order in which they occur.

Now that we have a lot of the theory of operation covered, we’re next going to dive into the full management capabilities of the appliance, first in the web user interface in Chapter 4, and then in the command-line interface in Chapter 5.