Chapter 7. Directories and Policies

In this chapter, you will learn the following:

• How to integrate ESA with LDAP directories, including Microsoft Active Directory

• How to create and use LDAP server profiles and basic and advanced LDAP queries

• How to create incoming and outgoing mail policies and tie those policies to directory information

Directory Integration

Email systems have a close relationship with other services in the network. Most prominent among these is Domain Name System (DNS), because of the dependency of Simple Mail Transfer Protocol (SMTP) on DNS. In most organizations today, however, directory services provided through Lightweight Directory Access Protocol (LDAP) are critical to email environments. In addition to providing information like address books for mail user agents (MUA), information in a directory can be used for routing email messages and applying policy to individual senders or recipients. Directories are also typically used to authenticate users for network services and applications, and email is no exception.

The Email Security Appliance (ESA) supports LDAP and LDAP over SSL/TLS, referred to as LDAPS. ESA does not provide any directory services—it’s only an LDAP client—but does cache results returned from queries.

The single-most common LDAP server in use is Microsoft’s Active Directory (AD). It’s so common that most of the ESA documentation refers to AD integration, instead of LDAP, and most configuration examples are targeted toward AD environments. In fact, the ESA has a Wizard mode that eases setup for AD, and most of the setup defaults use AD defaults. Because of its ubiquity, we refer to AD most often, including examples based on AD attributes and queries. Note, however, that the ESA speaks LDAP only, and AD is just another LDAP server as far as it is concerned.

The Need for Directory Integration

Directory integration with your ESA provides many important advantages for your email environment:

• Lower message volumes: Using LDAP recipient validation through LDAP accept queries reduces the number of messages your ESA has to accept. It’s a lot of unnecessary work to accept and filter messages that cannot be delivered because there’s no valid user with that address.

• Lower administration: Policy changes can be done at the directory instead of on the ESA. For example, grouping users by their language or location can make adding disclaimers or sending notifications in the correct language easy.

• Improved security: Using SMTP-conversation recipient validation with the dictionary attack prevention feature helps the ESA identify bad SMTP clients and keeps your valid email address list from being discovered and sold.

For some policy tasks, there’s simply no feasible alternative to using a directory. In large environments, the capacity of the ESA tables may become an issue, and administration of large static tables is a headache. The maintenance and security risk from other approaches far outweighs any drawbacks to ESA LDAP integration.

Security Concerns

The idea of exposing internal directory access to a gateway security device installed in the demilitarized zone (DMZ) is sometimes unpopular from a security perspective. The legitimate concern is that an attacker could gain access to an ESA and use it as a bastion to attack other systems; information installed in the directory could be stolen or modified. The first step is to prevent unauthorized access to an ESA in the first place. Cisco takes significant precautions against the exploitation of the ESA appliance software, and in fact, the appliance software doesn’t include the basic tools that could be used for such attacks. Appropriate network layer security can control access to the administrative interfaces on the ESA. Taking a few precautions minimizes any risk.

There are a few general recommendations for minimizing the perceived risks that LDAP queries from the ESA to an organization’s directory servers could create:

• The ESA should authenticate to LDAP with a username and password created specifically for the purpose.

• The username associated with ESA LDAP queries should have the minimum required privileges and be read-only if possible. Don’t use any kind of administrative account.

• Use LDAP over SSL/TLS (LDAPS) to encrypt the communications between the ESAs and the LDAP directory.

• Have an LDAP server available directly in the DMZ, eliminating the need to make LDAP connections inside; many LDAP servers support read-only instances to support this model, with synchronization performed by pushing updates from inside to the DMZ. If you plan to dedicate LDAP servers for messaging, it may be possible to expose only a portion of the directory to the ESA.

• If you’re crossing network boundaries, the firewalls or other network security devices should have rules that include the specific source (ESA) and destination IP and ports of the LDAP servers.

If direct access to LDAP servers is absolutely forbidden, you have other options for certain policy tasks. SMTP recipient validation can be performed with the Recipient Access Table (RAT). Recipient or sender policy grouping can be done with static entries in the appropriate policy. Both create a maintenance headache, and I’m left wondering this: Is having a list of email addresses in a static table on the ESA preferable to giving the ESA minimal directory access?

Another alternative introduced in AsyncOS 7.5 is SMTP call-ahead, which allows the ESA to use SMTP as a recipient validation protocol. It is only for recipient validation, so it can only replace LDAP accept queries. Call-ahead requires that there is an available SMTP server that can provide a clear acceptance and rejection response for the RCPT TO command.

Brief LDAP Overview

The LDAP was developed as a low-overhead alternative to the standard client protocols for X.500 directories. Although it is, technically, only a directory access protocol, it also necessarily defines the structure of the data being accessed. Under the hood, the directory can store its data in a variety of ways—LDAP does not constrain the architecture of the server software itself. Some servers use relational databases to store the directory attributes and values, while others use flat files, but the backend storage method does not impact ESA.

The current specification for the protocol is provided in RFC 4510, along with a number of others. The protocol defines a number of operations that can occur between a client and a server, but in the case of ESA and a directory server, only a few are relevant:

• StartTLS: Begins the negotiation for a TLS connection so that credentials and query data are sent over an encrypted channel. ESA supports secure connections for LDAP.

• Bind: In LDAP terminology, bind is the term for authentication of a client to an LDAP server. ESA uses a bind operation to connect to the server for making queries, but also as an authentication mechanism for administrators and end users.

• Search: Query the LDAP server for attributes of a specific value. Returns the count of matches and the matching values.

• Compare: Query the LDAP server to test if a named attribute has a specific value. LDAP is always case-insensitive when matching attribute names or values.

The ESA does not support any operations that modify the directory, like adding or removing entries or editing any attributes. It’s a read-only client.

LDAP envisions a directory as a tree starting at a root node. Every node is itself a directory tree, with subnodes that contain entries and other nodes. Entries in the directory are name: value pairs, called attributes, and can have one or more values.

Figure 7-1 shows a conceptual diagram of an LDAP directory tree.

Figure 7-1. Simplified Conceptual LDAP Tree

The unique path from root to the actual node where attributes are recorded is called the distinguished name (DN) and is often, but not always, recorded as an attribute on the entry itself. For example, the DN of the user account in Figure 7-1 is just a comma-separated list of all the node names:

dn:cn=Chris Porter,ou=users,dc=example,dc=com

Although LDAP is a binary protocol, it’s convenient to represent attributes in text format for our discussions, and the LDAP Interchange Format (LDIF) provides just that. An LDIF representation of the user account from Figure 7-1 might be displayed as

dn: cn=Chris Porter,ou=users,dc=example,dc=com
objectClass: person
cn: Chris Porter
c: US
mail: [email protected]
proxyAddresses: SMTP:[email protected]
proxyAddresses: smtp:[email protected]
memberOf: ou=ESA Admins,ou=groups,dc=example,dc=com
memberOf: ou=IT Security,ou=groups,dc=example,dc=com

At the top is the attribute dn, or Distinguished Name, which uniquely identifies this particular node in the LDAP tree by its complete path back to the root. Aside from this object’s DN, there are other attributes that have values that are LDAP DNs, which is perfectly valid, and a fact that I use later when we see LDAP group queries. I use this type of LDIF text layout in examples in this chapter. Because the ESA is focused on email, the LDAP objects you’ll deal with most are users—real people—so, the examples revolve around those kinds of entries.

There’s one note about LDAP directory servers being used with the ESA. The ESA has a fairly short LDAP query timeout, necessitated by the need to react at SMTP conversation time, and so expects the LDAP server to respond quickly. Most LDAP servers perform a process called indexing to speed up search times for commonly used attributes. Any attribute you intend to query with the ESA needs to be indexed. Configuring indexing varies with the LDAP server software, and most servers index the email-related attributes in their default configuration. If you’re querying other attributes, however, it makes sense to double-check that they are indexed, and be aware that unindexed attributes can be a cause of LDAP timeouts on the ESA. The ESA response to timeouts on LDAP recipient validation queries depends on your configuration. You can set it to accept messages if LDAP is unreachable, relying only on the RAT for recipient checks, or set it to temporarily reject messages until LDAP is restored. For other LDAP queries, the ESA queues messages until LDAP is available. Timeouts and other connection errors result in alerts being sent from the ESA.

LDAP Setup on ESA

To start using LDAP, you need one or more LDAP server profiles. You can set these up manually or, if you’re using AD, you can use the AD Wizard. The wizard makes LDAP setup faster because it preconfigures attributes common to most AD deployments. Either way, you can always modify an LDAP server profile later.

To create an LDAP server profile, you need all the following information, at a minimum:

• A name for the LDAP profile: This is just a friendly identifier to distinguish profiles from each other and to name the queries attached to the profile.

• Hostnames or IP addresses of one or more LDAP servers: This is the directory server or servers to contact for all the queries under this profile. The servers should be identical, meaning that they should support the same queries and contain the same data. Support for multiple servers is provided here for failover and load balancing. If you have multiple directories with disparate data, see the “Advanced LDAP Queries” section later in this chapter.

• Base DN: In the LDAP directory tree, this is the root node that all searches should start from. When you specify a base DN, all searches are relative to that base and the search is limited to nodes and subnodes of that base. You may be tempted to set an empty base, starting at the root of the LDAP directory. This is allowed, but narrowing the base DN to make it as specific as possible improves performance. In the example, we could choose dc=example,dc=com as the base, because all relevant entries are below that node, or you might go further and specify ou=users,dc=example,dc=com if you’re only querying users.

• Credentials for access to the directory: You need a username and password for an account that has at least read-only privileges; the ESA does not make changes to any directory. You can also use anonymous access if the directory permits.

• Port to contact the LDAP server on: LDAP customarily uses 389 for LDAP and 636 for TLS LDAP. AD uses these ports also, but you’re better off using the Global Catalog (GC) ports of 3268 and 3269.

• At least one query: Although you’re technically not required to create an LDAP profile, the queries are the fundamental part of any profile. We cover basic and advanced queries later in this chapter.

After the profile is created, you can use the associated queries in relevant parts of the configuration. Note that creating a profile and queries does not put the queries into use; you must configure other features to use LDAP before they actually apply to accepted or relayed messages. For example, creating and testing an LDAP accept query does not enable recipient validations until you assign that query to a listener. Group queries don’t sort users until you assign the query to a listener and either create a filter condition or define policy membership.

Figure 7-2 shows the basic setup of an LDAP server profile.

Figure 7-2. LDAP Server Profile Setup

This profile contains all the elements in the previous list, with a single recipient validation query configured.

Advanced Profile Settings

The Advanced drop-down on any LDAP profile provides configuration for a few connection related settings. Normally, the defaults are sensible, but it’s worth taking a look to understand how changing these settings can improve performance.

The first is caching for LDAP lookups. Caching makes lookups more efficient by storing both the positive and negative results of recent queries. The Time to Live (TTL) sets how long the results are good for, and the maximum cache entries is of course how many entries will be used. I recommend using longer TTLs, if you can. If the attributes that the ESA is querying don’t change that often, setting this to 4 hours (14,400 seconds) or even a whole day (86,400 s) will have benefits for performance. Increasing the cache size uses more memory on the ESA, but the default of 10,000 is fairly anemic; I recommend 4 hour TTLs and 100,000 entries in cache. In most environments, that’s enough to cache all the active email recipients.

The other advanced settings are for connections, controlling the number of connections the ESA can make and how to handle multiple hosts in a single profile. Multiple hosts can either be load-balanced so that queries are made round-robin to all the hosts in the list, or failover so that queries are normally made to the first LDAP server in the list and proceed to the others in the list only upon connection failure.

It’s worth explaining how the ESA uses LDAP connections. When you create an LDAP server profile and commit the change, the ESA immediately binds to the specified LDAP server and keeps the connection open. If the value in Maximum number of simultaneous connections for each host is greater than 1, the ESA attempts to connect and bind that number of times. This setting could be more accurately described as always maintain this number of active simultaneous connections. Having these connections open, even when no LDAP queries are active, reduces latency when a query is necessary. The ESA’s LDAP client uses one of the available open connections to send the query, avoiding the expense of the LDAP Bind overhead. The ESA keeps these connections open forever, if allowed to, and reopens any connection that is terminated by the LDAP server so that the maximum number is always being maintained. If you specify a load-balanced relationship among hosts, the ESA maintains the maximum number of connections to all the hosts in the list.

You can see the connection behavior of the LDAP client in the LDAP debug logs, which are discussed later in this chapter. These logs show the ESA opening new connections immediately as the LDAP server terminates them.

Basic Query Types

The basic ESA LDAP queries are

• Recipient validation: A simple check to determine whether a recipient exists in the directory. Essentially just a true/false query; any non-zero result from the directory is assumed to mean that the recipient exists.

• Group queries: Used for checking some attribute of a recipient or sender address to associate that address with some organizational group. A flexible query that can be used for various purposes.

• Routing query: I also refer to this as LDAP aliasing because these queries are designed to rewrite or redirect recipient addresses or both. Rewriting actually changes an individual address to one or more other addresses, so it can be used for either altering a single address or expanding a single address to multiple addresses, as with a distribution list. It can also redirect the recipient to a specific delivery host, either separate from or in conjunction with recipient rewrites. This capability is commonly used in Internet server provider (ISP) and university environments, less so in AD environments.

• Masquerading: You can think of masquerading as the opposite of aliasing. Instead of rewriting recipient addresses, it rewrites sender addresses. The purpose is to change any internal only addressing into externally resolvable full email addresses.

• SMTP authentication: Configures queries for the ESMTP extension SMTP-AUTH, which provides for authentication to mail servers. The ESA can act as either an SMTP AUTH client or server, but for LDAP purposes, we’re concerned with how remote clients authenticate to the ESA. SMTP-AUTH is specified in RFC 4954.

• External authentication: Provides for LDAP authentication to a directory for the purpose of logging into the ESA. If you want to allow administrative users that have accounts in AD or other LDAP directory to use those credentials for ESA administration, you must configure an LDAP query and define access levels.

• Spam quarantine authentication: Like external authentication, this is intended to allow login via credentials stored in an LDAP directory. In this case, however, the users are your user population accessing their stored spam messages in the spam quarantine. This is not the only means of allowing end user access to the quarantine, but if you want them to use the credentials they already have and are familiar with, you must configure the LDAP query first.

• Spam quarantine alias consolidation: If you are allowing end users to access the spam quarantine, you might also consider consolidating multiple email addresses per user into a single view. The quarantine can only do this if it can query LDAP to determine all the email addresses for a given user.

The most commonly used queries are the LDAP recipient validation and group queries, so I dedicate large sections on those subjects. Routing (aliasing) and masquerading are critical in certain environments, if the right directory information is available. If it’s at all possible, I strongly recommend using LDAP recipient validation, because it improves performance and security, so I discuss this next.

Recipient Validation with LDAP

The single most common use for LDAP on a Message Transfer Agent (MTA) is to validate recipients. This means querying an LDAP server with a simple “does this user exist?” question. A positive response means that the recipient (and message) is accepted; a negative response means that the recipient is rejected or bounced. Even with LDAP recipient validation, you still must list the valid recipient domains in the RAT, which the ESA checks before making an LDAP query.

Recipient validation has a few differences from the other LDAP query types:

• Can act at SMTP connection time. Lookups are performed while the client is still connected, at each instance of the SMTP RCPT TO command. The response is given back to the sender immediately. Most of the other query types are performed after the message is accepted.

• Negative results are harmful to messages. For most queries, a negative result means does not apply and message processing continues. Not so with LDAP accept: A negative response means that recipients are rejected or dropped. Note that error responses aren’t considered negative; a negative response is one where the connection and query succeeded, but the result was 0 matches.

• Only applied on HAT policies with accept behavior. LDAP accept is effectively bypassed on Relay (outgoing) behavior HAT mail flow policies, to avoid performing recipient validation on messages sent from your organization out to Internet recipients. Other LDAP query types are performed on all messages received at a particular listener.

For most corporate LDAP servers, the attribute named mail stores the valid email address for a user. It doesn’t matter if there are multiple instances of the same attribute because our query is essentially “does any mail attribute in the search path have value [email protected]?” The ESA doesn’t care what the values are, or how many matches there are, only whether there is a match or not. The query for the mail attribute is:

(mail={a})

Microsoft Exchange and AD use two attributes: mail and proxyAddresses. There’s an instance of the proxyAddresses attribute for every address for the user, and the value starts with smtp: or SMTP: for those types of addresses. The primary email address for the user is stored with a capital SMTP prefix and is copied into the mail attribute. Unfortunately, LDAP queries aren’t case sensitive, so ESA can’t see the difference between the primary and other proxyAddresses values. It’s no matter, however, because all we care about is whether there’s a matching value in the directory regardless of case, and this query accomplishes that. You’ll find that in the ESA web user interface (WUI), this query is the default for accept queries on AD profiles:

(proxyAddresses=smtp:{a})

When the recipient address isn’t found in the directory, you can choose to reject the recipient at SMTP conversation time or to bounce the message at the work queue. Of these two options, there’s not really much debate: You should use SMTP rejection. This is preferable because the sending client knows immediately that the recipient isn’t valid, and you avoid the trouble of accepting and bouncing the message later.

LDAP accept queries don’t have to be so simple. You can query multiple attributes and query using only a part of the recipient address with the {u} user or {d} domain variables. For example, to test only if the domain is valid, you can use a query like this:

(localdomain={d})

To test both the username and the full address, you can use a query like this:

(|(mail={a})(uid={u}))

The vertical bar or pipe character is a logical or condition, making the search result true if either an instance of the mail attribute matches the full recipient address or the uid attribute matches the username portion to the left of the @ sign. You don’t have to worry about the domain portion being local as long as the RAT is correctly configured. Logical and is the ampersand character, as in this test:

(&(mail={a})(externalMail=true))

That query tests whether the recipient address exists, but also that the attribute external-Mail has the string value, true. You use this sort of query to limit inbound messages to addresses that are specifically enabled to receive mail from outside the organization.

Recipient Routing with LDAP

LDAP routing on the ESA is designed to support the delivery of individual recipients to the appropriate host after the ESA finished message processing. The phrase routing actually refers to two specific capabilities: rewriting envelope recipient addresses (aliasing) and setting delivery destination host (routing). Routing LDAP queries are less frequently used than other LDAP features simply because most corporate LDAP directories don’t have appropriate attributes for it, and because most groupware servers, like Exchange and Notes, have the ability to route messages themselves for their local users.

As mentioned, routing queries have two parts: aliasing and destination host. You can use one or the other or both on incoming mail. For example, aliasing is the process of looking up an original recipient address, like

[email protected]

Rewriting the entire address to something else, like

[email protected]

The source and target address don’t need to be in the same domain. One common use case is to rewrite external-facing addresses to internal addresses that are only relevant inside your network. For example, an Internet email address like

[email protected]

might internally need to be transformed into

[email protected]

The ESA can perform this rewrite based on an LDAP lookup on each recipient, and will then route the message according to the new address, hence the term LDAP routing. In this example, the ESA delivers the message via SMTPRoutes or DNS lookup for boston.example.com. Only the envelope recipient addresses are rewritten; the visible headers, like To: and CC:, are not modified by this process.

The other feature of LDAP routing is to change the delivery destination for the recipient without rewriting. This requires that the directory have an attribute that stores the fully qualified domain name (FQDN) of the delivery destination for the user.

To use LDAP routing, you need to have this information stored in LDAP for this to work. Routing setup requires two things: a query to locate the user in the directory and the name of the attribute(s) to use when rewriting or rerouting. Locating the user works just like it does with LDAP accept, but the query is doing more than a simple yes/no test. The ESA is actually asking the LDAP server, “What is the value of attribute X for user with email address [email protected]?”, and then using that address to rewrite or reroute. The second part of the equation is the actual attribute to use for rewriting or rerouting. Here’s an LDIF excerpt that includes suitable attributes for use with both LDAP routing features:

dn: cn=alice smith,ou=users,dc=example,dc=com
cn: Alice Smith
mailRoutingAddress: [email protected]
mail: [email protected]
mail: [email protected]
mail: [email protected]
mailhost: webmail.example.com

In this case, the LDAP locator query will be:

(mail={a})

The attribute to use for rewrite is mailRoutingAddress and for destination host is mailhost. Again, you can use either one of those or both.

If you only have information for a subset of users, you can still use LDAP routing, because if the locator query fails to find the user, the rewrite step is simply bypassed and the recipient is routed as normal. No harm is done if the address can’t be found.

Sender Masquerading

If you’re using LDAP aliasing to hide internal SMTP architecture by rewriting external addresses to internal addresses for incoming mail, you’ll probably want to reverse that process for outgoing mail. That is, when a user sends a message from an internal address, the local address needs to be rewritten to an externally facing one.

In the most common case, masquerading is used simply to flatten internal subdomains into a single external domain, without changing the user portion, as in this transformation:

[email protected]: [email protected]

You don’t have to use LDAP for this; the static masquerade table on the listener accomplishes this directly. It’s available in the CLI command listenerconfig under the settings for individual listeners. The static table also supports individual user address rewrites. As is the case with most of the static tables, however, maintaining the user-by-user rewrites may be easier if the appropriate information is available in LDAP. Similar to LDAP routing, you must have both the source (internal) address and the destination (external) address in an attribute for each user, and the destination attribute must be single-instance.

When you are using masquerading, LDAP or otherwise, the important configuration option is deciding which headers to rewrite. For outbound messages, it’s possible that local (internal) addresses are used in To, CC, and Reply-To, in addition to the From header. The selection is made in the listener configuration when you’re assigning the query to the listener. Make sure that you enable these header rewrites appropriately; I recommend turning them all on, with the possible exception of the envelope sender address. The envelope sender should only be used in the event of a bounce or other system message, and the internal address is likely what you need to be used. For messages that are delivered, the From address determines replies and the like, and should be masqueraded.

Group Queries

When using LDAP to invoke policy on the ESA, I am generally referring to using LDAP group queries. Group queries might sound complex, but they are really quite simple: They query the directory using the supplied query string, and if there’s at least one match, the sender or recipient is considered a member of the group. Recipient addresses are tested (individually) for incoming mail, and sender addresses are tested for outgoing mail.

The complexity for these queries can come in the query string. These strings are essentially arbitrary—you can query anything. In general, you need at least two attributes to test: an attribute that identifies the user, whether by a username or full email address. I call this part of the query the locator. You also need at least one other attribute to test for group membership.

Here’s an example that uses the full email address as the locator and an attribute called departmentName as the group test:

(&(mail={a})(departmentName={g}))

When this is applied to incoming mail, a substitution occurs for the {a} and {g} variables. The variable {a} (for address) refers to either the sender or recipient, depending on context. (More on the context later in this chapter.) The {g} variable refers to the group name, which is supplied as either the argument to the policy membership, or by the argument to a content filter. In this example, we assume that this is incoming mail, and we have a policy called Legal, which uses the string Legal as the group query string. The result of the substitution is

(&([email protected])(departmentName=Legal))

which is passed to the LDAP directory. Assuming the query is a success, the address is considered a part of the group if there is a non-zero result set. Zero results means that the user is not considered a member of the group. You can use any query that works—if you can compose the LDAP filter logic to return what you want, you can use it here. For example, if I want to narrow the previous example to the Legal department in one country, I use this:

(&(mail={a})(departmentName={g})(countryCode=US))

In this example, I’m hard-coding the string US for the countryCode. Because you only get one group variable, {g}, you have to use it wisely. You can use it in more than one place, if you carefully consider what that means. For example, suppose you have this query:

(&(mail={a})(|(departmentName={g})(countryCode={g})))

You’re using this query with two policies that use the strings Legal and US as the group policy membership criteria. During message processing, the two policies cause these LDAP queries to be sent:

(&(mail={a})(|(departmentName=Legal)(countryCode=Legal)))

(&(mail={a})(|(departmentName=US)(countryCode=US)))

If only one query returns a positive match, obviously, the message falls into the corresponding policy. If both queries return a positive match, order determines the policy, based on top-down evaluation of policy order. If you have users that match both policies (because they are in the Legal department in the U.S.), you must decide which policy takes precedence and order it higher in the list.

This only begins to hint at some of the flexibility of group queries. You can use such a query for any attribute that distinguishes users, provided that they’re indexed for fast searches. You can use any of the LDAP logical operators, like and &, or |, and not !, and you’re not limited to one or two attributes. You don’t even necessarily have to use the {g} variable, as we see later in this chapter. Table 7-1 gives some examples with a description.

Table 7-1. Compound Group Query Examples

Authentication Queries

LDAP is used for a number of authentication tasks on the ESA. You can use LDAP to authenticate administrative users, replacing the local username and password logins. If you’re using the spam quarantine features, you can, optionally, force end users to authenticate. You can also authenticate connecting SMTP clients using SMTP-AUTH. All function in essentially the same way: Credentials are received from the client, and those credentials are used to bind to the relevant LDAP server. A successful bind means that the user is authenticated. SMTP AUTH queries also allow retrieval of a password attribute, to be compared with the supplied password, but that approach is uncommon.

External authentication for administrative users also provides the ability to group users based on organizational information in the directory. Based on this query, you can map group names in the directory to roles on the ESA.

AD Specifics

In commercial enterprise environments, AD is the most common directory server, and as a result, the ESA has default settings and specific features for supporting AD. It’s important to note, however, that the ESA still treats an AD server as an LDAP server.

There are a couple of important things worth knowing about AD in regards to the ESA LDAP client:

• AD can answer LDAP queries on port 389, the LDAP standard, and port 3268, which queries the AD global catalog. In multidomain environments, the AD GC contains information for all domains, while the LDAP port is limited to a specific domain. For this reason, LDAP queries to AD default to 3268, and unless you have good reason, you should always search the GC. The secure (LDAPS) port equivalents are 636 and 3269, respectively.

• The mail attribute only lists the primary SMTP address for a user or group. AD uses the proxyAddresses field to store all addresses for a user, and the primary proxyAddresses attribute is also copied to the mail attribute. The default ESA queries for AD use proxyAddresses, so you shouldn’t change that unless you have good reason.

• Some attributes aren’t indexed by default. Whatever attributes in AD you plan to query, either standard or custom, make sure that they’re indexed.

• AD doesn’t expose an FQDN hostname for the server storing the user’s messages. This means it’s not possible to use the ESA LDAP routing features directly, but it’s rarely necessary because Exchange servers can route messages between themselves. I discuss a workaround for this using group queries later in this chapter.

Testing LDAP Queries

The ESA provides tools to test your queries before putting them into production. You can test queries from the WUI, right as you create the LDAP server profile.

There are two tests available in the WUI:

• Server profile tests the server attributes only: Hostname or IP, port and bind credentials. The base DN is tested, too, because the base is part of the parameters to the LDAP bind operation.

• The query tests individually test the configured queries: To test the queries, you should have already tested the server profile, because there is no sense in testing the query format and attribute availability if you haven’t tested server connectivity.

Both sets of tests run against the values you currently have in the WUI boxes on the page. There’s no need to commit or even submit the page to make changes and test.

In addition, the CLI has the command ldaptest, which can be run interactively or in batch mode:

ldaptest active_directory.accept [email protected]

When you are testing LDAP queries, after you have a working query that is returning a successful result, be sure to test thoroughly. If you have entries in more than one directory, make sure to include those in a test. Test both negative and positive results—it is easy to create a query that always matches at least one record, defeating the purpose of accept and group queries.

Take particular care with LDAP accept queries, because a negative result here means the ESA will reject some recipients, causing lost, or at least delayed, messages.

Advanced LDAP Queries

After you have one or more basic LDAP queries created and tested, you have the option of using a couple types of advanced queries:

• Per-domain queries: Performs LDAP searches based on the domain portion of the recipient or sender email address. If may have information about different domains stored in different directories, this allows the ESA to select the relevant query for each domain. This allows ESA configurations that would otherwise normally require separate listeners or appliances.

• Daisy chained queries: Chained queries allow the LDAP subsystem to make multiple queries, one after the other, to find a match. This allows you to make queries to completely different directories, or even to the same directory, but with different search base and query filter. ESA follows the chain in order, stopping as soon as it gets a match.

Both advanced query types are created by combining two or more basic query types. You must first create the basic LDAP server profiles and their relevant queries before you have the option of creating advanced queries.

The per-domain queries are, of course, used where you have separate LDAP servers or queries for different domains or where you have LDAP for some domains, but not others. As an example, consider the situation when a company acquires a smaller one. We’ll use the example of Cisco acquiring IronPort. Within a week of the acquisition, all the users with email addresses @ironport.com need to also have @cisco.com addresses, but for continuity purposes, we need to continue to accept and route both domains on the same ESA infrastructure. Figure 7-3 conceptually shows this consolidated ESA environment.

Figure 7-3. Consolidated LDAP Environment Supported by Per-Domain LDAP Queries

To support this configuration on the ESA, follow these steps:

1. Create the LDAP server profile and query for the first environment. In my example, I’m using AD-Cisco for the cisco.com addresses. The accept query is called AD-Cisco.accept. Test the query and submit the change.

2. Create the LDAP server profile and query for the second environment. Here, I’m using AD-IronPort for the ironport.com addresses. The accept query is called AD-IronPort.accept. Test and submit. If you have more than two LDAP environments, repeat this for each one.

3. On the LDAP page of the System Administration menu, click Advanced and Add Domain Assignments. Give it a name, and choose the Accept query type in the drop-down. Add each individual recipient domain and select the appropriate query. You can select the same query for multiple domains, but in my case, I have one domain for each profile. Test and submit.

Figure 7-4 shows the resulting advanced query setup.

Figure 7-4. Per-Domain LDAP Query

Figure 7-5 shows the LDAP server profile and advanced query created in the WUI. The new per-domain accept query has its own unique name and can be used anywhere an LDAP accept query can be used.

Figure 7-5. LDAP Server Profiles and Per-Domain Query

Daisy-chained queries are useful when all the information for a given domain may reside in more than one LDAP server. For example, suppose that, in addition to the regular corporate user directory, your organization also has a client database application that stores customer email addresses and makes them available via LDAP queries. Your business requires that whenever one of your user emails a client, a copy of the message is sent to an email archiving address using BCC. The concept is illustrated in Figure 7-6.

Figure 7-6. LDAP Server Profiles and Per-Domain Query

The challenge here is this: How do you identify which are client addresses? You could maintain a static list of recipients in an outgoing mail policy, but you would have quite a bit of maintenance to keep that up to date. The better approach is, of course, to use LDAP and to compose a group query to the client database server. Because you’re already using your main organizational LDAP server for group queries, I’ll show you how to use a chained query to do both. A chained query can contact multiple LDAP servers until it gets a successful answer; a query fails only when all LDAP servers respond negatively. The queries in the chain don’t have to query the same attributes or use the same variables. In this case, we can ask the client database about the recipient address and the corporate LDAP server about the sender addresses. The client database query looks like this, keying on the objectclass attribute:

(&(contactEmail={a})(objectclass={g}))

While the regular corporate directory query looks like this:

(&(memberOf={g})(proxyAddresses=smtp:{a}))

First, create and test the two LDAP server profiles with their corresponding group queries, as you did with the per-domain advanced query. Here, however, you select Group from the drop-down, and there’s no need to assign domains for the queries. Figure 7-7 shows the resulting chained query in the ESA WUI.

Figure 7-7. Chained LDAP Query

Like any LDAP query, it needs to be assigned to the listener that handles outgoing mail. Once assigned, you still need to create an incoming mail policy that supplies the group membership values for {g} in the query. Because the business use case is concerned with outgoing mail sent to client recipients, we need to create an outgoing mail policy that uses LDAP group values. Be sure to click the radio button for Recipient. The resulting policy configuration for the client member in the ESA WUI is shown in Figure 7-8.

Figure 7-8. Recipient-Based Membership for Client Policy

For our local corporate users that are not sending to clients, the normal group rules apply, perhaps for policies corresponding to archiving, DLP, or encryption. Suppose you have an encryption policy for users in the Legal department. In this case, you want to create policies that apply to the sender, not the recipient, as shown in Figure 7-9.

Figure 7-9. Sender-Based Membership for Corporate Policy

You would follow the same pattern for creating your other corporate policies. Order matters here: not the order of LDAP queries, but on the order of the policies. In the case of our client archiving example, it’s possible that a message matches more than one policy, because the sender is in the legal department and the recipient is in the client database. The policy that’s applied is the first one in the list from top to bottom. If the client policy is first, the client archiving content filter will be applied, but not any of the Legal department filters. You can resolve this with a content filter.

Let’s suppose that the business rule requires the client archiving to take precedent over all the others, so it’s the first policy in the list. However, the Legal department still needs its encryption policy. If the client policy includes no filters to account for this, messages from the Legal department to a client may not be encrypted as mandated. The solution is to test for the Legal department senders and the encryption conditions in a filter, and apply that filter to the client policy. Figure 7-10 shows the content filter that accomplishes this.

Figure 7-10. Content Filter Testing for Senders in the Legal Department

As you can see, I’m just using the same string that we used for defining the Legal department policy in the envelope sender condition, in the Matches LDAP group field. In this way, we have effectively enforced an action based on both sender and recipient via a filter condition and a policy, respectively.

These examples focused on LDAP accept and group queries, but per-domain and daisy-chained advanced queries can be used for any type of query.

Troubleshooting LDAP

Using LDAP integration on the ESA is essentially creating a dependency between two systems, and as a result, it can be difficult to troubleshoot. First, I assume that you tested your LDAP profiles and queries thoroughly and now have put the queries into production.

First, how would you know that something is going wrong? Obviously, with so many different uses for LDAP queries, the symptoms differ, but most LDAP issues start with an alert from the ESA. These alerts are classified as critical and list the affected LDAP query. The alert text looks something like this:

LDAP: query AD.group result LDAP Query Syntax Error: Invalid character 'o' at
   position 29 of query "(&([email protected])(!objectclass=group))"

In many cases, the alert text will be enough to troubleshoot the issue. I should have tested this query before I committed it; it’s missing a pair of parentheses.

If the source of the problem is not immediately obvious, enable an LDAP debug log subscription. You’ll see the full text of the error and all the connection activity (or lack thereof) to each server.

The most challenging LDAP problems to solve are the intermittent ones—the problems affecting certain users or policies, but not all of them. If you know the user address or group that’s experiencing an issue, single it out and run ldaptest or use the WUI testing features to see the results of the queries. Try clearing the LDAP cache and test again. If the issue doesn’t appear obvious, you may have to resort to packet capture of the LDAP session.

Because the ldaptest tool is fairly limited unless you know precisely what you’re looking for, you may want to look at the LDAP directory in a different way. Most LDAP servers include tools to browse the directory hierarchy, graphically, and to test filters and queries. There are also third-party LDAP browsers that give you extraordinary visibility by listing all the available attributes and their values. It’s an easy way to troubleshoot access and privilege limitations, attribute and value spelling errors, and search scope (base DN). Your LDAP browser client will have the same view as the ESA if you log in with the same credentials and base DN.

If your configuration was tested and working, but all of a sudden isn’t, ask yourself what changed. Some of the most common issues I’ve seen with ESA LDAP hiccups in production environments have been

• Account expiration or password lockout: The user account being used by the ESA has had its password expired or been locked out because of incorrect password guesses. This can happen for many reasons, both automatic and manual. The account that the ESA uses for LDAP queries should be set to never expire and never be required to change the password. Any account restriction that could suddenly result in suspension should not be applied to the ESA account.

• Query timeouts: These are fairly obvious if you have an LDAP debug log running, and are usually caused by querying unindexed attributes. You should only query indexed attributes, and these kinds of timeouts should be uncovered in testing. Query timeouts can also occur on overburdened LDAP servers.

• Firewall rule changes: All it takes is one short-sighted change in rules to shut off LDAP access, resulting in an email outage. The ESA is somewhat resilient to this and can let mail in if accept queries can’t be resolved, and by queuing messages for the other query types. Luckily, this is fairly easy to troubleshoot because the alert messages are clear and a simple ldaptest can confirm connectivity.

• Routing updates: The ESA only supports static routing, so if network topology in the DMZ changes in a way that affects the ESA’s ability to reach DNS or LDAP, you will have an outage unless you manually update the routes on the ESA.

Incoming and Outgoing Mail Policies

Aside from using LDAP to verify local email addresses, the most common use of directory integration is to group senders and recipients for the purpose of applying policies. We discussed incoming and outgoing mail policies in Chapter 4, “ESA Web User Interface,” when describing the ESA pipeline, and earlier in this chapter. We look again at the effect of policies in Chapter 8, “Security Filtering,” regarding security scanning. I hope you get the sense that mail policies are a crucial topic in ESA administration!

Group-Based Policies

In this chapter, we’re interested in the application of policies based on LDAP information. This is done through group queries. Group queries are wonderfully flexible, allowing you to test the values of any attributes or even multiple attributes. A query that returns a match means that the recipient or sender address is part of the group, and hence a member of the policy. Figure 7-11 shows an outgoing mail policy showing membership by using a value to be tested in a group query.

Figure 7-11. Group Query Used as a Membership Test for a Policy

You can certainly add multiple group query conditions, providing different values for the membership test. If you’re using a chained or per-domain query, you can use a completely separate query as membership test. Lastly, you can also combine static email address, domains, or partial domains with LDAP membership, and freely mix conditions applying to senders and recipients.

Group Matches in Filters

You can also use the content filter conditions Envelope Sender or Envelope Recipient Matches LDAP Group to enact conditional rules on particular groups of senders and recipients.

When should you use policy membership versus one of the content filter conditions? In general, you should always use policy membership for recipient rules whenever possible, because of the problem with multirecipient messages, which was described in Chapter 3, “ESA Email Pipeline.” Using policy membership allows multirecipient messages to be processed correctly, with each recipient falling into the appropriate policy via the splintering process. The content filter conditions are evaluated after this point in the pipeline.

For sender rules, applying policy is easier, because there’s only one sender to any given message, and splintering doesn’t occur. It’s still a good idea to prefer policy-based sender sorting, however, simply because it’s easier to understand and maintain.

There may be cases where you need to use both policy and content-filter based user grouping. This comes up with any business rule that requires a specific policy on messages sent by specific groups of users to specific groups of recipients, like my earlier example of Legal department senders and external client recipients. Because mail policies can only group users by either recipient address or sender address, but not both, we have to use both policies and filters to accomplish the task.

Because of the multirecipient problem, you should prefer using recipient policy membership and content filter sender tests.

Other LDAP Techniques

You’re most likely to have a need for LDAP for recipient validation, and that’s where I recommend you start with directory integration. Recipient validation provides several tangible benefits in an email environment. Beyond that, you’re likely to need LDAP group queries for applying policy to inbound and outbound messages to meet business requirements.

In some cases, some requirements aren’t quite met by the built-in query types, at least not directly. Necessity is the mother of invention, however, and the flexibility of the LDAP features on ESA mean that we can put group queries to creative use.

Using Group Queries for Routing

Sometimes, you’ll have a need to make email delivery decisions based on non-hostname information in an LDAP directory. For example, suppose you have a set of recipient email addresses that are used by applications, not people, and these messages should all be delivered to the same host. The relevant attributes in the LDAP object for these applications look like this:

dn: cn=Customer Database,cn=users,dc=example,dc=com
memberOf: cn=applications,cn=users,dc=example,dc=com
cn:Customer Database
mail: [email protected]
proxyAddresses: SMTP:[email protected]
proxyAddresses: smtp:[email protected]

Because there’s nothing in this node that looks like a hostname, and the mail attribute doesn’t look like it can route messages, we can’t use LDAP routing. We can, however, use ESA group queries and filters to do the routing. I use the familiar group query recipe, which includes these ingredients:

1. Construct a group query that includes the user address {a} to locate the user and the group variable {g} that positively identifies the group. Test and then assign the group query to the listener.

2. Create an incoming mail policy whose membership is based on the LDAP group value you want to match. This value becomes the {g} variable in your query. Use the recipient condition for the test.

3. Create a content filter that performs the action you require—in this case, Send to Alternate Destination Host—and apply it to the policy you created in Step 2.

For the routing example, the group query needs to distinguish applications from users. It looks like we can use the memberOf field to do that, so our query is:

(&(proxyAddresses=smtp:{a})(memberOf={g})

The group variable that I’ll use in the policy membership test is in this case the value of memberOf in the LDAP node. This must be the actual value stored in the attribute, which, in this case, is itself an LDAP DN:

cn=applications,cn=users,dc=example,dc=com

Don’t let this confuse you! Even though we’re passing in an LDAP DN, it has no further meaning here; to the ESA, it’s just a string of bytes that needs to be compared to the values in the directory. The ESA isn’t going to parse that string DN value or use it for a separate query. The resulting queries, at runtime, will be passed to the LDAP server, like this:

(&(proxyAddresses=smtp:[email protected])(memberOf= cn=applications,cn=users,
   dc=example,dc=com))

If that query is successful and has at least one match, the group query is true and the address is considered part of the group. The incoming mail policy, with the example LDAP group query value, is shown in Figure 7-12.

Figure 7-12. Incoming Mail Policy for Applications

The content filter is simple in this case, with no conditions and a single action, Send to Alternate Destination Host. Just take care to apply this filter only to the relevant application mail policy, because it has no condition and hence applies to all messages that it runs on. Make sure that it only runs on application mail by restricting it to that policy. The content filter is shown in Figure 7-13.

Figure 7-13. Routing Content Filter to Be Applied to Application Mail Policy

Per-Recipient Routing with AD and Exchange

AD and Exchange warrant a special mention because it does not directly expose the fully qualified hostname that stores the user mailbox account, and so the ESA cannot use AD to route messages on a per-recipient basis. Instead, the DN of the user’s mailbox server is stored in the homeMDB attributes, like this:

homeMDB: CN=Mailbox Store (E2K7),CN=First Storage Group,CN=Information
   Store,CN=E2K7,CN=Servers,CN=first administrative group,CN=Administrative
   Groups,CN=EXAMPLE,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=example,
   DC=com

Using the previous technique, we can actually route using a group query that looks like this:

(&(proxyAddresses=smtp:{a})(homeMDB: CN=Mailbox Store ({g}),CN=First
   Storage Group,CN=InformationStore,CN={g},CN=Servers,CN=first administrative
   group,CN=Administrative Groups,CN=EXAMPLE,CN=Microsoft Exchange,CN=Services,
   CN=Configuration,DC=example,DC=com))

That query evaluates to true for any recipient whose mailbox is stored on that server. Combine that with an incoming mail policy, and an incoming content filter like the one shown in Figure 7-13, and you can deliver messages directly to the right Exchange server from the ESA. Repeat this for each server in your environment.

Using Group Queries for Recipient and Sender Validation

As explained, a group query is nothing more than a simple comparison, akin to asking the directory, “Do you have any entries with an attribute with this value?” You can exploit this simple query to do recipient validation, too. The idea is simply to group recipient addresses into two groups: those that are in the directory and those that aren’t. By using a group query, we can ascertain that the email address is in the directory and use that test to apply an incoming mail policy. The addresses that aren’t fall into the default policy.

Why would you want to do this, considering that LDAP accept is purpose-built for this very task? The answer is that LDAP accept is limited to a one of two actions on a negative result: either reject or drop. Either way, if the recipient address isn’t in the directory, it’s removed from the message and if that’s the only recipient, the whole message is dropped or rejected. In order to use it, you have to be certain that your directory is authoritative. If you want to take another action, like quarantine, you can’t use LDAP accept, but you can use LDAP group queries.

Like the other group query examples, we start with the query itself. This is pretty simple, because we’re just going to use an LDAP accept query:

(proxyAddresses=smtp:{a})

We don’t need the {g} variable because we’re not actually querying for it. This also makes the incoming mail policy simple, because the LDAP group membership value can be any string. The resulting incoming mail policies are shown in Figure 7-14.

Figure 7-14. Incoming Mail Policies to Support Recipient Validation via Group Query

The accepted mail policy will be followed for any recipient that appears in the directory, and the default mail policy will be followed for any recipient not in the directory. To quarantine or redirect the recipients that don’t resolve, create a content filter and apply it to the default. If you quarantine, the messages will be recoverable.

If you’re using this approach as a crutch to start doing recipient validation, without taking hard action, be aware that performance is better when you actually reject the recipients, because you’re not actually accepting the messages.

Summary

Almost every organization that uses the ESA can benefit from its LDAP integration features for security, routing, and policy. We covered how organizations can benefit, what the directory requirements are, and how to configure, test, and deploy LDAP queries. We explored the relationship between LDAP and incoming and outgoing mail policies, with examples of group queries for applying appropriate policies to individuals and groups of users.

We’ll use incoming and outgoing mail policies in the next chapter where configuration of the security engines are covered: anti-spam, anti-virus, and outbreak filters.