Email is a fairly essential service that needs special attention due to the large volume of unwanted messages, or spam. The volume of unsolicited commercial messages was already a rather painful problem when malware makers discovered that email-borne worms would work and started using email to spread their payload. During the early 2000s, the combined volume of spam and email-borne malware had increased to the point where running an SMTP mail service without some sort of spam countermeasures had become almost unthinkable.
Spam-fighting measures are almost as old as the spam problem itself. The early efforts focused on analysis of the messages’ contents (known as content filtering) and to some extent on interpretation of the messages’ rather trivially forgeable headers such as the purported sender address (From:) or the store and forward paths of intermediate deliveries recorded in the Received: headers.
When the OpenBSD team designed their spam-fighting solution spamd, first introduced with OpenBSD 3.3 in 2003, they instead focused on the network level and the immediate communication partner in the SMTP conversations, combined with any available information about hosts that tried to deliver messages. The developers set out to create a small, simple, and secure program. The early implementation was based almost entirely on creative use of PF tables combined with data from trusted external sources.
Note
In addition to the OpenBSD spam-deferral daemon, the content-filtering-based anti-spam package SpamAssassin (http://spamassassin.apache.org/) also features a program called spamd. Both programs are designed to help fight spam, but they take very different approaches to the underlying problem and do not interoperate directly. However, when both programs are correctly configured and running, they complement each other well.
The original spamd design is based on the observation that spammers send a lot of mail, and the likelihood that you are the first person receiving a particular message is incredibly small. In addition, spam is sent via a few spammer-friendly networks and numerous hijacked machines. Both the individual messages and the machines that send them will be reported to blacklist maintainers quickly, and the blacklist data consisting of known spam senders’ IP addresses forms the basis for spamd’s processing.
When dealing with blacklisted hosts, spamd employs a method called tarpitting. When the daemon receives an SMTP connection, it presents its banner and immediately switches to a mode where it answers SMTP traffic at the rate of 1 byte per second, using a tiny selection of SMTP commands designed to make sure that mail is never delivered, but rather rejected back into the sender’s queue once the message headers have been transferred. The intention is to waste as much time as possible on the sending end, while costing the receiver pretty much nothing. This specific tarpitting implementation with 1-byte SMTP replies is often referred to as stuttering. Blacklist-based tarpitting with stuttering was the default mode for spamd up to and including OpenBSD 4.0.
Note
On FreeBSD and NetBSD, spamd is not part of the base system, but available through ports and packages as mail/spamd. If you are running PF on FreeBSD 5.n or NetBSD 4.0 or newer, install the port or package now.
To set up spamd to run in traditional, blacklisting-only mode, you first put a special-purpose table and a matching redirection in pf.conf, and then turn your attention to spamd’s own spamd.conf. spamd then hooks into the PF rule set via the table and the redirection.
The following are the pf.conf lines for this configuration:
table <spamd> persist
pass in on $ext_if inet proto tcp from <spamd> to
{ $ext_if, $localnet } port smtp rdr-to 127.0.0.1 port 8025And here is the pre-OpenBSD 4.7 syntax:
table <spamd> persist
rdr pass on $ext_if inet proto tcp from <spamd> to
{ $ext_if, $localnet } port smtp -> 127.0.0.1 port 8025The table, <spamd>, is there to store the IP addresses you import from trusted blacklist sources. The redirection takes care of all SMTP attempts from hosts that are already in the blacklist. spamd listens on port 8025 and responds s-l-o-w-l-y (1 byte per second) to all SMTP connections it receives as a result of the redirection. Later on in the rule set, you would have a rule that makes sure legitimate SMTP traffic passes to the mail server.
spamd.conf is where you specify the sources of your blacklist data and any exceptions or local overrides you want.
Note
On OpenBSD 4.0 and earlier (and by extension, ports based on versions prior to OpenBSD 4.1), spamd.conf was in /etc. Beginning with OpenBSD 4.1, spamd.conf is found in /etc/mail instead. The FreeBSD port installs a sample configuration in /usr/local/etc/spamd/spamd.conf.sample.
Near the beginning of spamd.conf, you will notice a line without a # comment sign that looks like all:. This line specifies the blacklists you will use. Here is an example:
all: :uatraps:whitelist:
Add all blacklists that you want to use below the all: line, separating each with a colon (:). To use whitelists to subtract addresses from your blacklist, add the name of the whitelist immediately after the name of each blacklist, as in :blacklist:whitelist:
Next is the blacklist definition:
uatraps:
:black:
:msg="SPAM. Your address %A has sent spam within the last 24 hours":
:method=http:
:file=www.openbsd.org/spamd/traplist.gzFollowing the name (uatraps), the first data field specifies the list type—in this case black. The msg field contains the message to be displayed to blacklisted senders during the SMTP dialogue. The method field specifies how spamd-setup fetches the list data—in this case via HTTP. Other possibilities include fetching via FTP (ftp), from a file in a mounted file system (file), or via execution of an external program (exec). Finally, the file field specifies the name of the file spamd expects to receive.
The definition of a whitelist follows much the same pattern, but omits the message parameter:
whitelist:
:white:
:method=file:
:file=/etc/mail/whitelist.txtNote
The suggested blacklists in the default spamd.conf could exclude large blocks of the Internet, including several address ranges that claim to cover entire countries. If your site expects to exchange legitimate mail with any of the countries in question, those lists may not be optimal for your setup. Other popular lists have been known to list entire /16 ranges as spam sources, and it is well worth reviewing the details of the list’s maintenance policy before putting a blacklist into production.
Put the lines for spamd and the startup parameters you want in your /etc/rc.conf.local on OpenBSD, or in /etc/rc.conf on FreeBSD or NetBSD. Here is an example:
spamd_flags="-v -b" # for normal use: "" and see spamd-setup(8)
Here, we enable spamd and set it to run in blacklisting mode with the -b flag. In addition, the -v flag enables verbose logging, which is useful for keeping track of spamd’s activity for debugging purposes.
On FreeBSD, the /etc/rc.conf settings that control spamd’s behavior are obspamd_enable, which should be set to "YES" in order to enable spamd, and obspamd_flags, where you fill in any command-line options for spamd:
obspamd_enable="YES" obspamd_flags="-v -b" # for normal use: "" and see spamd-setup(8)
Note
To have spamd run in pure blacklist mode on OpenBSD 4.1 or newer, you can achieve the same effect by setting the spamd_black variable to "YES", and then restarting spamd.
Once you’ve finished editing the setup, start spamd with the options you want, and complete the configuration with spamd-setup. Finally, create a cron job that calls spamd-setup to update the blacklist at reasonable intervals. In pure blacklist mode, you can view and manipulate the table contents using pfctl table commands.
By default, spamd logs to your general system logs. To send the spamd log messages to a separate log file, add an entry like this to syslog.conf:
!!spamd daemon.err;daemon.warn;daemon.info;daemon.debug /var/log/spamd
Once you’re satisfied that spamd is running and does what it is supposed to do, you will probably want to add the spamd log file to your log rotations, too. After you’ve run spamd-setup and the tables are filled, you can view the table contents using pfctl.
Note
In the sample pf.conf fragment at the beginning of this section, the redirection (rdr-to) rule is also a pass rule. If your opted for a match rule instead (or if you are using an older PF version and chose to write a rdr rule that does not include a pass part), be sure to set up a pass rule to let traffic through to your redirection. You may also need to set up rules to let legitimate email through. However, if you are already running an email service on your network, you can probably go on using your old SMTP pass rules.
Given a set of reliable and well-maintained blacklists, spamd in pure blacklisting mode does a good job of reducing spam. However, with pure blacklisting, you catch traffic only from hosts that have already tried to deliver spam elsewhere, and you put your trust in external data sources to determine which hosts deserve to be tarpitted. For a setup that provides more immediate response to network-level behavior and offers some real gains in spam prevention, consider greylisting, which is a crucial part of how the modern spamd works.
Greylisting consists mainly of interpreting the current SMTP standards and adding a little white lie to make life easier.
Spammers tend to use other people’s equipment to send their messages, and the software they install without the legal owner’s permission needs to be relatively lightweight in order to run undetected. Unlike legitimate mail
senders, spammers typically do not consider any individual message they send to be important. Taken together, this means that typical spam and malware sender software are not set up to interpret SMTP status codes correctly. This is a fact that we can use to our advantage, as Evan Harris proposed in a paper titled “The Next Step in the Spam Control War: Greylisting,” published in 2003.[31]
As Harris noted, when a compromised machine is used to send spam, the sender application tends to try delivery only once, without checking for any results or return codes. Real SMTP implementations interpret SMTP return codes and act on them, and real mail servers will retry if the initial attempt fails with any kind of temporary error.
In his paper, Harris outlined a practical approach:
On first SMTP contact from a previously unknown communication partner, do not receive email on the first delivery attempt, but instead respond with a status code that indicates a temporary local problem, and store the sender IP address for future reference.
If the sender retries immediately, reply as before with the temporary failure status code.
If the sender retries after a set minimum amount of time (1 hour, for example) but not more than a maximum waiting period (4 hours, for example), accept the message for delivery and record the sender IP address in your whitelist.
This is the essence of greylisting. And fortunately, you can set up and maintain a greylisting spamd on your PF-equipped gateway.
OpenBSD’s spamd acquired its ability to greylist in OpenBSD 3.5. Beginning with OpenBSD 4.1, spamd runs in greylisting mode by default.
In the default greylisting mode, the spamd table used for blacklisting, as described in the previous section, becomes superfluous. You can still use blacklists, but spamd will use a combination of private data structures for blacklist data and the spamdb database to store greylisting-related data. A typical set of rules for spamd in default mode looks like this:
table <spamd-white> persist
table <nospamd> persist file "/etc/mail/nospamd"
pass in log on egress proto tcp to port smtp
rdr-to 127.0.0.1 port spamd
pass in log on egress proto tcp from <nospamd> to port smtp
pass in log on egress proto tcp from <spamd-white> to port smtp
pass out log on egress proto tcp to port smtpThis includes the necessary pass rules to let legitimate email flow to the intended destinations from your own network. The <spamd-white> table is the whitelist, maintained by spamd. The hosts in the <spamd-white> table have passed the greylisting hurdle, and mail from these machines is allowed to pass to the real mail servers or their content-filtering front ends. In addition, the nospamd table is there for you to load addresses of hosts that you do not want to expose to spamd processing, and the matching pass rule makes sure SMTP traffic from those hosts passes.
In your network, you may want to tighten those rules to pass SMTP traffic only to and from hosts that are allowed to send and receive email via SMTP. We will get back to the nospamd table in Handling Sites That Do Not Play Well with Greylisting in Detecting Out-of-Order MX Use.
The following are the equivalent rules in pre-OpenBSD 4.7 syntax:
table <spamd-white> persist
table <nospamd> persist file "/etc/mail/nospamd"
rdr pass in log on egress proto tcp to port smtp
-> 127.0.0.1 port spamd
pass in log on egress proto tcp from <nospamd> to port smtp
pass in log on egress proto tcp from <spamd-white> to port smtp
pass out log on egress proto tcp to port smtpOn FreeBSD, in order to use spamd in greylisting mode, you need a file descriptor filesystem (see man 5 fdescfs) mounted at /dev/fd/. To implement this, add the following line to /etc/fstab, and make sure the fdescfs code is in your kernel, either compiled in or by loading the module via the appropriate kldload command.
fdescfs /dev/fd fdescfs rw 0 0
To begin configuring spamd, place the lines for spamd and the startup parameters you want in /etc/rc.conf.local. Here is an example:
spamd_flags="-v -G 2:4:864" # for normal use: "" and see spamd-setup(8)
On FreeBSD, the equivalent line should go in /etc/rc.conf:
obspamd_flags="-v -G 2:4:864" # for normal use: "" and see spamd-setup(8)
You can fine-tune several of the greylisting-related parameters via spamd command-line parameters trailing the -G option.
This colon-separated list 2:4:864 represents the values passtime, greyexp, and whiteexp:
passtimedenotes the minimum number of minutesspamdconsiders a reasonable time before retry. The default is 25 minutes, but here we’ve reduced it to 2 minutes.greyexpis the number of hours an entry stays in the greylisted state before it is removed from the database.whiteexpdetermines how long a whitelisted entry is kept, in hours. The default values forgreyexpandwhiteexpare 4 hours and 864 hours ( just over one month), respectively.
Users and administrators at sites that implement greylisting tend to agree that greylisting gets rid of most of their spam, with a significant drop in the load on any content filtering they have in place for their mail. We will start by looking at what spamd’s greylisting looks like according to log files, and then return with some data.
If you start spamd with the -v command-line option for verbose logging, your logs will include a few more items of information in addition to IP addresses. With verbose logging, a typical log excerpt looks like this:
Oct 2 19:53:21 delilah spamd[26905]: 65.210.185.131: connected (1/1), lists: spews1 Oct 2 19:55:04 delilah spamd[26905]: 83.23.213.115: connected (2/1) Oct 2 19:55:05 delilah spamd[26905]: (GREY) 83.23.213.115: <gilbert@keyholes. net> -> <[email protected]> Oct 2 19:55:05 delilah spamd[26905]: 83.23.213.115: disconnected after 0 seconds. Oct 2 19:55:05 delilah spamd[26905]: 83.23.213.115: connected (2/1) Oct 2 19:55:06 delilah spamd[26905]: (GREY) 83.23.213.115: <gilbert@keyholes .net> -> <[email protected]> Oct 2 19:55:06 delilah spamd[26905]: 83.23.213.115: disconnected after 1 seconds. Oct 2 19:57:07 delilah spamd[26905]: (BLACK) 65.210.185.131: <bounce-3C7E4 [email protected]> -> <[email protected]> Oct 2 19:58:50 delilah spamd[26905]: 65.210.185.131: From: Auto lnsurance Savings <[email protected]> Oct 2 19:58:50 delilah spamd[26905]: 65.210.185.131: Subject: Start SAVlNG M0NEY on Auto lnsurance Oct 2 19:58:50 delilah spamd[26905]: 65.210.185.131: To: [email protected] Oct 2 20:00:05 delilah spamd[26905]: 65.210.185.131: disconnected after 404 seconds. lists: spews1 Oct 2 20:03:48 delilah spamd[26905]: 222.240.6.118: connected (1/0) Oct 2 20:03:48 delilah spamd[26905]: 222.240.6.118: disconnected after 0 seconds. Oct 2 20:06:51 delilah spamd[26905]: 24.71.110.10: connected (1/1), lists: spews1 Oct 2 20:07:00 delilah spamd[26905]: 221.196.37.249: connected (2/1) Oct 2 20:07:00 delilah spamd[26905]: 221.196.37.249: disconnected after 0 seconds. Oct 2 20:07:12 delilah spamd[26905]: 24.71.110.10: disconnected after 21 seconds. lists: spews1
The first line is the beginning of a connection from a machine in the spews1 blacklist. The next six lines show the complete records of two connection attempts from another machine, which each time connects as the second active connection. This second machine is not yet in any blacklist, so it is greylisted. Note the rather curious delivery address ([email protected]) in the message that the greylisted machine tries to deliver. There is a useful trick that we’ll look at in Greytrapping in Tracking Your Real Mail Connections: spamlogd. The (GREY) and (BLACK) before the addresses indicate greylisting or blacklisting status. Then there is more activity from the blacklisted host, and a little later we see that after 404 seconds (or 6 minutes, 44 seconds), the blacklisted host gives up without completing the delivery.
The remaining lines show a few very short connections, including one from a machine already in a blacklist. This time, though, the machine disconnects too quickly to see any (BLACK) flag at the beginning of the SMTP dialogue, but we see a reference to the list name (spews1) at the end.
Roughly 400 seconds is about the amount of time that naïve, blacklisted spammers hang around (according to data from various sites), and about the time it takes (at the rate of 1 byte per second) to complete the EHLO ... dialogue until spamd rejects the message. However, while peeking at the logs, you are likely to find some spammers that hang around significantly longer. For example, in the data from our office gateway, one log entry stood out:
Dec 11 23:57:24 delilah spamd[32048]: 69.6.40.26: connected (1/1), lists: spamhaus spews1 spews2 Dec 12 00:30:08 delilah spamd[32048]: 69.6.40.26: disconnected after 1964 seconds. lists: spamhaus spews1 spews2
This particular machine was already on several blacklists when it made 13 attempts at delivery from December 9 through December 12. The last attempt lasted 32 minutes, 44 seconds, without completing the delivery. Relatively intelligent spam senders drop the connection during the first few seconds, like the ones in the first log fragment. Others give up after around 400 seconds. A few hang on for hours. (The most extreme case we have recorded hung on for 42,673 seconds, which is almost 12 hours.)
Behind the scenes, rarely mentioned and barely documented, is one of spamd’s most important helper programs: the spamlogd whitelist updater. As the name suggests, spamlogd works quietly in the background, logging connections to and from your mail servers to keep your whitelist updated. The idea is to make sure that valid mail sent between hosts you communicate with regularly goes through with a minimum of fuss.
Note
If you have followed the discussion up to this point, spamlogd has probably been started automatically already. However, if your initial spamd configuration did not include greylisting, spamlogd may not have been started, and you may experience strange symptoms like the greylist and whitelist not being updated properly. Restarting spamd after you have enabled greylisting should ensure that spamlogd is loaded and available, too.
In order to perform its job properly, spamlogd needs you to log SMTP connections to and from your mail servers, just as we did in the sample rule sets in Chapter 5:
emailserver = "192.0.2.225" pass log proto tcp to $emailserver port $email pass log proto tcp from $emailserver to port smtp
On OpenBSD 4.1 and higher (and equivalents), you can create several pflog interfaces and specify where rules should be logged. Here’s how to separate the data spamlogd needs to read from the rest of your PF logs:
Create a separate
pflog1interface usingifconfig pflog1 create, or create a hostname.pflog1 file with just the lineup.Change the rules to the following:
pass log (to pflog1) proto tcp to $emailserver port $email pass log (to pflog1) proto tcp from $emailserver to port smtp
Add
-l pflog1tospamlogd’s startup parameters.
This separates the spamd-related logging from the rest. (See Chapter 8 for more about logging.)
With the preceding rules in place, spamlogd will add the IP addresses that receive email you send to the whitelist. This is not an ironclad guarantee that the reply will pass immediately, but in most configurations, it helps speed things significantly.
We know that spam senders rarely use a fully compliant SMTP implementation to send their messages, which is why greylisting works. We also know that spammers rarely check that the addresses they feed to their hijacked machines are actually deliverable. Combine these facts, and you see that if a greylisted machine tries to send a message to an invalid address in your domain, there is a good chance that the message is spam or malware.
This realization led to the next evolutionary step in spamd development—a technique dubbed greytrapping. When a greylisted host tries to deliver mail to a known bad address in our domains, the host is added to a locally maintained blacklist called spamd-greytrap. Members of the spamd-greytrap list are treated to the same 1-byte-per-second tarpitting as members of other blacklists.
Greytrapping as implemented in spamd is simple and elegant. The main thing you need as a starting point is spamd running in greylisting mode. The other crucial component is a list of addresses in domains your servers handle email for, but only ones that you are sure will never receive legitimate email. The number of addresses in your list is unimportant, but there must be at least one, and the upper limit is mainly defined by how many addresses you wish to add.
Next, you use spamdb to feed your list to the greytrapping feature, and sit back and watch. First, a sender tries to send email to an address on your greytrap list and is simply greylisted, as with any sender you have not exchanged email with before. If the same machine tries again, either to the same, invalid address or another address on your greytrapping list, the greytrap is triggered, and the offender is put into spamd-greytrap for 24 hours. For the next 24 hours, any SMTP traffic from the greytrapped host will be stuttered, with 1-byte-at-a-time replies.
That 24-hour period is short enough to not cause serious disruption of legitimate traffic, since real SMTP implementations will keep trying to deliver for at least a few days. Experience from large-scale implementations of the technique shows that it rarely produces false positives. Machines that continue spamming after 24 hours will make it back to the tarpit soon enough.
To set up your traplist, use spamdb’s -T option. In my case, the strange address[34] I mentioned earlier in Greylisting in Practice in Greylisting in Practice was a natural candidate for inclusion:
$ sudo spamdb -T -a [email protected]The command I actually entered was $ sudo spamdb -T -a "<[email protected]>". In OpenBSD 4.1 and newer, spamdb does not require the angle brackets or quotes, but it will accept them.
Add as many addresses as you like. I tend to find new additions for my local list of spamtrap addresses by looking in the greylist and mail server logs for failed attempts to deliver delivery failure reports to nonexistent addresses in my domain (yes, it really is as crazy as it sounds).
Warning
Make sure that the addresses you add to your spamtrap lists are invalid and will stay invalid. There is nothing quite like the embarrassment of discovering that you made a valid address into a spamtrap, however temporarily.
The following log fragment shows how a spam-sending machine is greylisted at first contact, and then comes back and clumsily tries to deliver messages to the curious address I added to my traplist, only to end up in the spamd-greytrap blacklist after a few minutes. We know what it will be doing for the next 20-odd hours.
Nov 6 09:50:25 delilah spamd[23576]: 210.214.12.57: connected (1/0) Nov 6 09:50:32 delilah spamd[23576]: 210.214.12.57: connected (2/0) Nov 6 09:50:40 delilah spamd[23576]: (GREY) 210.214.12.57: <[email protected]> -> <[email protected]> Nov 6 09:50:40 delilah spamd[23576]: 210.214.12.57: disconnected after 15 seconds. Nov 6 09:50:42 delilah spamd[23576]: 210.214.12.57: connected (2/0) Nov 6 09:50:45 delilah spamd[23576]: (GREY) 210.214.12.57: <bounce-3C7E40A [email protected]> -> <[email protected]> Nov 6 09:50:45 delilah spamd[23576]: 210.214.12.57: disconnected after 13 seconds. Nov 6 09:50:50 delilah spamd[23576]: 210.214.12.57: connected (2/0) Nov 6 09:51:00 delilah spamd[23576]: (GREY) 210.214.12.57: <[email protected]> -> <[email protected]> Nov 6 09:51:00 delilah spamd[23576]: 210.214.12.57: disconnected after 18 seconds. Nov 6 09:51:02 delilah spamd[23576]: 210.214.12.57: connected (2/0) Nov 6 09:51:02 delilah spamd[23576]: 210.214.12.57: disconnected after 12 seconds. Nov 6 09:51:02 delilah spamd[23576]: 210.214.12.57: connected (2/0) Nov 6 09:51:18 delilah spamd[23576]: (GREY) 210.214.12.57: <[email protected]> -> <[email protected]> Nov 6 09:51:18 delilah spamd[23576]: 210.214.12.57: disconnected after 16 seconds. Nov 6 09:51:18 delilah spamd[23576]: (GREY) 210.214.12.57: <bounce-3C7E40A [email protected]> -> <[email protected]> Nov 6 09:51:18 delilah spamd[23576]: 210.214.12.57: disconnected after 16 seconds. Nov 6 09:51:20 delilah spamd[23576]: 210.214.12.57: connected (1/1), lists: spamd-greytrap Nov 6 09:51:23 delilah spamd[23576]: 210.214.12.57: connected (2/2), lists: spamd-greytrap Nov 6 09:55:33 delilah spamd[23576]: (BLACK) 210.214.12.57: <[email protected]> -> <[email protected]> Nov 6 09:55:34 delilah spamd[23576]: (BLACK) 210.214.12.57: <bounce-3C7E40 [email protected]> -> <[email protected]>
As a side note, it looks like even though the spammer moved to send from a different machine, both the From: and To: addresses stayed the same. The fact that he is still trying to send to an address that has never been deliverable is a strong indicator that this spammer does not check his lists frequently.
There may be times when you need to view or change the contents of blacklists, whitelists, and greylists. These records are located in the /var/db/spamdb database, and an administrator’s main interface to managing those lists is spamdb.
Early versions of spamdb simply offered options to add whitelist entries to the database or update existing ones (spamdb -a nn.mm.nn.mm). You could delete whitelist entries (spamdb -d nn.mm.nn.mm) to compensate for shortcomings in either the blacklists used or the effects of the greylisting algorithms. Recent versions of spamdb offer some interesting features to support greytrapping.
If you run spamdb without any parameters, it lists the contents of your spamdb database, and it lets you add or delete both spamtrap addresses and traplist entries. You can also add whitelist entries on the fly.
If you want to add a host to your whitelist without adding it to your permanent nospamd file and reloading your rule set or the table, you could do it from the command line instead, like this:
$ sudo spamdb -a 213.187.179.198
If a spam sender managed to get a message delivered despite your best efforts, you could correct the situation by adding the spam sender to the spamd-greytrap list like this:
$ sudo spamdb -a -t 192.168.2.128
Adding a new trap address is just as simple:
$ sudo spamdb -a -T [email protected]
If you want to reverse either of these decisions, you would simply substitute -d for the -a option in both these commands.
Beginning with OpenBSD 4.1, spamd can keep greylisting databases in sync across any number of cooperating greylisting gateways. The implementation is via a set of spamd command-line options:
The
-Yoption specifies a sync target—that is, the IP address(es) of otherspamdrunning gateways you want to inform of updates to your greylisting information.On the receiving end, the
-yoption specifies a sync listener, which is the address or interface where thisspamdinstance is prepared to receive greylisting updates from other hosts.
For example, our main spamd gateway mainoffice-gw.example.com might have the following options added to its startup command line to establish a sync target and sync listener, respectively:
-Y minorbranch-gw.example.com -y mainoffice-gw.example.com
Conversely, minorbranch-gw.example.com at the branch office would have the hostnames reversed:
-Y mainoffice-gw.example.com -y minorbranch-gw.example.com
The spamd daemon also supports shared-secret authentication between the synchronization partners. Specifically, if you create the file /etc/mail/spamd.key and distribute copies of it to all synchronization partners, it will be used to calculate the necessary checksums for authentication. The spamd.key file itself can be any kind of data, such as random data harvested from /dev/arandom as suggested by the spamd man page.
Note
In situations where direct synchronization of spamd-related data is not practical, or if you simply want to share your spamd-greytrap with others, exporting the contents of your list of locally trapped spam senders to a text file may be desirable. The list format spamd-setup expects is one address per line, optionally with comment lines starting with one or more # characters. Exporting your list of currently trapped addresses in a usable format can be as simple as putting together a one-liner with spamdb, grep, and a little imagination.
OpenBSD 4.1 gave spamd the ability to detect out-of-order MX use. Contacting a secondary mail exchanger first instead of trying the main one is a fairly well-known spammer trick, and one that runs contrary to the behavior we expect from ordinary email transfer agents. In other words, if someone tries the email exchangers in the wrong order, we can be pretty sure that they’re trying to deliver spam.
For our example.com domain with main mail server 192.0.2.225 and backup 192.0.2.224, adding -M 192.0.2.224 to spamd’s startup options would mean that any host that tries to contact 192.0.2.224 via SMTP before contacting the main mail server at 192.0.2.225 will be added to the local spamd-greytrap list for the next 24 hours.
Unfortunately, there are situations where you will need to compensate for the peculiarities of other sites’ email setups.
The first email message sent from any site that has not contacted you for as long as the greylister keeps its data around will be delayed for some random amount of time, which depends mainly on the sender’s retry interval. There are times when even a minimal delay is undesirable. If, for example, you have some infrequent customers who always demand your immediate and urgent attention to their business when they do contact you, an initial delivery delay of what could be up to several hours may not be optimal. In addition, you are bound to encounter misconfigured mail servers that either do not retry at all or retry too quickly, perhaps stopping delivery retries after just one attempt.
Also, some sites are large enough to have several outgoing SMTP servers, and they do not play well with greylisting since they are not guaranteed to retry delivery of any given message from the same IP address used with the prior delivery attempt. Even though those sites comply with the retry requirements, it’s obvious that this is one of the few remaining downsides of greylisting.
One way to compensate for such situations is to define a table for a local whitelist to be fed from a file in case of reboots. To make sure SMTP traffic from the addresses in the table is not fed to spamd, add a pass rule to allow the traffic to pass:
table <nospamd> persist file "/etc/mail/nospamd" pass in log on egress proto tcp from <nospamd> to port smtp
In pre-OpenBSD 4.7 syntax, add a no rdr rule at the top of your redirection block and a matching pass rule to let SMTP traffic from the hosts in your nospamd table through, as shown here:
no rdr proto tcp from <nospamd> to $mailservers port smtp pass in log on egress proto tcp from <nospamd> to port smtp
Once you have made these changes to your rule set, enter the addresses you need to protect from redirection into the /etc/mail/nospamd file, and then reload your rule set using pfctl -f /etc/pf.conf. You can then use all the expected table tricks on the <nospamd> table, including replacing its content after editing the nospamd file. In fact, this approach is strongly hinted at in both man pages and sample configuration files distributed with recent versions of spamd.
At least some sites with many outgoing SMTP servers publish information about which hosts are allowed to send email for their domain via Sender Policy Framework (SPF) records as part of the domain’s DNS information.[35] To retrieve the SPF records for our example.com domain, use the host command’s -ttxt option like this:
$ host -ttxt example.comThe command would produce an answer roughly like this:
example.com descriptive text "v=spf1 ip4:192.0.2.128/25 -all"
Here, the text in quotes is the example.com domain’s SPF record. If you want email from example.com to arrive quickly, and you trust the people there not to send or relay spam, choose the address range from the SPF record, add it to your nospamd file, and reload the <nospamd> table contents from the updated file.
[31] The original Harris paper and a number of other useful articles and resources can be found at http://www.greylisting.org/.
[32] The relevant parts of RFC 5321 are identical to the corresponding parts of RFC 2821, which is obsolete. Some of us were more than a little disappointed that the IETF did not clarify these chunks of the text, now moving forward on the standards track. My reaction (actually it's quite a rant) is at http://bsdly.blogspot.com/2008/10/ietf-failed-to-account-for-greylisting.html.
[33] The relevant RFCs are mainly RFC 1123 and RFC 5321 (which obsoleted the earlier RFC 2821). Remember that temporary rejection is an SMTP fault-tolerance feature.
[34] Of course, this address is totally bogus. It looks like the kind of message ID the GNUS email and news client generates, and it was probably lifted from a news spool or some unfortunate malware victim’s mailbox.
[35] SPF records are stored in DNS zones as TXT records. See http://www.openspf.org/ for details.