In this chapter, we will present the best solutions that we can implement inside the Zimbra environment; that's why we call them internal solutions.
The way Zimbra handles spam is not very effective and it doesn't resolve most users' needs. In fact, it relies on SpamAssassin to filter spam/ham mails, and this heuristics-based method, in my opinion, is very old and inadequate, though it does catch a lot of spam. Besides, rules should be added/updated on a regular basis in order to face a spammer's new methods.
On the other hand, DSPAM has another approach to filter spam: a statistical method that "learns" better how to detect spam. This learning is simply done when you repeatedly show DSPAM examples of spam by marking them as junk. With time, DSPAM learns patterns (words and combinations of words) that exist mostly in your spam and ham mails. In terms of that knowledge, it can define what you should regard as spam and what you shouldn't. This is what keeps DSPAM up to date with the latest trends in spam. Besides, the language in which DSPAM is written is C, and compared to Perl (the language in which SpamAssassin is written), C is much faster.
Zimbra disabled DSPAM quite some time ago because of stability issues, and in our case the mileage varies; therefore, we have to enable DSPAM.
To enable DSPAM inside Zimbra, we need to act as the Zimbra user by running the following command:
abdelmonam@mail:~$ sudo -i [sudo] password for abdelmonam: root@mail:~# su - zimbra zimbra@mail:~$
Now we need to go to the amavisd.conf.in directory first:
zimbra@mail:~$ cd /opt/zimbra/conf/
You will see the dspam config by doing this:
zimbra@mail:~$ more amavisd.conf.in | grep dspam %%uncomment LOCAL:amavis_dspam_enabled%%$dspam = ''/opt/zimbra/dspam/bin/dspam'';
zimbra@mail:~$ more amavisd.conf | grep dspam #$dspam = ''/opt/zimbra/dspam/bin/dspam'';
Now you need to enable DSPAM using the command:
$ zmlocalconfig -e amavis_dspam_enabled=TRUE
The zmmtaconfig command will automatically reload Amavis within 2 minutes, or you can run the following command for an immediate effect:
$ zmamavisdctl reload
If you check the different parameters in the amavisd.conf file, you will see it is uncommented:
$ more amavisd.conf | grep dspam $dspam = ''/opt/zimbra/dspam/bin/dspam'';
With each update of Zimbra comes the latest release of the ClamAV engine.
But sometimes you may need the latest package of ClamAV without waiting for Zimbra updates, especially for urgent security purposes, such as alerts of detection of critical bugs in the ClamAV release, so that the ClamAV community can correct it in the following release.
In this section, we will show you how to do that in a safe way.
Note
The update of the ClamAV virus definitions base happens automatically every two hours by default, but you can configure it using the attribute: zimbraVirusDefinitionsUpdateFrequency
You can perform all the following actions either as a Zimbra user or any other user having sudo rights. If you do it all as a user other than a Zimbra user, remember to modify the ownership of the resulting clamav-X.Y.Z folder in /opt/zimbra to zimbra:zimbra.
If upgrading from anything below 0.90.x, please refer to: http://wiki.zimbra.com/wiki/ClamAV_-_Updating_from_versions_lower_than_0.90.0
To avoid tricky situations, we should guarantee a rollback plan by preparing a backup for the existing release. To do that, all you need is to run the following command, making sure to replace the release version with yours:
cp -a /opt/zimbra/clamav-X.Y.Z ~/clamav-X.Y.Z.backup
This is the main update step; we should start by getting the latest ClamAV source code from http://www.clamav.net/download (at the time of the writing of this book, the current stable version is 0.97.8).
Extract it to where you want; just don't forget our note about ownership if you are not logged in as a Zimbra user.
Assuming that the new ClamAV version is in the /home/abdelmonam directory:
tar -xvf clamav-0.97.8.tar.gz cd clamav-0.97.8
Next, run configure inside the clamav extract as following:
./configure --prefix=/opt/zimbra/clamav-0.97.8 --with-user=zimbra --with-group=zimbra
If this step is completed successfully, I mean the output of your configure doesn't contain any errors, we can move to the next step: running the make command. However, if you get errors in your configure output, you should fix them before continuing. After fixing the errors, make sure to run configure again and then would go well.
Now run:
make
If the make output goes well, we can move to the next step; else, we should fix the errors in the make output. Note that we should switch to the root user to be able to run the following commands; to do that under Ubuntu, you need to run this command:
sudo -i
Now run:
make check
After that, run:
make install
If you have no errors, we can say that you now have the new version installed into the /opt/zimbra/clamav-0.97.8 directory.
Before copying the clamav config file from the old release to the new one, you can compare your old clamd.conf and freshclam.conf files with the new ones:
cd /opt/zimbra/clamav-0.97.8/etc/ diff clamd.conf ../../clamav/etc/clamd.conf diff freshclam.conf ../../clamav/etc/freshclam.conf
Then, you can copy these config files from the previous version to the new version directory:
mv clamd.conf clamd.conf.org mv freshclam.conf freshclam.conf.org cd /opt/zimbra/conf cp clamd.conf /opt/zimbra/clamav-0.97.8/etc/ cp freshclam.conf /opt/zimbra/clamav-0.97.8/etc/
As a Zimbra user, run the following command to stop Zimbra:
zmcontrol stop
Now, delete the symbolic link and re-link it to the new install. As a root user, use the following command:
cd /opt/zimbra ls -la | grep clamav
(You should see clamav -> /path/to/previous clamAV)
If so:
rm -rf clamav
If you want to keep the old install and link around, you can easily exit; just use the following command:
mv clamav clamav.old
Then:
ln -s /opt/zimbra/clamav-0.97.8 /opt/zimbra/clamav
Create the directory /opt/zimbra/clamav/db:
mkdir /opt/zimbra/clamav/db
Now, you should make sure the Zimbra user owns all of clamav:
chown -R zimbra:zimbra /opt/zimbra/clamav-0.97.8
The Zimbra user also needs access to the freshclam.conf file:
chmod a+r /opt/zimbra/clamav/etc/freshclam.conf
Next, we need to update the virus database:
abdelmonam@mail:~$ sudo -i [sudo] password for abdelmonam: root@mail:~# su - zimbra zimbra@mail:~$
Run:
/opt/zimbra/clamav/bin/freshclam
If you get any warning, just run the command again to confirm that everything was successfully updated.
To start Zimbra, run:
zmcontrol start
Note that you may not need to stop Zimbra during this update. If you don't stop Zimbra, at this point just run:
zmantivirusctl restart
Run the following command to make sure the antivirus is running. If it is, you're good to go:
zmcontrol status
You should check /opt/zimbra/log/clamd.log, and freshclam.log, in the same directory, and also /var/log/zimbra.log for errors.
You can easily implement Anti-Spam SMTP Proxy (ASSP) with Zimbra as a separate appliance. All you need is a Linux distribution either on a simple machine or even a virtual machine; just configure ASSP to point to the Zimbra machine. You can either disable the antispam provided with Zimbra by default via admin console and use only ASSP, or keep it and use both internal and external antispam solutions to get better results.
You can also install ASSP on the same machine side by side with Zimbra; this implementation is a little bit more complex and can generate issues, especially when upgrading. The idea behind it is to change the receiving SMTP port inside Zimbra to a custom one that you choose, then install ASSP and configure it, firstly, to listen on the default port 25 and secondly, to forward mails to localhost at the customized Zimbra specified port number.
Despite its complexity and the upgrade issues that can occur, this method allows us to have both services on the same machine, which in turn reduces the amount of hardware or virtual instances to be maintained.
Independently of the method chosen to handle incoming e-mails, we must also configure all outgoing e-mails to be forwarded to the ASSP instance if you would like ASSP to maintain its automatic whitelist. Doing that is quite easy; all you need to do is specify the port and address of the ASSP instance in the Zimbra administration console:
- Navigate to Configure | Global Settings | MTA.
- Configure the Relay MTA for external delivery field from the administration console as shown in the following screenshot:
You will also need to ensure that the ASSP administrators and users can send e-mails to ASSP. The slickest way to do this, I think, is to create distribution lists in Zimbra with a single address similar to the desired ASSP reporting address. For example, create a distribution list named <[email protected]> and enter a single list member—<[email protected]>. Make the display name something like "ASSP: Forward Spam Here".
The whole point of this exercise is to get the ASSP reporting addresses into Zimbra's Global Address List (GAL). Use a naming convention for these addresses, so they all sort together in the GAL. In this example, each display name would begin with "ASSP:".
One should also put the prefix—in this case "ASSP"—in the first name blank and the function—"Forward Spam"—in the last name when creating the distribution list. Then, users can easily look up all the appropriate addresses in the GAL if they forget what they are.
There is more than one level to improve SpamAssassin; here is a list of them:
The best way to configure the SpamAssassin filtering rules is to edit the /opt/zimbra/conf/salocal.cf.in file. Just keep in mind to back up this file before each Zimbra upgrade, because it will be replaced during this operation.
This is the simplest way to add filtering rules for SpamAssassin, in fact to block all mails going from an address or domain. You just have to add the address or the domain into a blacklist entry, and vice versa if you want that mails from a specific address or domain should bypass all filtering rules, we just need to add this address or domain into a whitelist entry.
The following example shows how to add blacklist or whitelist entries to the salocal.cf.in file:
blacklist_from [email protected] whitelist_from [email protected] blacklist_from *@abc-xyz.net
Note that * is a wildcard. In this example, *@ abc-xyz.net indicates all e-mails from any user at abc-xyz.net.
Zimbra SpamAssassin needs to be restarted for each modification on the salocal.cf.in file; to do that, we have to run the following command as a Zimbra user:
zmmtactl restart && zmamavisdctl restart
As we saw, blacklists and whitelists are used to filter addresses and domain names; however, this is not sufficient to filter spam mails, which is why there is another way based on content filtering that SpamAssassin uses to detect spam via reading headers, content of mails, and applying rules to that content.
Rules can be in the form of a particular word or phrase, as well as a variety of built-in functions. When a rule is "hit" while evaluating an e-mail, a point score is added to that e-mail's total score. When an e-mail's total score exceeds a certain threshold (typically 5 on a Zimbra system), the e-mail is either marked as spam, or deleted automatically if the score is high enough.
Rules are in the form of a test followed by a score. The rule mechanism typically uses Perl regular expressions to search for specific content within an e-mail. Custom rules should be added to the salocal.cf.in file in the following format:
body LOCAL_RULE /sale/ score LOCAL_RULE 0.5
The preceding code creates a rule called LOCAL_RULE that searches the body of the message for the word "sale" in lower case. If it finds the word "sale" anywhere in the body, it adds 0.5 to the total score of the e-mail. Note that the score is only applied once—multiple instances of the word "sale" in the same e-mail will not be scored separately. Also note that you should always precede the name added by the user in his/her defined rules with the word LOCAL as in the given example, to distinguish them from built-in SpamAssasin rules and prevent accidental duplicate names.
Perl regular expressions are quite powerful mechanisms for locating text. The following are some additional examples of Perl regular expression-based rules:
Performs a case-insensitive search for the word "sale":
body LOCAL_SALE /sale/i
Searches for a line that starts with the words "hot stock tip" in any case:
body LOCAL_STOCK1 /^hot stock tip/i
Searches for any four capital letters in a row (generally a stock symbol):
body LOCAL_4CAPS /[A-Z][A-Z][A-Z][A-Z]/
Searches for three digits, a decimal point, and two more digits, and treats it as a word:
body LOCAL_MONEY /d?d?d?.dd/
We can also search for a combination of rules and apply a score to that combination by creating a "meta" rule in the following format:
body LOCAL_FOUR_CAPS /[A-Z][A-Z][A-Z][A-Z]/ body LOCAL_MONEY /d?d?d?.dd/ meta LOCAL_STOCK (LOCAL_MONEY && LOCAL_FOUR_CAPS) score LOCAL_STOCK 1
The given rule would add 1 to an e-mail's score only if both LOCAL_FOUR_CAPS and LOCAL_MONEY were hits. Be careful when creating meta rules, as it is easy to "over-score" an e-mail, as in the following code:
body LOCAL_FOUR_CAPS /[A-Z][A-Z][A-Z][A-Z]/ score LOCAL_FOUR_CAPS 1 body LOCAL_MONEY /d?d?d?.dd/ score LOCAL_MONEY 1 meta LOCAL_STOCK (LOCAL_MONEY && LOCAL_FOUR_CAPS) score LOCAL_STOCK 1
The given example could add 3 points to the e-mail score, if the meta rule hits.
After editing the salocal.cf.in file, restart Zimbra SpamAssassin by issuing the following command at the server prompt (as a Zimbra user):
zmmtactl restart && zmamavisdctl restart
The Sender Policy Framework (SPF) is an open standard specifying a technical method to prevent forgery of the sender address. More precisely, the current version of SPF—called SPFv1 or SPF Classic—protects the envelope sender address, which is used for the delivery of messages.
Read more about SPF and how to configure it for any domain at http://www.openspf.org/
Also, read about tools for testing your SPF settings at http://www.openspf.org/Tools
First of all, Zimbra SpamAssassin has no SPF enabled. Since the Perl environment is system-way-integrated, adding SPF support is fairly simple.
We can install it on Debian/Ubuntu-like systems via the following command:
sudo apt-get install libmail-spf-query-perl
For every other platform, you can install SPF by opening and configuring it; if you didn't, use the cpan command line utility and execute the following commands:
perl -MCPAN -eshell install Mail::SPF::Query
Razor is a collaborative spam filtering network allowing users to report and filter out matching spam. After adding SPF, we can add Razor2 to enhance scores.
To install Razor2, we should first of all get the razor-agents-sdk package from razor.sourceforge.net, then untar it and run the following commands as a root user (that will compile and then install it):
perl Makefile.PL make make install
Repeat the preceding action with the razor-agents package that you get from razor.sourceforge.net:
perl Makefile.PL make make install
Don't forget to open the firewall ports for Razor2 (TCP/2703 outgoing).
To configure Razor2, start by creating a .razor folder under /opt/zimbra/amavisd and, of course, don't forget to give Zimbra user permissions; follow this:
mkdir /opt/zimbra/amavisd/.razor; chown -Rf zimbra:zimbra /opt/zimbra/amavisd/.razor
Then, as the Zimbra user, create your Razor account:
razor-admin -home=/opt/zimbra/amavisd/.razor -create razor-admin -home=/opt/zimbra/amavisd/.razor -discover razor-admin -home=/opt/zimbra/amavisd/.razor -register
Finally, enable Razor (if it is not done already). To do that, edit /opt/zimbra/conf/spamassassin/v310.pre and uncomment the line:
loadplugin Mail::SpamAssassin::Plugin::Razor2
Like Razor, Pyzor too serves to increase spam scores. Now, we are going to add Pyzor support to increase (again) spam scores. We can install it to improve spam filtering inside Zimbra.
As we are using Ubuntu, to install Pyzor, we need to just run the following command:
sudo apt-get install pyzor
To configure Pyzor, start by creating the .pyzor folder into zimbra-amavisd home and, of course, don't forget to give Zimbra user permissions; follow this:
mkdir /opt/zimbra/amavisd/.pyzor; chown zimbra:zimbra /opt/zimbra/amavisd/.pyzor
Don't forget to open your firewall ports for Pyzor (UDP/24441 outgoing).
Finally, as a Zimbra user, run the following command:
pyzor --homedir /opt/zimbra/amavisd/.pyzor discover
At this point, we have all three filters: Pyzor, Razor, and SPF. However, to gain the advantages of these tools, we should enable them and set rules for them inside the SpamAssassin configuration. To do that, edit the /opt/zimbra/conf/spamassassin/local.cf file and add the following rules at the end; of course, you can change values as you want and customize rules to suit your needs:
ok_languages en fr ok_locales en fr trusted_networks 127. 172.16. use_bayes 1 skip_rbl_checks 0 use_razor2 1 #use_dcc 1 <<< WORK IN PROGRESS - SEE NEXT STEP use_pyzor 1 dns_available yes ## Optional Score Increases ## Choose your preferred values... score DCC_CHECK 4.000 score SPF_FAIL 10.000 score SPF_HELO_FAIL 10.000 score RAZOR2_CHECK 2.500 score PYZOR_CHECK 2.500 score BAYES_99 4.300 score BAYES_90 3.500 score BAYES_80 3.000 bayes_ignore_header Received: from mail.zimbra-essentials.com bayes_ignore_header Received: from localhost
To get the
required score parameter in Zimbra, we don't need to edit any config file. This value is calculated from a setting in the Zimbra admin page. Enter administration, and go to Global Settings | AS/AV. The required score parameter is tag percent x 0.2. So a Tag percent value of 33 will result in a required score of 6.6 (33 x 0.2=6.6).
We can also enable Distributed Checksum Clearinghouses (DCC) to enhance antispam content filtering.
The basic logic in DCC is that most spam mails are sent to many recipients. The same message body appears many times; therefore, it creates bulk e-mail. DCC identifies bulk e-mail by taking a checksum and sending that checksum to a Clearinghouse (server). The server responds with the number of times it has received that checksum. An individual e-mail will create a score of 1 each time it is processed. Bulk mail can be identified because the response number is high. The content is not examined.
To set up DCC, start by downloading it from http://www.rhyolite.com/dcc/.
Compile and install it, then change /etc/dcc/dcc_conf to read:
DCCUID=zimbra DCCD_ENABLE=off
After that, change /opt/zimbra/conf/spamassassin/v310.pre to enable the DCC plugin:
loadplugin Mail::SpamAssassin::Plugin::DCC
Finally, enable DCC on the firewall (UDP/6277 outgoing).