Objectives
To install and configure the MailMan email listserv
To create a simple mailing list
To configure some general mailing list attributes to make the list more usable and secure
To manage user subscriptions
Why some listserv email get rejected by large services such as AOL, EarthLink, and Gmail
To configure lists to circumvent the rejection problems caused by some large email services
Introduction
Mailing lists are important in many environments. They provide a single administrative point for lists rather than depending upon everyone maintaining their own local copy of an email list. MailMan is an excellent open source tool that is used for mailing lists. Of course, there are others, but once again it is free, has many useful features, and is quite configurable and fairly easy to install and use.
MailMan integrates with both your MTA for email services and the web server for administrative functions. Of course, all of the administrative tasks can be performed from the command line as well.
Installing MailMan
Mailman installation is simple. Most Red Hat–based Linux distributions have recent versions and Fedora usually has the most current in its repository.
MailMan is written in Python, another powerful language. Because Python is not installed by default, it will be installed as a dependency when we install MailMan.
Experiment 12-1
This also installs Python as a dependency because it is not installed by default.
Integrating MailMan with Apache
Using the Fedora package for MailMan ensures that the vast majority of configuration is performed for you. The mailman user account and group are added, the files are all installed, and integration with SendMail using SMRSH1 is done. There are only a couple remaining tasks. We need to ensure that all permissions are set properly and integrate the MailMan web interface with our Apache web server.
Experiment 12-2
Perform this experiment as the root user on StudentVM2.
The Fedora installation places a file named mailman.conf in the /etc/httpd/conf.d directory where it will be automatically read – along with other configuration files located there – by the HTTPD service as it starts up. There is an “include” line in the httpd.conf file that causes all of the files in this directory to be read into the configuration as the service starts.
You should take some time to examine the short /etc/httpd/conf.d/mailman.conf file to understand what it is doing. You should be able to interpret it by now.
Site mailing list
MailMan needs an internal site-wide mailing list created in order to work. This list is not one that will ever be used to send email by a user. It is used by MailMan to send email notifications, such as the monthly password reminders, to users.
Experiment 12-3
Perform this experiment as the root user on StudentVM2.
The first list we need to create is the “mailman” list. This list is used by MailMan to send monthly notifications and other administrative emails. Make /usr/lib/mailman/bin the PWD. List the contents of this directory to view the administrative programs to which you have access.
As root, copy the data from the “##” line through the “unsubscribe” line and paste it into the end of the /etc/aliases file. Then run the newalaises command with no options to regenerate the aliases database file. No changes need to be made to the “mailman” mailing list and you should not add users to it.
MailMan is now ready for us to create a “real” list.
Create a mailing list
The next list will be a “real” one that we can use for testing. The steps are the same as creating the site mailing list but using a different list name.
Experiment 12-4
Perform this experiment as the root user on StudentVM2. Create a list named “Testlist” using similar steps to those shown above.
Copy the data from the “##” line through the “unsubscribe” line and paste it in at the end of the /etc/aliases file. Then run the newalaises command with no options to regenerate the aliases database file.
There is no need to restart MailMan or any other services when creating new lists.
Configuring the new list
If configuration changes need to be made to new lists, they can be most easily configured using the web interface. There is a file that defines default configuration parameters for new lists and we will get to that soon. For now, let’s look at manual configuration via the web interface.
Experiment 12-5
Perform this experiment as the student user on StudentVM1. We could do this from StudentVM2, but this part can be done from anywhere with a web browser and network access to the server.
Open a web browser or a new tab and enter the URL www1.example.com/mailman/admin/ which will display a page with a welcome message but no lists displayed.
We can use the URL for a specific list, so enter the following: www1.example.com/mailman/admin/testlist. This URL is the mail administrator page for the Testlist on our MailMan web site. We will soon configure our list so that it will be displayed in the list of advertised lists.
On the General Options page, add a terse phrase to identify the list in the field by that name. It can be anything you want, but one suggestion would be “Test list.”
In the introductory description field, enter some text that says something like, “A list for testing MailMan.” Also enter this same text in the field for the new subscriber welcome message.
There are a few options that I always want set on lists that I create. Set the following items and take a bit of time to look around each of the configuration pages to see what is there.
Scroll down the General Options page and check the boxes, Conceal members address and Acknowledge members posting.
Concealing the address just makes it look like, “member at example.com” so that the address will be harder to decipher by page scrapers looking for email addresses to spam. And most lists will want to send an acknowledgment of a posting, especially if members do not receive their own postings. Most users I have worked with want to get both the acknowledgment and the posting itself.
In the Hostname this list prefers for email field, remove the actual hostname so that only the example.com domain name is left.
Scroll down to the bottom of this first page and click the Submit Your Changes button.
Now go to the Privacy options page. Click the radio button, Confirm and Approve. This means that new users who subscribe themselves will get an email requesting that they confirm their desire to join the list. This can prevent “friends” from signing you up for a list you don't want to be subscribed to. And the “Approve” part means that the list administrator must approve all subscription requests. Also on this page, there may be times when you do not want even other members of the list to see the rest of the subscribers; you may wish to check the radio button for List admin only.
Scroll to the bottom of this page and click the Submit Your Changes button. I assume that you will remember to click the Submit Your Changes button after changing anything on the configuration web pages.
On the Archiving options page, it is a good practice to make the archives private unless there is a specific reason to allow them to be indexed by Google and other web bots.
Now go to the Membership Management ➤ Mass Subscription page and add at least two members to the list. One should be [email protected]. Another should be an email address you have access to outside the classroom lab. Use the format “Student User <[email protected]>” including the <> characters. Each user should receive an email that verifies they have been added to the list.
After submitting your changes, the page refreshes and lists the successful – or unsuccessful – results.
As the user student, view the confirmation emails sent by MailMan. Then send an email to the list to verify that it gets through. Check the archive to verify that the message you sent is archived.
You may notice that SpamAssassin is still altering the subject line and attaching a report to the emails that are not spam. You can safely comment out the lines that do that in the mimedefang-filter program.
Changing list defaults
You can change the defaults for future lists. Changing these defaults does not change any currently existing lists. The master configuration file for your MailMan site is /usr/lib/mailman/Mailman/Defaults.py, but you should not change that file. Change the file /etc/mailman/mm_cfg.py instead.
Experiment 12-6
Perform this experiment as the root user on StudentVM2. Take a quick look through the mm_cfg.py file. Let’s change one of the defaults for an item we changed manually for the previous list. Note that some, like the description fields, can and should not have a default value.
The values we want to change are not in the mm_cfg.py file, so we will need to find them in the Defaults.py file and copy the line to mm_cfg.py and change the value there. If we change them in the Defaults.py file, our changes may be overwritten but future updates.
This will now allow the list of lists to be viewed at the administrative URL, www1.example.com/mailman/admin/. Note that none of the changes to the mm_cfg.py file require a restart of MailMan. Verify that the TestList is now visible at that URL.
To test the policy change, create a new list named Test2. Refresh the main list page and select the new Test2 list. Log in with the site or list password and verify that the policy setting is now “confirm and approve.”
Add the email address for the student user to this new list and send an email to the list to test it.
The user interface
So far we have talked about the administrative interface of MailMan, but users also have an interface. This interface is shown in a link at the bottom of every email sent from the list, including the monthly password reminder if configured and the initial welcome email message.
The MailMan user interface in Figure 12-1 allows users to request a subscription, unsubscribe, change their password, and edit their options. They can also view the list of subscribers to a list, assuming that the list is configured to allow that.
The MailMan user interface allows users to subscribe to the list, unsubscribe, change their password, edit their options, and view the list of subscribers to a list
Experiment 12-7
Using the web-based user interface for MailMan, subscribe to the list, TestList, we just created in Experiment 12-5.
Then, as the list administrator, ensure that it is configured so that users can view the list of subscribers. As the user you just subscribed, view the subscriber list.
Rejections from large email services
Many large email services, such as AOL, Yahoo, Gmail, EarthLink, att.net, Spectrum, and more, are flooded with huge amounts of spam every day. Much of the spam is from mailing lists, or at least exhibits some of the attributes of email from a mailing list. Junk like joke/recipe/meme/word/quote/etc/etc, of the day, floods these services. If not controlled, those services would eventually bog down and collapse under the onslaught.
To combat spam in general and that from lists in particular, these services have instituted some interesting but obstructive countermeasures. Much of the time these measures do not affect MailMan lists but sometimes the email services apparently adjust the threshold at which a message from a list is considered to be spam. This is similar to what we can do with SpamAssassin.
The problem is that many messages from listservs of all kinds, even valid messages, are rejected or just dropped without any type of notification to the sender or recipient. This can occur particularly with messages sent by users with a sending email address domain belonging to one of the large mail services to a listserv that has a different domain and then message recipients with the same large email domain as the sender.
The large ISPs are trying to block spam which can look like it originated with them. Here is what happens if the sender domain of the email user is the same as the recipient domain, but the email was actually sent by a listserv with a different domain. The ISP has filters in place that compare the original sender’s domain with the domain the email was resent from – the listserv. If they are different – which they always would be with a listserv – the logic applied is that any email from the original sender domain that is sent to a recipient in the same domain should never come from a mail server that does not belong to us. So it is labeled as spam and dropped or a return rejection message is sent.
MailMan has devised methods for encapsulating messages, but they are not perfect, and different combinations work best with different ISPs.
Experiment 12-8
Perform this experiment as the student user. If there is not already a browser tab open to the MailMan admin page for the Testlist list, do so now.
On the General Options page for Testlist, scroll down to the field that begins, Replace the From: header address, and click the Munge From radio button. Click Submit Your Changes at the bottom of the page.
Now go to Privacy options ➤ Sender filters. In the field Action to take when anyone posts to the list from a domain with a DMARC Reject/Quarantine Policy, choose Munge From radio button and click Submit Your Changes at the bottom of the page.
This pair of changes seems to work well most of the time.
Despite the fact that the changes made in Experiment 12-8 work well a good part of the time, the email services change things, and so list messages are once again rejected. There are other things that can be done, and the MailMan developers are constantly working to keep up with changes.
When I first started using MailMan many years ago, I joined their user’s list.2 MailMan administrators can discuss problems and get help on this list. One of the most common discussion topics is about various ISPs blocking email from lists. The best options for circumventing these issues are usually found here.
Documentation
The GNU web site contains complete documentation3 for MailMan. This includes an installation manual, a list administration manual, and even a manual for list users. There are also discussion lists that you can join for interaction with other MailMan users and administrators.
Note that most of the work required to install and configure MailMan has already been performed by the MailMan RPM we installed from the Fedora repository. Because of this, the documentation becomes more of a theoretical reference than a help.
Chapter summary
In this chapter, you have installed the MailMan listserv software and integrated it with both SendMail and Apache. You have created the required site list and two usable lists and sent some emails to the lists. You have also learned how to configure default settings that will apply to newly created lists and to deal with certain types of rejections of emails from a list by some of the large ISPs.
Exercises
- 1.
Where does MailMan place its web server configuration data?
- 2.
Where is the CGI file for MailMan located?
- 3.
Create a new list. Name it whatever you choose. Verify that the defaults are set according to the changes you made on the mm_cfg.py file.
- 4.
Add a couple members to this newest list and send an email to the list.
- 5.
Find some volunteers who have AOL, RoadRunner, Time Warner, Spectrum, or one of the other large ISPs to help you with this one. Create a list with those people, but do not set the Munge from options on the list. Send them messages via the list to determine if the emails are being dropped or rejected. If some emails are being rejected, set the Munge from options and test again. This could be quite time-consuming, so don’t spend too much time with it.