Chapter 11. Manage the Queue

Mail messages can be either delivered immediately or held for later delivery. Held messages are referred to as queued. They are placed into either a single holding directory (usually called mqueue) or several directories from which they are later delivered.

What’s New with V8.13

  • The qf file’s V line has been bumped to V8 in support of queue quarantining (Section 11.1.1 [V8.13]).

  • Queue quarantining is now officially supported (Section 11.1.2 [V8.13]).

  • The qf file’s new q line lists the quarantine reason (Section 11.1.5 [V8.13]).

  • You may now manage lost (Qf file) envelopes (Section 11.1.6 [V8.13]).

  • Tunable queue-processing delays (Section 11.1.4 [V8.13]).

  • The new confTO_QUEUERETURN_DSN mc macro (see Section 4.1.3 [V8.13]).

  • The new confTO_QUEUEWARN_DSN mc macro (see Section 4.1.4 [V8.13]).

The qf File’s V Line

The V line (11.11.19[3ed]) in the qf file (11.2.5[3ed]) corresponds to the version of sendmail that created it. Table 11-1 compares V line qf file versions to corresponding sendmail versions.

Table 11-1. Queue qf file versions

V version

sendmail version

1

8.6 and earlier

2

8.7 and 8.8

3

8.9

4

8.10 and 8.11 built with -D_FFR_QUEUEDELAY not defined

5

8.10 and 8.11 built with -D_FFR_QUEUEDELAY defined

6

8.12 built with -D_FFR_QUEUEDELAY not defined

7

8.12 built with -D_FFR_QUEUEDELAY defined

8

8.13

Note that the qf file’s V8 line corresponds to V8.13 sendmail, which added official support for queue quarantining.

Queue Quarantining

Queue quarantining is the process by which envelopes in the queue are marked as being ineligible for delivery. Such envelopes may then be manually or automatically reviewed. If review permits, each such envelope may then be delivered, bounced, or discarded. Queue quarantining employs the queue’s qf file, command-line switches, and the access database. Lost envelopes (covered in the next section) are also a part of this system.

Overview of quarantining

A quarantined message is an envelope containing one or more recipients that is held in the queue pending review. It can either be an inbound or outbound envelope that, for policy or security reasons, should not be sent or delivered immediately and should not be sent or delivered as is.

For example, consider a user who has a history of sending offensive email. You might want to intercept such a user’s email on its way out, so it can be screened for certain words or phrases about which the user has been previously warned.

V8.13 sendmail implemented quarantining by creating a new kind of queued file. Instead of storing the envelope information in a qf file, a quarantined message has its envelope information stored in an hf file. The different file allows sendmail to process messages normally (quarantined messages are invisible) unless you specifically ask it to handle quarantined messages (make them visible).

Note that the mailstats program (Section 5.1.1 Section 5.1.1[V8.13]) is an exception because it automatically includes the total count of quarantined messages in its output.

To insure that the reason for quarantining a message is not lost, a new qf file[12] line has been introduced. Called a q line (Section 11.1.5 [V8.13]), it stores the reason the message was quarantined. In parallel, a new macro has also been added. Called ${quarantine} (Section 21.1.7 [V8.13]), and intended for use in rule sets, it also contains the reason the envelope was quarantined.

Note that quarantining integrates well with all the other queuing facilities of sendmail and even works with envelope splitting.

In the following sections, we discuss how:

Quarantine command-line switches

The command-line can be used to quarantine and de-quarantine envelopes. V8.13 has added one new command-line switch and modified another. We will show the use of the modified switch first, then the new one.

The -qQ command-line switch

Normally, the queue is processed by invoking a -q command-line switch (11.8.1[3ed]). This switch causes all the normally scheduled (nonquarantined) envelopes to be processed. By combining that switch with a Q argument, you tell sendmail to process quarantined messages instead.

Note that it is not possible to operate on both normal and quarantined envelopes at the same time. That is, listing -q then -qQ does not process both, but only quarantined messages.

Unless limited with other -q letters, the -qQ switch processes all the quarantined envelopes currently in the queue. To further limit the envelopes to be processed, specify any of these additional switches in the same command line:

-qIident
                                             
                     
                      match any queue ID that contains 
                     ident (§11.8.2.3
                     [3ed])
-q!Iident        
                                     
                     
                      match any queue ID that does not contain 
                     ident (§11.8.2.4[3ed])
-qRrecip        
                                     
                     
                      match any recipient address that contains 
                     recip (§11.8.2.3[3ed])
-q!Rrecip        
                                     
                     
                      match any recipient address that does not contain 
                     recip  (§11.8.2.4[3ed])
-qSfrom
                                             
                     
                      match any sender address that contains 
                     from (§11.8.2.3[3ed])
-q!Sfrom        
                                     
                     
                      match any sender address that does not contain 
                     from  (§11.8.2.4[3ed])
-qGname
                                             
                     
                      match any queue group with the name 
                     name  (§11.8.2.5[3ed])
-qQreason                
                             
                     
                      match any queue group with the name 
                     name  (Section 11.1.2.2
                     [V8.13]

For example, the following command line processes only quarantined envelopes in the queue group okayclients that were sent by the user bob:

/usr/sbin/sendmail -qQ -qGokayclients -qSbob

The same switches can also be used to determine what the mailq command will print. For example, the following prints the status of all the currently quarantined envelopes:

mailq -qQ

The -Q command-line switch

When the -Q command-line switch is used with an argument (such as -Q"reason“), it causes the specified envelopes to become quarantined. When used without an argument, it causes the specified envelopes to become de-quarantined.

For example, the following command line causes all currently queued envelopes sent by the user bob to become quarantined:

/usr/sbin/sendmail [email protected] -Q"Bob resigned today"

Here, the [email protected] causes the queue to be searched for all envelopes that are from the sender (the -qS) bob at your domain. -Q is followed by the argument "Bob resigned today" so all those messages are quarantined using “Bob resigned today” as the reason.

To de-quarantine those same messages, you might use a command line such as the following, where the -Q is not followed by an argument:

/usr/sbin/sendmail -qQ [email protected] -Q

Here, the -qQ tells sendmail to operate only on quarantined envelopes. The -qS causes sendmail to search the quarantined envelopes for those from the sender bob at your domain. And finally, the -Q, without an argument, tells sendmail to de-quarantine all the envelopes found.

The mailq command’s display

When the -qQ command-line switch is specified, the mailq command displays only quarantined messages and the reason each was quarantined. For example:

# mailq -qQ
                /var/spool/mailqueue (1 request)
-----Q-ID----- --Size-- -----Q-Time----- ------------Sender/Recipient----------
h2VJcN3M012024   875429 Thu Mar 24 16:44 [email protected]
     QUARANTINE: Bob resigned today
                                         [email protected]
                 Total requests: 1

Here, the -qQ command-line switch causes mailq to print only the messages (there is only one in this example) that were quarantined in the queue. Information about the message is printed first. The reason the message was quarantined is printed next. Then the recipient or recipients of the message are printed last.

If you have set up a Milter to automatically quarantine messages (Section 7.1.2.1 [V8.13]), set up the access database to automatically quarantine messages (Section 11.1.2.5[V8.13]), or have created rule sets to automatically quarantine messages (Section 11.1.2.6 [V8.13]), you should run mailq with this -qQ command-line argument periodically to discover whether anything has been automatically quarantined.

Use Milter to quarantine

The end-of-message handler, inside a Milter, can call smfi_quarantine( ) to quarantine the envelope being screened. (This Milter routine is described in Section 7.1.2.1 [V8.13].)

Use the access database to quarantine

The access database (7.5[3ed]) provides a single, central database with rules to accept, reject, and discard messages based on the sender name, address, or IP number. It is enabled with the access_db mc configuration feature (7.5.1[3ed]).

A source text file used to create an access database might look (in part) like the following. Note that each line is composed of a key on the left and a value on the right, separated by tabs:[13]

Connect:192.167.23.45   REJECT
To:friend.gov           RELAY
From:we-spam-you.com    REJECT

Here, the first line says to reject the connection from the sending host that has the address 192.168.23.45. The second line says that it is okay to relay anything that is intended for delivery to (the SMTP RCPT TO:) the domain friend.gov. The last line says to reject any message that is from (the SMTP MAIL FROM:) the domain we-spam-you.com.

Beginning with V8.13 sendmail, the access database may also be used to quarantine envelopes. The key on the left may be any normally legal key (7.5.2[3ed]). To quarantine the envelope defined by that key, use the new term QUARANTINE as the value on the right.

                  
                     key                   
                            QUARANTINE
                  
                     key                   
                            QUARANTINE:
                  
                     reason                   

Note that the QUARANTINE term on the right may optionally be followed by a colon and the reason the envelope is being quarantined. The reason may contain whitespace but must not contain newlines and should not be quoted.

For example, consider the three following entries in a source file for an access database:

Connect:192.168.1.23      QUARANTINE:Bob's PC
To:your.competitor.gov    QUARANTINE:Review mail to our competitor
From:head.hunter.domain   QUARANTINE:Employee theft?

In the first line, Bob’s PC sends email by connecting to the SMTP port on the central mail server. Because of past behavior, or perhaps because of a worm or virus on Bob’s PC, we want to quarantine all outbound mail from that machine.

In the second line, management has requested that all mail (using an SMTP RCPT TO:) to the domain your.competitor.gov be quarantined for review before it is allowed to be sent.

The last line says that inbound mail addressed (using an SMTP MAIL FROM:) from the domain head.hunter.domain be quarantined so that it may be reviewed to see whether employee theft is being attempted.

One limitation of the access database is that it cannot conveniently be used to combine tests. If your tests are more complex than the access database can handle, you may employ rules in rule sets too.

Use rule sets to quarantine

Any of the check_ rule sets (7.1[3ed])[14] and any of the header screening rule sets (25.5[3ed]) may be used to quarantine envelopes. Any rule set that returns a $#error (20.4.4[3ed]) with a $@ part (19.5[3ed]) that is the literal quarantine will cause the message to be quarantined.

R $* < @ bad.site > $*         $#error $@ quarantine $: reason

Here, we show a rule in a rule set that returns a $#error. Because the $@ part is the literal quarantine, the message will be quarantined. Note that the $: part contains the reason the message is being quarantined.

Note that rule set quarantining affects all recipients of that envelope.

To illustrate rule set quarantining, consider the following mc configuration lines that cause any message that contains a special X-review: header to be held for review:

LOCAL_CONFIG
HX-review: $>Xreview

LOCAL_RULESETS
SXreview
R YES           $#error $@ quarantine $: X-review held for review

The first part of our example, the LOCAL_CONFIG part, defines a header. This header definition tells sendmail to pass all X-Review: header values through the Xreview rule set as specified using the $> with the H command.

The second part (LOCAL_RULESETS) defines the Xreview rule set (the S line), which contains a single rule that looks for a value that is the literal word YES. If that header’s value is YES, the message is quarantined with the reason shown. If that header is missing, or if it has any other value, this quarantine step is skipped.

Note that rules sets can detect whether a message has already been quarantined by checking the ${quarantine} sendmail macro (Section 21.1.7 [V8.13]). If that macro has a value, the message is already quarantined.

Logging

Whenever a message is quarantined, the fact that it was quarantined and the reason for doing so is logged using syslog(3). One log line is produced to record the quarantine event. Another is produced for each recipient to show that each was also quarantined.

The information logged for the quarantine event varies depending on the method used to quarantine. If a rule set is used, for example, a log line such as the following might be produced:

Oct  9 11:26:00 your.domain sendmail[4788]: f99IPuIH004788: ruleset=check_mail, 
[email protected], quarantine=Hold mail from competitor.gov

This line (wrapped to fit the page) shows that the check_mail rule set found the address [email protected] in its workspace and quarantined the message for the reason shown.

A Milter can also cause messages to be quarantined. The log line, produced by such a Milter event, might look like the following:

Oct  23 09:25:59 monkeyboy sendmail[52314]: f99IPuIH004787: milter=DocMilter, 
quarantine=Suspect application/ms-word attachment

Here, the Milter named DocMilter found a MIME type that indicated a possible Microsoft Word document was included as an attachment.

In addition to event logging, each recipient is also logged. For example, consider the following log line:

Nov 21 09:32:13 your.domain sendmail[33522]: fALHVwAQ033522: to=<[email protected]>, 
delay=00:00:06, mailer=local, pri=30029, quarantine=Suspect application/ms-word attachment,
stat=quarantine

Here the quarantine= equate shows the reason the message was quarantined, and the stat= equate prints the literal word quarantine.

When Milters, the access database, and rule sets are used to automatically quarantine messages, a script may be devised to detect the quarantine= equate in logging output. When run nightly, such a script might email the postmaster with a summary of quarantined messages for that day.

Manage Quarantined Envelopes with qtool.pl

The qtool.pl program is located in the contrib subdirectory of the source distribution. It is a perl(1) script that allows you to move envelopes between queues, bounce envelopes, and remove envelopes.

In general, if you use queue groups (11.4[3ed]), you should not use qtool.pl to move queued messages. However, it is always safe to move quarantined messages, because they are invisible to sendmail unless you manually cause sendmail to recognize them.

As of V8.13, the -Q command-line switch tells qtool.pl to operate on quarantined messages rather than on normal messages. For example, the following command causes all the quarantined messages in the main queue to be moved to a holding queue.

#./qtool.pl -Q /var/spool/hold /var/spool/mqueue

Also, as of V8.13, a new %msg hash variable has been introduced. Called quarantine_reason, it can be used to match strings in the quarantine reason. To illustrate, consider the need to bounce all messages that were quarantined with a reason that contained the word Virus.

#./qtool.pl -b -Q -e '$msg{quarantine_reason} =~ m/Virus/'

See the online manual for the qtool.pl program (contrib/qtool.8) for a complete guide to using that program.

The qf File’s q Line

As of V8.13, the qf file’s q line is used to store the reason that an envelope was quarantined. The q line should only appear in quarantined envelopes—that is, in hf files, not in qf files. If a q line appears in a qf file, that file will be silently converted into an hf file. Thus, it does no good to simply rename an hf file into a qf file.

The format of a q line looks like this:

qreason

There may only be one q line in an hf file. The reason is the reason the envelope was quarantined.

Handle Lost Envelopes

A problem with a qf file (such as a missing ending dot) can result in the loss of that file. When sendmail looses a qf file, it changes the first part of the filename into a Qf and logs that it did so. Other reasons an envelope can be lost are covered in 11.5[3ed].

You may now use command-line switches to handle lost envelopes. Note, however, that handling lost envelopes, without first repairing the problem that caused the loss, can be risky.

Beginning with V8.13, the new -qL command-line switch allows you to view and handle lost files.[15] One use for this new switch is to examine the mail queue to see whether any lost files exist:

% mailq -qL
                /var/spool/mqueue (1 request)
-----Q-ID----- --Size-- -----Q-Time----- ------------Sender/Recipient-----------
h7AJG4kr009003?     235 Sun Aug 10 13:16 <[email protected]>
                                         <[email protected]>
                Total requests: 1

Here, the -qL command-line switch was used with the mailq command to see whether any lost files were present. This output shows a lost file (called Qfh7AJG4kr009003) located in the /var/spool/mqueue directory. The ? character following the file’s name indicates that it is a lost envelope.

This -qL switch can be combined with other queue-handling switches (11.8.2.3[3ed]) to further limit what can be shown.



[12] We say qf file, even though this new line appears only in the new hf file type.

[13] Or other separation character specified by the -t command-line switch with makemap.

[14] Except the check_compat rule set.

[15] But not using qtool.pl, which handles only quarantined files, not lost files.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.239.15.34