For most debugging and lightweight monitoring purposes, logging the first packet in a connection provides enough information. However, sometimes you may want to log all packets that match certain rules. To do so, use the (all) logging option in the rules you want to monitor. After making this change to our minimal rule set, we have this:

block log (all)
pass log (all) quick proto tcp to port ssh keep state

This option makes the logs quite a bit more verbose. To illustrate just how much more data log (all) generates, we’ll use the following rule set fragment, which passes domain name lookups and network time synchronizations:

udp_services = "{ domain, ntp }"
pass log (all) inet proto udp to port $udp_services

With these rules in place, here’s an example of what happens when a Russian name server sends a domain name request to a server in our network:

$ sudo tcpdump -n -ttt -i pflog0 port domain
tcpdump: WARNING: pflog0: no IPv4 address assigned
tcpdump: listening on pflog0, link-type PFLOG
Sep 30 14:27:41.260190 212.5.66.14.53 > 194.54.107.19.53:[|domain]
Sep 30 14:27:41.260253 212.5.66.14.53 > 194.54.107.19.53:[|domain]
Sep 30 14:27:41.260267 212.5.66.14.53 > 194.54.107.19.53:[|domain]
Sep 30 14:27:41.260638 194.54.107.19.53 > 212.5.66.14.53:[|domain]
Sep 30 14:27:41.260798 194.54.107.19.53 > 212.5.66.14.53:[|domain]
Sep 30 14:27:41.260923 194.54.107.19.53 > 212.5.66.14.53:[|domain]

We now have six entries instead of just one.

Even with all but port domain filtered out by tcpdump, adding log (all) to one or more rules considerably increases the amount of data in your logs. If you need to log all traffic, but your gateway’s storage capacity is limited, you may find yourself shopping for additional storage. Also, recording and storing traffic logs with this level of detail is likely to have legal implications.

PF versions older than OpenBSD 4.1 offered only one pflog interface. That changed with OpenBSD 4.1, when the pflog interface became a cloneable device, meaning that you can use ifconfig commands to create several pflog interfaces, in addition to the default pflog0. This makes it possible to record the log data for different parts of your rule set to separate pflog interfaces, and easier to process the resulting data separately if necessary.

The required changes to your setup are subtle but effective. To log to several interfaces, make sure that all the log interfaces your rule set uses are created. You don’t need to create the devices before the rule set is loaded; if your rule set logs to a nonexistent interface, the log data is simply discarded.

When tuning your setup to use several pflog interfaces, you will most likely add the required interfaces from the command line, like so:

$ sudo ifconfig create pflog1

Specify the log device when you add the log keyword to your rule set, as follows:

pass log (to pflog1) proto tcp to $emailserver port $email
pass log (to pflog1) proto tcp from $emailserver to port smtp

For a more permanent configuration on OpenBSD, create a hostname.pflog1 file containing only up, and similar hostname.pflogN files for any additional logging interfaces.

On FreeBSD, the configuration of the cloned pflog interfaces belongs in your rc.conf file, in the following form:

ifconfig_pflog1="up"

On NetBSD, cloning pflog interfaces is not an option as of this writing.

As you saw in Chapter 6, directing log information for different parts of your rule set to separate interfaces makes it possible to feed different parts of the log data PF produces to separate applications. This makes it easier to have programs like spamlogd process only the relevant information, while you feed other parts of your PF log data to other log-processing programs.

One way to avoid storing PF log data on the gateway itself is to instruct your gateway to log to another machine. If you already have a centralized logging infrastructure in place, this is a fairly logical thing to do, even if PF’s ordinary logging mechanisms were not really designed with traditional syslog-style logging in mind.

As any old BSD hand will tell you, the traditional syslog system log facility is a bit naïve about managing the data it receives over UDP from other hosts, with denial-of-service attacks involving full disks one frequently mentioned danger. There is also the ever-present risk that log information will be lost under high load on either individual systems or the network. Therefore, consider setting up remote logging only if all hosts involved communicate over a well-secured network. On most BSDs, syslogd is not set up by default to accept log data from other hosts. (See the syslogd man page for information about how to enable listening for log data from remote hosts if you plan to use remote syslog logging.)

If you would still like to do your PF logging via syslog, the following is a short recipe for how to accomplish this. In ordinary PF setups, pflogd copies the log data to the log file. When you primarily want to store the log data on a remote system, you should disable pflog’s data accumulation by changing its log file to /dev/null, via the daemon’s startup options in rc.conf.local (on OpenBSD), like so:

pflogd_flags="-f /dev/null"

On FreeBSD and NetBSD, change the pflog_logfile= line in rc.conf as follows, and then kill and restart the pflogd process with its new parameters:

pflog_logfile="/dev/null"

Next, make sure that the log data, now no longer collected by pflogd, is transmitted in a meaningful way to your log-processing system instead. This step has two parts: First, set up your system logger to transmit data to the log processing system, and then use tcpdump with logger to convert the data and inject it into the syslog system.

To set up syslogd to process the data, choose your log facility, log level, and action and put the resulting line in /etc/syslog.conf. These concepts are very well explained in man syslog.conf, which is required reading if you want to understand system logs. The action part is usually a file in a local filesystem. For example, if you have already set up the system logger at loghost.example.com to receive your data, choosing log facility local2 with log level info, enter this line:

local2.info                 @loghost.example.com

Once you’ve made this change, restart syslogd to make it read the new settings.

Next, set tcpdump to convert the log data from the pflog device and feed it to logger, which will then send it to the system logger. Here, we reuse the tcpdump command from the basic examples earlier in this chapter, with some useful additions:

$ sudo nohup tcpdump -lnettti pflog0 | logger -t pf -p local2.info &

The nohup command makes sure the process keeps running even if it does not have a controlling terminal or it’s put in the background (as we do here with the trailing &). The -l option to the tcpdump commad specifies line-buffered output, which is useful for redirecting to other programs. The logger option adds the tag pf to identify the PF data in the stream and specifies log priority with the -p option as local2.info. The result is logged to the file you specify on the logging host, with entries that will look something like this:

pf: Sep 21 14:05:11.492590 rule 93/(match) pass in on ath0: 10.168.103.11.15842 >
82.117.50.17.80: [|tcp] (DF)
pf: Sep 21 14:05:11.492648 rule 93/(match) pass out on xl0: 194.54.107.19.15842 >
82.117.50.17.80: [|tcp] (DF)
pf: Sep 21 14:05:11.506289 rule 93/(match) pass in on ath0: 10.168.103.11.27984 >
82.117.50.17.80: [|tcp] (DF)
pf: Sep 21 14:05:11.506330 rule 93/(match) pass out on xl0: 194.54.107.19.27984 >
82.117.50.17.80: [|tcp] (DF)
pf: Sep 21 14:05:11.573561 rule 136/(match) pass in on ath0: 10.168.103.11.6430 >
10.168.103.1.53:[|domain]
pf: Sep 21 14:05:11.574276 rule 136/(match) pass out on xl0: 194.54.107.19.26281 >
209.62.178.21.53:[|domain]

This log fragment shows mainly web-browsing activities from a client in a NATed local network as seen from the gateway’s perspective, with accompanying domain name lookups.

The sequential information you get from retrieving log data basically tracks packet movements over time. In other contexts, the sequence or history of connections is less important than aggregates, such as number of packets or bytes that have matched a rule since the counters were last cleared.

At the end of Chapter 2, you saw how to use pfctl -s info to view the global aggregate counters, along with other data. For a more detailed breakdown of the data, track traffic totals on a per-rule basis with a slightly different form of pfctl command, such as pfctl -vs rules, to display statistics along with the rule, as shown here:

$ pfctl -vs rules
pass inet proto tcp from any to 192.0.2.225 port = smtp flags S/SA keep state
 label "mail-in"
  [ Evaluations: 1664158   Packets: 1601986   Bytes: 763762591   States: 0     ]
  [ Inserted: uid 0 pid 24490 ]
pass inet proto tcp from 192.0.2.225 to any port = smtp flags S/SA keep state
 label "mail-out"
  [ Evaluations: 2814933   Packets: 2711211   Bytes: 492510664   States: 0     ]
  [ Inserted: uid 0 pid 24490 ]

The format of this output is easy to read, and obviously designed for contexts where you want to get an idea of what is going on at a glance. If you specify even more verbose output with pfctl -vvs rules, you will see essentially the same display, with rule numbers added. On the other hand, the output from this command is not very well suited for feeding to a script or other program for further processing. To extract these statistics and a few more items in a slightly more script-friendly format (and make your own decisions about which rules are worth tracking), use rule labels.

Labels do more than identify rules for processing specific kinds of traffic; they also make it easier to extract the traffic statistics. By attaching labels to rules, you can store certain extra data about parts of your rule set. For example, you could use labeling to measure bandwidth use for accounting purposes.

In the following example, we attach the labels mail-in and mail-out to our pass rules for incoming and outgoing mail traffic, respectively.

pass log proto { tcp, udp } to $emailserver port smtp label "mail-in"
pass log proto { tcp, udp } from $emailserver to port smtp label "mail-out"

Once you have loaded the rule set with labels, check the data using pfctl -vsl:

$ sudo pfctl -vsl
    ❶     ❷      ❸       ❹       ❺      ❻        ❼      ❽
mail-in 1664158 1601986 763762591 887895 682427415 714091 81335176
mail-out 2814933 2711211 492510664 1407278 239776267 1303933 252734397

This output contains the following information:

❶ The label

❷ The number of times the rule has been evaluated

❸ The total number of packets passed

❹ The total number of bytes passed

❺ The number of packets passed in

❻ The number of bytes passed in

❼ The number of packets passed out

❽ The number of bytes passed out

The format of this list makes it very well suited for parsing by scripts and applications.

The labels accumulate data from the time the rule set is loaded until their counters are reset. And, in many contexts, it makes sense to set up a cron job that reads label values at fixed intervals, and then puts those values into permanent storage.

If you choose to run the data collection at fixed intervals, consider collecting the data using pfctl -vszl. The z option resets the counters once pfctl has read them, with the result that your data collector will then be fetching periodic data, accumulated since the command or the script was last run.