In the previous chapter, you saw how you might need to spend considerable time and energy making sure that the services you want to offer will be available even when you have strict packet filtering in place. Now, with your working setup in place, you will soon notice that some services tend to attract a little more unwanted attention than others.
Here’s the scenario: You have a network with packet filtering to match your site’s needs, including some services that need to be accessible to users from elsewhere. Unfortunately, when services are available, there is a risk that someone will want to exploit them for some sort of mischief.
You will almost certainly have remote login via SSH, as well as SMTP email running on your network—both are tempting targets. In this chapter, we’ll look at some ways to make it harder to gain unauthorized access via SSH, and then we’ll turn to some of the more effective ways to deny spammers use of your servers.
The Secure Shell service, commonly referred to as SSH, is a fairly crucial service for Unix administrators. It’s frequently the main interface to the machine and a favorite target of script kiddie attacks.
If you run an SSH login service that is accessible from the Internet, you’ve probably seen entries like this in your authentication logs:
Sep 26 03:12:34 skapet sshd[25771]: Failed password for root from 200.72.41.31 port 40992 ssh2 Sep 26 03:12:34 skapet sshd[5279]: Failed password for root from 200.72.41.31 port 40992 ssh2 Sep 26 03:12:35 skapet sshd[5279]: Received disconnect from 200.72.41.31: 11: Bye Bye Sep 26 03:12:44 skapet sshd[29635]: Invalid user admin from 200.72.41.31 Sep 26 03:12:44 skapet sshd[24703]: input_userauth_request: invalid user admin Sep 26 03:12:44 skapet sshd[24703]: Failed password for invalid user admin from 200.72.41.31 port 41484 ssh2 Sep 26 03:12:44 skapet sshd[29635]: Failed password for invalid user admin from 200.72.41.31 port 41484 ssh2 Sep 26 03:12:45 skapet sshd[24703]: Connection closed by 200.72.41.31 Sep 26 03:13:10 skapet sshd[11459]: Failed password for root from 200.72.41.31 port 43344 ssh2 Sep 26 03:13:10 skapet sshd[7635]: Failed password for root from 200.72.41.31 port 43344 ssh2 Sep 26 03:13:10 skapet sshd[11459]: Received disconnect from 200.72.41.31: 11: Bye Bye Sep 26 03:13:15 skapet sshd[31357]: Invalid user admin from 200.72.41.31 Sep 26 03:13:15 skapet sshd[10543]: input_userauth_request: invalid user admin Sep 26 03:13:15 skapet sshd[10543]: Failed password for invalid user admin from 200.72.41.31 port 43811 ssh2 Sep 26 03:13:15 skapet sshd[31357]: Failed password for invalid user admin from 200.72.41.31 port 43811 ssh2 Sep 26 03:13:15 skapet sshd[10543]: Received disconnect from 200.72.41.31: 11: Bye Bye Sep 26 03:13:25 skapet sshd[6526]: Connection closed by 200.72.41.31
This is what a brute-force attack looks like. Someone or something is trying by brute force to find a username and password to use to get into your system.
The simplest response would be to write a pf.conf rule that blocks all access, but that leads to another class of problems, including how to let people with legitimate business on your system access it. Setting up your sshd to accept only key-based authentication would help, but most likely would not stop the kiddies from trying. You might consider moving the service to another port, but then again, the ones flooding you on port 22 would probably be able to scan their way to port 22222 for a repeat performance.
Since OpenBSD 3.7 (and equivalents), PF has offered a slightly more elegant solution.
To thwart brute-force attacks, you can write your pass rules so they maintain certain limits on what connecting hosts can do. For good measure, you can banish violators to a table of addresses that you deny some or all access. You can even choose to drop all existing connections from machines that overreach your limits. To enable this feature, first set up the table by adding the following line to your configuration before any filtering rules:
table <bruteforce> persist
Then, early in your rule set, block brute forcers, as shown here:
block quick from <bruteforce>
Finally, add your pass rule:
pass inet proto tcp to $localnet port $tcp_services
keep state (max-src-conn 100, max-src-conn-rate 15/5,
overload <bruteforce> flush global)This rule is very similar to what you’ve seen in earlier examples. The interesting part in this context is the contents of the parentheses, called state-tracking options:
max-src-connis the number of simultaneous connections allowed from one host. In this example, it’s set to100. You may want a slightly higher or lower value, depending on your network’s traffic patterns.max-src-conn-rateis the rate of new connections allowed from any single host. Here, it’s set to 15 connections per 5 seconds, denoted as15/5. Choose a rate that suits your setup.overload <bruteforce>means that the address of any host that exceeds the preceding limits is added to the tablebruteforce. Our rule set blocks all traffic from addresses in thebruteforcetable.Once a host exceeds any of these limits and is put in the overload table, the rule no longer matches traffic from that host. Make sure that overloaders are handled, if only by a default block rule or similar.
flush globalsays that when a host reaches the limit, all states for its connections are terminated (flushed). Theglobaloption means that for good measure,flushapplies to states created by traffic from that host that matches otherpassrules, too.
As you can imagine, the effect of this tiny addition to the rule set is quite dramatic. After a few tries, brute forcers end up in the bruteforce table. That means that all their existing connections are terminated (flushed), and any new attempts will be blocked, most likely with Fatal: timeout before authentication messages at their end. You have created an adaptive firewall that adjusts automatically to conditions in your network and acts on undesirable activity.
Note
These adaptive rules are effective only for protection against the traditional, rapid-fire type of brute-force attempts. The low-intensity, distributed password-guessing attempts that were first identified as such in 2008 and have been recurring ever since (known among other names as The Hail Mary Cloud) do not produce traffic that will match these rules.
It’s likely that you may want some flexibility in your rule set, and allow a larger number of connections for some services, but would like to be a little more tight-fisted when it comes to SSH. In that case, you could supplement the general-purpose pass rule with something like the following one, early in your rule set:
pass quick proto { tcp, udp } to port ssh
keep state (max-src-conn 15, max-src-conn-rate 5/3,
overload <bruteforce> flush global)You should be able to find the set of parameters that is just right for your situation by reading the relevant man pages and the PF User Guide (see Appendix A).
Note
Remember that these sample rules are intended as illustrations, and your network’s needs may be better served by different rules. Setting the number of simultaneous connections or the rate of connections too low may block legitimate traffic with a clear risk of self-inflicted denial of service when the configuration includes many hosts behind a common NATing gateway, and the users on the NATed hosts have legitimate business on the other side of a gateway with strict overload rules.
The state-tracking options and the overload mechanism do not need to apply exclusively to the SSH service, and blocking all traffic from offenders is not always desired. You could, for example, use a rule like this:
pass proto { tcp, udp } to port $mail_services
keep state (max 1500, max-src-conn 100)Here, max specifies the maximum number of states that can be created for each rule (the number of rules loaded depends on what the $mail_services macro expands to) with no overload to protect a mail or web service from receiving more connections than it can handle. With this rule, the max value determines the maximum number of states that will be created for each resulting rule. Once the limit is reached, new connections will not be allowed until the old ones terminate. Alternatively, you could remove the max restriction, add an overload part to the rule, and assign offenders to a queue with a minimal bandwidth allocation (see the discussion of ALTQ in Chapter 7 for details on setting up queues).
Some sites use overload to implement a multitiered system, where hosts that trip one overload rule are transferred to one or more intermediate “probation” tables for special treatment. It can be useful in web contexts to not block traffic from hosts in the overload tables outright, but rather redirect all HTTP requests from these hosts to specific web pages (like in the authpf example near the end of Chapter 4).
With the overload rules from the previous section in place, you now have an adaptive firewall that automatically detects undesirable behavior and adds offenders’ IP addresses to tables. Watching the logs and the tables can be fun in the short run, but since those rules only add to the tables, we run into the next challenge: keeping the content of the tables up to date and relevant.
When you’ve run a configuration with an adaptive rule set for a while, at some point, you will discover that an IP address one of your overload rules blocked last week due to a brute-force attack was actually a dynamically assigned address, which is now assigned to a different ISP customer with a legitimate reason to communicate with hosts in your network.[29] If your adaptive rules catch a lot of traffic on a busy network, you may also find that the overload tables will grow over time to take up an increasing amount of memory.
The solution is to expire table entries—to remove entries that have not been referenced for a certain amount of time. In OpenBSD 4.1, pfctl acquired the ability to expire table entries based on the time since their statistics were last reset.[30] (In almost all instances, this reset time is equal to the time since the table entry was added.) The keyword is expire, and the table entry’s age is specified in seconds. Here’s an example:
$ sudo pfctl -t bruteforce -T expire 86400This command will remove bruteforce table entries that had their statistics reset more than 86,400 seconds (24 hours) ago.
Note
The choice of 24 hours as the expiry time is a fairly arbitrary one. You should choose a value that you feel is a reasonable amount of time for any problem at the other end to be noticed and fixed. If you have adaptive rules in place, it is a good idea to set up crontab entries to run table expiry at regular intervals with a command much like the preceding one to make sure your tables are kept up to date.
[29] In a longer-term perspective, it is fairly normal for entire networks and larger ranges of IP addresses to be reassigned to new owners in response to events in the physical, business-oriented world.
[30] Before pfctl acquired the ability to expire table entries, table expiry was more likely than not handled by the special-purpose utility expiretable. If your pfctl does not have the expire option, look for expiretable in your package system.