This chapter deals with two main topics, which taken separately or together have the potential to radically transform your networking experience. The common theme in this chapter is managing resource availability.
In the first part of the chapter, we look at how to use the ALTQ traffic-shaping subsystem to allocate bandwidth resources efficiently and according to a specified policy. The second part of the chapter explores how to make sure your resources stay available by using the redundancy features offered by the CARP and pfsync protocols.
ALTQ, short for ALTernate Queueing, is a very flexible mechanism for network traffic shaping, which lived a life of its own before it was integrated into PF on OpenBSD.[36] On OpenBSD, ALTQ was integrated into the PF code for the OpenBSD 3.3 release by Henning Brauer, with the configuration done in pf.conf mainly because it makes sense to integrate traffic shaping and filtering. PF ports to other BSDs followed suit, with at least some optional ALTQ integration.
Managing your bandwidth has a lot in common with balancing your checkbook or handling other resources that are either scarce or available in finite quantities. The resource is available in a constant supply with hard upper limits, and you need to allocate the resource with maximum efficiency, according to the priorities set out in your policy or specification.
The core of ALTQ bandwidth management is the queue concept. Queues are a form of buffers for network packets. The packets are held in a queue until they are either dropped or sent on their way according to the criteria that apply to the queue, and subject to the queue’s available bandwidth. Queues are attached to specific interfaces, and bandwidth is managed on a per-interface basis, with available bandwidth on a given interface subdivided into the queues you define.
Queues are defined with a specific amount of bandwidth, a specific portion of available bandwidth, or sometimes hierarchical priority. Priority in this context is an indicator of preference or which queue should be serviced with the shortest delay. Some queue types can even be configured with a combination of bandwidth allocation and priority. For even further refinement, with some queue types, you can allocate portions of each queue’s bandwidth share to subqueues, or queues within queues, which share the parent queue’s resources.
Once queue definitions are in place, you integrate traffic shaping into your rule set by rewriting your pass or match rules to assign traffic to a specific queue.
Note
In ALTQ setups, any traffic that you do not explicitly assign to a specific queue gets lumped in with everything else in the default queue.
In the default networking setup, with no ALTQ-style queueing, the TCP/IP stack and its filtering subsystem process the packets in order as they arrive on an interface. This is what we generally refer to as the first in, first out (FIFO) discipline.
ALTQ queues can be set up to behave quite differently, sometimes with startling effect. Each of the three queue scheduler algorithms, or disciplines, offer their own unique sets of options. The types are priq, cbq, and hfsc.
Priority-based queues are defined purely in terms of priority within the total bandwidth. For priq queues, the allowed priority range is 0 through 15, where a higher value earns preferential treatment. Packets that match the criteria for higher priority queues are serviced before the ones matching lower priority queues.
Class-based queues are defined as constant-size bandwidth allocations, as a percentage of the total available or in units of kilobits, megabits, or gigabits per second. A cbq queue can be subdivided into queues that are also assigned priorities in the range 0 to 7, and again a higher priority means preferential treatment.
Packets are kept in the queue until the bandwidth is available. For queues that are subdivided into queues with priority as well as bandwidth allocations, packets that match the criteria for a higher priority queue are serviced sooner.
The hfsc discipline uses the Hierarchical Fair Service Curve (HFSC) algorithm to ensure a “fair” allocation of bandwidth among the queues in a hierarchy. HFSC comes with the possibility of setting up queueing regimes with guaranteed minimum allocations and hard upper limits. You can even have allocations that vary over time, as well as fine-grained priority with a 0 through 7 range.
Both the algorithm and the corresponding setup are fairly complicated, with a number of tunable parameters. For that reason, most ALTQ practitioners stick with the simpler queue types. Yet the ones who claim to understand HFSC pretty much swear by it.
Enabling ALTQ so you can use the queueing logic in your rule sets may require some extra steps, depending on which operating system is your platform of choice.
On OpenBSD, all supported queue disciplines are compiled into the GENERIC and GENERIC.MP kernels. The only configuration you need to do involves editing your pf.conf.
On FreeBSD, you need to check that your kernel has ALTQ and the ALTQ queue discipline options compiled in. The default FreeBSD GENERIC kernel does not have ALTQ options enabled, as you may have noticed from the messages you saw when running the /etc/rc.d/pf script to enable PF. The relevant options are listed here:
options ALTQ options ALTQ_CBQ # Class Bases Queuing (CBQ) options ALTQ_RED # Random Early Detection (RED) options ALTQ_RIO # RED In/Out options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC)
options ALTQ_PRIQ # Priority Queuing (PRIQ) options ALTQ_NOPCC # Required for SMP build
The ALTQ option is needed to enable ALTQ in the kernel. On SMP systems, you also need the ALTQ_NOPCC option. Depending on which types of queues you will be using, you need to enable at least of one of ALTQ_CBQ, ALTQ_PRIQ, or ALTQ_HFSC. Finally, you can enable the congestion-avoidance techniques Random Early Detection (RED) and RED In/Out with the ALTQ_RED and ALTQ_RIO options, respectively. See the FreeBSD Handbook for information about how to compile and install a custom kernel with these options.
ALTQ was integrated in the NetBSD 4.0 PF implementation and is supported in NetBSD 4.0 and later releases. NetBSD’s default GENERIC kernel configuration does not include the ALTQ-related options. However, the GENERIC configuration file comes with all relevant options commented out for easy inclusion. The following are the main kernel options:
options ALTQ # Manipulate network interfaces' output queues options ALTQ_CBQ # Class-Based Queueing options ALTQ_HFSC # Hierarchical Fair Service Curve options ALTQ_PRIQ # Priority Queueing options ALTQ_RED # Random Early Detection
The ALTQ option is needed to enable ALTQ in the kernel. Depending on which types of queues you will be using, you need to enable at least of one of ALTQ_CBQ, ALTQ_PRIQ, or ALTQ_HFSC.
Using ALTQ requires that you compile PF into the kernel. The PF loadable module does not support the ALTQ functionality. Check the NetBSD PF documentation at http://www.netbsd.org/Documentation/network/pf.html for the most up-to-date information.
[36] The original research on ALTQ was presented in a paper for the USENIX 1999 conference. You can read Kenjiro Cho’s paper “Managing Traffic with ALTQ” online at http://www.usenix.org/publications/library/proceedings/usenix99/cho.html.