Gateway load balancing requires that you set up at least one gateway group, which consists of two or more WAN interfaces. This involves several steps:

1. Adding WAN interfaces and configuring them
2. Configuring DNS servers for each of the newly-created WAN interfaces
3. Forming gateway groups and adding the new interfaces to them
4. Adding firewall rules for each new gateway group

Adding and configuring additional WAN interfaces is not difficult. When you first set up pfSense, the WAN interface is automatically created and configured in the setup wizard. Configuring additional WAN interfaces is not that much different. It involves the following steps:

1. Navigate to Interfaces | (assign), and on the Interface Assignment tabs, there will be a tab showing all the existing interface assignments—at a minimum, the WAN and LAN interfaces will be listed here.
2. Select an unused network interface from the Available network ports drop-down box and click on the Add button corresponding to that drop-down box (on the right side of the row). This will add a new interface; the interface will have a generic name at first, such as OPT1. Hover over the leftmost column in the table and click on the new interface's name.
3. You can also select the new interface from the Interfaces menu.
4. On the interface configuration page, check the Enable checkbox. Enter an appropriate description for the interface in the Description field; this will become the interface's new name.
5. The IPv4 Configuration Type and IPv6 Configuraton Type drop-down boxes are where you select the appropriate configuration types for the interface IP. If the interface will be receiving an IP address from your ISP—and this likely represents the majority of cases—then the correct selection is either DHCP, DHCP6, or SLAAC for IPv6. If you choose either of these options or both, then there isn't much more you have to do as far as interface configuration goes. pfSense will automatically set up this interface as a gateway.

6. On the other hand, if you chose Static IPv4 and/or Static IPv6, you will have to manually configure the new interface as a gateway. To do so, scroll down to the Static IPv4/IPv6 Configuration section of the page and click on the Add new gateway button.
7. Clicking on this button will launch a dialog box; this will allow you to configure the most basic options for the gateway. Since the first WAN interface is already the default gateway, you should leave the Default gateway option disabled.
8. In the Gateway name edit box, you should give the gateway an appropriate name and enter a gateway IP address in the Gateway IPv4 or Gateway IPv6 edit box. The IP address of the gateway should be different compared to the interface’s IP address, but on the same subnet as the interface.
9. Enter a brief description in the Description field.
10. Click on the Add button when you're done. There isn't much more that needs to be done for the secondary WAN configuration, unless you have some other options you need to enter that are specific to your connection. For example, you may need to configure advanced DHCP options; if you have a PPP or PPPoE connection, you may have to enter a username and password.
11. You should probably check the Block private networks and loopback addresses checkbox and the Block bogon networks checkbox (to block non-IANA-assigned networks), as blocking such networks on WAN interfaces is a good practice. When you are done, click on the Save button at the bottom of the page, and when the page reloads, click on the Apply Changes button.
12. Repeats steps 1 through 11 for as many WAN-type interfaces as you have. If you are configuring interfaces for use with different ISPs, the options you choose in each case may be different.

Next, you must configure the DNS settings for each of the newly-created gateways. To do so, perform these steps:

1. Navigate to System | General Setup.
2. On the General Setup page, in the DNS Server Settings section, you should enter a DNS server for each of the new gateways (you can choose the gateway in the drop-down box adjacent to the edit box where you enter the IP address of the DNS server). There should be at least one unique DNS server per gateway in a multi-WAN setup. This is because we are looking to eliminate single points of failure in our configuration.
3. When you have finished configuring the DNS settings, click on the Save button at the bottom of the page.
4. Navigate to System | Routing and begin gateway configuration.

5. The Gateways tab should be the default tab, and on this tab, the newly created gateways should be listed in the table. The gateways will have the names you assigned to them when you created the gateways in the previous series of steps, but only if you configured the gateways manually. Otherwise, they will have names such as WAN2_DHCP or WAN2_DHCP6 (for gateways with interfaces that get their IPs from the ISP), and so on:
   • Two other options on the Gateways tab, new to pfSense 2.4, are the Default gateway IPv4 and Default gateway IPv6 dropdown boxes. This replaces the Default Gateway checkbox on the gateway Edit page, and provides an easy way of selecting and changing the default gateway.
6. Click on the Edit icon for any of the newly created gateways, and you will notice that there are many more options than there were in the dialog box when you click on the Add new gateway button on the interface configuration page:
   • The Disable this gateway option allows you to save the gateway configuration while forcing the gateway offline. This can be useful for troubleshooting.
   • The Interface drop-down box allows you to change the interface being configured.
   • Address Family lets you choose between IPv4 and IPv6 addresses.
   • Gateway allows you to enter the gateway IP address. If the interface is configured to use DHCP or DHCPv6, this field will be read-only.
   • There is a Default Gateway option which allows you to set this gateway as the default.
   • There is a Disable Gateway Monitoring option, which, if enabled, causes pfSense to consider the gateway to always be up.
   • There is a Monitor IP field, in which you may enter an alternative IP address to monitor the gateway. To determine whether a gateway is up or down, pfSense first pings a gateway; however, having the gateway ping a remote address is often a better indicator of whether a gateway is up. Therefore, if a gateway fails to respond to a ping and a monitor IP is specified, pfSense will have the gateway ping the monitor IP. It is useful to have this as a fallback when a gateway doesn’t respond to this ping; therefore, you should probably configure this option. To do so, enter a non-local IP address to ping. If you cannot think of a reliable site to ping, just enter the DNS server's IP address.
   • If the Mark Gateway as down option is enabled, pfSense will consider the gateway to be down.
   • There is a Description field where you can enter a brief, non-parsed description.
   • The Advanced section, which displays when you click the Display Advanced button, also has some interesting options. The Weight drop-down box allows you to select the weight for the gateway within a gateway group; higher numbers indicate that the gateway has more weight. As an example, consider a gateway group in which one gateway has a weight of 1 and the other gateway has a weight of 2. The gateway with a weight of 2 will have twice as many connections going through it than the gateway with a weight of 1.
   • The Data Payload field lets you set the -s parameter of the ping command. This allows you to set the number of data bytes to send (the default is 1).
   • The Latency thresholds edit boxes are the fields you can set to determine the low and high thresholds for latency in milliseconds. If the low threshold value is met, pfSense sends an alarm, while if the high threshold value is reached when pinging, the gateway’s status will be set to down. The defaults of 200/500 should be fine in most cases. There may be other cases where setting the right latency is critical, however, for example, you may have a high latency connection such as a satellite internet connection, and in such a case, you don't want the gateway's status changing to down when the connection is still good.
   • The Packet Loss threshold fields are where you can specify the low and high thresholds for packet loss. As with Latency, if the low threshold value is met, an alarm is sent, while if the high threshold value is met, the gateway is set to down.
   • The Probe Interval field lets you specify how often an ICMP ping probe is sent; the default value for this parameter is 500 ms.
   • The Loss Interval is the allowed latency of ping replies—this is the time interval allowed to lapse before packets are considered lost. The default is 2,000 ms.
   • The Time Period is the period over which results are averaged for packet loss. The default is 60,000 ms.
   • The Alert interval field lets you enter the time period between checking for an alert condition. The default is 1 second.
   • The final option on the page is Use non-local gateway. If this option is enabled, a gateway outside of the interface’s subnet can be used.
   • Click on the Save button when you have finished making changes, and then click on the Apply Changes button on the main Routing page.

Once the gateways are configured, we can set up the actual gateway group. This can be done by following this procedure:

1. From the main Routing page, click on the Gateway Groups tab. This tab will display a table that shows all the configured gateways (it should be empty at first).
2. Click on the Add button at the bottom of the table to add a new gateway.

3. On the Edit page for the gateway group, you have a few options:
   • In the Group Name field, enter the name of the group.
   • Under Gateway Priority, you have two configurable options, each with its own set of drop-down boxes. The Tier drop-down box allows you to select the tier on which a gateway exists. You might recall that with tiers, lower numbers have priority over higher numbers. Therefore, gateways on the same tier are load balanced with each other, while gateways on a higher tier are inactive as long as a gateway on a lower tier is still up (a failover setup). If we set WAN_DHCP to Tier 1 and WAN2_DHCP to Tier 2, then WAN_DHCP will get all the traffic for the gateway group while WAN2_DHCP will not be used unless WAN_DHCP goes down. But if both WAN_DHCP and WAN2_DHCP are set to Tier 1, both gateways will be active at the same time and handle a share of the total traffic. Set the tiers to the same or different values depending on whether you want a load balancing or failover gateway group.
   • The Virtual IP drop-down box is where you can select the virtual IP for each gateway; however, this only applies to instances in which the gateway group is used as an endpoint for a local Dynamic DNS (DDNS), IPsec, or OpenVPN connection. In most cases, you can leave it set to Interface Address.
   • There is only one Trigger Level drop-down box for each gateway group. Here, you can specify when to trigger the exclusion of a gateway member. You have the following choices:
     • Member down: This one is obvious. A member is excluded when it fails to respond to a ping attempt/fails to ping the monitor IP.
     • Packet Loss: Exclusion is triggered when packet loss is unacceptably high (the actual threshold is set during gateway configuration; see the previous points for more details).
     • High Latency: Exclusion is triggered when latency is unacceptably high (again, the actual threshold is set during gateway configuration).
     • Packet Loss or High Latency: In this case, a group member is excluded when either packet loss or high latency is too high.
   • You can enter a brief description in the Description field. When you are done making changes, click on the Save button, and then click the Apply Changes button on the main Routing page.

One procedure you can follow for the sake of completeness is to configure failover groups for each of the gateways (assuming you didn’t configure the first group as a failover). This is a relatively simple process:

1. On the Gateway Groups tab of the Routing page, click on the Add button.
2. In the Group Name edit box, type an appropriate name for the group (for example, FAILOVER1). In the Gateway Priority Tier drop-down box, select the appropriate tier for each connection. The first WAN connection should be set to Tier 1, and the second should be set to Tier 2.
3. In the Trigger Level drop-down box, set the desired trigger level.
4. Type an appropriate description in the Description field. Then, click on the Save button and click on the Apply Changes button.
5. Now, you want to create another failover group, only this time reverse the settings for the tiers so that they are the opposite of what they were for the first group. Click on the Add button again, or, if you want to make life even easier, click on the Copy icon in the entry for the first failover group. This will make a new gateway group, with all values defaulting to the values of the first failover group. Configure another gateway group, but set the second WAN connection to Tier 1 and the first WAN connection to Tier 2. When you are done, click on the Save button, and then on the main Routing page, click on Apply Changes.

The gateway group is now configured, but without firewall rules for the group, no network traffic will be directed through the group. You could create a rule for each interface that will be using the group, but in cases where the group will be used on more than a single network, this would seem to be a good opportunity to use a floating rule. To create a floating rule for the gateway group, follow these steps:

1. Navigate to Firewall | Rules, and from there, click on the Floating tab. To add a new rule, click on one of the Add buttons.
2. Keep the Action set to Pass; we are, after all, trying to pass traffic.
3. In the Interface listbox, select every interface that will be using the gateway group (this is potentially every non-WAN interface, but isn't necessarily so).
4. Rules involving gateways can only be one-way rules, so in the Direction drop-down box, select out (it applies to traffic leaving the interface).
5. In the Protocol drop-down box, select any.
6. The Source field should be set to an alias that refers to all the interfaces selected in the Interface listbox. If you want to keep things simple, then leave Source set to any. The Destination field can be left set to any.
7. Scroll down to the Advanced Options found in the Extra Options section. Click on the Show Advanced button to show additional options.
8. The third-to-last option will be the Gateway drop-down box. In this drop-down box, select the first gateway group you created previously.
9. Click the Save button on the main Firewall page and click Apply Changes.

Although it is easier to set up a floating rule, if you want to set up per-interface policy-based routing, you can do that as well:

1. Navigate to Firewall | Rules and click on the tab for the interface for which you want to set the policy. On that page, click on one of the Add buttons at the bottom of the page.
2. If all you want to do is create a general rule directing all interface traffic to a specific gateway, there are very few default values that need to be changed. Change the default Protocol from TCP to any, and change the Gateway to the interface that you want to direct traffic to (again, this can be found by clicking on the Display Advanced button and scrolling down).
3. If your policies are more granular, you should make the necessary changes to the rule so that the rule only matches traffic you want to be directed to the gateway.
4. You can enter a brief Description in the description field.
5. Click on the Save button when finished.
6. Although you are making specific rules for each interface, you may still want to have a floating rule as a fallback. This way, all traffic that doesn’t match the policy-based routing rules will go to the gateway group. If you create such a floating rule, make sure that the Quick option on the floating rule is disabled. Otherwise, the floating rule will take precedence over all other policy rules. As always with rules, make sure the order is correct.
7. To be completely thorough, you will need to create rules for the two failover gateway groups (if you created such groups previously). To do so, click the Copy icon in the table entry for the new rule, and create a new rule for each of the failover groups. The only changes you need to make are in the Gateway drop-down box and you also probably want to change the Description field). Click on Save when you have finished created all the necessary rules.
8. On the main Rules page, click on the Apply Changes button when you are done creating and editing rules.

In most cases, you will not have to configure static routes. This is because when a route needs to change as a result of a gateway going offline, pfSense will automatically create a temporary static route to re-route traffic to a gateway that is up. In some cases, however, we will need static routes. As an example, consider traffic that originates from pfSense. This includes such traffic as traffic from services, ping requests, and similar things. Policy-based routing only works with traffic that enters the router from the outside. It does not apply to traffic that originates from pfSense. Such traffic cannot be tagged for alternate routing. The solution in such cases is to create a static route. Fortunately, adding a static route is easy:

1. Navigate to System | Routing; from there, click on the Static Routes tab.
2. At the bottom of the page, click on the Add button. This will take you to the Edit page for the static route.
3. On this page, enter the destination network for the route in the Destination network field. This involves entering an IP address and a CIDR.
4. Select the correct gateway for the route in the Gateway drop-down box. There is also a link that allows you to create a new gateway.
5. Note that there is a Disable this static route option if we ever have to disable this route. You can enter a description in the Description field.
6. When finished, click on the Save button and then the Apply Changes button.

There are also two settings of interest available by navigating to System | Advanced, clicking on the Miscellaneous tab, and scrolling to the Gateway Monitoring section:

• The State Killing on Gateway Failure option, if enabled, causes all states to be flushed when a gateway goes down. If this option is not enabled, active states from a gateway that goes down will be transferred to other gateways in the gateway group. This may be undesirable if you don't want persistent connections to be transferred; hence, this option is available.
• The Skip rules when gateway is down option, if enabled, will change the default behavior regarding what happens to a rule specifying a gateway in cases where the gateway is down. The default behavior is to make an implicit rule without the gateway that is down. However, you may want to just skip the rule, so this option is provided.

If you completed the preceding steps, then your gateway group should be fully functional. Nevertheless, you will probably want to check to ensure that the gateway group is functional. To do so, follow these steps:

1. First navigate to Status | Gateways; there are two tabs on this page: Gateways and Gateway Groups. Stay on the Gateways tab first.
2. On the Gateways tab, there is a table showing all configured gateways. The meaning of the Name, Gateway, Monitor, and Description fields should be pretty obvious by now. The following not-as-obvious fields also contain critical information:
   • RTT (Round Trip Time): This measures the ping round trip time in milliseconds. RTT is averaged over the calculation interval specified for the gateway.
   • RTTsd (Round Trip Time standard deviation): Introduced in pfSense 2.3, this is the standard deviation of the round trip time. Once again, RTTsd is averaged over the calculation interval specified for the gateway.
   • Loss: Packet loss (over the calculation interval)
   • Status: Only two values—either online or offline
3. You can also test the gateway group monitoring by unplugging each of the WAN interfaces one by one. Then, see how long it takes for the Gateway status page to report the gateway as offline.
4. If the amount of time it takes to report a gateway as down, you may have to revisit the Gateway Groups page and adjust the Trigger Level setting for the group. You may also have to adjust the Latency or Packet Loss threshold for the gateway.