Check Point has an extensive set of clustering options which have evolved throughout the years. In the early days, only high availability was available, and full load balancing required the use of third-party software such as StoneBeat FullCluster or RainWall. Today, Check Point offers a clustering feature set called ClusterXL, which has both high availability and load-sharing capabilities. ClusterXL also handles state synchronization to maintain connections in case of a failover.
In addition to Check Point’s ClusterXL, several OPSEC-compliant third-party clustering solutions are available. This chapter details the ClusterXL feature set and introduces some of these third-party clustering solutions. Whether you have a large or a small network, the fact that you rely on uptime to allow your customers access to your protected applications is critical. In fact, any sort of network downtime is almost unacceptable today. Fortunately, as demand for high availability increases, it is becoming easier to set up this capability and ensure optimal network performance for your customers.
ClusterXL is a software-based solution for Check Point gateways that offers both active/ passive and active/active high availability. It operates in two different modes. In High Availability mode, one gateway is active while the other is passive. In this sense, active means the gateway is processing traffic, and passive means the gateway is only in a backup state and is not processing packets. In Load Sharing mode, both firewalls are active in the sense that they are both processing traffic at the same time. Figure 8.1 shows the different capabilities of each mode.
To set up ClusterXL, you must first determine which mode you want to use on your network, and you need to make sure you meet the prerequisites for ClusterXL. The prerequisites include a distributed installation, the same operating system, the same Check Point versions, and a proper license to activate ClusterXL. The use of the same NTP server for each cluster member is highly recommended, because it’s all about timing. If a member goes down, you want to ensure that the backup gateway has the correct time so that it doesn’t drop any connections by thinking that the connection is too old.
When running in High Availability mode, you have two different recovery methods, whether you are using Legacy High Availability or New Mode High Availability (both which we will cover later in this chapter). Whenever failover occurs, traffic is processed to the backup node, which then becomes the active member. When the faulty original active member recovers from its failed state, you can decide whether you want it to resume as active or to stay as passive. Figure 8.2 shows this decisive configuration option. In the figure, whichever member is first in the list receives a primary status by default. The second member in the list is the standby machine.
In Figure 8.2, Corporate-Cluster-1-member-A is listed first, and therefore it has primary status; Corporate-Cluster-1-member-B has standby status. You can change the order of these members by selecting the gateway in question and then selecting Increase Priority or Decrease Priority. You may decide to give a certain member priority over another simply because one machine has a quicker CPU and more RAM, or because you simply prefer one over the other.
The Cluster Control Protocol (CCP) is a proprietary protocol that Check Point developed for ClusterXL. It is used to transfer the stateful information from each firewall so that they can be synchronized in case of failure. CCP uses User Datagram Protocol (UDP) port 8116; if you have any issues with state synchronization, verify that your firewall rules permit this traffic even though it should be allowed through the implied rules. CCP is also used to send members’ state information regarding link failures and overall status. It is important for each member to be aware of the other to ensure a smooth transition in case of failure.
CCP is passed along the sync network. The sync network should be a dedicated network and shouldn’t have any other traffic passing along it. Heavy amounts of traffic will be passed on this network, and other types of traffic shouldn’t be able to interfere with it.
Legacy High Availability mode used to be the default mode for Check Point, but it isn’t really used anymore. Instead, Check Point offers the New Mode High Availability mode, starting with NG FP3 (Version 5.3), and today this mode is highly recommended for new installs. However, if you are still working with Legacy High Availability mode (and we hope few people are), you should be aware that the gateways will share the same Internet Protocol (IP) addresses and Media Access Control (MAC) addresses with each other. Also note that there are some switch (Layer 2 forwarding) considerations that you need to be aware of prior to configuration. Historically, most switches have had difficulties with multicast packets, so you’ll need to ensure that your switch is capable.
New Mode High Availability mode is different from Legacy High Availability mode and is the setup of choice for most administrators. New Mode is easier to set up and doesn’t require any complicated devices. In New Mode, each gateway has a unique IP address and uses the Virtual Internet Protocol (VIP). To visualize the notion of a VIP, picture that .1 is gateway A and .2 is gateway B. The VIP will be .3 and will be shared between .1 and .2 depending on who is the active member. The active member will respond to all Address Resolution Protocol (ARP) requests for the VIP with its own MAC address. The VIP will most often be the default gateway for all your devices internally. When using the VIP as the default gateway for your devices, they will never know who is actually processing traffic. When the active member fails, the VIP will be moved to the passive gateway which will now take the role of the active member. New Mode High Availability uses Unicast in its communication process through broadcast messages, and sometimes uses gratuitous ARP to advertise who will be responding to the VIP.
When working with New Mode High Availability, you can see who is responding to the VIP by connecting to the switch it faces and looking at the ARP entries. We suggest that you try this to test for failover. When a failover occurs and you verify the ARPs once again, the ARP for the VIP should be given the MAC address of the new active member.
Load sharing is the ultimate challenge, and once this capability is properly configured it is the quickest solution around whether you are using Multicast or Unicast. With Multicast, regardless of who has requested the VIP address, the request will be sent to all members of the cluster. Once the Multicast packet hits the cluster members, the members will decide who will process the packet. This decision factor is the core of ClusterXL and is proprietary to Check Point.
In Multicast mode, when a packet arrives through gateway-a and then returns through gateway-b, the Check Point state information will permit the traffic to go through because it is in the state table.
The difference between Unicast mode and Multicast mode is that in Unicast mode, a single cluster member processes all initial packets. Check Point identifies this member as being the Pivot machine. The Pivot machine accepts all initial traffic and decides which member should process it. The Pivot machine takes a slightly heavier load than the remaining members because it handles all initial requests. The Pivot machine decides who should process the packets by using a proprietary decision algorithm.
The Pivot machine is an active member in the sense that it is participating in the processing of packets inbound and outbound. It doesn’t sit idle; however, it can assign itself fewer connections to process.
Configuring ClusterXL requires some planning prior to implementation. Figure 8.3 represents the basic four-network topology we will use in the rest of this chapter. Figure 8.3 shows an external network, an internal network, a DMZ, and a sync network. The three-line connector for each network except for the sync network consists of member-A, member-B, and the VIP for the network in question. The sync network does not have a VIP because it communicates directly to each other and nobody else.
Once you have decided on the type of clustering method you want to use, you must activate ClusterXL. You can configure ClusterXL when you’re installing the gateway. During gateway installation, ClusterXL will ask whether the gateway will participate in a cluster. If you answer no to that question, you don’t have to reinstall the software. If you don’t want to configure ClusterXL when you’re installing the gateway, you can activate it at a later time by using the command cpconfig. One of your choices with cpconfig is to enable cluster membership, as shown in the following code snippet:
[member-A]# cpconfig This program will let you re-configure your Check Point products configuration. Configuration Options: ––––––––––––––––––––– (1) Licenses (2) SNMP Extension (3) PKCS#11 Token (4) Random Pool (5) Secure Internal Communication (6) Enable cluster membership for this gateway (7) Automatic start of Check Point Products (8) Exit
Enter your choice by typing the number that corresponds to the option you want to configure.
Note that enabling cluster membership at this point will require a reboot, which you must do on all members that will participate in the cluster. Prior to configuring the remaining options (which you will do with the Dashboard), you should double-check your member interfaces to ensure that the netmask and the default gateway are identical. If they are not, this will cause some topology errors later when you create the Check Point cluster object within the Dashboard.
Once you have configured each gateway member at the operating system level, it’s time to define the objects within the Dashboard. You have the option to either create the cluster object by first creating two separate gateways and then adding them to the cluster object, or creating the cluster object and then defining the two gateways. Figure 8.4 displays the creation of the cluster and then adding the gateway members to the cluster by selecting Manage | Network Objects | New | Check Point | VPN-1/UTM Cluster | Simple Mode.
Notice in Figure 8.4 the information you are asked to provide. The first field you must fill in asks for the cluster name, which in this example is syngress-cluster. Keep in mind that the cluster name should be relevant to your environment or location; if several Check Point objects are defined in your SCS, having logical names will help you to know where the Check Point is physically located. Some international examples for naming conventions that seem to work well are airport codes. Airport codes are already defined, and most cities, even small ones, have an airport close by.
The second field that you must fill in asks for the main cluster’s IP address. The cluster’s IP address should be the external routable address, wherever the default gateway points to. You can create Check Point objects with the external routable IP address for virtual private network (VPN) certificate exchanges and secure internal communication (SIC) activation keys.
The third field you must fill in asks for the type of clustering method being used. The default method is via ClusterXL; however, you can use other solutions, including Nokia VRRP, Nokia IP Clustering, and OPSEC. OPSEC covers a wide range of third-party solutions. Visit www.opsec.com for a complete list of Check Point’s OPSEC-certified partners.
Once you have filled in these fields, click Next; this will bring you to the Cluster members’ properties page, as shown in Figure 8.5.
The Cluster members’ properties page is where you have the option to add gateway members. Clicking Add brings you to two options: New Cluster Member and Add Gateway to Cluster. If you haven’t already defined your gateway objects within the Dashboard, select New Cluster Member and enter the IP address and the SIC activation key. If your gateway already exists on the SCS, when you add the gateway to the cluster, the cluster will inform you that the gateway will inherit new properties from the cluster. This is more of an information message than a warning. Whichever way you feel comfortable adding members to the cluster, keep in mind that you can always add or remove them later on.
At this point, you should have a cluster object with a minimum of two gateways participating in the cluster. The next step is to configure the topology information for the cluster. The topology definition for the cluster is essential with ClusterXL. To configure the cluster topology options, you must select your cluster object through the Dashboard and then select Topology | Edit Topology. Figure 8.6 shows the cluster topology information for our cluster object, Corporate-Cluster-1.
To receive information similar to that shown in Figure 8.6, select Get all members’ topology. Within the Edit Topology section of the cluster object are five columns that you can modify.
The first column, labeled “Network Objective,” is where you set the objective of the defined network interface. You can choose from the following: Cluster, Cluster 1st Sync, Cluster 2nd Sync, Cluster 3rd Sync, 1st Sync, 2nd Sync, 3rd Sync, Monitored Private, and Non Monitored Private.
The Cluster option is the most common option because it will represent the cluster’s VIP. This Cluster IP is the IP address that will be shared among the cluster members. The Cluster IP has to be different from any other gateway member interface.
The 1st Sync, 2nd Sync, and 3rd Sync options are the synchronization network definitions that the two gateways will use to pass traffic to each other regarding state information and cluster member status. The synchronization network has all the state information and is extremely important for the functionality of your cluster. As noted earlier, the sync network should be on a dedicated network and should not have any other traffic passing on it. The 2nd Sync and 3rd Sync options give you the option of using a second or third network for synchronization.
The Cluster 1st Sync, Cluster 2nd Sync, and Cluster 3rd Sync options aren’t recommended because they leave an unsecured option for using an internal cluster interface to serve as the sync network. You don’t necessarily want to use a network for Sync and Cluster; however, the option remains should your environment require it.
The Monitored Private option is not related to ClusterXL and no VIP will be used on this network. Devices that are placed on this network will not be using a VIP as their default gateway, but rather the actual IP address of the cluster member(s).
The Non-Monitored private option also isn’t related to ClusterXL and will be used only if you do not want to have an interface/network monitored or offer high availability.
In addition, the Private option is not related to ClusterXL; however, a third-party cluster configuration such as Nokia can use it. Depending on your third-party requirements, you may have to set this Private network which will most likely be used for the state sync that the third party utilizes.
In the second column, you can input different options depending on what you chose in the Network Objective column. If you chose Cluster, 1st Sync, 2nd Sync, or 3rd Sync, you can use a logical name, IP address, and subnet information. If you selected any other option, you are not required to input any information in this column.
The third and fourth columns contain the topology interface information for each cluster member, and any discrepancies should be fixed first on the OS layer. At that point, you can try to redo a Get all members topology information so that it corrects itself.
The fifth column, which is labeled “Topology,” will populate itself based on calculated topology request information from the cluster.
Once you have created the cluster object, you are ready to configure the security rules. The security rules won’t require any special modifications, unless you notice that the CCP is being blocked, in which case you may have to enable a specific rule to allow the traffic to go through. The security rules will work in conjunction with your network address translator (NAT) rules, where you’ll want to use Hide NAT for your internal networks and the cluster external IP as the hidden address.
Having the ability and the knowledge to monitor the cluster is crucial. You should be informed when the cluster has switched from a high-availability state, or simply whether a member is down. Figure 8.7 displays the ClusterXL SmartView Monitor, which provides the overall status for Corporate-Cluster-1-member-A. The green checkmarks in the ClusterXL section of the figure indicate that everything is of. This section also provides basic information such as the cluster’s working mode and member state; click More, and you’ll receive the member ID with the number of connections and interfaces.
Using the limited free features of SmartView Monitor will give you a graphical representation of the current cluster status for all members individually. To verify the status of the cluster members use the cphaprob stat command. You must run this command separately on each cluster member. If you run it without any parameters, you will receive a help menu, as shown in the following example:
[Expert@member1]# cphaprob Usage: cphaprob state cphaprob [-a] if The following commands are NOT applicable for 3rd party: cphaprob -d <device> -t <timeout(sec)> -s <ok|init|problem> [-p] register cphaprob -f <file> register cphaprob -d <device> [-p] unregister cphaprob -a unregister cphaprob -d <device> -s <ok|init|problem> report cphaprob [-i[a]] [-e] list cphaprob igmp .................. IGMP membership status cphaprob [-reset] ldstat ....... Sync serialization statistics cphaprob [-reset] syncstat ..... Sync transport layer statistics cphaprob fcustat ............... Full connectivity upgrade statistics cphaprob tablestat ............. Cluster tables [Expert@member1]#
The most common command for viewing the status of your cluster member is cphaprob state. The following will result in the command from each active cluster member:
[Expert@member1]# cphaprob state Cluster Mode: New High Availability (Primary Up) Number Unique Address Assigned Load State 1 (local) 10.100.0.1 100% Active 2 10.100.0.2 0% Standby [Expert@member1]# [Expert@member2]# cphaprob state Cluster Mode: New High Availability (Primary Up) Number Unique Address Assigned Load State 1 10.100.0.1 100% Active 2 (local) 10.100.0.2 0% Standby [Expert@member2]#
The cphaprob state command shows you who is active (depending of the cluster mode) and who is on standby. If the cluster was using load sharing, it would indicate as such in the Cluster Mode section.
A major advantage that Check Point gateways have over those of other manufacturers is that Check Point has always offered administrators the ability to work with other specialized manufacturers through the OPSEC Alliance. Check Point’s core business is stateful inspection, with its VPN-1/FireWall-1 product line. Since its formation in 1997, Check Point has partnered with several manufacturers specializing in clustering technologies, including Rainfinity (RainWall), Stonesoft (StoneBeat FullCluster), and Nokia (VRRP).
Over time, partners offering third-party clustering solutions have come and gone; however, Check Point has continued to develop its own technologies, and although its NGX R65 product lacks some of the features of past legends, such as the RainWall product suite, it is simple to set up. Nevertheless, the loss of partners such as Rainfinity has enabled other manufacturers to enter the market and test themselves. Two of these manufacturers, Resilience and Crossbeam, offer another approach to clustering solutions that includes specialized hardware and software mechanisms. We discuss each of these solutions in the following sections.
The Resilience solution for VPN-1/FireWall-1 is an appliance and software-based solution. The appliance offers a combination of Check Point VPN-1/FireWall-1 and Resilience hardware and software, designed to provide integrated High Availability (iHA).
Each gateway appliance module is a self-contained computer comprising a motherboard, interface cards, a power supply, and a disk drive. It’s basically an Intel system with preconfigured sync networks. Different Resilience solutions provide one or two different synchronization networks depending on the size of your enterprise.
Third-party solutions almost always use a modified hardened flavor of the Linux operating system, and this is true with the Resilience product. You may view this as a good thing because you can modify several parameters; however, not always supported by them. The Check Point software is preloaded on the appliances and only needs to be activated according to the type of high availability you will be performing. Resilience offers two different high-availability systems: iHA Hot Standby and iHA with ClusterXL.
The iHA Hot Standby system combines two appliances and Resilience iHA to provide a hot standby solution with the ability to simply fail over either manually or automatically. The nice thing about this is that it requires only one Check Point VPN-1 license because the SCS sees the Resilience appliances as a single system.
The iHA with ClusterXL system provides full high availability because both members are active. It combines the Resilience infrastructure with Check Point’s ClusterXL to provide full state synchronization and automatic failover with no sessions lost.
The Check Point/Nokia alliance has existed for a long time. For those of you still using the uber-popular IP330 or IP440 appliance, it’s definitely time to change! For starters, those classic but super-old machines are slow; furthermore, the IP440 doesn’t support IPSO 4.2.
IPSO is the Intel-based operating system used in Nokia appliances. IPSO is derived from FreeBSD and was created by Ipsilon Networks, which Nokia acquired in the late 1990s. The names of the Nokia appliances start with the letters IP, followed by numbers; the higher the number, the faster the appliance. The Nokia version of clustering uses the Virtual Router Redundancy Protocol (VRRP). VRRP is not a proprietary protocol in that any system that is running Linux can use it. Nokia provides an easy-to-use interface in the creation of virtual routers which the Check Point cluster will recognize as VIPs. Figure 8.8 shows the creation of the VRRP, which you can access by connecting to the Nokia appliance through HTTP(S).
Crossbeam offers two hardware appliance product lines: the C-Series and the X-Series. The C-Series provides an integrated solution that allows for a set of applications to run on the appliance simultaneously. The flagship X-Series solution is a bladed solution that also allows for multiple applications to run on one appliance. The X-Series Crossbeam appliance is one of the fastest Check Point solutions available, in that it can hardware-accelerate traffic once the Check Point rule base has accepted a connection. For more information, go to http://crossbeamsystems.com/.
If your corporation does not require a cluster and decides to go with a single physical machine, you may want to look at the ClusterXL ISP Redundancy option. For example, if your Internet service provider (ISP) link goes down, no matter how many gateway members you have you won’t have Internet access. To use ISP Redundancy you must be running Red Hat Linux 7.2 or later, SecurePlatform, and/or IPSO.
The configuration options for the ISP Redundancy feature require that you have two different default gateways; however, you need to enter only one default gateway, the primary ISP. When configuring the external interfaces of your gateway, decide which one will be used as the primary ISP. The primary ISP is normally the fastest link, but it could be the least expensive link. Ensure that a default route is added toward the primary ISP. When configuring the second interface of the gateway’s ISP, do not add a second default route. The Check Point ISP redundancy script will handle switching the default route.
To configure Check Point ISP Redundancy you have to modify the gateway object. From the gateway object, select Topology | ISP Redundancy, as shown in Figure 8.9.
Within the ISP Redundancy configuration page you have two different modes that you can configure: Load Sharing and Primary/Backup. In a load-sharing environment, both ISPs will be used, whereas in a primary/backup environment, you must define a primary ISP where all traffic will default to and a backup ISP in case the primary link goes down. When you select Primary/Backup, the order of priority is determined by whichever link will be displayed first.
Figure 8.10 displays the configuration options in Primary/Backup mode. Note that we have given the name ISP1_10MB for the primary link and ISP2_ADSL for the backup link. If you want to switch the order of priority, select the link in question and move its priority by using the Up-arrow key or the Down-arrow key.
The Check Point redundancy script can monitor devices by sending Internet Control Message Protocol (ICMP) packets from each link. You can configure this by selecting the link in question and then selecting Edit | Advanced tab, as shown in Figure 8.11.
Within the ISP link’s Advanced tab, you can specify which hosts you want to monitor. If these hosts stop responding to ICMP echo requests sent from the gateway, the link in question will be considered down and the gateway will switch to ISP2 for all traffic. When the gateway decides to switch gateways, the ISP redundancy script will issue a new default route automatically. You can find the default route on the General tab of the ISP Link properties window. The General tab is where you can set the next hop IP address, which is the new default gateway should you have a change in ISPs.
You also can enable or modify a few different settings for tracking options. For instance, you can track the ISP link failure and ISP link recovery. The options are the same general settings which Check Point comes bundled with.
Note
Once you configure the ISP Redundancy feature, you must check that it’s working properly. You can test it by removing a physical interface or via the command line. The command line will simulate a link failure, thus forcing traffic through the second link.
The fw isp_link command will simulate a failure, as exemplified in the following code snippet:
fw isp_link Corporate-gw ISP1_10MB down
By itself, the fw isp_link command will display the proper usage in case you forget the proper arguments. In terms of the preceding example, keep the following arguments in mind:
Corporate-gw is the object name of the gateway.
ISP1_10MB is the name of the primary ISP link that is active.
down sets the link status to down so that it switches to the secondary ISP.
Testing the ISP failure should produce excellent results, and you should find that you’re not losing any packets. A good test is to start a File Transfer Protocol (FTP) transfer with the hash marks on before conducting the simulated failure. When the transfer begins, the hash marks will be displayed, thus showing the progress of the transfer. When switching, the hash marks should pause for about a second and then continue the transfer. Whether you are in a VPN or not, the failover should happen smoothly.
A complete ISP redundancy solution will require that you maintain your domain name system (DNS) Start of Authority (SOA). When clients make inbound connections toward your Web server, for example, it will first do a DNS query to see what IP it belongs to. If you are not managing your DNS server, the responsible must be aware of the new ISP link and netblock. Even if the DNS server is hosted elsewhere, to what IP will they be redirecting clients? ISP1 or ISP2? How will they know which link is up or down? Check Point resolves this important point by providing a built-in mini DNS server, as shown in Figure 8.12. To configure the built-in DNS server to respond to public DNS requests, select Enable DNS proxy | Configure.
Figure 8.12 lists the DNS entry for the Web page. Notice that we must enter the full host name and the public routable IP from each ISP. Using the preceding example, if we were in a Primary/Backup configuration and ISP1 was active, anybody querying for www.syngress.com would be directed to 143.100.75.10. Should ISP1 fail, all DNS requests would receive 172.16.2.10, assuming that this was a routable IP address. For this reason, it is important to advise your ISP DNS administrators to change the DNS SOA for your domain to each physical IP address of the gateway. When you do so, all DNS requests will go to the gateway and a security rule must be enabled to allow the query through. In addition, all security rules and, more important, NAT rules must be created for each ISP segment to allow inbound traffic to each destination.
 | High Availability mode enables one active member at a time. |
| When the active member fails, traffic is taken over by the secondary member. |
| If the primary member resumes from a failover, you have the option to allow it to regain control or to stay in standby mode. |
| Load Sharing mode must be properly thought out for and symmetric routing must be taken into consideration. |
| Unicast Load Sharing uses a Pivot cluster member through which all traffic is routed. |
| Multicast Load Sharing sends all ARP requests to all cluster members. |
| Third-party solutions provide various solutions for high availability and load sharing. |
| The Nokia solutions require specific hardware and use VRRP. |
| You configure Nokia clusters through Nokia Web Voyager access. |
| ISP redundancy is built into ClusterXL via SecurePlatform. |
| All ISP redundancy configuration options are made through the gateway object defined through the Dashboard. |
| Primary/Backup is the simplest ISP redundancy solution and is easily configurable and reliable. |
| Check Point provides a built-in mini DNS server to respond to inbound DNS requests to ensure full inbound ISP redundancy. |