16. Through the Firewall

Something there is that doesn’t love a wall,
That wants it down.

—ROBERT FROST
MENDING WALL

Sometimes even when security isn’t a concern for a product, it can still be a nuisance. One technology in particular is adept at causing developers of Internet applications grief—the firewall. Generally, any time your application needs to run over a network, and installs network-aware code on a client’s machine, you need to consider the impact of different firewall strategies.

A common goal of system administrators is to protect naïve users from running untrusted code. A common goal of the application developer is to allow end users who want to run any application to do so. Although we certainly understand and respect the wishes of administrators, this book caters to the programmer. Therefore, in this chapter we concern ourselves with how best to make applications work with even the most draconian firewalls. We examine situations that cause firewalls to block your applications, and show you how to design around those hurdles.

If you would like a more in-depth discussion of firewall technology to better understand the context of this discussion, we recommend Cheswick and Bellovin’s Firewalls and Internet Security 2nd ed. [Cheswick, 2001].

Basic Strategies

Firewalls are typically concerned with filtering traffic between networks to enforce an access control policy for one of those networks. If you are writing a client application that needs to connect to a server, then you will generally only be worried about firewalls from the point of view of a client. People who install your server can just open up their firewall for all traffic destined to the application (although see the discussion on server proxies later).

Many firewalls are configured to allow all connections that originate from the inside. This means that you can test your software from many different networks and still not notice that it won’t work with some firewalls. This is a serious testing challenge.

Occasionally, an administrator will want to control the kinds of traffic that can originate from the local network. There are a lot of good reasons for this kind of policy. It can keep attackers who find a flaw in your network from using it as a base for further attacks. It can stop malicious Trojan horses from contacting their creators. It can also stop software you download that may not be of the best quality from exposing itself through your network. For example, imagine a client for a new streaming media format, in which the client has a buffer overflow. An attacker could present you with a bogus media clip, or inject data into a valid media clip that would exploit the overflow and run code on your machine.

There are two different ways in which administrators tend to control outbound traffic. The first is port-based packet filtering. Under this scheme, traffic destined to particular ports (which usually indicate particular protocols) is allowed to pass, but everything else is blocked.

The second way administrators control outbound traffic is by using an application proxy. In this scheme, the firewall lets no traffic cross. The internal machines are not actually capable of routing packets past the local network. However, the firewall can see both networks, and has inwardly visible proxies for supported applications. A proxy is a program that mimics the server to the client, and mimics the client to the server, acting as an intermediary. Generally, a proxy machine offers more security than simple packet filtering. Proxies are usually very simple pieces of code, written to be robust. It generally shouldn’t be possible to buffer-overflow the proxy. An attacker may get the proxy to pass on a buffer overflow to a real client. However, proxies greatly limit the risk of a successful exploit.

Note that application firewalls are a good example of a case when your clients probably do not have a valid IP address. Instead, the network behind the firewall has IP addresses that are not Internet routable. For example, machines behind the firewall may be assigned IP addresses starting with 10., such as 10.1.1.15. They will be able to reach the firewall, and perhaps other machines that are also behind the firewall. The application proxy will act on behalf of the client. Your server will see a valid client coming from a valid IP address, but this will be the IP address of the firewall.

There are other configurations in which the client can have an IP address that cannot be routed, including ones that do not involve a firewall at all. For example, an organization may be given a single IP address that can be routed on the Internet, yet have dozens of machines they wish to connect to the Internet. The standard way to do this is by using network address translation (NAT). The mechanics of NAT are very similar to those of a proxy firewall.

The practical consequence of these configurations is that your server-side application must recognize that two different clients running on two different machines can connect from the same IP address.

One of the most common strategies for clients needing to circumvent a firewall is to place servers on the HTTP port (port 80). The theory goes that everybody allows outbound HTTP traffic, so this should work well. In practice, this strategy works in most places, but is not a universal solution because application proxies do not pass through the traffic.

The next thing people tend to try is to make traffic “look” like HTTP traffic. This may work for proxies positioned on port 80, but it doesn’t easily work for proxies placed on different ports. Web browsers have options to support proxies that know the port to which they should send traffic. Of course, your application will not know a priori what port is used for HTTP proxying. You can try every single port and see which one works, but that can cause intrusion detection systems to enable (which often results in the client getting cut off from the Internet).

Additionally, even if the proxy is on port 80, many firewalls examine the traffic to make sure it is valid for the protocol. In this case, you need to wrap your connections in HTTP requests and responses. For some applications, this can be a bit tricky, considering that HTTP is a “command/response” protocol. The client always causes the server to respond; the server never sends messages out of the blue. The new protocol SOAP can help automate tunneling protocols over HTTP. However, people who find tunneling over HTTP an unwanted abuse will find it easy to detect and drop SOAP transactions.

Sending traffic to port 443 (the HTTPS [secure HTTP] port) is a more portable solution than sending it to port 80. Many firewalls let HTTPS traffic through that proxy regular HTTP requests. However, it is possible also to proxy HTTPS.

Generally we recommend tunneling through port 443 because it is a more universal solution, and because it is less susceptible to traffic analysis. Using an actual SSL connection is a good idea. A traffic analyzer can automatically detect non-SSL traffic. See Chapter 11 for information on setting up an external SSL tunnel.

When designing protocols for a client/server application, there is one important guideline that you should follow: Avoid any server-initiated connections. Proxying firewalls can deal with such a situation, even though it makes the proxy more complex. However, packet-filtering firewalls often have great difficulty with such situations. A significant difference between a packet-filtering firewall and a proxy-based firewall is that proxy-based firewalls tend to make an attempt to understand the state of an entire connection, whereas packet filters tend to look at individual packets and decide whether to let them through, saving themselves from the inefficiencies of even a limited protocol analysis. Thus, packet filters do not keep any application-level state, and cannot know when a client has agreed to open up a port for remote connections.

This problem is best illustrated by looking at the FTP protocol. In the default mode of operations, the client negotiates a file to download with the server, then the client opens up a port to which the server connects and sends the file. A packet-filtering firewall may know that is has an outbound connection to an FTP port, but when it receives an incoming connection request it has no way of knowing that the request is really part of an FTP session that has already been established. If the firewall rules are adequate, the connection request will be blocked.

Because FTP is such a commonly used protocol, most packet-filtering firewalls have a built-in proxy that can support it. However, your application is not likely to work in such an environment. You can provide your own proxy, as we discuss in the next section. However, most organizations that use packet-filtering firewalls are unwilling to install a proxy, even if it is a generic one (such as SOCKS discussed later). Therefore, you should aim to design a protocol that relies only on connections initiated by the client.

Client Proxies

As we previously mentioned, tunneling through port 443 isn’t going to make your application work with every firewall. If you want to be able to support all potential users, then you need to have your software work well with proxies. One approach to this problem is to write your own proxy server for your application. If people are interested in using your application and they are behind a proxying firewall, then they can install your proxy on their firewall.

Conceptually, an application proxy is simple. An application proxy listens for data on an internal network. When a client establishes a connection, the server opens a connection on behalf of the client on the external, Internet-enabled interface. From then on, the proxy sits in the middle, passing data back and forth between the two parties. Sometimes, the proxy rewrites data at the application level to provide the client with some privacy. For example, anonymous HTTP proxies may change any machine-specific information sent by the client to be that of the firewall. Additionally, instead of each client machine having its own set of cookies, the proxy could automatically block cookies.

Generally, proxying firewalls are UNIX machines. However, Windows machines are becoming more popular as time goes on. If you are going to write your own proxy server and wish to set things up so that your application really can run anywhere, then you should probably write a cross-platform UNIX version first and only then consider a Windows version.

Although application proxies are meant to be simple, there are a number of subtle points to consider if you wish to build a good one. Good proxy design requires that you respect the paranoia of the people who run a proxy-based firewall. You should make a conscious effort to meet their needs.

First, make your proxy as simple as humanly possible. Set things up so that someone who manually audits the software can easily make the determination that your code is trustworthy. Firewall expert Marcus Ranum suggests that your proxy should be simple enough to not need comments, because comments are “an indication that code is too complex to be trusted.”

Because your application proxy must speak to your protocol, there may be some concern with showing people code (especially if you wish your protocol to remain proprietary). In such cases we recommend that you put in place a publicly acknowledged policy to allow people to audit the proxy if they sign a nondisclosure agreement. Additionally, you should encourage the people who do audit your code to give you their assessment on review, so that you can demonstrate that many eyeballs have seen the code.

Second, you should design your code to use no special privileges. Have the port to which it binds be configurable, and default to something over 1024. Additionally, have your proxy automatically run in a chroot directory, or at least make it so that your program could easily be chroot-ed.

Third, you should log any data that may be of interest to the administrator. At a bare minimum, log the IP address of the client machine, the IP address of the remote server, the time at which the connection took place, and the time at which it ended. Additionally, you should consider logging information about each transaction. For example, if you are writing an SMTP proxy to proxy outgoing e-mail, then you may log the entire header of each outgoing message.

You should also consider supporting proxy-level authentication. That is, the client application should be required to present credentials to establish a session with the proxy. Authentication is desirable from an administrative standpoint because it improves the audit trail, especially if you also communicate with the proxy over an encrypted link. Without these features, smart users could very well spoof other people on the local network.

Similarly, you should support proxy-level access control, so that administrators can provide services selectively to those who need to use them. Such access control should support both per-user and per-address control.

Finally, you should design your application proxy to be robust. It should never crash for any reason. However, if it does crash, it should not be possible to lose any important information. For example, consider smap, an SMTP proxy. smap could forward all data from a client to the target remote server outside the firewall. Data would never need to touch the disk. It could essentially pass from one connection to another with a minimum of processing. However, for the sake of robust behavior, smap is actually a very small SMTP server that queues messages.

Server Proxies

If you wish to install servers in organizations that are highly paranoid, you should also consider writing a simple server proxy. Clients (or a suitable proxy) connect to the server proxy, which communicates with the actual server. In the ultimate configuration, the actual server would not be able to be routed. It should only be able to communicate with the proxy server, and should be required to provide machine-specific authentication credentials to communicate with the proxy.

Such a configuration is desirable, because it prevents outsiders from dealing directly with a large, complex program that probably requires privilege and may have security problems. If an attacker does manage to break the server while connecting through the server proxy, the only way to launch an attack on an external network would be to back out the proxy. Such an attack would likely be difficult if your proxy does not allow the server to initiate arbitrary connections to the outside world. The external damage a successful attacker could inflict would, at worst, be limited to those hosts connected through the proxy during a compromise. Of course, if the proxy is incapable of stopping an attack, then nothing would protect a working exploit from totally trashing the data files on the server.

SOCKS

Proxy-based firewalls have the potential to provide much more security than the typical packet-filtering firewall. The significant downside to this kind of firewall is the difficulty of getting new applications to run with them. There is a real need for a generic proxy server that can proxy arbitrary applications.

SOCKS is a protocol for proxying arbitrary TCP connections. Version 5 adds support for UDP (Unreliable Datagram Protocol) proxying as well. In this protocol, a SOCKS server accepts connections on a single port from the internal network. Clients connect to that port, provide the server with connection information, and then the server establishes and proxies the actual connection.

The end application needs to know how to speak to the SOCKS server. Some programs build in SOCKS support as an option (including, for example, Netscape and Internet Explorer). However, many applications can be run or recompiled with SOCKS support without having to change the actual code. SOCKS implementations usually come with two libraries that replace standard network calls: one that can be statically linked with a program and the other that can be dynamically linked. Both libraries replace the common C networking API with SOCKS-aware versions of the calls.

The dynamic library is useful because it allows applications to use SOCKS directly without recompiling. Additionally, one can turn off SOCKS support by removing the appropriate library from the library load path. The drawback of the dynamic library is that it does not work with setuid programs. Such programs can only work if statically compiled with SOCKS support.

SOCKS doesn’t encrypt data between the application and the server. Additionally, prior to version 5, SOCKS lacked any sort of authentication. Even now, authentication is all but worthless because of the lack of encryption.

The primary drawback to SOCKS is that any suitable application can connect through a firewall, as long as the end user links against the appropriate library. The administrator cannot prevent users from running applications on a per-protocol basis. The typical access control restrictions are based on the destination address, which can, of course, be forged.

In short, SOCKS is an outdated technology. In fact, it has largely been superseded by NAT (address masquerading). NAT on a packet-filtering firewall provides the exact same functionality that SOCKS provides, with the minor exception of authentication. Unlike SOCKS, NAT works at the operating system level, and looks like a router to other machines on the internal network. Thus, client applications do not need to be aware of the fact that they are behind a firewall.

If SOCKS were to add encryption to the server, it may be worthwhile for limiting outbound traffic. However, end users would be encouraged to take per-application authentication information and use it in another program in order to “punch” through the firewall. Because a protocol-level proxy must necessarily be generic, it cannot do protocol-level validation of a connection, and is thus a far less secure alternative to hand-rolled application proxies.

Nevertheless, SOCKS can be worth supporting, because people use it. Thankfully, it generally requires very little effort to support. The organizations that use it are usually willing to link against SOCKS libraries themselves. Therefore, you don’t necessarily have to support SOCKS directly, as long as your protocol only calls for client-initiated connections. Fortunately, SOCKS works equally easily on Windows and UNIX-based platforms.

If you wish to provide a more user-friendly on/off switch for SOCKS in your application, then you can link to the static library in such a way that the replacement calls are available under slightly different names. For example, Dante, a free SOCKS implementation for UNIX platforms (http://www.inet.no/dante/), includes the following replacement calls, all taking the same arguments as the original:

• rconnect

• rbind

• rgetsockname

• rgetpeername

• raccept

• rsendto

• rrecvfrom

• rwrite

• rsend

• rsendmsg

• rread

• rrecv

The SOCKS reference implementation is freely available for noncommercial use, but otherwise requires licensing. It is available from http://www.socks.nec.com/.

Peer to Peer

Peer-to-peer communication, a term popularized by the success of Napster, refers to client-to-client communication. For example, consider an instant message service. Generally, all users log in to a central server (or network of servers). The server tells the user which people of interest are currently connected to the server. When Alice wishes to send a message to Bob, the server may either act as an intermediary for the messages, or it may put Alice and Bob in direct contact. This second strategy is the peer-to-peer model, which is effective at removing burden from the server. Additionally, this strategy can provide privacy benefits to the end user, because potentially sensitive messages or documents can be exchanged directly between users, instead of through an intermediary that may not be trusted.

The peer-to-peer model is quite appealing. However, firewalls provide a large barrier to implementing a successful system.

In most peer-to-peer applications, clients stay directly connected to a server, which, at the very least, brokers point-to-point connections. We’ve already covered client/server communication in the presence of a firewall, and the same techniques apply here.

However, what should be done when it comes time for two clients to communicate? The clients need to establish a direct connection: one client opening a server socket to which the other client connects. The first problem is that good firewalls prevent someone from opening a server socket on an arbitrary port. Some firewalls may actually allow this as long as the server binds to the HTTP port (or some other common server port), but this doesn’t often work for many reasons. First, services may already be running on that port. Second, the user may not have privileges to bind to the port in question.

The same problem can happen without a firewall. A client using NAT may be able to bind to a local port, but that port is only visible inside the local network unless the NAT server specifically knows to redirect external connection requests.

Also, just because one of the clients is able to establish a server port successfully doesn’t mean that the other client’s firewall permits a connection to that port, especially if the serving client is only able to bind to a high port.

If neither client can open up a server port that the other can successfully use, then peer-to-peer connectivity is not possible. There are two options. First, you can revert to using the server as an intermediary. Second, you can allow clients to configure a proxy. Proxies could exist anywhere on the Internet, as long as you are willing to give the proxy server away. Generally, the proxy server runs on port 443, or some other port that is widely reachable.

Allowing anyone to run a proxy is desirable, because some administrators may decide that your proxy is a deliberate attempt to thwart their network policy, and thus refuse to route traffic to your proxy. If proxies can spring up anywhere, the end user is more likely to be able to run your application.

We recommend that you determine a user’s firewall capabilities the first time that person starts the client and logs in to the server. Additionally, you need to take into account that some users are mobile and are behind different firewalls at different times. The firewall capability determination process is something that the user should be able to rerun when necessary.

A good strategy for determining a firewall configuration is to try to bind a server socket to two ports: the HTTPS port and a high port. Pick the same high port every time, so that people who specifically want to poke holes in their firewall for your application can do so. Then, for each successful bind operation, the server should try to connect to each port to see if outside entities can reach. If both of those ports are inaccessible, then the client in question should be labeled as unable to serve documents. Additionally, the client should explicitly try to connect to your chosen high port on the server, to see if the client can make outgoing connections to servers on that port.

If you choose to support proxy servers (which you should, because some people with draconian firewalls may need it), then the user should be able to enter in an address for such a server and decide whether to use it always or only to use it when no direct connection is otherwise possible.

When two clients wish to connect, it is the server’s responsibility to take the stored information and determine who should perform the action to initiate a connection. If one of the clients can serve on the application-specific port and the other can reach it (directly or through a proxy), then do that. If the clients can talk with one acting as the server on the HTTPS port (again, directly or through a proxy), then do that. Otherwise, you can have each connect to a proxy, and then have the two proxies communicate, or you can revert to sending everything across the central server.

Unless a client is configured to use a proxy always, you should give a warning whenever a direct peer-to-peer connection is not possible, because some users may be counting on their privacy.¹

1. Note that peer-to-peer links should be encrypted. The most secure option for the client is for you to set yourself up as a PKI, and have clients make validated SSL connections to each other. This is by far the most difficult option. Another reasonable option is to have the client and server perform a Diffie-Hellman key exchange through the central server (see [Schneier, 1996]). As a last resort, the server could generate a symmetric key and give the key to both clients.

Conclusion

We’ve seen throughout this book that security commonly trades off against usability. This is certainly the case with firewalls, which are widely cursed whenever they prevent someone from doing what that person wants to do.

Although the person setting policy on a firewall may wish to keep users from running your application, your goal is the opposite. You usually want users to be able to run your application, even if it violates the intended policy on some networks.

If your application is client/server based, then you have an easy time of it, because most firewalls pass through traffic if your server runs on port 443. For the few that don’t, you may wish to provide an application proxy. If you do, keep it simple!

Getting peer-to-peer services to work transparently though a firewall tends to be much more difficult. The best strategy is to be flexible enough to support every common security configuration.