Cyber Security Education
The Merits of Firewall Exercises
Evan Damon1, Jens Mache1, Richard Weiss2, Kaleb Ganz1, Claire Humbeutel1 and Miles Crabill1, 1Lewis & Clark College, Portland, OR, USA, 2Evergreen State College, Olympia, WA, USA
Educational tools related to firewalls and firewall configurations are central to the development of cybersecurity education, due to the integral role firewalls play in this field. To assess the current state of firewall education, we developed a set of criteria that educational exercises should meet. They should be competitive, interactive, extensible, and approachable. These criteria were then applied to three existing firewall exercises in order to gauge their strengths and weaknesses. With that information in mind, we then looked to emerging trends in firewalls and firewall education. We found that many of the emerging trends align with our criteria, but diverge from them in other important ways.
Keywords
computer security; firewall; computer science education; lab exercises; interaction; extensibility; approachability; competition
Information in this chapter
Introduction
Firewalls are an essential part of network defense, and studying them captures aspects of confidentiality and availability. Educating students about them is therefore an essential part of cyber-security education. In this chapter, we describe and evaluate three firewall exercises. We develop four criteria and assess how well each exercise meets the criteria we set out. Finally, we explore emerging trends in firewalls and firewall education, and we make recommendations based on our observations.
Firewalls remain a key element of computer security despite being among the oldest security strategies. Although they have become more sophisticated, port controls and packet inspection techniques are still a mainstay for defending the perimeter of a network. Firewalls originated as the relatively rare packet filter in the late 1980s; they have since become ubiquitous, existing in various forms on nearly every network-connected router, computer, and server. This widespread implementation makes firewalls an integral part of computer security. Strangely, though, firewall education was not emphasized in computer science until recently; firewalls and other security concepts were considered elective material in 2001 [1]. This was probably due to a multitude of factors, such as the lack of a universal firewall syntax and the difficulty of setting up a security laboratory for simulating computer networks and firewalls. Some exercises did exist, but they were not particularly deployable; one exercise required a set of laptops to be transported and set up on-site [2]. The seeming dearth of educational firewall exercises leaves computer security students without a complete understanding of computer networks. Knowledge of firewalls, how they work, and how they are administered is imperative to understanding and practicing computer security.
It is for these reasons that we have evaluated the current state of firewall education. Firewall exercises do exist, after all. The three exercises we selected were chosen for their availability and deployability within an existing classroom or laboratory setting. These exercises came from varied sources. The first was a standalone simulation designed by Professor Ken Williams [3]. This tool, called FireSim, pits players against each other in a competitive simulation. Players assume the role of a firewall administrator and write rules that prevent other players from potentially compromising the security of their network. Simultaneously, they conduct probes against each other, taking advantage of vulnerabilities in the firewall. The second exercise was from Vincent Nestler’s Principles of Computer Security[4] and used a cloud-based virtual network, called the RAVE. The RAVE exercises give the user control over a virtual network of four machines. The students configure the firewall of one machine and test it from another. The third exercise, DETERlab [5], is also a cloud-based network. Designed by the University of South California’s Information Sciences Institute in collaboration with the University of Utah, DETERlab allows students to remotely log in to the VMs. In our case, these were two machines running a Linux environment. One student works with iptables to configure the firewall to prevent unwanted access while allowing desirable services, while the other uses network tools, such as nmap, to test the effectiveness of their other machine’s firewall. These three exercises are evaluated in more detail later. In addition, we discuss new firewall exercises by Palo Alto Networks and EDURange. EDURange can be used to create competitive, interactive exercises and is independent of platform.
Criteria for firewall education
In order to predict the educational effectiveness of firewall education exercises, we developed a set of criteria that have shown promising results in classroom environments. Exercises can engage students through interactivity and competition. To teach the security mindset, they should involve real-time problem-solving and analysis. These skills are necessary when configuring an enterprise firewall. Third, exercises should be extensible. That is, instructors or other users should be able to add scenarios or other features to the exercise in a reasonably simple manner. This ensures that the exercise remains up-to-date and teaching points or important concepts can easily be added. Finally, an exercise must be easy to use (approachable). The exercise should achieve the educational goals without adding complexities such as difficult-to-use syntax or tools. In summary, our criteria specifies that an exercise should be competitive, provide some manner of interactivity, have a degree of extensibility, and be easy for a student to understand without being too simplistic.
These criteria were heavily influenced by the curricular guidelines outlined by the Association for Computing Machinery (ACM) and the Institute of Electrical and Electronics Engineers (IEEE) [6]. These guidelines stipulate that firewall education is “generally essential” in undergraduate curricula. The guidelines advise that, at a minimum, eighty percent of the Tier-2 Core subjects (of which firewalls are an element) be covered. The criteria outlined above hopes to facilitate this goal.
Evaluation of firewall exercises
With the above criteria in mind, we evaluated the three previously mentioned exercises and assessed their educational strengths and weaknesses.
FireSim
FireSim was created using Java applets and XML files. The setup involves downloading a group of files to a networked machine that serves as the host machine. Players then connect to the host machine and choose a username. Once the administrator/instructor uses the GUI to start the game, players are able to conduct simulated attacks against one another in a competitive setting. Through the GUI, players select an attack and direct it at a particular opponent. If the attack is successful, the attacker receives a point while the defender loses a point. After 60 seconds, the same attack can be launched against the same opponent. Players write firewall rules using the Cisco firewall syntax. The goal is to create a list of firewall rules that filter specific packets. This can take many forms, but most rules block or allowed access to certain ports, depending on the source and destination IP addresses.
To introduce complexity, FireSim allows the administrator/ instructor to give players new “tasks” or injects. Tasks are stored in the XML files used by the simulation, allowing for more to be added. Each task may require players to create additional firewall rules that mitigate the new situation or vulnerability. The task is announced to the players, and after 60 seconds, players may begin to attack each other using it.
FireSim does many things well. It does a good job of simulating a real network and its network map includes many servers at specific IP addresses. This is one of FireSim’s greatest strengths. The underlying network simulation gives the exercise a great deal of potential for further development.
FireSim has weaknesses as well. The tasks are not as clear as they could be. Some are “trick” tasks that do not require additional firewall rules. These are intended to show the user that, when creating a firewall, extending your whitelist too far is dangerous. However, this is not obvious to the user. If a player does nothing and is impervious to attack, they may not learn this lesson. On the other hand, players who incorrectly extend the whitelist may have difficulty understanding why they lose points. Firesim could be made more effective by providing more feedback to the students for them to achieve a deeper understanding.
The scenarios are often about blocking access to a particular service, which is easy enough. However, the zero-sum nature of the point system may promote an offensive strategy, where players spend time attacking other players as frequently as possible. For some players, this may take emphasis away from the firewall configuration and place much of it on attacking. In summary, FireSim is moderately successful as a firewall exercise. It is competitive, since players test each other’s firewalls and try to expose vulnerabilities. The competitive structure, though, is a zero-sum structure. This puts a great deal of emphasis on offensive strategies, while the educational goal should concentrate on defense. FireSim does a good job of giving the students hands-on experience. The simulation puts players in full control of a firewall. The main shortcoming is that some of the scenarios are less than successful, but that can be remedied. The simulation offers some extensibility through its XML files, which store the tasks and actions available to players. Finally, FireSim is fairly approachable. The only confounding factors are the syntax and lack of feedback. The feedback level is relatively low, but further development of FireSim or a similar exercise could provide better feedback.
The RAVE
The RAVE lab firewall exercises are designed to teach students how to configure a firewall in Linux. Each student accesses their own virtual machines on which they can configure iptables. The steps for the lab are explicitly spelled out. The goal of the firewall lab is to teach students how to use iptables (using the Uncomplicated Firewall (UFW) [7] syntax) and the effects of different rules through hands-on work.
The advantages of this approach are mostly found in its simplicity, resistance to mistakes, and opportunity to practice what the student are learning about in the related book [4]. The instructions are easy to understand and follow, and they progress in a logical manner. The pictures that are provided make it easy to check how the lab is progressing and to find any part of the GUI that the lab requires.
The use of virtual machines that can be quickly restored from snapshots gives the exercise resistance to mistakes. That way, if a firewall is misconfigured and the student is locked out, the problem can be corrected by restoring the previous state.
A major disadvantage is the lack of scalability. If a number of people are connecting at the same time, the servers can become overloaded. Also, it requires an Internet connection. The RAVE lab does not meet all of our criteria. The exercises are not competitive. Students explore defensive scenarios, but there is no adversary to find their mistakes. RAVE is also not particularly extensible; the configuration is hidden from the instructor. However, RAVE is interactive. The exercise guides the players through the steps, allowing the players to apply what they learn. Unfortunately, this advantage is somewhat diminished by the simplicity of the exercises. The step-by-step nature helps to show the user how to proceed, but does not require the user to apply much analysis. The RAVE is extensible since it is based on virtualization. Currently, most instructors do not have the tools for creating their own exercises or modifying the existing one.
DETERlab
DETERlab offers an exercise in which users configure a firewall on live Linux machines. The DETERlab exercise that covers firewalls also covers UNIX file permissions. The tasks for the two are independent of each other, but descriptions for the two are intermixed. This can make it lengthy to find relevant information for each of the tasks on the walkthrough web page. This section of the chapter focuses exclusively on the firewall aspects of the exercise.
The firewall tutorial portion of this exercise starts out with a description of stateless and stateful firewalls, what the difference between them is, and a little bit of history behind the development of the two. A description follows of how a firewall policy should be designed and how to implement one using iptables, the default Linux firewall. It briefly covers the syntax and a few example rules. The lab then goes on to describe four different network tools. These are nmap, ifconfig, telnet, and netcat. Each of these has about a sentence- to a paragraph-long description and an example of what the tool returns in the terminal when used.
The interactive portion of the lab is done on two remotely accessed nodes. Accessing these nodes requires the user to connect to their DETERlab account through Secure Shell (SSH), then from SSH into the two control nodes. Each of these two nodes serves a different function. One node is their firewall. The other is a machine through which users test the firewall rules they have implemented on their server node.
The firewall rules are applied using a script. The user writes the rules, then runs the script to enable the firewall using those rules. It is also easy for the user to turn off all rules and remove them by running another script that is provided on the server node. The tasks that the lab asks the user to do are: write a rule that prevents spoofing; allow access to OpenSSH, Apache, and MySQL on their standard ports; allow UDP access to specified ports; allow ICMP ping requests; allow all established and related traffic; and lastly, to drop all other traffic to any unspecified port. If the user accidently configures iptables in a way as to get locked out, the user must reboot the instance, which can take some time. The user can test the firewall by using nmap or telnet to a port in question; however, there is no test provided that allows the user to determine if the user’s anti-spoofing rules are functional.
The best feature of using this DETERlab exercise is that it is realistic. Since it uses common firewall syntax, students must remotely connect to the environment to make changes, and they have a broad spectrum of network testing tools at their command. It is also possible for the instructor to create new scenarios that require the students to modify their firewalls to fit this new situation. Many of the advantages of using this exercise for learning and teaching firewall rules have the potential to also be disadvantages, depending on the proficiency of the students and instructor. Iptables has a complex syntax. Also, the possibility that students may lock themselves out of the environment is both an advantage and a disadvantage in that it teaches them about the possibility of doing so in a real IT position; but it takes time to reboot the system and can be frustrating. Other disadvantages are that some overhead is required to set up the experiment, and students must wait for the systems to become available to connect to them.
Based on the criteria stated above, DETERlab falls short in its competitive aspects and its approachability. The exercise poses no competition, as it is in an isolated environment in which the student is working alone, and as such, doesn’t capture the real-world situation of defending against an attacker in real time. Due to the range of network tools available and the complicated syntax of iptables, this exercise can be challenging as a first approach to learning about firewalls and writing firewall rules. However, this exercise fulfills our criteria for interactivity, since the students have full control over how they wish to set up their firewalls to complete the tasks and the order in which they wish to tackle them. DETERlab is highly extensible due to the way that the network/device configuration is done. Its system uses specific .ns files that allow the instructor to specify the number of nodes and how they are connected, as well as other network configuration specifics.
Satisfying the criteria
It is desirable to provide students with an active learning environment that focuses on analysis skills. We still need to think about what we can teach and how to provide laboratory exercises for undergraduate students. There are also some pedagogical issues. While the exercises received high scores on student interest [8], they only had moderate coverage of the security education principles that instructors value. This suggests the need for additional work to extend the scope and intellectual complexity of these exercises.
One option is EDURange, an in-development tool intended to be an extensible framework for creating interactive and competitive cyber-security scenarios. Implemented on cloud-based computing platforms (currently Amazon Web Services’ Elastic Compute Cloud [9]), it is simple to set up a virtual network. Each student uses SSH to login to a gateway VM and from there logs in to their own VM. The student can use either UFW syntax or iptables to create firewall rules and can then use network testing tools to probe the firewalls of fellow students. By using UFW’s syntax, the scenario becomes much more approachable than either FireSim or the DETERlab exercise, which use the more complicated iptables or Cisco syntax. This allows students to focus on comprehension of the concepts of firewalls without having to master the syntax. Also, having all of the students in the same network space, and giving them tools to test and potentially penetrate their fellow students’ machines, adds a component of competition to the scenario. EDURange is designed to be extensible. Instructors can modify which services the students’ firewalls should allow or deny. The network configuration can also be changed. These changes are made by editing a configuration file, written in YAML [10], which is parsed using Ruby scripts. This file sets up the network and machines as specified by the instructor. These YAML files can even be used to create a variety of virtual network configurations to suit different types of scenarios. The instructor does not need to manually change network configurations or write any scripts to do so. For example, the scripts can be parameterized so that the instructor could choose the size of the network and the services running on the hosts.
Another option is firewall exercises by Palo Alto Networks. The Palo Alto device is a so-called “next generation” firewall, which has recently become more prevalent. Palo Alto’s approach involves the control of general service categories, such as a “file-sharing” category, as opposed to specific ports and services. For example, it can detect that a host is sending an IRC message over an sftp port. It also makes use of innovative prevention measures, such as traffic classification and data filtering. The classifications of the traffic logs are used to examine what services on the network are being used, and if any are harmful. To train students to use their firewall devices, Palo Alto offers lab modules [11]. These modules lead students through step-by-step labs to accomplish different tasks such as creating user groups and blocking specific types of websites. These labs cover firewall administration, network interface configuration, network layer configuration, application identification, content identification, and packet decryption. The lab modules are not competitive; in most cases, two students use a single router and must work together. However, competition could easily be introduced through time limits, scoring, or direct competition between an attacker and defender, provided the network is isolated from the Internet.
Future exercises and projects like EDURange and Palo Alto’s educational modules can take the identified criteria into account as they are developed. As has been discussed, the inclusion of hands-on, competitive aspects in cyber-security education promotes understanding and the development of analysis skills. In order to better teach cyber-security students about firewalls (and thereby general network functions and security), educators and exercise designers can incorporate some of the strengths found in FireSim, RAVE, DETERlab, and the Palo Alto activities. To reiterate, these strengths were the competitive structure and approachable layout found in FireSim and the hands-on approach found in the DETER and RAVE labs.
Figure 31.1 summarizes the strengths and weaknesses of the different exercises (EDURange is excluded because it has not reached a usable level of completion).
As can be seen in 31.1, some of the exercises are more appropriate than others in teaching cyber-security students about firewall configuration. Particularly of note are the possibilities offered by Palo Alto’s exercises and fully functional hardware, though this is currently off-set by the high cost of the devices. Also of note is that FireSim was the only exercise to involve competition as a feature.
Emerging trends in firewall education
Various trends have become apparent in personal and commercial firewalls. Firewall companies like Palo Alto, Cisco, and Barracuda have begun to use new strategies when defending networks. Barracuda [12] and Palo Alto [13] offer virtual implementations of their firewalls. Both companies have also introduced “next-generation” firewall techniques, primarily emphasizing application awareness: Palo Alto’s App-ID [14] and Barracuda’s NG Firewall [15] can identify applications by their traffic signature and can control traffic on the application layer (e.g., allow Skype calls but prevent Skype file transfers). Though virtualization and application awareness are beyond the scope of a personal firewall, firewall software has become part of general security suites. Commercial anti-virus suites like those offered by Kaspersky [16] and Norton [17] include firewalls; free software like Microsoft Security Essentials [18] and the open-source Snort IDS function as firewalls and even include some application layer functionality [19].
These trends may begin to appear in firewall education tools. Application awareness may be difficult to simulate, but the virtual firewall implementations could allow students to use them in isolated, special purpose virtual environments. Companies like Palo Alto offer training programs; if educational licenses of their virtual firewalls are available, instructors may be able to use them to educate their students on the emergent firewall technologies. However, this would be a proprietary system, and we are interested in teaching analytical skills not just tools. Many of the features such as identifying signatures for applications are not open, but the student could write their own regular expression recognizer. EDURange could also potentially utilize the power of regular expressions as part of iptables.
Conclusion
In 2001, the ACM and IEEE’s curriculum guidelines listed firewalls and additional security concepts as optional. In the 2013 guidelines, firewall knowledge was elevated to a core topic. Recently, training platforms have been created that allow students to work with firewalls without the need for physical devices [20].
Based on undergraduate students’ experience with three of these firewall training exercises, we concluded that all three are workable firewall education tools, but they all could be improved. FireSim is competitive, interactive, and entertaining, but the educational aspects are limited by the low levels of feedback and guidance. DETERlab was the most realistic of the exercises, involving fully configured machines. However, accessing and following the exercise was rather involved, and could be daunting for introductory-level students. This could be improved with additional tutorials. Finally, the RAVE labs are focused and accessible. The exercises are well-written, easy to follow, and educational. Since they run on virtual machines, students can recover from mistakes using snapshots. A Web connection is required, as well as the vSphere client running on Windows, before students can begin to use RAVE.
Each exercise had strengths and weaknesses. We recommend RAVE for beginning security students; the simple instructions and stable setup give introductory students a good environment to work in. DETERlab is useful for more advanced students. The structure gives students greater access to the command-line interface and software management, which may be daunting for beginning students. Students must also be aware of security pitfalls (e.g., blocking SSH from within an SSH session) and other problems, meaning introductory-level users could easily have trouble. FireSim is the only competitive exercise of these three; we recommend it as an in-class exercise. Finally, although Palo Alto’s lab exercises may require expensive hardware, instructors could consider them for actual hands-on experience with newer trends in the firewall market. Eventually, virtual devices may be available.
Of course, every exercise has issues. No “silver bullet” exists for firewall education. Some exercises sacrifice simplicity for realism, while others exchange educational depth for approachability. Future firewall exercises will have to take this into account along with the recent developments in firewall configuration. For example, the EDURange project attempts to utilize cloud-based virtual machines for availability. Hopefully, future projects will be able to balance the competition, interactivity, extensibility, and approachability.
Acknowledgments
Partial support for this work was provided by the National Science Foundation’s “Transforming Undergraduate Education in Science, Technology, Engineering and Mathematics (TUES)” program under Awards No. 1141314 and 1141341, by the John S. Rogers Science Research Program of Lewis & Clark College, and by the James F. and Marion L. Miller Foundation. We would also like to thank Michael Locasto and Stefan Boesen for their contributions.
References
1. Computing Curricula: Computer Science. Association for Computing Machinery. [Internet]. Retrieved from: <http://www.acm.org/education/curric_vols/cc2001.pdf>; 2001 [accessed 06.13].
2. A Portable Computer Security Workshop. ACM Digital Library. [Internet]. Retrieved from: <http://dl.acm.org/citation.cfm?doid=1248453.1248456>; 2006 [accessed 06.13].
3. Firewall Simulation. Williams.comp.ncat.edu. [Internet]. Retrieved from: <http://williams.comp.ncat.edu/FireSim/index.htm>; [accessed 05.13].
4. Nestler V, White G. Conklin WMA Principles of computer security: CompTIA Security+ and beyond. Lab Manual McGraw Hill 2011.
5. About DeterLab. DeterLab.net. [Internet]. Retrieved from: <http://info.deterlab.net/about>; [accessed 05.13].
6. Computer Science Curricula. Stanford AI Lab. [Internet]. Retrieved from: <http://ai.stanford.edu/users/sahami/CS2013/ironman-draft/cs2013-ironman-v1.0.pdf>; 2013 [accessed 06.13].
7. UFW. Community Ubuntu Documentation. [Internet]. Retrieved from: <https://help.ubuntu.com/community/UFW>; 2013 [accessed 07.13].
8. Weiss R, Mache J, Nilsen E. Top 10 hands-on cybersecurity exercises. Journal of Computing Sciences in Colleges. 2013;29.
9. Amazon Elastic Compute Cloud. Amazon Web Services. [Internet]. Retrieved from: <http://aws.amazon.com/ec2/>; 2013 [accessed 05.13].
10. YAML Ain’t Markup Language. YAML 1.2. [Internet]. Date of article [accessed on year mon day]. Retrieved from: <http://www.yaml.org/>; 2011 [accessed 05.13].
11. Firewall Configuration Essentials 101 Course. [Internet]. Retrieved from: https://support.paloaltonetworks.com/101_course/player.html.
12. Next Generation Firewall Vx. Barracuda Networks. [Internet]. Retrieved from: <https://www.barracuda.com/products/ngfirewall/vx>; 2013 [accessed 07.13].
13. Virtualized Firewalls. Palo Alto Networks. [Internet]. Retrieved from: <https://www.paloaltonetworks.com/products/platforms/virtualized-firewalls/vm-series/overview.html>; 2013 [accessed 06.13].
14. App-ID. Palo Alto Networks. [Internet]. Retrieved from: <https://www.paloaltonetworks.com/products/technologies/app-id.html>; 2013 [accessed 06.13].
15. Next Generation Firewall - Features. Barracuda Networks. [Internet]. Retrieved from: https://www.barracuda.com/products/ngfirewall/features; 2013 [accessed 06.13].
16. Internet Security 2014. Kaspersky Lab. [Internet]. Retrieved from: http://usa.kaspersky.com/products-services/home-computer-security/internet-security; 2013 [accessed 07.13].
17. Internet Security. Norton. [Internet]. Retrieved from: <http://us.norton.com/internet-security/#>; 2013 [accessed 07.13].
18. Microsoft Security Essentials Product Information. Windows. [Internet]. Retrieved from: <http://windows.microsoft.com/en-us/windows/security-essentials-product-information#tabs1=features>; 2013 [accessed 07.13].
19. SNORT - First Line of Defense for Web application attacks. Packet Storm Security. [Internet]. Retrieved from: <http://dl.packetstormsecurity.net/papers/IDS/snort-firstline.pdf>; [accessed 07.13].
20. Weiss R, Mache J, Nestler V, Dodge R, Hay B. Teaching cybersecurity through interactive exercises using a virtual environment. J Comput Sci Colleges. 2012;28:10.