When a testbed does more than testing

When a testbed does more than testing The Internet-Scale Event Attack and Generation Environment (ISEAGE) providing learning and synthesizing experiences for cyber security students. Julie A. Rursch, Doug Jacobson Department of Electrical and Computer Engineering Iowa State University Ames, IA jrursch@iastate.edu, dougj@iastate.edu Abstract The importance of laboratory exercises for students is recognized unilaterally by engineering and technology programs. As engineering educators whose academic focus is information assurance and cyber security, we believe students in cyber security need the same type of access to hands on opportunities as their counter parts in hardware design or circuit design. Students should be able to configure and run their own networks, as well as explore the vulnerabilities, exploits, and remediatios needed in a cyber security professional s tool kit. Further, they need exposure to working in the complexity of the Internet. While some might argue that simulation software could be a solution, it often lacks realism. In this paper we show how our institution goes beyond the providing the standard, formalized laboratory activities for our cyber security students by developing a unique, highly configurable testbed called Internet-Scale Event and Attack Generation Environment (ISEAGE pronounced ice age ) that allows us to imitate the Internet. ISEAGE provides a controlled environment that allows real attacks to be played out against the students networks and demonstrates to them real world security concepts. This paper provides an overview of how the ISEAGE security testbed functions, as well as illustrates how ISEAGE provides our students five different types of opportunities for real world experience: support of formalized classroom work; cyber defense competitions for high school, community college and four year students; inquiry-based learning in a playground environment for high school, as well as college students; testing environment for network devices such as firewalls, data loss protection, intrusion detection; research environment for senior and graduate student work. Keywords testbed, information assurance education, cyber security, laboratory exercises. I. INTRODUCTION Engineering and technology programs across the United States recognize the importance of laboratory exercises (or hands on experiences) for their students to truly synthesize concepts. As engineering educators whose academic focus is information assurance and cyber security, we want to provide our students with the most realistic of laboratory experiences to help them hone their skills and develop the depth of thinking needed in this complex and every changing world we live in. Students in cyber security need the same type of access to hands on opportunities as their counter parts in hardware design or circuit design. They should be able to configure and run their own networks, as well as explore the vulnerabilities, exploits, and remediations. However, putting students in labs full of physical equipment can be expensive to do, even with virtualization. And, if given an unending supply of servers, hard disk and RAM to create local networks, it is still a challenge to mimic the complexity of the Internet. Others might argue that simulation software could be a solution, but it often lacks realism. So, we are presented with a hard problem. How do we create a realistic Internet for cyber security students to use? In this paper, we show how our institution goes beyond the providing the standard, formalized laboratory activities for our cyber security students. We have developed a unique, highly configurable testbed called Internet-Scale Event and Attack Generation Environment (ISEAGE pronounced ice age ) that allows us to imitate the Internet. ISEAGE provides a controlled environment that allows real attacks to be played out against the students networks and demonstrates real world security concepts. The ISEAGE security testbed has an air gap proxy server through which students can connect to the Internet to download operating systems and patches or search for additional information about configuration problems, but no other traffic can escape. While ISEAGE can follow all TCP/IP protocols, it also allows manipulation of those protocols and traffic capture to demonstrate specific types of attacks such as Distributed Denial of Service. This paper will provide an overview of how the ISEAGE security testbed was developed and functions, as well as future work underway with the testbed. However, the major focus of the paper is to illustrate how ISEAGE provides the environment so that our students have five different types of opportunities for real world experience. It is divided into four sections. Section II provides a technical overview of ISEAGE and how it is configured. Section III enumerates the five different types of experiences our students are provided by the ISEAGE security testbed: A) support of formalized classroom work; B) cyber defense competitions for high school, community college and four year students; C) inquiry-based learning in a playground environment for high school, as well as college students; D) testing environment for network The ISEASGE security testbed was developed through Department of Justice funding. 978-1-4673-5261-1/13/$31.00 2013 IEEE

devices such as firewalls, data loss protection, intrusion detection; E) research environment for senior and graduate student work. Section IV provides the conclusions and future work. II. OVERVIEW OF ISEAGE There have been several successful network testbeds, the most widely recognized name in this area is DeterLabs [1], but ISEAGE is designed specifically for use in security research and offers several advantages over a conventional network testbed. It has four unique features as part of its highly reconfigurable architecture which allows for very small or very large network testing and rapid setup. Each of ISEAGE s features (architecture, tool set, data collection availability, and scalability) are described in the following sections. A. Architecture As shown in Fig. 1, the core of the ISEAGE security testbed is a routable IP network that supports the traffic to and from the networks and systems under test. Because of ISEAGE s internal programming, called ISEFlow, the architecture simulates the cloud environment of the Internet with multihops, but the traffic all stays contained within the security testbed. However, ISEFlow makes traffic appear as if it has routed through the Internet. ISEAGE is unlike conventional testbeds where each router represented by either a real router or a software router running on a computer. ISEFlow creates the external subnets as well as a large number of internal networks without needing to instantiate a separate router for each network. This internal cloud network represents a cluster of routers. If an external computer performed a traceroute to a server in a different network, it would see a number of hops between itself and server as if there were real routers between it and the server. The TTL field in the IP header would also indicate the traffic traversed multiple routers just as it would if it were traveling via the Internet. Figure 1. ISEAGE Architecture As is also shown in Fig. 1, the networks and equipment under test are directly connected into the ISEAGE s routable network. Generally, this equipment is considered to be servers and end user machines. These types of devices are end points and view ISEAGE as an Internet connection. However, there is also the ability to connect external devices to insert additional routing infrastructure. These types of devices can use the data flow through ISEAGE and can be used to test routers, firewalls and other devices that interconnect multiple networks. As was discussed in the introduction, the ISEAGE security testbed was created to allow work conducted in a secure environment. As shown in Fig. 1, there is a separate command and control network for configuring both the ISEAGE routable network and the networks and equipment under test. Further, Fig. 1 also shows the air gap proxy which is the only egress point for traffic leaving the ISEAGE security testbed. It allows web and ftp traffic to pass out of the networks attached to it, but restricts all other traffic. This configuration provides an isolated network environment in which the networks can run. It was intentionally developed to avoid the inevitable misconfiguration or unwanted attacks on the real world network by a student learning about or incorrectly configuring a device. Finally, the tool repository is shown in Fig. 1 and is discussed in Section B below. ISEAGE is highly configurable and highly scalable. The authors have copies running on computers ranging in size from modest laptops to multi-machine installations. One of ISEAGE s unique features is the ability to have an unlimited number of routers connected. The only limitation is the size of the computer or server on which the framework is running. ISEAGE is currently built on ESXi servers allowing for the quick addition of additional resources for any project. As a simple network configuration example for this paper, Fig. 2 is provided. For the sake of this discussion, it will be assumed the entire ISEAGE testbed is installed on one ESXi server. However, there are many different configurations that could occur in ISEAGE, depending upon the complexity required. Fig. 2 shows the network using five copies of ISEFlow which is depicted as a red box. Again, the number of ISEFlows can vary with the complexity of the installation. In this simple configuration between one and three routers are configured in each ISEFlow. Each of the numbered routers within the ISEFlow has an ingress and an egress address in public IP space. Again, because the traffic doesn t escape the ISEAGE security testbed, ISEAGE can use public IP address space. The red boxes labeled B1, B2, and B3 are the ISEFlows that allow systems to be attached into the ISEAGE network. Outside of the red boxes at the top of Fig. 2 are five additional IP address ranges. Again, these are in the public space. The systems under test use these additional IP ranges. The devices are either attached as endpoints and view ISEAGE as an Internet connection or they are inline devices that use ISEAGE to test the data flow through multiple networks. In either case, the systems under test point their outbound traffic to the outermost router interface in the ISEFlow configuration to which they are attached. When traffic leaves the devices of interest, the cluster of routers in ISEFlow will alter the traffic as if the traffic traversed each router in turn.

The red box labeled B4 exists to connect the ISEAGE network to the real outside Internet through an air gap proxy that allows only web and ftp traffic out. Air Gap 1 also functions as an internal DNS server so all systems under test can have name resolution. The red box labeled B5 is a data collection port and can record all traffic in the ISEAGE network. The green box labeled Backplane allows ISEFlows to communicate with each other. Figure 2. Simple network B. Tool Set There are multiple tools designed to support ISEAGE. The tools use a common command and control protocol to allow easy integration into the ISEAGE command and control network. Below is a brief description of the tool set. 1) Attack Amplifier & Condenser The attack amplifier is used to convert an attack launched from a single computer into an attack that appears to be launched from multiple computers. This tool allows researchers to study distributed and flooding based attacks. With this tool researchers can create attacks that appear to come from thousands or even tens of thousands of computers. The attack condenser works with the attack amplifier. Often distributed attacks create a large number of responses back to the attacker or responses that have been redirected to another target. The attack condenser will take the responses and condense them into a small number of responses. It can also be configured to respond to the messages. For example, if there is a redirected distributed attack pointed to a machine, the attack condenser can become that machine and absorb the attack and respond when necessary. 2) Packet Changer/Responder The packet changer/responder can be used to modify packets in real-time as they flow through the network. This tool can be used to create man-in-middle attacks or can be used to generate traffic in response to certain incoming packets. 3) Attack Collector/Watcher/Replayer These three tools are used to collect information to be replayed within the virtual internet. The attack collector is a honey pot / honey net that is used to collect host based attacks. The attack watcher is an intrusion detection system that captures network attacks. The attack replayer replays the attack inside the virtual internet. The attack collector and watcher are connected to remote sites via the internet using encrypted connections. 4) Attack Tool Repository An extensive library of attack tools is maintained. The library allows the launch of a wide array of attacks. By feeding the attacks through the tools described above the testbed network allows examination of and design of mechanisms for defense against real attacks 5) Traffic Collector/Replayer This tool captures traffic patterns from the actual Internet at particular locations so they can be replayed with ISEAGE. The collector captures traffic patterns only; it does not capture the data. The replayer then reconstructs traffic from the captured data to recreate as close as possible the background traffic seen at a given location on the actual Internet. C. Data Collection Point Another interesting feature of ISEAGE is its ability to capture all traffic through what is called a tap port. This is very useful when teaching about intrusions or when wanting a log of all network traffic during a research experiment. D. Scalability In addition to the full scale ISEAGE testbed several smaller versions have been developed. ISEAGE is capable of running on a single machine running virtualization. These smaller versions of ISEAGE are used in most of the setting discussed in this paper. The full scale ISEAGE is used primarily for research and device testing. The smallest version of ISEAGE runs on a laptop. The only limiting factor for the number of routers and nodes that can be used in the ISEAGE testbed is the size of the hardware on which ISEAGE is installed. III. ISEAGE USES There are five different uses of the ISEAGE security testbed in which student can gain valuable, real world experience with networks and security. The first three listed

below (classroom, cyber defense, and playground) are very similar, but have distinct differences based upon who builds the network that is being tested and who does the testing. A. ISEAGE in the Classroom The ISEAGE security testbed is used to support several classes at Iowa State University (ISU). Two of them are discussed in this paper: an undergraduate introduction to networking and graduate level capstone course. Both of these courses use the ISEAGE architecture as a connection to the Internet. Both courses allow the students to build networks by setting up servers and running services on them. However, the undergraduate course focuses on teaching basic principles such as IP address space, network connectivity, and simple protocols. This course was not taught in the engineering college, but in the business college as part of the Management Information Systems (MIS) program. The hands on activities include designing and configuring a mail server, a domain name service (DNS), a firewall, and a web server using their assigned public IP address and domain name in the ISEAGE testbed. They also had to use Windows Server 2008 as their operating system. The course was taught in the traditional lecture manner with students needing to complete the hands on exercises on a computer on their own time; no lab was associated with this course. Students self-selected into teams of four to five to complete the project. The course included a project planning phase, an implementation phase, and an evaluation phase for each team of students. The graduate course is an information assurance course capstone course developed at ISU to enable distance education students to complete the requirements for a Master s of Engineering in Information Assurance without setting foot on campus [2]. This course is entirely lab-based and is comprised of three parts: the planning and implementing phase (six weeks); the defending and attacking phase (four weeks); and the infrastructure assessing phase (five weeks). While the undergraduate course had the planning and assessment phases, the graduate course was more rigorous and included an attack and defend phase. Here individual students create their own networks instead of working as a team. They adhered to a provided scenario or story line of what services have to be run. The students could select any operating system as long as it was open source, had a demonstration period that lasts throughout the semester, or was site-licensed by ISU. Additionally, the graduate students had to produce their own preliminary network plan that included a diagram of their network, as well as the rationale for selecting the operating systems and applications used. Again, the graduate students were assign public IP addresses and domain names in the ISEAGE testbed which they had to implement. During the attack and defend phase each graduate student tries to identify vulnerabilities and weaknesses in his classmates networks. Students are allowed to exploit these vulnerabilities on others networks, as well as capture predefined flags or planting flags. Students must document their discoveries and activities as part of their final report. Additionally, while they are trying to exploit others networks, they must defend their own networks and protect their flags. In summary, in the MIS class the students build their own networks as teams using a Microsoft environment, but the networks were only tested by the faculty member to verify the systems were working. In the graduate course, a single student built his entire network using any operating system and then attacked other students networks. In both cases the requirements for building the network were specified ahead of time and did not change during the semester. B. ISEAGE as a Cyber Defense Environment The goal of a cyber defense competition (CDC) is to have students design, and configure a set of servers and a network in a secure manner and in a relatively short, one-month period of time [3, 4]. Then, the students attend a two-day competition to defend their network from attackers. During the two-day competition, their goal is to prevent, if possible, any security violations or attacks on their network, as well as report and correct any problems that arise. They also must maintain full functionality of their systems for the end users. Students in ISU competitions are required to configure their networks as described by a scenario that details the services that they have to implement in a short story format, as well as their network address space. They are told they are the IT support staff for a company or school and have to implement services such as email, web mail, remote programming, file sharing, and web hosting. They are also told they are responsible for their own Domain Name Service (DNS) and it would be wise to implement a firewall to help protect their networks. They are also given some service, generally a web server, that is a legacy installation which must be supported in a present state. The legacy system provides some inherent security vulnerabilities that they have to protect against. Teams also have to protect flags from being captured by the attackers. These flags are encrypted files which contain a unique string and are required to be stored in a specific directory location on specific servers the teams are running. In our competitions, these student teams are known as the Blue Team. Depending upon the competition, the Blue Team may be comprised of high school students, community college students, four-year institution students or IT professionals/faculty members. In addition to defending their network, the students also participate in numerous activities (called anomalies) throughout the competition which are designed to keep them engaged and slightly off balance just as real IT staffs get engaged in new projects and may overlook intrusions or security risks in new implementations. The people who activate these anomalies are the Green Team. This team of people are assigned to play the role of end users of the teams' networks. They can request changes to be made to the Blue Teams' networks throughout the competition. Some of these Green Team requests may run counter to the goal of having secure systems or may be to have the teams install some of the latest software that opens holes in their servers. The Blue Team must then decide how, or if, to implement the request on their network and how to implement it security. The Green Team members are recruited from undergraduate student population, less technical corporate partners, and ISU faculty across campus. This wide variety of computer skill levels provides true tests of usability for the

Blue Teams. The addition of the Green Team is what helps keep the students focused on providing a useable network, as well as a secure one. Generally, the anomalies occur with a frequency of every 60 to 90 minutes during the competition. However, during the cyber defense competition an anomaly may be developed based upon a common characteristic found in the networks. The group who tests each Blue Team network for vulnerabilities and plays the role of attackers in the competition are IT professionals, as well as Computer Engineering faculty and graduate students specializing in Information Assurance (IA), and are called the Red Team. The Red Team is led by an ISU IA faculty member and/or a member of industry who specializes in penetration testing. Since the competitions occur over a Friday and Saturday, their job is to conduct reconnaissance work on Friday and early Saturday morning when the teams are still setting up to determine what kinds of networks the teams are running or to carry out social engineering. Then, at the designated time on Saturday morning, the Red Team begins active network scans and active penetration testing against the Blue Teams networks. Once vulnerabilities are found, the Red Team may act on those to gain access to the servers of interest. First, they must capture the flag on that server to prove that they have access to the box. Then, once they have the flag, they can reconfigure it, install additional software on it, install a virus on it or take any variety of steps that an attacker might take on a production server. The Blue Teams may recognize the Red Team's advances and may take actions to stop them or recover from them, as well as report the breach. However, the Blue Team may not attack or block the Red Team in an effort to protect their systems. All competitions need a group to oversee the event and keep everyone in the competition honest. The White Team performs the role of adjudicator, as well as records scores for the Blue Teams given by the Green Team and Red Team on usability and security, respectively. The White Team also reads the security reports and scores them for accuracy and countermeasures. The White Team leader is usually an upper level undergraduate student or a graduate student. In summary, for the CDC the Blue Team built their network and defended it, but the Red Team was the group doing the attacking. Also, the requirements for the network was constantly changing based upon the Green Team anomalies. C. ISEAGE as a Playground Environment Although taking a one semester class that uses ISEAGE allows students some valuable experience to build and protect networks, the pitfall is that the course only runs one semester. Then the equipment is reset and the networks that the students build are put back to original clean state. As part of the course evaluations, we discovered that it would be useful to allow students access to a centralized virtual lab year round so students could practice skills in an ungraded and experimental environment. Therefore a centralized virtual lab called the playground was created which is available throughout the year for students to build and experiment with different operating systems and security measures. In the case of the playground, there are no requirements made of the students wanting to work in it. They are not given any network specifications or required to complete any reports. They are given open access to perform inquiry-based learning in their own time and speed. The systems that are under test would be the systems that they build and there is no one attacking them, but they could complete their own penetration testing as part of the inquiry-based learning. D. ISEAGE as a Testing Environment ISEAGE has been used to test data loss prevention devices for a major networking trade magazine and new testing series of different products is scheduled. Generally, the projects are the testing of commercial-off-the-shelf (COTS) products in a controlled environment. As part of each testing process the methodology and the metrics have to be developed. Here the student work is in development of the test metrics, performance of the tests, and the evaluation and results documentation. The first three examples listed above (A-C) engaged students with creating and implementing networks of their own design. In this case, the network that is being developed in the ISEAGE testbed is focused on data flows being pushed through or to external devices connected into the testbed. There is more standardization of the network configuration. The creativity and learning comes from the development of metrics and the evaluation of the test results. E. ISEAGE as a Research Environment The ISEAGE security testbed was designed to provide an environment to conduct state of the art research in computer security and security tool development. ISEAGE is currently being used in several projects related to the modeling of critical infrastructures. Two projects are discussed in this paper. First, the ISEAGE security testbed is being used to model the State of Iowa cyber infrastructure with the goal of being able to determine interdependencies between systems and any weakness in the system. Additionally, what if scenarios are developed to help the state develop contingency plans in case of a cyber attack. Once deployed, the State of Iowa will not only be able to test the infrastructure, but will be able to use ISEAGE to provide training of the staff and to try out new protection systems in a controlled environment. The second project using the ISEAGE testbed is focused on the development a meta framework that allows modeling of critical infrastructure and assets with physical data which can be used for training, preparedness, and real-time reaction. This unified model is the Critical Infrastructure Modeling and Response Environment (CIMoRE) [pronounced see more ] which represents a new paradigm for disaster planning and response. CIMoRE accounts for all critical infrastructure components such as roads, bridges, rail systems, water treatment facilities, power grids and telephone systems, cyber networks, as well as their interdependencies, in its single, unified framework. Because it is built upon the ISEAGE testbed, CIMoRE provides for a varying level of complexity in the inclusion or exclusion of critical infrastructure components. CIMoRE gives emergency planners and disaster responders the opportunity to view the physical locations of the critical

infrastructure components, assess their interconnectedness, identify their failing health state, determine and avoid congestion, visually play out mitigation options, document analysis decisions and record the recovery of the critical components. IV. CONCLUSIONS AND FUTURE WORK The use of the ISEAGE security testbed has provided the environment so that students have five different types of opportunities for real world experience: support of formalized classroom work; cyber defense competitions for high school, community college and four year students; inquiry-based learning in a playground environment for high school, as well as college students; testing environment for network devices such as firewalls, data loss protection, intrusion detection; research environment for senior and graduate student work. The extension of the ISEAGE security testbed from graduate level research into the mainstream of undergraduate and graduate education has proven to be an outstanding way for college students to solidify concepts and gain real world skills in information assurance and network security. Undergraduate and graduate experiences, both in the classroom and out of the classroom, have become an integral part of much larger ISEAGE research project. While originally developed as a testbed for security research, the extension of ISEAGE s reach into general IA student education has been valuable. By using the ISEAGE testbed, we provide our cyber security students with multiple opportunities to create and evaluate throughout their academic career. In addition to providing more numerous opportunities to participate in higher order thinking skills, ISEAGE also provides a wider array of activities with which to engage. The ISEAGE security testbed is a continual work-inprogress. Currently a lab extender is under development to extend the ISEAGE closed infrastructure across the actual Internet, by placing a lab extender in a remote location connected via the Internet to a lab extender connected to the ISEAGE. The extender will use compression and special protocols to increase the effective bandwidth between two extenders. The lab extender can be used to provide remote testing of infrastructure components. The lab extender will also be used to setup remote virtual Internets for collaboration on research projects with other universities, agencies or businesses. The authors are in the process of releasing the ISEAGE security testbed to other academic institutions. At the time of writing, we have two institutions using ISEAGE in their classrooms. One community college is using ISEAGE for a CDC-type activity for their own students in a network security course. The other is a state university that will be using ISEAGE to teach two introductory networking classes. These introductory networking classes will be patterned after the course taught in MIS at ISU and described above. The authors plan to distribute to additional institutions that have asked for similar configurations over the next several months. REFERENCES [1] (2013, July 9). Deter Lab. Available: http://deterproject.org/deter-operatedeterlab-cyber-security-research-lab [2] N. Evans, B. Blakely, and D. Jacobson, "A Security Capstone Course: An Innovative Practical Approach to Distance Education," presented at the 39th ASEE/IEEE Frontiers in Education Conference, San Antonio, TX, 2009. [3] D. Jacobson and J. A. Rursch, "Cyber Defense Competitions as learning tools: Serious applications for information warfare games," in Serious Games as Educational, Business, and Research Tools, M. M. Cruz- Cunha, Ed., ed, 2012. [4] D. Jacobson and J. A. Rursch, "Engaging Millenials with Information Technology: A Case Study Using High School Cyber Defense Competitions," presented at the 12th Colloquium for Information Systems Security Education, Dallas, TX, 2008.