Designing and Implementing a Cyberwar Laboratory Exercise for a Computer Security Course

Designing and Implementing a Cyberwar Laboratory Exercise for a Computer Security Course Paul J. Wagner and Jason M. Wudi Department of Computer Science University of Wisconsin-Eau Claire Eau Claire, WI 54701 {wagnerpj,wudijm}@uwec.edu Abstract The development of a cyberwar laboratory exercise for a computer security course raises many pedagogical and management issues relating to the structure of the laboratory, its network and the exercise itself. We have designed, implemented and given such an exercise, and faced many of these issues. Evaluation of this exercise leads to multiple insights about the proper goals, structure and implementation of such an exercise. Categories and Subject Descriptions K.3 [Computers & Education]: Computer & Information Science Education - Computer Science Education. General Terms Management, Design, Experimentation, Security. Keywords Cyberwar, Security, Cybersecurity, Laboratory, Exercise. 1 Introduction We taught a computer security course for the first time at the University of Wisconsin Eau Claire in the Spring 2003 semester. As part of this class, we developed a large cyberwar laboratory exercise where course students worked in teams to secure a computer system and then try to gain access to other systems present on the network, including the systems used by the other teams. We gave this exercise during the final week of classes as a way for students to demonstrate cumulative understanding of the various security issues, strategies and tools we discussed during the semester. There are a large number of pedagogical and management issues that arise in the design and development of this exercise, and we discuss these issues below. 2 Background 2.1 History The idea of using a cyberwar laboratory exercise in computer security curriculum is not new, though from anecdotal reports at computer security workshops and conferences a solid majority of schools offering computer security course work are still in the process of developing such exercises. These exercises may involve a variety of security activities, but usually include the defense (through system hardening) and attack (through various methods, including technological, physical and social) of computer systems on a network. One of the best known of these exercises is the annual cyberwar competition between the four branches of the United States military academies [2]. Other cyberwar laboratory structures, courses and exercises have been presented at prior SIGCSE conferences [1,4]. Unlike these previous discussions, we focus here solely on the pedagogical and management issues that arise from developing a large cyberwar laboratory exercise. 3 Student Preparation for Exercise 3.1 Student Preparation Goals To prepare the students for this exercise over the course of the semester, we had two goals. First, we wanted to give them enough experience with the issues, strategies, tactics and tools of computer and network system attackers and defenders so that they could apply these readily during the exercise. Second, we wanted to give them an ethical foundation for the application of the knowledge they gained so that they were able to maintain proper conduct during the exercise and in their future work as computer professionals. 3.2 Course Work The computer security course itself was a combination of lecture and laboratory exercises developed to build expertise in both security concepts and particular tools. The benefits of a laboratory component for a computer security course have been noted repeatedly [1,3,4]. We focused primarily on work with Linux tools in our laboratory exercises, though we also used Windows versions where available to see the different issues that can arise with Windows and to gain experience with some of the tools across multiple platforms. The labs we used were on the following topics, with particular tools used in parentheses: Ethics

Policies and Social Engineering General Information Gathering (ping, traceroute, finger, whois, nslookup/dig, arp, netstat, etc.) Packet Sniffing (tcpdump, ethereal) Password Cracking (john the ripper, l0phtcrack) Cryptography (PGP) Port Scanning (nmap) Vulnerability Assessment (nessus, chkrootkit) Intrusion Detection (snort) By the end of the semester, the students were fairly well aware of the issues and representative tools used in computer security scenarios. However, in future offerings of the course we will add other laboratories on topics such as system hardening (with tools such as bastille) and network/firewall configuration to improve the students background. 3.3 Ethical Issues We were clear to the students from the beginning that the purpose of the course, the lab exercises during the course, and the cyberwar exercise at the end of the course were presented for the purpose of better understanding defense and design of computer systems and networks through the study of both attack and defense strategies. We felt it was ethically important to stress that we were only looking at attack strategies to better understand them from a defensive standpoint. Students were required to sign forms at the beginning of the semester that indicated their willingness to act ethically, follow directions, and not use the tools we were working with outside the scope of the course. Students were warned that violation of this agreement during the cyberwar lab exercise could result in a failing grade in the course and possible University sanction. Contrary to a report from another institution of a cyberwar exercise deteriorating as students became destructive, we had no such problems during the exercise. 4 The Cyberwar Exercise 4.1 Exercise Goals As mentioned above, our goal for the exercise was to give our students further experience with the major issues, strategies and tools involved in computer security and to see how they synthesized the information presented earlier in the course. Specifically, we wanted to achieve the following: give them real-world team-based experience with system defense in a live environment, let them experience attack in order to better understand the strategies, tactics and mindset of the attacker, and to be able to respond defensively in a real-time environment, give them experience with technological, physical and social engineering security. 4.2 Exercise Structure We structured the exercise as a combined defend and attack scenario. The students were divided into teams of 3-4, each assigned to a specific computer system. The overall exercise was a variant of capture the flag that could be more accurately described as plant the flag i.e. attempt to place a file with certain contents on the root or administrative directory of as many other systems as possible. We partitioned the exercise into two phases: defend and attack. First, we told each team what operating system they would be working with, and gave them root access to their system. Each team then had 24 hours to harden and secure their system as well as possible, within certain guidelines. Specifically, they could not change the base operating system, but could only patch the one they were given. We decided to allow them to patch the Linux kernel, even though in some cases this amounted to a partial system upgrade. Also, they had to keep certain services available (a mail server, ssh) to keep the exercise more realistic. The isolated network meant that no automatic update systems (e.g. Redhat Network up2date [7]) could be used. Teams had to download individual libraries, patches, RPMs, and kernels on other machines, and used removable media and sneaker net to patch their systems and bring in software tools for attack and defense. While this may not always be realistic in the noneducational world, it did force the teams to think carefully and specifically about each change they made to their system. Second, each team had another 24 hours to attack any other system on the network (either a team system or one of the three unattended miscellaneous systems) and attempt to plant a flag as specified above. Teams could also further harden their system, and fix any weaknesses discovered during this period. They could not remove a flag, in order to ensure that any successful attacks were counted. Finally, once the exercise was completed, each team had to write a final report that outlined the following: each patch, defensive technique or hardening done during the defense period, each defensive change done during the attack period, and what attack this was in response to, each attack attempted on other systems, each attack that was successful, any miscellaneous comments, issues and/or suggestions that the students felt were important regarding the lab exercise, and a list of locations and sources from which the students gathered their information. This summary not only helped us evaluate how well the exercise met its objectives and how much the students learned during the semester, but gave us a number of new ideas for future lab exercises and discussion topics. 5 Laboratory Setup 5.1 Laboratory Goals Our goal for the laboratory setup was to present a heterogeneous (i.e. multiple operating systems) and realistic network that allowed a large range of attack and defend activity while still maintaining the integrity of the general department and campus network.

5.2 Laboratory Structure While we worked most of the semester with an open network in our security lab (so that the lab could be used by other students as well), we did completely isolate the network for the purposes of our cyberwar exercise, and made the laboratory itself off-limits to non-security class students. We think this is essential, as it is very hard to predict the limits and possible side effects of a cyberwar exercise, and thus it is difficult to protect a campus network from the possible fallout from the exercise without such isolation. We configured eight Linux machines with Redhat Linux [6] v.7.3 for the students. We chose a non-current distribution that had known security holes. The installation was out of the box, with no patches. Several configuration changes were made to further weaken the system. We used Norton Ghost [5] to copy our development image to each system, which ensured that each team system was exactly alike and saved much administrative time. To each team system we added several extra realistically-named accounts (e.g. backup, tomcat, logwd) with weak passwords, in order to allow possible openings through password analysis or even direct login attempts. We also added several other machines to the network, specifically: a Linux machine running a yet older operating system (Redhat 6.2) with several services running: sendmail, apache, bind, nfs, and samba a Windows NT 4 server, service pack 4 with Internet Information Server (IIS) 4, no additional patches a Windows 2000 server with IIS, no additional patches The purpose of these extra machines was to provide a heterogeneous environment, have some systems with no defenders and known security holes, and make it easier for some of the less knowledgeable teams to be successful in the attack phase of the exercise. Finally, we added a heavily secured laptop for monitoring the exercise and our closed network. Our goal was to monitor most or all network activity, but we found it difficult to filter traffic down to a reasonable level, and we were quickly overwhelmed by the amount of network traffic generated by the exercise. It should be mentioned that the assistance of department and/or campus system administrators and network administrators is essential if the instructor is not an expert in these areas and/or does not have authority or ability to modify hardware configurations. The hard work and joint effort of instructors and several IT staff significantly improved the exercise in terms of what we were able to provide to the students. 6 Major Exercise Issues A number of major issues surfaced during the planning and delivery of this exercise. Some of these questions were obvious, but others were more subtle, and only appeared or fully presented themselves during the exercise. The more such issues can be dealt with in advance, the more smoothly such an exercise will go. We present the major issues we dealt with below. First, which services should a machine have to keep active? As our goal was to make the exercise realistic, we required a minimum set of services: mail server and ssh. In retrospect, we would expand this list significantly, and require the availability of other services such as a web server, a database server, and perhaps some sort of application server. Second, how much physical access should be allowed to a nonteam system? We wanted to have the students thinking about physical security, and make them aware of the issues in protecting console access to a system. On the other hand, we didn t want to allow a team to easily gain access to a system directly just because they came into the lab in the middle of the night (as the students couldn t move their machine to a secure room, they were limited in how much physical security they could achieve). Our original non-ideal compromise was to allow keyboard and mouse access, but not allow the placement of bootable or non-bootable floppy disks or CDs into another team s system. We soon found a problem with this rule as most teams put password protection on their system s BIOS and boot process, a team could cause another machine s team to hang by rebooting it from the keyboard and letting it hang at a BIOS or boot password prompt. We ended up prohibiting keyboard or mouse access to other team s systems as well, requiring attack attempts to be entirely through the network. Third, should denial of service (DoS) attacks be allowed? While DoS attacks are certainly a realistic and significant security problem, and knowing how to deal with such attacks is important, we felt with our limited exercise that the negatives of a clogged network and entirely downed systems could prevent the learning gained from being able to work on continually available systems. Thus, we prohibited any DoS attacks during the exercise. We are considering developing a separate laboratory exercise for the course to allow the students to work with DoS attacks and defense against them. 7 Laboratory Experience 7.1 Specific Events Several interesting and not entirely expected events occurred during this exercise. Although none were directly planned for, we would now attempt to ensure the inclusion of these events and/or awareness of the issues behind these events as part of any future presentations of this exercise. First, the extra accounts created were more of a hole than expected. While we had spent an entire earlier exercise on password cracking, not all teams checked their password files during the defense phase. As a result, access was gained to some systems through these added accounts when their passwords were discovered (of course, since the same accounts were on all systems, the better teams tried attacking through these accounts as soon as they discovered the account passwords). Second, the most successful teams were the ones that combined attack strategies. While this was to be expected as the path to making significant progress in the exercise, fewer teams were able to achieve this than expected. For example, several teams gained non-root access through the above-mentioned added accounts. However, only one team was able to upgrade this exploit to gain root access, and this was accomplished through a buffer overflow attack. This reminded us and the students that computer security problems are often based on a series of events rather than a single event, and continued effort and vigilance are essential in computer security matters. Third, while most teams focused on the technological aspects of security for their system attacks, one team attempted a social engineering attack. This was done through spoofed email from the instructor asking each team to set up an account with the

instructor s username and a given password so that he could resolve alleged questions regarding the existence of flags on each system. While we had discussed social engineering at length in the course, most students seemed to think that it was not a significant issue. However, approximately half of the students in the class (including a student system administrator) were fooled by the spoofed message to the extent of creating the requested account. Only the rather lucky occurrence of two students showing up in the instructor s office at the same time to ask about this request exposed the true nature of the message fairly quickly, the accounts were quickly shut back down and no points were gained by this attack. We think that the result would have been different had the instructor not been in his office late that evening. At any rate, we were very glad that this social engineering technique was used during the exercise. We discussed the attack in length afterwards with the class, looking at what aspects worked and what was suspicious about the email message (and what could be used to determine its true invalid nature on closer inspection), and the students came out of the exercise with a much better understanding of the power of social engineering. 7.2 Overall Experience The exercise was considered to be very successful educationally from both the instructor s point of view and from the students point of view. Comments on the exercise reports were very positive both in terms of educational value and enjoyment. When various faculty members went into the lab during this exercise to observe, they found a very high percentage of the students in the lab at any one time, and the energy, engagement and activity levels were very high. While not directly quantifiable from our evaluation, the combination of enthusiasm, work effort, interaction and learning shown by the students in this exercise was higher than any other course in our curriculum except perhaps in our software engineering courses during the final days of a team project. 8 Benefits Based on our evaluation as well as the students evaluation of exercise, we found a number of significant benefits to the use of our cyberwar laboratory exercise in the context of our computer security course. First, the students reported an increased appreciation of security as a process, not as an absolute state. While the work most teams did during the defense phase was quite comprehensive, all teams reported making defensive adjustments during the attack phase. The students better understood the principle we d passed on the first day of the course that security is a process, not a product [8]. Second, the students gained increased appreciation of the use of the tools of the security trade, and how those tools can successfully be used. They reported a better understanding of the use of attacking tools to perform vulnerability assessment of their own network and systems. Third, an exercise like this generates much student enthusiasm. As mentioned above, the students spent a very large amount of time on the exercise, including significant time overnight more than could be expected or demanded late in the semester. When the instructor went into the lab several times to check on progress of the exercise, the lab was essentially full each time. Word spread outside of the class as well, and other students have become aware of and interested in taking our computer security classes in the future. 9 Problems / Lessons Learned for Future Offerings of the Exercise As with any laboratory exercise, problems surfaced during the exercise, and others became apparent as we reflected on the exercise. Several significant ones are discussed below. 9.1 Problems First, the instructor and the students agreed that the time period for each phase of the exercise was too short. They correspondingly recommended that it should have been longer than 24 hours for each of the two periods of attack and defense, in order to allow them more time to research issues that arose as well as to be less intrusive in regard to their other classes. More comments were received in this area regarding the attack period, with most recommending extending it by at least another 24 hours. Second, the students felt that the exercise was given too late in the semester. While they realized the need for background information, the last week of classes had too many other obligations (in terms of projects to finish, starting to study for final exams, etc.) to allow them to spend as much time as they wanted on the exercise. While the students did spend a large amount of time on the exercise, they felt that they wanted to and would have spent even more time if the exercise was slightly earlier in the semester. The instructor also noted some burnout toward the end of the exercise, with some teams and individuals giving up in the final hours. Third, a number of students felt that they didn t have sufficient background for the exercise, primarily in the area of system administration. Many students pick up these skills through jobs, internships and/or personal experience, but a significant number (approximately 25%) of the students felt this was a problem for them. This occurred primarily in the defense portion of the exercise, where students needed to know how to upgrade packages, use Unix utilities like tar and gzip, and in some cases configure upgraded software packages. While the instructor assigned the teams to try to ensure that every team had at least one student who was strong in system administration, the other students sometimes felt left out during the defense phase. Fourth, as this was the first offering of our computer security course, we lowered the prerequisites from the desired Networking course to a more student-friendly Data Structures course. While the enrollment was quite high (30 students), this meant that we needed to teach some networking basics. We thus lost time that we could have spent on other topics. As noted in [4], this is not optimal, and in the long run we will work to add the Networking course prerequisite without significantly dropping the enrollment level. Fifth, some students reported problems when hardening their systems because they didn t know what underlying hardware they were working on. Several incompatibilities were uncovered when patches or upgrades weren t compatible with system hardware.

9.2 Improvements for Future Offerings As a result of the above issues and problems, we are considering the following changes in the next offering of the course exercise (currently scheduled for Spring 2004): Extend the cyberwar exercise to somewhere between 36 and 48 hours or even longer, with the defense phase being at least 12-24 hours and the attack phase being at least 24 hours. Move the exercise to the second to the last week of class to try to avoid interference with other class projects and studying for final exams. Keep the systems more realistic, in terms of requiring additional services (web server, database server, application server) to be up on each system and providing actual data for these services. We will also monitor the systems more closely during the exercise to ensure that teams are keeping the services active. Assign a Windows system as well as a Linux system to each team, which while requiring additional system administration knowledge will make the environment more realistic and open up a larger set of vulnerabilities, exploits and tools for each team. Inject other traffic into the environment, so that the students need to filter legitimate traffic from attacks. This could include teams or the instructor periodically using the services (e.g. web server, database server) on each system to ensure it is still available. Give students a hardware description of their environment so they can determine if there are incompatibilities with patches, updated software, and drivers that they may want to install. Develop a centrally controlled team logging mechanism so that teams can keep an electronic journal during the exercise. This will not only save time for the teams, but assist us in our collection of information from and evaluation of the exercise. Find a better way to monitor the network during the exercise, so that we can collect more information as the exercise proceeds. We think there is a lot to learn from such information if we can find a way of filtering and organizing it. We may need to rely on a commercial tool for this. Require each team to develop and turn in, as part of the exercise, a map of what they think the network looks like what systems they find, what services are available on each, what operating systems and versions are running, etc., which will help the students better understand how an attacker can footprint a system and perform vulnerability assessment. Investigate the use of VMWare [9] to run the exercise in a virtual environment. While we and others have used this product to allow Windows and Linux systems to be installed and available simultaneously on the same machine without rebooting, we can envision this tool allowing us to support a much larger number of machines as part of a cyberwar exercise. 10 Conclusion Our cyberwar exercise was very successful overall in terms of both student learning and the structure of the exercise from a faculty point of view. The team exercise reports and the overall student evaluations graded the exercise very positively. We felt we built on and improved on several other models of such exercises at other institutions, yet also learned much ourselves from issues and problems that occurred during the exercise. As we modify and improve the exercise to build on this experience, the cyberwar exercise will continue to be an integral part of the future offerings of our computer security course. This work is partially supported by NSF Grant 0309818 (CCLI/A&I grant, May 2003). References [1] Hill, John, Carver, Curtis, Humphries, Jeffrey and Pooch, Udo, Using an Isolated Laboratory to Teach Advanced Networks and Security; SIGCSE Bulletin: Proc. 32 nd Technical Symposium on Computer Science Education, v.33, n.1, March 2001, pp. 36-40. [2] Jackson, William, Cadets Keep NSA Crackers At Bay; Government Computer News, May 20, 2002. [3] Mateti, Prabhaker, A Laboratory-Based Course on Internet Security; SIGCSE Bulletin: Proc. 34 th Technical Symposium on Computer Science Education, v.35, n.1, March 2003, pp. 252-256. [4] Micco, Mary and Rossman, Hart, Building a Cyberwar Lab: Lessons Learned; SIGCSE Bulletin: Proc. 33 rd Technical Symposium on Computer Science Education, v.34, n.1, March 2002, pp. 23-27. [5] Norton Ghost, http://www.symantec.com/ghost [6] Redhat Linux, http://www.redhat.com/ [7] Redhat Network up2date Network Automatic Upgrade System, http://rhn.redhat.com/ [8] Schneier, Bruce, Security is not a Product; It s a Process; Cryptogram Online Security Newsletter, http://www.schneier.com/crypto-gram- 9912.html#SecurityIsNotaProductItsaProcess, December 1999. [9] VMWare, http://www.vmware.com