Incident Cost Analysis and Modeling Project

Transcription

1 FINAL REPORT Incident Cost Analysis and Modeling Project I-CAMP II A Report to the USENIX Association 1

2 Project Staff Virginia Rezmierski, Ph.D. Project Director Director, CIO's Office of Policy Development and Education The University of Michigan Adriana Carroll, M.P.P. Project Coordinator and Research Associate Gerald Ford School of Public Policy The University of Michigan Jamie Hine, B.A. Research Associate Gerald Ford School of Public Policy The University of Michigan We are thankful for and acknowledge here the valuable part-time assistance of Todd Lee, M.P.P. and Jason Weller, M.P.P., during data gathering and project design phases of this project. Project Advisory Board Robert Charette Chief Executive Office Risk Management Consultant ITABHI Corporation Kathy Kimball, M. S. Security Director Computer Information Systems Pennsylvania State University Dennis Steinauer Computer Security Division National Institute for Standards and Technology George Cubberly Assistant Risk Manager Office of Associate V. P. for Finance Department of Risk Management The University of Michigan Eugene Spafford, PhD Professor of Computer Sciences Director, CERIAS Project Laboratory Purdue University Larry Stephens, AIC, EPCU, ARM Director of Risk Management Department of Risk Management Indiana University 2

3 Acknowledgments First and foremost, we are thankful to the USENIX Association for their vision and interest in understanding the impact, type, frequency, and cost of IT related incidents in college and university settings. Without their interest, support, and the project funding this work could not have been accomplished. USENIX is the Advanced Computing Systems Association. Since 1975 the USENIX Association has brought together the community of engineers, system administrators, scientists, and technicians working on the cutting edge of the computing world. The USENIX Association and its members are dedicated to: problem-solving with a practical bias, fostering innovation and research that works, communicating rapidly the results of both research and innovation, and providing a neutral forum for the exercise of critical thought and the airing of technical issues. USENIX supports its members' professional and technical development through a variety of on-going activities: Annual technical and system administration conferences, a highly regarded tutorial program, SAGE, a special technical group for system administrators, student programs, and awards programs. Special thanks also go to the members of our Project Advisory Board for I-CAMP II. We thank them for their attendance at I-CAMP II Board Meetings in light of their demanding travel and work responsibilities. Most importantly we thank them for their critical thinking and input to the project. They were not hesitant to make suggestions, to criticize ideas or procedures, or to make themselves available to us when questions arose. Their guidance kept the research team moving forward. We also thank each of the study's participating schools and their personnel who gave of their time and expertise. Instead of cautiously hiding incident data and refusing to openly discuss problems in data collection, these people were sincerely and professionally interested in trying to understand and improve IT-related incident handling on campuses. Special thanks go to Robert Bartlett, Andrea Basing, Mark Bruhn, David Brumley, Katrina Cook, Jacqueline Craig, Jane Drews, Bob Foertsch, Helen Green, Clair Goldsmith, Steve Griese, Stephen Hansen, Susan Levy Haskell, Margie Hodges Shaw, Kathy Kimball, Jim Knox, Doug Nelson, Rodney Peterson, Chris Pruess, Steve Romig, Roger Safian, Jeffrey Savoy, Sara Staebler, Kevin Unrue, Elaine Ward, and Ed Zawacki. Finally, special gratitude goes to the staff of the Office of Policy Development and Education at The University of Michigan for their continued input and support to this project. Thanks go especially to our editor, Kathleen Young for her help with the final report, to David Nesom and Jon Leonard for technical advice, and to our Office Assistant Joyce Ruppert for her support and scheduling of Advisory Board meetings. The original I-CAMP report was funded by the Chief Information Officers of the Committee on Institutional Cooperation (CIC). That report, describes and analyzes thirty technology-related incidents occurring on university campuses and provides discussion of factors that seem to affect both cost and occurrence of the incidents. It is available through the CIC representative by sending to rbaumgar@uiuc.edu. 3

4 TABLE OF CONTENTS TABLE OF CONTENTS 4 EXECUTIVE SUMMARY 7 PREFACE 8 INTRODUCTION 9 The Problem 9 Personnel Skills and Knowledge 9 Unfavorable Trends 9 Time and Skill Required 9 Management Implications 10 I-CAMP STUDY OVERVIEW 10 PURPOSE OF THE I-CAMP II STUDY 11 I-CAMP II METHODOLOGY 11 FIGURE I - ICAMP II PROJECT OVERVIEW 13 EXPANDING STUDY PARTICIPATION 14 PART I -COST ANALYSIS SECTION 15 Purpose 15 Expanding the Sample of Incidents 15 Table I - INCIDENT CATEGORIES TO BE COST ANALYZED 16 Providing Comparison Data 16 Procedure for Incident Identification 16 Procedure for Incident Cost Analysis 16 Assumptions 17 I-CAMP II Methodology for Calculating User-Side Costs 19 Refining and Increasing the Robustness of the Cost Analysis Model 19 Methodologies for calculating User-Side Costs 20 Table II - CALCULATIONS FOR WILLINGNESS TO PAY OF ONE HOUR OF STUDY 22 FIGURE 2 - WILLINGNESS TO PAY AND OPPORTUNITY COST 22 Method of calculation 23 4

5 Table III - Summary of Results from the Incident Cost Analysis 24 Recommendations Regarding Cost Analyzing IT-Incidents in Academic Environments25 Questionnaire Template 25 Calculation Template 25 Machine cost 25 Reports 25 Part II - FREQUENCY SECTION 26 Database Analysis and Categorizations 26 Designing a Methodology and Gathering Frequency Data 26 Availability of Databases 26 Aggregated Data 26 Problems in Data Collection 27 Summarization of Interview Results 27 FIGURE 3 -DATABASE CREATOR ROLE AND PURPOSE 29 Status of School Categorizations 30 Fairness and Justice in Incident Handling 30 Similarities in Incident Categories 30 Factors Appear Again-Lack of Knowledge/Information, Lack of Resources 31 Frequencies from Database Schools 31 Frequencies from Expert Estimates 32 Who Were the Experts? 32 Methodology for Expert Estimates 32 TABLE IV -A, B, C - EXPERT ESTIMATES - MAIL BOMBS 34 TABLE V - A, B, C - EXPERT ESTIMATES - SYSTEM PROBES 37 TABLE VI - A, B, C - EXPERTS ESTIMATES - WAREZ SITES 40 Results of Expert Estimates 43 Towards a Comprehensive Categorization Scheme 43 Literature Review 43 I-CAMP II Recommended Categorization Scheme 43 FIGURE 4 - INTERFACE OF USERS, DATA AND OPERATING SYSTEMS IN THE ACADEMIC ENVIRONMENT 45 FIGURE 5 - I-CAMP II CATEGORIZATION SCHEME 47 SUMMARY AND CONCLUSIONS 48 5

6 FINAL RECOMMENDATIONS 51 APPENDIXES A: QUESTIONNAIRE TEMPLATE B: CONVENTIONS FOR COST VARIABLES C: I-CAMP II INCIDENTS D: QUESTION TEMPLATE E: CATEGORIES AND INCIDENT TYPES USED BY DATABASE SCHOOLS F: INCIDENT FREQUENCY COUNT G: GLOSSARY 6

7 EXECUTIVE SUMMARY Information technology related incidents are occurring on college and university campuses. Some threaten the reliability or integrity of systems or data, operations, reputation, or resources and constitute a risk. To manage these risks senior managers need data to measure potential costs. The I-CAMP II study was designed to provide incident data. The study, funded by the USENIX Association, had two major objectives. The first was to refine the costing model for calculating user costs for IT incidents. I-CAMP I showed that it was easy to calculate worker costs in IT-incidents. The number of hours - needed to resolve the incident was multiplied by the employee's hourly wage. When users are disrupted in using networks or other IT-resources, on which they are increasingly dependent, real costs also exist. This study improved the model for calculating user-side costs. The study confirmed the usefulness of a common template for gathering data on IT-related incidents and for guiding the cost analysis process. It expanded the number and the geographical representation of participating schools in the study to eighteen. It expanded the collection of cost-analyzed incidents by 15 including incidents of compromised access, harmful code, denial-of-service, hacker attacks, and copyright violations. For the 15 incidents, 90 employees were involved, 506 hours were devoted to resolution, and $59,000 in cost, was incurred. The assumption that, though their frequency might be high, the costs of resolving these selected types of incidents would be low, was generally confirmed. The average cost for incidents of access compromise was $1,800, for harmful code was $980, for denial of service was $22,350, for hacker attacks was $2,100, and for copyright violations was $340. The second major objective of the I-CAMP II study was to investigate the availability of incident frequency data and the incident categorization schemes. The study found that only 38% of the participating schools had incident databases. After analysis, the team concluded that colleges and universities are not currently equipped to understand the types of IT-related incidents that are occurring on their campuses. They are not currently able to identify the number or type of incidents or assess the level of organizational impact caused by IT-incidents because of the lack of robust database tools and insufficient staffing. The study team analyzed the currently existing diverse incident categorization schemes. The category schemes reflected the specific roles of their creators, the individual institutional cultures, and organizational needs. Therefore, the frequency counts recorded in these disparate systems could not be statistically compared and aggregated across schools. The team gathered expert estimates of the annual number of occurrences of mail bombs, system probes, and Warez sites. It concluded that expert estimates of incidents logged and handled annually were very similar to actual frequency counts for those same type incidents when compared to data from school databases. It concluded that school size did not appear to affect the level of expert estimates for any of the three types of incidents. They found that, in general, experts as a group, believe they are identifying and handling only about 28% of mailbombs occurring campus-wide-an average of 15 incidents per year, 12% of system probes-an average of 565 per year, and 28% of Warez sites-an average of 15 per year. The study report provides nine specific recommendations for future study and practice. It provides a model for a comprehensive categorization system encompassing both operating system vulnerabilities and interpersonal and policy violations. 7

8 PREFACE This final project report has two major parts; Part I covers Cost Analysis and Part II covers Frequency Analysis. In Part I we describe the problem of information technology incidents on campuses, and provide highlights from the initial Incident and Cost Analysis and Modeling Project (I-CAMP I) study. In the section of Part I, entitled Cost Analysis, we provide the new methodology for calculating user-side costs, analysis of fifteen new and specifically selected IT-related incidents and a guideline for cost analyzing IT-incidents in the academic environment. In Part II of the report we describes the investigation of incident frequencies, the state of existing incident databases, the results of expert frequency estimates, and provide a categorization scheme to assist administrators. The report concludes with a set of recommended best practices. The appendices of this report also provide valuable information. They include, among other information, the actual descriptions and analyses of incidents, the categories currently used by seven of the schools that have incident databases, and an explanation of the cost conventions used in this study. 8

9 INTRODUCTION Information technology related incidents are occurring on college and university campuses. Incidents that threaten the reliability or integrity of systems or data, can constitute risks for the organization, risks to its operations, its reputation, its resources, and effect the trust that members of its community have in the organization. To manage these risks senior managers need to understand them, have sufficient data to measure their potential costs, and make informed management decisions. The I-CAMP 11 study is designed to help provide data regarding such information technology related incidents. The Problem The implementation and rapid evolution of information technology (IT) resources at colleges and universities have increased the number of security and risk management issues for institutions of higher education. Physical and electronic security processes, common to the mainframe environment, are often not suitable in the more distributed computing environment that exists today. Personnel Skills and Knowledge Individuals who handle these distributed services as system administrators have differing levels of sophistication regarding the technology, laws, and ethics governing data security. To guarantee a viable computing environment, colleges and universities are becoming aware that they must provide education and training for the system administrators who manage the environment. They must also ensure increased community awareness of key features of data protection and security, hardware maintenance, software compatibility, disaster planning and recovery, and basic security standards for configuring personal computing devices. Finally, they must take steps to ensure that administrators and senior management understand the legal, fiscal, and ethical implications of physical theft, infrastructure failure, and employee incompetence or inexperience and receive sufficient on-going data to manage responsibly the associated risks within these technology dependent and networked environments. Unfavorable Trends Recent news regarding denial-of-service attacks aimed at Internet sites and unauthorized access and modification of data at major government and commercial sites has raised awareness of potential security threats. Additional information from a recently completed Computer Security Institute survey showed that "90% of respondents (primarily large corporations and government agencies) had detected computer security breaches; 75% reported a variety of serious security breaches such as theft of proprietary information, financial fraud, system penetration from outsiders, denial of service attacks and so on; and 74% acknowledged financial losses due to computer breaches." ' This information is just beginning to cause the administrators within most organizations, profit or non-profit, to devote sufficient fiscal and human resources to security. Time and Skill Required Increased dependency on networks and technology brings with it a growing demand for the availability and reliability of information technology systems. Administrators realize that both time and skill are needed to keep systems and networks operating. "Availability" and "Reliability,"' concepts that have long been recognized as key components of security by security professionals, auditors, and risk managers are now being recognized by students, staff, and faculty as requiring resource commitment, and the time and attention of system administrators. 9

10 Time and skill are required to address known vulnerabilities in operating system and in various applications-to patch known operating system holes. Time and skill are also required to maintain a current and appropriate level of security knowledge. In the fast-changing technology environment, system administrators can not rely on the information they obtained even one year ago to do their current jobs with sufficient competency. The time of systems administrators is also required to provide security education to users and to set standards for ethical, legal, and appropriate use of resources. Time and skill are needed to detect those vulnerabilities that are not readily obvious, to thoroughly understand who is accessing systems and data and whether they have authority to do so. This requires that system administrators gather systems log data, analyze existing patterns, and monitor access and use of resources. These important activities contribute valuable data to address the concerns of auditors, risk managers, and security professionals-data regarding the processes of identification, authentication, and authorization. Without robustness in these three features of security-"identification", "Authentication", and "'Authorization"- and without knowledge of who is accessing and using networks and systems, no resources or data can be secure. Management Implications Risk managers within organizations are finding it difficult to learn about the capabilities of technology at a pace fast enough to understand the implications of new and emerging applications. Risk managers, accustomed to thinking in terms of risks against which the organization can insure, find themselves behind innovation in the area of information technology. IT incidents involving risk, liability, and even significant financial loss are difficult to comprehend unless they are related to physical losses such as fire, flood, or theft-the more commonly known risks for management. Only recently have the large insurance companies begun to recognize that risks and losses within university information technology and networking areas may also require protection and insurance. Like system administrators, risk managers find it difficult to convince senior management of the need for more attention to the management of IT risks and of the importance of systems security. Often it takes a major incident before sufficient attention is paid to security. The absence of data, the press of conflicting demands for fiscal resources, and an environment of rapid technological change can combine to create a climate in which administrators prefer not to hear about-more problems. As a result, a tendency can develop to underestimate the frequency with which incidents occur or to consider individual incident costs as insignificant when compared with the institution's overall expenses. Such an approach, however, leaves an organization open to the possibility of serious financial liability. A single incident may cost only $2,000. If, however, that type of incident is repeated 60 times per month, then the costs to the organization are significantly increased to $120,000 per month or $1,440,000 per year-an amount that would be considered far from insignificant when compared with any institution's overall expenses. Determining the multiplier for specific types of information technology incidents requires data. We need to know about the nature of information technology incidents, about their different types, about the costs associated with each of the different types, and ultimately about the frequency of occurrence on each campus. Do these data exist? I-CAMP STUDY OVERVIEW In 1997, the first "Incident Cost Analysis and Modeling Project," I-CAMP, was funded by the Chief Information Officers of the CIC (Committee for Institutional Cooperation/Big 10) Universities. The objective of the study was to design a cost analysis model for IT-related incidents and to gather and analyze a sample of such incidents. 10

11 University of Michigan staff and graduate students developed the model. They gained access to information about such incidents through partnership with security professionals at each of the 13 CIC campuses. They also described the incidents and provided detailed cost analysis information about 30 such incidents. No particular incident type was sought for this study. Rather, it was the goal to establish a mechanism for gathering information about any IT-related incidents occurring on the campuses and to collect cost data for the analysis. The study began identifying factors that appear to influence the occurrence of IT incidents and those that appear to affect the cost of incidents once they occur. For purposes of the first study, and extended to the present study (I-CAMP II), "incident" was defined as: "Any event that takes place through, on, or constituting information technology resources requiring a staff member or administrator to investigate and/or take action to reestablish, maintain, or protect the resources, services, or data of the community or of its individual members." In summary, the first I-CAMP study examined 30 IT-related incidents and researchers found that: 210 employees were involved in incident investigation/resolution, 9,078 employee hours were devoted to incident investigation/ resolution 270,805 computer /network users were affected by the incidents, and, Calculated costs for the 30 incidents exceeded $1,000,000. Although a model had been developed and beginning data regarding the cost of IT-related incidents was made available through the I-CAMP I study, it was important to refine the cost analysis model, analyze additional incidents to ensure the usefulness of the model, and begin to collect data regarding incident frequencies to allow managers to evaluate organizational risks and costs. A second study was undertaken. This report, parts I and II, describe the work of this second study, ICAMP II. PURPOSE OF THE I-CAMP II STUDY The purpose of the study is threefold: First the study provides guidelines to cost analyze IT-incidents in the academic environment. Through the use of a template, IT personnel are able to identify true costs and follow a guide in analyzing them. This template is used to guide an interview process for the gathering of cost information in each incident. Second, the study analyzes the status of the databases of the participating institutions and their categorization schemes for classifying incidents. It also begins the examination of the frequencies of occurrence for specific types of incidents in three different periods of time (periods of high, medium, and low academic activity.) Finally, the study provides a categorization scheme as a guide to encourage more incident data gathering and to encourage consistency in the classification process. I-CAMP II METHODOLOGY The I-CAMP II study, and this report, have two major parts as represented in Figure 1. (In this figure the circles rep resent data gathering efforts and the rectangles represent output from project activities.) Part I, the Cost 11

12 Analysis section, provides information about the gathering of 15 IT-incidents from the total participant pool-incidents of service interruption and copyright violation. The specific objectives for this section of the project were to increase the robustness of the cost-analysis model, to enhance the collection of IT-incidents, and to confirm the variables and factors that affect the cost and resolution of incidents. Part II, the Frequency Analysis section, provides information from a subset of the total participant pool-those schools with incident databases. In this section of the project, data were gathered in three time periods, April, July, and October. To meet the objectives of: understanding database conditions and categorization schemes, and gaining knowledge of the frequencies for particular types of incidents, the project team completed in-depth interviews with participants and gathered expert estimates of incident occurrences from each of the campuses. As a result of this section, the study then developed and proposed a new categorization scheme for IT-related incidents within academic environments. 12

13 FIGURE 1 - ICAMP II PROJECT OVERVIEW 13

14 EXPANDING STUDY PARTICIPATION In the initial I-CAMP study, existing expertise and cooperation from each of the participating campuses facilitated the study. These partners consisted primarily of the security and policy officers /professionals of the CIC schools. One of the problems that has existed in the past, as individuals have tried to understand the occurrence of IT-related incidents in profit and non-profit organizations, has been an unwillingness on the part of key personnel to share incident information. Their fear is that such incidents may reflect poorly either on the personnel within the organization or on the organizations themselves. In for-profit organizations, such incidents could have direct effects on a customer's trust and therefore on the profitability of the company. In all organizations where prestige and reputation is important, loss of trust can negatively impact the organization's success. Therefore, incident data historically has been difficult to obtain and study. Through the CIC partnerships, this was not the case for the I-CAMP study. For I-CAMP II, we sought to retain the same partners as in the initial study. However, we determined that expanding the pool would be beneficial. This would provide more representative data by providing input from schools with different populations and experiences than the CIC schools. It would potentially increase the dissemination of results and increase the overall investment in the process of incident data analysis. We decided to include large West Coast, East Coast, and central states universities-universities that have had a history of information technology development and use. The same CIC universities as in the initial I-CAMP study were encouraged, and agreed, to participate. These included: Indiana University, Michigan State University, Northwestern University, The Ohio State University, The Pennsylvania State University, Purdue University, The University of Chicago, University of Illinois at Chicago, University of Illinois at Urbana-Champaign, The University of Iowa The University of Michigan-Ann Arbor, University of Minnesota, The University of Wisconsin-Madison. Participation was expanded to include: Stanford University, The University of California, Berkeley, Cornell University, The University of Maryland, and, The University of Texas at Austin. 14

15 PART I -COST ANALYSIS SECTION Purpose The I-CAMP II study was designed to provide system administrators and others with additional information about which IT-related risks are of highest priority to address, about factors relating to their occurrence and costs, and about how best to manage certain risks. The USENIX Association funded the study. The I-CAMP II study first sought to improve the methodology for cost-analyzing IT-related incidents. We wanted to learn more about particular types of incidents that were not included in the first study. We also wanted to create a classification scheme for incidents-a scheme that would help system administrators and others understand how to manage these incidents more effectively. Finally, we wanted to see if it was possible to gather frequency data regarding incident occurrences on campuses. Expanding the Sample of Incidents The first goal for I-CAMP II, noted above, was to enhance the existing model. The best way to determine which factors affect the cost and occurrence of incidents was to expand the sample of same-type incidents. The study team examined the original I-CAMP incidents and discussed the need for more data with IT security personnel from each of the participating schools. We determined that if IT-related incidents were put into two categories-those that were the result of purposeful acts and those that were the result of unwitting acts or accidents--it was the first category about which security professionals were most concerned. Systems security personnel indicated that they needed more data regarding the costs of service interrupts and copyright violations. They believed that while these incidents may be small in cost, these incidents are occurring with high and growing frequency on campuses. The aggregate costs of these types of incidents may be significant. Project staff continued also to be interested in data and identity theft. However, little information is currently available regarding such incidents. While it appears that a combination of data stewards, such as university registrars or directors of personnel, and law enforcement/campus safety personnel, are in the best position to learn about incidents of this type, it also appears that incidents of this type are only starting to be recognized on college and university campuses. Because of the paucity of information, I-CAMP II did not include this type of incident in the study. The I-CAMP II study gathered and cost analyzed data regarding Purposeful/ Malicious behaviors of two types: 1) Service Interruptions- specifically, compromised access, insertion of harmful code, and denial of service, and 2) Copyright Violations- specifically, distribution of MP3 and Warez distribution of illegal software. The study goal was to augment the first sample of incidents (N=30) by adding the analysis of a small sample of these specific type incidents (N=15). (15 of these were analyzed and appear in Appendix C of this report). The incident categories collected in I-CAMP II appear in Table I. 15

16 Providing Comparison Data The current study sought to "gather more same-type incidents to facilitate analysis and comparisons and refine the cost analysis model." It was determined that within the scope and time allowed for this study it would not be possible to gather a large enough number of same-type incidents to provide a statistically significant sample (40-50) for comparisons. The project Advisory Board recommended that instead, we narrow the study--collect and analyze three of each type of the targeted incidents if possible. Three same-type incidents would allow the project team to begin comparing costs and actions that were necessary to resolve the incidents. Two or more of each of the specific incident types shown in Table I were collected. Procedure for Incident Identification The system administrators of the participating schools identified incidents. As in the initial study, a specific process was used to gain access to campus information. Authorization came directly from campus Chief Information Officers (CIO's) who identified the key staff members from whom data should initially be sought and who relayed those names to the study team. Participating personnel in each of the 18 universities were then asked to identify incidents of service interrupt and copyright violations. Specifically, they were asked to inform the I-CAMP II team of access compromises, insertion of harmful code such as NetBus, Back Orifice, and others, denial of service attacks such as mail bombs, ping attacks, smurf attacks and others, and incidents involving distribution of MP3 and Warez. (See Glossary for definition of terms.) Procedure for Incident Cost Analysis Once an incident was identified, data gathering was accomplished in person through a visit to the campus, or by telephone call using a questionnaire template (see Appendix A). Often an incident required follow-up activities 16

17 to clarify particular aspects of the event, gather a piece of needed data, or ask a question when some aspect of the event was omitted. Additional details were exchanged using electronic mail or file transfer. (The standard template for cost analyzing IT-incidents appears in Appendix A.) Assumptions This section of the report details the assumptions and methods used in gathering data and the manner in which cost variables were treated. This information should be used as a guide for understanding the subsequent incident analyses. (Refer to Appendix B for a detailed description of the conventions used in calculating costs.) Assumption I - Truthful Information We assumed that the information that we received from the people directly involved in an incident was truthful to the best of their knowledge. Other than an occasional log of employee actions, we depended primarily on the person's best recollection of events. We attempted to gather data as close to the incident's occurrence as possible to minimize data loss due to memory lapses. While some measure of error exists when recalling past events, we have no grounds for disputing the information conveyed. If an incident was too old to gather valid data, it was not included in the study. All incident data collected were subjected to a final review by the provider of the data prior to inclusion in this report. Assumption 2 - Appropriate Data Suppliers We assumed that the individuals identified to provide data about incidents were appropriate and valuable for the purposes of this study. Within each participating school, others who had some involvement with a particular incident were identified to us for purposes of more complete data gathering. We recognized, however, that as a result of their association with the information technology organizations of these colleges and universities, they would identify incidents of one type more often than might individuals in non-technology departments. Assumption 3 - User-Side Costs Regarding the costs on the user side of the equation, we assumed that: a) The tuition fee is the basis for the calculation using this methodology. This fee includes all the academic resources that the university offers to the student: libraries, professors, rooms, places to study, networked services, computer rooms, restrooms, etc. It is too difficult to separate which part of the overall fee corresponds to the particular service offered. b) For each credit hour, the student incurs three "study hours." The total number of study hours includes class time plus required preparation. For example, a 3- course entails 3 hours in the classroom plus 9 hours of additional study and hence, a total of 12 hours of study per week. Note: This calculation, three times the number of in-class hours for each credit is an accepted standard device for estimating preparation time in most universities. c) The fee/ student cost used for calculations depends on the number of in-state and out-of-state students in a particular university. From these numbers we can calculate an average student cost. To illustrate this ideal, we took the weighted student cost for a semester for one of the participating schools and we calculated, $17, d) It is "virtually impossible to speak to every affected user to determine her or his real loss in dollars. We expect that when the network manager is learning about the resolution of an incident, she or he can ask the users involved questions about how much time the user lost as a result of the IT-incident. Alternatively, the network engineer can estimate the traffic of users connected to the computer network at a specific period of time. These estimations can help to identify (either by increasing or decreasing) the number of users affected when there is 17

18 an incident that affects the entire community (e.g., a probe that results in a massive denial-of-service for members of the campus.) We do not estimate any loss of hours due to an IT-incident. The following methodology (see "I-CAMP II Methodology for Calculating User-Side Costs") for the user side is based on the concept of the willingness to pay and the individual opportunity cost. Both methodologies will be used in instances where the user cannot perform an alternative activity; e.g., if the paper that the student was writing was stored on a hard drive that is compromised and cannot be obtained to be turned in on time. When quantifying an incident, we do so from a university's perspective. However, we made the assumption that the term "university" implies the entire community of students, faculty, and staff. (Some feel that if the cost of an incident is not a direct cost to a university department it is not a cost to the university. This is not the position taken here.) Thus, costs borne by students, for example, from the inability to complete work as a result of a server crash, are considered a real cost to the university and estimated for reporting even though the university may not directly pay out resources. Generally, any quantifiable cost borne by any member of the community as a result of an incident, if we are able to estimate it reasonably, is included in our calculations. Otherwise, the real costs and their implications are described qualitatively in the incident report. There were those who, when reading the first I-CAMP study results, commented that costs associated with system administrators who repair computers or networks as a result of unauthorized intrusions, for example, should not be included because they were already assumed as part of their salaries and their expected work. We might assume that if a system administrator spent all of his/her time managing IT-related incidents, there would be a real cost to the institution. The system administrator would be totally unavailable to perform the other duties associated with their position such as configuring machines, trouble-shooting network and system problems, supporting users, and so on. Likewise, we might assume that if no one spent time and effort dealing with the IT-related incidents that are occurring, there would be a real cost to the institution through unmanaged risk, and potential liability. Depending on the frequency and impact of the incident and between total and no response lies the real cost to the institution. For purpose of this study, we assumed that employee effort to detect and manage IT-related incidents should be considered a real cost to the institution, and therefore be included in the analysis. In I-CAMP I, user costs for faculty and staff were calculated on an hourly basis from their stated salaries. In that study, student costs, where the number of students affected by an incident was known, were calculated according to an average part-time wage commensurate with undergraduate employment wages and graduate employment wages. It was assumed that students would have been working if they had not been interrupted by the downtime caused by the incident. Assumption 4 - Limiting Incident Scope For the purposes of this study, we concluded our cost analysis when the network, system, LAN, PC, or other environment was returned to its pre-incident condition. The decision of when to close the quantification of an incident is debatable and, in some sense, arbitrary. Often it involves a judgment in terms of natural closure-that is, a judgment of when the incident really ended. In general, we attempted to capture the essence of an incident without carrying it out too far. Therefore, if a security audit or review was stimulated by the event and was performed after the problem had been resolved or the hole closed, it was not included in the cost analysis. For the purposes of this study, we did not consider these additional events and their concomitant costs to be directly related to the cost analysis of the incident. 18

19 Assumption 5 - Common Costs Excluded There are specific variables common to all incidents that we did not attempt to quantify unless they presented themselves as inordinately large in proportion to and specifically related to the overall incident. Generally, these variables did not provide any clearer sense of the situation, but would have required a significant commitment to data gathering. Included in this category are office supply costs (such as paper and pens), telephone bills, and costs of secretarial support to the individuals involved. Assumption 6 - Study Team Cost Excluded We did not include the time spent by the researchers of this project as part of the overall costs to a university. Under normal circumstances, an incident would not include an investigation by a separate party; thus, we did not want to skew the results of an incident analysis by including the commitment of the project team. I-CAMP II Methodology for Calculating User-Side Costs Refining and Increasing the Robustness of the Cost Analysis Model We felt that it was important to focus a fair amount of attention to refining this part of the cost analysis model because students and their time, as well as faculty and staff, are the users of these information technology systems. Students are the customers of colleges and universities. Even though their losses in productivity do not reflect directly on the budget expenditures of the organization, such losses reflect on and affect perceptions of the value of the educational experience at a college or university, and the satisfaction of the students -therefore its reputation. In calculating the costs of an IT-related incident, there are actual costs related to actions that need to be taken to manage the incident and return the environment to its original state. These actions may include incident investigation, patching identified vulnerabilities, repairing or replacing systems or applications, managing publicity, supporting and informing users, and so on. There are also costs related to the effects of the incident on users. The original I-CAMP model underscored the difficulty in estimating the costs to users when an IT-related incident occurs. The authors of the first report specifically identified three areas of difficulty: "First, it is virtually impossible to speak to every affected user to determine his or her real loss in dollars. Second, we cannot say for certain what the user's time is worth. Wage rates are traditional measures of a person's time, but it is difficult to put a number on, for instance, a student who is not employed. Third, opportunity costs are always involved. If a person cannot retrieve needed information from the network, he or she may be able to do some alternative activity that provides some utility. Measuring the difference between the real loss and gain in utility from an alternative activity is a difficult task at best.,, 2 A revised model for calculating the student user-side costs was developed for the ICAMP II project. This model is based on -the theory of willingness to pay as an approach to the concept of an allied market. It provides what we consider to be an improved methodology for calculating the user cost of IT-related incidents. Although the cost of the user side is not a direct cost for the university, it will have an important relevance when it is included as a shadow price. The I-CAMP II team learned that there is no single methodology for cost analyzing the user side of an incident. Cost varies according to the type of incident. The two incident types we gathered for I-CAMP II (service interruptions and copyright violations) have two potentially different cost approximations: a) the marginal cost 19

20 to access the network, and b) the willingness to pay for one hour of study. These different cost approximations-this methodology for examining costs-is based on the work of economist Edward Gramlich's concept of allied markets. He states: "... Many times in a benefit-cost study there will be changes in quantities for which there is no market...' Further, "...the simplest answer is to -try to find some allied market where the price or quantity change can be used to infer valuations for the missing market". " Methodologies for calculating User-Side Costs Marginal cost to access the network If an IT-incident denies access to a university's network, the user could pay for a connection to another service provider. In such cases, our proposed methodology uses the marginal average cost of the connection to another server times the hours the user spent connected to it (or, when appropriate, the fixed cost). The marginal average cost of the connection to another server can be obtained from Internet Service Providers. It is the simple average cost of one hour of connection time. Willingness to pay for one hour of study If the user loses worktime because he or she cannot access the hard drive of the computer that has been compromised or shut down, the cost analysis methodology depends on the type of user. If the user is a professor or a staff member, the analysis technique will be the individual opportunity cost, approximated by using the hourly wage rate. If, on the other hand, the user is a student, the methodology is the willingness to pay for one hour of study. Based on the concept of allied markets, the methodology of willingness to pay can be used to derive missing values that previously were roughly estimated. The willingness to pay is a good alternative methodology. As shown in Figure 2, we derive marginal income from the consumer demand curve. It is interpreted as the willingness to pay where consumers, at quantity Q, are willing to pay just P for the last unit of product, but no more. Thus, the price that consumers are willing to pay exactly equals the marginal utility for the good they receive. Therefore, at the economically efficient allocation point, marginal income equals marginal cost (the willingness to pay is equal to the opportunity cost). But when there is an imbalance, markets suffer from irrationality problems. Such discontinuity makes the willingness to pay different from the opportunity cost. If this occurs, we must ask how to approach the imperfect market-from the demand side (the willingness to pay) or from the supply side (the opportunity cost). The willingness to pay is a direct measurement obtained through an estimation of demand. That is, the amount of money the consumer is willing to pay for one additional unit of good. Conversely, the opportunity cost is an indirect measure addressed by the offer. In other words, the market states a price for the last unit of good, without differentiating if the price is correctly allocated. 20

21 To illustrate this concept more clearly, let's assume that in the unskilled labor market, students are offered a job. The unskilled labor worker is defined as the market where people having a high school education or less are offered a job. Students fall under this category because employers do not differentiate between a student and an unskilled worker until the students have received their college degrees.' When a student is willing to work while he/she attends school the opportunity cost is an hourly salary rate of $6.00. As can be seen on the left side of Figure II, the unskilled labor market represents the opportunity cost. Conversely, as shown on the right side of Figure 2, we assume a "study market" denoted by a fixed supply curve representing the university's student cost (or tuition fee)" and a demand curve for study (without differentiating quality or quantity). We also assume that students are rational agents that will search for a university that best matches their preferences, depending on costs of tuition fee and living, quality of the degree they are pursuing and services offered by the university. Under such assumptions, the willingness to pay for one hour of study is equal to $15.00, the cost of a marginal hour of study (based on an average weighted student cost of $10,000.) What is "one hour of study"? To study in an academic environment requires interaction with the computer. Thus, "one hour of study" would likely involve computers and networks. Using for communication between the school and the student, or between students, has become the norm. Students use computers to write essays and articles, work with data, work in the laboratories, and so on. The computer network is an essential part of the academic experience. 21

22 FIGURE 2 22

23 Method of calculation As we showed early in the assumptions, on average, students take 4 classes per term with 3 credits for each class. Each class entails 12 hours of study per week (3 hours in class plus 9 additional hours of study). Therefore, in a month, a student is required to study 48 hours per week, or the equivalent of 192 hours per month. On average a semester lasts 3.5 months. Taking the student cost of X dollars and dividing it by the total hours per month times the duration of the semester, we obtained the student's willingness to pay for one hour of study, or the marginal hour of study equal to $ In summary, when the user is a student' and he/she is involved in an IT-related incident that causes a loss of time, the cost for the user's time should be $15.00 per hour, the weighted average of in- and out-of-state student cost plus fees. If the user is a. professor, the cost should be calculated at his/her hourly wage rate. In reality, each university has different student costs. When applying this methodology, each university will have to calculate its own weighted average. For example, for the calculations of the user cost in the IT-incidents we cost analyzed, use the respective tuition fee (out-of-state vs. in-state) and student enrollment figures from each respective university. We applied this methodology in the incidents entitled, "Experts Lying", "Jumping Hacker", "'Possessed Mouse", and "Post Fourth of July", since these incidents involved specific user costs. (Table III provides a summary of results.) To augment the incident sample, a total of 15 incidents of the two types-service interruptions and copyright violations- was gathered and cost analyzed. Within the service interruption category, the sample contains three examples of compromised accounts, three of hacker attacks, three of harmful code insertion, and three denial-of-service attacks. Within the copyright violation category, the sample contains three MP3 incidents. 23

24 Table III - Summary of Results from the Incident Cost Analysis Examples of five types of selected incidents were collected, described, and cost analyzed. In these 15 incidents, we found the following: 90 employees were involved in incident investigation and resolution. 506 employee hours were devoted to incident investigation and/or resolution. The estimated number of computer and network users who were affected by the incidents was not available. Calculated costs for the 15 incidents totaled $59,250. The average calculated cost for the (2) compromises of access incidents was $1,800. The average calculated cost for the (3) harmful code incidents was $980. The average calculated cost for the (2) denial-of-service incidents was $22,350. The average calculated cost for the (3) hacker attacks incidents was $2,100. The average calculated cost for the (5) copyright violations incidents was $340. At first review, the cost figures for these incidents appear so small as to be entirely insignificant. However, it is important to remember two key points: We purposefully solicited these types of incidents because security and policy experts in the participating schools perceived that the frequency of occurrence of these types of incidents was high or rising, therefore, the overall costs to the organization may indeed be significant. We have used the most conservative figures for calculating costs in all cases. For these types of incidents it is extremely difficult to understand user costs because knowing the number of users that were actually affected in a denial-of-service attack, for instance, is impossible. The average costs are small and they provide only the beginning insights into the overall costs of these incidents for a campus. 24