SQUARE Project: Cost/Benefit Analysis Framework for Information Security Improvement Projects in Small Companies

Transcription

1 SQUARE Project: Cost/Beneft Analyss Framework for Informaton Securty Improvement Projects n Small Companes System Qualty Requrements Engneerng (SQUARE) Team Nck (Nng) Xe Nancy R. Mead, Advsor Contrbutors: Peter Chen Marjon Dean Llan Lopez Don Ojoko-Adams Hasan Osman November 2004 Techncal Note CMU/SEI-2004-TN-045 Networked Systems Survvablty Program Unlmted dstrbuton subject to the copyrght.

2 Ths work s sponsored by the U.S. Department of Defense. The Software Engneerng Insttute s a federally funded research and development center sponsored by the U.S. Department of Defense. Copyrght 2004 Carnege Mellon Unversty. NO WARRANTY THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. Use of any trademarks n ths report s not ntended n any way to nfrnge on the rghts of the trademark holder. Internal use. Permsson to reproduce ths document and to prepare dervatve works from ths document for nternal use s granted, provded the copyrght and No Warranty statements are ncluded wth all reproductons and dervatve works. External use. Requests for permsson to reproduce ths document or prepare dervatve works of ths document for external and commercal use should be addressed to the SEI Lcensng Agent. Ths work was created n the performance of Federal Government Contract Number F C-0003 wth Carnege Mellon Unversty for the operaton of the Software Engneerng Insttute, a federally funded research and development center. The Government of the Unted States has a royalty-free government-purpose lcense to use, duplcate, or dsclose the work, n whole or n part and n any manner, and to have or permt others to do so, for government purposes pursuant to the copyrght lcense under the clause at For nformaton about purchasng paper copes of SEI reports, please vst the publcatons porton of our Web ste (

3 Contents Abstract...v 1 Introducton The Problem A Framework for Cost/Beneft Analyss The Acme Company System Overvew Busness Goals Securty Objectves Cost/Beneft Analyss Framework Terms and Concepts Methodology Stakeholder Involvement Evaluaton Crtera Maxmzng System Value Wthn Real-Lfe Budget Constrants Cost/Beneft Analyss Framework n Practce Msuse Cases Categores of Threats Rsk Exposures Archtectural Recommendatons Polcy Recommendatons Total System Value Versus Total Implementaton Costs Beneft/Cost Rato Versus Total Implementaton Costs Total Implementaton Costs Versus Rsk Exposures Values Versus Rsk Exposures Lessons Learned Msuse Cases Estmaton of Losses Estmaton of Costs Cost Structures of Securty Improvement Projects...24 CMU/SEI-2004-TN-045

4 4.5 Values of Securty Improvement Projects Conclusons Future Work References CMU/SEI-2004-TN-045

5 Lst of Fgures Fgure 1: Categorzaton of Threats, Msuse Cases, and Incdents... 6 Fgure 2: Total System Value vs. Total Implementaton Costs Fgure 3: Beneft/Cost Rato vs. Total Implementaton Costs Fgure 4: Total Implementaton Costs vs. Rsk Exposures Fgure 5: Values vs. Rsk Exposures CMU/SEI-2004-TN-045

6 v CMU/SEI-2004-TN-045

7 Lst of Tables Table 1: Terms and Concepts Used n the Framework... 4 Table 2: Example Msuse Case Table 3: Cost Estmates for Archtectural Recommendatons Table 4: Cost Estmates for Polcy Recommendatons CMU/SEI-2004-TN-045 v

8 v CMU/SEI-2004-TN-045

9 Abstract Many companes rely on hstorcal data to buld predctablty models for cost/beneft justfcaton of future projects. Unfortunately, for small companes, whch generally do not have a process for collectng securty data, the costs and the benefts of nformaton securty mprovement projects have been very dffcult to estmate and justfy. In addton, detaled attack data are smply not avalable to be used as references n cost estmatons. Gven these dffcultes, many small companes choose to gnore entrely the securty vulnerabltes n ther systems, and many suffer the consequences of securty breaches and sgnfcant fnancal loss. Small companes that do mplement securty mprovement projects often have problems understandng the cost structures of ther mprovement ntatves and how to translate rsk exposures nto costs that can be passed on to ther customers. To deal wth the aforementoned problems, ths report descrbes a general framework for herarchcal cost/beneft analyss amed at provdng acceptable estmatons for small companes n ther nformaton securty mprovement projects. The framework classfes msuse cases nto categores of threats for whch natonally surveyed rsks and fnancal data are publcly avalable. For each category of threats, costs, benefts, baselne rsks, and resdual rsks are estmated. The framework then generates all permutatons of possble solutons and analyzes the most optmal approach to maxmze the value of securty mprovement projects. The framework analyzes the problems from fve dmensons: Total Implementaton Costs, Total System Value, Net Project Value, Beneft/Cost Rato, and Rsk Exposures. The fnal proposed system wll be derved from the comparsons of these dmensons, takng nto consderaton each company s specfc stuaton. Ths report s one of a seres of reports resultng from research conducted by the System Qualty Requrements Engneerng (SQUARE) Team as part of an ndependent research and development project of the Software Engneerng Insttute. CMU/SEI-2004-TN-045 v

10 v CMU/SEI-2004-TN-045

11 1 Introducton The purpose of a cost/beneft analyss s to provde a set of quanttatve metrcs to assst companes n ther decson makng. In nformaton securty mprovement projects, such analyss can provde nsghts about whch vulnerabltes and/or desgn flaws to fx, n what order of mportance, and for how much nvestment. By assocatng a calbrated monetary amount wth each rsk, vulnerablty, cost tem, and recommendaton, a cost/beneft analyss enables companes to compare and contrast avalable alternatves and to arrve at a sound decson wth fnancal justfcaton. 1.1 The Problem Informaton securty data has tradtonally been very dffcult to collect. In small companes, where human resources are especally scarce, the process of collectng data on the annual number of securty breaches and ther resultng fnancal losses s typcally non-exstent. Ths creates the problem that, on one hand, small companes need relable data to make good decsons, and on the other hand, they cannot have data when no one has tme to collect data. In addton, securty rsk s often an unknown quantty, because no one can predct the exact tme and methods of future securty ncdents. Busnesses can only hope to reduce rsk and potental loss by mplementng securty solutons. At a detaled level, there s often a manyto-many relatonshp between rsks and securty mprovement measures, and t s dffcult to compute the actual rsk versus the cost for each specfc msuse and attack. Wthout relable hstorcal data and/or comparable thrd-party data, small companes are usually at a loss about whether to mplement ther securty mprovement projects. Many small companes choose to gnore entrely the securty vulnerabltes n ther systems, and many suffer the consequences of securty breaches and sgnfcant fnancal loss when attacks occur. Small companes that do mplement the securty mprovement projects often have problems understandng the cost structures of ther mprovement ntatves and how to translate rsk reducton nto costs that can be passed on to ther customers. 1.2 A Framework for Cost/Beneft Analyss To deal wth the aforementoned problems, we have devsed the Cost/Beneft Analyss Framework, a general framework for herarchcal cost/beneft analyss amed at provdng acceptable estmatons for small companes n ther nformaton securty mprovement projects. The framework classfes msuse cases nto categores of threats for whch natonally surveyed rsks and fnancal data are publcly avalable. For each category of threats, costs, benefts, baselne rsks, and resdual rsks are estmated. The framework then CMU/SEI-2004-TN-045 1

12 generates all permutatons of possble solutons and analyzes the most optmal approach to maxmze the value of securty mprovement projects. The framework s descrbed n detal n Secton The Acme Company Throughout ths report we wll use the Acme Company as the alas of our real-lfe clent. The Acme Company s a small start-up software company. Its core product has attracted nterests from several large prospects. However, before deals can be sgned, these prospect companes demand that the Acme Company show them that the product s reasonably secure when deployed n large, heterogeneous enterprse envronments. Because of customer demands, the Acme Company s plannng to ntate a project to mprove the securty of ts product. Before the project s undertaken, however, ts costs must be justfed relatve to ts benefts. An applcaton of the framework to the Acme Company example s dscussed n Secton System Overvew The Acme Company s core product s a web-based n-ter asset management system wth browser clents, web servers, applcaton servers, and database components. It has an exstng clent nstallaton base. Currently t s undergong a major mgraton to a new verson. It remans to be shown whether the system can be reasonably secure when deployed n a large, heterogeneous enterprse envronment Busness Goals As wth any busness, one of the Acme Company s man objectves s to make a proft. In addton to the securty objectves presented n ths document, Acme wants to keep focus on ts busness goals of ncreasng profts and market share n the ndustry. Hence, ncorporatng securty mprovements should work n parallel wth the orgnal objectves rather than aganst them Securty Objectves The followng are Acme s securty objectves for ts asset management system. They are lsted alphabetcally. Avalablty: The busness purpose of the system can be met, and the system s accessble to those who need to use t [SANS 03]. Confdentalty: Informaton s not made avalable or dsclosed to unauthorzed ndvduals, enttes, or processes (.e., to any unauthorzed system entty) [SANS 03]. 2 CMU/SEI-2004-TN-045

13 Integrty: The system performs ts ntended functon n an unmpared manner, free from delberate or nadvertent unauthorzed manpulaton. Data n the system are not changed, destroyed, or lost n an unauthorzed or accdental manner [Allen 99]. CMU/SEI-2004-TN-045 3

14 2 Cost/Beneft Analyss Framework 2.1 Terms and Concepts Terms and concepts used n the Cost/Beneft Analyss Framework are defned n Table 1. Table 1: Terms and Concepts Used n the Framework Category of Threats Category of Preventons Baselne Rsk Bypass Rate Resdual Rsk Net Present Value (NPV) a set of related msuses and attacks that pose threats to the organzaton a set of recommendatons that suffcently mtgate a Category of Threats. A Category of Preventons has a one-to-one relatonshp wth a Category of Threats. ncdent rsk to the organzaton f no securty solutons are n place probablty that an attack wll penetrate a gven securty soluton and result n observable damage. A 100% bypass rate means the securty soluton does not stop any ncdents; a 0% bypass rate means the securty soluton stops all ncdents. 1 ncdent rsk to the organzaton f securty solutons are properly nstalled, utlzed, and montored. Resdual Rsk = Baselne Rsks x Bypass Rate. the present value of an nvestment s future net cash flow mnus the ntal nvestment 2.2 Methodology The Cost/Beneft Analyss Framework derves ts cost and beneft fgures from msuse cases and the archtectural and polcy recommendatons needed to mtgate these msuse cases. We 1 The authors gratefully acknowledge the deas expressed by Arora et al. that all securty solutons are subject to rate of falures (bypass), whch needs to be accounted for n the rsk reducton analyss. (Arora, Ashsh; Hall, Denns; Pnto, C. Arel; Ramsey, Dwayne; & Telang, Rahul. An Ounce of Preventon vs. a Pound of Cure: How Can We Measure the Value of IT Securty Solutons? Carnege Mellon CyLab, 2004.) 4 CMU/SEI-2004-TN-045

15 wll not explan n depth how to generate msuse cases and recommendatons, snce they are very company and project specfc. To llustrate our ponts, we wll show examples of msuse cases and recommendatons n Secton 3. The framework categorzes related msuses nto Categores of Threats, whch are sets of related msuses and attacks that pose threats to the organzaton. Examples of Categores of Threats nclude denal of servce, system penetraton, and sabotage of data. Categorzaton has several benefts. Frst, categores are hgh level and easy to understand by busness users. Second, categorzaton reduces the scope and the dmensons of the problem by aggregatng on top of related msuse cases, whch themselves are aggregates of ncdents. Thrd, categores are relatvely dstnct from each other. We are assumng that the effects of mtgatng rsks n one Category of Threats are neglgble to other Categores of Threats. Ths assumpton allows us to compute ndependently costs of mplementatons for each Category of Preventons wthout worryng about overlappng cardnaltes. Fnally, attack and loss data for Categores of Threats can be found n natonal surveys, whch provde reasonable estmates for small companes wthout forcng them to nvest large amounts of human resources n data collecton or research. The most dffcult problem for any small company s the lack of hstorcal data or comparable external data to base ts analyss on. Therefore, reasonable assumptons need to be made n the areas of expected probabltes and consequences when the company s subject to msuses and attacks. In most cases, even for large companes, we cannot accurately predct when and how an attack wll happen. However, these challenges can be overcome wth threat categorzaton. Annual natonal surveys have shown that over the perod of a year, Categores of Threats have average probabltes of occurrences and ranges of fnancal losses due to exposures to these Categores of Threats. Because these Categores of Threats are general and encompassng, they can be assumed to nclude most of the msuses and attacks that a small company s lkely to face. By not concentratng on each specfc msuse or attack that a company may face, small companes can avod gettng consumed by over-detaled rsk modelng that they have no resources or relable data to do. Instead, by focusng on mtgatng Categores of Threats, small companes wll have reasonable estmatons of ther expected loss f they were to take no actons aganst a set of probable msuse cases. From Categores of Threats they can quantfy and prortze sets of securty mprovement measures wth respect to ther hgh-level securty and busness goals. We call these securty mprovement measures Categores of Preventons. They have one-to-one relatonshps wth Categores of Threats. CMU/SEI-2004-TN-045 5

16 Enterprse Securty Legend Entty Connector And Category of Threats 1 Category of Threats N Or Msuse Case 1 Msuse Case S Msuse Case 1 Msuse Case T Incdent 1 Incdent Incdent 1 Incdent j Incdent 1 Incdent k Incdent 1 Incdent l Fgure 1: Categorzaton of Threats, Msuse Cases, and Incdents The framework takes fnancal and probablstc data from annual natonal surveys for each Category of Threats. The prncpal assumpton s that a small company s subject to attacks and msuses at probabltes at or near natonal average. If the company cannot provde an estmate for the expected loss when msuses happen, lower ends of natonally surveyed loss are used as cost avodance tems for mplementng securty mprovement measures. We use the lower end because small companes typcally do not have as many assets to lose as larger companes. The goal of the framework s to support better decson-makng to ensure that resources are effectvely allocated n the lfetme of the project. Typcally, a securty mprovement project runs for M number of years and there are N possble Categores of Preventons to mplement. If and only f all the archtectural and polcy recommendatons n a Category of Preventons are mplemented do we consder the rsks n ts correspondng Category of Threats mtgated; otherwse Category of Threats s consdered not to have been mtgated. Let s defne the followng: X = 1 f we are gong to mplement a Category of Preventon ( = 1, 2 N) = 0 f we are not gong to mplement a Category of Preventon ( = 1, 2 N) 6 CMU/SEI-2004-TN-045

17 Usng the aforementoned probabltes from the surveys, Margn of Safety and Rsk Exposures of a company s exstng system can be calculated. Margn of Safety s the probablty that none of the categores of threats happen at all wthn a year. Therefore, t s the accumulatve product of (1 probabltes of a Category of Threats happenng). The probablty of a Category of Threats happenng wll dffer dependng on whether the gven Category of Threats has been mtgated. When unmtgated, a partcular Category of Threats wll have Baselne Rsk (ncdent rsk to the organzaton f no securty solutons are n place) assumed at natonal average; when mtgated, the same Category of Threats wll have only Resdual Rsk, whch s the ncdent rsk to the organzaton even f securty solutons are properly nstalled, utlzed, and montored. However, even wth proper securty solutons n place, an attack stll mght penetrate the securty solutons and result n observable damage. The rate of such occurrence s thus defned as the Bypass Rate. A 100% Bypass Rate means the securty soluton does not stop any ncdents; a 0% Bypass Rate means the securty soluton stops all ncdents. For small companes, whch typcally do not have volumnous data on ther nformaton securty, a reasonable estmate of Bypass Rate can be used. Ths s the case n the Acme Company example n Secton 3. For = 1, 2.N number of possble Categores of Preventons: Resdual Rsk = Baselne Rsk x Bypass Rate Margn of Safety (1 P{ attacked } ) N = 1 N = 1 ( 1 P ) P = Baselne_R sk f X = 0 Resdual_R sk f X = 1 Example 1: Rsk Exposure = 1 Margn of Safety If a company currently has a 60% lkelhood of encounterng msuse ncdents n Category A and a 30% lkelhood of encounterng msuse ncdents n Category B, then: Baselne Rsk (A) = 60% Baselne Rsk (B) = 30% Margn of Safety = (1-60%) x (1 30%) = 28% Rsk Exposure = 1 28% = 72% when no acton s taken. CMU/SEI-2004-TN-045 7

18 Example 1, contnued: The company can take steps to mtgate Category A and/or Category B. The avalable soluton for A s hghly effectve (Bypass Rate of 5%), but the avalable soluton for B s not effectve (Bypass Rate of 70%), so: Resdual Rsk (A) = 60% * 5% = 3% Resdual Rsk (B) = 30% * 70% = 21% When steps to address both A & B are mplemented: Margn of Safety = ( 1 3% ) x (1 21%) = 77% Rsk Exposure = 1 77% = 23% The Cost/Beneft Analyss Framework employs the formula of Annualzed Loss n each category multpled by Baselne Rsk n each category to calculate the Baselne Cost n each category. The Baselne Cost s the amount n dollars that an organzaton s expected to lose by takng no acton aganst a Category of Threats. The Annualzed Loss s then used to derve the Tangble Benefts n the Benefts secton (cost avodance) of the recommendatons for each category, f the recommendatons were to be mplemented. The cost avoded by mplementng the securty solutons s the amount n dollars reduced from the total possble loss by the effectveness of the securty solutons. The effectveness of a securty soluton s essentally the amount of rsk reducton a Category of Preventons can acheve. Annualzed Loss (AL) = Surveyed_Average f no data avalable Avg_Incde nt_loss Est_Frequency f data or estmaton avalable Baselne Cost Resdual Cost = Baselne_R sk AL = Resdual_R sk AL = Baselne_R sk Bypass_Rate Tangble Beneft = 0 f X = 0 Baselne_Cost Resdual_Cost f X = 1 Intangble Beneft = 0 f X = 0 Custom_Ben eft f X = 1 AL 8 CMU/SEI-2004-TN-045

19 Total Benefts = = N = 1 N = 1 Tangble_B eneft + N = 1 (AL Baselne_Rsk X Intangble_Beneft (1 Bypass_Rate )) + N = 1 (X Custom_Beneft ) Example 2: If the company loses $50,000 for each msuse ncdent n Category A and there are 10 ncdents per year n Category A, and the company loses $100,000 for each msuse ncdent n Category B and there are 2 ncdents per year n Category B: Annualzed Loss (A) = $50,000 x 10 = $500,000 Annualzed Loss (B) = $100,000 x 2 = $200,000 Usng fgures from Example 1: Baselne Cost (A) = $500,000 x 60% = $300,000 Resdual Cost (A) = $500,000 x 3% = $15,000 Baselne Cost (B) = $200,000 x 30% = $60,000 Resdual Cost (B) = $200,000 x 21% = $42,000 Tangble Beneft (for mtgatng A) = $300,000 - $15,000 = $285,000 Tangble Beneft (for mtgatng B) = $60,000 - $42,000 = $18,000 Suppose that the company can get a $50,000 government award for havng effectvely guarded aganst msuses n Category A, then: Intangble Beneft (for mtgatng A) = $50,000 Intangble Beneft (for mtgatng B) = $0 Total Benefts = ($285,000 + $18,000 ) + ($50,000 + $0) = $353,000 Wth stakeholders feedback, msuse cases n each Category of Threats can be dentfed as hgh, medum, or low n prorty. We found that small companes typcally wll only have the resources to mtgate hgh-prorty msuse cases. Gven such constrants, t s mportant to note that msuses and attacks wth low to medum rsk can stll occur. Therefore, the Bypass Rate shall not be too low when medum- and low-prorty rsks have not been mtgated. The recommendatons that correspond to hgh-prorty msuse cases are used n the calculatons of the Cost/Beneft Analyss Framework. Cost Avodance s used as the beneft for each Category of Preventons. If there are any other ntangble benefts, they should be ncluded as well. Costs of mplementaton for each recommendaton need to be estmated, checked wth stakeholders, and then adjusted based on ther feedback. Total System Value, Total CMU/SEI-2004-TN-045 9

20 Implementaton Costs, Net Project Value, and Beneft/Cost Rato (B/C) are then calculated. For more detals, see Secton 2.4, Evaluaton Crtera. 2.3 Stakeholder Involvement Stakeholders must be regularly nvolved n ths Cost/Beneft Analyss Framework to ensure reasonably accurate results, especally durng the msuse case dentfcaton phase and the cost estmaton phase for mplementng recommendatons. After the stakeholders reply wth ther feedback and suggestons for change, cost/beneft calculatons should be updated and mproved on n a reteratve process over a span of several weeks. Small companes probably do not have months of tme to analyze a project. Therefore, we recommend that the Cost/Beneft Analyss be done wth an exstng set of templates nstead of renventng the wheel. Also, t s mportant to keep n mnd that the proposed system and alternatves may change, dependng on a company s nternal assessment of ts assets, vulnerabltes, development tmeframes, and rsks and ther assocated costs, among other varables. 2.4 Evaluaton Crtera The crtera for evaluatng alternatves are based on fve key metrcs: Total Implementaton Costs, Net Project Value, Total System Value, Beneft/Cost Rato, and Rsk Exposures. These fve crtera serve dfferent purposes. Total Implementaton Costs can help small companes make decsons as to how much money they can spend wthout jeopardzng growth n other areas of need. Net Project Value demonstrates the extent to whch a partcular securty soluton can contrbute to the overall system. Total System Value takes nto consderaton the fact that unmtgated threats stll cost a company some amount of money n rsks. It accounts for scenaros where the Net Project Value s hgh whle the overall value of the system s low because the soluton dd not address costly threats. A postve Net Project Value s a strong key ndcator that the soluton s worthwhle to mplement; a large Total System Value suggests that the system wll be mproved by mplementng the project; and a large B/C Rato relatve to other solutons ndcates that the soluton should be mplemented frst because t s more cost effectve. Combned wth Rsk Exposures after mplementng the proposed system versus mplementng alternatves, these fve crtera form the bass of correlaton between benefts of desred securty mprovement, costs wthn avalable fscal budget, and tolerance of acceptable Rsk Exposures. The proposed system and the alternatves wll be chosen from a fnte set of possble solutons that small companes may wsh to mplement or gnore, based on comparng and analyzng present values of these metrcs. Let s assume that Categores of Preventons have the followng characterstcs, whch we can calculate by dong a cost/beneft spreadsheet on each category. All values are NPV. 10 CMU/SEI-2004-TN-045

21 Category of Preventons ( P ) Baselne Cost A ) f X = 0 ( Resdual Cost R ) f X = 1 ( Implementaton Cost C ) f X = 1 ( 1 A 1 R 1 C 1 2 A 2 R 2 C 2 N A N R N C N Total Implementaton Costs Total Implementaton Costs are the present value costs calculated over the length of the project. Because there mght be overlappng n costs of mplementng archtectural and polcy recommendatons when some recommendatons (e.g., good password management) may be necessary to mtgate multple Categores of Threats, total mplementaton costs are the sum of all present value costs of mplementaton mnus any overlappng costs. Total Implementaton Costs N = = 1 j= 1 k= j+ 1 N N ( C X ) Overlap _ Cost X X jk j k Net Project Value Net Project Value s the present value of savngs (loss) from the total benefts of mplementng recommendatons mnus total costs of mplementng recommendatons. It demonstrates the value that the project can delver to the overall system. The hgher the Net Project Value s, the better. Net Project Value (NV) = Total Benefts Total Implementaton Costs Total System Value Total System Value s the present value of Net Project Value mnus the present value of expected loss from unmtgated threats. It takes nto consderaton that unmtgated threats stll cost companes some amount of money n rsks. If a Category of Threats s mtgated, then ts Resdual Cost s used; otherwse ts Baselne Cost s used. Total System Value accounts for scenaros where the Net Project Value s hgh whle the overall value of the system s low because the soluton dd not address costly threats. It evaluates the system s overall value after mplementng the project and provdes hgh-level gudance to the busness objectve beyond the project tself. The hgher the Total System Value s, the better. Total System Value (TV) = Net Project Value - costs of unmtgated rsks N N = Net Project Value ( X R + ( 1 X ) A ) Theoretcally the hgher TV s, the better; but t needs to be taken nto consderaton wth Rsk Exposures and other company-specfc factors. Because X s ether 0 or 1 (2 choces) and = 1 = 1 CMU/SEI-2004-TN

22 there are N categores, there are 2 N possble solutons. For small N ths can be easly calculated va a computer program (e.g., Mcrosoft Excel), whch s the case n the Acme Company example. In fact, ths s where categorzaton helps out small companes n terms of estmaton efforts because t reduces the sze of N. Beneft/Cost Rato (B/C) Beneft/Cost Rato pertans to the rato between the net beneft n mplementng a securty soluton and the costs of mplementaton. It demonstrates the capablty for the organzaton to proft (cost savngs) from ts securty nvestments. The hgher the B/C Rato, the better an nvestment s. BC = Total_Benefts Total_Implementaton_Costs Example 3: If t costs $200,000 to mplement solutons for A and $150,000 to mplement solutons for B, wth $40,000 of overlappng hardware costs, then: Total Benefts = $335,000 (from Example 2) Resdual Costs (A) = $15,000 (from Example 1) Resdual Costs (B) = $42,000 (from Example 1) Total Implementaton Costs = $200,000 + $150,000 - $40,000 = $310,000 Net Project Value = $335,000 - $310,000 = $25,000 Total System Value = $25,000 ($15,000 + $42,000) = -$32,000 Beneft/Cost Rato = $335,000/$310,000 = 108% 2.5 Maxmzng System Value Wthn Real-Lfe Budget Constrants We have untl now presented a framework that analyzes the proposed system versus the alternatves assumng that there are no lmts and no varatons to yearly budgets. For the sake of convenence, we assumed that the budget s gong to be so large that these varables could be gnored. However, we know from real-lfe experence that ths s often not the case, especally n small companes where captal s at a premum. Companes wth lttle ntal budgets and large future budgets wll make ther decsons sgnfcantly dfferent from companes that have large ntal budgets but lttle future budgets. In such cases, to deal wth real-lfe budget constrants, we must fnd a lnear soluton n whch all constrants are lnear functons of the decson varables. Some or all of the decson varables must have nteger values (0 or 1, do or not do). In mathematcs terms, the model to solve these knds of problems s called Integer Model [Camm 00]. We are able to do so because decsons to mplement Categores of Preventons are essentally concrete (yes/no), and the optons come from a fnte set of Categores of Preventons that are suffcently dstnct from each other. 12 CMU/SEI-2004-TN-045

23 Back to the problem, there are N possble Categores of Preventons to mplement, each of whch runs for M number of years, wth the followng Total System Values and yearly mplementaton costs. Category of Preventons ( P ) 1 2 N Implementaton Cost ( C t ) f X = 1 Y 0 Y 1 C C C Y M C 1 M C C 2 M C N 0 C N1 C NM And we have avalable budget for each year: B 0, B 1, budget. B M, wth B 0 beng the ntal Now we have to decde whch set of Categores of Preventons to mplement n order to maxmze our returns wthn the budget constrants. We know that yearly costs for mplementatons must also be wthn the yearly budget. The constrants for the Lnear Model problem are then the lnear sum of mplementaton costs for each category. If a Category of Preventons was mplemented, then t would contrbute ts cost aganst the budget; otherwse t wll count as 0. N =1 C t X B t or (t=0, 1, 2 M) C 10 X 1 + C20 X C N 0 X N B0 11 X 1 + C21 X CN1 X N B1 (year 0) C (year 1). C1 X 1 + C2 X C X B (year M) M M NM N There could be Z number of solutons to ths set of equatons, where Z s less than or equal to 2 N. We can exhaustvely apply every set of possble ( X 1, X 2 M X N ) values to calculate fnancally feasble solutons under the budget constrant. In fact, n most cases we expect the exhaustve method to be used because t s easy to understand and easy to calculate when N s not too large. However, should there be a stuaton where N s very large, the Branch and Bound method may be used. Brand and Bound s an algorthmc technque to fnd the optmal soluton by keepng the best soluton found so far [NIST 04]. In the Brand and Bound method, f a partal soluton cannot mprove on the best value, t s abandoned. The method systematcally enumerates a fracton of feasble solutons, whle stll guaranteeng CMU/SEI-2004-TN

24 that the most optmal nteger soluton s found. Several commercally avalable software packages support the Brand and Bound method, ncludng Mcrosoft Excel. In the end, we should get a set of TV values and a set of ( X 1, X 2 X N ), from whch we derve the proposed system and possble alternatves. When we analyze them wth ther assocated Beneft/Cost Rato and Rsk Exposures, we can fnd the best paths to take for nformaton securty mprovement projects n small companes. 14 CMU/SEI-2004-TN-045

25 3 Cost/Beneft Analyss Framework n Practce The Cost/Beneft Analyss Framework s appled on the Acme Company to help t determne how to meet ts securty and busness objectves at the same tme wthn reasonable costs. 3.1 Msuse Cases Before the cost/beneft analyss can be done, msuse cases must be dentfed n order to accurately access the mpact of msuses when they happen. Ths report wll not go nto detal about how these msuse cases are generated. The msuse case documentaton shown n Table 2 s provded as an example of the level of detal msuse cases need n order to derve comprehensve archtectural recommendatons and polcy recommendatons. Attack trees for msuse cases may also be used to ensure that the lst of archtectural and polcy recommendatons s complete. Table 2: Example Msuse Case Number: MC-xx Name: Users gan sys admn rghts on the server (elevaton of prvleges). Scope: User Authorzaton Concerns Prorty: Low Medum x Hgh Deployment Envronment: Ms-actors: Access Rght Levels: x Intranet Extranet/Internet Users Low-Level System User x Medum-Level System User x Hgh-Level System User Sys-Admn-Level System User x Other Network User Pont of Entry: Network x Host Applcaton Securty Attrbutes affected: Descrpton: Sophstcaton: Pre-condtons: Assumptons: Post-condtons: x Confdentalty x Integrty Avalablty A user attempts to gan sys admn rghts on the server and succeeds. Low Medum x Hgh The user has unntended logon rghts to the Wndows 2003 server. The user s not already a sys admn. The user does not have expressed permsson to gan sys admn rghts. Worst Case Threat: Wanted Preventon Guarantee: The user gans sys admn rghts on the server and then tampers wth system and/or user data. Hs/her actons are never caught. Enforce machne access control lst (ACL) securty polcy (role-based user authentcaton). CMU/SEI-2004-TN

26 Potental Msactor Profles: Stakeholders and Threats: Related Use Cases: Related Threats: Archtectural Recommendaton: Polcy Recommendaton: Wanted Detecton Guarantee: Wanted Recovery Guarantee: Hghly sklled users wth hgh crmnal ntent. Logon attempts are logged and vewed by system admnstrators. Remove users unauthorzed logon rghts on the server. Acme Company s clent: loss of data ntegrty and/or confdentalty Acme Company: loss of reputaton, loss of current and potental clents UC-06, UC-07, UC-08 Elevaton of prvleges, unauthorzed access to admnstraton nterfaces, unauthorzed access to confguraton stores Store audt nformaton n a separate locaton from the servers and the workstatons. Implement a strong role-based authentcaton control. Patch applcatons and operatng systems routnely (bmonthly). Ensure that users do not have rghts or access levels beyond those prescrbed by ther job responsbltes. Revew audt nformaton routnely (monthly). Store and cross-revew confguraton changes (monthly). Enforce strong password polces. Password protect any necessary shared documents. Requre users to change ther passwords perodcally (monthly). Perodcally revew user actvtes (bmonthly). Requre users to log out of the system or close ther browser as soon as ther actvtes are done. Requre users never to reveal ther account names and passwords. Perform routne system and data backup (weekly). 3.2 Categores of Threats The Cost/Beneft Analyss Framework categorzes all msuse cases nto seven Categores of Threats: Denal of Servce System Penetraton Sabotage of Data Theft of Propretary Info Unauthorzed Access by Insders Vrus Actve Wretappng [Rchardson 03] Fnancal and probablstc data are avalable for these categores from the 2003 CSI/FBI Computer Crme and Securty Survey [Rchardson 03]. Gven that the Acme Company had not pad much attenton to ts own securty efforts up to ths pont n tme, we assumed that Acme wll have Baselne Rsks at or near natonal average wthn each Category of Threats. The lower end of reported losses s ntally used as the estmate of Annualzed Loss f attacks were successful n achevng observable damages. Later the Acme Company performs an nternal estmaton and determnes a more precse set of fnancal numbers that get ncorporated n the analyss nstead. 16 CMU/SEI-2004-TN-045

27 3.3 Rsk Exposures The cost/beneft analyss assumes that the Bypass Rate s approxmately 10%. Subsequent calculatons show that when none of the seven Categores of Threats are mtgated, the Acme Company s core product s Rsk Exposures to some combnaton of msuses are above 90%. Because the probablty of msuses/attacks s very hgh, the cost/beneft analyss s needed n order to manage and mtgate the company s Rsk Exposures for ts core product. 3.4 Archtectural Recommendatons From our work, we have dscovered that archtectural recommendatons tend to have costs that are heavly front-loaded (e.g., ntal mplementaton costs). It ntutvely makes sense because archtectural mprovements need to be mplemented, tested, and deployed before benefts can be realzed over the lfetme of a project. The Acme Company prefers to vew the costs of mplementaton n terms of man-hours of effort. We have no objectons to ths method of evaluaton. In fact, we would recommend the man-hour estmaton method to our future clents because t s a standard way of makng engneerng estmates. We can then multply man-hours wth average hourly wage rates to arrve at a good estmaton of total costs. Other than costs of salares, there are mantenance costs (also calculated va manhours), thrd-party software costs, and hardware costs. Table 3 shows the format we used to break down the types of costs that archtectural recommendatons have. Table 3: Cost Estmates for Archtectural Recommendatons Archtectural No. Recommendaton AR-01 All shared drves on the network should enforce authentcaton polces. AR-02 Antvrus software s nstalled on the server. Related Msuse Cases Mant. Cost ($/ year) Software Cost [Type]/($) Hardware Cost [Type]/ ($) Category of Implementaton Prorty Threat Cost ($/ year) MC-01 Hgh U $xxx $xxx $xxx $xxx MC-17 Hgh V $xxx $xxx $xxx $xxx 3.5 Polcy Recommendatons Polcy recommendatons tend to recur over the lfetme of the project. The cost of tranng and the cost of enforcement are dffcult to quantfy on the macro level. However, the feedback we ganed from the Acme Company s that t s much easer to vsualze the efforts n terms of man-hours per user per year. The total costs can then be calculated by multplyng man-hours per user per year wth an estmated number of users and wth average hourly wage rates. Table 4 shows the format we used to break down the types of costs that polcy recommendatons have. CMU/SEI-2004-TN

28 Table 4: Cost Estmates for Polcy Recommendatons No. Polcy Recommendaton PR-01 All nstallaton must be approved and revewed by managers. PR-02 Applcatons and operatng systems must be patched routnely (bmonthly). Related Msuse Cases MC-13, MC-15 MC-01, MC-03, MC-13, MC-15, MC-16, MC-17, MC-18, MC-19, MC-20, MC-21, MC-22 Prorty Category of Threat Tranng Cost ($) Enforcement Cost ($) Other Costs [Type]/ ($) Hgh U, W $xxx $xxx Name/$xxx Hgh U, P $xxx $xxx Name/$xxx 3.6 Total System Value Versus Total Implementaton Costs The Total System Value vs. Total Implementaton Costs graph n Fgure 2 shows us there are optmal and non-optmal solutons among the securty solutons that the Acme Company may choose to mplement. The solutons wth hgher Total System Value are better solutons. The four colored boxes (solutons) are better solutons wthn ther respectve cost ranges because they have the hghest Total System Value compared to other solutons on the same vertcal lnes n the graph. The pnk soluton represents the Total System Value of the current system. It has zero total mplementaton costs. The Blue soluton (Alternatve 2) represents the total value of the system when every archtectural and polcy recommendaton has been mplemented. The brown soluton (Alternatve 1) and the red soluton (Proposed System) have the hghest Total System Value, meanng that by mplementng ether one the Acme Company can obtan the best value for ts system over the next three years of project lfetme. From a strctly fnancal perspectve, solutons wth hgher Total System Value and lower Total Implementaton Costs are preferred. Therefore, the graph suggests that Alternatve 1 s a better soluton than the Proposed System or Alternatve 2. However, t s not mmedately apparent from ths vew the extent to whch Rsk Exposures are reduced. We shall examne Rsk Exposures n later sectons. It s worth notng, however, that Alternatve 1 s a subset of the Proposed System. 18 CMU/SEI-2004-TN-045

29 Total System Value vs. Total Implementaton Costs $0 A1 PS $0 $50,000 $100,000 $150,000 $200,000 $250,000 ($50,000) ($100,000) CS ($150,000) A2 Total Value of System Poly. (Total Value of System) ($200,000) Legend Proposed System ($250,000) Alternatve 1 Alternatve 2 ($300,000) Current System ($350,000) Total Implementaton Costs Fgure 2: Total System Value vs. Total Implementaton Costs 3.7 Beneft/Cost Rato Versus Total Implementaton Costs How effectve are the solutons n delverng results? Beneft/Cost Rato gves us the trend pattern when compared aganst the Total Implementaton Costs. From the graph, B/C Rato brefly ncreases before droppng as costs of mplementng securty recommendatons go up. Small companes often gan sgnfcant benefts by mplementng a small set of selected securty mprovement recommendatons but then lose the benefts when they start to mplement addtonal securty solutons. The Beneft/Cost Rato vs. Total Implementaton Costs graph n Fgure 3 suggests that there are hghly cost-effectve securty solutons that should be mplemented frst. The Current System s not present on ths graph because there s no mplementaton cost nvolved wth takng no acton. Smlar to the Total System Value vs. Total Implementaton Costs graph, the three solutons wth hgher Total System Value are more cost effectve when compared aganst other securty solutons that have the same mplementaton cost. Alternatve 1 appears to be more cost effectve than the red or blue soluton. However, when compared to the prevous graph, we note that Alternatve 1 and the Recommendaton have the same total value. Ths suggests that the addtonal nvestment wth the Recommendaton mtgates the cost of addtonal rsk at or near 100% Beneft/Cost Rato, whch s the case when we see that the Recommendaton has approxmately 100% Beneft/Cost Rato. The trend lne also suggests that f the Acme Company nvests more resources to become more secure, ts return on the nvestment wll declne precptously. Wthout ntangble benefts such as new CMU/SEI-2004-TN

30 revenue opportuntes, large nvestments assocated wth makng many securty mprovements are probably dffcult to justfy beyond an acceptable level of rsk tolerance. Beneft/Cost Rato vs. Total Implementaton Costs % A % PS 80.00% A2 BC Rato Poly. (BC Rato) 60.00% Legend 40.00% Proposed System Alternatve % Alternatve 2 Current System 0.00% $0 $50,000 $100,000 $150,000 $200,000 $250,000 $300,000 Total Implementaton Costs Fgure 3: Beneft/Cost Rato vs. Total Implementaton Costs 3.8 Total Implementaton Costs Versus Rsk Exposures The Total Implementaton Costs vs. Rsk Exposures graph n Fgure 4 shows us that ntally securty mprovements can be costly. Securty mprovements may be best when done together wth mplementng multple Categores of Preventons. The solutons that mtgate more rsks wth lower costs are better solutons. There are several thngs to be notced wth the graph. Frst, costs go up when Rsk Exposures go down, whch s to be expected. Second, the smallest Rsk Exposure s not near zero. Ths s due to the fact that a small company such as the Acme Company may not have the resources to mplement and enforce every sngle recommendaton. Therefore, ts Bypass Rates and Resdual Rsks for securty breaches are stll hgh, whch causes ts Rsk Exposures to be hgh. More detaled studes are warranted f the Acme Company needs to reduce ts rsk exposures further. However, from the trend projecton, we can see that the cost goes up sgnfcantly as Rsk Exposures become smaller and smaller. It s an ndcaton that the costs needed to cover edge scenaros may be very expensve and may only be justfed wth large ncreases n the benefts (such as new revenue opportuntes) that addtonal securty mprovements would brng. 20 CMU/SEI-2004-TN-045

31 The varance around the trend lne s extremely hgh when rsks are not mtgated. Ths suggests a few possble scenaros. Frst, there are Categores of Threats wth low rates of return and hgh costs to fx. Therefore, they should only be mplemented after other categores wth hgher Beneft/Cost Rato. Second, strateges that focus on mtgatng only very small number of Categores of Threats may be nether cost effectve nor rsk averse. $600,000 Total Implementaton Costs vs. Rsk Exposures $500,000 $400,000 Total Implementaton Costs Log. (Total Implementaton Costs) $300,000 Legend Proposed System $200,000 A2 Alternatve 1 Alternatve 2 $100,000 PS A1 Current System $0 CS 0.00% 20.00% 40.00% 60.00% 80.00% % % Rsk Exposures Fgure 4: Total Implementaton Costs vs. Rsk Exposures 3.9 Values Versus Rsk Exposures The Values vs. Rsk Exposures graph n Fgure 5 shows us what happens when Rsk Exposures are taken nto consderaton. The graph shows the relatonshps between Net Project Value, Total System Value, and Rsk Exposures. The gap between Total System Value and Net Project Value represents the amount of costs n unmtgated rsks the Acme Company s subject to wth respect to each possble soluton. Because costs of unmtgated rsks are the product of multplyng probabltes of occurrence by Annualzed Loss (when msuses happen), they are essentally approxmatons of costs of uncertanty. The hgher Rsk Exposures are, the hgher the uncertanty and volatlty s. As Rsk Exposures decrease, gaps become smaller and Total System Value becomes more predctable. Therefore, the Proposed System s a much more rsk-averse soluton that delvers the same results when compared to Alternatve 1. So t s a better soluton, wth same Total System Value, less volatlty, hgher predctablty, and smaller Rsk Exposures. Furthermore, the dotted lne of Total System Value w/o Resdual Costs s shown to demonstrate the extent to whch Resdual Rsks can have an affect on the Total System Value. CMU/SEI-2004-TN

32 When securty solutons are hghly effectve (.e., Bypass Rate s small), the gap between the two Total System Values wll be small; otherwse the gap wll be large. The gap between the two Total System Values represents the costs of the Resdual Rsks that the project s avalable securty solutons cannot mtgate. In order to reduce the Resdual Costs, the Acme Company needs to consder mplementng medum- and low-prorty recommendatons. Values vs. Rsk Exposures $40,000 $20,000 A1 PS CS $0 0.00% 20.00% 40.00% 60.00% 80.00% % % ($20,000) PS A1 "Total System Value" ($40,000) ($60,000) ($80,000) ($100,000) ($120,000) ($140,000) ($160,000) A2 CS Legend "Net Present Value of Project" Total System Value w/o Resdual Costs Proposed System Alternatve 1 Alternatve 2 Current System ($180,000) Rsk Exposures Fgure 5: Values vs. Rsk Exposures 22 CMU/SEI-2004-TN-045

33 4 Lessons Learned 4.1 Msuse Cases The Cost/Beneft Analyss Framework s bult on msuse cases. Even though the generaton and valdaton of msuse cases are not dscussed n ths report, the comprehensveness of msuse cases wll drectly mpact the accuracy of the results n the cost/beneft analyss. We have dscovered that a cost/beneft analyss contrbutes to more clarfcaton and better understandng of the project s msuse cases. The average probabltes of occurrence and expected loss gve nsghts nto the prortzaton of msuse cases when costs of rsks are ranked. In addton, t provdes quantfable mappng from descrptons to mplementaton choces for archtectural and polcy recommendatons. Understandng man-hour and captal expendture requrements helps stakeholders plan the project wth respect to ther stuatons. 4.2 Estmaton of Losses The framework ntally used estmated cost fgures from the lower end of natonally surveyed losses for each Category of Threats. Later on, we worked wth the Acme Company to come up wth a set of loss fgures for each msuse case per ncdent. We multpled estmated frequences (per year) by estmated ncdent losses (for all msuse cases n a category) to derve the Annualzed Loss for each Category of Threats. Through ths process we found that Lower ends of natonally surveyed losses may be used as estmatons for tangble losses (productvty loss, fxng cost, etc.). Surveyed losses cannot suffcently account for ntangble losses (loss of reputaton, loss of confdental data, etc.), snce these values are hghly company and project specfc. Intangble losses often exceed tangble losses for many Categores of Threats. For small companes, loss of reputaton may be a very mportant tem of nterest, and t can contrbute sgnfcantly to ntangble losses. Therefore, for better accuracy, we hghly recommend that losses are estmated for each msuse case. 4.3 Estmaton of Costs Our experence s that the Acme Company strongly prefers the use of man-hours to estmate costs of mplementaton. Its senor techncal and project leads make effort estmatons n man-hours. The company provdes average cost fgures for employees n dfferent roles. CMU/SEI-2004-TN

34 Costs are then calculated on the number of man-hours multpled by average hourly wage rates. We found ths process of cost estmaton to be very effectve. We wll strongly recommend ths process n our future work. 4.4 Cost Structures of Securty Improvement Projects We found that the costs of ensurng polcy complance heavly domnate n the costs of mplementaton for vrtually every Category of Threats. Ths suggests that securty mprovement projects are very human-effort ntensve n ther cost structures. The costs wll be spread over the lfetme of these projects. Such costs are often seen as hdden costs that many companes tradtonally have dffcultes n quantfyng. The framework can provde sgnfcant nsght nto the hdden costs of polcy complance by examnng and then summng up the efforts for every recommendaton. However, because costs are accumulated over multple years, companes that take on securty mprovement projects need to look at ther nvestments from a long-term perspectve. The cost structures of securty mprovement projects wll be determned prmarly by the companes wllngness to nvest n ther employees on securty awareness and polcy enforcement. 4.5 Values of Securty Improvement Projects The Acme Company s most optmal Total System Value s stll negatve. There are two possble explanatons for ths phenomenon. Frst, Resdual Rsks stll cost companes a certan amount. Real-lfe experences have shown us that no securty soluton s 100% secure. Therefore, even the best effort of securty mprovement may not reduce rsks to zero. Second, securty mprovement may need to be vewed from a lose-less perspectve rather than the proft-more perspectve that typcal IT projects are judged on. Lose-less s another way of proftng by mnmzng the rsks of havng msuses and attacks. 24 CMU/SEI-2004-TN-045

35 5 Conclusons The objectve of the Cost/Beneft Analyss Framework s to provde a quantfable fnancal analyss framework that small companes can apply on ther securty mprovement projects. Wthn ths scope, we show that unmtgated rsks can be translated nto costs, and we demonstrate the estmaton methods for calculatng costs of mplementaton for archtectural and polcy recommendatons. Most mportantly, we show through the example of the Acme Company that small companes can obtan optmal results for mprovng the securty of ther systems and the optmal results can be acheved wth reasonable reductons n Rsk Exposures. The reductons n Rsk Exposures n turn enable small companes to have less volatlty n ther Total System Value. The ncrease n predctablty of results by mplementng optmal securty solutons wll enable small companes to proft from securty mprovements and to plan for future growth. CMU/SEI-2004-TN

36 6 Future Work There are several questons that drve future work on the Cost/Beneft Analyss Framework: Can the Acme Company s trend patterns be wtnessed n other small companes and ther securty mprovement projects? How would the estmate values compare to emprcal data f we were to follow through wth the Acme Company over the lfetme of ts project? Are there any other varables that we have not accounted for n the framework? If so, why do they exst and how can we account for them? What f Categores of Threats cannot be assumed to be ndependent from each other? So far we have assumed that the effects of mtgatng threats n one category are neglgble to the rsks n other categores. If ths assumpton no longer holds, how do the resultng nterdependences affect the framework? Can the framework be appled to larger companes? The overall goal of the framework s to provde a way for small companes to be able to accurately estmate the cost of ther securty mprovement projects. By ncorporatng lessons learned from the Acme Company, we wll strve n the future to refne the estmaton methods to facltate further analyss use the estmaton methods wth other companes to see f smlar trend patterns exst develop a general set of cost/beneft profles and metrcs for projects wth dfferent types of system archtectures desgn a spreadsheet to automate the cost/beneft calculatons and to select the most optmal soluton formalze the relatonshp between the fve metrcs we proposed n Secton CMU/SEI-2004-TN-045

37 References URLs are vald as of the publcaton date of ths document. [Allen 99] [Camm 00] [NIST 04] [Rchardson 03] [SANS 03] Allen, J.; Chrste, A.; Fthen, W.; McHugh, J.; Pckel, J.; & Stoner, E. State of the Practce of Intruson Detecton Technologes (CMU/SEI-99-TR-028, ADA375846). Pttsburgh, PA: Software Engneerng Insttute, Carnege Mellon Unversty, tr028app-a.html. Camm, Jeffrey D. & Evans, James R. Management Scence & Decson Technology. South-Western College Publshng, Natonal Insttute of Standards and Technology. branch and bound. (2004). Rchardson, Robert CSI/FBI Computer Crme and Securty Survey. Computer Securty Insttute. (2003). SANS Insttute. SANS Glossary of Terms Used n Securty and Intruson Detecton. (2003). CMU/SEI-2004-TN

38 28 CMU/SEI-2004-TN-045

39 REPORT DOCUMENTATION PAGE Form Approved OMB No Publc reportng burden for ths collecton of nformaton s estmated to average 1 hour per response, ncludng the tme for revewng nstructons, searchng exstng data sources, gatherng and mantanng the data needed, and completng and revewng the collecton of nformaton. Send comments regardng ths burden estmate or any other aspect of ths collecton of nformaton, ncludng suggestons for reducng ths burden, to Washngton Headquarters Servces, Drectorate for nformaton Operatons and Reports, 1215 Jefferson Davs Hghway, Sute 1204, Arlngton, VA , and to the Offce of Management and Budget, Paperwork Reducton Project ( ), Washngton, DC AGENCY USE ONLY (Leave Blank) 2. REPORT DATE November REPORT TYPE AND DATES COVERED Fnal 4. TITLE AND SUBTITLE 5. FUNDING NUMBERS SQUARE Project: Cost/Beneft Analyss Framework for Informaton F C-0003 Securty Improvement Projects n Small Companes 6. AUTHOR(S) Nck (Nng) Xe 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Software Engneerng Insttute Carnege Mellon Unversty Pttsburgh, PA SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) HQ ESC/XPK 5 Egln Street Hanscom AFB, MA SUPPLEMENTARY NOTES 8. PERFORMING ORGANIZATION REPORT NUMBER CMU/SEI-2004-TN SPONSORING/MONITORING AGENCY REPORT NUMBER 12A DISTRIBUTION/AVAILABILITY STATEMENT Unclassfed/Unlmted, DTIC, NTIS 13. ABSTRACT (MAXIMUM 200 WORDS) 12B DISTRIBUTION CODE Many companes rely on hstorcal data to buld predctablty models for cost/beneft justfcaton of future projects. Unfortunately, for small companes, whch generally do not have a process for collectng securty data, the costs and the benefts of nformaton securty mprovement projects have been very dffcult to estmate and justfy. In addton, detaled attack data are smply not avalable to be used as references n cost estmatons. Gven these dffcultes, many small companes choose to gnore entrely the securty vulnerabltes n ther systems, and many suffer the consequences of securty breaches and sgnfcant fnancal loss. Small companes that do mplement securty mprovement projects often have problems understandng the cost structures of ther mprovement ntatves and how to translate rsk exposures nto costs that can be passed on to ther customers. To deal wth the aforementoned problems, ths paper descrbes a general framework for herarchcal cost/beneft analyss amed at provdng acceptable estmatons for small companes n ther nformaton securty mprovement projects. The framework classfes msuse cases nto categores of threats for whch natonally surveyed rsks and fnancal data are publcly avalable. For each category of threats, costs, benefts, baselne rsks, and resdual rsks are estmated. The framework then generates all permutatons of possble solutons and analyzes the most optmal approach to maxmze the value of securty mprovement projects. The framework analyzes the problems from fve dmensons: Total Implementaton Costs, Total System Value, Net Project Value, Beneft/Cost Rato, and Rsk Exposures. The fnal proposed system wll be derved from the comparsons of these dmensons, takng nto consderaton each company s specfc stuaton. Ths report s one of a seres of reports resultng from research conducted by the System Qualty Requrements Engneerng (SQUARE) Team as part of an ndependent research and development project of the Software Engneerng Insttute.

40 14. SUBJECT TERMS cost/beneft analyss, nformaton securty mprovement, nformaton securty costs, msuse cases 15. NUMBER OF PAGES PRICE CODE 17. SECURITY CLASSIFICATION OF REPORT Unclassfed 18. SECURITY CLASSIFICATION OF THIS PAGE Unclassfed 19. SECURITY CLASSIFICATION OF ABSTRACT Unclassfed 20. LIMITATION OF ABSTRACT NSN Standard Form 298 (Rev. 2-89) Prescrbed by ANSI Std. Z UL