Guidance for Best Practices in Information Security and IT Audit

Transcription

1 September 2009 Guidance for Best Practices in Information Security and IT Audit IT Policy Compliance Group

2 Contents Executive Summary Practices Covered 2 Key Findings 2 Only One-in-Ten Experience the Best Outcomes 2 Some Practice Guidance Drives Better Outcomes 2 Baseline Practices for Better Outcomes 3 Top 10 Percent: Top 10 Practices 3 Outcome-based Practices 4 Practice Improvements Pay Off 4 Guidance for Best Practices 4 Organization of the Report 5 Benchmark Findings Distribution of Outcomes 6 Practices and Better Outcomes in IT 6 CobiT, ISO and Practices for Better Outcomes 7 Which Practices Work 8 Differences in Implementation of Practices by Domains 9 Significant Gaps for Individual Practices 9 Baseline Practices for Better Outcomes 11 Practice Improvements Pay Off 12 Top Line Business Results 12 Audit Expenses 13 Time and Labor Costs for Audit in IT ` 14 Business Downtime 15 Financial Exposure and Loss: Data Loss and Theft 16 Taking Action to Improve Outcomes 17 Appendix A: Guidance for Practices 18 Appendix B: Practices Tested 23 About the Research 24 About IT Policy Compliance Group 25 3

3 Executive Summary Practices Covered This report covers benchmarked practices within information security and IT audit functions across more than 3,000 organizations that are directly related to better operational and financial outcomes, for managing: The integrity of information Compliance with regulatory audit Business risks related to the use of IT In addition, guidance for practices related to information security practices and procedures, and information security policies are also covered in this report. Extent of practices covered Communications about management aims and directions Managing IT processes, organization, relationships Managing the Information architecture Managing human resources Acquiring and managing IT assets Managing information and data Managing operations Ensuring systems security Monitoring and evaluating Managing quality Managing risks Governance More than 100 benchmarked practices for information security and IT audit are grouped into 12 practice domains, including: managing communications about directions and aims; managing IT processes, the organization and relationships; managing the information architecture; managing human resources; managing information and data; managing operations; ensuring systems security; monitoring and evaluating; managing quality; managing risks and governance. Key Findings Only One-in-Ten Experience the Best Outcomes Operating outcomes related to IT About 1 in 10 organizations (12 percent) experience the best outcomes for information security and IT audit with the lowest levels of data loss or theft, least business disruption and fewest problems with audit A majority of organizations, nearly 7 in 10 (69 percent) are experiencing higher rates of data loss or theft, higher levels of business disruptions from IT failures, and more difficulty with passing regulatory audits in IT Almost 2 in 10 organizations (19 percent) experience the worst outcomes with the highest rates of data loss or theft, the highest levels of business downtime, and the most difficulties passing audits in IT. Financial outcomes among best performing organizations Organizations with the best operational outcomes are also experiencing better financial outcomes, including: Fifty percent less money being spent on regulatory audit each year Fifty percent less time spent supporting audit in IT Financial exposure and loss from customer data loss or theft is the lowest Customer retention rates are typically six percent higher Annual revenue is uniformly eight percent higher Profits are six percent higher Some Practice Guidance Drives Better Outcomes The best performing organizations uniquely rely on primary and secondary forms of guidance to inform and adjust information security and IT audit practices for better results, including: Primary practice guidance: CobiT and COSO The use and implementation of CobiT and COSO guidance for information security and IT audit practices is between 20 and 30 times higher among organizations with the best outcomes. 2

4 Secondary practice guidance: ISO, PCI, NIST and regulatory audit mandates Supplementary practice guidance from ISO, PCI, NIST and regulatory mandates to guide information security and audit practices is five to ten times higher, even among organizations not subject to PCI and FISMA audits. Baseline Practices for Better Outcomes Baseline Best Practices Twelve key-practices, one in each practice domain, are uniformly implemented for information security and IT audit by organizations with the best outcomes. These twelve are joined by the top 10 practices making the most difference in outcomes. Top 10 Percent: Top 10 Practices Of 104 of the most common practices for information security and IT audit tested by the benchmarks (Appendix B), ten percent of these practices are most responsible for the disparity in outcomes between best performing organizations and all others. Although overlap exists between the baseline and top 10 practices, some baseline practices are unique and as such can be considered a minimum set of practices. Some of the unique baseline practices include: Distributing IT policies for exceptions and adoption throughout the organization Maintaining an updated list of IT assets for managing outcomes Controls implemented to detect and prevent sensitive data from leaking Automating the detection and prevention of unauthorized changes to critical IT assets While most baseline and top 10 practices are commonsense, there is nothing common about their implementation: only four-to-six of these practices are implemented by organizations with normative outcomes, while fewer than three are typically implemented by organizations experiencing the worst outcomes (Table 1). Table 1: Baseline and Top 10 Practices Baseline Best Practices in Domains Distribute IT policies for exceptions and adoption Manage information security from a non-it operations role Protect information security, IT audit and customer data Deliver training to employees and contractors Maintain an updated list of IT assets Use controls to detect and prevent sensitive data from leaking Automate detection and prevention of unauthorized changes to critical IT systems Continuously monitor critical IT assets and verify against IT security standards Conduct daily and weekly assessments of critical IT technical controls Conduct monthly assessments of procedural controls Define business risks based on objectives and standards for integrity, availability and confidentiality Conduct ongoing assessments of business conditions, risks and controls Top 10 Practices Frequent control assessments Protecting IT security data Using common controls between IT security and IT audit Automating the majority of procedural and technical controls Conducting self-assessments of procedural controls Improving IT controls Protecting IT audit data Protecting customer data Reviewing changes against IT policies and standards Managing information security from a non-it operations role 3

5 Outcome-based Best Practices The findings contained in this report are based on practices that are implemented by organizations posting the best outcomes. The benchmarked outcomes used to identify the practices of the best performing organizations include: Lowest losses or thefts of customer and other sensitive data Most resilient IT systems and highest IT service levels Outcome-based best practices Lowest losses of sensitive data Lowest business downtime Fewest problems with regulatory audit Least problems with audit in IT The findings contained in this report are based on a few simple measures: 1) organizations with best, normative and worst outcomes are identified; 2) practices are grouped by outcomes and 3) differences by size, industry and practices are compared to identify the most impact to outcomes. Practice Improvements Pay Off Improving practices for information security and IT audit pay off. Whether measured by lower financial loss and risk from lost or stolen customer data, more resilient systems and less business downtime, less money spent on audit, or less time spent in IT to support and maintain audit results, the financial incentives for making improvements to information security and IT audit practices are multifaceted and compelling (Table 2). Table 2: Practice improvements pay off Worst practices (2 in 10) Normative practices (7 in 10) Best practices (1 in 10) Customer retention, revenue and profit Annualized loss rates for data loss and theft Range of business downtime from IT problems -8 % to -12% lower 1% to 7% of revenue 60 hours and more 0% 6% to 8% higher 0.2% to 2% of revenue 4 to 40 hours 0.1% to 0.2% of revenue Less than 4 hours Spend on audit 75% of norm 100% 50% of norm Time spent on audit 80% of norm 100% 50% of norm Guidance for Best Practices Organizations with the best outcomes implement the baseline and top 10 practices, while using practice guidance from CobiT and COSO to a degree that is not matched by other organizations. Guidance for best practices includes: Implement the missing or partially implemented baseline and top 10 practices to improve outcomes Adopt CobiT and COSO as primary sources of guidance for information security and IT audit practices Adopt ISO, PCI, NIST, vendor guides, CSI benchmarks and regulatory mandates as secondary sources of guidance For some organizations, implementing the baseline and top 10 practices may simply be augmenting existing practices or adding the few that are missing from current standard operating procedures. However, for other organizations, implementing the baseline and top 10 practices may involve changes to current practices and the addition of practices that are not currently part of standard operating procedures. 4

6 Guidance for Implementing Best Practices Based on the findings, guidance for implementing practices for better outcomes includes establishing objectives and standards, orienting the organization, and implementing the baseline and top 10 practices (Table 3). Table 3: Guidance for implementing best practices Guidance Establish objectives Define unacceptable business risks related to the use of IT Focus on core business risks and objectives for controls Fewer than 30 is optimal Establish standards Organizational structure Practices Protect critical assets and information Continuous assessment and reporting Define internal standards for integrity, availability and confidentiality of information and IT assets. Define objectives for controls to manage business risks related to the use of IT. Manage information security from a non-it operations role Implement the baseline practices Implement the top 10 practices Employ guidance for practices, including: 1. CobiT and COSO 2. ISO and PCI 3. ITIL, vendor guides, NIST, CIS benchmarks, regulatory mandates Protect critical information, especially IT security data, IT audit data, and customer data Conduct very frequent assessments of the effectiveness of controls against standards for integrity, availability and confidentiality - Once per week for sensitive data - Once bi-weekly for IT general controls - Once per month for procedural controls Use common controls between IT audit and information security for - Backup, archive and data quality tests and reports - Procedural and technical tests and reports on controls - Audit and IT security tests and reports - Procedural and technical controls Organization of the Report There are four sections to this report, as follows: Executive Summary: contains the key findings of the research and recommendations for best practices for information security and IT audit. Benchmark Findings: covers in detail the principal benchmark findings for practices as these relate to outcomes, and steps to consider taking to improve practices for information security and IT audit. Appendix A: includes detailed findings related to guidance employed by organizations for managing the integrity of information, managing information security practices and procedures, managing compliance with regulatory audit, managing information security policies, and managing business risks related to the use of IT. Appendix B: contains the list of 104 of the most common practices for information security and IT audit tested by the benchmarks, and aligned by outcomes across more than 3,000 organizations 5

7 Benchmark Findings Distribution of Outcomes The best performing organizations share several characteristic outcomes that set their practices apart from all other organizations. These characteristics include the: Least loss or theft of sensitive information Lowest business downtime due to IT failures or disruptions Fewest audit deficiencies in IT However, best performing organizations represent just slightly more than 1 in 10 organizations (12 percent). In contrast, nearly 7 in 10 organizations (69 percent) are experiencing normative outcomes with higher rates of data loss or theft, increased levels of business downtime due to IT failures or disruptions, and elevated levels of audit deficiencies in IT that must be corrected to pass audit. Unfortunately, nearly 2 in 10 organizations (19 percent) are suffering from the highest levels of data loss or theft, the worst business disruptions from IT failures or disruptions, and the most difficulty maintaining and passing regulatory audits in IT (Figure 1). Figure 1: Distribution of outcomes The most recent benchmark results covering practice guidance conducted with 319 organizations are nearly identical to operating and financial outcomes being experienced by the larger population of more than 3,000 organizations. A few key results show that although there are slight skews in outcomes based on size and industry, the primary 2-in-10, 7-in-10, and 1-in-10 distribution of outcomes (worst, normative and best) is primarily due to practices implemented for information security and IT audit. Moreover, the research findings reveal similarity of outcomes across all three measures. For example, almost all (97 percent) of the organizations with the least loss or theft of sensitive information are the same organizations with the least business downtime and the fewest problems with audit deficiencies in IT. A majority (93 percent) of the organizations with normative outcomes for data loss or theft are the same organizations with normative outcomes for the other two. Lastly, a majority (76 percent) of the organizations with the worst outcomes for audit deficiencies are the exact same organizations with the highest rates of data loss or theft, and the highest level of business impact due to IT failures and disruptions. Practices and Better Outcomes in IT The findings show that almost all organizations employ an empirical approach to assessing and reporting on the effectiveness of controls, using IT audit and IT security test reports as a basis for evaluating the integrity of information, the integrity of practices and controls, for managing regulatory audit, managing business risks related to the use of IT, and for managing information security policies. However, the characteristics unique to best performing organizations include 6

8 well defined internal standards for two primary focus areas: 1) confidentiality, integrity and availability standards, and 2) unacceptable business risks related to the use of IT (Appendix A). In addition, best performing organizations uniquely employ CobiT and COSO guidance to inform and guide practices, while supplementing these with guidance from ISO, PCI, NIST, vendor guides and regulatory mandates (Table 4). Although supplementary forms of guidance from ISO, PCI, ITIL, vendor guides, CIS benchmarks, NIST guidelines and regulatory mandates are all employed, CobiT and COSO are used for practice guidance by 10, 20 and 30 times more of organizations experiencing the best outcomes (Appendix A). Table 4: Information security and audit practices and guidelines for better outcomes Practice area Table stakes for all organizations Primary practice guidance unique to best performers Supplementary practice guidance unique to best performers Managing the integrity of information Backup, archive and data quality test reports CobiT and COSO ITIL and vendor guides Managing information security practices and procedures Procedural and technical controls assessments, audit and IT security test reports CobiT, COSO, PCI NIST FIPS, FISMA, ITIL, vendor guides, CIS benchmarks Managing compliance with regulatory audit Procedural and technical controls assessments, audit and IT security test reports CobiT, COSO PCI and ISO ITIL, NIST, FIPS, FISMA, vendor guides Managing information security policies Internal standards for confidentiality, integrity and availability, legal counsel and advice Unacceptable business risks, vendor guides, standards for confidentiality, integrity, and availability Regulatory mandates (PCI, GLBA, HIPAA, FISMA, SOX), Industry standards (ISO, CobiT, NIST, FIPS), legal advice and counsel Managing business risks related to IT Risk assessment reports, audit and IT security test reports CobiT, COSO, Octave, RiskNav and similar tools CobiT, ISO and Practices for Better Outcomes The benchmark outcomes of 3,280 organizations, aligned by outcomes, are mapped against specific practice guidance defined by guidance contained in ISO (ISO 17799) and CobiT (CobiT 4.1). The 104 practices selected (Appendix B) are the most common practices implemented by most organizations for information security and IT audit. Occupying more than 21 practices domains in ISO and CobiT, the 104 practices are simplified into 12 practice domains for ease of presentation. The 21 domains include: communicating management aims and directions; managing the information architecture; managing IT processes, the organization and relationships, managing human resources, managing quality, assessing and managing IT risks, acquiring and maintaining the technology infrastructure; managing changes; enabling operation and use; installing and accrediting changes and solutions; ensuring continuous service; ensuring NIST Practice coverage - Communicating management aims and directions - Managing IT processes, organization, relationships - Managing the Information architecture - Managing human resources - Acquiring and managing IT assets - Managing information and data - Managing operations - Ensuring systems security - Monitoring and evaluating - Managing quality - Managing risks - Governance systems security; educating and training users; managing the configuration; managing problems; managing data; managing the physical environment; managing operations; monitoring and evaluating internal control; ensuring compliance with external requirements; and providing IT governance. Practices in some domains are not tested by the benchmarks. These include: managing the strategic IT plan; managing the IT investment; managing projects; identifying automated solutions; acquiring and maintaining application software; 7

9 managing third party services; managing performance and capacity; and managing the service desk and incidents. As such, incomplete or untested benchmarks findings for these domains are not included. Once aligned by outcomes, implementation levels for each practice are counted and totaled to provide the frequency of implementation: a percentile of implementation by outcome. The differences in the range of percentile implementations are grouped by the 12 domains, with the largest gaps and leading baseline practices for each domain identified. Which Practices Work As the benchmark results show, far more organizations with the best outcomes are implementing practice guidance contained in CobiT and COSO. Of the 104 most common practices tested, the findings reveal that on average, organizations with normative outcomes are half as likely (42 percent to 58 percent) to implement the same practices as the best performing organizations, and with somewhere between 4 and 6 of the top baseline or best practices implemented. It should be no surprise that organizations with the worst outcomes are one-tenth as likely to implement the same practices as the best performing organizations and with fewer than three of the baseline and top 10 practices. The range of practice implementations is striking in its direct relationship with outcomes being experienced by organizations (Figure 2). The level of practice implementation by outcomes produces two clear findings, as follows: 1) Better outcomes are directly related to practice improvements for information security and audit, and 2) Some practices matter more than others The evidence from the benchmarks clearly shows that to improve outcomes, most organizations with normative results will have to improve current practices. However, not all practices will have to be improved to the same extent. For example, among organizations with normative outcomes, some practices in managing the information architecture may already be close enough to the same practice implementation of the best performers, while other practices may have to be improved dramatically. Figure 2: Range of CobiT and ISO domain practices, by outcomes Outcomes and practices Better outcomes are directly related to practice improvements for information security and IT audit: some practices matter more than others. The range of practice implementation shows the worst performing organizations have the most ground to cover to improve practices and outcomes, while for a majority of organizations operating at the norm the amount of effort to improve outcomes and practices is going be much less. The actual practices that need to be improved depend on whether practices are implemented or not, and the level of implementation. Even the best performing organizations are not uniformly implementing all 104 of the most common practices. Rather than attempt to implement all 104 of the most common practices, it makes more sense to identify the practices most commonly implemented by the majority of the best performing organizations for comparison. Ideally, a self-assessment comparing current practices and the most commonly employed practices of the best performing organizations will deliver prescriptive guidance into practices that are close enough to defer and practices most in need of 8

10 improvement. Gaps between the most commonly implemented practices by the best performing organizations will identify how close, or far away, current practices are to achieve best outcome levels. Differences in Implementation of Practices, by Domains Knowing which practices need to be improved, and by how much, as well as practices that can be deferred or ignored, is extremely helpful when aiming to improve outcomes and results. Based on the benchmarks with 3,280 organizations, some stark differences in practice areas become readily apparent when the practices of organizations with different outcomes are compared with those being implemented by the best performing organizations. When the practices in the domains are grouped, differences in practice implementations reveal two important points: (1) some practices are adequate enough, and 2) other practices are in need of more improvement, with some in need of the most improvement to replicate the success being enjoyed by organizations with better outcomes for information security and audit (Figure 3). Figure 3: Differences in practice domain implementations 9 Practice improvements among nearly best performing organizations Organizations experiencing close to the best outcomes exhibit very little difference in practices when compared to the best performers. Nevertheless, small incremental improvements can be made to specific practices for monitoring and evaluating, quality, governance and managing the information architecture to improve outcomes further. Practice improvements for organizations with normative outcomes In comparison, most organizations experiencing normative outcomes exhibit much greater differences in the extent of practice implementations for information security and IT audit when compared with the best performers. The primary focus areas of improvement among normative performing organizations should be practices for monitoring and evaluating. After this, practices for governance, managing the information architecture, and managing quality contain the next most deficient practices that need to be addressed to further improve results. Practice improvements for organizations with the worst outcomes Organizations experiencing the worst outcomes related to information security and IT audit exhibit the largest differences in the extent of practices implemented when compared with the best performers. The primary focus areas of improvement among worst performing organizations include monitoring and evaluating, governance, managing the information architecture, and managing quality. Significant Gaps for Individual Practices In addition to comparing practice domain implementation levels, significant gaps exist between the level of implementation of specific practices between most organizations and the best performers. For example, the practices with the most consistent significant gaps include:

11 The frequency of controls assessments Employing common controls between IT audit and information security Conducting self-assessments of procedural and technical controls Automating procedural and technical controls Protecting IT security data These five practices contain the largest gaps between the levels of practice implementation when compared with the best performers for almost 9 of every 10 organizations. The rank ordered list of the top must improve practices with the largest gaps compared to the best performing organizations changes as organizations approach nearly best-in-class. However for most organizations, the 9 in 10 not close-enough to best-in-class results, the top 5 practices only change order and are not as nuanced (Table 5). Table 5: Prioritized practices, based on gaps to best outcomes Prioritized improvements to practices Organizations with the worst outcomes Organizations with normative outcomes Organizations with nearly best outcomes* 1 Increase the frequency of controls assessments Increase the frequency of controls assessments Automate procedural and technical controls 2 Protect IT security data Protect IT security data Produce IT operational level quality reports 3 Conduct self assessments of procedural controls 4 Automate procedural and technical controls 5 Use common controls between IT security and IT audit Use common controls between IT security and IT audit Automate procedural and technical controls Conduct self assessments of procedural controls Produce Electronic dashboard reports Produce IT audit and IT security test reports Produce financial and business impact summaries 6 Protect customer data Improve IT controls and procedures Produce IT policy compliance reports 7 Improve IT controls and procedures 8 Protect IT audit and reporting data 9 Implement strong information security and audit functions outside of the IT operations function Protect IT audit and reporting data Protect customer data Review and communicate changes to IT policies and standards Use common controls between IT security and IT audit Produce financial and business impact summaries Protect design data Source: IT Policy compliance Group, 2009 Missing from the table of prioritized practices are important targets for practices that are being implemented by the best performing organizations. Covering a range of metrics collected by the benchmarks that include frequencies of control assessments, and the number of control objectives that are directly linked to audit costs and operating outcomes, the practice targets provide a way to assess how well practices and outcomes align with best performing organizations, and by how much practices may need to be improved. Experience of one large organization For example, a large entertainment organization with multiple business divisions went through the process of identifying all of its control objectives managing risk related to the use of IT. Across multiple business lines the organization found more than 120 different and distinct control objectives. The organization found duplication of effort and additional costs due to the number of control objectives. After a period of review, the organization whittled the list of 120 objectives to 30 that were common across multiple divisions. After another period of business and financial review, the original 120 control objectives were reduced to 36 after the first year, and 22 in the second year. The 22 business risk control objectives are now making it possible for the organization to 10

12 be laser-focused in the procedural and technical controls needed to manage business risks related to IT, while instituting new practices to improve outcomes. Most of the best practices identified by the benchmarks are now implemented by this organization with substantial reductions in audit costs, vast improvements in the protection of sensitive information, and much better assurance that information assets are better protected today than in the past. As the experience at this organization illustrates, and the benchmark findings demonstrate, knowing which practices are more important than others needs to be accompanied by specific targets for practice improvements to yield better outcomes (Table 6). Table 6: Practice Targets based on Best Outcomes Practice Targets Focus Frequency of control assessments Once per week Once bi-weekly Once monthly Sensitive information IT general controls (includes technical controls for applications, systems and information security) Procedural controls Risk reporting Coincident with assessments Core business, regulatory and legal risks Improvements to controls Automation of controls Common controls between IT audit and IT security Controls Control objectives At least a 50/50 mix of procedural and technical controls. At least 50 percent of controls and reporting fully automated At least 60 percent of controls for IT audit should be identical to those employed for information security Key technical controls include user access permissions, critical systems, applications and information Reduce to core set of control objectives base don key business risks Higher levels of technical controls 75 percent improve outcomes. Higher levels of automation 75 percent of all procedures and technical controls improve outcomes Higher levels of common controls 80 percent and more improve outcomes Key procedural controls include IT change management, segregation of duties, and employee training If the list exceeds 30 core control objectives, it is too large Baseline Practices for Better Outcomes Irrespective of the largest gaps in implementation rates of practices, or specific practice targets implemented by organizations with the best outcomes, there are baseline practices implemented by best performing organizations at levels far above most other organizations. These baseline practices, by domain, include: distributing IT policies to relevant constituents for exceptions and adoption among employees and contractors; and placing responsibility for the management of information security into a specialized role, including a chief information security officer (CISO), a chief risk officer (CRO) or a chief compliance officer (CCO) outside of IT operations. Whereas operations is goaled and rewarded by what happens, namely maintaining always-on operations for business purposes, the information security function is goaled and rewarded for what does not happen, namely avoiding losses in integrity to information and IT assets, and retaining confidentiality for critical information assets. Working across business operations, legal counsel, IT operations, physical plant security, regulatory audit, internal audit, and finance, the organizations with the best outcomes are consistently managing the information security function in a non-it operations role. Another baseline practice among best performing organizations is the use of automation to detect and prevent unauthorized changes to critical IT assets. By preventing changes to assets and information that are violations of policy, the best performing organizations are eliminating problems and costs associated with recovering from problems that are more prevalent among organizations with less well developed practices. Further baseline practices include controls being used to detect and prevent the leakage of sensitive data, consistent training delivered to employees and contractors, and continuous monitoring of critical IT assets to maintain adherence to policy. 11

13 An additional baseline practice includes an actively maintained inventory of all IT assets. Knowing what is on the network, and whether the devices and services are valid, and whether an unauthorized change occurred is one of the basic practices being implemented by all of the best performing organizations. Although the baseline practices may not exhibit the largest gap between best performing organizations and others, these practices are either close seconds, or are the same practices. As such, the baseline practices should be considered holistically as best practices and implemented, even if some of the top 10 best practices may take additional time to fully implement (Table 7). Table 7: Baseline Practices by Domain Practice domain Baseline practices in the domain Management communications Managing IT processes and the organization Managing the information architecture Managing human resources Acquiring and maintaining information assets Managing information and data Managing operations Ensuring systems security Monitoring and evaluating Managing quality Managing risks Governance Distributing IT policies to relevant constituents, policy adoption and enforcement Managing information security from a non-it operations role Protection of information security, IT audit, reporting, and customer data Delivering training and awareness for employees Maintaining an updated list of assets with continuous monitoring and reporting Maintaining controls to manage sensitive data from leaking Automating procedures to detect and prevent unauthorized changes to critical IT assets Continuous monitoring of critical assets and verification with IT security standards Daily and weekly assessments of critical IT technical controls Monthly self assessments of non-automated procedural controls Defining unacceptable business risks related to IT based on organizational objectives for integrity, availability and confidentiality Ongoing assessment of changes in business conditions, risk profiles and controls. Practice Improvements Pay Off The benchmarks also track financial results being experienced by organizations, including when latent data loss or theft event rates result in financial loss, and loss experiences from lost or stolen customer data, including: capital reductions in shareholder value, direct expenses incurred for litigation, penalties, settlements, cleanup, restoration, customer notifications, and credit services. Additional metrics tracked by the benchmarks include customer retention, revenue declines, profit declines, overall spend on audit, time spent by IT to demonstrate audit, the frequency of business disruptions due to IT failures or disruptions, and the length of business outages due to IT failures and disruptions. Given the large dataset, consistency of findings and verifications against publicly available data, the findings provide a fairly reliable picture of the financial ramifications for managing outcomes related to information security and IT audit practices, including: Top line results, including customer retention, revenue and profitability Annual audit expenses Time spent labor in IT on audit IT resiliency and business service levels Financial risk from data loss or theft The next sections summarize some of the major financial findings related to information security and IT audit practices across the thousands of organizations measured by the benchmarks. Top Line Business Results 12

14 Organizations with the best practices for information security and audit consistently demonstrate the best top-line results when compared to peers and other organizations. When differences in revenue, profit and customer retention are aligned by outcomes for data loss or theft, business downtime occurring from IT disruptions, and audit deficiencies in IT, there is a consistent finding that shows better outcomes are aligned with better practices for information security and IT audit. Whether the metric is the retention of existing customers, revenues or profits, organizations with better information security and IT audit practices outpace all other organizations for top line outcomes (Figure 4). Conversely, organizations with the worst practices consistently experience the largest declines in customer retention rates, revenues and profits than peers. Furthermore, an almost linear relationship exists between top-line results and the effectiveness of practices for information security and audit. Although it can be argued that better managed organizations consistently post better operating financial results with happier customers, the benchmarks do not measure the connection. Rather, the relationship measured by the benchmarks is between practices for information security and IT audit, and top line business results. Obviously, improvements in revenue between 8 and 12 percent, or improvements in profits between 6 and 8 percent are not solely due to improvements in information security and audit practices. However, it would be disingenuous not to recognize the relationship between customer retention, revenue, profit, and better protected customer data, or negative brand impacts associated with not protecting information assets and information. Although information security and audit practices are most certainly not solely responsible for improvements in customer retention, revenue and profit, the benchmarks show a direct relationship between better practices and better results, even if the margin of contribution has not been benchmarked. Figure 4: Top line business results and practices 13 Audit Expenses The total amount of money spent on audit each year differs considerably by the industry and size of organizations: of the two, size plays the most important factor in how much is spent on audit. This is due to several factors identified by the benchmarks, including: more audits, more conformance and more reporting requirements driven by local geographic, political and legal requirements; an increase in the number of audits impacting IT; differences in audit requirements across geographies and regulatory regimes for different lines of business. In addition to size, there are differences in total spend on audit by industry, with some industries more, or less, than what other industries spend on audit. However, when annual spend on audit is aligned by outcomes being experienced by audit deficiencies in IT, the relative amount spent on audit reveals a consistent parabola-looking relationship with maximum spending occurring at or near the mid-point for outcomes, where the majority of organizations (7 in 10) operate. The consistency of the findings, even within industries and similar size organizations include: Organizations with normative practices are overspending, by as much as two times more Organizations with the worst practices are overspending, by as much as one-third more The findings reveal that for most organizations, the 7 in 10 operating in the normative range, the annual savings from reductions in rounds of testing, more focused testing based on a reduced set of objectives linked to core business risks

15 and regulatory requirements, reductions in consulting fees, and reduced internal labor costs associated with demonstrating conformance with audits can be as much as 54 percent annually. Similarly, for 2 in 10 organizations with the worst practices, the annual savings on audit expenses can be as much as 33 percent annually (Figure 5). Figure 5: Money spent on annual audit expenses by practices The organizations with the best outcomes are uniformly spending much less on audit each year, with all of these best performing organizations being from different industries and of different size. The results clearly indicate that it is the practices that are responsible for lower spend in audit, not size or the industry of an organization. Even when organizations of similar size in the same industry are compared, the same relative spending curve, based on practices, is replicated. The benchmarks confirm that improved practices in information security and audit result in lower audit fees. Although spend by industry cannot be changed for a given organization, and spend by size is unlikely to change for a given organization, lower spending on audit for any organization is directly related to the practices implemented for information security and IT audit. While impressive, achieving a 50 percent reduction in annual spend on audit is going to depend on several factors, including: 1) Implementing better practices for information security and IT audit 2) Near-term expense reductions from fewer rounds of audit testing 3) Longer-term reductions as contracts and labor expenses are reduced Actual experience from the benchmarked organizations shows that improved practices for information security and IT audit are directly resulting in lower expenses for audit. Time and Labor Costs for Audit in IT The amount of time spent by IT to support audits depends on the number of audits impacting IT, the size and industry of an organization. Of the three, size is the most important factor. A narrow range of between 25 percent and 39 percent of time is spent in IT to support audit for most organizations, with an average of 32 percent being spent by IT to support audit. In addition, some variation occurs by industry, with some industries spending more or less time to support audits in IT than other industries. However, this variation is similarly constrained to a narrow band varying by 5 to 10 percent. However, when the time spent by IT to support audit is aligned by outcomes and practices for information security and IT audit, the relative amount of time being spent to support audits in IT reveals a similar parabola with the maximum time spent to support audit in IT occurring near the mid-point, where a majority of 7 in 10 organizations are operating. Much like money spent on audit, time being spent to support audit in IT reveals: Organizations with normative practices spend almost twice as much time in IT to support audit Organizations with the worst practices spend almost one-third as much time in IT to support audit 14

16 For most organizations, the 7 in 10 operating in the normative practices range, the annual savings from reductions in internal labor costs can be as much as 52 percent annually. Similarly, for 2 in 10 organizations with the worst practices, the annual savings on audit expenses can be as much as 30 percent annually (Figure 6). Figure 6: Time spent to support audit by IT by practices The research reveals mixed approaches are being used by organizations for the savings in time. For example, some of the best performing organizations are reducing operating expenses because skills are not transferrable, while many others are redeploying skills to improve practices and outcomes. Of the two approaches, the organizations spending the savings from IT audit on improvements to practices dominate among best performing organizations. Business Downtime Improving practices for information security and IT audit are accompanied by less business downtime from IT failures or disruptions. Whether the reason for business downtime is due to malware, losses of information integrity, or to natural events or loss of power impacting service-levels, the findings show organizations with the best practices experience much less business downtime (Figure 7). Figure 7: Reductions in business downtime 15

17 The evidence indicates a close relationship exists between business service levels, operational resiliency and best practices for control objectives, controls, operational efficiency, and ultimately customer retention rates, revenue and profit. Financial Exposure and Loss: Data Loss and Theft Benchmarked financial loss, occurring after the loss or theft of sensitive customer is directly related to information security and IT audit practices. Improve the practices and the loss decreases and the likelihood of loss is lowered. Annualized loss exposure tracked by the benchmarks and actual industry experience reveals declining rates as practices in information security and IT audit are improved (Figure 8). For example, an organization of $1 billion in annual revenue with the worst information security and IT audit practices has an expected annualized loss rate that is slightly more than 2 percent of annual revenue. In contrast, such an organization with normative practices is exposed to an expected annualized loss rate that is slightly less than 1 percent of revenue. An organization with best practices however, has a much lower annualized exposure rate that is between 0.1 and 0.2 percent of revenue. Whether the experience is more recent such as the breaches that are reported to have occurred at Amuse, Mitsubishi, Fiserv, the U.S. Army, Network Solutions, HSBC, Anthem, Virginia Department of Health, or less recent experience such as the reported data breaches at TRW, Heartland Payment Systems, CheckFree, Norwegian Tax Authority, or the U.K. Ministry of Defence, the publicly available data shows some interesting findings, including: Organizations with the worst practices experience multiple loss events in shorter periods of time Roughly half of the losses are due to inadequately managed procedural controls About half of the losses occur due to inadequately managed technical controls Improving practices for information security and IT audit are shown to reduce the occurrence rates of data loss or theft and the onset of these events from resulting in financial loss. The lesson from the benchmarks and industry experience is clear: improve practices for information security and audit and financial exposure and loss is dramatically lower, and occurs much less frequently. Figure 8: Reductions in financial risk from data loss or theft Although some of the publicly documented loss experiences are higher and lower than the benchmarked experience of 3,280 organizations, the benchmark findings and publicly reported loss experiences fall within the predictable ranges as depicted. In addition to the benchmarks, a reliable source for industry experience can be found in the data maintained by OSF at Annual accruals of expected loss from the risk of customer data loss or theft, while not common, are being recorded internally against operating expenses at some of the organizations with the best practices. Among most other organizations however, these losses are only recorded on publicly visible quarterly operating statements after latent loss and theft rates from poorer practices result in financial loss from litigation, clean-up expenses, fines and penalties. 16

18 Whether the objective is reducing financial exposure, increasing customer retention, maximizing revenue and profit, avoiding business disruptions, reducing audit fees, or reallocating labor in IT for more critical projects, the findings provide strong financial evidence it is worth the effort to improve practices for information security and IT audit. Taking Action to Improve Outcomes Current practices may have to be modified, augmented, or added to improve outcomes and results. Change is always difficult, but is a necessary part of improving results for the 9 of 10 organizations not operating with the best outcomes. While useful, practice guidance from CobiT, COSO, PCI, ITIL, NIST and regulatory mandates can only point an organization in the correct direction: guidelines cannot substitute for action that results in better outcomes. The primary actions taken by the best performing organizations to improve practices and results include: (1) prioritizing and managing the risks; (2) establishing objectives and measuring results; (3) rationalizing IT policies and procedures; (4) delivering training for employees; (5) segmenting and limiting access to customer and sensitive corporate data; (6) improving IT controls and procedures; (7) automating the collection of IT audit and information security data; and (8) frequent assessment and reporting. The guidance for improving practices for information security and IT audit contained is this report is shown to improve operational outcomes, reduce costs and financial risks. This guidance, based on the results of more than 3,280 organizations, will hopefully serve as guideposts for improvement (Table 8). Table 8: Guidance for Best Practices in Information Security and Audit Establish objectives Establish standards Organizational structure Practices Protect critical assets and information Continuous assessment and reporting Guidance Define and maintain what constitutes unacceptable business risks related to the use of IT. Focus on core business risks and objectives for controls: fewer than 30 is optimal. Define and maintain internal standards for confidentiality, integrity and availability of information assets. Maintain objectives for controls being used to manage the primary business risks related to the use of IT. Manage information security from a non-it operations role. Implement the baseline practices Implement the top 10 practices Employ guidance for practices, including: 1. CobiT and COSO 2. ISO and PCI 3. ITIL, vendor guides, NIST, CIS benchmarks, regulatory mandates IT security data, IT audit data, customer data, critical IT assets, and sensitive organizational data, systems, applications, networks, and user accounts. Conduct very frequent controls assessments against internal standards for confidentiality, integrity and availability (CIA standards - Once per week for sensitive data - Once bi-weekly for IT general controls - Once per month for procedural controls Use common controls between IT audit and information security for - Backup, archive and data quality tests and reports - Procedural and technical tests and reports on controls Improvements worth the effort The findings provide strong financial evidence it is worth the effort to improve practices for information security and IT audit. 17

19 Appendix A: Guidance for Practices Recent benchmarks completed with 319 organizations determined the form of guidance being employed by organizations for practices implemented in information security and IT audit. The results are organized as practices and guidance for six key focus areas, and an assessment of the level of customization required, as follows: Managing the integrity of information Managing information security practices and procedures Managing compliance with regulatory audit Managing information security policies Managing business risks related to the use of IT Level of customization required Managing the integrity of information Perhaps the most critical area of focus for most organizations is how well information and IT assets are maintained free from corruption or unintended change. The two dominant practices employed by most organizations to maintain the integrity of information include: Backup and archive test reports Data integrity checks and quality test reports These test reports are followed by vendor guides and the ISO 9000 and ISO quality standards at much lower incidence levels. Given the large proportion of organizations employing backup, archive, and data integrity test reports to ensure information integrity, these practices can be considered as table-stake practices that must be implemented. When the findings are directly linked to outcomes being experienced by organizations, the picture that emerges reveals a dominant use of CobiT and COSO as practice guidance for ensuring the integrity of information among organizations with the best outcomes. The findings are conclusive: almost thirty times the number of organizations with the best outcomes rely on CobiT and COSO for practice guidance for managing the integrity of information (Figure 9). Figure 9: Guidance for managing the integrity of information, by outcomes In contrast, a majority of organizations with normative outcomes are relying on SCAP, CVE, and SDLC guidelines for managing the integrity of information. As a protocol for updating patches to systems and a common method for classifying vulnerabilities, these two guidelines, while useful for classifying and remediating problems, are apparently not contributing materially to maintaining the integrity of information. 18

20 Lastly, organizations with the worst outcomes are relying on combinations of backup, archive, integrity test reports, vendor guides and ISO quality standards as the primary practice guidance for information integrity without relying on assistance from CobiT, COSO, ITIL SCAP or CVE. The findings clearly show that backup, archive, data integrity and quality reports, along with practice guidance from CobiT and COSO form a foundation for best practices to maintain the integrity of information and better outcomes. Managing information security practices and procedures The two dominant guidelines employed for managing information security practices and procedures for most organizations include: IT audit and IT security test reports Assessments of procedural and technical controls Between four and five of every ten organizations rely on these two empirical approaches test reports and assessments of controls to manage information security practices in the organization. Although other forms of guidance are employed, such as vendor guides and NIST guidance, these are employed at much lower levels than test reports and assessments of controls. When practices are directly linked to outcomes being experienced, the evidence reveals organizations with the best outcomes are relying primarily on guidance from CobiT, COSO, and to a somewhat lesser extent on PCI, NIST, FIPS and FISMA. More than 20 times the population of the best performers employ CobiT and COSO for guidance on information security practices and procedures, followed by supplementary use of PCI, NIST, FIPS, and FISMA practice guidance (Figure 10). Figure 10: Guidance for information security practices and procedures, by outcomes In contrast, the primary guidance employed by a majority of organizations with normative outcomes is the practice guidance contained in the ISO information security practices, including ISO 17799, and 27002). However, fewer organizations with normative outcomes use CobiT, COSO, PCI, or ITIL as guidance for information security practices. Lastly, organizations with the worst outcomes are relying primarily on vendor guides and not much else. Although most vendor guidelines are critically important to the effective implementation and maintenance of technical controls and among some the maintenance of non-technical or procedural controls, by themselves vendor guidelines are apparently insufficient to improve outcomes. The findings indicate the to be effective, vendor guidelines have to be led by audit and IT security test reports, procedural and technical control assessments, and guidance from CobiT, COSO, PCI and NIST among other forms of guidance. When the practices are aligned by outcomes and checked against audits, the results show that PCI is being employed for guidance among best performing organizations even when there are no PCI audits. 19