Operational Efficiencies of Proactive Vulnerability Management Return on investment analysis
Table of Contents Automation Brings Efficiencies 3 Survey Results 3 Cost Elements for 4 Cost Assumptions 4 VMA 1 Inventory Assets 4 Sample asset inventory operational costs 6 VMA 2 Assess Vulnerabilities 6 Sample calculations 7 VMA 3 Correlate Threats 7 Sample calculations 8 VMA 4 Remediate and Validate 9 Sample calculations 10 Conclusion 10 About Cooper Research Associates 10 About McAfee, Inc. 10
In the security industry, the high costs associated with security risk management are often difficult to quantify. How do you calculate the value of good will lost in the wake of a major denial of service attack that prevents customers from accessing their accounts? How do you calculate the costs associated with data corruption or the extent to which the integrity of information resources has been violated as a result of a malicious attack on your network? Automation Brings Efficiencies The answers to these questions vary from industry to industry and from company to company, and they can be very difficult and time consuming to measure. This paper details the operational and economic efficiencies associated with the implementation of proactive, integrated, and automated vulnerability management compared to the inefficiencies of a reactive, manual, and fragmented security operation. When activities are managed or implemented in a manual and non-integrated manner, the operational cost of vulnerability management rises in proportion to the number of devices, systems, and the complexity of the enterprise network. However, if vulnerability management activities (VMAs) are automated and integrated, then the operational cost of vulnerability management can be significantly reduced by several orders of magnitude while actually elevating the security posture of the enterprise networking environment. VMAs are those functions performed by network and security personnel to protect against, prevent, and recover from security events that can challenge the integrity of enterprise information assets or threaten to disrupt business continuity. Survey Results To establish a benchmark of awareness of the crucial elements and activities associated with managing enterprise security operations, CRA Reports surveyed 149 security officers and IT managers who have security responsibilities. Among the survey s findings: 88.6 percent of respondents have initiatives in place to inventory network assets 75.8 percent have programs that prioritize network assets in terms of criticality to the enterprise 59.1 percent currently correlate known threats to their organization s critical information assets Only 38.3 percent have automated the process of remediating vulnerabilities on their enterprise systems 77.9 percent report that clearly defined policies for security compliance are in place in their organizations 58.4 percent regularly measure the performance of their security operations to established security policies Only 54.4 percent report requirements to provide senior management with regular reports on the security posture of their organization More than 65 percent report that they have not established ROI metrics for security risk management initiatives in their organizations. Further analysis of the survey results reveals that most companies overspend on discrete aspects of their security risk management operations, while underinvesting in broad categories of security activities. Along with financial investments, most organizations allocate disproportionate human and technical resources on some aspects of their security risk management operations, while ignoring other often critical elements of their security operations. The reason for the discrepancy has to do with the way organizations organize their security risk management operation, often resulting in the following: The need for manual reconciliation of silos of security automation Very high costs from this approach A lack of resources needed to perform all of the functions to maximize the security posture 3
Cost Elements for Although specific risks, vulnerabilities, and threats tend to be unique to each organization, many of the procedures and costs associated with managing security vulnerabilities like any business process share common elements that are consistent and that can be tracked, measured, and quantified. In other words, while the staff, networks, and devices maintained by different organizations vary, there are a common set of VMAs that provide a common basis for comparative analysis. This report identifies those quantifiable elements, defines and describes them, and provides a matrix against which individual organizations can compare the actual costs of running a manual vulnerability management operation against automated approaches. In identifying elements that can be quantified and analyzed objectively, we have developed a simple and effective operational cost of vulnerability management (OCVM) formula to which each element must apply. The formula is as follows: Devices (number of live devices on your network) Time (time for each VMA) Cycles (the number of times you will complete this activity) Operational Cost of (D x T x C = OCVM) Cost Assumptions VMAs that could not be quantified by this formula were not included in this analysis. We identified four VMAs that were applicable to the OCVM formula. They are: VMA 1 Inventory assets VMA 2 Assess vulnerabilities VMA 3 Correlate threats VMA 4 Remediate issues and validate fixes To provide a basis for financial analysis, we attached a labor rate to the time associated with VMAs. According to a recent security professional salary survey conducted by Dice/Datamation, the most widely listed network security job is a LAN/network administrator, with a starting salary of $71,000, or $36/hour. We have adopted this hourly rate as a conservative basis for assessing the financial costs of vulnerability management. VMA 1 Inventory Assets Almost 90 percent of respondents to the CRA Reports Security Survey reported having a concerted effort in place to inventory network assets (Figure 1). However, the approaches taken by different organizations to inventory and classify their network resources vary significantly depending on the size of the organization and the industry in which they operate. My organization currently prioritzes its network assets. 100% 80% 88.60% 60% 40% 20% 0% Agree 10.70% Disagree 0.70% No Opinion Figure 1. Percentage of respondents with efforts to inventory network assets. (Source: CRA Reports) 4
A full 75 percent of respondents to the CRA Reports survey indicated their organizations engage in efforts to prioritize network assets in terms of criticality to their business operations (Figure 2). My organization currently prioritizes its network assets in terms of criticality to the business/agency. 80% 75.80% 60% 40% 20% 0% Agree 20.10% Disagree 4% No Opinion Figure 2. Percentage of respondents whose organization currently prioritizes its network assets in terms of criticality to the business/agency. (Source: CRA Reports.) There is broad consensus that using traditional manual techniques represents a labor- and resourceintensive effort, even for organizations that have asset management systems in place. This is especially true as the size of the organization grows. Nevertheless, there is overwhelming agreement that regular, rigorous, and comprehensive surveys must be conducted of all devices, applications, databases, and processes that make up or are connected to the enterprise network to maintain or improve security posture. The security-oriented inventory assessment steps include: 1. Identify or query all devices in the enterprise. In general, it can take a relatively small organization with up to 256 devices as little as five minutes to fully account for all systems attached to the network. A mid-sized company with up to 65,500 devices can take five hours. And a large Fortune 1000 firm can take as many as 20 hours to identify and query as many 250,000 devices. Identifying subnets, segmentation devices, owners of subnets and devices, and entering findings into a spreadsheet adds further time to this phase of the identification process. It can take two weeks to perform this for up to 5,000 devices, up to three weeks for 10,000 devices, four weeks for 20,000 devices, and five weeks for 40,000. 2. Categorize and prioritize assets by function. Categorizing and prioritizing assets is the most time consuming element of this process. It can take between three to five minutes to appropriately classify an asset once the data gathering phase of the research is done. Because this is such a time-consuming effort, few organizations categorize their assets comprehensively. Most organizations have taken a pragmatic approach and applied the 80-20 rule (20 percent of assets actually represent more than 80 percent of the value to the organization) and categorize those assets accordingly. 3. Make regular updates. Because of the time-intensive nature of the effort, few organizations update their asset inventories more than three or four times per year. It is estimated that each update takes 50 percent of the time associated with performing the original inventory/query. If there are significant changes, additions, or removals, then the updates can take significantly more time to perform accurately. 5
Sample asset inventory operational costs Sample calculations on the operational cost of asset inventory for security are listed below. Manual Assumptions 1 minute per device Subsequent cycles = 50% of first cycle Automated Assumptions 15 hours for all devices Subsequent cycles = 50% of first cycle Manual Process Costs Automated Process Costs Devices 380,000 380,000 Time/Device 1 minute 15 hours* Cycle 1 $228,000 $540 Cycle 2 $114,000 $270 Cycle 3 $114,000 $270 Total Cost $456,000 $1,080 * This assumes that the security or IT professional is monitoring the automated process in real time. Table 1. Operational cost of asset inventory for security purposes. VMA 2 Assess Vulnerabilities The assess vulnerabilities activity provides the foundation for determining the vulnerabilities and the severity of the vulnerabilities, such as misconfigurations or missing patches that are present on the enterprise network. Ideally, a vulnerability assessment will: Look for weaknesses in the network architecture and devices Provide current information to guide how security measures should be implemented Provide the necessary logic to prioritize mission-critical assets The vulnerability assessment steps include: Leverage asset inventory findings Using the data gathered from the asset discovery process, an assessment tool can be used to analyze all live hosts Identify weaknesses, risk exposures, and misconfigurations If this process is implemented frequently, the time it takes to complete each analysis will be relatively short. The process may take longer due to delays between analyses. Match findings of this analysis to inventory Manual correlation between asset spreadsheets and assessment output can take several days. If the analysis is broader for example at a business unit or operational unit level then it could take a few additional days to analyze the impact of the analysis. Review and distribute reports to appropriate personnel This process also has a high level of variability. It is highly dependent on the size and complexity of the enterprise network, as well as the organizational/reporting structure. Generally speaking, it takes approximately one week to segment, summarize, and distribute reports that provide an accurate snapshot assessment of an organization s vulnerability to threats. Since most companies do not have a centralized system that manages inventory and vulnerabilities, the vulnerability assessment activity is typically extremely time consuming and resource intensive. It s difficult for many organizations to organize, correlate, and act on disparate and inconsistent results from desktop assessment tools. 6
Sample calculations Sample calculations on the operational cost of vulnerability assessment are listed below. Manual Assumptions Assessment of 2 minutes per device Subsequent cycles = 50% of first cycle Automated Assumptions Assessment of 20 hours for all devices Subsequent cycles = 50% of first cycle Manual Process Costs Automated Process Costs Devices 380,000 380,000 Time/Device 2 minute 20 hours* Cycle 1 $456,000 $720 Cycle 2 $228,000 $360 Cycle 3 $228,000 $360 Total Cost $912,000 $1,440 * This assumes that the security or IT professional is monitoring the automated process in real time. Table 2. Operational cost of vulnerability assessment. VMA 3 Correlate Threats After scanning for vulnerabilities, the vulnerability management operation should proceed with a threat correlation analysis. Organizations must be constantly aware of emerging threats to enterprise systems, and develop the ability to determine how those threats can potentially affect the security of the organization. And yet, according to the CRA Reports Security Survey, less than 60 percent of organizations have systems in place for correlating known threats to critical information assets (Figure 3). My organization has a system(s) for correlating threats to our critical information assets. 80% 60% 59.10% 40% 33.60% 20% 7.40% 0% Agree Disagree No Opinion Figure 3. Percentage of respondents with a system for correlating threats to critical information assets. (Source: CRA Reports) 7
Threat correlation aggregates potential and known threats against specific assets in the target environment. Often, administrators scour the web to get news, information, and alerts from different sources to identify new threats that might affect their enterprise systems. Done properly, this provides information security officers with an opportunity to anticipate and proactively implement countermeasures before their systems are exposed to threats. The threat correlation steps include: Search and monitor the Internet for information on the most recently identified threats This has become a routine part of an IT or security professional s job description. It can take up to one hour per day. (IT staffs typically don t spend that much time because automated services send a synopsis of threats on a daily basis.) Correlate the results to vulnerability assessment reports When a likely threat is identified, searching through reports for the correct information might take 30 minutes and could take many hours in a large organization Threat correlation is probably the most time-consuming and imprecise process. Because many organizations do not regularly inventory their assets, most systems administrators and security directors do not know all the systems they have in the environment. They therefore do not have a clear idea of which assets may be susceptible to an attack. Once a potential threat is identified, matching it to systems and then checking if vulnerabilities actually exist can also be extremely time consuming. Sample calculations Sample calculations on the operational cost of threat correlation are listed in Table 3. Manual Assumptions 15% of assets reporting potential vulnerabilities 20 minutes per device Subsequent cycles = first cycle Automated Assumptions 15% of assets reporting potential vulnerabilities 2 hours for all devices Subsequent cycles = first cycle Manual Process Costs Automated Process Costs Devices 57,000 57,000 Time/Device 20 minutes 2 hours* Cycle 1 $684,000 $72 Cycle 2 $684,000 $72 Cycle 3 $684,000 $72 Total Cost $2,052,000 $216 * This assumes that the security or IT professional is monitoring the automated process in real time. Table 3. Operational cost of performing threat correlation. 8
VMA 4 Remediate and Validate Less than 40 percent of the CRA Reports Security Survey respondents indicated that they have automated the process of remediating and validating vulnerabilities on their organizations enterprise systems (Figure 4). The vast majority of organizations perform this function manually or do not remediate all vulnerabilities against critical threats. For those companies that do not have a way of prioritizing mission-critical assets, this can elevate risk unnecessarily. My organization has an automated process to remediate vulnerabilities. 60% 50% 40% 30% 20% 38.30% 55% 10% 6.70% 0% Agree Disagree No Opinion Figure 4. Percentage of respondents with an automated process to remediate vulnerabilities. (Source: CRA Reports). Steps must be taken to fix or remediate severe vulnerabilities discovered during the assessment phase. Once the misconfiguration has been fixed or the asset has been patched, then the asset should be tested to ensure that the vulnerability has been fixed correctly. The steps for remediating and validating remediation are: Leverage vulnerability assessment Some form of assessment is a prerequisite for remediation Package vulnerability reports for appropriate personnel Reports need to be packaged and sent out to the network administrators who are charged with protecting and maintaining those assets. In many organizations, the actual remediation activity is performed by IT staffs, not security personnel. Disseminating these reports and work orders often takes a full work day. A technician visits the asset to remediate and validate Managers should budget between 10 to 45 minutes to remediate all high- and medium-risk vulnerabilities on a machine. Low-risk vulnerabilities are usually ignored. Process steps include:»» Technician performs patch and remediation»technician» re-scans box to determine if vulnerability is still there (this step is often not carried out at all and is usually left up to the security team to do any re-scanning)»» Technician generates report and sends report back to security management team»» Security management re-scans for vulnerabilities (for instance, bulk verification) to confirm remediation The remediation and validation process is often convoluted especially in larger organizations. A relatively small percentage of organizations have developed enterprise-wide standard operating procedures for remediation and validation. Several administrators typically manage a large environment, and each will have a certain expertise and will be responsible for discreet parts of a network. Often, different approaches are taken to prioritizing which vulnerabilities get patched first. This can create confusion, as managers or executives receive inconsistent reports. 9
Sample calculations Sample calculations on the operational cost of performing remediation and validation are listed below. Manual Assumptions 15% of assets reporting potential vulnerabilities 15 minutes per device Subsequent cycles = first cycle Automated Assumptions 15% of assets reporting potential vulnerabilities 10 minutes per device Subsequent cycles = first cycle Manual Process Costs Automated Process Costs Devices 380,000 380,000 Time/Device 15 minutes 10 minutes* Cycle 1 $513,000 $342,000 Cycle 2 $513,000 $342,000 Cycle 3 $513,000 $342,000 Total Cost $1,539,000 $1,026,000 * This assumes that the security or IT professional is monitoring the automated process in real time. Table 4. Operational cost of performing remediation and validation. Conclusion When security activities are managed or implemented in a manual and non-integrated manner, the operational cost of vulnerability management rises in proportion to the number of devices, systems, and complexity of the enterprise network. However, if VMAs are automated and integrated with each other, then the operational cost of vulnerability management can be significantly reduced (by several orders of magnitude) while actually improving overall security posture. About Cooper Research Associates The research in this report was prepared by CRA Reports. Founded in 1994, Cooper Research Associates (CRA) is an independent reporting agency with offices in San Francisco, CA and Washington, DC that analyzes user trends in business technology. CRA Reports explore the role that technology products and services play in the overall economy and/or in specific vertical industries. To view a list of current white papers, please visit www.cooperresearchassociates.com. About McAfee, Inc. McAfee, Inc., headquartered in Santa Clara, California, is the world s largest dedicated security technology company. McAfee is relentlessly committed to tackling the world s toughest security challenges. The company delivers proactive and proven solutions and services that help secure systems and networks around the world, allowing users to safely connect to the Internet, browse and shop the web more securely. Backed by an award-winning research team, McAfee creates innovative products that empower home users, businesses, the public sector, and service providers by enabling them to prove compliance with regulations, protect data, prevent disruptions, identify vulnerabilities, and continuously monitor and improve their security. http://www.mcafee.com. McAfee, Inc. 3965 Freedom Circle Santa Clara, CA 95054 888 847 8766 www.mcafee.com McAfee and the McAfee logo are registered trademarks or trademarks of McAfee, Inc. or its subsidiaries in the United States and other countries. Other marks and brands may be claimed as the property of others. The product plans, specifications and descriptions herein are provided for information only and subject to change without notice, and are provided without warranty of any kind, express or implied. Copyright 2010 McAfee, Inc. 8946wp_grc_op-eff-vuln-mgmt_0310_ETMG