How to Get from Scans to a Vulnerability Management Program Gary McCully Any views or opinions presented are solely those of the author and do not necessarily represent those of SecureState LLC.
Synopsis Many organizations falsely equate a vulnerability scanner with a Vulnerability Management Program. A scanner is important to the overall program, but can only help with a few processes on its own. This paper discusses the processes involved in a Vulnerability Management Program, while focusing on tasks that vulnerability scanners like Qualys and Nexpose can either directly perform or assist with. Table of Contents Vulnerability Checklist... 3 Discovery... 5 System Inventory and Business Owner Identification... 5 Create Scan Templates Specific to Each Baseline... 6 Scan... 6 False Positive Removal... 7 Prioritization... 7 Remediation and Security Control Implementation... 8 Re-scan... 8 Root Cause Analysis... 9 Monitoring... 9 Ongoing Tracking, Trending, and Analysis... 10 Conclusion... 10 2
Vulnerability Checklist Developing a Vulnerability Management Program is crucial for an organization to successfully manage their vulnerabilities. Often times, a Vulnerability Management Program is mistaken for a vulnerability scanner, which is false. Although a vulnerability scanner is an important part of the program, there are only a few processes that the scanner is able to perform on its own; most require human intervention. Even though a vulnerability scanner may assist in performing tasks like a root cause analysis, remediation, and management of false positives, it is unable to perform such tasks without human involvement. Unfortunately, there are several additional tasks a vulnerability scanner is unable to assist with, including defining roles and responsibilities as well as the creation of policies and procedures used to drive the Vulnerability Management Program. The table below lists the processes required for a solid Vulnerability Management Program while focusing on the specific tasks a vulnerability scanner can either directly perform or assist with. Cannot Can Help Task Roles and Responsibilities - Roles and responsibilities must be created for individuals involved in the Vulnerability Management Program. Discovery - This is the process of determining what devices are active on the organization's network. System Inventory and Business Owner Identification - This is the process of creating an inventory of the contents of the devices the discovery scan identified. This inventory should include both technical (Operating System, Applications, Services, etc) as well as non-technical (Types of Data the system stores/transmits/process, the asset owner, etc). Data Classification - The organization must clearly articulate what data they value most. It is also important to know what data the organization stores/processes/transmits that must meet regulatory requirements. Asset Classification - Assets should be grouped according to asset criticality. Asset criticality should take into account the data that is stored/transmitted/processed on the server, business impact, etc. Create High Level Policies and Procedures that Support Vulnerability Remediation Efforts - High level policies must be created at the beginning of the Vulnerability Management Program. These policies must articulate service level agreements that are based on asset criticality and vulnerability severity. Procedures should be established for testing system level changes before making changes to production servers. Create Baselines of Classified Assets - Determine what kind of vulnerabilities pose the greatest risk to a specific asset group. Determine acceptable vulnerability baselines for such assets. Create Scan Templates Specific to Each Baseline - Customize scan templates to meet the needs of each asset group. Asset groups should contain assets of similar asset value. Scan - Use a vulnerability scanner to scan devices on the network for vulnerabilities. 3
Cannot Can Help Task False Positive Removal - Remove false positives from scan data. Prioritization - Prioritize vulnerability remediation efforts based on asset classification and vulnerability severity. Remediation and Security Control Implementation - Remediate the vulnerabilities that the scanner identified on vulnerable devices. If the vulnerability cannot be remediated, compensating controls must be implemented to mitigate the risk that the vulnerability poses to the organization. Rescan - Rescan devices that the scanner reported as containing vulnerabilities in order to verify vulnerabilities are remediated. Compensating controls effectively addresses the risk that the vulnerability poses to the organization. Root Cause Analysis - Determine the root cause of the identified vulnerabilities and eliminate the root cause of such vulnerabilities. Create Final Policies and Procedures Regarding Network Segmentation, Patch Management, Minimum Security Baseline Implementation, and Remediation of Vulnerabilities - At this point, the Vulnerability Management Program has finished its first scan, remediated identified vulnerabilities, and addressed root causes of the problems. The next step is to finalize policies and procedures related to the Vulnerability Management Program. Policies and Procedures surrounding Network Segmentation, Patch Management, Minimum Security Baseline Implementation, etc. may also need to be updated and/or created depending on the analysis of the root cause. Monitoring - Monitor the network for new vulnerabilities by performing ongoing security assessments. Manual Assessments to Identify Holes that Scanners Cannot Find and Verify Security Controls and an Effective Vulnerability Management Program - Vulnerability assessments are primarily a tool-based assessment. The problem lies within the fact that tools cannot find certain vulnerabilities (such as flaws in business logic). Manual assessments such as External Attack and Penetration Assessments, Internal Attack and Penetration Assessments, Whitebox, Greybox, and Blackbox Web Application Assessments should be performed in order to determine the effectiveness of the security controls the organization has put in place. Ongoing Tracking, Trending, and Analysis - A Vulnerability Management Program is not a onetime assessment. Instead, it is an ongoing process that should provide continual analysis and trending of the vulnerabilities identified on the network over long periods of time. It should be able to generate reports that clearly reflect the technical security posture of the organization over time. 4
This list reveals some pretty interesting items regarding a scanner's role in a Vulnerability Management Program. First, there are several tasks that a scanner cannot perform within a Vulnerability Management Program. Second, there are very few tasks a vulnerability scanner can do in and of itself. The third item that should be noted is the fact that the vulnerability scanner can assist with many tasks that are not specific to vulnerability scanning. In this paper, I will address items related to the Vulnerability Management Program that a scanner can either perform or assist. When I discuss specific vulnerability scanners, I will focus most of my time on Qualys QualysGuard and the Enterprise Version of Nexpose. I have found that both of these vulnerability scanners fit very well into an overall Vulnerability Management Program. Discovery I have always loved war movies. I find that one of the most nerve racking scenarios is when an enemy sniper is hiding and the good guys do not know of the location. The sniper takes aim and starts inflicting damage upon the good guys. The good guys cannot address the problem until the sniper s location has been identified. Once the location is identified, the good guys are able to stop the threat (normally with heavy artillery). This scenario is correlated to the discovery phase of a Vulnerability Management Program. A device may exist on the network which poses a significant risk to the organization's network. However, the threat cannot be addressed until the organization is aware of the device s existence on the network. Discovery involves identifying what assets are active on the organization's network. In fact, this is one task that vulnerability scanners are great at. In order to scan a device for vulnerabilities, the scanner needs to know what devices are active on the network. The scanner will use a number of methods (TCP Packets, ICMP Requests, ect.) in order to identify what systems are alive on the network segment. Once identified, the organization can perform system inventory and owner identification. Qualys & Nexpose: These scanners allow organizations to run discovery scans. This scan uses various types of packets in order to determine which devices are active on the organization's network. It should be noted that a discovery scan s purpose is not to identify vulnerabilities. In fact, its purpose is to determine what assets are active on the organization's network. Both of these vulnerability scanners will also identify which hosts are active on the organization's network as part of the normal vulnerability scanning process. System Inventory and Business Owner Identification System inventory involves cataloging the applications, hardware, operating system, services, data, etc. on the specific device. Most vulnerability scanners, such as Qualys, Nexpose, and Nessus, fingerprint the operating system and the externally facing services. This information is part of the scan results that can be reviewed after a vulnerability scan is completed. Although this information is helpful, it is only a subset of the technical information on a device that should be inventoried. Organizations can purchase software which specializes in tracking system inventory and can be used to pull required inventory information as needed. Additionally, technical information should be inventoried, such as applications, operating systems, services, and the type of data that the device stores. It is critical that the organization knows which devices store/transmit/process sensitive data. In addition to inventorying technical information and what data the system houses, it is important that the owner of the device is tracked. This information should be tracked for a number of reasons. First, if something were to happen to the device during a vulnerability scan (such as it stops responding), the person in charge of running the scans knows who to 5
contact. The second reason is so that the person in charge of vulnerability scanning can contact the asset owners of devices containing vulnerabilities as indicated by the scanner. Qualys & Nexpose: Both of these scanners can provide limited information about the devices that they scan. This includes items such as externally accessible services, operating system, etc. Neither can provide the necessary information for a full inventory of technical components, data, and asset owner. Create Scan Templates Specific to Each Baseline Let's suppose that you own two watches. One is a $9,000 Rolex and the other is a $10 Timex watch. Needless to say, you will treat these watches differently! Which watch would you wear when jogging? Which watch would be worn to an important black tie event? You would use these watches differently due to the disparity in value between the two watches. Similar to an individual s watches, all of an organization's assets do not have the same value. Due to the discrepancy in value, the organization should not treat all of their assets the same way. Some assets contain credit card data, others hold sensitive and/or proprietary information, some will have a high business impact on the organization (such as mail server), some maintain applications that are business critical, and others maintain applications that are not business critical. In reality, an organization's assets have different functionality and value. Not all of the organization's assets should be treated the same. It is important to create scan templates for each asset group that meets its specific needs. Qualys and Nexpose: Both of these vulnerability scanners allow users to create various scan templates that can be used to scan different asset groups. Organizations should customize these scan templates to meet the needs of the devices that they are scanning. Scan I remember the first time I watched Star Wars, Return of the Jedi. The Rebellion had created an elaborate plan to destroy the second Death Star. The Starfighters could not destroy the second Death Star until its shield generator was eliminated. Once the team on Endor eliminated the shield generator, the Starfighters could blow the Death Star to pieces. This is similar to the scan portion of the Vulnerability Management Program. A solid Vulnerability Management Program has many steps that must be performed before the actual scan takes place (Roles and Responsibilities, Discovery, Data Classification, Asset Classification, etc.). Once all the preparatory work is complete, it is time to destroy the Death Star!...or run the scan. This is where the vulnerability scanner shows that it is able to do what it claims to do. Vulnerability Scanners should be able to detect vulnerabilities in various operating systems, services, applications, databases, web applications, etc. Qualys & Nexpose: Both of these scanners do a nice job of identifying vulnerabilities on an organization's network. These scanners can identify missing patches, insecurely configured software and services, etc. In addition to running non-credentialed scans against organization's assets, both Qualys and Nexpose can run credentialed scans as well. Credentialed scans generally identify several additional flaws in devices that non-credentialed scans cannot. The reason for this is the fact that a credentialed scan accesses the application as an authenticated user, thus granting less restrictive access to the operating system and installed applications. 6
False Positive Removal Have you ever had a conversation that went something like the following: Security Personnel: Good afternoon Mr. Smith, I am calling to inform you that device X.X.X.X. is vulnerable to vulnerability H1-N1. Technical Personnel: Isn't that a vulnerability from over 50 years ago? If I recall correctly, it had something to do with the device broadcasting its username and password every 5 seconds to each device on the network. Is this the vulnerability you are referring to? Security Personnel: That is the exact vulnerability! As you know this is an extreme vulnerability allowing an attacker to gain access to every device on the network at once! Technical Personnel: This is a brand new server. I just installed the newest version of Windows last week. I also verified that it is completely updated. H1-N1 is a vulnerability on UNIX and is 50 years old! This server is not vulnerable to H1- N1! No matter how good a vulnerability scanner is, it is going to report some false positives. These false positives can skew reporting and make it difficult to perform trending and analysis. Not to mention the fact that if the person performing the vulnerability scanning continues to send the same false positives to the same asset owners month after month, these asset owners will tend to get angry at the person performing the scanning. Both Qualys and Nexpose have effective ways to address false positives. Qualys gives the person in charge of running the scans the ability to ignore specific vulnerabilities on specific assets. That person can then run reports that list all ignored vulnerabilities at a later time. This is an important step because vulnerabilities should never be ignored indefinitely. The vulnerability scanner can run an ignored vulnerability report once every three to six months. This report should be reviewed in order to verify that labeling a vulnerability a false positive is still valid. Nexpose allows organizations to mark vulnerabilities as false positives and choose to ignore them for a specific amount of time. When the time comes, the vulnerability will once again appear during the vulnerability scan. This is a good feature because it makes sure that false positives are not ignored indefinitely. This forces the organization to periodically review false positives in order to review the business justification as to why these vulnerabilities are considered false positives. Prioritization Imagine running a scan against an internal network. At the end of the scan, you get a report that shows 700 vulnerabilities on 200 individual systems. Some of these vulnerabilities pose a significant risk to the device the vulnerability was identified on; others pose a substantial risk to the affected device, but are not as severe as some of the other vulnerabilities. Some devices with identified vulnerabilities are part of a test lab, which is fully segmented from the rest of the network. Further vulnerabilities were identified on servers that stored critical business data, and other vulnerabilities were found on systems storing data that must comply with specific regulations. The person in charge of performing vulnerability scanning is responsible for leading remediation efforts. With all the various vulnerabilities and affected systems, it can be difficult to prioritize the remediation efforts. 7
Qualys allows grouping of assets into "asset groups". Basic asset criticality information can be assigned to these "asset groups". Qualys is able to run reports correlating asset criticality information with the severity of the vulnerability, serving as a basic remediation roadmap. Nexpose allows assets to be placed in "asset groups". Basic asset criticality can be assigned to each asset group. The asset criticality information is correlated with vulnerability severity in order to prioritize vulnerability remediation efforts. This correlated information can help with the prioritization of remediation efforts. Remediation and Security Control Implementation One of the most important tasks of a vulnerability scanner is to provide clear and detailed reports. These reports should include a vulnerability description, information regarding the threat that the vulnerability poses to the device, details on how to remediate the vulnerability, vulnerability severity, etc. It is critical to know if a vulnerability scanner has the ability to articulate how it found a particular vulnerability as well as a detailed remediation outline. Without this information, the organization may need to spend several hours locating solutions to address the identified vulnerabilities. Qualys & Nexpose scanners both provide detailed information regarding the identified vulnerabilities and what steps are required to remediate the issue. When an organization finishes running their first vulnerability scan and sees all the vulnerabilities on their network, they may feel a little overwhelmed. Often times a scan will reveal hundreds of vulnerabilities on the network that pose a significant risk. Tracking remediation efforts for each of these vulnerabilities can quickly become an overwhelming task! Many individuals attempt to track remediation efforts using spreadsheets; however, they quickly realize a spreadsheet is inadequate to track all the remediation efforts. Larger organizations may significantly benefit by using ticketing systems to track remediation efforts. Both Qualys and Nexpose come with built-in ticketing systems that can be used to support vulnerability remediation efforts. These ticketing systems are nice because they keep all data relating to remediation efforts in one centralized location. Re-Scan Once the organization has finished remediating vulnerabilities and implementing compensating controls, it s time to rescan the assets. The rescan is performed to verify that the vulnerabilities have effectively been addressed. More than once, I have been in situations where I was told that a vulnerability had been remediated. Upon rescanning the affected device, I found that it was still present. I contacted the person who assured the vulnerability had been remediated and informed them that the vulnerability was in fact, still present in the affected asset. A few days later, I was contacted again and told the vulnerability was remediated. Once again, I rescanned the device and found that it was still vulnerable. I proceed to contact the person who let me know that the vulnerability had been remediated and asked them how they attempted to remediate the vulnerability. Upon further inquiry I realized that the person who was working on remediating the vulnerability had no clue as to what the vulnerability was or how to remediate it. The point is, just because someone informs you that a vulnerability has been remediated, does not make it true. It is important to rescan the affected system to verify that the vulnerability has truly been addressed. 8
Both Qualys and Nexpose have the ability to scan a specific device multiple times to verify the vulnerability has been addressed. Root Cause Analysis Imagine that you have a broken window in your house. Within a few days, you notice copious amounts of flies have filled the rooms. You bug bomb every room in the house and the flies are pretty much eliminated, however, you never fixed the broken window. A few days later, the flies have started to fill the house once again. Unfortunately, this is a game that many security professionals take part in on a regular basis. The vulnerability scanner identifies a number of missing patches on the network and the organization starts applying patches to all the identified areas. When it's time to run a vulnerability scan once again, you will never guess what the scanner finds...missing Patches! Vulnerabilities are normally the result of a much larger flaw in a secondary process. If the vulnerability scanner keeps identifying missing patches, then maybe it's time to review the patch management program. If the vulnerability scanner keeps identifying applications that have vulnerabilities as a result of insecure configurations (Default Usernames and Passwords, Blank SA, etc.), then maybe it's time to review system hardening procedures, minimum security baselines, policies and procedures regarding the application of minimum security baselines, etc. The point is, if the root of the vulnerability is not addressed, the organization will be forced to deal with the same vulnerabilities month after month. One of the main ways that vulnerability scanners can assist with root cause analysis is by providing robust reporting. Comparative reports are one of the most important reports that can be used for trending and analysis. These reports compare a scan performed at one point in time with a scan performed at a secondary point in time. The report highlights the differences between the two scans, enabling the organization to quickly identify how their security posture has improved and/or deteriorated over time. These reports also can quickly highlight systematic problems with the patch management program, system hardening guidelines, minimum security baselines, and policies and procedures that enforce the application of system hardening guidelines and minimum security baselines, etc. Both Qualys and Nexpose scanners have the ability to report the comparison between two scans performed at separate points in time. Monitoring Imagine that you have a new car. Your maintenance manual states that the oil in the car needs changed every 5,000 miles. The first 5,000 miles is up and you bring the car into the shop to have the oil changed. You follow the maintenance manual s guidelines for changing the oil for the first couple of years. After a while, the novelty of the new car starts to wear off and you stop monitoring the number of miles until the next oil change. A year has passed and you re driving your car down the highway when the lack of oil changes finally catches up with your car. The car suddenly breaks down and when you open the hood, you are greeted by a cloud of black smoke. Congratulations! You just destroyed your car's engine! Once an organization reaches their ideal level of security, they may feel as though "they have finally arrived." Organizations must fight this temptation and continue to monitor their security posture on an ongoing basis. If the organization does not continue to monitor their security posture, they will quickly fall into a deteriorated state of security. A Vulnerability Management Program is not a one-time assessment. It is a continual process utilizing assessments such as vulnerability scanning on a regularly scheduled basis. At a very minimum, the discovery task of the Vulnerability 9
Management Program should be performed once a month in order to discover what new devices were placed on the network. As new devices are identified, a full vulnerability scan should be performed to verify no vulnerabilities are contained in these devices. A full vulnerability scan of every device on the network should be performed once a quarter (every three months). Both Qualys and Nexpose scanners can be used on a continual basis to scan the organization's network. Ongoing Tracking, Trending, and Analysis The vulnerability scanner should be able to quickly generate reports that can be used for tracking, trending, and ongoing analysis. It s important have the ability to quickly generate reports showing the current technical vulnerabilities in the organization s network and the threats they pose. Data of vulnerabilities obtained over multiple scans should be able to be quickly correlated and compared. It is recommended that the organization have policies which require scan data to be saved for a minimum of two years. This data should be kept in a format that the scanner can read and use in its reporting engine. Both Qualys and Nexpose scanners are able to generate clear and detailed reports used for Tracking, Trending, and Analysis. Conclusion Qualys and Nexpose are good vulnerability scanners that can be used as a solid part of a much larger Vulnerability Management Program. It is important for organizations to understand the role vulnerability scanners play in an overall Vulnerability Management Program. It is also important to know the limitations of such scanners. They are an important part of a Vulnerability Management Program; however, they are not the program itself. 10