IBM Managed Security Services Vulnerability Scanning:

IBM Managed Security Services August 2005 IBM Managed Security Services Vulnerability Scanning: Understanding the methodology and risks Jerry Neely Network Security Analyst, IBM Global Services

Contents 2 About scanning 2 Caveats 4 Internet scanning 4 Intranet scanning 6 Conclusion About scanning Remote vulnerability scanning uses a scan appliance to actively probe a computer network, cataloging pertinent information about that network and the machines attached to it. The goal of remote vulnerability scanning is to identify exploitable weaknesses that could compromise the network s security. Vulnerability scanning is a continual process. Because new vulnerabilities are found regularly, a machine that was previously considered as operating in a security-rich manner can suddenly become vulnerable to an attack. Likewise, as new systems are added to a network, it is important that they be scanned to verify that they are operating in a security-rich manner. A single vulnerable system can compromise the security of an entire network. For the security professional, vulnerability scanning is an important tool for determining the security posture of a network. Regular scanning presents a clear picture of the network at several levels: Which IP addresses are active Which ports are open on each of these active servers What services are listening on open ports (such as HTTP or FTP) What remotely exploitable vulnerabilities exist for each active service Server compliance to policy standards (such as unauthorized service running or user account with no password or default password) Knowing exactly what IP addresses are in use helps ensure that unauthorized machines that might compromise the security of a network have not been connected to it. Likewise, knowing what ports are active helps ensure that new services that might compromise the security of a network have not been enabled. Understanding what vulnerabilities exist on a network is the first step in remediating these problems and verifying that any security holes are patched before they can be exploited. Caveats Unlike most of the security services offered by IBM Managed Security Services, vulnerability scanning is an active service. This means that rather than passively monitoring a customer s network or systems, the act of scanning has a tangible affect on the environment being scanned. Systems and network appliances are probed to determine whether known vulnerabilities exist.

Highlights Scanning servers has a negligible effect on system performance and observable performance degradation on less powerful machines Vulnerability scanning is not without risks. For example, in the vast majority of cases, the most noticeable effect of this activity on a given host system is an increase in CPU utilization as the target system responds to the network requests it receives from the scanner. Systems that would commonly be deployed as servers are generally powerful enough to withstand any such increase in network activity, so the scanning would have a negligible effect on overall system performance. When a less powerful machine is scanned, the impact can cause an observable degradation in performance for the duration of the scan. Another risk is that a scan can cause an actual disruption in service. There are generally two situations where this is likely to happen. The first of these is during the service recognition phase. The scanner sends a number of queries to the port being interrogated, looking for a recognizable response that the scanner can use to identify the service running on that port. Some servers, usually those with older software, can react badly to this recognition process. This can result in a disruption of the affected server, usually requiring a manual restart of the process. Buffer overflows attempt to identify a vulnerability that can be used by an attacker The other problematic situation occurs when certain classes of tests are run against a given server. Most of these tests fall under the category of buffer overflows. A buffer overflow attempts to identify a vulnerability that can be used by a malicious attacker to execute arbitrary code on the server. Rather than exploiting the vulnerability by passing executable instructions, the scanner attempts to overflow the buffer only with a garbage string to cause the server process to terminate. With most modern software, this results only in the termination of one of many child processes servicing user requests, which is immediately respawned. Some older software lacks this sophistication and a successful buffer overflow test can result in disruption of that server, usually requiring the process to be manually restarted. Before your first scan, update and fully patch all server software In general, older software tends to be more sensitive to vulnerability scanning. Before your first scan, it is a good practice to update and fully patch all server software. An initial service discovery scan can be run to assist in identifying the software running on a network. This will aid in the patch process to prepare for scanning.

Highlights Internet scanning The Internet should be viewed as a hostile environment. Machines connected to the Internet are open to attack from anywhere in the world. Furthermore, because of political boundaries, it is often impossible to prosecute or otherwise seek relief from such attacks. Therefore, it is imperative that any server connected to the Internet be fully patched against known vulnerabilities. IBM Managed Security Services uses an extensive battery of tests when scanning devices connected to the Internet. There are some risks associated with internet scanning. The comprehensive set of tests IBM uses to perform vulnerability scanning of Internet-facing devices includes tests that might cause a Denial of Service (DoS) condition on the target machine. For example, repeated attempts to authenticate by using weak passwords might cause an account to be locked due to excessive failures. However, in an environment like the Internet, this would be likely to happen even without a vulnerability scanner. Carefully consider differences between Internet and intranet access when developing vulnerability scanning strategy Intranet scanning The typical private network, or intranet, differs significantly from the Internet. The Internet is accessible in some form to nearly anyone in the world. Although firewalls may limit access to an Internet-facing system, it is still, to some degree, exposed to anyone who cares to attack it. This is not the case for intranet machines. Access to intranets and the machines attached to them are restricted, both physically and logically. In general, only employees of a company are granted access to an intranet. When developing a strategy for performing vulnerability scanning, these differences must be carefully considered in order to optimize the benefits of the scan while reducing the risks of disruption to vital systems. Vulnerability scanning of intranet systems is an important practice that helps reduce internal threats While it is more tightly controlled than the Internet, an intranet should still be viewed as a potentially hostile environment. The majority of security breaches come from inside a business, rather than outside. For this reason, vulnerability scanning of intranet systems is an important practice that helps reduce these internal threats.

Highlights Attacker access can be limited on an intranet; this may not be possible with the Internet Because of the increased control of the environment, once an incident has been identified, dealing with it is much easier than it would be if the attack were coming from the Internet. The attacker s access to the network can be effectively terminated or controlled as necessary; often this is not possible on the Internet. Therefore, it is not as important to harden systems against brute force attacks that generally take some time to execute and can be readily detected by IDS systems or even by monitoring system logs. Likewise, it is not as imperative to scan for vulnerabilities that would be attacked in such a manner. In fact, scanning for such vulnerabilities on an intranet might actually create a DoS condition on the target machines if they are configured to automatically lock out accounts or services when a brute force attack is detected. This is one example of the kind of consideration that should come into play when you are formulating a security policy and deciding on a profile for intranet scanning. One other thing to remember is that any time a remote scanner can cause a DoS situation on a given server, that server is clearly vulnerable to a malicious attacker using the same vector to create the same DoS condition. The primary difference then becomes one of when this activity takes place; when the network owner is prepared for it and ready to patch the servers or at a time of the attacker s choosing. Augmenting with a network service discovery scan gives an overview of network changes and a detailed vulnerability scan of vital servers Another factor to consider in intranet scanning is the list of targets to be scanned. All machines on the Internet should be scanned vigorously and frequently; however this may not be the best policy for an intranet. Networkattached printers, for example, are not typically plagued by remotely exploitable vulnerabilities, but they are notoriously susceptible to service disruptions when scanned. Scanning desktop PC s is another activity of questionable value. Since remote scanning focuses primarily on vulnerabilities found in server software, many businesses choose to focus intranet scans on multiuser servers. The downside to scanning only servers is that it does not inform the network owner of new devices or services on the network. This can be addressed by augmenting a targeted vulnerability scan with a network service discovery scan. This provides both an overview of any changes on the network and a detailed vulnerability scan of vital servers.

Highlights Any business should carefully consider the risks and benefits of vulnerability scanning on an intranet. Many of the risks of scanning have been described here. A given business might choose to disable certain classes of tests in order to avoid potential disruptions in service. However, this may leave vulnerabilities undiscovered on the network. The cost of one of these vulnerabilities being exploited would likely far outweigh the inconvenience of having access to a server disrupted because of a vulnerability scan. Remote vulnerability scanning is an important tool for safeguarding a computer network Conclusion Remote vulnerability scanning is an important tool for safeguarding a computer network, whether it is a private intranet or part of the Internet. Vulnerability scanning does present some potential dangers, but these are far outweighed by its benefits. The following activities can go a long way toward mitigating these potential risks: Taking the time to prepare the network by updating and patching software Determining which machines and TCP/UDP ports should be targeted for scanning Choosing the set of tests to be performed on each network layer As with most preventative measures, these activities will take some time and effort, but the anticipated payoff will be a security-enhanced network with a limited risk of scan-related disruptions. For more information To learn more about IBM Managed Security Services and IBM Global Services, contact your IBM representative or visit: ibm.com/services

Copyright IBM Corporation 2005 IBM Global Services Route 100 Somers, NY 10589 U.S.A. Produced in the United States of America 08-05 All Rights Reserved. IBM, the IBM logo, and the On Demand Business logo are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Other company, product, and service names may be trademarks or service marks of others. References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates. The IBM home page on the Internet can be found at ibm.com G510-6138-00