CIO Update: Enterprise Security Moves Toward Intrusion Prevention

Transcription

1 IGG J. Pescatore, R. Stiennon Article 4 June 2003 CIO Update: Enterprise Security Moves Toward Intrusion Prevention As targeted hacker attacks increase, intrusion prevention is gaining importance while intrusion detection is fading away. Gartner has defined three key criteria for true network-based and hostbased intrusion prevention. Targeted hacker attacks on enterprises have been increasing, and are generally launched by more sophisticated and motivated attackers. Intrusion prevention is gaining importance while intrusion detection is fading away. Gartner has defined three key criteria for true network-based and hostbased intrusion prevention. Intrusion Detection Has Problems As enterprises became disenchanted by the performance of intrusion detection products, security vendors stopped using the word detection and began relabeling their products as intrusion prevention or intrusion protection. However few products provide the features that Gartner believes are necessary for true intrusion prevention. In the post-internet boom, with the dawn of Web services and with the network becoming an integral part of business operations, security solutions such as intrusion prevention systems are being introduced that offer real protection for the enterprise. Similar to intrusion detection, intrusion prevention can be separated into two broad categories host-based and network-based. Mandatory Requirements for Intrusion Prevention An intrusion prevention system must meet three key criteria: It must not disrupt normal operations. When it is inserted online, a network-based intrusion prevention system must not introduce unacceptable or unpredictable latency into a network. A hostbased intrusion prevention system must not use more than 10 percent of a system s resources. Normal network traffic and host-based processes should operate identically, whether an intrusion prevention system is running or not. Blocking actions must occur in real time or near real time, with latencies in the range of tens of milliseconds (not seconds). It must block malicious actions using multiple algorithms. Intrusion prevention systems must provide blocking capabilities that include signature-based blocking of known attacks. However, intrusion prevention systems must also move beyond simple signature-based approaches such Gartner Entire contents 2003 Gartner, Inc. All rights reserved. Reproduction of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice.

2 as those used by antivirus and intrusion detection systems to at least support policy, behavior and anomaly-based detection algorithms. These algorithms must operate at the application level in addition to standard, network-level firewall processing. It must have the wisdom to know the difference (between attack events and normal events). As intrusion prevention systems mature, they will be able to positively identify and block higher percentages of attacks than today s first-generation intrusion prevention systems (that is, firewalls) do. However, they will never be perfect, and it will always be necessary to flag suspicious activity for further human investigation. Thus, the intrusion detection market will be relegated to mere first-alert status. Host-Based Intrusion Prevention Host-based intrusion prevention is software that resides on a server and prevents cyberattacks against the operating system or applications. Products from Okena and Entercept Security Technologies have had early success in protecting servers, particularly against the Code Red and Nimda attacks. Host-based intrusion prevention is an immediate cure for vulnerabilities in servers, but because of the costly overhead of managing security software on many diverse platforms within the enterprise, host-based intrusion prevention systems will not see the same adoption rate as network-based intrusion prevention. Host-based intrusion prevention technology can apply policies based on predefined rules or learned behavior analysis to block malicious server or PC actions. Host-based intrusion prevention can stop attackers from implementing buffer overflow strikes, changing registry keys, overwriting Dynamic Link Libraries or engaging in other approaches to obtain control of the operating system. Host-based intrusion prevention can be implemented as software shims that intercept calls between applications and the underlying operating system, or as kernel modifications that apply morestringent security controls than those built into commercial operating systems. Examples of software shims are: Network Associates/Entercept Security Technologies Cisco Systems/Okena Sana Security GreenBorder Technologies Examples of kernel modifications are: Argus Systems Group Sun Microsystems Trusted Solaris Operating System Hewlett-Packard s Virtual Vault

3 Host-based software that simply locks down the host and only allows certain applications to execute does not meet Gartner s criteria for host-based intrusion prevention, because it does not protect against flaws in permitted applications. Network-Based Intrusion Prevention The advantages of network-based intrusion prevention systems include the reduced importance of constant monitoring, and that an attack does not set off chimes and claxons that cause a chaotic scramble to react. Network administrators know that Code Red attacks have become part of the background radiation of the Internet. Therefore, the time spent logging and responding to such attacks is wasted. Once identified, the affected session should simply be dropped. Thus, not only are valuable resources conserved, but also a better overall security posture is achieved. The defining characteristics and benefits of network intrusion prevention are: Firewalls and gateway antivirus systems are examples of first-generation, network-based intrusion prevention systems. However, firewalls primarily operate at the network protocol level, and antivirus systems largely implement simple, reactive (that is, non-real-time), signature-based detection and blocking. A true network-based intrusion prevention system must: Operate as an in-line network device that runs at wire speeds. Perform packet normalization, assembly and inspection. Apply rules based on several methodologies to packet streams, including (at a minimum) protocol anomaly analysis, signature analysis and behavior analysis. Drop malicious sessions don t simply reset connections. To do all that, network-based intrusion prevention must perform deep packet inspection of all traffic, and generally must use special-purpose hardware to achieve gigabit throughput. Software-based approaches that run on general-purpose servers may be sufficient for small enterprise use, and blade-based approaches may scale up to some large enterprises. However, for complex networks running at gigabit rates, Gartner believes that application-specific integrated circuits and dedicated network security processors will be required to perform deep packet inspection, and to support blocking at wire speeds. Vendors include: TippingPoint Technologies IntruVert Networks NetContinuum ipolicy Networks Fortinet

4 Characteristics and Benefits of Network Intrusion Prevention The defining characteristics and benefits of network intrusion prevention are: Inline position: Rather than tapping into a data stream from the switch or other device, the products sit inline with the data stream. Inline systems can analyze and identify packets and sessions, verify which are malicious and drop the associated stream of packets. That is essential to the products protective abilities. Stateful signature: To efficiently handle multigigabit traffic streams, some form of stateful inspection must be used. The state of a particular communication going over a network includes the ability to have session knowledge about the packets being analyzed. Some awareness of state enables the engine to parse only the pieces of the session that are applicable to the attack signature. That provides high throughput and low latency, which are also required for enterprise applications. Combined algorithms: No single methodology can catch the maximum number of intrusion attempts while minimizing false positives. Intrusion prevention systems must use a combination of methodologies: Signature analysis is the most powerful method, but it must be augmented with protocol/packet anomaly detection. Protocol/packet anomaly detection focuses on signatures within the protocol or packet that have been defined as hostile, malformed, out of sequence or potential zombies, which are some distributed denial-of-service (DDoS) relay kits that can serve as transmitters for floods of packets to be sent to DDoS targets servers. The relay kits use Internet Relay Chat channels to communicate back to controlling hackers, who can direct the relay kits to start attacking certain Web sites. By bombarding sites with bogus traffic, hackers can make it impossible for a site to respond to legitimate connections. Behavior-based statistics are less exact, but they can provide a valuable function. This technique involves analyzing baseline metrics of known traffic patterns, then setting the alert threshold when extreme traffic pattern changes occur, such as massive flooding that may indicate a denial-of-service attack. (Flooding may also indicate a legitimate network traffic surge. Thus, notification can maintain or alert required infrastructure changes to meet valid traffic demand.) Dropping malicious traffic: Once a malicious session is identified, it is simply dropped, which protects the destination server or device. Logging and alerting are functions of these devices. Intrusion Prevention Summary Some facts about intrusion prevention are: Firewalls are intrusion prevention devices. As intrusion prevention begins to include application-level attack blocking, products must meet a minimum set of criteria before enterprises can take them into consideration.

5 Intrusion detection will always be required to give warnings about activities that are suspicious but not necessarily hostile. Most enterprises will require hardware-based intrusion prevention products to protect highspeed networks. Bottom Line As processing power and security algorithm performance increase, intrusion prevention will grow in importance, while intrusion detection will shrink. However, through 2006, enterprises should deploy a combination of both capabilities to meet security best practices. Written by Edward Younker, Research Products Analytical source: John Pescatore and Richard Stiennon, Gartner Research This article is an excerpt of a chapter from a new report, Securing the Enterprise: The Latest Strategies and Technologies for Building a Safe Architecture. The report is an offering of the Gartner Executive Report Series, a new business venture of Gartner Press that provides buyers with comprehensive guides to today s hottest IT topics. For information about buying the report or others in the Executive Report Series, go to For related Inside Gartner articles, see: Management Update: Security Strategies for Enterprises Using Web Services, (IGG ) CIO Update: Gartner s IT Security Management Magic Quadrant Lacks a Leader, (IGG ) CIO Alert: Follow Gartner s Guidelines for Updating Security on Internet Servers, Reduce Risks, (IGG )