A Survey of Intrusion Detection Systems Daniele Sgandurra 1 1 Istituto di Informatica e Telematica, CNR, Pisa, Italy 1/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Outline 1 Introduction Attacks and Threats 2 Characteristics of 3 Static Analysis Run-Time Support 2/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats Broad New Hacking Attack Detected Wall Street Journal (18/02/2010): Hackers in Europe and China successfully broke into computers at nearly 2.500 companies and government agencies over the last 18 months in a coordinated global attack that exposed vast amounts of personal and corporate secrets to theft. [...] infiltrating some 75.000 computers and touching 196 countries. The highest concentrations of infected computers are in Egypt, Mexico, Saudi Arabia, Turkey and the U.S. 3/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats Broad New Hacking Attack Detected 4/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats Mariposa Botnet It is considered the largest botnet, consisting of 12,7 million hosts comprised of systems in businesses, universities, government agencies, and in homes of more than 190 countries. Now it s dead. The stolen data included bank account details, credit card numbers, user names, passwords, etc., belonging to more than 800.000 users. 5/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats The Top Cyber Security Risks Featuring attack data from TippingPoint intrusion prevention systems protecting 6.000 organizations. Vulnerability data from 9.000.000 systems compiled by Qualys. Additional analysis and tutorial by the Internet Storm Center and key SANS faculty members. September 2009. 6/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats The Top Cyber Security Risks Priority One: client-side software that remains unpatched. Priority Two: Internet-facing web sites that are vulnerable. Operating systems continue to have fewer remotely-exploitable vulnerabilities that lead to massive Internet worms. Rising numbers of zero-day vulnerabilities. 7/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats The Top Cyber Security Risks The number of vulnerabilities being discovered in applications is far greater than the number of vulnerabilities discovered in OS. 8/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats IBM s annual X-Force Trend and Risk Report The number of software vulnerabilities fell overall in 2009, but the number of bugs in document readers and multimedia applications increased by 50 %. Of the 5 most prevalent Web site exploits, 3 involved PDF files. The other two exploits involved Flash and an ActiveX control. 9/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats IBM s annual X-Force Trend and Risk Report Browsers had the most client-side vulnerabilities: Firefox had twice the number of critical/high vulnerabilities as IE. More than half of the critical/high client-side vulnerabilities affected just 4 vendors: Microsoft, Adobe, Mozilla and Apple: while on average most vendors patch 66 % of those outstanding vulnerabilities, Apple proved the worst, patching just 38%. 10/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats Targeted Attacks 2008/2009/2010 11/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats Application Patching is Much Slower than Operating System Patching 12/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats Key Predictions for 2010 and Beyond Trend Micro 2010 Annual Threat Roundup: No global outbreaks, but localized and targeted attacks. It s all about money, so cybercrime will not go away: mobile devices will become greater targets for cybercrime. Windows 7 will have an impact since it is less secure than Vista in the default configuration. Risk mitigation is not as viable an option anymore even with alternative browsers/alternative operating systems. 13/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats Key Predictions for 2010 and Beyond Malware is changing its shape every few hours. Drive-by infections are the norm: one Web visit is enough to get infected. New attack vectors will arise for virtualized/cloud environments. Bots cannot be stopped anymore, and will be around forever. Company/Social networks will continue to be shaken by data breaches. 14/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Attacks and Threats Types of Threats Two types of threats: insider and outsider. Insider threat: hard to detect and quantify. Outsider threat: attacks from over the Internet: ubiquitous: background radiation: on average, hosts are probed every 90 sec. medium-size site: 10.000 of remote scanners each day; what do they scan for? A wide and changing set of services/vulnerabilities, attacked via auto-rooters or worms; what are they after? They seek zombies for DDOS slaves, spamming, bots-for-sale,... 15/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Definitions Introduction Characteristics of Intrusion: a set of actions aimed to compromise: integrity, confidentiality, or availability, of a computing and networking resource. Intrusion detection (ID): the process of identifying and responding to intrusion activities, i.e. entities attempting to subvert in-place security control: Intrusion Detection Systems () are SW and/or HW components that monitor the events in a computer or in a network and analyze the activities for signs of possible violations of computer security policies. Intrusion prevention: extension of ID with access control to protect computers from exploitation. Intrusion Detection and Prevention Systems (IDPS). 16/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Intrusion Detection Introduction Characteristics of An intrusion detection system (IDS) finds anomalies. The IDS approach to security is based on the assumption that a system will not be secure, but that violations of security policy (intrusions) can be detected by monitoring and analyzing system behavior. (Forrest 98) The IDS requires: training the IDS (training); looking for anomalies (detection). 17/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Intrusion Detection Systems Characteristics of A Network IDS (NIDS) attempts to identify unauthorized, illicit and anomalous behaviors based on network traffic A Host IDS (HIDS) attempts to identify violations of the security policies on a specific device. A signature-based IDS examines the activities for predetermined attack patterns known as signatures. An anomaly based-ids firstly builds a model of the normal usage of the monitored system and, based on this model, it then monitors the system s activities by classifying them as either normal or anomalous. 18/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Characteristics of Characteristics of 19/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Key Functions IDS Technologies Characteristics of Monitor and analyze events to identify incidents. Record information related to observed events. Notify security administrators of important observed events. Producing reports. IPS also attempt to prevent a threat from succeeding: stop the attack itself; change the security environment; change the attack content. 20/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Network IDS (NIDS) Introduction Characteristics of Network IDS attempts to identify unauthorized, illicit, and anomalous behavior based solely on network traffic: using either a network tap, span port, or hub collects packets. Using the captured data, the IDS system processes and flags any suspicious traffic. The role of a network IDS is passive, only gathering, identifying, logging and alerting. 21/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
NIDS Placement Introduction Characteristics of 22/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
NIDS Example: SNORT Characteristics of Open source IDS. Snort rules. Sample: alert tcp any any -> 192.168.1.0/24 111 (content:" 00 01 86 a5 "; msg: "mountd access";) Rule Header: Action, Protocol, Src+Port -> Dest+Port Rule Options: Alert messages and Packet Content 23/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Host Based (HIDS) Introduction Characteristics of Host-based intrusion detection attempts to identify unauthorized, illicit, and anomalous behavior on a specific device. HIDS generally involves an agent installed on each system, monitoring and alerting on local OS and application activity. The installed agent uses a combination of signatures, rules, and heuristics to identify unauthorized activity. 24/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
HIDS Block Diagram Introduction Characteristics of 25/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
HIDS Example: OSSEC Characteristics of OSSEC is an Open Source Host-based IDS. Log analysis. File integrity checking. Policy monitoring. Rootkit detection. Real-time alerting. Active response. 26/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
OSSEC Example Logs Introduction Characteristics of SSH: May 21 20:22:28 slacker sshd[21487]: Failed password for root from 192.168.20.185 port 1045 ssh2 ProFTPD: May 21 20:21:21 slacker proftpd[25530] proftpd.lab.ossec.net (192.168.20.10[192.168.20.10]): no such user dcid-inv Bind: Aug 29 15:33:13 ns3 named[464]: client 217.148.39.4#32769: query (cache) denied Apache: 127.0.0.1 - - [28/Jul/2006:10:27:32-0300] "GET /hidden/ HTTP/1.0" 404 7218 Windows: Nov 2 17:23:16 192.168.1.100 security[failure] 529 NT AUTHORITY\SYSTEM Logon Failure: Reason:Unknown user name or bad password User Name:Jeremy Lee Domain:IBM17M Logon Type:2 Logon Process:User32 Authentication Package:Negotiate Workstation Name:IBM17M Cisco IOS: Sep 6 09:20:44 RouterName 86: Sep 6 14:20:35.991: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) 27/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Host vs Network IDS Introduction Characteristics of 28/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Physical (Physical IDS) Introduction Characteristics of Physical intrusion detection is the act of identifying threats to physical systems. Examples of: security Guards; security Cameras; access control systems (card, biometric); firewalls; man traps; motion sensors. 29/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Network Behavior Analysis (NBA) Characteristics of Network Behavior Analysis (NBA) examines network traffic to identify threats that generate unusual traffic flows: distributed denial of service (DDoS) attacks; certain forms of malware (e.g., worms, backdoors); policy violations (e.g., a client system providing network services to other systems). Monitor flows on an organization s internal networks. Monitor flows between internal networks and external networks. 30/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
NBA Sensor Architecture Example Characteristics of 31/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Wireless IDS Introduction Characteristics of Wireless IDS monitors wireless network traffic and analyzes its protocols to identify suspicious activity in the protocols. It cannot identify suspicious activity in the application or higher-layer network protocols (e.g., TCP) that the wireless traffic is transferring. Deployed within range of an organization s wireless network, but also to locations where unauthorized wireless networking could occur. 32/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Wireless IDS Placement Characteristics of 33/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Characteristics of Comparison of IDPS Technology Types 34/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Honeypot Introduction Characteristics of Honeypot Systems are decoy servers or systems setup to gather information regarding an attacker or intruder into your system. Can be setup outside or in the DMZ although they are most often deployed inside of a firewall for control purposes. In a sense, they are variants of standard IDS but with more of a focus on information gathering and deception. 35/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Honeypot Introduction Characteristics of 36/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Honeypot Introduction Characteristics of 1 Learn how intruders probe and attempt to gain access to your systems: gain insight into attack methodologies to better protect your real production systems. 2 Gather forensic information to aid in the prosecution of intruders: to provide law enforcement officials with the details to prosecute. 37/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Signature-Based Detection Characteristics of A signature is a pattern that corresponds to a known threat. Signature-Based Detection is the process of comparing signatures against observed events to identify possible incidents. Examples: a telnet attempt with a username of root, which is a violation of an organization s security policy an e-mail with a subject of Free pictures! and an attachment filename of freepics.exe, which are characteristics of a malware an operating system log entry with a status code value of 645, which indicates that the host s auditing has been disabled. 38/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Signature-Based Detection Characteristics of Very effective at detecting known threats but largely ineffective at: detecting previously unknown threats, threats disguised by the use of evasion techniques, variants of known threats. If an attacker modified the previous malware to attach freepics2.exe, a signature looking for freepics.exe would not match it. 39/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Anomaly-Based Detection Characteristics of Anomaly-based detection is the process of comparing definitions of what activity is considered normal against observed events to identify significant deviations. An IDS using anomaly-based detection has profiles that represent the normal behavior. The profiles are developed by monitoring the characteristics of typical activity over a period of time. 40/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Anomaly-Based Detection Characteristics of The IDS uses statistical methods to compare the characteristics of current activity to thresholds related to a profile. They can be very effective at detecting previously unknown threats. An initial profile is generated over a period of time (training). Ex.: user Joe only logs in from host ABC, usually at night. 41/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Specification-Based Detection Characteristics of Core idea: codify a specification of what a sites policy permits; look for patterns of activity that deviate. Example: user Joe is only allowed to log in from host ABC. Pro: Con: potentially detects wide range of attacks, including novel; framework can accommodate signatures, anomalies; directly supports implementing a site s policy. specifications require significant development & maintenance; hard to construct attack libraries. 42/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Stateful Protocol Analysis Characteristics of Stateful protocol analysis is the process of comparing predetermined profiles of generally accepted definitions of benign protocol activity for each protocol state against observed events to identify deviations. Relies on vendor-developed universal profiles that specify how particular protocols should and should not be used. The stateful in stateful protocol analysis means that the IDS is capable of understanding and tracking the state of network, transport, and application protocols that have a notion of state. 43/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Sensor or Agent Introduction Characteristics of Sensors and agents monitor and analyze activities. The term sensor is typically used for that monitor networks, including network-based, wireless, and network behavior analysis technologies. The term agent is typically used for host-based IDS technologies 44/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Management Server Introduction Characteristics of A management server is a centralized device that receives information from the sensors or agents and manages them. Sometimes perform analysis on the events provided by sensors/agents to identify events that the individual sensors or agents cannot: matching event information from multiple sensors/agents, such as finding events triggered by the same IP, is known as correlation. 45/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Database Server and Console Characteristics of A database server is a repository for event information recorded by sensors, agents, and/or management servers. A console is a program that provides an interface for the IDS s users and administrators. 46/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
False Positives/Negatives Characteristics of All suffer from the twin problems of false positives and false negatives: not minor, but an Achilles heel. False positives occur when the IDS erroneously detects a problem with benign traffic. False negatives occur when unwanted traffic is undetected. Both create problems for security administrators and may require that the system be calibrated. False positives can burden administrator with cumbersome amounts of data. False negatives do not afford administrators an opportunity to review the data. 47/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Base-rate Fallacy Introduction Characteristics of Suppose that your doctor performs a test that is 99% accurate: when the test was administered to a test population all of whom had the disease, 99% of the tests indicated disease; when the test population was known to be 100% free of the disease, 99% of the test results were negative. Upon visiting your doctor to learn the results he has good and bad news: the bad news is that you tested positive for the disease; the good news is that out of the entire population the rate of incidence is only 1/10.000 (only 1 in 10.000 people have this ailment). What is the probability of you having the disease? 48/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Base-rate Fallacy Introduction Characteristics of If S denotes Sick and S denotes healthy and P denotes a positive test results and P a negative test results, we have P(P S) = 0, 99, P( P S) = 0, 99, P(S) = 1/10.000 P(S P) =? Since P(A B) = then P(S P) = P(A) P(B A) P ni=1 P(A i ) P(B A i ) P(S) P(P S) P(S) P(P S)+P( S) P(P S) and P(P S) = 1 P( P S) = 1% and P( S) = 1 P(S) then P(S P) = 1/10.000 0,99 = 0, 00980... 1% 1/10.000 0,99+(1 1/10.000) 0,01 49/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
The Problem of Evasion Characteristics of Consider the following attack URL: http://./c/winnt/system32/cmd.exe?/c+dir Easy enough to scan for cmd.exe, right? What if you consider: http://./c/winnt/system32/cm%64.exe?/c+dir Okay, we need to handle % escapes. What about: http://./c/winnt/system32/cm%25%54%52.exe?/c+dir 50/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
The Problem of Evasion Characteristics of Consider passive measurement: scanning traffic for a particular string ( USER root ) Easiest: scan for the text in each packet: not good: text might be split across multiple packets. Okay, remember text from previous packet: not good: out-of-order delivery. Okay, fully reassemble byte stream: costs state and still evadable. 51/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Characteristics of Evading Detection Via Ambiguous TCP Retransmission 52/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
List of Host Introduction Characteristics of AIDE-Advanced Intrusion Detection Environment CSP Alert-Plus eeye Retina eeye SecureIIS Web Server Protection GFI EventsManager Hewlett Packard-Unix (HP-UX) 11i Host Intrusion Detection System (HIDS) IBM RealSecure Server Sensor integrit Lumension Application Control McAfee Host Intrusion Prevention NetIQ Security Manager iseries Osiris OSSEC HIDS PivX preempt Samhain Tripwire Enterprise Tripwire for Servers 53/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
List of Network Introduction Characteristics of Arbor Networks Peakflow ArcSight Bro Check Point IPS Software Blade Check Point VPN-1 Power Check Point VPN-1 Power VSX Cisco ASA 5500 Series IPS Edition Cisco Catalyst 6500 Series Intrusion Detection System Services Module (IDSM-2) Cisco Guard XT Cisco Intrusion Detection System Appliance IDS-4200 Cisco IOS IPS Cisco Security Agent Enterasys Dragon Network Defense ForeScout CounterAct Edge IBM Proventia SiteProtector Imperva SecureSphere Intrusion SecureNet IDS/IPS ipolicy Intrusion Prevention Firewall Family 54/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
List of Network (cont.) Characteristics of Juniper Networks IDP Lancope StealthWatch McAfee IntruShield Network IPS Appliances NIKSUN NetDetector NitroSecurity NitroGuard Intrusion Prevention System PreludeIDS Technologies Q1 Labs QRadar Radware DefensePro SecurityMetrics Appliance Snort snort_inline Sourcefire 3D Sensor Sourcefire Intrusion Prevention System StillSecure Strata Guard Symantec Critical System Protection TippingPoint Intrusion Prevention System Top Layer IPS Webscreen 55/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
List of Wireless Introduction Characteristics of AirMagnet AirSnare AirTight Networks SpectraGuard Enterprise Aruba Wireless Intrusion Detection & Prevention (WIDP) Kismet Motorola AirDefense Enterprise Newbury Networks WiFi Watchdog 56/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Standard Introduction Characteristics of The Internet Engineering Task Force (IETF) has a working group to develop a common format for IDS alerts: the design involves sending XML based alerts over an HTTP like communications format; a lot of attention has been paid to the needs of IDS analysis, and to making the protocol work through firewalls. http://www.ietf.org/old/2009/ids.by.wg/idwg.html Intrusion Detection Exchange Format Working Group (IDWG) Intrusion Detection Message Exchange Format (IDMEF) Intrusion Detection Exchange Protocol (IDXP) 57/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Static Analysis Run-Time Support Static Analysis An example of a HIDS based on the expected behavior of the program (static analysis) and virtualization (run-time monitoring): Process self: valid sequences of system calls (traces) and invariants for the process executing the program to be protected: traces are statically deduced from the program. invariant on program variables at system call invocations are inferred from the semantics of the program. 58/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Static Analysis Run-Time Support Grammar of System Call Sequences A tool computes a context-free grammar that models the legal system call traces that the process can issue: the tool automatically generates the grammar by linearly scanning each function defined in the program s source code. At run-time, a sequence of system calls is valid only if it is a prefix of at least one string generated by the grammar. 59/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Static Analysis Run-Time Support Run-Time Architecture Exploiting virtual machines (VMs): transparency; visibility; robustness. 60/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Static Analysis Run-Time Support Run-Time Architecture The Monitored VM executes the process to be monitored; The Introspection VM monitors the protected process through introspection: stream-oriented parser; assertion checker; introspection library. 61/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Static Analysis Run-Time Support Run-Time Checks Each time the monitored process invokes a system call, the Monitored VM is suspended. The Introspection VM checks that: 1 the system call trace is coherent with the grammar; 2 the assertions paired with the system call are verified. If the trace is not coherent with the grammar, or an assertion is false attack. 62/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Static Analysis Run-Time Support Example of Invariant Evaluation 63/64 Daniele Sgandurra A Survey of Intrusion Detection Systems
Static Analysis Run-Time Support Questions? 64/64 Daniele Sgandurra A Survey of Intrusion Detection Systems