Intrusion Detection Systems

Intrusion Detection Systems Sebastian Abt Selected Topics in IT-Security Lecture 05 Summer term 2012

Motivation STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 2

Motivation» Why do we need intrusion detection systems? Computer systems commonly contain sensitive data Vulnerabilites inherent to computer systems or software Flaws in system design or software development Misconfiguration of a system or software No or inappropriate operational processes Attackers try to gain access to systems by exploiting vulnerabilities Insider vs. outsider threat Industry espionage Internet underground economy STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 3

Motivation» Intrusion detection systems in real life? Car alarms House alarms Fire detectors Earthquake detectors Tsunami warning systems STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 4

Learning Objectives» Understanding necessity of IDS» Understanding principles of IDS» Understand capability of IDS» Be able to classify IDS» Be able to evaluate IDS performance» Get an impression of open source Snort IDS STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 5

Definitions» A vulnerability is an exlpoitable flaw in a system or software» An attacker is a person seeking unauthorized access to systems or data» An attack is an attempt to violate a security goal» An intrusion is a successful attack» An exploit is a software written for exploiting a specific vulnerability» Malware is malicious software used during an attack STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 6

Definitions» Intrusion detection (ID) is the process of monitoring the events occurring in a computer system or network and analyzing them for signs of possible incidents, which are violations or imminent threats of violation of computer security policies, acceptable use policies, or standard security practices» An intrusion detection system (IDS) is a software that automates the intrusion detection process» An intrusion prevention system (IPS) is a software that has all the capabilities of an intrusion detection system and can also attempt to stop possible incidents STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 7 Source: NIST Special Publication 800-94, Guide to Intrusion Detection and Prevention Systems (IDPS)

IDS vs. IPS» IPS can actively stop malicious activities Enables pro-active mitigation of attacks IPS has to be deployed in-band with data stream» Possible mitigation measures? Terminate malicious network connections Terminate malicious processes» Further on, we will not distinguish between IDS and IPS STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 8

IDS vs. Extrusion Detection System» IDS usually monitor data inbound to computer systems Detecting attacks targeting specific systems» An extrusion detection systems (EDS) monitors data outbound to computer systems Protecting others from a specific system Usually used to detect illegal activities originating from computer systems Data leakage (e.g. industry espionage) Misuse of corporate systems and networks (e.g. file sharing, watching policy-defined illegal web sites) STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 9

Architecture and Components of IDS Generalized architecture 0 1 0 0 0 1 1 0 Sensor or agent 1 0 0 1 1 1 0 1 Pre-proc. data Event database Get/update event info Detection engine Get/update knowledge Domain knowledge Get event info Alert Update knowledge Console Command View STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 10

Architecture and Components of IDS Components» Sensor or agent Sensors and agents capture and pre-process activity» Detection engine A component that receives information from sensors or agents and anylses them, utilizing domain knowledge» Event database A repository for event information recorded by sensors and agents or detection engines» Console A program that provides an interface for the IDS users and administrators STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 11

Architecture and Components of IDS Generic processing pipeline» Monitoring of data» Analysis of data» Detection of attacks» Response to attacks IDS Data Monitoring Analysis Detec4on Response Ac4on STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 12

Classification of IDS» IDS are usually classified according to their locality of deplyoment and their incorporated ID technique Locality of deployment Host-based IDS Network-based IDS Application-based IDS Intrusion detection technique Misuse detection Anomaly detection Specification-based detection» As usual, hybrid solutions are possible... STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 13

Locality of Deployment Host-based IDS (1)» (Usually) Software deployed on a computer system» Monitors host activity Sequence of system calls Network sockets Library loading» Pros Most complete view on data Fine-grained analysis of host activity» Cons Attackable from host (e.g. rootkit) Attacker has already reached host system STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 14

routers, switches). 7.1.2 Network Architectures Locality of Deployment The network architecture for host-based IDPS deployments is typically very simple. Because the agents are deployed to existing hosts on the organization s networks, the components usually communicate over those networks instead of using a separate management network. Most products encrypt their communications, preventing eavesdroppers from accessing sensitive information. Appliance-based agents are typically deployed inline immediately in front of the hosts that they are protecting. Figure 7-1 shows an example of a host-based IDPS deployment architecture. Host-based IDS (2) STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 15 Figure 7-1. Host-Based IDPS Agent Deployment Architecture Example Source: NIST Special Publication 800-94, Guide to Intrusion Detection and Prevention Systems (IDPS)

Locality of Deployment Network-based IDS (1)» Either software deployed on a router or firewall, or specialized hardware/software attached to network links» Monitors network traffic In-band: data streams are traversing IDS to reach destination Out-of-band: data streams are mirrored to IDS Active mirroring: SPAN port on network device Passive mirroring: network TAP STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 16

Locality of Deployment Network-based IDS (2)» Pros: Can monitor network traffic of many hosts Global view on activities Early stage detection» Cons: No state information from host Possibly limited by encryption High volume of network traffic STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 17

GUIDE TO INTRUSION DETECTION AND PREVENTION SYSTEMS (IDPS) Locality of Deployment Network-based IDS (3) GUIDE TO INTRUSION DETECTION AND PREVENTION SYSTEM (b) Out-of-Band deployment (a) In-Band deployment 4.3 Security Capabilities Figure 4-3. Passive Network-Based IDPS Sensor Architecture Example STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 18 Source: NIST Special Publication 800-94, Guide to Intrusion Detection and Prevention Systems (IDPS) Figure 4-2. Inline Network-Based IDPS Sensor Architecture Example Passive. A passive sensor is deployed so that it monitors a copy of the actual network traffic;

Locality of Deployment Application-based IDS» Software components or sandbox of specific application» Monitors data inside and state of application Data fed into application, e.g. SQL queries Log statements» Pros Application-specific view on data Possibility to effectively prevent exploitation of vulnerabilities» Cons Run-time overhead Development overhead STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 19

Intrusion Detection Technique Misuse detection» IDS uses attack signatures to perform intrusion detection» Signatures describe known attacks Hypothesis: attacks of the same kind show same patterns Need attack model detects known attacks only Signatures have to be generated timely and reliably» Well-known systems Snort Intrusion Detection System Bro Network Security Monitor STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 20

Intrusion Detection Technique Anomaly detection» IDS uses a model of normality to perform intrusion detection» Model of normality (profile) shall describe normal system behaviour Hypothesis: attacks cause deviation from profile Can possibly detect yet unknown attacks But: how to model normal system behaviour? Expert knowledge: manually construct profile AI/ML: learn profile from monitoring data Issues? STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 21

Intrusion Detection Technique Specification-based detection» IDS uses specification of normal activities to perform intrusion detection» Policies describe permitted events and activities Hypothesis: attacks differ from policy Explicit models of normality Permitted/certified software Matrix of network communication» Technique commonly deployed in network packet filters STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 22

Signature/Model Generation» IDS need signatures (misuse detection) or profiles (anomaly detection) to detect intrusions» How can signatures/models be generated? Identify characteristics of specific class of events Characteristics of attacks Characteristics of normal behaviour Characteristics are usually referred to as features Multiple (distinctive) features are combined to form a feature vector Process of generating feature vector is called feature extraction Feature vector has to be encoded appropriately STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 23

Lifecycle of a Vulnerability Zero-day attacks, anomaly detection Known attacks, misuse detection information is not public information is public discovery exploit disclosure patch available patch installed creation t creat t disco t explo t discl t patch t insta t t explo t patch t disco pre-disclosure risk post-disclosure risk t insta post-patch risk Source: Frei et al., Modelling the Security Ecosystem - The lifecycle of a vulnerability defined by distinctive The Dynamics events. of (In)Security The exact sequen een vulnerabilities. STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 24

Efficiency of IDS» IDS efficiency can be evaluated according to different criteria: Accuracy Proper detection of attacks Absence of false alarms Throughput Amount of data analysed per time unit Fault tolerance Availability of IDS Resistance to attacks targeting IDS Timeliness Time elapsed between intrusion and detection STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 25

Accuracy of IDS Definitions» Correlation of detection result and reality of event E High false negative rate leads to undetected attacks High false positive rate leads to annoyed operators Q: Which technique is prone to what errors? Detection result(e) Valid event Attack Reality(E) Valid event Attack True negative False positive False negative True positive STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 26

Accuracy of IDS Base-rate fallacy» Suppose 1% of network traffic belongs to malicious activities IDS accuracy is 90% Malicious traffic classified as intrusion with probability 0.9 Valid traffic classified as intrusion with probability 0.1» Probability that an alarm indicates an intrusion? Prob(Intrusion occured Alarm is raised) Conditional probability STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 27

Accuracy of IDS Conditional probability» Conditional probability Prob(X Y) is the probability that event X occurs if event Y is known to occur» Conditional probability of X given Y is defined as Prob(X Y )= Prob(X \ Y ) Prob(Y ) Q: Constraint on Prob(Y)?» Prob(X Y) denotes joint probability of X and Y Prob(X \ Y )=Prob(X Y )Prob(Y ) STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 28

» Can be used to compute conditional entropy Prob(X Y )= Prob(X \ Y ) Prob(Y ) = = Prob(X\Y ) Prob(X) Prob(Y ) Prob(X) Prob(Y X)Prob(X) Prob(Y ) Accuracy of IDS Bayes theorem Often, Prob(X Y) has to be computed and Prob(Y X) is known. STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 29

Accuracy of IDS Base-rate fallacy» Suppose 1% of network traffic belongs to malicious activities IDS accuracy is 90% Malicious traffic classified as intrusion with probability 0.9 Valid traffic classified as intrusion with probability 0.1» Probability that an alarm indicates an intrusion? Prob(Intrusion occured Alarm is raised) Conditional probability STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 30

Accuracy of IDS Base-rate fallacy» The scene... Let M denote malicious activity, Prob(M) = 0.01 A denote an alarm raised by IDS, Prob(A) =? M denote benign activity, Prob( M) = 1-Prob(M) = 0.99 A denote no alarm raised by IDS, Prob( A) =?» Probability of true positive Prob(A M) = 0.9 obtained when testing our IDS» Probability of false positive Prob(A M) = 0.1 obtained when testing our IDS STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 31

» Probability of true negative Prob( A M) = 1 Prob(A M) = 0.9» Probability of false negative Prob( A M) = 1 Prob(A M) = 0.1» Probability that an alarm indicates an intrusion Bayesian detection rate Accuracy of IDS Base-rate fallacy Prob(A M)Prob(M) Prob(M A) = Prob(A M)Prob(M)+Prob(A M)Prob( M) 0.9 0.01 0.9 0.01 = = 0.9 0.01 + 0.1 0.99 0.108 =0.083 STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 32

» Probability that an alarm indicates an intrusion Prob(M A) = 0.083 8.3% Probability that an alarm indicates no intrusion Prob( M A) = 1-Prob(M A) = 0.916 91.6%» False alarm rate is approximately 92%! 92 out of 100 alarms do not indicate intrusions Accuracy of IDS Base-rate fallacy» Effect due to low base-rate of malicious activity Malicious activity is rare, compared to total activity See Axelsson, 1999, The Base-Rate Fallacy and its Implications for the Difficulty of Intrusion Detection for further details STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 33

Accuracy of IDS Bayesian detection rate vs. base-rate STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 34

Visualisation of IDS performance» Receiver operating characteristic (ROC) curve 120 True positive rate 100 80 60 40 20 IDS1 IDS2 0 0 10 20 30 40 50 60 70 80 90 100 False positive rate STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 35

Visualisation of IDS performance» Receiver operating characteristic (ROC) curve 120 True positive rate 100 80 60 40 20 IDS1 IDS2 Ideal IDS 0 0 10 20 30 40 50 60 70 80 90 100 False positive rate STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 36

Snort IDS» Well-known and established network-based IDS Open source Signature-based http://www.snort.org/ Commercial support and signatures http://www.sourcefire.com/ Snort IDS Data Monitoring Analysis Detec4on Response Ac4on STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 37

Snort IDS Data» Create your own network tap as illustrated below» Connect a free computer to tap ports» Configuration example, assuming FreeBSD and Intel networking cards: ids# ids# ids# ids# ids# ifconfig bridge create! ifconfig bridge0 addm em0 addm em1 monitor up! ifconfig em0 up! ifconfig em1 up! tcpdump -ni bridge0! STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 38

Snort IDS Monitoring» Three operating modes Sniffer Packet logger Network-based IDS ids# snort -de -l /ids -c /etc/snort.conf -D! -d inspect payload data -e inspect ethernet layer -l log directory -c snort configuration file -D run as daemon STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 39

Snort IDS Analysis» Snort pre-processors Run before detection engine Own pre-processors can be written in C» Example pre-processors Frag3 IP defragmentation Stream5 TCP/UDP stream reassemly Protocol awareness HTTP Inspect HTTP decode and normalization {SMTP,POP,IMAP,Telnet,FTP,RPC} pre-processors STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 40

Snort IDS Detection (1)» Signature-based detection using rules» Rules are applied to pre-processed data» Rule format RULE_HEADER (RULE_OPTIONS) RULE_HEADER ACTION PROTO SRC_IP SRC_PORT DIR DST_IP DST_PORT ACTION = {alert,log,pass,activate,dynamic,drop,reject,sdrop} PROTO = {IP,TCP,UDP,ICMP} SRC_IP,DST_IP = IP addresses or ranges SRC_PORT,DST_PORT = Port numbers or ranges DIR = {->, <>} STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 41

Snort IDS Detection (2)» Signature-based detection using rules» Rules are applied to pre-processed data» Rule format RULE_HEADER (RULE_OPTIONS) RULE_OPTIONS Define detection parameters RULE_OPTIONS = {OPTION} OPTION = KEYWORD : ARGUMENTS ; Four categories of rule options general information about the rules itself payload payload specific options non-payload non-payload effects post-detection post-detection triggers STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 42

Snort IDS Detection (3)» Example rules x86 Linux shellcode sequence alert ip $EXTERNAL_NET any -> $HOME_NET any! (! msg:"shellcode Linux shellcode";! content:" 90 90 90 E8 C0 FF FF FF /bin/sh";! fast_pattern:only;! reference:arachnids,343;! classtype:shellcode-detect;! sid:652;! rev:11;! )! STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 43

Snort IDS Detection (4)» Example rules TT-bot trying to contact CnC server alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS! (! msg:"botnet-cnc TT-bot botnet contact to C&C server attempt";! flow:to_server,established;! content: TT-Bot";! nocase;! http_header;! pcre:"/^user-agent\x3a[^\r\n]*tt-bot/mi";! reference:url,anubis.iseclab.org/index.php? action=result&format=html&task_id=1494581651ca480640538ead93feabed2;! classtype:trojan-activity;! sid:16493;! rev:7;! )! STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 44

Snort IDS Response» Basically defined in rule s action alert Generate specific alert (Pager, SMS, email) log Log packet to Snort log file pass Ignore packet activate Alert and run dynamic rule dynamic Remain idle until actived drop Drop and log packet (only in inband/ips mode) reject Block and log packet, and send TCP RST or ICMP unreachable sdrop Drop, without logging (performance) STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 45

Summary» IDS can detect known and unknown attacks» IDS do not need to be placed inline with data stream» IDS can operate on different OSI layers» IDS show different types of errors» High accuracy and low false alarm rate are essential for operational gain of IDS» Regular update of models/signatures is elementary» Alarm correlation and filtering are important research topics STITS, Lecture 05: Intrusion Detec4on Systems 04.06.12 46