Security Monitoring and Architectures for Security Logging Christer Andersson 15 December 2009 1 December 2008 Introduction to myself Christer Andersson Information Security Consultant at Combitech Karlstad office, 2 employees 2007 - now PhD Engineering in Computer Security Privacy and Security (PRISEC) group at KaU Anonymous communication in mobile networks 2002 2007 2 December 2008 Combitech 7 Service Areas Information security (IS) Logistics Mechanical engineering System development System integration System security Customer adapted solutions President Combitech: Marie Bredäng 3 December 2008 1
Information Security (IS) More than 100 employees IS Manager Jerker Löf Divided into three units ISA Information Security Architectures ISS Information Security Solutions IST Information Security Technologies Clients Swedish defence (~55%) FMV largest customer Swedish institutions (~20%) Skatteverket, Banverket, Rikspolisstyrelsen, KMB, etc. Private companies (~25%) Telecom companies, private defence industry (SAAB, Kockum, etc.) 4 December 2008 Information Security Architectures (ISA) 14 employees distributed in 6 cities Växjö, Karlstad, Linköping, Stockholm, Kristianstad, Enköping ISA Manager Lena Johansson Two focus areas 2008 for ISA 1. Monitored security and logging 2. Service-based security architectures Intra-IS cooperation with other units 5 December 2008 Security Monitoring and Logging Can both seen as one combined area and two distinct areas: Security Information Management (SIM) Refers to the collection of data (typically log files; e.g. event logs) into a central repository for trend analysis SIM is mainly focused on the discovery of bad behavior Security Event Management (SEM) A SEM stores large amount of logs in disk devices for historical storage purposes It typically offers integrity and confidentiality A SEM is slow but can typically store years of logs Sometimes combined into a SEIM Note that terminology confusion often occurs Often a SIM claims to be SEM by implementing a minimal set of SEM functionalities, and vice versa 6 December 2008 2
Motivation There is a lot of traffic in corporate nets Between clients and business applications To / from internal servers Between clients and Internet Problems How to find intrusions from Internet How to find misbehaving users / compromised machines in the internal network Commercial value When an incident happen we want a quick reaction time and traceability Additional motivation: compliance Bonus: will detect badly configured network applications 7 December 2008 A possible solution Security monitoring and logging architectures (SIM and SEM) SIM and SEM is becoming increasingly more important and prioritized among companies. It is essential for mapping vulnerabilities and threats against the information security in an organization Benefits Keep control over business application and IT system logs System-wide picture of the information security status React quickly on incidents (real-time analysis or analysis on regular basis) Provides rich material for efficient investigation of security incidents 8 December 2008 Phases in Security Monitoring and Logging ANALYZE Requirement analysis IMPLEMENT & DRIVE Proof of concept deployment Acquisition/development support Usage guidelines Full scale installation FOLLOW UP Monitoring and analysis Training Maintenance My involvement in projects so far includes: Networked-based intrusion detection and security monitoring using OSSIM (Svenska Räddningsverket) Assessment of current logging situation and writing guidelines and directives (Arbetsförmedlingen, Ludvika kommun) Certification in open-source based SIM tool OSSIM MONITORED SECURITY 9 December 2008 3
Typical Monitoring Architecture One or several servers One or several databases Several sensors using agent-based or agent-less collection Several log sources Analysis Server(s) for correlation and normalization Sensor(s) Console with GUI Real-time Forensic database Long-term Forensic database agent agent agent agent Syslog SMTP SNMP Cisco IDS CheckPoint Windows XP 10 Firewall-1 Nessus December 2008 Example of SIM and SEM products ArcSight RSA envision IBM Consul InSight Symantec Enterprise Security Manager CiscoWorks SIMS Network Intelligence NetForensics Intellitactics NetIQ IBM (Tivoli) Netcool/Neusecure Novell Sentinel OSSIM (open source) 11 December 2008 Example SIM: OSSIM 12 December 2008 4
Sensors/probes Sensors/probes collect log data from different log sources and/or generates security events based on monitored logs The log data (or security events) are sent to the central monitoring server, possibly after being normalized to a standard format Sensor components are mostly passive (do not generate traffic) Collection can both be agent-based and agent-less 13 December 2008 Sensors components Network-based Intrusion Detection Systems (NIDS) Monitors network traffic in order to detect intrusions Often matches network packets against known signatures Can also detect new attacks by looking for suspicious behavior Often deployed outside and inside firewall, on backbones and subnets, etc. Host-based Intrusion Detection Systems (HIDS) Detects intrusions on important hosts (servers, routers, switches, clients, etc.) Performs integrity checking on files, log monitoring of different logs, etc. Log collector components Collect/monitor logs from hosts in the network Examples of log sources: firewall logs, OS logs (syslog / windows event logs), logs from business applications, etc. 14 December 2008 Logging component and log sources A logging component parses log data from different log sources Extracts fields like date, IP address, free text information, log generator, etc. One example method of parsing is regular expressions Common log sources Windows event logs Syslog in Linux / Unix SNMP (Simple Network Management Protocol) Traps Business application logs Firewall logs Logs may be sent to server both in original and normalized format Collecting all all logs from all all monitored components to to a central server requires a lot lot of of bandwidth and and performance 15 December 2008 5
Example NIDS: Open-source based NIDS developed by Sourcefire Statefull and stateless analysis Stateless analysis is based on network traffic signatures Stateless analysis discovers known attacks Signatures are grouped and collected in rule sets Common rule sets are Sourcefire s official rules and the Bleeding Edge rules Statefull analysis is done by different preprocessors in Snort Statefull analysis may discover unknown attacks (anomaly detection) Example preprocessors: spade, http_inspect, stream, portscan, etc. False positives are common and serious problem The NIDS must be configured according to the monitored network and the signature rule sets must be fine tuned 16 December 2008 The Snort Process libpcap Packet Decoder Works through the TCP/IP stack Preprocessor Examines packets using often using statefull inspection Detection Engine Matches against the rule set with signatures Output plugin 17 December 2008 Example Snort Rule Payload Plain Display Download of Payload Download in pcap format length = 376 000 : 04 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01... 010 : 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01... 020 : 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01... 030 : 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01... 040 : 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01... 050 : 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01... 060 : 01 DC C9 B0 42 EB 0E 01 01 01 01 01 01 01 70 AE...B...p. 070 : 42 01 70 AE 42 90 90 90 90 90 90 90 90 68 DC C9 B.p.B...h.. 080 : B0 42 B8 01 01 01 01 31 C9 B1 18 50 E2 FD 35 01.B...1...P..5. 090 : 01 01 05 50 89 E5 51 68 2E 64 6C 6C 68 65 6C 33...P..Qh.dllhel3 0a0 : 32 68 6B 65 72 6E 51 68 6F 75 6E 74 68 69 63 6B 2hkernQhounthick 0b0 : 43 68 47 65 74 54 66 B9 6C 6C 51 68 33 32 2E 64 ChGetTf.llQh32.d 0c0 : 68 77 73 32 5F 66 B9 65 74 51 68 73 6F 63 6B 66 hws2_f.etqhsockf 0d0 : B9 74 6F 51 68 73 65 6E 64 BE 18 10 AE 42 8D 45.toQhsend...B.E 0e0 : D4 50 FF 16 50 8D 45 E0 50 8D 45 F0 50 FF 16 50.P..P.E.P.E.P..P 0f0 : BE 10 10 AE 42 8B 1E 8B 03 3D 55 8B EC 51 74 05...B...=U..Qt. 100 : BE 1C 10 AE 42 FF 16 FF D0 31 C9 51 51 50 81 F1...B...1.QQP.. 110 : 03 01 04 9B 81 F1 01 01 01 01 51 8D 45 CC 50 8B...Q.E.P. 120 : 45 C0 50 FF 16 6A 11 6A 02 6A 02 FF D0 50 8D 45 E.P..j.j.j...P.E 130 : C4 50 8B 45 C0 50 FF 16 89 C6 09 DB 81 F3 3C 61.P.E.P...<a 140 : D9 FF 8B 45 B4 8D 0C 40 8D 14 88 C1 E2 04 01 C2...E...@... 150 : C1 E2 08 29 C2 8D 04 90 01 D8 89 45 B4 6A 10 8D...)...E.j.. 160 : 45 B0 50 31 C9 51 66 81 F1 78 01 51 8D 45 03 50 E.P1.Qf..x.Q.E.P 170 : 8B 45 AC 50 FF D6 EB CA.E.P... Snort rule syntax: Snort rule syntax: alert udp $EXTERNAL_NET any -> $HOME_NET 1434 (msg:"sql Wormpropagation attempt"; flow:to_server; content:" 04 "; alert udp $EXTERNAL_NET any -> $HOME_NET 1434 (msg:"sql Wormpropagation attempt"; flow:to_server; content:" 04 "; depth:1; content:" 81 F1 03 01 04 9B 81 F1 01 "; content:"sock"; content:"send"; ; classtype:misc-attack; sid:2003; rev:12;) depth:1; content:" 81 F1 03 01 04 9B 81 F1 01 "; content:"sock"; content:"send"; ; classtype:misc-attack; sid:2003; rev:12;) 18 December 2008 6
Example HIDS: Snare and OSSEC A Snare agent monitors windows event logs from Security, Application and System logs in Windows Can be configured to send collected logs to a central log server For example a syslog server on a Linux host or a Snare server This central log server can in turn be monitored by a SIM/SEM server Advanced HIDS for Linux / Unix / Windows Central OSSEC server/distributed OSSEC agents hierarchy Log analysis, integrity checking, Windows registry monitoring, rootkit detection, real-time alerting and active response, etc. 19 December 2008 Example SIM: OSSIM 20 December 2008 Server and Central Repository The server collect log data and security events from the sensors The events are being correlated and prioritized Events may generate alarms / incidents Alarms / incidents may trigger automatic actions The server often includes active components that generate traffic Example: vulnerability scanning (Nessus), port scanning (Nmap), etc. 21 December 2008 7
Correlation Problems Multitudes of events/alarms from different sensors Multitudes of false positives Possibility of false negatives Solution Cross correlate events/alarms in a correlation engine Detect patterns of events/alarms A high degree of fulfillment of specific pattern the greater the reliability of a real attack Can decrease false positives and false negatives significantly 22 December 2008 Example of a Snort pattern 23 December 2008 Correlation directives in OSSIM 24 December 2008 8
Priorization Problem How should a security analyst know which security events he/she should invest resources to investigate? Solution A system for prioritizing security event must be included in the SIM/SEM In Snort events are prioritized between priority value 1 3 In OSSIM events are prioritized between risk value 1 10 Priorities are calculated based on parameters such as (i) probability that an attack has occurred, (ii) the resulting level of severity if the attack would succeed, and (iii) value of the attacked host In OSSIM, the RISK = (RELAIABILITY * PRIORITY * ASSET) / 25 A security event of RISK >= 1 generates a security alarm 25 December 2008 Priorization 26 December 2008 Example SIM: OSSIM 27 December 2008 9
Storage databases A hierarchy of databases should be used for archiving the log data and the security events / alarms Databases for both (i) short term (fast) real-time analysis and (ii) longterm (slow) forensic analysis should preferably be used Storage medium (database, file system, etc.) is chosen according to requirements (performance, reliability, etc.) Fast online search queries on the last days of collected security events and log data Front-end Real-time Forensic database Back-end Long-term Forensic database Slower offline search queries on the complete databse of collected security events and log data Integrity of data is important 28 December 2008 Example SIM: OSSIM 29 December 2008 Console Web Interface Different usersbelonging to different roles administrate and use the SIM from (often) a web interface Typically, the GUI can be configured so that the boss, the analyst, the administrators, etc., have different (sometimes overlapping) views of the GUI 30 December 2008 10
Practical Demonstration 31 December 2008 11