Application-Integrated Data Collection for Security Monitoring

Transcription

1 Application-Integrated Data Collection for Security Monitoring Magnus Almgren and Ulf Lindqvist System Design Lab, SRI International The work described here was funded by DARPA under contract number F C-0149 and F C The views herein are those of the authors and do not necessarily reflect the views of the supporting agency.

2 Introduction How many network IDSs can detect this? How many network IDSs can detect this? 2

3 Outline Part One Traditional NIDS and HIDS Host-located data collection Application-integrated module Proof-of-concept implementation Implementation architecture and data flow Part Two 3

4 Traditional Network IDS Problem areas: Encrypted traffic Evasion tricks Network speed Session/transaction reconstruction and statefulness Timely preemption difficult Advantages: Passive, non-invasive Hidden Can monitor multiple hosts from one location 4

5 Traditional Host IDS Data sources Audit data useful, but limited insight into application data Application/system log files limited content, disk space management Usually, data produced after the fact Blind to most network-level attacks 5

6 Host-located Real-time Event Data Collection on Multiple Levels Application OS Network Application-integrated IDS IDS analyzing audit trail (system calls) Network IDS (located on network or host) Security violations manifest themselves differently on different levels The data sources are complementary 6

7 Application-integrated Module Advantages Unencrypted information Independent of network speed Detailed information available True session and transaction decoding and reconstruction Opportunity for preemptive capability Disadvantages Tailored for a specific application Invasive: Could impact application performance and stability Complements data collection on other levels 7

8 Proof-of-concept: One type of application Web server Web is a popular, ubiquitous service Allowed through most firewalls Existing EMERALD analysis engine for HTTP Many Web servers allow custom extensions Apache Web server Apache ~60% of market according to Netcraft Open-source, well-documented module interface 8

9 Implementation Architecture and Data Flow Host Data collection module expert-http Web server EMERALD libraries Transaction records as EMERALD messages Alerts 1. The Web server receives a request 2. Module produces transaction data 3. Message is sent to expert-http 4. expert-http performs analysis 9

10 Outline Part Two Inside the Apache Server Performance evaluation Evasion techniques Problems with this approach Future work Related approaches Conclusions 10

11 Inside the Apache Server Apache uses a request loop Hooks are available in all phases of the loop Our module is hooked to the logging phase Currently no feedback from analysis unit Passive data collection [Stein 1999] 11

12 Performance Experiment Setup Goal: Measure impact on user experience Used WebLoad from RadView Software Set up to request a single URL repeatedly Measured transaction round-trip time Each run was 60 minutes with 10 virtual clients on a single physical host Static page: text and one image Dynamic page: CGI program 12

13 Performance Results: Static Page 50 KB text + 12 KB JPEG Round-trip (s) No IDS W/ IDS Impact median % average % std dev

14 Performance Results: Dynamic Page Execution of a CGI program Round-trip (s) No IDS W/ IDS Impact median % average % std dev

15 Evasion Techniques Using lower protocol levels [Ptacek and Newsham] Crafting ambiguous HTTP request 1. GET Tab /cgi-bin/phf 2. GET /%00cgi-bin/phf 3. GET / HTTP 1.1 Host: victim Content-Length: 3 123GET /cgi-bin/phf The evasion techniques work because Web servers and NIDS decode them differently 15

16 Problems 1) Invasive Because the module must run within the server application, it could impact stability Testing is difficult Server applications typically do not run from batch input Forced to use scripts rather than data files 2) Application-specific A module must be written specifically for every application you want to monitor Every application has its own interface (or none) for customized module But: For many services, only a few major brands 16

17 Future Work Other Web server brands iplanet prototype ready Other services FTP SMTP (sendmail) Remote access (telnet, rlogin) Databases Great potential for improved analysis In many cases, knows how the request was serviced (e g document or CGI program) Could detect evasion attempts Knows the exact local filename request refers to Check expected control flow of program 17

18 Preemption: Two-tiered approach Problem: Complete analysis in module performance hit Analysis on separate host excludes preemption Solution: Two-tier analysis Module contains simple (fast) analysis engine: hash of suspicious source/cgi program expert-http performs complete (slower) analysis and keeps a global state expert updates the module s (simple) knowledge Three options for server module: serve request, deny request, or wait for further analysis 18

19 Per-request Granularity Single requests can be stopped or delayed (awaiting analysis) Select requests for analysis depending on the type (static, dynamic, directory) Compare with a packet-filtering firewall IP address/port granularity Can only block, not delay 19

20 Related Approaches mod_id by Burak Dayioglu ( Performs simple CGI name matching inside module (development discontinued) TripWire for Web pages Limited to stopping altered content from being served (MD5 checksums) Interfacing Trusted Applications with Intrusion Detection Systems Marc Welz and Andrew Hutchison, RAID

21 Conclusions Application-integrated data collection for IDS complements data from other levels and locations Addresses the three most severe problems NIDS are currently facing: Encryption Evasion Network speed Prototype integrated with Apache and with EMERALD infrastructure and analysis engine 21