Incident Response Using Splunk for State and Local Governments

Transcription

1 Copyright 2013 Splunk Inc. Incident Response Using Splunk for State and Local Governments Bert Hayes Solu=ons Engineer #splunkconf

2 Legal No=ces During the course of this presenta=on, we may make forward- looking statements regarding future events or the expected performance of the company. We cau=on you that such statements reflect our current expecta=ons and es=mates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward- looking statements, please review our filings with the SEC. The forward- looking statements made in this presenta=on are being made as of the =me and date of its live presenta=on. If reviewed aver its live presenta=on, this presenta=on may not contain current or accurate informa=on. We do not assume any obliga=on to update any forward- looking statements we may make. In addi=on, any informa=on about our roadmap outlines our general product direc=on and is subject to change at any =me without no=ce. It is for informa=onal purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obliga=on either to develop the features or func=onality described or to include any such feature or func=onality in a future release. Splunk, Splunk>, Splunk Storm, Listen to Your Data, SPL and The Engine for Machine Data are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respeccve owners Splunk Inc. All rights reserved. 2

3 Introduc=on

4 About Bert! 15 years experience in Systems Administra=on & Network Security! 10 years experience in IT security for Texas state government Texas Educa=on Agency University of Texas at Aus=n Department of Informa=on Resources Texas Higher Educa=on Coordina=ng Board! 5 years experience using Splunk for IT security 4

5 Agenda! Incident Handling at State.gov! Must- Have Data Sources for Basic Incident Handling! Post- Incident Data Collec=on! Crea=ng a Timeline of File System Meta Data! Did 10e9_SSNs.csv Leak Out?! Sharing with Others 5

6 Incident Handling at State.gov

7 Incident Handling at State.gov! Increased likelihood of storing sensi=ve data! Typically limited resources! Legisla=ve mandates to report security breaches! Internal agency to agency or state to state informa=on sharing 7

8 We discovered, inves=gated and closed an open invita=on to aeackers in less than a few hours. Without Splunk Enterprise, we would not have known the device was compromised for weeks, at best. Kim Munoz IT Manager Nevada DOT 8

9 Splunk has been a tremendous help to the Informa=on Security Office at ERS. We now have the visibility to research malware from the actual point of entry and we can actually see when the user clicks on the malicious link. It s been a great asset in incident response. Victoriano Casas III, MPA CISSP GSLC GSEC Informa=on Security Officer Employee Re=rement System of Texas 9

10 Covering the Bases: Pre- Incident Data Collec=on

11 Pre- Incident Data Collec=on Collect This Now and Always! Firewall logs! HTTP proxy logs! DNS server logs! DHCP server logs! Network flow data! Extra credit for IDS/IPS logs 11

12 Firewall Logs You Keep on Knockin! Sudden increase in outbound DENY events! Sudden increase in inbound DENY events! Unusual des=na=on IPs! Unusual des=na=on ports! Off- Site DNS? 12

13 HTTP Traffic All Your Webs Are Belong to Us! Malicious code used to use Internet Relay Chat (IRC) for Command and Control (C&C) traffic! Modern malware increasingly using HTTP for C&C traffic GET hep:// Data exfiltra=on over HTTP POST hep:// 13

14 14

15 DNS Sever Logs Places Named AVer Numbers! If C&C is not over HTTP, web proxy will not log or block! irc.evil.br! ssl.evil.br! DNS itself as a C&C channel 15

16 DHCP Sever Logs > CA:FE:DE:AD:BE:EF! IP addresses are transient Track an incident by MAC address! Track host s presence on the LAN Disable switch ports! OVen username is presented and requested as hostname! Make sure you re tracking the correct IP based reports over =me 16

17 NetFlow When I Get My Flow, I m Dr. On The Go! Will record Command & Control meta data, regardless of protocol! Will record bytes of data transferred! Use to determine how much data was transferred when, to whom! Correlate against other data sources to determine incident severity 17

18 Post Incident Data Collec=on

19 Data You ll Want AVer an Incident Diving Deeper! Packet capture! RAM dump! Hard drive image! Server logs 19

20 Time Lining File System Meta Data

21 Crea=ng a Timeline of File System Meta Data log2%meline hep://kleinco.com.au/thoughts- events/item/forensic- =meline- splunking 21

22 The Tools The Sleuth Kit Open source digital forensic tools hep:// Wrieen by Brian Carrier hep:// evidence.org/fsfa/ Command line tools hep://wiki.sleuthkit.org/index.php? =tle=tsk_tool_overview Screenshot here Timescanner Front end for log2=meline Wrieen by Kris=nn Gudjonsson 22

23 Crea=ng the Super Timeline 23

24 props.conf 24

25 transforms.conf 25

26 26

27 Did 10e9_SSNs.csv Leak Out?

28 Sensi=ve Data is in Known Loca=on! Searching for post- incident file access is now trivial 28

29 Sensi=ve Data is in Unknown Loca=on! Locate it!! Use SENF! The Sensi=ve Number Finder! heps://senf.security.utexas.edu! It s free! 29

30 30

31 How Severe is the Incident?! Was sensi=ve data accessed?! Correlate file access =mes with network flow, other logs! Evidence that aeack has spread?! Correlate server logs with network flow! No sensi=ve data access? No spread of aeack? BORING REPORT! 31

32 Sharing with Others

33 Sharing with Others! Many state and local governments have Informa=on Sharing and Analysis Centers (ISAC)! Inter- agency informa=on sharing is common! Collect key elements of forensic inves=ga=on into new index Use the collect command! Export key elements of forensic inves=ga=on as raw data 33

34 34

35 35

36 Summary Splunk Enterprise is the Best Tool You Already Have for Incident Handling Begin with established logging and indexing data from network devices and network services Add system forensic =meline and op=onally packet capture analysis post- incident Resul=ng data set can show if and when sensi=ve data was accessed correlated with network ac=vity to determine if data was likely to leak Screenshot here 36

37 Key Takeaway Splunk Enterprise Can Keep Your Name Out of the Papers Incident handling anywhere Incident handling: state & local gov Need to log and monitor network devices and network services Need to determine root cause of incident Need to determine extent of incident Higher likelihood of sensi=ve personal informa=on Need to determine how incident is reported Public en==es leaking sensi=ve data makes BIG HEADLINES More informa=on sharing within and between agencies 37

38 Demo 38

39 Q & A 39

40 Next Steps 1 Download the.conf2013 Mobile App If not iphone, ipad or Android, use the Web App 2 Take the survey & WIN A PASS FOR.CONF2014 Or one of these bags! 3 Check other Security sessions All PPTs are on the Mobile App Recordings will be available aver.conf

41 THANK YOU