Building a Security Operations Center Lessons Learned active threat protection Yves Beretta October 8, 2013
INTRODUCTION 10/10/2013 Building a Security Operations Center Lessons Learned Slide 2
Yves Beretta Profile 20 years of IT experience IT Management Operations Management Network Security Built and managed a Security Operations Center for the past 7 years 10/10/2013 Building a Security Operations Center Lessons Learned Slide 3
Why a Security Operations Center? Multiplicity and Volume of Cyber Security Threats Social engineering Advanced Persistent Threats (APT) Internal threats Bring Your Own Device (BYOD) - Cloud security - HTML5 Botnets Source: Check Point /Forbes Source: Symantec 10/10/2013 Building a Security Operations Center Lessons Learned Slide 4
Security Operations Center 101 Security focused Operational Centralized 24x7 10/10/2013 Building a Security Operations Center Lessons Learned Slide 5
SOC: Expectations Watch and protect the infrastructure Monitor Network Traffic, watching for anomalies Protect Users Internal and External Threat detection Alert and Escalate Internal and External Threat mitigation.and also Monitor Users Systems Configuration Data Loss Prevention Forensics Analysis Threat modeling 10/10/2013 Building a Security Operations Center Lessons Learned Slide 6
SOC: Functional Definition Network Traffic / Events Response IDS/IPS/ SIEM/NGFW 24x7 Orientation Security Analysis Aggregation/ Correlation Systems Goal #1: Real-time threat detection and mitigation 10/10/2013 Building a Security Operations Center Lessons Learned Slide 7
SOC: Key Components KEY COMPONENTS Technology People Process Monitoring, Analysis, Mitigation, Reporting (FW, NGFW, IDS, IPS, SIEM, Web Security Gateway, AV, etc) and its own network/air gap. Security Analyst, Senior Security Analyst, Team Lead/Manager Training, Monitoring, Analyzing, Mitigating, Alerting, Escalating 10/10/2013 Building a Security Operations Center Lessons Learned Slide 8
TECHNOLOGY 10/10/2013 Building a Security Operations Center Lessons Learned Slide 9
Technology: SOC Private Network Tap or port mirroring from: - Core managed switch - DMZ segment - All choke points Event Logging / SysLogging enabled and fetched to one central location Secure communication between IDS/IPS and Management Systems Lab (separate network, sandboxing, training) 10/10/2013 Building a Security Operations Center Lessons Learned Slide 10
Technology: Security Sensor Role - Capture and store Inbound & Outbound network traffic - Meta data and raw data - Rule-based notifications - Behaviour-based notifications Components - Secure OS. - Secure communication (e.g. SSH, stunnel) - SW suite (e.g. Snort, Suricata, EasyIDS or Security Onion[Bro, Snort, Sguil, ELSA, Snorby, etc]) - Packet Capture (e.g. tcpdump, sguil) 10/10/2013 Building a Security Operations Center Lessons Learned Slide 11
Technology : Security Sensor Placement Domain Controllers and Servers DMZ Switch Switch Corporate Network Internet Corporate Firewall Users Security Sensor 10/10/2013 Building a Security Operations Center Lessons Learned Slide 12
Technology : More Monitoring Domain Controllers Domain Controllersand and Servers Servers DMZ WIFI Access Point Remote Laptops/Devices Internet Switch DMZ Switch Primary Security Sensor Network Traffic Analysis (Logging, Replay, Intrusion, Detection, Flows) Secondary Security Sensor System/Configuration (Integrity, Log Analysis, Shipping) System/Configuration Integrity SIEM/Logging, Managed Systems, Flows, Configuration Verification Users 10/10/2013 Building a Security Operations Center Lessons Learned Slide 13
Technology: Management System Role - Visualize events - Monitor key infrastructure and security devices Components - Secure OS. - SW (e.g. LAMP, Splunk, Nagios) 10/10/2013 Building a Security Operations Center Lessons Learned Slide 14
Technology: Traffic Analysis Role - Analyze events - Qualify Threats Components - Secure OS - SW (Wireshark, tcpdump) Internal port scan 10/10/2013 Building a Security Operations Center Lessons Learned Slide 15
Technology: Dashboard & Traffic Analysis 10/10/2013 Building a Security Operations Center Lessons Learned Slide 16
Technology: Lab Role Run Malware Profile Malware Test new detection rules Training Components Secure VM Software (e.g. Cuckoo sandbox) 10/10/2013 Building a Security Operations Center Lessons Learned Slide 17
Technology: Events to Actionable Data 10/10/2013 Building a Security Operations Center Lessons Learned Slide 18
Technology: Physical Security SOC Access should be restricted (policy, badge) Two factor authentication when possible Access to the SOC should be monitored (logs) Activities within the SOC should be monitored (video) 10/10/2013 Building a Security Operations Center Lessons Learned Slide 19
Technology: Lessons Learned Technology = Enabler Control technology Technology does cost money Craft technology so that it follows process It 10/10/2013 Building a Security Operations Center Lessons Learned Slide 20
PEOPLE 10/10/2013 Building a Security Operations Center Lessons Learned Slide 21
People: Roles Manager/Security Expert Team Lead Senior Security Analyst Security Analyst Sensor on Network 10/10/2013 Building a Security Operations Center Lessons Learned Slide 22
People: Security Analyst Skills Computer/network security Network administration Analytical mindset Process oriented Attention to detail Ability to work shifts Ability to work in a fast-paced and deadline-driven environment Team player University/College graduate 10/10/2013 Building a Security Operations Center Lessons Learned Slide 23
People: Senior Security Analyst Skills: Security Analyst + the following Security certifications (ISC2, ISACA, CISCO, SANS, Offensive Security, EC-Council, etc.) Network Security Experience Demonstrable initiative to continuous improvement 10/10/2013 Building a Security Operations Center Lessons Learned Slide 24
People: Team Lead / Manager Skills: Senior Security Analyst + the following Security background Process focus Superior inter-personal skills Client centric 10/10/2013 Building a Security Operations Center Lessons Learned Slide 25
People: Hiring the Right Candidates Finding Candidates - Targeted schools with an InfoSec or a CSI program - Security groups (TASK, LinkedIn groups) - Personal network Selecting Candidates - Test network and InfoSec knowledge - Evaluate inter-personal skills - Evaluate communication (verbal and written) - Test resistance to pressure and stress! 10/10/2013 Building a Security Operations Center Lessons Learned Slide 26
People: Team Building Context: Critical Security Operations People need to rely on each other Team work and collaboration first Invest in the team 10/10/2013 Building a Security Operations Center Lessons Learned Slide 27
People: SOC in Action Qualified People Context Rule -based Detection Expertise Technology Web Traffic Monitoring THREAT DETECTION QUALIFICATION and ERADICATION EXEcutable Downloads DLP Experience Behaviour -based Detection Policies 10/10/2013 Building a Security Operations Center Lessons Learned Slide 28
People: Lessons Learned Hire on core requirements Then train on specific needs Re-train as often as needed Bet on a great team rather than on a champion Audit knowledge and adherence to policies 10/10/2013 Building a Security Operations Center Lessons Learned Slide 29
PROCESS 10/10/2013 Building a Security Operations Center Lessons Learned Slide 30
Process: OODA Loop Workflow OBSERVE ORIENT DECIDE ACT Client Query Implicit Guidance & Control (SOC Team Lead, Management) Client Security Posture Implicit Guidance & Control Unfolding Circumstances Client Culture Client Defense Template Decision (Hypothesis) Action Outside Information (Other Client Data) Unfolding Interaction With Environment Observations Previous Experience (History) New Information (Deep Dig) Effective Feedback leads to Appropriate Action Taken Alert Intercept Unfolding Interaction With Environment 10/10/2013 Building a Security Operations Center Lessons Learned Slide 31
Process: Training Threat Landscape and attacks Specific Tools Specific Process Safe Practice (Lab) Hands-on Practice (Shadowed) Include Quiz 10/10/2013 Building a Security Operations Center Lessons Learned Slide 32
Process: Scheduling Nights/days/weekends Busy vs non busy? Vacations/sick etc Overtime: pay 10/10/2013 Building a Security Operations Center Lessons Learned Slide 33
Process: Threat Qualification Network/machine/user behaviour Follow the evidence Context Sandboxing Inspect Meta Data Identify the initiator of the traffic Confirm protocol/nature of traffic If web traffic involved Check Referrer & User Agent Check Traffic before and after suspicious activity Inspect Raw Data External Malicious Intent Confirmed and/or Internal Host Compromised 10/10/2013 Building a Security Operations Center Lessons Learned Slide 34
Process: Spamming Machine Spamming Confirmation: Live TCP dump on port 25 Rule out mail server responding to spam If spamming confirmed, kill port 25 for that machine Deep dig (TCP, BW, and web traffic) to identify vector of infection and/or back channel(s) Kill back channel(s) Alert and requesting to pull the machine from network and AV scan it Lift the block on port 25 once the machine has been cleaned up 10/10/2013 Place a watch on that machine for another 24 hours Building a Security Operations Center Lessons Learned Slide 35
Process: Threat Mitigation Core Tools - FW blocks - NGFW blocks - IDS Rules in kill mode - Whitelisting tools - Blacklisting tools - Manual TCP kill 10/10/2013 Building a Security Operations Center Lessons Learned Slide 36
Process: Threat Mitigation Traffic blocked by FW/NGFW No Traffic blocked by IDS No IP/domain blacklisted No Manual TCP kills Yes Yes Yes Yes Confirmation Mitigation confirmed 10/10/2013 Building a Security Operations Center Lessons Learned Slide 37
Process: Incident Categorization Could be invented but. It already exists! US Federal Agency Incident Categories (http://www.us-cert.gov/government-users/reporting-requirements) Category Name Description CAT 0 Exercise/Network Defense Testing This category is used during state, federal, national, international exercises and approved activity testing of internal/external network defenses or responses. CAT 1 Unauthorized Access In this category an individual gains logical or physical access without permission to a federal agency network, system, application, data, or other resource CAT 2 Denial of Service (DoS) An attack that successfully prevents or impairs the normal authorized functionality of networks, systems or applications by exhausting resources. This activity includes being the victim or participating in the DoS. CAT 3 Malicious Code Successful installation of malicious software (e.g., virus, worm, Trojan horse, or other code-based malicious entity) that infects an operating system or application. Agencies are NOT required to report malicious logic that has beensuccessfully quarantined by antivirus (AV) software. CAT 4 Improper Usage A person violates acceptable computing use policies. CAT 5 Scans/Probes/ Attempted Access This category includes any activity that seeks to access or identify a federal agency computer, open ports, protocols, service, or any combination for later exploit. This activity does not directly result in a compromise or denial of service. CAT 6 Investigation Unconfirmed incidents that are potentially malicious or anomalous activity deemed by the reporting entity to warrant further review. 10/10/2013 Building a Security Operations Center Lessons Learned Slide 38
Process: Alerting Confirm Policy states to Alert Select Template Provide - Origin - Destination - Description - Action taken - Recommendation Proof Alert Send Alert Acknowledgement? Escalate 10/10/2013 Building a Security Operations Center Lessons Learned Slide 39
Process: Incident Management Pre-incident Analysis and Incident Identification Incident Categorization Incident Handling (alerting, containment, recovery) Forensics Analysis Root Cause Analysis Threat Intelligence Integration 10/10/2013 Building a Security Operations Center Lessons Learned Slide 40
Process: Ticketing Keeps track of - Alerts - Client Requests (time to resolve/respond) - Change Requests (and approval) Enables - SLA measure - Stats & Reports 10/10/2013 Building a Security Operations Center Lessons Learned Slide 41
Process: Knowledge Sharing Essential to a SOC A SOC deals with a huge amount of heterogeneous information - Networks Topologies - IDS Configuration - Policies - Procedures - Threat Intelligence Knowledge Sharing - Requires a central repository, one or several DB - Custom application 10/10/2013 Building a Security Operations Center Lessons Learned Slide 42
Process: Lessons Learned Process helps reinforcing the right behaviour Ambiguity = potential error Revisit and improve regularly Communicate clearly on new/revised processes Audit process adherence 10/10/2013 Building a Security Operations Center Lessons Learned Slide 43
WHAT IS NEXT? 10/10/2013 Building a Security Operations Center Lessons Learned Slide 44
Talent Retention Monitor Intercept Alert & Escalate VA/PenTest Threat Modeling Cyber Security Research Continuous Monitoring Threat Intelligence Support Optimization Reporting Custom Request Process Automation Process Improvement Tools 10/10/2013 Building a Security Operations Center Lessons Learned Slide 45
Threat Intelligence - Collaboration with ISACs (Information Sharing and Analysis Center) - Collaboration with CERTs (Computer Emergency Readiness Team) - Clearing house - Threat research - Sandboxing - Threat modeling (Common Attack Pattern and Enumeration) 10/10/2013 Building a Security Operations Center Lessons Learned Slide 46
Threat Intelligence (continued) Sandboxing Illustrated 10/10/2013 Building a Security Operations Center Lessons Learned Slide 47
Compliance and Maturity Model Compliance contributes to - Maturing the processes - Predictable processes - A culture of control - Periodic Audits Acceptance 10/10/2013 Building a Security Operations Center Lessons Learned Slide 48
If Outsourcing a SOC? Questions to ask to your outsourced SOC provider: - What is the primary focus of the provider? - How is the SOC structured (people s perspective)? - What is the SOC employee turnover? - Could we have a copy of the SOC DR plan? - What is the SLA on Median Time to Alert? - How many sites are monitored? - Is the technology used tailored to Security Operations? - Could you provide a reference in a vertical similar to ours? - Could we have a copy of your latest Internal or external audit report? 10/10/2013 Building a Security Operations Center Lessons Learned Slide 49
EPILOGUE 10/10/2013 Building a Security Operations Center Lessons Learned Slide 50
Epilogue: a few DON Ts Assume about anything Underestimate the technology cost Resist to change Keep C players 10/10/2013 Building a Security Operations Center Lessons Learned Slide 51
Epilogue: DOs Think repeatable process Be persistent Be obsessive about the Client (Internal or External) Bet on people as the strength of a SOC, or when A Difference makes THE Difference Train and retrain Identify metrics as early as possible in the process Continuous Improvement: Audit Technology, People, Process 10/10/2013 Building a Security Operations Center Lessons Learned Slide 52
Thank you! Yves.Beretta@esentire.com active threat protection