Practical examples of Big Data, security analytics and visualization

Practical examples of Big Data, security analytics and visualization Jeff McGee, Data Scientist Josh Stevens, Enterprise Security Architect

Objective Identify problems in Security that could be solved with better analytics Discuss recent efforts on Big Data and Visualization Share examples of how HP s Cyber Defense Center has leveraged these capabilities 3

Big Data and the data overload

Challenge: There is more noise Good guys are making things less predictable Mobile Bring your own device Virtual machines and the Cloud SAAS New sources of logs HIPPA, SOX, PCI 5

Challenge: There is less signal And bad guys know how to stay inside the bell curve. Known: Easier to detect Unknown: Harder to detect Matches a signature Goes to a bad place Works in the clear Unauthorized Use Outside of baseline Within monitored infrastructure New behavior Goes to an approved place Works encrypted Authorized Use Inside of baseline Outside monitored infrastructure 6

Solutions to big data problems Let s take techniques originally built for other domains and apply them to security: Map-reduce Columnar Data Stores Machine Learning Visualization Tools 7

Tools and technologies Hadoop Framework for distributed computing Vertica Columnar database Tableau Visualization software Numpy/Scikit-learn Machine learning tools ArcSight Vertica analytic platform Hunt teams Security intelligence 8

Overview: The Vertica analytic platform Purpose built for Big Data from the first line of code Real time Analytics Rapid iterative conversations with your data Proven Scalability Store & Analyze PBs Ingest 30 TB/hour Open & Extensible Works with Hadoop, R Ecosystem of Visualization Tools, SDKs and Community Low TCO Efficient compressed storage Scale-out architecture Easy to setup & manage Software Only Private Cloud Public Cloud Appliance Flexible to deploy 9

Network-based IDS/IPS /Suspicious Network-based IDS/IPS /Recon Network-based IDS/IPS /Informational Router Network-based IDS/IPS /Compromise Content Security Firewall /Normal Firewall Applications Firewall Applications /Informational Network Monitoring /Informational Operating System /Informational Firewall Network Monitoring Security visualization Practical examples bust6 Category Device Type Applications Content Security Firewall Host-based IDS/IPS Mainframe Network Monitoring Network-based IDS/IPS Operating System Router VPN 10

Security management

View at a glance 100M fromafarsourcept Category Significance Count of Destination Port 0 20,000,000 40,000,000 66,854,010 90M Applications Count of Destination Port 80M 0 66,854,010 Firewall Applications 70M 60M Host-based IDS/IPS Network Monitoring 50M Applications Co 40M Network-based IDS/IPS 30M Firewall Operating System 20M Network-based IDS/IPS 10M Network-based IDS/IPS Network-based IDS/IPS 0M Network-based IDS/IPS Firewall VPN Informational/Error /Hostile /Compromise /Suspicious /Normal /Informational/Warning /Recon /Informational 12

Proportional relationships bust6 Category Device Type Applications Content Security Firewall Host-based IDS/IPS Router Firewall Operating System /Informational Mainframe Network Monitoring Network-based IDS/IPS Operating System Router Network-based Network-based IDS/IPS /Compromise Firewall VPN IDS/IPS /Suspicious Network-based IDS/IPS /Recon Applications Applications /Informational Content Security Firewall Network-based IDS/IPS /Informational Firewall /Normal Network Monitoring Network Monitoring /Informational 13

Security analysts

Starting points DeviceSeveritybyDevice Category Device Type 700M 600M 500M 400M 300M Cou 200M 100M 0M 15 Applications Content Security Database Firewall Host-based IDS/IPS Mainframe Network Monitoring Networkbased ID.. Operating System Policy Man agement Count of Device Severity for each Category Device Type. The view is filtered on Category Device Type, which keeps 13 of 20 members. Router Security Mangement VPN

Carving in desthostnameattemptfailsuccess 350M Category Outcome / Category Significance /Attempt /Failure /Success 300M 250M 200M 150M 100M 50M 0M /Informational/Error /Suspicious /Compromise /Informational /Compromise /Informational /Suspicious /Comp /Informational /Recon /Normal /Informational/Alert /Informational/Error /Informational/Warning /Recon /Normal Null Null Cou /Informational/Alert /Informational/Error /Informational/Warning Count of Destination Host Name for each Category Significance broken down by Category Outcome. The data is filtered on Destination Host Name, which excludes Null. The view is filtered on Exclusions (Category Outcome,Category Significance) and Category Outcome. The Exclusions (Category Outcome,Category Significance) filter keeps 35 members. The Category Outcome filter excludes Failure. 16

Trending attempts 17

Successes desthostnameattemptfailsuccess 350M Category Outcome / Category Significance /Attempt /Failure /Success 300M 250M 200M 150M 100M 50M 0M /Suspicious /Compromise /Informational /Compromise /Informational /Suspicious /Comp /Informational /Recon /Normal /Recon /Normal Null Null Cou 18 /Informational/Error /Informational/Alert /Informational/Error /Informational/Warning /Informational/Alert /Informational/Error /Informational/Warning Count of Destination Host Name for each Category Significance broken down by Category Outcome. The data is filtered on Destination Host Name, which excludes Null. The view is filtered on Exclusions (Category Outcome,Category Significance) and Category Outcome. The Exclusions (Category Outcome,Category Significance) filter keeps 35 members. The Copyright 2014 Hewlett-Packard Category Development Outcome filter Company, excludes L.P. Failure. The information contained herein is subject to change without notice.

Trending success by hostname 19

Actual hostnames 20

Keep in mind this is demo data however a quick internet search shows this domain has a reputation as a bullet proof server, delivering malware. Our visualization shows us it s been accessed every day for the last 30 days. 21

Bullet proof servers White spac e 22

Hunt teams Use case 1

60 days IPS data 90.. count of device severity 30 dall all scatter Device Receipt Time Device Severity High Medium Unknown Very-High 80.. 70.. 60.. 50.. Cat Network 40.. 30.. 20.. 10.. 0K 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 24

30 days 140K IPS Events Device Severity High Medium Unknown Very-High 120K 100K 80K Cat Network Co 60K 40K 20K 0K 25 Jun 3 Jun 8 Jun 13 Jun 18 Jun 23 Jun 28 Jul 3 Jul 8 Jul 13 Jul 18 Hour of Device Receipt Time [2014]

By technique 26

a Aggregate from victim tenacle fromvictim Category Technique /Exploit/Vulnerability /Policy/Breach /Traffic Anomaly/Network Layer 600 /Traffic Anomaly/Network Layer/Flow /Traffic Anomaly/Network Layer/IP Fragments 500 400 300 CouNetwork 200 100 0 Jun 7 27 Jun 12 Jun 17 Jun 22 Jun 27 Minute of Device Receipt Time [2014] Jul 2 Jul 7 Jul 12

Hunt teams Use case 2

Sonar 29

Sonar trend Destinations Source addresses 30

Hunt teams Use case 3

Bottom of the stack Informational fromafar2infoonly Network Monitoring Category Device Type Applications Content Security Database Firewall Host-based IDS/IPS Mainframe Network Monitoring Network-based IDS/IPS Operating System Policy Management Security Mangement VPN Applications Firewall Host-based IDS/IPS VPN Network-based IDS/IPS 32

VPN logging 33

Who s scanning via VPN? 34

For more information After the event Contact your sales rep Visit the HP Security Product Blog: hp.com/go/securityproductsblog 35

Please give me your feedback Session TB3273 Speaker Joshua Stevens, Jeff McGee Please fill out a survey. Hand it to the door monitor on your way out. Thank you for providing your feedback, which helps us enhance content for future events. 36

Thank you