Big Data in Action: Behind the Scenes at Symantec with the World s Largest Threat Intelligence Data

Size: px

Start display at page:

Download "Big Data in Action: Behind the Scenes at Symantec with the World s Largest Threat Intelligence Data"

Scarlett McKinney
8 years ago
Views:

1 Big Data in Action: Behind the Scenes at Symantec with the World s Largest Threat Intelligence Data Patrick Gardner VP Engineering Sourabh Satish Distinguished Engineer Symantec Vision Big Data in Action 1

2 Agenda Symantec s Data Analytics Platform Creating Powerful Big Data Applications Big Data Enabling Targeted Attack Detection Symantec Vision Big Data in Action 2

3 Symantec Data Analytics Platform Symantec Vision Big Data in Action 3

4 SYMANTEC DATA ANALYTICS PLATFORM A generic platform for converting data into intelligence High volume loading Analytics Data mart Massively-parallel data store Symantec Vision Big Data in Action 4

5 All Security Telemetry in One Place File/user/site associations File heuristics Behavior heuristics Industry feeds Traffic Hygiene Parent program File name/path Instruction use File structure Digital signature Has a GUI Settings changes In program menu Vendor A sent us this file IP address Y sends spam has malicious URL File hash File hash File hash File hash IP/URL IP/URL IP/URL Machine ID Machine ID Machine ID Network traffic IP source IP destination Vulnerability ID SSL certification Domain Level of VeriSign SSL certification Honeypot sensors Suspicious traffic from IP address X Web site details Popularity PII fields Site age Hundreds of additional features File hash File hash IP/URL IP/URL IP/URL IP/URL Machine ID Machine ID Machine ID Symantec Vision Big Data in Action 5

6 SYMANTEC DATA ANALYTICS PLATFORM Raw features Big Data System Intelligence driven applications Examples: Downloads Web site visits Intrusion alerts Malware alerts Behaviors File appearance Crashes Symantec Data Analytics Platform trillion rows of data 55,000 rows added every second File URL Crash Behavior Forms SONAR engine File Insight Scam Insight URL Insight Crash Ratings Intelligence Symantec Vision Big Data in Action 6

7 Symantec BIG DATA Platform Tracks more than 13.8 Billion files Tracks more than 21.3 billion URLs FILES + MACHINES + URLs are viewed as a huge graph comprising of 152 billion nodes that expresses relations between them that drives our unique hygiene based ability to rate files and URLs Advanced machine learned predictive models (using this data) rates files and URLs We respond to more than ~10Billiion queries per day to protect users from security risks Billions of artifacts Tracked or Analyzed Per Day We aggregate malware metadata information on our analytics platform from various internal and external sources that helps us accurately rate files and URLs. This is currently 150 TB loaded into our DB. We have loaded a total of 2.1 trillion rows, growing monthly at the rate of 100+ billion rows Symantec Confidential - Features for CY2013 Releases Symantec Vision Big Data in Action 7

8 Big Data Applications Symantec Vision Big Data in Action 8

9 Symantec Big Data Applications 1. File & URL Insight blocks malicious files and URLs based on the wisdom of the crowd 2. File & Behavioral Heuristics predicts risk about files and processes using classifiers 3. Scam Insight predicts if a web site might steal your personal information (e.g. CC) 4. Mobile Insight predicts security, privacy, and performance of mobile apps 5. Fraud Detection Services uses endpoint reputation for intelligent authentication and fraud detection 6. Synapse - Endpoint, , and Network correlation - correlates events across control points 7. Stability Ratings predicts if a program will crash your machine Symantec Vision Big Data in Action 9

10 File Insight Overview File X just arrived on computer Y 1 Collect data 2 Place data in a central store File X has a low reputation 4 3 Deliver reputation scores Analyze relationships to calculate reputations Symantec Vision Big Data in Action 10

11 Insight makes decisions based on who downloads what from where 150+ Billion associations Symantec Vision Big Data in Action 11

File and Behavioral Heuristics Collects millions of programs Community Watch File & behavior profiles Over 500 million profiles, hundreds of attributes 1 2 3 Machine learning engine Analyzes patterns

12 File and Behavioral Heuristics Collects millions of programs Community Watch File & behavior profiles Over 500 million profiles, hundreds of attributes Machine learning engine Analyzes patterns of good and bad programs Changes DNS settings Modifies browser homepage Disables UAC Changes security settings Adds desktop shortcut Is signed by good CA Distributed to our products LiveUpdate Symantec Vision Big Data in Action Symantec Security Response Classification rules undergo rigorous certification Classification rules Creates rules for classifying files as good or bad 12

at $10/month now we block this, protecting over 10k potential victims per day We know about every web site (traffic volume, age, SSL, referrals)

13 Scam Insight Detects sites that try to steal key information like your credit card number or cell phone number These sites aren t traditional phishing - Counterfeit products - Small banks - Easy cash/loans/awards We found one that tricks users into signing up for a premium SMS service at $10/month now we block this, protecting over 10k potential victims per day We know about every web site (traffic volume, age, SSL, referrals) We see which sites ask for credit cards, passwords, etc. We warn users about new sites asking for this data Symantec Vision Big Data in Action 13

14 STAR MOBILE INSIGHT App automation Advanced static analysis & heuristics Symantec Data Analytics Platform Machine learning and rules create new insight Collect Inspect Safe? From mobile devices, app stores, and partners Run Attributes Trustworthy? Privacy leak? Battery drain? Rapid forced runtime analysis in a VM Insight Telemetry and feeds already in SDAP & more Symantec Vision Big Data in Action 14

Solving the Challenges: Advanced Threat Protection Synapse correlation of events across control points Email.

15 Solving the Challenges: Advanced Threat Protection Synapse correlation of events across control points .cloud Events Provides meaningful prioritization for incident responders, saving time Closes the loop from network event to target machine or user Symantec Cloud Events Events Gateway SEP Symantec Vision Big Data in Action 15

16 INTELLIGENCE ANALYSIS April 12, 2012 May 7, 2012 July 10, 2012 Symantec Vision Big Data in Action 16

17 INTELLIGENCE ANALYSIS Symantec Vision Big Data in Action 17

18 INTELLIGENCE ANALYSIS Symantec Vision Big Data in Action 18

19 INTELLIGENCE ANALYSIS Symantec Vision Big Data in Action 19

20 INTELLIGENCE ANALYSIS Symantec Vision Big Data in Action 20

21 INTELLIGENCE ANALYSIS Symantec Vision Big Data in Action 21

22 INTELLIGENCE ANALYSIS Symantec Vision Big Data in Action 22

23 Thank you! Please take a few minutes to fill out the short session survey available on the mobile app the survey will be available in the mobile app shortly after the session ends. And then watch for and complete the more extensive post-event survey that will arrive via a few days after the conference. To download the app, go to or search for Vision 2014 in the itunes or Android stores. 23

24 Thank you! Patrick Gardner Sourabh Satish Copyright 2014 Symantec Corporation. All rights reserved. Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice. Symantec Vision Big Data in Action 24

How Shared Security Intelligence Can Better Stop Targeted Attacks

How Shared Security Intelligence Can Better Stop Targeted Attacks SESSION ID: SPO3-T07 Piero DePaoli Senior Director Global Product Marketing Symantec Corporation Targeted Attacks are an Increasing Issue