1 Troubleshooting & Traffic Mining(TM) Stefan Burschka
2 Agenda Intro: Who, What, Problems with big traffic data How to address problems with brain & tools Lots of examples and exercises
3 What we do: Products: Network Troubleshooting, Forensics, Security TRANALYZER(T3): High Speed and Volume Traffic Analyzer TRAVIZ3: Graphical Toolset for Tranalyzer Complete Tool Sets for TM & Forensics Artificial Intelligence Plugins Research: Brain support 4 multi-dimensional datasets Encrypted TM Big Data Visualization (Traviz) Malware and covert channel detection Nifty stuff 3
4 Data (Traffic) Mining Forensics Input: Huge Tiny Answer Output: Tiny Large Puzzle Almost Everything Find what everybody missed Find patterns, anomalies and the undetectable. Almost Nothing , 80 Tell me everything and especially Owner, gender, colour of hair and underpants, talks to pigs?, OS, type of computer... 4
5 Traffic Mining(TM): Hidden Knowledge: Listen See, Understand, Invariants, Model Application in Troubleshooting, Security (Classification, Encrypted TM ) Netzwerk usage (VoiP, P2P traffic shaping, application/user profiling) Profiling & Marketing (usage performance- & market- index) Law enforcement and Legal Interception (Indication/Evidence) 5
6 Right! But why all this? What is the problem?
7 Problem: Controlled Flight into Terrain There is a union for the lifeform SW I m on strike Seldom problem oriented KISS Engineering High Complexity, small fault tolerance Time- and economic pressure, info overload Customer = Test Lab Banana Principle: Conditioning of cutomers SW became a weapon 7
8 The Network is slow, The Network is insecure; NO, it's not Microsoft, shut up, It wasn't me... Manager (MBA) Always right, DoR License to Powerpoint Production (poor Techie) Knows, Always warned, Always his fault: FUBAR License to get fired Finance (MBA) Knows basic calculus License to Excel We didn't find the problem in 4 months, can you do the job in 2 weeks? (We supply only 11TB data)
9 Large unstructured Traffic Datasets(/s) Problem Comprehension? WTF am I looking for? Data Selection: I d like to have Eat what you get! Preprocessing: Dimension Reduction, Feature Selection Data Integrity: Churchill / Murphy Formats: XML, EXCEL Binary, (txt) Tools/Interfaces: Microsoft, Java Bash, awk,db Operating Storage: Cache Memory SSD Disk FBHDD Visualization, AI 9
10 10 Exercise: What is wrong here?
11 See the disaster now? Now you have context! 11
12 Preprocessing/Context/Dimension Reduction Versatile Flow Compression A B Definition: (6-Tuple) Vlan(s), srcip, srcpport, dstip, dstport, L4Protocol Or why not a bit more context and meaning? srcwho, dstwho srcnetwork, dstnetwork Bad, Good Internal / External 12
13 Tranalyzer Flow Example A x9B x x C :0f:1f:cf:7c:45_00:00:0c:07:ac:0a_6387 http x00 0x42 0x x18 0xF900 0x0000 0x03 0x x B x9B x C x :d0:00:64:d0:00_00:0f:1f:cf:7c:45_8272 http x00 0x42 0x x18 0x1B00 0x0000 0x03 0x x
14 Yeah sure, lots of numbers. But encryption prevents TM! Which features you want to look at?
15 Encrypted TM: Packet Length Magic Distinguish from by listening Gap in tracks Tump Tump Tump Tump Tump Tump Tump Tump Sound ~ F d p dt dm dt v m dv dt dm dt dm dpkt dpkt dt 15 Packet Length Packet Fire Rate (Interdistance)
16 3D Statistical Application / User profiling Packet length-interdistance Statistics: Fingerprinting Vulnerable against TM 16
17 Packet Signal: Encrypted VoIP Mining Packet Length 17 time
18 Exercise: Multiple Flow Packet Length Signal See the features? Codec training Burschka (Fischkopp) Linux Dominic (Student) Windows SN Ping min l =3 18
31 Clustering of Multidimensional Data ESOM: Nonlinear Mapping Retina 31
32 ESOM Anomaly Picture: 13 Dim statistical T2 Flow parameters Now conceivable by human brain Bot Scanner DNS Zone Transfer 32
33 Mooooment Loriot
34 Exercise: AI yourself Microsoft 34
35 Exercise: Knowledge Extraction Data: Bad Weather - Low Pressure Rain Coat - Clouds Storm - Clouds Rain Noodle - Sun Rain Clouds - Spring Autumn - Pressure sinking Heating Data: Good Weather - Sun Sea Bomb - High Pressure Isobar Sun - Pressure climb Grill Steak - Sun Friends Beer - Sun Beach Clouds - Summer Sun Good Weather = Sun & / Rain Words or Word Chains, which separate most of data sets correctly, have highest Information gain. 35
36 Questions / Comments Try me! Who wants Bootcamp? Google: Dataming for Hackers 36
10 Things Your Next Firewall Must Do Introduction Without question, your network is more complex than ever before. Your employees are accessing any application they want, using work or personal devices.
A Trend Micro Research Paper Suggestions to Help Companies with the Fight Against Targeted Attacks Jim Gogolinski Forward-Looking Threat Research Team Contents Introduction...3 Targeted Attacks...4 Defining
32 Big Data: present and future Big Data: present and future Mircea Răducu TRIFU, Mihaela Laura IVAN University of Economic Studies, Bucharest, Romania firstname.lastname@example.org, email@example.com
Getting Physical with the Digital Investigation Process Brian Carrier Eugene H. Spafford Center for Education and Research in Information Assurance and Security CERIAS Purdue University Abstract In this
A Websense White Paper ADVANCED PERSISTENT THREATS AND OTHER ADVANCED ATTACKS: THREAT ANALYSIS AND DEFENSE STRATEGIES FOR SMB, MID-SIZE, AND ENTERPRISE ORGANIZATIONS REV 2 ADVANCED PERSISTENT THREATS AND
A First Encounter with Machine Learning Max Welling Donald Bren School of Information and Computer Science University of California Irvine November 4, 2011 2 Contents Preface Learning and Intuition iii
White Paper Application Visibility and Monitoring > An integrated approach to application delivery Application performance drives business performance Every business today depends on secure, reliable information
MODEL ATC-2004 TCP/IP TO RS-232/422/485 CONVERTER User s Manual 1.1 Introduction The ATC-2004 is a 4 Port RS232/RS485 to TCP/IP converter integrated with a robust system and network management features
IceWarp Unified Communications Reference Version 11.1 Published on 11/4/2014 Contents... 4 About... 5 The Big Picture... 7 Reference... 8 General... 8 Dial Plan... 9 Dial Plan Examples... 12 Devices...
On Designing and Deploying Internet-Scale Services James Hamilton Windows Live Services Platform ABSTRACT The system-to-administrator ratio is commonly used as a rough metric to understand administrative
How Fail Today s Networks And Why Will Prevail Why your current firewall may be jeopardizing your security, and how you can counter today s threats, manage web 2.0 apps and enforce acceptable-use policies.
What is SIP? What s a Control Channel? History of Signaling Channels Signaling and VoIP Complexity Basic SIP Architecture Simple SIP Calling Alice Calls Bob Firewalls and NATs SIP URIs Multiple Proxies
Application Performance Management for Enterprise Applications White Paper from ManageEngine Web: Email: firstname.lastname@example.org Table of Contents 1. Introduction 2. Types of applications used
TABLE OF CONTENTS Introduction... 3 The Importance of Triplestores... 4 Why Triplestores... 5 The Top 8 Things You Should Know When Considering a Triplestore... 9 Inferencing... 9 Integration with Text
INTELLIGENT BUSINESS STRATEGIES W H I T E P A P E R Architecting A Big Data Platform for Analytics By Mike Ferguson Intelligent Business Strategies October 2012 Prepared for: Table of Contents Introduction...
Open Learning Universiteit Unit 3 Learning Unit 3 Architectural patterns Contents Introduction............................................... 35 3.1 Patterns..............................................
Hacker Intelligence Initiative Man in the Cloud (MITC) Attacks 1. Executive Summary In this report, we demonstrate a new type of attack we call Man in the Cloud (MITC). These MITC attacks rely on common
Firewalls in the Data Center: Main Strategies and Metrics Joel Snyder, PhD Senior Partner, Opus One What You Will Learn Measuring performance in networks has usually involved looking at one number: throughput.
WHITE PAPER VoIP Networks August 2013 Keys to Minimizing Echo in VoIP Networks Table of Contents Executive Summary 3 Section 1: The Challenge 4 Minimizing Echo in VoIP Systems Section 2: The Opportunity
ZyWALL 5 Internet Security Appliance Support Notes Version 4.02 Dec. 2006 INDEX Application Notes...12 Seamless Incorporation into your network...12 Using Transparent (Bridge Mode) Firewall...12 Internet
D5.1 Version: 0.7 Date: 2008-07-30 Author: UNITN Dissemination status: PU Document reference: D5.1 State of art in the field of Adaptive Service Composition Monitoring and Management Project acronym: COMPAS
AN INTRODUCTION TO Data Science Jeffrey Stanton, Syracuse University INTRODUCTION TO DATA SCIENCE 2012, Jeffrey Stanton This book is distributed under the Creative Commons Attribution- NonCommercial-ShareAlike
I Know Where You are and What You are Sharing: Exploiting P2P Communications to Invade Users Privacy Stevens Le Blond Chao Zhang Arnaud Legout Keith Ross Walid Dabbous MPI-SWS, Germany NYU-Poly, USA INRIA,
Securing Enterprise Applications Version 1.1 Updated: November 20, 2014 Securosis, L.L.C. 515 E. Carefree Highway Suite #766 Phoenix, AZ 85085 T 602-412-3051 email@example.com www.securosis.com Author