CLASSIFYING NETWORK TRAFFIC IN THE BIG DATA ERA

Size: px
Start display at page:

Download "CLASSIFYING NETWORK TRAFFIC IN THE BIG DATA ERA"

Transcription

1 CLASSIFYING NETWORK TRAFFIC IN THE BIG DATA ERA Professor Yang Xiang Network Security and Computing Laboratory (NSCLab) School of Information Technology Deakin University, Melbourne, Australia

2

3 Major Research Themes Security and Privacy Large-scale attacks and defence Malware modelling and classification Trusted computing and authentication IP traceback Networking Network analytics Traffic classification Big data analytics CPS, IoT, and RFID Social networks

4 Publications Related to This Talk Jun Zhang, Yang Xiang, Yu Wang, Wanlei Zhou, Yong Xiang, Yong Guan, Network Traffic Classification Using Correlation Information, IEEE Transactions on Parallel and Distributed Systems, vol. 24, no.1, pp , Jun Zhang, Chao Chen, Yang Xiang, Wanlei Zhou, and Yong Xiang, Internet Traffic Classification by Aggregating Correlated Naive Bayes Predictions, IEEE Transactions on Information Forensics and Security, vol. 8, no. 1, pp. 5-15, Jun Zhang, Chao Chen, Yang Xiang, Wanlei Zhou, and Athanasios V. Vasilakos, "An Effective Network Traffic Classification Method with Unknown Flow Detection", IEEE Transactions on Network and Service Management, vol. 10, no. 2, pp , 2013.

5 Agenda Introduction Related Work Our Innovations Conclusion and Future Directions

6 Agenda Introduction Related Work Our Innovations Conclusion and Future Directions

7 Introduction The big data Features: 3Vs: volume, velocity, and variety The phenomenon behind big data

8 Big Network Traffic Data Internet traffic doubles every year according to CISCO New applications emerging every day No existing devices can record all network traffic

9 Challenges to the Future Networks Things are significantly changed in future networks: Infrastructure exposed Human involved Devices increased Network traffic increased

10 Traffic Classification Global Internet Traffic Expected to Quadruple by 2015 By 2015, about 3 billion people will be using the Internet 1 zettabyte in traffic per year (A zettabyte is equal to 1,000,000,000,000,000,000,000 bytes)

11 Traffic Classification What is in the traffic?

12 Traffic Classification A mixture of everything!

13 Traffic Classification Do you want to tell which is which? Technique: Classifying network traffic flows by their generation applications

14 Traffic Classification: Edge Link Example Internet ART-TC Classification Result Flow #2 #1 #3 Link: Ethernet II Internet: IPv > > > Transport: TCP UDP port 7845->port >port 3074->port Application: HTTP Gaming (streaming) (web (XBOXLIVE) browsing) The Real-Time Traffic Classification Engine

15 Traffic Classification VS. Packet Classification Packet classifier is actuator It applies a sequence of pre-defined rules to incoming packets A predicate over some packet header fields A decision to be taken upon the matching packets Challenge: huge set of rules & high speed links Traffic classifier is predictor It observes/extracts some features of incoming flows/packets Packet header fields, payloads, flow statistics It then predicts the underlying applications and applies labels Challenge: accuracy, efficiency, human efforts It uses packet classifiers to classify packets to flows It is usually used to generate rules for packet classifiers

16 Methods of Traffic Classification The Unit of traffic in consideration is usually flows (also called connections, sessions, conversations) Port number fields Application payload (Deep Packet Inspection) Flow statistics Describe flows with feature vectors by extracting pre-defined features Data points in the feature space Data are labelled: supervised learning Data are unlabelled: clustering Inter-packet Packet size: Flow time: max/min/mean/std.dev std.dev duration

17 Agenda Introduction Related Work Our Innovations Conclusion and Future Directions

18 Traffic Classification: Techniques Example Traffic Statistics: 13 4 FTP-DATA vs. TELNET (x-axis: avg. inter-packet-time; y-axis: avg. packet size) Flow Reassemble Link: Feature Extraction Feature Extraction 1 Packet Header 2 2 Ethernet II Internet: IPv > Transport: TCP port >port 80 Packet Header Intelligent Decision Engine ART-TC Packet Payload Packet Payload Machine Learning ART-TC TCP DST Port 80 Classification result Flow Statistics 7 Internet Flow Statistics HTTP (Web browsing Google) HTTP Security & QoS Control Traffic Analytics Advanced Data Mining User Profiling

19 Traffic Classification Methods Chapter 5, WAN and Application Optimization Solution Guide, CISCO

20 Flow Statistical Feature Based Methods Supervised classification Parametric classifiers (C4.5 decision tree, neural network) Non-parametric classifiers (k-nn) Unsupervised classification Clustering + Mapping Difficult to map a large number of clusters to a small number of applications

21 Supervised Traffic Classification Supervised algorithms + flow statistical feature Naïve Bayes (Moore and Zuev 2005) C4.5 decision tree (Williams et al. 2006) k-nn (Roughan et al. 2004) Bayesian network (Williams et al. 2006) Neural network (Auld et al. 2007) SVM (Kim et al. 2008, Este et al. 2009) Supervised algorithms + IP payload Naïve Bayes, AdaBoost, EM (Haffner et al. 2005) SVM (Finamore et al. 2010)

22 Unsupervised Traffic Classification Traffic clustering EM (McGregor et al. 2004) AutoClass (Zander et al. 2005) k-means (Bernaille et al. 2006) DBSCAN (Erman et al. 2006) Combine flow statistical features and IP payload information (Wang et al. 2010; Finamore et al. 2011) Semi-supervised clustering k-means + few supervised samples (Erman et al. 2007)

23 Agenda Introduction Related Work Our Innovations Conclusion and Future Directions

24 Challenges of Traffic Classification for Big Network Data Challenge 1: Big network data, small samples Challenge 2: Processing traffic accurately, with highspeed Challenge 3: Unknown applications

25 Our Innovations Solving Challenge 1: Big network data, small samples Solving Challenge 2: Processing traffic accurately, with high-speed Solving Challenge 3: Unknown applications

26 Innovation 1: Traffic Classification Using Correlation Information Problem Big network data, small samples Observation Correlation among flows Be Benefit to traffic classification Idea Supervised classification using flow correlation Effectively improve classification accuracy when a small number of supervised training samples are available

27 Major Contributions New approach Propose a novel non-parametric approach to incorporate flow correlation into classification process Theoretical study Provide a detailed theoretical analysis on the novel classification approach and its performance benefit Empirical study Validate the effectiveness by comparing classification performance of the proposed approach and state-of-the-art methods

28 Correlation Analysis: Example Video Text Image

29 System Model: TCC

30 Correlation Analysis 3-tuple heuristic: In a certain period of time, the flows sharing the same 3- tuple {des_ip, dst_port, protocol} form a Bag of Flows (BoF) In this example, flows AD, BD, and CD are generated by the same application, which can form a BoF.

31 Performance Benefit

32 Performance Benefit

33 Classification Method

34 Performance Evaluation Datasets Experiments Statistical features Performance metrics Results Overall performance Per-experiment performance Per-class performance Comparison with other existing methods Summary

35 Real-world Network Traffic Datasets wide: P2P, DNS, FTP, WWW, CHAT, MAIL isp: BT, DNS, ebuddy, FTP, HTTP, IMAP, MSN, POP3, RSP, SMTP, SSH, SSL, XMPP, YahooMsg

36 Statistical Features

37 Performance Metrics Overall accuracy Ratio of the sum of all correctly classified flows to the sum of all testing flows Measure the accuracy of a classifier on the whole testing data F-measure F measure= 2 precision recall/ precision+recall Evaluate the per-class performance

38 Overall Performance - wide

39 Overall Performance - isp

40 Summary Overall Performance With comparison to the NN classifier, the proposed methods can effectively improve the overall performance of traffic classification.

41 Per-Experiment Performance 10 training samples per class

42 Per-Experiment Performance 20 training samples per class

43 Summary - Per-Experiment Performance The proposed methods can improve the classification accuracy in a robust way and consistent improvement is achieved in almost every experiment.

44 F-measure Per-Class - wide

45 F-measure Per-Class - isp

46 F-measure Per-Class - isp

47 Summary F-measure Per-Class The proposed methods can improve the F-measure of every class and significant improvements are obtained in most classes.

48 Comparison with Other Methods - wide

49 Comparison with Other Methods - isp

50 Summary - Comparison TCC is superior to the existing traffic classification methods since it demonstrates the ability of applying flow correlation to effectively improve traffic classification performance.

51 Innovation 2: Bag of Flow Framework Problem: Processing traffic accurately, with high-speed We propose a new traffic classification scheme to utilize the information among the correlated traffic flows generated by an application We provide a theoretical study on the proposed scheme Theoretical framework of classifier combination Analyze the sensitivities to prediction errors of different aggregation rules employed in the proposed scheme

52 Classification Process of Correlated Traffic

53 Evaluation

54 Evaluation

55 Evaluation

56 Innovation 3: Compound Classification Framework Problem: Unknown applications

57 Statistics-based Traffic Classification Very high accuracy Training Set - labelled data HTTP FTP SMTP Testing Set - unlabelled data??? Predict the classes Classifier (by supervised learning)

58 Unknown Classes are Overlooked Training In classifier Set design, - known most previous works Testing assumed: Set - known classes classes All classes are known during training HTTP All classes have sufficient data for training HTTP FTP In evaluation, they got good results by excluding unwanted data FTP SMTP Classifiers were trained with a limited number of classes BitTorrent (unknown class) SMTP Classifiers were tested against only data from the trained classes

59 Innovation 3: Compound Classification Framework Problem: Unknown applications We aim to tackle the problem of unknown flows in a semisupervised framework This work considers very few labelled training samples and investigates flow correlation in real world network environment, which makes it different to previous works Flow label propagation to automatically label relevant flows from a large unlabelled dataset We proposed the compound classification to jointly identify the correlated flows in order to further boost the classification accuracy We provide the theoretical justification on performance benefit of applying these two new techniques to network traffic classification

60 System Model

61 Flow Label Propagation

62 Nearest Cluster-based Classifier

63 Compound Classification

64 Impact of Unknown Applications

65 Overall Accuracy and F-Measure

66 F-Measure on isp Data

67 Comparison against Other Methods

68 Comparison against Other Methods

69 Comparison against Other Methods

70 Comparison against Other Methods

71 Agenda Introduction Related Work Our Innovations Conclusion and Future Directions

72 Conclusion and Future Directions We proposed three frameworks to deal with three major challenges of the network traffic classification problems in big data era Solving Challenge 1: Big network data, small samples Solving Challenge 2: Processing traffic accurately, with high-speed Solving Challenge 3: Unknown applications

73 Future Directions Cloud computing: classifying encrypted traffic More than half of the traffic is HTTP: further classifying HTTP traffic Building user profile based on traffic classification CPS/IoT/Cloud: classifying data link layer traffic

74 Thank You! More about? Yang Xiang Yang Xiang

Robust Network Traffic Classification

Robust Network Traffic Classification IEEE/ACM TRANSACTIONS ON NETWORKING 1 Robust Network Traffic Classification Jun Zhang, Member, IEEE, XiaoChen, Student Member, IEEE, YangXiang, Senior Member, IEEE, Wanlei Zhou, Senior Member, IEEE, and

More information

Encrypted Internet Traffic Classification Method based on Host Behavior

Encrypted Internet Traffic Classification Method based on Host Behavior Encrypted Internet Traffic Classification Method based on Host Behavior 1,* Chengjie GU, 1 Shunyi ZHANG, 2 Xiaozhen XUE 1 Institute of Information Network Technology, Nanjing University of Posts and Telecommunications,

More information

Identification of Network Applications based on Machine Learning Techniques

Identification of Network Applications based on Machine Learning Techniques Identification of Network Applications based on Machine Learning Techniques Valentín Carela Español - vcarela@ac.upc.edu Pere Barlet Ros - pbarlet@ac.upc.edu UPC Technical Report Deptartament d Arqutiectura

More information

Aggregating Correlated Naive Predictions to Detect Network Traffic Intrusion

Aggregating Correlated Naive Predictions to Detect Network Traffic Intrusion Aggregating Correlated Naive Predictions to Detect Network Traffic Intrusion G.Vivek #1, B.Logesshwar #2, Civashritt.A.B #3, D.Ashok #4 UG Student, Department of Computer Science and Engineering, SRM University,

More information

A Preliminary Performance Comparison of Two Feature Sets for Encrypted Traffic Classification

A Preliminary Performance Comparison of Two Feature Sets for Encrypted Traffic Classification A Preliminary Performance Comparison of Two Feature Sets for Encrypted Traffic Classification Riyad Alshammari and A. Nur Zincir-Heywood Dalhousie University, Faculty of Computer Science {riyad, zincir}@cs.dal.ca

More information

Traffic Classification with Sampled NetFlow

Traffic Classification with Sampled NetFlow Traffic Classification with Sampled NetFlow Valentín Carela-Español, Pere Barlet-Ros, Josep Solé-Pareta Universitat Politècnica de Catalunya (UPC) {vcarela,pbarlet,pareta}@ac.upc.edu Abstract The traffic

More information

CYBER SCIENCE 2015 AN ANALYSIS OF NETWORK TRAFFIC CLASSIFICATION FOR BOTNET DETECTION

CYBER SCIENCE 2015 AN ANALYSIS OF NETWORK TRAFFIC CLASSIFICATION FOR BOTNET DETECTION CYBER SCIENCE 2015 AN ANALYSIS OF NETWORK TRAFFIC CLASSIFICATION FOR BOTNET DETECTION MATIJA STEVANOVIC PhD Student JENS MYRUP PEDERSEN Associate Professor Department of Electronic Systems Aalborg University,

More information

Forensic Network Traffic Analysis

Forensic Network Traffic Analysis Forensic Network Traffic Analysis Noora Al Khater Department of Informatics King's College London London, United Kingdom noora.al_khater@kcl.ac.uk Richard E Overill Department of Informatics King's College

More information

ATCM: A Novel Agent-based Peer-to-Peer Traffic Control Management

ATCM: A Novel Agent-based Peer-to-Peer Traffic Control Management Journal of Computational Information Systems 7: 7 (2011) 2307-2314 Available at http://www.jofcis.com ATCM: A Novel Agent-based Peer-to-Peer Traffic Control Management He XU 1,, Suoping WANG 2, Ruchuan

More information

Classifying P2P Activities in Netflow Records: A Case Study (BitTorrnet & Skype) Ahmed Bashir

Classifying P2P Activities in Netflow Records: A Case Study (BitTorrnet & Skype) Ahmed Bashir Classifying P2P Activities in Netflow Records: A Case Study (BitTorrnet & Skype) by Ahmed Bashir A thesis submitted to the Faculty of Graduate and Postdoctoral Affairs in partial fulfillment of the requirements

More information

CLASSIFICATION BASED NOVEL FRAMEWORK FOR NETWORK TRAFFIC ANALYSIS IN CLOUD COMPUTING

CLASSIFICATION BASED NOVEL FRAMEWORK FOR NETWORK TRAFFIC ANALYSIS IN CLOUD COMPUTING CLASSIFICATION BASED NOVEL FRAMEWORK FOR NETWORK TRAFFIC ANALYSIS IN CLOUD COMPUTING Sourav Debnath 1, Vijay Kumar Jha 2 1 Student, M. Tech, Department of Information Technology, Birla Institute of Technology,

More information

Protocols. Packets. What's in an IP packet

Protocols. Packets. What's in an IP packet Protocols Precise rules that govern communication between two parties TCP/IP: the basic Internet protocols IP: Internet Protocol (bottom level) all packets shipped from network to network as IP packets

More information

Online Classification of Network Flows

Online Classification of Network Flows 2009 Seventh Annual Communications Networks and Services Research Conference Online Classification of Network Flows Mahbod Tavallaee, Wei Lu and Ali A. Ghorbani Faculty of Computer Science, University

More information

The Applications of Deep Learning on Traffic Identification

The Applications of Deep Learning on Traffic Identification The Applications of Deep Learning on Traffic Identification Zhanyi Wang wangzhanyi@360.cn Abstract Generally speaking, most systems of network traffic identification are based on features. The features

More information

Network Traffic Characterization using Energy TF Distributions

Network Traffic Characterization using Energy TF Distributions Network Traffic Characterization using Energy TF Distributions Angelos K. Marnerides a.marnerides@comp.lancs.ac.uk Collaborators: David Hutchison - Lancaster University Dimitrios P. Pezaros - University

More information

Detection. Perspective. Network Anomaly. Bhattacharyya. Jugal. A Machine Learning »C) Dhruba Kumar. Kumar KaKta. CRC Press J Taylor & Francis Croup

Detection. Perspective. Network Anomaly. Bhattacharyya. Jugal. A Machine Learning »C) Dhruba Kumar. Kumar KaKta. CRC Press J Taylor & Francis Croup Network Anomaly Detection A Machine Learning Perspective Dhruba Kumar Bhattacharyya Jugal Kumar KaKta»C) CRC Press J Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint of the Taylor

More information

How is SUNET really used?

How is SUNET really used? MonNet a project for network and traffic monitoring How is SUNET really used? Results of traffic classification on backbone data Wolfgang John and Sven Tafvelin Dept. of Computer Science and Engineering

More information

A Survey of Methods for Encrypted Traffic Classification and Analysis

A Survey of Methods for Encrypted Traffic Classification and Analysis INTERNATIONAL JOURNAL OF NETWORK MANAGEMENT Int. J. Network Mgmt 2014; 00:1 24 Published online in Wiley InterScience (www.interscience.wiley.com). A Survey of Methods for Encrypted Traffic Classification

More information

Implementation of Naive Bayes as a Quality of Service Determination on Traffic Network Communication Protocol

Implementation of Naive Bayes as a Quality of Service Determination on Traffic Network Communication Protocol RESEARCH ARTICLE OPEN ACCESS Implementation of Naive Bayes as a Quality of Service Determination on Traffic Network Communication Protocol Sudarma, M.*, Pramana, D.H ** * Computer System and Informatics,

More information

SDN 交 換 機 核 心 技 術 - 流 量 分 類 以 及 應 用 辨 識 技 術. 黃 能 富 教 授 國 立 清 華 大 學 特 聘 教 授, 資 工 系 教 授 E-mail: nfhuang@cs.nthu.edu.tw

SDN 交 換 機 核 心 技 術 - 流 量 分 類 以 及 應 用 辨 識 技 術. 黃 能 富 教 授 國 立 清 華 大 學 特 聘 教 授, 資 工 系 教 授 E-mail: nfhuang@cs.nthu.edu.tw SDN 交 換 機 核 心 技 術 - 流 量 分 類 以 及 應 用 辨 識 技 術 黃 能 富 教 授 國 立 清 華 大 學 特 聘 教 授, 資 工 系 教 授 E-mail: nfhuang@cs.nthu.edu.tw Contents 1 2 3 4 5 6 Introduction to SDN Networks Key Issues of SDN Switches Machine

More information

Traffic Analysis of Mobile Broadband Networks

Traffic Analysis of Mobile Broadband Networks Traffic Analysis of Mobile Broadband Networks Geza Szabo,Daniel Orincsay,Balazs Peter Gero,Sandor Gyori,Tamas Borsos TrafficLab, Ericsson Research, Budapest, Hungary Email:{geza.szabo,daniel.orincsay,

More information

EXPLORER. TFT Filter CONFIGURATION

EXPLORER. TFT Filter CONFIGURATION EXPLORER TFT Filter Configuration Page 1 of 9 EXPLORER TFT Filter CONFIGURATION Thrane & Thrane Author: HenrikMøller Rev. PA4 Page 1 6/15/2006 EXPLORER TFT Filter Configuration Page 2 of 9 1 Table of Content

More information

Breaking and Improving Protocol Obfuscation

Breaking and Improving Protocol Obfuscation Breaking and Improving Protocol Obfuscation Technical Report No. 2010-05, ISSN 1652-926X Erik Hjelmvik Independent Network Security and Forensics Researcher Enköping, Sweden erik.hjelmvik@gmail.com Wolfgang

More information

Kick starting science...

Kick starting science... Computer ing (TDDD63): Part 1 Kick starting science... Niklas Carlsson, Associate Professor http://www.ida.liu.se/~nikca/ What do you have in the future? What do you have in the future? How does it keep

More information

Review on Analysis and Comparison of Classification Methods for Network Intrusion Detection

Review on Analysis and Comparison of Classification Methods for Network Intrusion Detection Review on Analysis and Comparison of Classification Methods for Network Intrusion Detection Dipika Sharma Computer science Engineering, ASRA College of Engineering & Technology, Punjab Technical University,

More information

Early Recognition of Encrypted Applications

Early Recognition of Encrypted Applications Early Recognition of Encrypted Applications Laurent Bernaille with Renata Teixeira Laboratoire LIP6 CNRS Université Pierre et Marie Curie Paris 6 Can we find the application inside an SSL connection? Network

More information

An apparatus for P2P classification in Netflow traces

An apparatus for P2P classification in Netflow traces An apparatus for P2P classification in Netflow traces Andrew M Gossett, Ioannis Papapanagiotou and Michael Devetsikiotis Electrical and Computer Engineering, North Carolina State University, Raleigh, USA

More information

Internet Protocol: IP packet headers. vendredi 18 octobre 13

Internet Protocol: IP packet headers. vendredi 18 octobre 13 Internet Protocol: IP packet headers 1 IPv4 header V L TOS Total Length Identification F Frag TTL Proto Checksum Options Source address Destination address Data (payload) Padding V: Version (IPv4 ; IPv6)

More information

Understand the OSI Model

Understand the OSI Model Understand the OSI Model Part 2 Lesson Overview In this lesson, you will learn information about: Frames Packets Segments TCP TCP/IP Model Well-known ports for most-used purposes Anticipatory Set Review

More information

Email Classification Using Data Reduction Method

Email Classification Using Data Reduction Method Email Classification Using Data Reduction Method Rafiqul Islam and Yang Xiang, member IEEE School of Information Technology Deakin University, Burwood 3125, Victoria, Australia Abstract Classifying user

More information

Network Traffic Classification and Demand Prediction

Network Traffic Classification and Demand Prediction Chapter 12 Network Traffic Classification and Demand Prediction Mikhail Dashevskiy and Zhiyuan Luo Reliable classification of network traffic and accurate demand prediction can offer substantial benefits

More information

Computer Networks. Secure Systems

Computer Networks. Secure Systems Computer Networks Secure Systems Summary Common Secure Protocols SSH HTTPS (SSL/TSL) IPSec Wireless Security WPA2 PSK vs EAP Firewalls Discussion Secure Shell (SSH) A protocol to allow secure login to

More information

Training on multiple sub-flows to optimise the use of Machine Learning classifiers in real-world IP networks

Training on multiple sub-flows to optimise the use of Machine Learning classifiers in real-world IP networks Training on multiple sub-flows to optimise the use of Machine Learning classifiers in real-world IP networks Thuy T.T. Nguyen, Grenville Armitage Centre for Advanced Internet Architectures Swinburne University

More information

Traffic Identification Based on Applications using Statistical Signature Free from Abnormal TCP Behavior *

Traffic Identification Based on Applications using Statistical Signature Free from Abnormal TCP Behavior * JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 31, 1669-1692 (2015) Traffic Identification Based on Applications using Statistical Signature Free from Abnormal TCP Behavior * HYUN-MIN AN 1, SU-KANG LEE

More information

Flow Analysis Versus Packet Analysis. What Should You Choose?

Flow Analysis Versus Packet Analysis. What Should You Choose? Flow Analysis Versus Packet Analysis. What Should You Choose? www.netfort.com Flow analysis can help to determine traffic statistics overall, but it falls short when you need to analyse a specific conversation

More information

Transport and Network Layer

Transport and Network Layer Transport and Network Layer 1 Introduction Responsible for moving messages from end-to-end in a network Closely tied together TCP/IP: most commonly used protocol o Used in Internet o Compatible with a

More information

Near Real Time Online Flow-based Internet Traffic Classification Using Machine Learning (C4.5)

Near Real Time Online Flow-based Internet Traffic Classification Using Machine Learning (C4.5) Near Real Time Online Flow-based Internet Traffic Classification Using Machine Learning (C4.5) Abuagla Babiker Mohammed Faculty of Electrical Engineering (FKE) Deprtment of Microelectronics and Computer

More information

Security in IPv6. Basic Security Requirements and Techniques. Confidentiality. Integrity

Security in IPv6. Basic Security Requirements and Techniques. Confidentiality. Integrity Basic Security Requirements and Techniques Confidentiality The property that stored or transmitted information cannot be read or altered by an unauthorized party Integrity The property that any alteration

More information

Bypassing Firewall. @ PISA AGM Theme Seminar 2005. Presented by Ricky Lou Zecure Lab Limited

Bypassing Firewall. @ PISA AGM Theme Seminar 2005. Presented by Ricky Lou Zecure Lab Limited Bypassing Firewall @ PISA AGM Theme Seminar 2005 Presented by Ricky Lou Zecure Lab Limited Firewall Piercing (Inside-Out Attacks) Disclaimer We hereby disclaim all responsibility for the following hacks.

More information

An Overview of Knowledge Discovery Database and Data mining Techniques

An Overview of Knowledge Discovery Database and Data mining Techniques An Overview of Knowledge Discovery Database and Data mining Techniques Priyadharsini.C 1, Dr. Antony Selvadoss Thanamani 2 M.Phil, Department of Computer Science, NGM College, Pollachi, Coimbatore, Tamilnadu,

More information

A Novel Distributed Denial of Service (DDoS) Attacks Discriminating Detection in Flash Crowds

A Novel Distributed Denial of Service (DDoS) Attacks Discriminating Detection in Flash Crowds International Journal of Research Studies in Science, Engineering and Technology Volume 1, Issue 9, December 2014, PP 139-143 ISSN 2349-4751 (Print) & ISSN 2349-476X (Online) A Novel Distributed Denial

More information

Approaching Real-time Network Traffic Classification

Approaching Real-time Network Traffic Classification 22 Approaching Real-time Network Traffic Classification ISSN 1470-5559 Wei Li, Kaysar Abdin, Robert Dann and Andrew Moore RR-06-12 October 2006 Department of Computer Science Approaching Real-time Network

More information

Classifying P2P Activity in Netflow Records: A Case Study on BitTorrent

Classifying P2P Activity in Netflow Records: A Case Study on BitTorrent IEEE ICC 2013 - Communication Software and Services Symposium 1 Classifying P2P Activity in Netflow Records: A Case Study on BitTorrent Ahmed Bashir 1, Changcheng Huang 1, Biswajit Nandy 2, Nabil Seddigh

More information

Analysis of Communication Patterns in Network Flows to Discover Application Intent

Analysis of Communication Patterns in Network Flows to Discover Application Intent Analysis of Communication Patterns in Network Flows to Discover Application Intent Presented by: William H. Turkett, Jr. Department of Computer Science FloCon 2013 January 9, 2013 Port- and payload signature-based

More information

An Implementation Of Network Traffic Classification Technique Based On K-Medoids

An Implementation Of Network Traffic Classification Technique Based On K-Medoids RESEARCH ARTICLE OPEN ACCESS An Implementation Of Network Traffic Classification Technique Based On K-Medoids Dheeraj Basant Shukla*, Gajendra Singh Chandel** *(Department of Information Technology, S.S.S.I.S.T,

More information

A Novel Approach for Network Traffic Summarization

A Novel Approach for Network Traffic Summarization A Novel Approach for Network Traffic Summarization Mohiuddin Ahmed, Abdun Naser Mahmood, Michael J. Maher School of Engineering and Information Technology, UNSW Canberra, ACT 2600, Australia, Mohiuddin.Ahmed@student.unsw.edu.au,A.Mahmood@unsw.edu.au,M.Maher@unsw.

More information

Lecture 28: Internet Protocols

Lecture 28: Internet Protocols Lecture 28: Internet Protocols 15-110 Principles of Computing, Spring 2016 Dilsun Kaynar, Margaret Reid-Miller, Stephanie Balzer Reminder: Exam 2 Exam 2 will take place next Monday, on April 4. Further

More information

Cisco IOS Flexible NetFlow Technology

Cisco IOS Flexible NetFlow Technology Cisco IOS Flexible NetFlow Technology Last Updated: December 2008 The Challenge: The ability to characterize IP traffic and understand the origin, the traffic destination, the time of day, the application

More information

A Study Of Bagging And Boosting Approaches To Develop Meta-Classifier

A Study Of Bagging And Boosting Approaches To Develop Meta-Classifier A Study Of Bagging And Boosting Approaches To Develop Meta-Classifier G.T. Prasanna Kumari Associate Professor, Dept of Computer Science and Engineering, Gokula Krishna College of Engg, Sullurpet-524121,

More information

AIR FORCE INSTITUTE OF TECHNOLOGY

AIR FORCE INSTITUTE OF TECHNOLOGY CLASSIFICATION OF ENCRYPTED WEB TRAFFIC USING MACHINE LEARNING ALGORITHMS THESIS William Charles Barto AFIT-ENG-13-J-11 DEPARTMENT OF THE AIR FORCE AIR UNIVERSITY AIR FORCE INSTITUTE OF TECHNOLOGY Wright-Patterson

More information

Signature-aware Traffic Monitoring with IPFIX 1

Signature-aware Traffic Monitoring with IPFIX 1 Signature-aware Traffic Monitoring with IPFIX 1 Youngseok Lee, Seongho Shin, and Taeck-geun Kwon Dept. of Computer Engineering, Chungnam National University, 220 Gungdong Yusonggu, Daejon, Korea, 305-764

More information

Packet Flow Analysis and Congestion Control of Big Data by Hadoop

Packet Flow Analysis and Congestion Control of Big Data by Hadoop Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 4, Issue. 6, June 2015, pg.456

More information

Assuring Your Business Continuity

Assuring Your Business Continuity Assuring Your Business Continuity Q-Balancer Range Offering Business Continuity, Productivity, and Security Q-Balancer is designed to offer assured network connectivity to small and medium business (SME)

More information

Defending Networks with Incomplete Information: A Machine Learning Approach. Alexandre Pinto alexcp@mlsecproject.org @alexcpsec @MLSecProject

Defending Networks with Incomplete Information: A Machine Learning Approach. Alexandre Pinto alexcp@mlsecproject.org @alexcpsec @MLSecProject Defending Networks with Incomplete Information: A Machine Learning Approach Alexandre Pinto alexcp@mlsecproject.org @alexcpsec @MLSecProject Agenda Security Monitoring: We are doing it wrong Machine Learning

More information

Trends and Differences in Connection-behavior within Classes of Internet Backbone Traffic

Trends and Differences in Connection-behavior within Classes of Internet Backbone Traffic MonNet a project for network and traffic monitoring Trends and Differences in Connection-behavior within Classes of Internet Backbone Traffic Wolfgang John, Sven Tafvelin and Tomas Olovsson Department

More information

Realtime Classification for Encrypted Traffic

Realtime Classification for Encrypted Traffic Realtime Classification for Encrypted Traffic Roni Bar-Yanai 1, Michael Langberg 2,, David Peleg 3,, and Liam Roditty 4 1 Cisco, Netanya, Israel rbaryana@cisco.com 2 Computer Science Division, Open University

More information

International Journal of Computer Science Trends and Technology (IJCST) Volume 2 Issue 3, May-Jun 2014

International Journal of Computer Science Trends and Technology (IJCST) Volume 2 Issue 3, May-Jun 2014 RESEARCH ARTICLE OPEN ACCESS A Survey of Data Mining: Concepts with Applications and its Future Scope Dr. Zubair Khan 1, Ashish Kumar 2, Sunny Kumar 3 M.Tech Research Scholar 2. Department of Computer

More information

Machine Learning Based Encrypted Traffic Classification: Identifying SSH and Skype

Machine Learning Based Encrypted Traffic Classification: Identifying SSH and Skype Machine Learning Based Encrypted Traffic Classification: Identifying SSH and Skype Riyad Alshammari and A. Nur Zincir-Heywood Abstract The objective of this work is to assess the robustness of machine

More information

Top 10 Algorithms in Data Mining

Top 10 Algorithms in Data Mining Top 10 Algorithms in Data Mining Xindong Wu ( 吴 信 东 ) Department of Computer Science University of Vermont, USA; 合 肥 工 业 大 学 计 算 机 与 信 息 学 院 1 Top 10 Algorithms in Data Mining by the IEEE ICDM Conference

More information

Hadoop Technology for Flow Analysis of the Internet Traffic

Hadoop Technology for Flow Analysis of the Internet Traffic Hadoop Technology for Flow Analysis of the Internet Traffic Rakshitha Kiran P PG Scholar, Dept. of C.S, Shree Devi Institute of Technology, Mangalore, Karnataka, India ABSTRACT: Flow analysis of the internet

More information

HMM Profiles for Network Traffic Classification

HMM Profiles for Network Traffic Classification HMM Profiles for Network Traffic Classification Charles Wright, Fabian Monrose and Gerald Masson Johns Hopkins University Information Security Institute Baltimore, MD 21218 Overview Problem Description

More information

Top Top 10 Algorithms in Data Mining

Top Top 10 Algorithms in Data Mining ICDM 06 Panel on Top Top 10 Algorithms in Data Mining 1. The 3-step identification process 2. The 18 identified candidates 3. Algorithm presentations 4. Top 10 algorithms: summary 5. Open discussions ICDM

More information

Getting the Most Out of Your Existing Network A Practical Guide to Traffic Shaping

Getting the Most Out of Your Existing Network A Practical Guide to Traffic Shaping Getting the Most Out of Your Existing Network A Practical Guide to Traffic Shaping Getting the Most Out of Your Existing Network A Practical Guide to Traffic Shaping Executive Summary As organizations

More information

Introducing IBM s Advanced Threat Protection Platform

Introducing IBM s Advanced Threat Protection Platform Introducing IBM s Advanced Threat Protection Platform Introducing IBM s Extensible Approach to Threat Prevention Paul Kaspian Senior Product Marketing Manager IBM Security Systems 1 IBM NDA 2012 Only IBM

More information

Social Media Mining. Data Mining Essentials

Social Media Mining. Data Mining Essentials Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

More information

Data Mining Part 5. Prediction

Data Mining Part 5. Prediction Data Mining Part 5. Prediction 5.1 Spring 2010 Instructor: Dr. Masoud Yaghini Outline Classification vs. Numeric Prediction Prediction Process Data Preparation Comparing Prediction Methods References Classification

More information

A host-based firewall can be used in addition to a network-based firewall to provide multiple layers of protection.

A host-based firewall can be used in addition to a network-based firewall to provide multiple layers of protection. A firewall is a software- or hardware-based network security system that allows or denies network traffic according to a set of rules. Firewalls can be categorized by their location on the network: A network-based

More information

Live Traffic Monitoring with Tstat: Capabilities and Experiences

Live Traffic Monitoring with Tstat: Capabilities and Experiences Live Traffic Monitoring with Tstat: Capabilities and Experiences Maurizio M. Munafò Alessandro Finamore Marco Mellia Michela Meo Dario Rossi WWIC - Luleå, June 3, 2010 Outline Motivations Tstat - TCP STatistic

More information

Mobile Phone APP Software Browsing Behavior using Clustering Analysis

Mobile Phone APP Software Browsing Behavior using Clustering Analysis Proceedings of the 2014 International Conference on Industrial Engineering and Operations Management Bali, Indonesia, January 7 9, 2014 Mobile Phone APP Software Browsing Behavior using Clustering Analysis

More information

Distributed Systems. 2. Application Layer

Distributed Systems. 2. Application Layer Distributed Systems 2. Application Layer Werner Nutt 1 Network Applications: Examples E-mail Web Instant messaging Remote login P2P file sharing Multi-user network games Streaming stored video clips Social

More information

Clustering Big Data. Anil K. Jain. (with Radha Chitta and Rong Jin) Department of Computer Science Michigan State University November 29, 2012

Clustering Big Data. Anil K. Jain. (with Radha Chitta and Rong Jin) Department of Computer Science Michigan State University November 29, 2012 Clustering Big Data Anil K. Jain (with Radha Chitta and Rong Jin) Department of Computer Science Michigan State University November 29, 2012 Outline Big Data How to extract information? Data clustering

More information

Using Data Mining for Mobile Communication Clustering and Characterization

Using Data Mining for Mobile Communication Clustering and Characterization Using Data Mining for Mobile Communication Clustering and Characterization A. Bascacov *, C. Cernazanu ** and M. Marcu ** * Lasting Software, Timisoara, Romania ** Politehnica University of Timisoara/Computer

More information

Monitoring of Tunneled IPv6 Traffic Using Packet Decapsulation and IPFIX

Monitoring of Tunneled IPv6 Traffic Using Packet Decapsulation and IPFIX Monitoring of Tunneled IPv6 Traffic Using Packet Decapsulation and IPFIX Martin Elich 1,3, Matěj Grégr 1,2 and Pavel Čeleda1,3 1 CESNET, z.s.p.o., Prague, Czech Republic 2 Brno University of Technology,

More information

CISC 1600 Introduction to Multi-media Computing

CISC 1600 Introduction to Multi-media Computing CISC 1600 Introduction to Multi-media Computing Spring 2012 Instructor : J. Raphael Email Address: Course Page: Class Hours: raphael@sci.brooklyn.cuny.edu http://www.sci.brooklyn.cuny.edu/~raphael/cisc1600.html

More information

International Journal of Computer Science Trends and Technology (IJCST) Volume 3 Issue 3, May-June 2015

International Journal of Computer Science Trends and Technology (IJCST) Volume 3 Issue 3, May-June 2015 RESEARCH ARTICLE OPEN ACCESS Data Mining Technology for Efficient Network Security Management Ankit Naik [1], S.W. Ahmad [2] Student [1], Assistant Professor [2] Department of Computer Science and Engineering

More information

Finding the real source of Internet crimes

Finding the real source of Internet crimes Finding the real source of Internet crimes Professor Wanlei Zhou Chair of Information Technology and Head School of Information Technology, Deakin University, Melbourne campus at Burwood, Victoria, Australia

More information

An Anomaly-Based Method for DDoS Attacks Detection using RBF Neural Networks

An Anomaly-Based Method for DDoS Attacks Detection using RBF Neural Networks 2011 International Conference on Network and Electronics Engineering IPCSIT vol.11 (2011) (2011) IACSIT Press, Singapore An Anomaly-Based Method for DDoS Attacks Detection using RBF Neural Networks Reyhaneh

More information

A Review of Anomaly Detection Techniques in Network Intrusion Detection System

A Review of Anomaly Detection Techniques in Network Intrusion Detection System A Review of Anomaly Detection Techniques in Network Intrusion Detection System Dr.D.V.S.S.Subrahmanyam Professor, Dept. of CSE, Sreyas Institute of Engineering & Technology, Hyderabad, India ABSTRACT:In

More information

Botnet Detection Based on Degree Distributions of Node Using Data Mining Scheme

Botnet Detection Based on Degree Distributions of Node Using Data Mining Scheme Botnet Detection Based on Degree Distributions of Node Using Data Mining Scheme Chunyong Yin 1,2, Yang Lei 1, Jin Wang 1 1 School of Computer & Software, Nanjing University of Information Science &Technology,

More information

Steven C.H. Hoi School of Information Systems Singapore Management University Email: chhoi@smu.edu.sg

Steven C.H. Hoi School of Information Systems Singapore Management University Email: chhoi@smu.edu.sg Steven C.H. Hoi School of Information Systems Singapore Management University Email: chhoi@smu.edu.sg Introduction http://stevenhoi.org/ Finance Recommender Systems Cyber Security Machine Learning Visual

More information

Innovative, High-Density, Massively Scalable Packet Capture and Cyber Analytics Cluster for Enterprise Customers

Innovative, High-Density, Massively Scalable Packet Capture and Cyber Analytics Cluster for Enterprise Customers Innovative, High-Density, Massively Scalable Packet Capture and Cyber Analytics Cluster for Enterprise Customers The Enterprise Packet Capture Cluster Platform is a complete solution based on a unique

More information

CompTIA Network+ (Exam N10-005)

CompTIA Network+ (Exam N10-005) CompTIA Network+ (Exam N10-005) Length: Location: Language(s): Audience(s): Level: Vendor: Type: Delivery Method: 5 Days 182, Broadway, Newmarket, Auckland English, Entry Level IT Professionals Intermediate

More information

Internet Firewall CSIS 4222. Packet Filtering. Internet Firewall. Examples. Spring 2011 CSIS 4222. net15 1. Routers can implement packet filtering

Internet Firewall CSIS 4222. Packet Filtering. Internet Firewall. Examples. Spring 2011 CSIS 4222. net15 1. Routers can implement packet filtering Internet Firewall CSIS 4222 A combination of hardware and software that isolates an organization s internal network from the Internet at large Ch 27: Internet Routing Ch 30: Packet filtering & firewalls

More information

Measurement of the Usage of Several Secure Internet Protocols from Internet Traces

Measurement of the Usage of Several Secure Internet Protocols from Internet Traces Measurement of the Usage of Several Secure Internet Protocols from Internet Traces Yunfeng Fei, John Jones, Kyriakos Lakkas, Yuhong Zheng Abstract: In recent years many common applications have been modified

More information

Analysis of Network Packets. C DAC Bangalore Electronics City

Analysis of Network Packets. C DAC Bangalore Electronics City Analysis of Network Packets C DAC Bangalore Electronics City Agenda TCP/IP Protocol Security concerns related to Protocols Packet Analysis Signature based Analysis Anomaly based Analysis Traffic Analysis

More information

Bro at 10 Gps: Current Testing and Plans

Bro at 10 Gps: Current Testing and Plans U.S. Department of Energy Bro at 10 Gps: Current Testing and Plans Office of Science Brian L. Tierney Lawrence Berkeley National Laboratory Bro s Use at LBL Operational 24 7 since 1996 Monitors traffic

More information

Network Intrusion Detection and Prevention

Network Intrusion Detection and Prevention Ali A. Ghorbani Wei Lu Mahbod Tavallaee Network Intrusion Detection and Prevention Concepts and Techniques )Spri inger Contents 1 Network Attacks 1 1.1 Attack Taxonomies 2 1.2 Probes 4 1.2.1 IPSweep and

More information

Role of Social Networking in Marketing using Data Mining

Role of Social Networking in Marketing using Data Mining Role of Social Networking in Marketing using Data Mining Mrs. Saroj Junghare Astt. Professor, Department of Computer Science and Application St. Aloysius College, Jabalpur, Madhya Pradesh, India Abstract:

More information

IT services for analyses of various data samples

IT services for analyses of various data samples IT services for analyses of various data samples Ján Paralič, František Babič, Martin Sarnovský, Peter Butka, Cecília Havrilová, Miroslava Muchová, Michal Puheim, Martin Mikula, Gabriel Tutoky Technical

More information

HMC: A Novel Mechanism for Identifying Encrypted P2P Thunder Traffic

HMC: A Novel Mechanism for Identifying Encrypted P2P Thunder Traffic HMC: A Novel Mechanism for Identifying Encrypted P2P Thunder Traffic Chenglong Li* and Yibo Xue Department of Computer Science & Techlogy, Research Institute of Information Techlogy (RIIT), Tsinghua University,

More information

IPFIX IE Extensions for DDoS Attack Detection draft-fu-dots-ipfix-extension-01

IPFIX IE Extensions for DDoS Attack Detection draft-fu-dots-ipfix-extension-01 IPFIX IE Extensions for DDoS Attack Detection draft-fu-dots-ipfix-extension-01 Tianfu Fu futianfu@huawei.com Dacheng Zhang dacheng.zdc@alibaba-inc.com Liang Xia (Frank) frank.xialiang@huawei.com Min Li

More information

International Journal of Recent Trends in Electrical & Electronics Engg., Feb. 2014. IJRTE ISSN: 2231-6612

International Journal of Recent Trends in Electrical & Electronics Engg., Feb. 2014. IJRTE ISSN: 2231-6612 Spoofing Attack Detection and Localization of Multiple Adversaries in Wireless Networks S. Bhava Dharani, P. Kumar Department of Computer Science and Engineering, Nandha College of Technology, Erode, Tamilnadu,

More information

Tackling Network Management Problems using Machine Learning Techniques

Tackling Network Management Problems using Machine Learning Techniques Tackling Network Management Problems using Machine Learning Techniques Ph.D. Thesis Proposal Yu Jin Advisor: Professor Zhi-Li Zhang 1 Contents 1 Introduction 3 1.1 Related Work..........................................

More information

Networking Basics and Network Security

Networking Basics and Network Security Why do we need networks? Networking Basics and Network Security Shared Data and Functions Availability Performance, Load Balancing What is needed for a network? ISO 7-Layer Model Physical Connection Wired:

More information

Cyber Essentials. Test Specification

Cyber Essentials. Test Specification Cyber Essentials Test Specification Contents Scope of the Audit...2 Assumptions...3 Success Criteria...3 External systems...4 Required tests...4 Test Details...4 Internal systems...7 Tester pre-requisites...8

More information

Large-Scale TCP Packet Flow Analysis for Common Protocols Using Apache Hadoop

Large-Scale TCP Packet Flow Analysis for Common Protocols Using Apache Hadoop Large-Scale TCP Packet Flow Analysis for Common Protocols Using Apache Hadoop R. David Idol Department of Computer Science University of North Carolina at Chapel Hill david.idol@unc.edu http://www.cs.unc.edu/~mxrider

More information

Critical Considerations in Choosing a Network Firewall. Version 5.4.3 July 2014

Critical Considerations in Choosing a Network Firewall. Version 5.4.3 July 2014 Critical Considerations in Choosing a Network Firewall Version 5.4.3 July 2014 Why today s Firewalls are broken Visibility No visibility into user behavior No control over applications Manageability No

More information

A Study of Technology in Firewall System

A Study of Technology in Firewall System 2011 IEEE Symposium on Business, Engineering and Industrial Applications (ISBEIA), Langkawi, Malaysia A Study of Technology in Firewall System Firkhan Ali Bin Hamid Ali Faculty of Science Computer & Information

More information

Chapter 6. The stacking ensemble approach

Chapter 6. The stacking ensemble approach 82 This chapter proposes the stacking ensemble approach for combining different data mining classifiers to get better performance. Other combination techniques like voting, bagging etc are also described

More information

On detecting Internet-based criminal threats with XplicoAlerts: Current design and next steps

On detecting Internet-based criminal threats with XplicoAlerts: Current design and next steps On detecting Internet-based criminal threats with XplicoAlerts: Current design and next steps Carlos Gacimartín, José Alberto Hernández, Manuel Urueña, David Larrabeiti Universidad Carlos III de Madrid,

More information