1 Passive Measurement in CSTNET Chunjing Han Aug CSTNET, CNIC
2 Topics 1. Passive measurement systems introduction in CSTNET 2. Large - scale distributed traffic analysis system in IPv6
3 1, Passive measurement systems introduction in CSTNET
4 Background introduction IPv4 address will soon be depleted the Asia-Pacific region(april 2011) In the last three years, the number of ipv4 addresses assigned has a downward trend in China. This figure comes from CNNIC We need migrate to the IPv6 network and do not wait
5 Background introduction China Next Generation Internet (CNGI) IPv6 high-speed network IPv4-IPv6 dual network and support IPv6 in the whole CSTNET Traffic increasing continuously, congestion link, IPv6 ARP attack No-commercial IPv6 capable network management and traffic monitoring system We need monitor IPv6 traffic analyzers
6 Passive measurement systems CNMS Cloud network management system LDTM Large - scale distributed network monitoring system
7 Large - scale distributed traffic analysis system
8 LDTM( Large scale distribution traffic monitor) Netflow and DPI technology Goal: IPv6 support and can monitor the backbone and boundary of CSTNET Using cloud plus probe to implement LDTM Cloud: collect, process, and merge the raw flows, storage the aggregated records, and provide visual service. probe: create the improved Netflow packets and send network data to Cloud.
9 One cloud processing center, multi probes Deployment of LDTM This is typical implementation of multi - tenant cloud services model the collecting servers web servers, database, storage device.
10 Multiple layers for traffic collector Software architecture of LDTM
11 Flexible deployments Software deployment
12 An uniform CMDB Distributed collector design model Distributed collectors regularly synchronize the configuration from the CMDB, guarantee the correct monitoring scope of links and monitor objects Using the multi-process data loader tools and data dispatch map, to load big block data into the database cluster correctly and quickly
13 Key technology Property of IP address Continent, country, city, organization, customer information Enrich this information to each raw flows real-time Advance a IPv4-IPv6 attribution information searching algorithm based Patricia Tree 128 bits IPv6 address Provide uniform interface for IPv4&IPv6 A root node and two sub-tree Left IPv4 sub-tree Right IPv6 sub-tree
14 Key technology Aggregate huge raw flow recording Time granularity: min, hour, day, week, month Function aggregation: continent, country, protocol, application, host, session, packet size Advance the visual link concept: merge the traffic of multiple links as a visual link Long-term storing the raw flow records Create the raw flow file every five minutes Put it in the exclusive storage resource and do not occupy the space of the collecting servers.
15 Key technology Storage technology improvement for massive data Parallel database and database clusters Min granularity record small files stand-alone database Hour, day, week and month granularity record cluster big files database Cluster GBase 8a MPP GCluster sg01 sg02 sg03 sg 千 兆 局 域 网 数 据 分 发 机 &8a 单 机 & 后 台 服 务 器
16 Features list Traffic weather map Distribution analysis Top N analysis Configuration Overview volume Top host Collecting Application Top session Exporter Organization Top protocol Link Region Continents Packe size Raw flows research IP utilization IP location Visual link Monitor object Application Traffic billing
17 Traffic weather map A geo-view traffic distribution Two levels of zoom: continent, region The IPv6 and IPv4 view separately Traffic of Top IP session, organization and region distribution,by using threedimensional flex model
18 Traffic distibution and Top N Organization distribution Top IPv6 address Top IP session Application distribution
19 Raw flow analysis Our can display a back trace of the current and history transferring flows. Enrich the raw flow using the GeoIP and institutes information of CSTNET. For IPv6, information of GeoIP is very limit. Provide the filter configuration of IP,port, protocol and link Provide downloading function of the raw flow records
20 Active IP address statistic Active IP address statistic is very important to evaluate the IP utilization. A statistics once a month---the active and inactive IP address We have stored the result in to the database and have not a good visualization for IPv6, due to the long address format and huge number of inactive IP address A time-consuming work
21 Traffic billing For the ISP and customer, LDTM provides a traffic billing inquiry of each customer in every day and month. For a customer, LDTM can query a special IP address traffic, including every application traffic and detail traffic records.
22 Providing a IP location service IP address location
23 Future work The active and passive measurement of IPv6 performance. Perfosonar Performance Probe passive data The active IPv6 address statistic A good algorithm to quickly get the result Suitable visualization, huge IPv6 address number Research of measurement method of IPv6 transition Dual stack Translation Tunnel THANK YOU
Network Monitoring and Traffic Analysis in CSTNET Chunjing Han Aug. 2013 CSTNET, CNIC Topics 1. The background of network monitoring 2. Network monitoring protocols and related tools 3. Network monitoring
Institut für Technische Informatik und Kommunikationsnetze Kirila Adamova Anomaly Detection with Virtual Service Migration in Cloud Infrastructures Master Thesis 263-8-L October 22 to March 23 Tutor: Dr.
Toward a lightweight framework for monitoring public clouds Kun Ma, Runyuan Sun, Ajith Abraham Shandong Provincial Key Laboratory of Network Based Intelligent Computing University of Jinan, Jinan, China
Network Monitoring Based on IP Data Flows Best Practice Document Produced by CESNET led working group on Network monitoring (CBPD131) Authors: Martin Žádník March 2010 TERENA 2010. All rights reserved.
McAfee NGFW Reference Guide for Firewall/VPN Role 5.7 NGFW Engine in the Firewall/VPN Role Legal Information The use of the products described in these materials is subject to the then current end-user
http://www.cse.wustl.edu/~jain/cse567-06/ftp/net_traffic_monitors2/ind... 1 of 11 SNMP and Beyond: A Survey of Network Performance Monitoring Tools Paul Moceri, email@example.com Abstract The growing
Developing Fleet and Asset Tracking Solutions with Web Maps Introduction Many organizations have mobile field staff that perform business processes away from the office which include sales, service, maintenance,
Magic Quadrant for Security Information and Event Management 25 June 2014 ID:G00261641 Analyst(s): Kelly M. Kavanagh, Mark Nicolett, Oliver Rochford VIEW SUMMARY EVIDENCE Broad adoption of SIEM technology
CHAPTER 1 LAN Design Objectives Upon completion of this chapter, you will be able to answer the following questions: How does a hierarchical network support the voice, video, and data needs of a small-
Software Defined Network Support for Real Distributed Systems Chen Liang Project for Reading and Research Spring 2012, Fall 2012 Abstract Software defined network is an emerging technique that allow users
Unified Security Monitoring Best Practices June 8, 2011 (Revision 1) Copyright 2011. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of
UNIVERSITY OF OSLO Department of Informatics Performance Measurement of Web Services Linux Virtual Server Muhammad Ashfaq Oslo University College May 19, 2009 Performance Measurement of Web Services Linux
WHITE PAPER 1ntroduction... 2 Zenoss Enterprise: Functional Overview... 3 Zenoss Architecture: Four Tiers, Model-Driven... 6 Issues in Today s Dynamic Datacenters... 12 Summary: Five Ways Zenoss Enterprise
VMware vcloud Networking and Security Overview Networks and Security for Virtualized Compute Environments WHITE PAPER Overview Organizations worldwide have gained significant efficiency and flexibility
Building A Better Network Monitoring System A report submitted in fulfillment of the requirements for the degree of Bachelor of Computing and Mathematical Sciences with Honours at The University of Waikato
IBM i on Power - Performance FAQ February 5, 2013 IBM Corporation Table of Contents 1 Introduction 8 1.1 Purpose of this document 8 1.2 Overview 8 1.3 Document Responsibilities 8 2 What Is Performance?
Redefining Microsoft SQL Server Data Management APRIL Actifio 11, 2013 PAS Specification Table of Contents Introduction.... 3 Background.... 3 Virtualizing Microsoft SQL Server Data Management.... 4 Virtualizing
Design and implementation of an MPLS based load balancing architecture for Web switching Radu Dragos, Sanda Dragos & Martin Collier School of Electronic Engineering - DCU Switching and Systems Laboratory
PRTG NETWORK MONITOR Installed in Seconds. Configured in Minutes. Master Your Network for Years to Come. PRTG Network Monitor is... NETWORK MONITORING Network monitoring continuously collects current status
Infosys Labs Briefings VOL 11 NO 1 2013 Big Data: Testing Approach to Overcome Quality Challenges By Mahesh Gudipati, Shanthi Rao, Naju D. Mohan and Naveen Kumar Gajja Validate data quality by employing
Performance Evaluation of Online Backup Cloud Storage Xiao Zhang School of Computer Science Northwestern Polytechnical University 127 Youyi xi Road, Xi an Shaanxi China firstname.lastname@example.org Wan Guo School
2014 Implementing a Data Warehouse on AWS in a Hybrid Environment INFORMATICA CLOUD AND AMAZON REDSHIFT Contents Abstract... 3 Before You Get Started... 4 Overview of Informatica Cloud... 5 Amazon Redshift