Passive Measurement in CSTNET

Transcription

1 Passive Measurement in CSTNET Chunjing Han Aug CSTNET, CNIC

2 Topics 1. Passive measurement systems introduction in CSTNET 2. Large - scale distributed traffic analysis system in IPv6

3 1, Passive measurement systems introduction in CSTNET

4 Background introduction IPv4 address will soon be depleted the Asia-Pacific region(april 2011) In the last three years, the number of ipv4 addresses assigned has a downward trend in China. This figure comes from CNNIC We need migrate to the IPv6 network and do not wait

5 Background introduction China Next Generation Internet (CNGI) IPv6 high-speed network IPv4-IPv6 dual network and support IPv6 in the whole CSTNET Traffic increasing continuously, congestion link, IPv6 ARP attack No-commercial IPv6 capable network management and traffic monitoring system We need monitor IPv6 traffic analyzers

6 Passive measurement systems CNMS Cloud network management system LDTM Large - scale distributed network monitoring system

7 Large - scale distributed traffic analysis system

8 LDTM( Large scale distribution traffic monitor) Netflow and DPI technology Goal: IPv6 support and can monitor the backbone and boundary of CSTNET Using cloud plus probe to implement LDTM Cloud: collect, process, and merge the raw flows, storage the aggregated records, and provide visual service. probe: create the improved Netflow packets and send network data to Cloud.

9 One cloud processing center, multi probes Deployment of LDTM This is typical implementation of multi - tenant cloud services model the collecting servers web servers, database, storage device.

10 Multiple layers for traffic collector Software architecture of LDTM

11 Flexible deployments Software deployment

12 An uniform CMDB Distributed collector design model Distributed collectors regularly synchronize the configuration from the CMDB, guarantee the correct monitoring scope of links and monitor objects Using the multi-process data loader tools and data dispatch map, to load big block data into the database cluster correctly and quickly

13 Key technology Property of IP address Continent, country, city, organization, customer information Enrich this information to each raw flows real-time Advance a IPv4-IPv6 attribution information searching algorithm based Patricia Tree 128 bits IPv6 address Provide uniform interface for IPv4&IPv6 A root node and two sub-tree Left IPv4 sub-tree Right IPv6 sub-tree

14 Key technology Aggregate huge raw flow recording Time granularity: min, hour, day, week, month Function aggregation: continent, country, protocol, application, host, session, packet size Advance the visual link concept: merge the traffic of multiple links as a visual link Long-term storing the raw flow records Create the raw flow file every five minutes Put it in the exclusive storage resource and do not occupy the space of the collecting servers.

15 Key technology Storage technology improvement for massive data Parallel database and database clusters Min granularity record small files stand-alone database Hour, day, week and month granularity record cluster big files database Cluster GBase 8a MPP GCluster sg01 sg02 sg03 sg 千 兆 局 域 网 数 据 分 发 机 &8a 单 机 & 后 台 服 务 器

16 Features list Traffic weather map Distribution analysis Top N analysis Configuration Overview volume Top host Collecting Application Top session Exporter Organization Top protocol Link Region Continents Packe size Raw flows research IP utilization IP location Visual link Monitor object Application Traffic billing

17 Traffic weather map A geo-view traffic distribution Two levels of zoom: continent, region The IPv6 and IPv4 view separately Traffic of Top IP session, organization and region distribution,by using threedimensional flex model

18 Traffic distibution and Top N Organization distribution Top IPv6 address Top IP session Application distribution

19 Raw flow analysis Our can display a back trace of the current and history transferring flows. Enrich the raw flow using the GeoIP and institutes information of CSTNET. For IPv6, information of GeoIP is very limit. Provide the filter configuration of IP,port, protocol and link Provide downloading function of the raw flow records

20 Active IP address statistic Active IP address statistic is very important to evaluate the IP utilization. A statistics once a month---the active and inactive IP address We have stored the result in to the database and have not a good visualization for IPv6, due to the long address format and huge number of inactive IP address A time-consuming work

21 Traffic billing For the ISP and customer, LDTM provides a traffic billing inquiry of each customer in every day and month. For a customer, LDTM can query a special IP address traffic, including every application traffic and detail traffic records.

22 Providing a IP location service IP address location

23 Future work The active and passive measurement of IPv6 performance. Perfosonar Performance Probe passive data The active IPv6 address statistic A good algorithm to quickly get the result Suitable visualization, huge IPv6 address number Research of measurement method of IPv6 transition Dual stack Translation Tunnel THANK YOU