Who is Generating all This Traffic?

Size: px

Start display at page:

Download "Who is Generating all This Traffic?"

Shon Brown
10 years ago
Views:

1 Who is Generating all This Traffic? Network Monitoring in Practice Luca Deri

2 Who s ntop.org? Started in 1998 as open-source monitoring project for developing an easy to use passive monitoring application. Several project spin-off Accelerated packet capture 1 and 10 Gbit packet capture NetFlow/sFlow probes Peer-to-Peer VPN 2

3 ntop.org at a Glance 3

4 Who is Using ntop Products? International Domestic 4

5 Some ntop Partners 5

6 Some Common Monitoring Questions [1/2] Top N talkers (those who transmit most traffic). Top N conversations (the host pairs that transmit most traffic between each other). Top N Applications (e.g. SAP is using 70% of the available bandwidth). Data volume per entity basis (link, location, region/subnet, class of users/cluster). 6

Top N conversations (the host pairs that transmit most traffic between each other).

7 Some Common Monitoring Questions [2/2] Data volume and rates per AS (e.g. do we need to sign a new peering contract?). QoS marking per application or entity basis (e.g. does BGP reports us that we re sending the traffic on the optimal path?). Reports about traffic we don t expect to see on the network (e.g. why host X is sending IPX packets although we speak pure IP?). 7

). Reports about traffic we don t expect to see on the network (e.g.

8 Some Challenges SNMP is good for element management (e.g. router and server monitoring) but poor for traffic measurement. Not all routers/switches speak NetFlow/ sflow: we need to deploy soft probes. 1 and 10 Gbit networks can produce a lot of monitoring data: our monitoring apps must be able to handle all this traffic. 8

1 and 10 Gbit networks can produce a lot of monitoring data: our monitoring apps

9 Networks are Changing [1/2] Wireless Edge Intranet Internet Central Mgmt Core Distribution Wired Edge 9

10 Networks are Changing [2/2] Without edge control there s no real network control. Central traffic monitoring isn t enough anymore: not all traffic flows through the center. Edge equipment is often very basic and it means that there s no visibility at the edge: this about this before purchasing your network equipement. 10

11 Typical Monitoring Deployment: LAN 11

12 Typical Monitoring Deployment: Internet Traffic 12

13 Typical Monitoring Deployment: Cloud and Intra-VM Monitoring 13

14 Some Lessons Learnt In order to monitoring the traffic we need to deploy a probe where the traffic is flowing. We need to make sure we can handle both NetFlow and sflow if we want to have complete network visibility. Cloud computing and server virtualization push us to monitor in-vm virtual networks. 14

We need to make sure we can handle both NetFlow and sflow if we want to

15 What if we Upgrade to 10 Gbit? Be prepared to: handle 10x as much traffic as with 1 Gbit. be able to handle encapsulations (GRE, GTP) and tagging (MPLS, VLANs) in your monitoring software. buy 10 Gbit probes (costly and rare). Good news: 10 Gbit adapters are now commodity (< Euro/port). nprobe support 10 Gbit already. 15

be able to handle encapsulations (GRE, GTP) and tagging (MPLS, VLANs) in your

16 How can ntop help me? Central network monitoring console already integrated into Würth-Phoenix NetEye. Software NetFlow/sFlow probes that can be deployed across the network. 10 Gbit packet capture acceleration and filtering using nprobe. Ability to handle billion of flows with subsecond response time. 16

17 What is ntop? 17

18 Network Inventory 18

19 Traffic Trends 19

20 Host Health 20

21 VoIP Support 21

22 ntop Scripting using Python HTTP(S) <html> </body>... </body> </html> handlepythonhttprequest(...) 22

23 Where is my Traffic Going To? 23

24 nprobe: IPFIX/NetFlow Soft Probe sflow NetFlow Packet Capture nprobe Flow Export Data Dump Würth-Phoenix NetEye Raw Files / MySQL / SQLite / FastBit 24

25 ntop on-the-go [1/2] Apple iphone is commonly used as mobile web pad. Accessing ntop information in mobility is often required by network administrators. The ntop web GUI can be accessed via Apple Safari, however a tighten and more comprehensive interface was necessary. Ability to control several ntop instances via a single device. Access traffic information as well as configuration information. Available (soon) on the AppleStore. HTTP(S) JSON ntop 25

26 ntop on-the-go [2/2] 26

27 nprobe: Main Features Ability to keep up with Gbit speeds on Ethernet networks handling thousand of packets per second without packet sampling on commodity hardware. Support for major OS including Unix, Windows and MacOS X. Full NetFlow v9/ipfix support V9 extensions: payload, network/application latency, VoIP, RTP. Ability to extend the probe with user-written plugins. BGP Peering with the router for full AS monitoring. Würth-Phoenix NetEye can be used as collector for nprobe. 27

28 nprobe: Network Performance and Response Time 28

29 nprobe: Network Awareness 29

30 Handling Billion Flows nprobe+fastbit vs MySQL Query MySQL nprobe + FastBit Q Q Q Q Q nprobe+fastbit vs nfdump nprobe+fastbit nfdump All measurements are in seconds 30

31 How can I Improve my Internet Presence? 31

32 Interactive Data Search 32

33 10 Gbit Wire-rate Traffic Monitoring with Commodity Hardware 33

High-speed Network and Service Monitoring. Luca Deri <[email protected]>

High-speed Network and Service Monitoring Luca Deri Who s ntop.org? Started in 1998 as open-source monitoring project for developing an easy to use passive monitoring application. Several