NfSen Plugin Supporting The Virtual Network Monitoring Vojtěch Krmíček krmicek@liberouter.org Pavel Čeleda celeda@ics.muni.cz Jiří Novotný novotny@cesnet.cz
Part I Monitoring of Virtual Network Environments in FEDERICA Network Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 2 / 24
Virtual World of FEDERICA Network Virtualization several virtual links inside one physical link. Virtual nodes + virtual links virtual network infrastructure. VN2 VP VN3 Virtual Slice N.... Virtual Slice II VN1 VP VP VN2 VP VP VP VN4 VP VN3 Virtual Networks (slices) VN1 VN4 VN3 Virtual Slice I VN1 VP VP VN4 VP Virtual Nodes GARR IT DFN DE CESNET CZ PSNC PL Physical Infrastructure Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 3 / 24
VLAN Networks and NetFlow IEEE 802.1Q also known as VLAN tagging. Multiple bridged networks share the same physical link. Default NetFlow record doesn t contain VLAN tag field. We need to add VLAN tag information to the flow record. Physical Line VLAN 1203 VLAN 1202 VLAN 1201 (4B1h) Ethernet Frame 00:0C:29:11:79:C3 Destination MAC 00:0C:29:62:C7:EC Source MAC 81 00 04 B1 802.1Q Header C0 A8 01 03 Src IP C0 A8 01 01 Dst IP Payload NetFlow Record extended with VLAN field 30.276 Duration TCP 192.168.1.3:2545 Proto Src IP : Port 192.168.1.1:80 Dst IP : Port.A.R.. Flags 9240 Packets 220 Bytes 1201 VLAN Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 4 / 24
NetFlow VLAN Support in FEDERICA Project NetFlow VLAN Issues NetFlow version 5 doesn t support VLAN tags. NetFlow version 9 defines VLAN tags (see RFC 3954). Routers and probes doesn t support VLAN export. NetFlow collectors doesn t support VLAN handling. Proposed Solution Dedicated FlowMon probes with VLAN support. We have added VLAN tag information as DST_AS field. Flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Intf VLAN 06:49:55.049 299.996 ICMP 192.168.3.2:0 -> 192.168.3.1:0.0 969 1.3 M 8 1203 06:49:55.657 299.997 ICMP 192.168.3.1:0 -> 192.168.3.2:8.0 969 1.3 M 9 1203 06:51:10.255 299.752 ICMP 192.168.3.2:0 -> 192.168.1.1:8.0 968 1.3 M 8 1203 06:51:10.255 299.752 ICMP 192.168.1.1:0 -> 192.168.3.2:0.0 968 1.3 M 9 1203 06:51:36.593 299.824 ICMP 192.168.1.3:0 -> 192.168.1.1:0.0 1936 2.6 M 6 1201 06:51:37.189 299.848 ICMP 192.168.1.1:0 -> 192.168.1.3:8.0 1936 2.6 M 7 1201 VLAN tag information is crucial for virtual circuits monitoring! Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 5 / 24
Architecture of NetFlow Monitoring System DFN DE PSNC PL Virtual Node Cloud GARR IT CESNET CZ Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 6 / 24
Architecture of NetFlow Monitoring System Probe Probe DFN DE PSNC PL Virtual Node Cloud GARR IT CESNET CZ Probe Probe Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 6 / 24
Architecture of NetFlow Monitoring System Probe Probe DFN DE PSNC PL Data Analysis Center Virtual Node Cloud GARR IT CESNET CZ Probe Probe Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 6 / 24
Architecture of NetFlow Monitoring System DFN DE PSNC PL Virtual Node Cloud GARR IT CESNET CZ Current System Deployment Probe Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 6 / 24
Single Node Monitoring Using Tapped Traffic Copper/Multimode/ Monomode TAP Copper/Multimode/ Monomode TAP Copper/Multimode/ Monomode TAP Copper/Multimode/ Monomode TAP FlowMon Probe 8000 Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 7 / 24
Block Structure of NetFlow Monitoring System FlowMon Probe 8000 Web Interface NfSen Collector Plugins Backend Frontend Processing and Presentation Layer NetFlow Data Storage NFDUMP Toolset Collector Layer packets flows FlowMon Exporter Fiber TAP Packet Data Inside VLANs FEDERICA Traffic FlowMon Exporter flows packets Fiber TAP flows FlowMon Exporter packets Fiber TAP NetFlow Generation Layer Network Layer Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 8 / 24
Part II NfSen Default Collector Features Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 9 / 24
NetFlow Processing with NFDUMP Available Flow Statistics Raw NetFlow data. Top N statistics. Flow filtering (via IP addresses, protocols, VLAN,... ). Flow aggregation (IP addresses, protocols, VLAN,... ). VLAN tags and interface numbers. Flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Intf VLAN 06:49:55.049 299.996 ICMP 192.168.3.2:0 -> 192.168.3.1:0.0 969 1.3 M 8 1203 06:49:55.657 299.997 ICMP 192.168.3.1:0 -> 192.168.3.2:8.0 969 1.3 M 9 1203 06:51:10.255 299.752 ICMP 192.168.3.2:0 -> 192.168.1.1:8.0 968 1.3 M 8 1203 06:51:10.255 299.752 ICMP 192.168.1.1:0 -> 192.168.3.2:0.0 968 1.3 M 9 1203 06:51:36.593 299.824 ICMP 192.168.1.3:0 -> 192.168.1.1:0.0 1936 2.6 M 6 1201 06:51:37.189 299.848 ICMP 192.168.1.1:0 -> 192.168.1.3:8.0 1936 2.6 M 7 1201 06:54:55.355 299.997 ICMP 192.168.3.2:0 -> 192.168.3.1:0.0 969 1.3 M 8 1203 06:54:55.964 299.996 ICMP 192.168.3.1:0 -> 192.168.3.2:8.0 969 1.3 M 9 1203 06:56:10.317 299.781 ICMP 192.168.1.1:0 -> 192.168.3.2:0.0 968 1.3 M 9 1203 06:56:10.317 299.781 ICMP 192.168.3.2:0 -> 192.168.1.1:8.0 968 1.3 M 8 1203 06:56:36.649 299.916 ICMP 192.168.1.3:0 -> 192.168.1.1:0.0 1936 2.6 M 6 1201 06:56:37.245 299.941 ICMP 192.168.1.1:0 -> 192.168.1.3:8.0 1936 2.6 M 7 1201 06:57:01.952 0.000 UDP 194.132.52.193:138 -> 194.132.52.195:138 2 513 5 1200 Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 10 / 24
NfSen Profiles The profile is defined by its name, type and profile filter(s). The profile applies to the graphical and to the numerical view. The profiles are set manually by network administrator. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 11 / 24
NfSen Alerts The alerts allow to execute actions based on conditions. Triggered alert typically sends an email to administrator. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 12 / 24
NfSen Plugins The plugins allow to extend NfSen with new functionality. The plugins run automated tasks every 5 minutes. The plugins allow display any results of NetFlow measurement. Plugin Report Automatic run every 5 min Notification.pm Register Output Email nfsen.conf Web Interface Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 13 / 24
Part III NfSen Plugin Supporting The Virtual Network Monitoring Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 14 / 24
Plugin Motivation and Goals Plugin Motivation No VLAN monitoring tool in FEDERICA. No analysis of VLAN traffic. No visualization of VLAN traffic. But we need to observe traffic in slices. Plugin Goals Detailed and long-term VLAN stats. Regular reporting to email. Visualization of VLAN data. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 15 / 24
Plugin Architecture Plugin consists of three components: Plugin frontend, plugin backend and database. FlowMon Probe 8000 NfSen Collector Graphs Stats Reports Plugin Frontend NfSen WWW Frontend Reports DB Update Plugin Backend DB Query NetFlow Data Storage PostgreSQL Database flows FlowMon Exporter FlowMon Exporter flows FlowMon Exporter packets packets Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 16 / 24
Plugin Frontend - VLAN Overview Main navigation bar with stats, reporting and about. Traffic visualization divided by flows, packets and traffic. Detailed traffic statistics. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 17 / 24
Plugin Frontend - VLAN Overview Main navigation bar with stats, reporting and about. Traffic visualization divided by flows, packets and traffic. Detailed traffic statistics. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 17 / 24
Plugin Frontend - VLAN Overview Main navigation bar with stats, reporting and about. Traffic visualization divided by flows, packets and traffic. Detailed traffic statistics. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 17 / 24
Plugin Frontend - VLAN Details I Graph visualization divided by protocols. Detailed traffic statistics by protocols. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 18 / 24
Plugin Frontend - VLAN Details I Graph visualization divided by protocols. Detailed traffic statistics by protocols. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 18 / 24
Plugin Frontend III - VLAN Details II Protocol statistics for top 5 ports in chosen VLAN. Protocol statistics for top 5 talkers in chosen VLAN. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 19 / 24
Plugin Frontend III - VLAN Details II Protocol statistics for top 5 ports in chosen VLAN. Protocol statistics for top 5 talkers in chosen VLAN. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 19 / 24
Plugin Frontend - VLAN Reporting I Possibility to add a new email address for reporting. Listing of existing email addresses for reporting. Activation/inactivation of particular email address. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 20 / 24
Plugin Frontend - VLAN Reporting I Possibility to add a new email address for reporting. Listing of existing email addresses for reporting. Activation/inactivation of particular email address. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 20 / 24
Plugin Frontend - VLAN Reporting I Possibility to add a new email address for reporting. Listing of existing email addresses for reporting. Activation/inactivation of particular email address. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 20 / 24
Plugin Frontend - VLAN Reporting II Example of the email report. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 21 / 24
Part IV Conclusion Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 22 / 24
Conclusion NetFlow Based Monitoring Monitoring system delivers detailed traffic information. Used tools support NetFlow with full VLAN processing. NetFlow data are provided via NfSen collector. NfSen Plugin Supporting VLAN Monitoring Provides detailed statistics about VLAN traffic. Gives the graphical representations of the traffic structure. Allows regular reporting to the email. Generally supports monitoring of VLAN networks. Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 23 / 24
Thank You For Your Attention www.fp7-federica.eu NfSen Plugin Supporting The Virtual Network Monitoring Vojtěch Krmíček krmicek@liberouter.org Pavel Čeleda celeda@ics.muni.cz Jiří Novotný novotny@cesnet.cz FlowMon Probe Krmíček, Čeleda, Novotný NfSen Plugin Supporting The Virtual Network Monitoring 24 / 24