APPLICATION NOTE MONITORING NETWORK TRAFFIC USING sflow TECHNOLOGY ON EX SERIES ETHERNET SWITCHES Exporting sflow to Collectors Through a Separate Virtual Routing Instance Copyright 2010, Juniper Networks, Inc.
Table of Contents Table of Figures Introduction......................................................................................... 1 Scope.............................................................................................. 1 Design Considerations................................................................................ 1 sflow Overview of Juniper Networks EX Series Ethernet Switches.......................................... 1 Packet Flow.................................................................................... 1 Packet Flow Sampling........................................................................... 1 Packet Flow Record............................................................................. 1 Counter Sampling.............................................................................. 1 sflow Agent.................................................................................... 1 sflow Collector................................................................................. 1 Description and Deployment Scenario................................................................... 2 sflow Implementation on EX Series Switches........................................................ 2 Traffic Monitoring Using sflow on EX Series Switches................................................. 2 Exporting sflow-sampled Records to a Remote Collector in a Different Virtual Routing Instance............. 6 Summary.......................................................................................... 15 About Juniper Networks............................................................................. 15 Figure 1: sflow monitoring in a regular network......................................................... 2 Figure 2: sflow sampled packets sent to the collector..................................................... 4 Figure 3: Packet header of the sflow sampled packets................................................... 5 Figure 4: Details of the sflow sampled packets.......................................................... 6 Figure 5: sflow monitoring with a collector in separate virtual routing instance.............................. 7 Figure 6: sflow sampled packets sent to the collector 7.0.0.10............................................ 11 Figure 7: sflow sampled packets sent to the collector 1.0.0.10............................................ 12 Figure 8: sflow sampled packets sent to the collector 8.0.0.10............................................ 12 Figure 9: Details of the sflow sampled packets sent to collector 8.0.0.10................................... 13 Figure 10: Details of the sflow interface counters sent to collector 8.0.0.10................................. 14 ii Copyright 2010, Juniper Networks, Inc.
Introduction The sflow (RFC 3176) technology is designed for monitoring high-speed switched or routed networks and provides visibility into the type of network traffic to help detect anomalies in traffic flows. This statistical sampling-based network monitoring technology samples network packets and sends the samples to a monitoring station, where it gives the network administrator visibility into network behavior. Scope This application note will describe how sflow technology can be deployed on the Juniper Networks EX3200 Ethernet Switches and EX4200 Ethernet Switches in a typical switched or routed network environment. It will also discuss how to export the sflow sampling data records to remote monitoring collectors through network ports in a separate virtual routing instance on Juniper Networks EX Series Ethernet Switches. Design Considerations The results of the tests described below are based on the use of Juniper Networks EX4200 Ethernet Switches. The EX3200 Ethernet Switches could be substituted for the EX4200 switches. sflow Overview of Juniper Networks EX Series Ethernet Switches Before discussing details about deploying sflow on EX Series switches, it would be useful to first provide an overview of basic sflow terminologies. Packet Flow A packet flow is defined as a set of packets moving through a networking device such as a switch or router. Packets are received on an ingress interface, and a switching or routing decision is made for the egress interface. Packet Flow Sampling Packet flow sampling refers to arbitrarily choosing some packets out of a specified number, reading the first 128 bytes, and exporting the sampled datagram for meaningful analysis. Packet Flow Record The packet flow record contains two kinds of information: first, some basic information about the sample datagram such as encapsulation and header information; and second, information related to selection of the forwarding path. Counter Sampling Counter sampling performs periodic, time-based sampling or polling of counters associated with an interface enabled for sflow. Interface statistics from the counter record are gathered, and the agent constructs a datagram which it sends to the collectors, depending on which collector addresses are configured. sflow Agent The sflow agent provides an interface for configuring sflow instances. The interfaces may be command-line interface (CLI) or SNMP MIBs (in the feature roadmap). The sflow agent is also responsible for making the datagrams and sending them to the collectors. sflow Collector The sflow collector is a piece of hardware/software that can receive sflow datagrams and present a view of traffic and other network parameters which are output as type, length, and value (TLV) in the datagrams. The sflow collectors can also read and configure sflow-managed objects. Copyright 2010, Juniper Networks, Inc. 1
Description and Deployment Scenario sflow Implementation on EX Series Switches An sflow agent is typically embedded in a switch s ASIC hardware, where it collects different samples at regular intervals. The datagrams are sent at regular intervals to the sflow collector whose address is configured as an IP address, UDP port pair. The collector reads the datagram, extrapolates the traffic pattern, and generates a traffic report. The sflow technology provides Layer 2-7 visibility and can also scale to 10-Gigabit Ethernet interfaces. The sflow agent does sampling in two phases. Packet flow sampling consists of statistical data gathered from individual flows, while counter sampling involves the periodic polling of counters to gather interface data. The datagrams are output to the UDP port default as 6343. Flow samples are then bundled into a datagram. Counter sampling is done at regular intervals to provide details about the interfaces, backplane, and so on. The sflow agent can be configured using CLI or, in the future, by SNMP variables. Communication between the agent and the collector is bidirectional; the agent sends datagrams to the collector, while the collector may configure some SNMP variables in the sflow agent or may read some of the SNMP MIB using UDP packets, as they work efficiently in times of congestion. Traffic Monitoring Using sflow on EX Series Switches The following shows a typical deployment for sflow on EX Series switches in order to monitor network traffic. The next section will cover the details step by step. All configurations have been verified in Juniper Networks Junos operating system release 9.5R1.8. EX4200-1 EX4200-3 sflow sampling data GE-0/0/0 GE-0/0/12 sflow enabled sflow sampling GE-0/0/0 data GE-0/0/13 GE-0/0/0 Network Traffic Stream EX4200-2 sflow collector (1.0.0.10) Figure 1: sflow monitoring in a regular network As shown in Figure 1, a bidirectional traffic stream is being transferred across two EX4200 switches, with interface ge-0/0/12 on EX4200-1 and interface ge0/0/0 on EX4200-2. The traffic stream is between two endpoints which are not shown in the diagram with IP addresses 5.0.0.10 and 6.0.0.10. The ge-0/0/12 on EX4200-1 is configured as a Layer 3 interface with IP address 20.0.0.15/24. root@ex4200-1# show interfaces ge-0/0/12 unit 0 { family inet { address 20.0.0.15/24; root@ex4200-1# 2 Copyright 2010, Juniper Networks, Inc.
sflow is enabled on ge-0/0/12 on EX4200-1 so that the sflow agent can sample the ingress traffic stream on this particular interface. As of now, sflow can only be enabled on Layer 2 or Layer 3 physical interfaces. root@ex4200-1# show protocols sflow { polling-interval 20; sample-rate 100; collector 1.0.0.10 { udp-port 6343; interfaces ge-0/0/12.0; root@ex4200-1# Up to four collectors can be configured on each EX Series switch, and each collector can receive the same set of sflow data record samples. The sflow data record samples are UDP packets and the default UDP port is 6343, although this is configurable. The polling interval is the interval between each port statistic polling update message, which can range from 0 to 3600 seconds. The sample rate means one out of N packets in the traffic stream will be sampled, and this can be different for various interfaces. The range of sample rate is from 100 to 1 million. In an EX Series switch implementation, the sflow datagram cannot be routed over the management Ethernet interface (me0) or virtual management interface (vme0). It only can be exported over the network Gigabit Ethernet or 10-Gigabit Ethernet ports using valid route information in the routing table. The most important thing here is that the switch must have a route in the default global routing table to point to the next hop via a network port through which it can reach the remote collector s IP address (in this case, the collector has the IP address 1.0.0.10). In this example, as shown in Figure 1, a static route is configured on EX4200-1, which tells the switch that the interface ge0/0/0 on EX4200-3 is the next hop for it to reach the remote collector 1.0.0.10. root@ex4200-1# show routing-options static { route 1.0.0.0/24 next-hop 30.0.0.13; root@ex4200-1# root@ex4200-1> show route 1.0.0.10 inet.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1.0.0.0/24 *[Static/5] 00:35:14 {master:0 root@ex4200-1> Copyright 2010, Juniper Networks, Inc. 3
With the network data traffic stream sending, the sflow sampling data records and counter statistics record have been received on the remote collector, capturing the following information via the version 1.1.3 Wireshark tool. Figure 2: sflow sampled packets sent to the collector As shown in Figure 2 above, the collector with IP address 1.0.0.10 received the sflow data records from the sflow agent 20.0.0.15 on EX4200-1, and most data records have seven sflow sampled datagrams bundled. The next step is to take a closer look at each sflow data record packet. 4 Copyright 2010, Juniper Networks, Inc.
Figure 3: Packet header of the sflow sampled packets As shown in Figure 3, the sampled sflow record packet that was sent from the sflow agent EX4200-1 to the sflow collector 1.0.0.10 is a UDP packet with the destination port number 6343. Copyright 2010, Juniper Networks, Inc. 5
Figure 4: Details of the sflow sampled packets Figure 4 above shows the detailed information for one of the seven flow samples that were bundled together in one sflow data record packet sent from the EX Series switch to the collector. It is possible to discern the following about the network traffic stream that enters the sflow-enabled interface ge-0/0/12 on EX4200-1: The IP source address is 6.0.0.10. The IP destination address is 5.0.0.10. The DiffServ code point (DSCP) value of the packets in the stream is set to 0x28(EF). The traffic is UDP traffic type FTP with destination port 21. Exporting sflow-sampled Records to a Remote Collector in a Different Virtual Routing Instance As mentioned earlier, in an EX Series switch implementation, sflow-sampled records will not be exported out of management interfaces (me0 or vme0) to avoid the possibility of overwhelming the CPU. In certain situations, the need to export sflow data to a remote collector through the management network path still exists. To accommodate this requirement, it is possible to use the virtual routing instance feature on the EX Series Ethernet Switch and utilize one of the network ports as a dedicated management port to have sflow exported through the management network. 6 Copyright 2010, Juniper Networks, Inc.
In the lab network used in this example (shown in Figure 5), network port ge-0/0/0 on the EX4200-1 is used as a dedicated management interface that is connected to the management network. To isolate the interface ge-0/0/0 from other network interfaces on the switch, interface ge-0/0/0 is placed in a separate virtual routing instance called mgnt_net. In this example, two sflow collectors are sitting in the management network data center, while the third sflow collector is connected to EX4200-1 through the regular network port. VRF: mgnt_net sflow collector (7.0.0.10) GE-0/0/0 sflow sampling data EX4200-1 GE-0/0/12 sflow enabled GE-0/0/7 Network Traffic Stream EX4200-2 Management Network GE-0/0/0 sflow sampling data sflow sampling data sflow collector (1.0.0.10) sflow collector (8.0.0.10) Figure 5: sflow monitoring with a collector in separate virtual routing instance All three sflow collectors are configured under protocol sflow on EX4200-1: root@ex4200-1# show protocols sflow { polling-interval 20; sample-rate 100; collector 1.0.0.10; collector 8.0.0.10; collector 7.0.0.10; interfaces ge-0/0/12.0; root@ex4200-1# The following configuration places ge-0/0/0 on EX4200-1 into the separate virtual routing instance mgnt_net to isolate it from other network interfaces on the switch. Static routes are also configured for out-of-band management for switch EX4200-1. root@ex4200-1# show routing-instances mgnt_net { instance-type virtual-router; interface ge-0/0/0.0; routing-options { static { Copyright 2010, Juniper Networks, Inc. 7
route 0.0.0.0/0 next-hop 30.0.0.13; route 1.0.0.10/32 next-hop 30.0.0.13; route 8.0.0.10/32 next-hop 30.0.0.13; root@ex4200-1# The routing table on switch EX4200-1 shows the following: root@ex4200-1# run show route inet.0:7 destinations, 7 routes (7 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 5.0.0.0/24 *[Direct/0] 23:23:22 > via ge-0/0/5.0 5.0.0.15/32 *[Local/0] 1d 03:20:14 Local via ge-0/0/5.0 6.0.0.0/24 *[Static/5] 1d 03:17:49 > to 20.0.0.16 via ge-0/0/12.0 7.0.0.0/24 *[Direct/0] 00:26:17 > via ge-0/0/7.0 7.0.0.15/32 *[Local/0] 00:26:17 Local via ge-0/0/7.0 20.0.0.0/24 *[Direct/0] 1d 03:23:07 > via ge-0/0/12.0 20.0.0.15/32 *[Local/0] 1d 03:23:07 Local via ge-0/0/12.0 mgnt_net.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 0.0.0.0/0 *[Static/5] 00:26:17 1.0.0.10/32 *[Static/5] 00:26:17 8.0.0.10/32 *[Static/5] 00:26:17 30.0.0.0/24 *[Direct/0] 00:26:17 > via ge-0/0/0.0 30.0.0.15/32 *[Local/0] 00:26:17 Local via ge-0/0/0.0 root@ex4200-1# The sflow agent EX4200-1 needs the routing information in its default routing table to reach the sflow collectors, so that it can export the sflow data records to the collectors through the network interfaces. As shown above, EX4200-1 can reach the collector 7.0.0.10, since this collector is connected to EX4200-1 through interface ge-0/0/7 which belongs to the default global routing instance. Hence, there is a route pointing 7.0.0.0/24 subnets in its default global routing table. The other collectors, 8.0.0.10 and 1.0.0.10, are connected to EX4200-1 through the pseudo management interface ge-0/0/0 which belongs to a separate virtual routing instance mgnt_net. Therefore, EX4200-1 doesn t have routes to reach these two collectors in the default global routing table, and the routes only show up in the mgnt_net virtual routing table. 8 Copyright 2010, Juniper Networks, Inc.
In order for EX4200-1 to export sflow data records to collectors 8.0.0.10 and 1.0.0.10 through the pseudo management interface ge-0/0/0, these routes (8.0.0.10/32, 1.0.0.10/32) must be advertised from mgnt_net virtual routing instance to the default global routing instance. First, the policy statement must be set up so that the policy statement sflow_collector will advertise two routes 1.0.0.10/32 and 8.0.0.10/32 from virtual routing instance mgnt_net to the default global routing instance. root@ex4200-1# show policy-options policy-statement sflow_collector { term t1 { from { instance mgnt_net; route-filter 1.0.0.10/32 exact; then accept; term t2 { from { instance mgnt_net; route-filter 8.0.0.10/32 exact; then accept; term default { then reject; root@ex4200-1# Next, the policy statement must be attached to the default global instance: root@ex4200-1# show routing-options instance-import sflow_collector; root@ex4200-1# A quick review of the routing table shows that two more routes (1.0.0.10/32 and 8.0.0.10/32) have been redistributed from the mgnt_net routing instance and the outgoing interface is ge-0/0/0. The forwarding table also shows the next hop to be 30.0.0.13. root@ex4200-1# run show route inet.0:9 destinations, 9 routes (9 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1.0.0.10/32 *[Static/5] 00:01:08 5.0.0.0/24 *[Direct/0] 23:23:22 > via ge-0/0/5.0 5.0.0.15/32 *[Local/0] 1d 03:20:14 Local via ge-0/0/5.0 6.0.0.0/24 *[Static/5] 1d 03:17:49 > to 20.0.0.16 via ge-0/0/12.0 7.0.0.0/24 *[Direct/0] 00:26:17 > via ge-0/0/7.0 Copyright 2010, Juniper Networks, Inc. 9
7.0.0.15/32 *[Local/0] 00:26:17 Local via ge-0/0/7.0 8.0.0.10/32 *[Static/5] 00:01:08 20.0.0.0/24 *[Direct/0] 1d 03:23:07 > via ge-0/0/12.0 20.0.0.15/32 *[Local/0] 1d 03:23:07 Local via ge-0/0/12.0 mgnt_net.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 0.0.0.0/0 *[Static/5] 00:26:17 1.0.0.10/32 *[Static/5] 00:26:17 8.0.0.10/32 *[Static/5] 00:26:17 30.0.0.0/24 *[Direct/0] 00:26:17 > via ge-0/0/0.0 30.0.0.15/32 *[Local/0] 00:26:17 Local via ge-0/0/0.0 root@ex4200-1# run show route forwarding-table destination 1.0.0.10 Routing table: default.inet 1.0.0.10/32 user 0 30.0.0.13 ucst 1315 7 ge-0/0/0.0 Routing table: juniper_private1.inet default perm 0 rjct 116 1 Routing table: juniper_private2.inet default perm 0 rjct 196 1 Routing table: master.anon.inet default perm 0 rjct 1286 1 Routing table: mgnt_net.inet 1.0.0.10/32 user 0 30.0.0.13 ucst 1315 7 ge-0/0/0.0 root@ex4200-1# run show route forwarding-table destination 8.0.0.10 Routing table: default.inet 8.0.0.10/32 user 0 30.0.0.13 ucst 1315 7 ge-0/0/0.0 Routing table: juniper_private1.inet default perm 0 rjct 116 1 Routing table: juniper_private2.inet 10 Copyright 2010, Juniper Networks, Inc.
default perm 0 rjct 196 1 Routing table: master.anon.inet default perm 0 rjct 1286 1 Routing table: mgnt_net.inet 8.0.0.10/32 user 0 30.0.0.13 ucst 1315 7 ge-0/0/0.0 After starting traffic between EX4200-1 and EX4200-2, sflow sampled records are being sent to all three collectors, which are actually in different virtual routing instances. Figure 6 below shows the captures on collector 7.0.0.10. Figure 6: sflow sampled packets sent to the collector 7.0.0.10 Copyright 2010, Juniper Networks, Inc. 11
Figure 7 below shows the captures on collector 1.0.0.10: Figure 8 below shows the captures on collector 8.0.0.10: Figure 7: sflow sampled packets sent to the collector 1.0.0.10 Figure 8: sflow sampled packets sent to the collector 8.0.0.10 12 Copyright 2010, Juniper Networks, Inc.
A close look at the sflow sampled data records exported to collector 8.0.0.10 shown in Figure 9 shows that the network traffic stream which enters the sflow-enabled interface ge-0/0/12 on EX4200-1 has the following patterns: The stream s IP source address is 6.0.0.10. The stream s IP destination address is 5.0.0.10. The DSCP value of the packets in the stream is 0x28(EF). The traffic is UDP traffic type FTP with destination port 21. Figure 9: Details of the sflow sampled packets sent to collector 8.0.0.10 Copyright 2010, Juniper Networks, Inc. 13
Figure 10 below shows the captured sflow counter sample record which is exported to collector 8.0.0.10. Figure 10: Details of the sflow interface counters sent to collector 8.0.0.10 14 Copyright 2010, Juniper Networks, Inc.
Summary The sflow technology is used for monitoring traffic in data networks containing switches and routers. With the sflow implementation on Juniper Networks EX Series Ethernet Switches, sflow data records and counters can be sampled and exported to up to four collectors in different virtual routing instances to provide clear visibility into network traffic patterns. About Juniper Networks Juniper Networks, Inc. is the leader in high-performance networking. Juniper offers a high-performance network infrastructure that creates a responsive and trusted environment for accelerating the deployment of services and applications over a single network. This fuels high-performance businesses. Additional information can be found at www.juniper.net. Corporate and Sales Headquarters Juniper Networks, Inc. 1194 North Mathilda Avenue Sunnyvale, CA 94089 USA Phone: 888.JUNIPER (888.586.4737) or 408.745.2000 Fax: 408.745.2100 www.juniper.net APAC Headquarters Juniper Networks (Hong Kong) 26/F, Cityplaza One 1111 King s Road Taikoo Shing, Hong Kong Phone: 852.2332.3636 Fax: 852.2574.7803 EMEA Headquarters Juniper Networks Ireland Airside Business Park Swords, County Dublin, Ireland Phone: 35.31.8903.600 EMEA Sales: 00800.4586.4737 Fax: 35.31.8903.601 To purchase Juniper Networks solutions, please contact your Juniper Networks representative at 1-866-298-6428 or authorized reseller. Copyright 2010 Juniper Networks, Inc. All rights reserved. Juniper Networks, the Juniper Networks logo, Junos, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other trademarks, service marks, registered marks, or registered service marks are the property of their respective owners. Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice. 3500162-002-EN May 2010 Printed on recycled paper 15