ADVANCED NETFLOW
Business and IT are Changing Like Never Before Drastic Change in Application Type, Delivery, and Consumption Public/Hybrid Cloud SaaS/IaaS Storage Users/ Machines Proliferation of Devices THE NETWORK Private Cloud VDI IaaS 60% of IT professional cites performance as key challenge for cloud How Applica<on applica<ons Type of applica<ons are Consumed Delivered Database
Cisco Network Devices Embedded Instrumenta<on
Applica<on Visibility and Control What is Needed App Visibility & User Experience Report App BW Transac&on Time NFv9/IPFIX SAP 3M 150 ms Sharepoint 10M 500 ms High Med Low Repor<ng Tools Applica<on Recogni<on Repor&ng Perf. Tool Collec<on & Expor<ng Management Tool Control Iden<fy applica<ons using L3 to L7 informa<on Collect applica<on performance metrics, and export to management tool Advanced repor<ng tool aggregates and reports applica<on performance Control applica<on network usage to improve applica<on performance
Applica<on Visibility and Control Enabled Technologies App Visibility & User Experience Report App BW Transac&on Time NFv9/IPFIX SAP 3M 150 ms Sharepoint 10M 500 ms High Med Low Repor<ng Tools Applica<on Recogni<on Repor&ng Perf. Tool Collec<on & Expor<ng Management Tool Control NBAR2 Metadata Unified Monitoring - Traffic Sta<s<cs - Response Time - Voice/Video Monitoring - URL Collec<on Cisco Prime Infrastructure QoS (w/ NBAR2) PfR
What do we want to monitor? Traffic Sta<s<cs URL Visibility Applica<on Response Time Media Performance Applica<on Usage per client IP/ subnet/site Top clients per applica<on Most visited web- site Per- URL applica<on response <me Per- applica<on end- to- end latency Applica<on response <me & transac<on <me Applica<on processing <me Top conversa<on per applica<on Per- stream jider and packet loss RTP conversa<ons
Evolu<on of Applica<ons Sta<c port classifica<on is no longer enough Increasing use of Encryp<on and Obfusca<on Applica<on consists of mul<ple sessions (Video, Voice, Data)
Define Your Own Applica<on in NBAR2 Port Payload HTTP URL New TCP or UDP 16 sta<c ports per applica<on Range of ports (1000 maximum) Search the first 255 bytes of TCP/UDP payload ASCII (16 characters) Hex (4 bytes) Decimal (1-4294967295) Variable (4 bytes Hex) URI regex Host regex 9
NBAR2 Regular Updates In- service Applica<on Defini<on Update PPX (Major) 1M PPX.1 (Minor) 1M PPY (Major) 1M PPY.1 (Minor) protocols~ 10 updates and fixes Bug fixes small updates Protocols~10 updates and fixes Bug fixes small updates Protocol Pack Includes all supported Protocols / Applica<ons Support Traffic categoriza<on and Adributes Available (as Default protocol pack) in DATA image Periodic releases and Offers SLA Protocol Pack Protocol1 Protocol2 Protocoln NBAR2 PP 4.1 Available
NBAR2 Protocol Pack Example Add new applica<ons recognized by NBAR2 without IOS upgrade or router reload New protocol pack is published every two months on CCO Single IOS CLI to enable the protocol pack
Applica<on Response Time HQ Key Features 27 Applica<on Response Time (ART) Metrics Interact with NBAR2 for Applica<on ID Standard NFv9 and IPFIX export Benefits Visibility into applica<on usage and performance Quan<fy user experience Troubleshoot applica<on performance Track service levels for applica<on delivery Branch Delay Network Delay Datacenter Delay PA WAN1 (IP- VPN) ASR PA ASR PA Repor<ng Tool ISR ISR ISR ISR My email is slow! WAN2 (IPVPN, DMVPN) My query is taking long &me! How do I ensure my SLA is met PA
Media Monitoring - Voice and Video Performance FNFv9 Alarm Syslog FNFv9 Alarm Syslog Management Tool Voice/video Endpoints WAN Voice/video Endpoints Media Monitoring Key Features Monitor media performance metrics, i.e. jitter, loss Integrate with NBAR2 to identify applications Setting threshold and generating alert/alarm Standard FNFv9 export Benefits Real-time monitoring of voice and video performance across network Accelerate troubleshooting identify what, where, when is the problem Proactive troubleshooting Validate SLA
Flexible NetFlow (FNF) Expor<ng Process: NetFlow v9 and IPFIX Sta<c Flow Export Format Flexible & Extensible Flow Export Format Neplow Version 5 Neplow v9 / IPFIX Flow record Describe flow format A Exporter Flow record Flow record Collector Exporter Describe flow format B Flow record A Flow record A Collector Flow record Flow record B Fixed number of fields (18 fields) e.g. source/des<na<on IP & port, input/ output interfaces, packet/byte count, ToS Users define flow record format Flow format is communicated to collector
Version 5 Flow Format Flow Key vs. Non- Key Field From/to Usage Time of Day Port U&liza&on QoS Packet count Byte count Start sysuptime End sysuptime Input ifindex Output ifindex Type of service TCP flags Protocol Source IP address Des<na<on IP address Source TCP/UDP port Des<na<on TCP/UDP port Next hop address Source AS number Dest. AS number Source prefix mask Dest. Prefix mask Applica&on Rou&ng and Peering
NetFlow Cache Example 1. Create and update flows in NetFlow cache Srclf SrclPadd Dstlf DstlPadd Protocol TOS Flgs Pkts Src Port Src Msk Src AS Dst Port DstM sk Dst AS NextHop Bytes/ Pkt Active Idle Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 00A2 /24 15 10.0.23.2 1528 1745 4 Fa1/0 173.100.3.2 Fa0/0 10.0.227.12 6 40 0 2491 15 /26 196 15 /24 15 10.0.23.2 740 41.5 1 Fa1/0 173.100.20.2 Fa0/0 10.0.227.12 11 80 10 10000 00A1 /24 180 00A1 /24 15 10.0.23.2 1428 1145.5 3 Fa1/0 173.100.6.2 Fa0/0 10.0.227.12 6 40 0 2210 19 /30 180 19 /24 15 10.0.23.2 1040 24.5 14 2. Expira&on Inac&ve &mer expired (15 sec is default) Ac&ve &mer expired (30 min is default) => change it 1 min NetFlow cache is full (oldest flows are expired) RST or FIN TCP flag Srclf SrclPadd Dstlf DstlPadd Protocol TOS Flgs Pkts Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 00A2 /24 15 10.0.23.2 1528 1800 4 Src Port Src Msk Src AS Dst Port Dst Msk Dst AS NextHop Bytes/ Pkt Ac&ve Idle 3. Aggrega&on 4. Export version Non- aggregated flows export version 5 or 9 5. Transport protocol (UDP, SCTP) Export Packet Header Payload (Flows) E.g., Protocol- Port Aggrega&on Scheme Becomes Protocol Pkts SrcPort DstPort Bytes/Pkt 11 11000 00A2 00A2 1528 Aggregated Flows Export Version 8 or 9
NetFlow Export Version 5 and Main Cache Configura<on Example Router(config)# interface <slot/port/subinterface> Router(config-if)# ip flow ingress Router(config-if)# ip flow egress Router(config)# ip flow-cache entries <number> Router(config)# ip flow-cache timeout active <minutes> Router(config)# ip flow-cache timeout inactive <seconds> Router(config)# ip flow-export version 5 peer-as Router(config)# ip flow-export destination 10.10.10.10 1234 Router(config)# ip flow-export source loopback 0
NetFlow Flow Keys on the Router By default, the 7 flow keys are: Source IP address, des<na<on IP address, source port, des<na<on port, Layer 3 protocol type, TOS byte (DSCP), input interface The 12 NetFlow aggrega<ons allow to reduce/change the number of flow keys Example: source prefix aggrega<on = source network, source interface Can be seen as a different view of the main cache Egress NetFlow, MPLS- aware NetFlow, etc. Specify new flow keys Note: on the Cisco Catalyst, we speak of the flow mask This effec<vely specify the flow keys 18
Flow Keys on the Cisco Catalyst 6500/7600 The Flow Mask (before SUP2T) Full-Interface VLAN SRC IP DST IP IP Protocol Src Port Dst Port Full VLAN SRC IP DST IP IP Protocol Src Port Dst Port Destination-Source-Interface VLAN SRC IP DST IP IP Protocol Src Port Dst Port Source-Only VLAN SRC IP DST IP IP Protocol Src Port Dst Port Destination-Only VLAN SRC IP DST IP IP Protocol Src Port Dst Port Destination-Source VLAN SRC IP DST IP IP Protocol Src Port Dst Port Flow Keys in Orange
Extensibility and Flexibility Requirements Phases Approach Tradi<onal NetFlow with v5 or v8 NetFlow export New requirements: build something flexible and extensible Phase One: NetFlow Version 9 Advantages: extensibility Integrate new technologies/data types quicker (MPLS, IPv6, BGP next hop, etc.) Integrate new aggrega<ons quicker Note: for now, the template defini<ons are fixed Phase Two: Flexible NetFlow Advantages: cache and export content flexibility User selec<on of flow keys User defini<on of the records Expor&ng Process Metering Process 20
Expor<ng Process versus Metering Process and NetFlow Evolu<on Expor<ng Process versus Metering Process are IPFIX (IP Flow Informa<on export) terms: the NetFlow term doesn t make the dis<nc<on The Metering Process generates Flow Records. Inputs to the process are packet headers, characteris<cs, and Packet Treatment observed at one or more Observa<on Points. Tradi<onal NetFlow Flexible NetFlow Metric Media<on Agent (Media<on func<on expor<ng performance metrics) and some others The Expor<ng Process sends IPFIX Messages to one or more Collec<ng Processes = the export protocol NetFlow export version 5, version 7, version 8, version 9 IPFIX (RFC 7011), which is version 10 (as it s based on NetFlow version 9)
Expor<ng Process versus Metering Process and NetFlow Evolu<on Export Metric Media<on Agent Infrastructure Use cases evolu<on and hence informa<on elements evolu<on + Different sources of informa<on (different metering processes) => we need some aggrega<on and correla<on in the router => we need a super metering process: the Metric Media<on Agent FNF NBAR2 Perf Mon PA (ART) QoS PfR firewall WAAS 22
NetFlow Partners Traffic Analysis Denial of Service Billing CS- Mars hgp://www.cisco.com/en/us/prod/iosswrel/ps6537/ps6555/ps6601/networking_solu&ons_products_genericcontent0900aecd805ff728.html
NetFlow Open Source Tools Product Name Primary Use Comment OS Cflowd Traffic Analysis No longer supported UNIX Flow-tools Collector Device Scalable UNIX Flowd Collector Device Support V9 BSD, Linux FlowScan Reporting for Flow-Tools UNIX IPFlow Traffic Analysis Support V9, IPv4, IPv6, MPLS, SCTP, etc.. Linux, FreeBSD, Solaris NetFlow Guide Reporting Tools BSD, Linux NetFlow Monitor Traffic Analysis Supports V9 UNIX Netmet Collector Device V5, support v9 Linux NTOP Security Monitoring UNIX Stager Reporting for Flow-Tools UNIX Nfdump/nfsen Traffic Analysis Support V5 and v9 UNIX Different costs: implementa<on and customiza<on
NetFlow Version 9 Version 9 is an export protocol No changes to the metering process Version 9 is based on templates and separate flow records Templates expressing type and length Flow records expressing template ID and list of values Sent the template regularly (configurable), because of UDP Support: 800, 1700, ISR (1800, 2800, 3800), ISR- G2 (1900, 2900, 3900), 2600, 3200, 3600, 3750, 4400, cat 3850, cat4500, cat6500, cat 5760 (wireless controller), Cloud Services Router CSR- 1000v, 7200, 7300, 7500, 7600, 10000, 12000 (IOS and IOS- XR), CRS- 1, ASR 1000, ASR 9000, ASA 5580, Nexus 7000 and Nexus 1000V RFC3954 Cisco Systems NetFlow Services Export Version 9 NetFlow patent: intellectual property right statement at the IETF website
NetFlow Version 9 Export Packet Template 1 Template 2 H E A D E R Template FlowSet Template Record Template ID #1 (Specific Field Types and Lengths) Template Record Template ID #2 (Specific Field Types and Lengths) Data FlowSet FlowSet ID #1 Data Record (Field Values) Data Record (Field Values) Data FlowSet ID #1 FlowSet ID #2 Data Record (Field Values)
NetFlow Version 9 Export Packet Op<ons Template FlowSet Specifies the Scope: Cache, System, Template, etc. Template 3 H E A D E R Op&ons Template FlowSet Op&on Template Record Template ID #3 (Specific Scope, Field Types and Lengths) Data FlowSet FlowSet ID #3 Op&on Data Record (Field Values) Op&on Data Record (Field Values)
Interface Name Export with NetFlow Version 9 Example of op<ons template FlowSet: NetFlow exports the ifindex Instead of the collector polling the ifname MIB variable for a specific ifindex, the matching (ifindex, ifname) is sent in an op<on data record Router(config)# ip flow-export interface-names 28
NetFlow Version 9 Main Cache Configura<on router(config)# ip flow-export version [5 9] [origin-as peer-as] [bgp-nexthop] router(config)# ip flow-export template options export-stats router(config)# ip flow-export template options timeout-rate 5 router(config)# ip flow-export template options refresh-rate 20 router(config)# ip flow-export template timeout-rate 5 router(config)# ip flow-export template refresh-rate 20 router(config)# ip flow-export destination 10.10.10.10 9996 (Op&ons) Templates Sent Every (Op&ons) Templates Sent Every Five Minutes or 20 Should you export from the main cache with or Every Packets 20 Packets NetFlow Version 5 or Version 9? 29
NetFlow Version 9 Aggrega<on Cache Configura<on router(config)# ip flow-aggregation cache bgp-nexthop-tos router(config-flow-cache)# export destination 11.11.11.11 9999 router(config-flow-cache)# export version? 9 Version 9 export format router(config-flow-cache)# export version 9 router(config-flow-cache)# enabled In this case, we have only version 9. Why?
Flexible NetFlow High- Level Concepts and Advantages Flexible NetFlow feature allows user configurable NetFlow record formats, selec<ng from a collec<on of fields: Key, non- key, counter, <mestamp Advantages: Tailor a cache for specific applica<ons, not covered by exis<ng 21 NetFlow features in tradi<onal NetFlow Different NetFlow caches: per subinterface, per direc<on (ingress, egress), per sampler, per Beder scalability since flow record customiza<on for par<cular applica<on reduces number of flows to monitor 31
Flexible NetFlow Mul<ple Monitors with Unique Key Fields Traffic Key Fields Packet 1 Source IP 3.3.3.3 Destination IP 2.2.2.2 Source Port 23 Destination Port 22078 Layer 3 Protocol TCP - 6 TOS Byte 0 Input Interface Ethernet 0 Flow Monitor 1 Non-Key Fields Packets Bytes Timestamps Next Hop Address Flow Monitor 2 Key Fields Source IP Dest IP Input Interface SYN Flag Packet 1 3.3.3.3 2.2.2.2 Ethernet 0 0 Security Analysis Cache Non- Key Fields Packets Timestamps Source IP Dest. IP Input I/F Flag Pkts Traffic Analysis Cache 3.3.3.3 2.2.2.2 E0 0 11000 Source IP Dest. IP Source Port Dest. Port Protocol TOS Input I/F Pkts 3.3.3.3 2.2.2.2 23 22078 6 0 E0 1100
Flexible NetFlow Model Interface Monitor A Monitor B Monitor C Exporter M Record X Exporter M Record Z Exporter N Record Y A single record per monitor Poten<ally mul<ple monitors per interface Poten<ally mul<ple exporters per monitor
Service Planning FNF Configura<on - Example 1. Configure the Exporter Router(config)# flow exporter my-exporter Where do I want my data sent? Router(config-flow-exporter)# destination 1.1.1.1 2. Configure the Flow Record Router(config)# flow record my-record Router(config-flow-record)# What data do I match want to ipv4 meter? destination address Router(config-flow-record)# match ipv4 source address Router(config-flow-record)# collect counter bytes 3. Configure the Flow Monitor Router(config)# flow monitor my-monitor Router(config-flow-monitor)# How do I want to cache exporter informa&on? my-exporter Router(config-flow-monitor)# record my-record 4. Apply to an Interface Router(config)# interface s3/0 Which interface do I want to monitor? Router(config-if)# ip flow monitor my-monitor input
Flexible NetFlow User- Defined Record Configura<on Router(config)# flow record my-record Router(config-flow-record)# match Router(config-flow-record)# collect Router(config-flow-record)# match? application Application Fields datalink Datalink (layer 2) fields flow Flow identifying fields interface Interface fields ipv4 IPv4 fields ipv6 IPv6 fields routing routing attributes transport Transport layer field Router(config-flow-record)# collect? application Application Fields counter Counter fields datalink Datalink (layer 2) fields flow Flow identifying fields interface Interface fields ipv4 IPv4 fields ipv6 IPv6 fields routing IPv4 routing attributes timestamp Timestamp fields transport Transport layer fields Specify a Key Field Specify a Non- Key Field
Flexible Flow Record: Key Fields NEW NEW Flow Sampler ID Direction Class ID Interface Input Output Layer 2 Source VLAN Dest VLAN Dot1q VLAN Dot1q priority Source MAC address Des<na<on MAC address IPv4 IP (Source or Destination) Prefix (Source or Destination) Mask (Source or Destination) Minimum-Mask (Source or Destination) Protocol Fragmentation Flags Fragmentation Offset Identification Header Length Total Length Payload Size Packet Section (Header) Packet Section (Payload) TTL Options bitmap Version Precedence DSCP TOS IPv6 IP (Source or Destination) Prefix (Source or Destination) Mask (Source or Destination) Minimum-Mask (Source or Destination) Protocol Traffic Class Flow Label Option Header Header Length Payload Length Payload Size Packet Section (Header) Packet Section (Payload) DSCP Extension Headers Hop-Limit Length Next-header Version
Flexible Flow Record: Key Fields NEW: 2 or 4 bytes NEW Rou<ng src or dest AS Peer AS Traffic Index Forwarding Status IGP Next Hop BGP Next Hop Input VRF Name Transport Destination Port Source Port ICMP Code ICMP Type IGMP Type* TCP ACK Number TCP Header Length TCP Sequence Number TCP Window-Size TCP Source Port TCP Destination Port TCP Flag: ACK TCP Flag: CWR TCP Flag: ECE TCP Flag: FIN TCP Flag: PSH TCP Flag: RST TCP Flag: SYN TCP Flag: URG UDP Message Length UDP Source Port UDP Destination Port Application Application ID Multicast Replication Factor* RPF Check Drop* Is-Multicast NEW TCP Urgent Pointer RTP SSRC *: IPv4 Flow only NEW
Flexible Flow Record: Non- Key Fields Counters Timestamp IPv4 IPv4 and IPv6 Bytes Bytes Long Bytes Square Sum Bytes Square Sum Long Packets Packets Long Bytes replicated Bytes replicated Long Packets replicated Packets Replicated Long sysuptime First Packet sysuptime First Packet Absolute first packet Absolute last packet NEW Total Length Minimum (*) Total Length Maximum (*) TTL Minimum TTL Maximum NEW Total Length Minimum (**) Total Length Maximum (**) Plus any of the poten<al key fields: will be the value from the first packet in the flow (*) IPV4_TOTAL_LEN_MIN, IPV4_TOTAL_LEN_MAX (**)IP_LENGTH_TOTAL_MIN, IP_LENGTH_TOTAL_MAX
Flow Exporter Configura<on New in 15.3.1(T) and IOS XE 3.8 flow exporter <exporter-name> destination <ipv4-address> [vrf <vrf-name>] dscp <value> export-protocol [netflow-v5 netflow-v9 ipfix ] option {exporter-stats interface-table sampler-table vrf-table application-table application-attributes c3pl-class-table c3pl-policy-table } timeout <value in sec> source <interface-name> template data timeout <value in sec> transport udp <destination-port> ttl <value> output-features Eight Types of Op&ons Data Record New in 12.4(20)T NetFlow Exported Packets Go Through QoS, Crypto- Map, etc
Cisco Prime Infrastructure
Cisco Prime Infrastructure Realizing the vision of One Management Lifecycle Simplified deployment and configura<on Compliance Regulatory requirements and best prac<ces Assurance Improved Applica<on Delivery
Cisco Prime Infrastructure Management of Wired and Wireless Devices Configura&on of features, Config Archive and Image Management Monitoring and Performance Trending Neslow collec&on and visibility of traffic flowing through the infrastructure Consolidated Repor&ng and dashboards Trending and Analysis
Consistent Visibility across the Enterprise Data Center Cisco NAM Appliance Cisco Nexus 1000V VM VM VM Netflow and AVC And SNMP Cisco ASR VM Cisco ISR SNMP and Medianet NAM on Nexus 1110 Netflow and AVC And SNMP SNMP and Medianet Cisco WAAS Cisco WAAS San Jose Branch Cisco Prime WAN NAM on ISR - Netwflow and AVC and SNMP Cisco- ISR SNMP and Medianet Cisco ISR SNMP Poll Amsterdam Branch Branch to Branch Traffic SNMP and Medianet Cisco WAAS London Branch
Assurance Use Case
Network Performance Site is experiencing bandwidth congestion. Troubleshoot and identify the users/applications responsible for bandwidth congestion
SNMP Polling of Interface U<liza<on Top WAN interfaces Bandwidth u<liza<on over <me
Applica<on U<liza<on over <me Applica<on u<liza<on over <me
Top N reports for the interface Top Users by bandwidth Top Applica<ons by bandwidth
Top talkers for applica<ons Find the users who are using the most bandwidth for the site
QoS Se{ng for the Interface Class Map Sta<s<cs DSCP marking of Traffic
Op<mizing the bandwidth (Control) QoS - > Enable QOS on the interface for bandwidth is op<mized for the cri<cal applica<ons
End User Experience Jack Fields is having performance issues with accessing his critical applications.
Search and find user Search and find user by name or IP Address
Iden<fy Users Devices Jack Fields has 2 Wireless and 1 Wired Client User 360 View of Jack Fields 54
Iden<fy User and their applica<ons Devices and network performance Applica&ons and bandwidth
Known what the user is doing Jack Fields conversa<ons to/from Users Jack Fields Voice Conversa<ons
User compared to his site Users Site devices Worst Voice calls Applica<on Response 57
Iden<fy authen<ca<on issues (Wireless) Select troubleshoo<ng Now we get Select a full the report device on with what the could have gone wrong connec<vity in the Auth. problem process This results in a real- <me connec<vity Now we test, get in the this full case Auth. History of this device with Auth. respec<ve fails user to the respec<ve ISE sever, click on the failure reason Integra<on with ISE becomes very useful in this stage, select the ISE budon
Iden<fy access issues (wireless) Click on the loca<on Thanks to colora<on of Clients with interferers we can locate connec<vity issues Everyone can do heatmaps
User End to End Performance Connec&vity Cisco Switches Cisco ISR/ASR Cisco NAM Users, their end points and applica<ons [Neslow, NBAR, NBAR2, AVC, Medianet] End point to User mapping Cisco Prime Authen&ca&on and Access Authenticated Wireless Users Authenticated Wired Users Cisco ISE Users devices