Performance Metrics for Multilayer Switches

Transcription

1 1 1 Performance Metrics for Multilayer Switches Session 2

2 Objective To understand the how to measure a device s network ability to protect application networking resource performance and scalability. 3 Agenda Definition of Terms Performance Test Tools Testing Methods and Metrics Example Buffering and Architecture Example Flow Lookup and Architecture Example WRR and Architecture Example SLB and Architecture 4

3 Definition of Terms Device Under Test Device under test (DUT) A single stand-alone or modular unit which receives frames on one or more of its interfaces and then forwards them to one or more interfaces according to the addressing information contained in the frame 5 Definition of Terms (Cont.) 1.1.X.X 1.2.X.X System Under Test Device Under Test 1.3.X.X 1.4.X.X System Under Test (SUT) The collective set of network devices to which stimulus is offered as a single entity and response measured 6

4 Definition of Terms (Cont.) Queue No Queue Queuing buffer mechanisms used to control congestion by temporarily storing packets in memory 7 Definition of Terms (Cont.) ngle Latency Frame Latency (RFC 1944) Time in microseconds which a DUT takes to transmit one packet with same size packet at X rate 8

5 Definition of Terms (Cont.) ngle Latency Frame Latency (RFC 1944) Time in microseconds which a DUT takes to transmit one packet with same size packet at X rate 9 Definition of Terms (Cont.) All Frames tagged with latency and sequence Latency (RFC 2285) Time in microseconds which a DUT takes to transmit all packets with sequence and latency tags 10

6 Definition of Terms (Cont.) All Frames tagged with latency and sequence Latency (RFC 2285) Time in microseconds which a DUT takes to transmit all packets with sequence and latency tags 11 Definition of Terms (Cont.) Bit Bucket =0 Throughput Highest rate in packets-persecond through a DUT which frame loss does not occur 12

7 Definition of Terms (Cont.) Bit Bucket Frame loss Percentage of lost frames when running at said frame size often confused with throughput 13 Definition of Terms (Cont.) Input Gigabit Ports 768 nsec IFG (Inter-Frame-Gap) Input Gigabit Ports Burst 96 nsec IFG (Inter-Frame-Gap) Back-to-back burst The minimum interval between bursts which the DUT can process with no frame loss 14

8 Definition of Terms (Cont.) MAC IP TCP/UDP Application Application 128-->1518+ Layer 7 48 bytes Layer 4 32 bytes Layer 3 12 bytes Layer 2 Depth of inspection Amount of packet information processed to make a switching decision 15 Agenda Definition of Terms Performance Test Tools Testing Methods and Metrics Example Buffering and Architecture Example Flow Lookup and Architecture Example WRR and Architecture Example SLB and Architecture 16

9 ML-7710 TX INIT TRIG CLS RX ERR LINK DPLX 100M PAUSE ML-7710 TX INIT TRIG CLS RX ERR LINK DPLX 100M PAUSE Real PC User Keyboard Real Application with Overhead TCP IP OS UDP NDIS Driver PC Based Testing No Industry Standard Application (Packet) Scripts TCP IP OS UDP NDIS Driver Test Tools Application to Application Measurement A Few Scripts Metrics Unknown Metrics Unknown Metrics Unknown Metrics Unknown Not Wire Speed No Accuracy Testing with Appliance Device to Device Measurement Scales to Many Users Setup Timing etc Latency, Jitter etc. No OS Delay Throughput, Loss, etc. Wire Speed Nanosec Accuracy ML-7710 TX INIT TRIG CLS RX ERR LINK DPLX 100M PAUSE Source Source Network No Network Measurement Full Visibility of Network Elements Network Destination Destination Only Round Trip Times One Way Times, Accurate GPS timing 17 Real PC User Keyboard Real Application with Overhead TCP IP OS UDP NDIS Driver PC Based Testing No Industry Standard Fake Application (Packet) Scripts TCP IP OS UDP NDIS Driver Test Tools Application to Application Measurement A Few Scripts Metrics Unknown Metrics Unknown Metrics Unknown Metrics Unknown Not Wire Speed No Accuracy Testing with Appliance Device to Device Measurement Scales to Many Users Setup Timing etc Latency, Jitter etc. No OS Delay Throughput, Loss, etc. Wire Speed Nanosec Accuracy ML-7710 TX INIT TRIG CLS RX ERR LINK DPLX 100M PAUSE Source Source Network No Network Measurement Full Visibility of Network Elements Network Destination Destination Only Round Trip Times One Way Times, Accurate GPS timing 18

10 ML-7710 TX INIT TRIG CLS RX ERR LINK DPLX 100M PAUSE ML-7710 TX INIT TRIG CLS RX ERR LINK DPLX 100M PAUSE Application Testing Only true application test User feel valuable but intangible No standard metrics Not wire speed Real PC User Application Keyboard Real Application with Overhead TCP IP OS UDP NIC Driver Real Network 19 Appliance Testing End-to-End Latency Multi-user Multi-network GPS SYNC Great for L3 & L2 OK for L4 GPS ML-7710 TX INIT TRIG CLS RX ERR LINK DPLX 100M PAUSE Source ISP-Network Your Network GPS Your Network Your Network GPS 20

11 ML-7710 TX INIT TRIG CLS RX ERR LINK DPLX 100M PAUSE ML-7710 TX INIT TRIG CLS RX ERR LINK DPLX 100M PAUSE Why Ping-Pong is all Wrong ngle user D C B ngle network A1 ngle Sample Asynchronous Routing Issues E A2 Minimal accuracy A1+A2 B+C+D+E=RTT 21 Why Ping-Pong is all Wrong Multi-user Multi-network Multi-Sample End-to-End Latency GPS timing A1+B+C+D=RTT1 1 D E GPS C A1 A2 B D+E+A2 =RTT2 *Netcom s SmartISP GPS 2 22

12 Verify your Service Service Level Agreements CIR-Committed Information rate With Respect to time 9AM-5PM 5PM-12AM 12AM-9AM CIR Latency & Deviation Throughput & Deviation 5 0 te-a te-b 23 Performance Test Tools PC Test Tool Appliance Application to Application Measurement Device to Device Measurement A Few Scripts Scales to Many Users Metrics Unknown Setup Timing Etc Metrics Unknown Metrics Unknown Latency, Jitter Etc. No OS Delay Metrics Unknown Thru Put, Loss, Etc. Not Wire Speed No Accuracy Wire Speed Nanosec Accuracy No Network Measurement Full Visibility of Network Elements Only Round Trip Times One Way Times, Accurate GPS Timing 24

13 Performance Test Tools Links PC Test Tool LanMark Pro etestinglabs /filters/benchmarks/ WebBench NetBench Appliance SmartWindows SmartFlow SmartTCP TCLGui 25 Application vs. Performance Test Tool Application Tool Not Scalable Real World Combine and Prosper Performance Tool Scalable Not Real World 26

14 Type of Test 1.Performance 2.Capacity 3.Correct Operation 4.Resilience 5.Security Types of Tests Typical Tests Tests for delays in sending data or setting up calls Tests for variation in delay and effect on applications Tests for user application responsiveness Tests use of theoretical bandwidth Tests number of addresses/flows supported Tests data integrity and data loss/ RFC conformance Tests of device/network functionality under load Network ability to recover from errors or changes Map to Service Level Agreement Tests loss of security under load Tests loss of security based on port scanners Tests loss of security based on new attacks 27 Agenda Definition of Terms Performance Test Tools Testing Methods and Metrics Example Buffering and Architecture Example Flow Lookup and Architecture Example WRR and Architecture Example SLB and Architecture 28

15 Resource Constraints and Bottlenecks Application Characteristics Buffering, queuing, and latency Interface and pipe sizes, speed Control plane, IP flow speed (if flow-based) Memory, IP flow capacities, buffering capacities, control plane and data plane 29 Relational Perflow Metrics Latency Latency Capacity Capacity Rate Any One Change Affects the Other 30

16 Relational Layer 2 Metrics MAC MAC Addresses Addresses Latency Latency Frame ze Packets-Per-Second 31 Relational Layer 3 Metrics Latency Latency Sequence Sequence IP IP Addresses Addresses IP IP Flows Flows + Layer 2 IP Flow Setup Rate 32

17 Relational Layer 4 Metrics TCP/UDP TCP/UDP Session Session Latency Latency Sequence Sequence Capacity Capacity + Layer 3 TCP/UDP Flow Setup Rate 33 Relational Layer 7 QoS Latency Latency Sequence Sequence Application- Application- Definitions Definitions + Layer 4 Application Recognition Rate 34

18 Relational Layer 7 Termination Latency Latency Sequence Sequence Terminated- Terminated- Sessions Sessions + Layer 7QoS Application Termination Rate 35 Agenda Definition of Terms Performance Test Tools Testing Methods and Metrics Example Buffering and Architecture Example Flow Lookup and Architecture Example WRR and Architecture Example SLB and Architecture 36

19 Buffering and Switching Architectures Congestion management Multi- Gigabit Switch Fabric Switching decision CPU 37 Congestion Management Required when multiple ports are contending for the same port Congestion occurs during short bursts 38

20 Closet/Server Buffering File Server Output Buffer Remember that all networks have hot spots of congestion Output ports only transmit one frame-at-a-time 39 High-Speed Buffering Output Buffer 1 Gigabit Ethernet port at 100% fills a 500 KB buffer in: 4.1 ms with 1500 Byte MTU 5.5 ms with 64 Byte MTU 40

21 Instantaneous Over Subscription Output Buffer Even if bandwidth of two input ports are ½ the speed of the output port congestion will occur! 41 Dynamic Buffer Queuing Three 64-Byte Frames Use 192 Bytes of Memory Unused Memory Each buffer element fixed in small increments (for example, 64 bytes each) Allows for efficient use of buffers More expensive One 256-Byte Frame Uses 256 Bytes of Memory 42

22 Fixed Buffer Queuing One 64-Byte Frame Uses One 2000-Byte Buffer Wasted Memory One 256-Byte Frame Uses One 2000-Byte Buffer Buffer element is a fixed size Inefficient use of buffers Less controller overhead by 31 times easier to make a faster switch! 43 Dynamic vs. Fixed Buffering Dynamic Ethernet Frame ze Voice Data Throughput Large Small Buffers Available Fixed ATM Cell ze Voice Data Requires SAR Only Small Buffers Available 44

23 Buffering Tests 2:1 Backbone Test Two 1000 Mbps one 1000 Mbps Burst frames for 2:1 gigabit Ethernet ports at 5 50% utilization Packet Blaster Device Under Test 45 Buffering Capacity Packets Buffered Dynamic 500K Buffer 64 byte Elements 64 Byte Fixed 500K Buffer 2 Kbyte Elements 512 Byte 1518 Byte 46

24 Buffering Capacity Packets Buffered Byte 512 Byte 1518 Byte Dynamic 500K Buffer 64 byte Elements Fixed 500K Buffer 2 Kbyte Elements 47 Buffering Test 2:1 Packet Blaster Output Port at 100% Utilization 96 nsec IFG Backplane to Port Down Speed reduces IFG Input ports at 50% Utilization 768 nsec IFG (Interframe-Gap)* Device Under Test *Note: Input ports should vary 5 50% pertrial thus output utilization % 48

25 Buffering Test 2:1 Packet Blaster Output Port at 100% Utilization 96 nsec IFG (Inter-frame-Gap) Burst 3000 packets with 96 nsec IFG (Inter-frame-Gap) Device Under Test 49 Fixed Queuing Latency Distribution 2:1 Number of Packets % Input 20% Input 30% Input 40% Input 50% Input <=10uSec <=100uSec Latency Distribution 50

26 Dynamic Latency Distribution 2:1 Number of Packets % Input 20% Input 30% Input 40% Input 50% Input <=10uSec <=100uS <=500uS <=1000uS <=5000uS Latency Distribution 51 Buffering Results 2:1 Percent Lost with 64 Byte Frames Input Utilization 10% 20% 30% 40% 50% Static Buffer Dynamic Buffer 52

27 Buffering Tests 40:1 Packet Blaster Closet Test Mbps One 1000 Mbps Burst frames for 20:1 100 Mbps Ethernet ports at 1 5% utilization Device Under Test 53 Test Results 64 bytes 10 ppb % Throughput Dynamic Buffering Fixed Buffers Number of Ports 54

28 Test Results 500 bytes 10 ppb % Throughput Number of Ports Dynamic Buffering Fixed Buffers 55 Test Results 1518 bytes 10 ppb % Throughput Number of Ports Dynamic Buffering Fixed Buffers 56

29 Test Results 64 bytes 24 ppb % Throughput Number of Ports Dynamic Buffering Fixed Buffers 57 Test Results 500 bytes 24 ppb % Throughput Number of Ports Dynamic Buffering Fixed Buffers 58

30 Test Results 1518 bytes 24 ppb % Throughput Number of Ports Dynamic Buffering Fixed Buffers 59 Agenda Definition of Terms Performance Test Tools Testing Methods and Metrics Example Buffering and Switch Architecture Example Flow Lookup and Architecture Example WRR and Architecture Example SLB and Architecture 60

31 Flow Lookup and Switch Architectures Congestion management Multi- Gigabit Switch Fabric Switching decision CPU 61 Route Caching (Flow-Based Switching) First packet processed switched by route processor Flow cache enabled (centrally and/or locally) Route Processor A3 BB C1 Routing Table Ethernet 1 Gig E-net 1/0 Fast Ethernet 2/0 Central Routing Cache SA DA Interface C1 BB FF 2/0 Switching Fabric Local Route Cache Local Route Cache 62

32 CPU-Control Plane CPU-Control Plane Run routing protocols Maintain routing tables Check for CLI commands Accounting counters ICMP Queue packets, etc Multi- Gigabit Switch Fabric CPU-Control Plane Issues Control Plane must manage flow setup and system management What happens to Spanning Tree, VRRP and routing updates if the CPU is busy processing flows? CPU 63 Cisco Express Forwarding (FIB-Based Switching) No process switching necessary FIB based on routing table entries, not traffic flows Route Processor Forwarding Information Base A3 BB C1 Routing Table Ethernet 1 Gig E-net 1/0 Fast Ethernet 2/0 SA DA Interface C1 BB FF 2/0 Switching Fabric FIB can be kept central or distributed Local FIB Local FIB 64

33 IP-Host Flow Test Input Port Cache 1000 IP Flows 1000 IP Flows IP Destination 4 bytes Mask 4 bytes Output Port ID Device Under Test 1.1.X.X X.X IP-Recursive Flow Test Input Port Cache 1000 IP Flows 1000 IP Flows IP Destination 4 bytes Mask 4 bytes Output Port ID Subnet-Mask System Under Test Device Under Test 1.1.X.X 1.2.X.X 1.3.X.X 1.4.X.X 66

34 Flow Test Results Loss Packets Transmitted DUT-A DUT-B DUT-CEF Number of Frames Received out of Flow Test Results Jitter Latency in Microseconds DUT-A DUT-B DUT-CEF Maximum Latency Minimum Latency 68

35 Flow Test Results Raw using Smart Windows L3Tests 1 Port VTE Sequence TxTime RxTime DUT-A Flow Test Results-Setup Time 70

36 Agenda Definition of Terms Performance Test Tools Testing Methods and Metrics Example Buffering and Switch Architecture Example Flow Lookup and Architecture Example WRR and Architecture Example SLB and Architecture 71 ToS TX r IP Flows 11 IP Weighted-Round Robin Queues Test Input Port Cache and Buffer RX r IP Lookup and/or Classification Queue Depth Device Under Test WRR Scheduler 1.1.X.X X.X

37 IP WRR Baseline No Congestion 73 IP WRR With Congestion Scheduling Latency in MicroSeconds Scheduling Interval in Number of Packets 74

38 Agenda Definition of Terms Performance Test Tools Testing Methods and Metrics Example Buffering and Switch Architecture Example Flow Lookup and Architecture Example WRR and Architecture Example SLB and Architecture 75 Centralized L4 Switching More memory = More connections Centralized State Centralized Persistence Physical port independent Central CPU Forwarding or Routing Table TCP Port! Real Server Switching Fabric 76

39 Centralized L7 Switching One place to keep persistent information Cookies URL SSL Works with and without interface caching Interface punts to CPU Cache examples- Vito,C6k(Dispatch) IOSSLB & ASLB, Arrowpoint Central CPU Forwarding or Routing Table Layer7! Real Server Switching Fabric 77 Layer 4 Load Balancing MAC IP TCP/UDP 48 bytes Layer 4 SYN SYN ACK ACK HTTP Get HTTP Res.1.. FIN CPU L4 Lookup SYN SYN ACK ACK HTTP Get HTTP Res.1.. FIN 78

40 Layer 7 Load Balancing HTTP 1.0 ngle HTTP Get Per-Session MAC IP TCP/UDP HTTP GET Seq Seq.200 TCP Seq SYN Termination SYN ACK ACK HTTP Get1 HTTP Res1.. FIN CPU URL Lookup Seq.400 SYN SYN ACK ACK HTTP Get1 HTTP Res1.. FIN 79 Layer 7 Load Balancing HTTP 1.1 Multi HTTP Get Per-Session MAC IP TCP/UDP HTTP GET SYN SYN ACK ACK HTTP Get1 HTTP Res1 HTTP Get2 HTTP Res2 FIN TCP Termination CPU URL Lookup SYN SYN ACK ACK HTTP Get1 HTTP Res1 HTTP Get2 HTTP Res2 FIN 80

41 Layer 4 Testing-SmartTCP MAC IP TCP/UDP SYN SYN ACK ACK 48 bytes Layer 4 CPU L4 Lookup SYN SYN ACK ACK 3-Packet Connection Setup Time-Number of Connections 81 Layer 4 Testing-SmartTCP 4 Client s 1M-60k Open Connections at 3,000 Conns-Per-Second Per-card DUT 4 Server s Responding Test Completes as a BURST of connections NOT continuous 82

42 Layer 4 Testing-SmartTCP 3-Packet Connection Setup Rate Connection Setup Time Session Capacity Session Rate 6-Packet 83 Layer 4 Testing-SmartTCP 84

43 Layer 4 Testing-SmartTCP RATE x (NUMBER OF CLIENT CARDS) = TOTAL NUMBER OF CONNECTIONS PER SECOND 85 Layer 4 Testing (no data) MAC IP TCP/UDP SYN SYN ACK ACK FIN FIN ACK ACK 48 bytes Layer 4 CPU L4 Lookup SYN SYN ACK ACK 6-Packet Connection Setup Time-Number of Connections FIN FIN ACK ACK 86

44 Layer 4 Testing Input Port Cache 2000 CPS per Client- Server Pair 2000 CPS per Client- Server Pair IP Destination 4 bytes Mask 4 bytes TCP Port Sequence# Cookie URLID Output Port ID Device Under Test 87 Layer 7 Testing 1.0 vs. 1.1 MAC IP TCP/UDP HTTP GET SYN SYN ACK ACK HTTP Get1 HTTP Res1 HTTP Get2 HTTP Res2 FIN TCP Termination CPU URL Lookup SYN SYN ACK ACK HTTP Get1 HTTP Res1 HTTP Get2 HTTP Res2 FIN 88

45 Layer 7 Testing 1.0 vs. 1.1 Device Lookup Table 2000 CPS per Client- Server Pair 2000 CPS per Client- Server Pair IP Destination 4 bytes Mask 4 bytes TCP Port Sequence# Cookie URLID Output Port ID Device Under Test 89 Layer 7 Testing 1.0 vs. 1.1 WebStone WebBench-Ziffdavis (400cps) Best functionality tool Tarantula-Cisco/Arrowpoint (4000cps) High Performance Socrates-Very Easy to setup Good functionality tool 90

46 Performance Metrics for Multilayer Switches Session 91 Please Complete Your Evaluation Form Session 92

47 93