Here is a summary of the recommendations that have been reviewed and approved by NETS engineers:

Transcription

1 QOS Review: This is a review of NCAR s current campus network and the potential FRGP QOS implementation. Since NCAR s initial QOS deployment, new hardware and IOS versions have been deployed in the network, CatOS has been converted to IOS, and the QOS toolset has changed. The intent of this document is to validate the existing QOS configuration and/or propose new QOS configurations using QOS best practices outlined in Cisco s Enterprise QoS Solution Reference Network Design (Enterprise QoS SRND) as a guideline. The intent of NCAR s QoS deployment is to ensure VOIP traffic has priority over other traffic in the face of network congestion. QoS is not a one size fits all configuration it is highly dependent on company policy decisions (i.e. prioritize application X) and on the overall composition of network traffic. NCAR s QoS implementation should be reviewed on a regular and as-needed basis to ensure QoS is functioning as designed when new hardware or changes in traffic profiles occur. Here is a summary of the recommendations that have been reviewed and approved by NETS engineers: Cisco s Enterprise QoS and Medianet Campus Qos SRNDs will be used as the basis for QoS configurations Input queuing and policing will not be used Only voice traffic will be prioritized Transmit queue design will use the default settings applied by the mls qos command except in a few cases where the default deviates from the SRND Cut and paste QoS files will be created for each queue type and be used as the default configuration for new cards Packet drops per queue and threshold will be monitored to ensure QoS policy is working properly Netflow will be enabled on internal NCAR traffic for additional QoS monitoring and statistics This document is organized in the following manner: 1. QoS Overview Review of QoS concepts and config examples 2. NCAR s Existing QoS configuration - Review of NCAR s current QoS configurations 3. NCAR s queue types 4. NCAR s traffic profile VOIP traffic vs Other traffic 5. QoS Configuration Recommendations 6. Monitoring QoS 7. Telepresence and QoS at NCAR 8. Telepresence and QoS at the FRGP 9. References 1. QoS Overview

2 There are many different ways to implement QoS but the SRND recommends using DSCP markings to ensure consistent PHB (per hop behavior) of packets. A DSCP based QoS deployment consists of the following components: 1. Input Queuing 2. Classification and Marking 3. Policing 4. Transmit Queuing The example commands presented below are for a 6500 with 1p3q8t queue. To complicate matters, the QoS implementation varies across the different hardware platforms and IOS versions. Commands may vary slightly depending on switch and/or queue type, however the basic procedure remains the same. Input Queuing Ingress queues (RX) are extremely difficult to congest. Ingress congestion implies that the combined ingress rates of traffic exceed the switch fabric channel speed, and thus would need to be queued simply to gain access to the switching fabric. On newer platforms, such as the Catalyst 6500 Sup720, this means that a combined ingress rate of up to 40 Gbps per slot would be required to create such an event*. The 6500 line cards do have ingress queues in the event that congestion does occur, but since this is highly unlikely, only the egress (TX) queues will be addressed in this document. * End-to-end qos network design, By Tim Szigeti, Christina Hattingh Classification and Marking The Enterprise QoS SRND follows standards-based DSCP PHB markings to ensure interoperability and future expansion. A general rule of thumb is to classify traffic as close to the source as possible and to establish trust boundaries. At the edge of our network, the Cisco IP phones appropriately mark their traffic as voice bearer (COS 5, DSCP 46 EF) and voice signaling traffic (COS 3, DSCP 24 CS3), so we ll trust the DSCP markings from the phone. A PC attached to the phone s PC port will not be trusted - should the PC try to send packets marked other than DSCP = 0, the switch will remark the packets to DSCP = 0. By default, the 6500s, with QoS enabled, will mark packets with DSCP 0 from any untrusted source. The trust boundary can be established on a per port or per vlan basis. Cisco recommends per vlan when possible. Once classification is performed at the edge for inbound traffic, the router/switch may need to map COS to DSCP markings (and vice-versa) before transmitting the packet. Use the IOS commands below to configure the mapping (global config):

3 mls qos map cos-dscp mls qos map dscp-cos Here is the CatOS command (global): set qos cos-dscp-map set qos dscp-cos-map The DSCP/COS values are then trusted on the uplinks from Access to Distribution, and Distribution to Core. Policing Policing can be used to markdown excess traffic in a class or drop offending traffic. Policing will not be used in NCAR s campus network at this time. Transmit Queuing Transmit queue types are hardware dependant. IOS command show interface capabilities -> QoS scheduling displays the type: ml-16c-c1-gs#show interfaces capabilities mod 4 GigabitEthernet4/1 Model: WS-X6148A-GE-45AF Type: 10/100/1000BaseT Speed: 10,100,1000,auto Duplex: half,full Trunk encap. type: 802.1Q,ISL Trunk mode: on,off,desirable,nonegotiate Channel: yes Broadcast suppression: none Flowcontrol: rx-(off,on,desired),tx-(off,on,desired) Membership: static Fast Start: yes QOS scheduling: rx-(1q2t), tx-(1p3q8t) The CatOS command is show qos info config 1p3q8t tx In this example, the transmit queue is 1p3q8t. 1p = one priority queue. Traffic in the priority queue is serviced until the queue is empty before the other queues are serviced. 3q = there are three queues 8t = each queue has 8 drop thresholds that can be configured (does not apply to the priority queue) For each queue type, the following needs to be configured: Map traffic to queue

4 Map traffic to queue Set queue buffer size Set queue WRR weights Set queue WRED thresholds There are a number of RFC s (2474, 2597, 3246, 4595) which describe the PHB for the different DSCP markings. Proper queue design (buffer size, WRR weights, and WRED thresholds) determines whether the queue behaves as Best Effort or CS3 or some other type. DSCP marked traffic must be mapped to a queue designed to handle that type of traffic. Cisco has taken all these factors into account and provided queuing models in the SRND for all the different types of queues found in their hardware: Book.html Here are the IOS commands (per interface): wrr-queue cos-map <queue> [<thresh>] <cos> <cos> priority-queue cos-map <queue> <cos> <cos> Here is the CatOS command (global) set qos map 1p2q2t tx 3 1 cos 5 Set queue buffer size The size of the buffers must be set. Typically the lower weighted queues will have larger buffers since they are not serviced as often. Here is the IOS command to set the buffers (per interface): wrr-queue queue-limit priority-queue queue-limit Here is the CatOS command (global): set qos txq-ratio 1p2q2t Set queue WRR weights With WRR, the switch uses a configured weight value for each egress queue. This weight value determines the implied bandwidth of each queue. The higher the weight value, the higher the priority that the switch applies to the egress queue. Priority queues do not have assigned weights and are serviced before other queues, until empty. For example, consider the case of a Catalyst 3550 switch configured for QoS and WRR. The Catalyst 3550 uses four egress queues. If queues 1 through 4 are configured with weights 50, 10, 25, and 15, respectively, queue 1 can utilize 50 percent of the bandwidth when there is congestion. Here is the IOS command to set the queue weights (per interface):

5 wrr-queue bandwidth Here is the CatOS command (global): set qos wrr 1p2q2t Set queue WRED thresholds There is a minimum and maximum WRED (weighed random early detection) threshold configured. The minimum indicates the point at which packets will be randomly dropped for that queue/threshold and the MAX threshold determines when tail drop (i.e. all pkts in the queue) will begin. Here is a link to how WRED works: 9 IOS example (per interface):! Sets Min WRED Threshold for Q1T1 to 80% and all others to 100% wrr-queue random-detect min-threshold ! Sets Max WRED Threshold for Q1T1 to 100% and all others to 100% wrr-queue random-detect max-threshold CatOS example (global): set qos wred 1p3q8t tx queue 3 <[thr1lo:]thr1hi> <[thr8lo:]thr8hi> 2. NCAR s Existing QoS configurations The existing QOS policy is, where possible, to use the default configuration when mls qos global command is applied. Exceptions to the default configurations are COS to DSCP mappings, and queue/threshold mappings for COS 3 and COS 5 marked traffic. Here is a typical configuration from one of our CATOS switches. set qos enable set qos map 2q2t tx 2 1 cos 3 set qos map 2q2t tx 2 2 cos 5 set qos wrr 2q2t set qos map 1p2q2t tx 2 1 cos 3 set qos wrr 1p2q2t set qos wred 1p2q2t tx queue 1 40:80 70:100 set qos wred 1p2q2t tx queue 2 40:80 70:100

6 set qos map 1p2q1t tx 2 1 cos 3 set qos map 1p3q8t tx 3 3 cos 3 set qos cos-dscp-map set qos ipprec-dscp-map clear qos acl all #ACL-IP-PHONES set qos acl ip ACL-IP-PHONES trust-cos ip any any #ACL-TRUST-DSCP set qos acl ip ACL-TRUST-DSCP trust-dscp ip any any #ACL-TRUST-IPPREC set qos acl ip ACL-TRUST-IPPREC trust-ipprec ip any any # commit qos acl all set qos acl map ACL-IP-PHONES 700 A survey of all the CATOS switches showed that queue mapping is performed for all the queues, but buffer sizes, weights, and thresholds are not consistently applied to the different queues. A survey of IOS switches shows that qos is enabled globally, but are currently using the defaults for all other settings. In CATOS, show qos info config 1p3q8t tx shows the current queue settings (for IOS its show queueing interface <interface name> ): QoS setting in NVRAM for 1p3q8t transmit: QoS is enabled Queue and Threshold Mapping for 1p3q8t (tx): Queue Threshold CoS Tx drop thresholds: Tx drop-thresholds feature is not supported for this port type. Tx WRED thresholds: Queue # Thresholds - percentage %:100% 100%:100% 100%:100% 100%:100% 100%:100% 100%:100% 100%:100% 100%:100% 2 70%:100% 100%:100% 100%:100% 100%:100% 100%:100% 100%:100% 100%:100% 100%:100% 3 40%:70% 40%:70% 50%:80% 50%:80% 60%:90% 60%:90% 70%:100% 70%:100% Tx queue size ratio: Queue # Sizes - percentage %

7 2 15% 3 15% 4 5% Tx WRR Configuration of ports with 1p3q8t: Queue # Ratios NCAR Queue Types Model Description Queue Type Scheduler VS-S720-10G Supervisor Mod G (enabled/disabled G ports) 1p3q4t/1p7q4t DWRR or SRR WS-X6K-S2-MSFC2 2 port 1000BaseX Supervisor Mod 2 (GBIC) 1p2q2t WS-X6K-S2U-MSFC2 2 port 1000BaseX Supervisor Mod 2 (GBIC) 1p2q2t WS-X6K-SUP2-2GE 2 port 1000BaseX Supervisor Mod 2 (GBIC) 1p2q2t WS-SUP32-10GE-3B 2 port 10GBaseX Supervisor module 1p3q8t DWRR or SRR WS-SUP720-3B Supervisor Mod 720 base board 1p2q2t WRR WS-SUP720-3BXL Supervisor Mod 720 base board 1p2q2t WRR WS-SUP720-BASE Supervisor Mod 720 base board 1p2q2t WRR WS-X6148-RJ45V 48 port 10/100BaseTX (RJ-45) 2q2t WRR WS-X6348-RJ port 10/100BaseTX (RJ-45) 2q2t WRR WS-X6148A-GE-45AF 48 port 10/100/1000BaseTX (RJ-45) 1p3q8t WRR WS-X6408A-GBIC 8 port 1000BaseX (GBIC),Enhanced QoS 1p2q2t WRR WS-X6416-GBIC 16 port 1000BaseX (GBIC) 1p2q2t WRR WS-X6516A-GBIC 16 port 1000BaseX (GBIC) 1p2q2t WRR WS-X GE 4 port 10 GE 1p7q8t DWRR WS-X GE 16 port 10 GE 1p7q4t DWRR or SRR WS-X6748-GE-TX 48 port 10/100/1000 (RJ-45) 1p3q8t DWRR WS-X6748-SFP 48 port 1000Base FX (SFP GBIC) 1p3q8t DWRR 3560E 1p3q3t SRR IE3000 Industrial Ethernet Switch 1p3q3t SRR NME-XD-48ES-2S-P NME-XD-48ES-2S-P: EtherSwitch SM 48 10/100T PoE + 2 SFP 1p3q3t SRR 4. NCAR Traffic Profile Table 1: NCAR Queue Types To get a rough sense of the percentage of VoIP traffic on the network vs. other traffic, in/out bps was pulled from all the VLANs at ML, FL, and CG for a 31 day period. For voice vlans, the max values were pulled along with average values to evaluate the worst case scenario. The table below shows percent of VoIP (all traffic on voice vlans) traffic per campus, which was typically less than 1%.

8 Inbound (%) Outbound (%) % CGRA VOIP traffic % CGRA MAX VOIP traffic % FLRA VOIP traffic % FLRA MAX VOIP Traffic % MLRA VOIP Traffic % MLRA MAX VOIP Traffic QoS Configuration Recommendations AutoQoS vs. Manual configuration AutoQoS will automatically apply QoS configurations on a switch. Lab tests show that AutoQoS selected different buffer sizes, weights and thresholds than what is recommended in the SRND. Manual configuration is recommended. Input Queuing and Policing Input queuing and Policing will not be used at this time per the reasons discussed above. Classification and Marking: The SRND classifications recommendations assume that all traffic will be classified and have configured transmit queues accordingly. To keep things simple, only voice bearer and call signaling traffic will be explicitly prioritized on NCAR s campus. Queue configurations will therefore need to be adjusted accordingly. The proposed classification of NCAR s campus traffic is as follows: 1. Prioritize voice bearer traffic as COS 5, DSCP 46 EF 2. Prioritize call signaling traffic as COS 3, DSCP 24 CS3. 3. Router/switch control traffic (i.e. routing protocols, HSRP, etc) are marked as CS6 by the router/switch. 4. All other traffic is marked with the default COS 0, DSCP 0. All of NCAR s campus traffic will be placed into a 4 level classification policy. Additional classification levels can easily be added should we need to prioritize traffic other than voice. Application L3 Classification Voice EF, DSCP 46 Network Control CS6, DSCP 48 Call Signaling CS3, DSCP 24 All Other DSCP 0

9 Trust boundaries will be extended to the IP phones, all other *access* ports will be untrusted. All uplinks between access and distribution/core switches will be trusted. Here is a sample IOS configuration that extends the trust boundary to the IP phone: Per Interface Configs: interface gig <mod/port> mls qos vlan-based switchport voice vlan 700 <- Applied on interface Vlan700 no ip address service-policy input ACL-IP-PHONES-policy Global Configs: class-map match-all ACL-IP-PHONES-1-class match access-group name ACL-IP-PHONES-1! policy-map ACL-IP-PHONES-policy class ACL-IP-PHONES-1-class trust dscp Here is a sample IOS configuration to enable trust on an uplink (per interface): mls qos trust cos COS to DSCP and DSCP to COS mappings will remain unchanged. Transmit Queuing, Generic Proposal: Transmit queuing is partially determined by the hardware (1p3q8t, 2q2t, etc) and IOS version being used on the switch/router. While different types of queuing and thresholds (WRR, SRR, WRED, WTD, etc) are implemented on the different platforms, the user has the ability to control buffer size, weights, and thresholds regardless of the type. The queues must behave in an appropriate manner as determined by the type of traffic (DSCP/COS) assigned to the queue. Based on the proposed classification of traffic, here are a couple of generic queue design proposals. Option 1: Use the SRND as a guide for designing queues, making adjustments as necessary based on our classification policy. 1. Mapping of DSCP/COS to queues will be performed per the SRND even though NCAR will only use 4 classification levels. In the future, should we decide to prioritize traffic other than voice, the mappings will already be in the standard configuration. I don t see any real downside to doing this.

10 2. Buffer space will only be allocated to queues with DSCP/COS values for which traffic has been explicitly mapped. For example, 1p7q8t has a total of 8 queues available 1 priority and 7 other queues. The proposed classification calls for at most 4 different queues, so 4 of the 1p7q8t queues will have a buffer size of 0 assigned since no traffic would ever be assigned to these queues. The SRND will be used as a guide to determine buffer size. It will be necessary to deviate from the SRND because we will be marking all traffic except voice as DSCP/COS 0 and a larger buffer will be needed to accommodate this traffic. 3. Bandwidth servicing weights will only be allocated to queues that have buffers assigned. The SRND will be used as a guide to determine bandwidth servicing weights. For the most part, these should follow pretty closely with the SRND except where buffers have been set to Buffer thresholds will be explicitly defined to queue s that have buffers assigned, for all DSCPs assigned to the queue regardless of whether or not any traffic will be marked with that DSCP value. The SRND will be used as a guide. For example, the SRND for 1p3q8t queues assigns CS7 to q3t7. The proposed classification does not include CS7, however, the q3t7 threshold will still be set. This will have no affect on traffic that is using the queue. PROs: CONs: Follows Cisco s recommendation for queue design which is standards based. Queue design is known and less likely to change due to IOS version changes. Cisco s queue design is based on marking/classifying ALL traffic. NCAR is only marking/classifying VOIP traffic, so adjustments will need to be made for buffers, weights, and thresholds. Queue design results in 14+ lines of config per port depending on queue type. Option 2 (RECOMMENDED): As a sanity check, we ran option1 past David and Option 2 came from that review. Option 2 primarily relies on the default values that result from enabling the mls qos global command (Currently implemented in NCAR s campus network CATOS devices). 1. Use the default mapping of DSCP/COS to queues that result from enabling the mls qos global command except where it differs for DSCP 24 (call signaling) and DSCP 46 (voice traffic). In those cases, follow the SRND. 2. Use the default buffers, bandwidth servicing weights, and thresholds that result from enabling the mls qos global command. For the most part, these settings are in line with SRND and where they deviate, it s hard to quantify how it might affect QoS.

11 PROs: Easier admin, fewer config lines per port Still maps COS to queues per SRND CONs: Queue design could change, if defaults change due to IOS upgrade Depending on queue type, some resources could go unused i.e. buffers, but it s hard to quantify how this would affect QoS (see 1p3q8t example below). Note that once a Tx queuing option has been agreed upon, it will need to be applied to all queue types described in Table 1. Transmit Queuing 1p2q2t: Below are some examples of 1p2q2t (older 1G uplink queue type) queue design based on option 1 and option 2. Option 1 compares three queue designs: SRND queue design if the SRND is exactly followed, Default queue design created by enabling mls qos global command (same for IOS and CATOS), and Proposed the proposed design as described above. Option 2 compares two queue designs: Default queue design created by enabling mls qos global command, and Proposed the proposed design as described above. The grayed out COS/DSCP values represent unused COS/DSCP values. For example, no traffic will be marked as AF21, CS2, COS2. The blue and orange cell colors are used to visually separate the queues and have no other meaning.

12 OPTION 1 SRND 1p2q2t - Buffer Size% & Threshold BW Servicing Weight COS to Threshhold Map Value n/a EF, Cos5 n/a Default 1p2q2t - Buffer Size% & Threshold BW Servicing Weight COS to Threshhold Map Value n/a EF, Cos5 n/a Proposed (based on SRND) 1p2q2t - Buffer Size% & Threshold BW Servicing Weight COS to Threshhold Map Value n/a EF, Cos5 n/a Priority Queue - 30% 70 CS7; Cos7 q2t2 80:100 CS6; Cos6 AF31,CS3; Cos3 AF21,CS2; Cos2 q2t1 40:80 AF41,CS4; Cos4 Queue 2-40% 30 DF; Cos0 q1t2 80:100 CS1; Cos1 q1t1 40:80 Priority Queue - 15% 255 CS7; Cos7 q2t2 70:100 CS6; Cos6 AF41,CS4; Cos4 q2t1 40:70 Queue 2-15% 100 (5 catos) AF21,CS2; Cos2 AF31,CS3; Cos3 CS1; Cos1 DF; Cos0 q1t2 70:100 q1t1 40:70 Priority Queue - 15% 70 CS7; Cos7 CS6; Cos6 q2t2 80:100 AF31,CS3; Cos3 AF21,CS2; Cos2 q2t1 40:80 AF41,CS4; Cos4 Queue 2-15% 30 DF; Cos0 q1t2 80:100 CS1; Cos1 q1t1 40:80 Queue 1-30% Queue 1-70% Queue 1-70%

13 Option 1: Proposed IOS Queue Configuration Commands (per interface): wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue bandwidth wrr-queue random-detect 1 wrr-queue random-detect 2 wrr-queue random-detect min-threshold wrr-queue random-detect max-threshold Option 1: Proposed CATOS Queue Configuration Commands (per queue type): set qos map 1p2q2t tx 2 1 cos set qos map 1p2q2t tx 2 2 cos 6 7 set qos map 1p2q2t tx 1 1 cos 1 set qos map 1p2q2t tx 1 2 cos 0 set qos wred 1p2q2t tx queue 2 40:80 80:100 set qos wred 1p2q2t tx queue 1 40:80 80:100 set qos wrr 1p2q2t 30 70

14 Option 2 Default 1p2q2t - Buffer Size% & Threshold BW Servicing Weight COS to Threshhold Map Value n/a EF, Cos5 n/a Proposed (based on default) 1p2q2t - Buffer Size% & Threshold BW Servicing Weight COS to Threshhold Map Value n/a EF, Cos5 n/a Priority Queue - 15% 255 CS7; Cos7 q2t2 70:100 CS6; Cos6 AF41,CS4; Cos4 q2t1 40:70 Priority Queue - 15% 255 CS7; Cos7 q2t2 70:100 CS6; Cos6 AF41,CS4; Cos4 AF31,CS3; Cos3 q2t1 40:70 Queue 2-15% 100 (5 catos) AF21,CS2; Cos2 AF31,CS3; Cos3 CS1; Cos1 DF; Cos0 q1t2 70:100 q1t1 40:70 Queue 2-15% 100 AF21,CS2; Cos2 q1t2 70:100 CS1; Cos1 DF; Cos0 q1t1 40:80 Queue 1-70% Queue 1-70% Option 2: Proposed IOS Queue Configuration Commands (per interface): wrr-queue cos-map Option 2: Proposed CATOS Queue Configuration Commands (per queue type): set qos map 1p2q2t tx 2 1 cos 3 set qos wrr 1p2q2t Transmit Queuing 1p3q8t: Below are some examples of 1p3q8t queue design based on option 1 and option 2. Option 1 compares three queue designs: SRND queue design if the SRND is exactly followed, Default queue design created by enabling mls qos global command (different defaults for IOS and CATOS), and Proposed the proposed design as described above. Option 2 compares two queue designs: Default queue design created by enabling mls qos global command (different defaults for IOS and CATOS), and Proposed the proposed design as described above. The grayed out COS/DSCP values represent unused COS/DSCP values. For example, no traffic will be marked as AF21,CS2,COS2

15 OPTION 1 SRND BW Servicing Weight n/a p3q8t - Buffer Size% & COS to Threshhold Map EF, Cos5 Threshold Value n/a Priority Queue - 30% CS7; Cos7 q3t5 90:100 CS6; Cos6 q3t4 80:90 AF31,CS3; Cos3 q3t3 70:80 AF21,CS2; Cos2 q3t2 60:70 AF41,CS4; Cos4 q3t1 50:60 Queue 3-40% DF; Cos0 q2t1 80:100 Queue 2-25% CS1; Cos1 q1t1 80:100 Queue 1-5% Default CatOS BW Servicing Weight n/a p3q8t - Buffer Size% & Threshold COS to Threshhold Map Value EF, Cos5 n/a Priority Queue - 5% CS7; Cos7 CS6; Cos6 AF41,CS4; Cos4 q3t7 70:100 q3t5 60:90 AF31,CS3; Cos3 q3t3 50:80 Queue 3-15% CS1; Cos1 AF21,CS2; Cos2 q2t1 70:100 Queue 2-15% DF; Cos0 q1t1 70:100 Queue 1-65% Default IOS BW Servicing 1p3q8t - Buffer Size% & Threshold Weight COS to Threshhold Map Value n/a EF, Cos5 n/a Priority Queue - 15% CS7; Cos7 q3t1 70:100 CS6; Cos6 Queue 3-15% AF31,CS3; Cos3 AF41,CS4; Cos4 CS2:Cos2 q2t2 70:100 q2t1 40:70 40:70 Queue 2-20% CS1; Cos1 q1t2 70:100 DF; Cos0 q1t1 40:70 Proposed (based on SRND) BW Servicing 1p3q8t - Buffer Size% & Threshold Weight COS to Threshhold Map Value EF, Cos5 n/a n/a Priority Queue - 10% CS7; Cos7 q3t5 100:100 CS6; Cos6 q3t4 90: AF31,CS3; Cos3 q3t3 80:90 AF21,CS2; Cos2 q3t2 70:80 AF41,CS4; Cos4 q3t1 70:80 Queue 3-15% DF; Cos0 q2t1 80: Queue 2-75% CS1; Cos1 q1t1 80:100 Queue 1-0% 100 Queue 1-50%

16 Option 1: Proposed IOS Queue Configuration Commands (per interface): wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue queue-limit wrr-queue bandwidth wrr-queue random-detect 1 wrr-queue random-detect 2 wrr-queue random-detect 3 wrr-queue random-detect min-threshold wrr-queue random-detect max-threshold wrr-queue random-detect min-threshold wrr-queue random-detect max-threshold wrr-queue random-detect min-threshold wrr-queue random-detect max-threshold Option 1: Proposed CATOS Queue Configuration Commands (per queue type): set qos map 1p3q8t tx 1 1 cos 1 set qos map 1p3q8t tx 2 1 cos 0 set qos map 1p3q8t tx 3 1 cos 4 set qos map 1p3q8t tx 3 2 cos 2 set qos map 1p3q8t tx 3 3 cos 3 set qos map 1p3q8t tx 3 4 cos 6 set qos map 1p3q8t tx 3 5 cos 7 set qos txq-ratio 1p3q8t set qos wrr 1p3q8t set qos wred 1p3q8t tx queue 3 70:80 40:70 70: : : : : :100 set qos wred 1p3q8t tx queue 2 80: : : : : : : :100 set qos wred 1p3q8t tx queue 1 80: : : : : : : :100

17 Option 2 Default CatOS BW Servicing Weight n/a p3q8t - Buffer Size% & Threshold COS to Threshhold Map Value EF, Cos5 n/a Priority Queue - 5% CS7; Cos7 CS6; Cos6 AF41,CS4; Cos4 q3t7 70:100 q3t5 60:90 AF31,CS3; Cos3 q3t3 50:80 Queue 3-15% CS1; Cos1 AF21,CS2; Cos2 q2t1 70:100 Queue 2-15% DF; Cos0 q1t1 70:100 Proposed (based on Current/Default) BW Servicing 1p3q8t - Buffer Size% & Threshold Weight COS to Threshhold Map Value n/a EF, Cos5 n/a 20 Queue 1-65% Default IOS BW Servicing 1p3q8t - Buffer Size% & Threshold Weight COS to Threshhold Map Value n/a EF, Cos5 n/a 200 Priority Queue - 15% CS7; Cos7 q3t1 70:100 CS6; Cos Priority Queue - 15% AF31,CS3; Cos3 q3t3 70:100 CS7; Cos7 CS6; Cos6 q3t1 70:100 Queue 3-15% AF41,CS4; Cos4 q2t2 70:100 CS2:Cos2 q2t1 40:70 Queue 2-20% CS1; Cos1 q1t2 70:100 DF; Cos0 q1t1 40: Queue 3-15% AF31,CS3; Cos3 AF41,CS4; Cos4 CS2:Cos2 q2t2 70:100 q2t1 40:70 40:70 Queue 2-20% CS1; Cos1 q1t2 70:100 DF; Cos0 q1t1 40:70 Queue 1-50% 100 Queue 1-50%

18 Option 2: Proposed IOS Queue Configuration Commands (per interface): wrr-queue cos-map wrr-queue cos-map Option 2: Proposed CATOS Queue Configuration Commands (per queue type): set qos map 1p3q8t tx 1 2 cos 1 set qos map 1p3q8t tx 2 1 cos 2 set qos map 1p3q8t tx 2 2 cos 4 set qos map 1p3q8t tx 3 1 cos 6 7 set qos map 1p3q8t tx 3 3 cos 3 set qos txq-ratio 1p3q8t set qos wrr 1p3q8t set qos wred 1p3q8t tx queue 3 70:100 40:70 70:100 50:80 60:90 60:90 70:100 70:100 set qos wred 1p3q8t tx queue 2 40:70 70: : : : : : :100 set qos wred 1p3q8t tx queue 1 40:70 70: : : : : : :100 ****Recommend using Option 2**** Transmit Queuing, 3560E/3750/IE3000: The 3560E uses SRR (shaped round-robin) queue s with WTD (weighted tail drop). Weighted Tail Drop is a simpler congestion avoidance mechanism than WRED. The weight sets the % of the queue that, when full, will begin tail drop. The queue s can be configured as 4qt3 or 1p3q3t. We ll configure it to use 1p3q3t. We will add this configuration information based upon the decisions for the other module types. Transmit Queuing, Voice Gateways: As of T and later, the voice gateways, by default will mark voice traffic with DSCP 46 EF and call signaling with DSCP 24 CS3. Yet another queuing configuration format is used for voice gateways - HQF (Hierarchical queuing Framework). Mar-26a-c1-gw uses HQF on its T1 link. The SRND s for Enterprise QoS and Unified Communications don t explicitly cover this. From a BW perspective, the links are lightly used and even if all PRI s were in use, the max load on the uplinks would be : 3 pri * 24 calls/pri * 80kbps/call = 7.68 Mbps. For Gig uplinks, that would consume less than 1% of available BW. Recommendation: No changes.

19 Applying QoS Configurations: Smart port macros are available, and we could create one macro per queue configuration type. The other option is to follow our current setup of configuring the modules as we insert them. The settings are determined by the modules, so this would be a good way to keep us in sync as we change modules. When the macro is applied to the interface, the configuration shows all the commands and which macro was applied. Here is an example of a 1p3q8t port configured using a macro: interface GigabitEthernet3/1 no ip address shutdown wrr-queue bandwidth wrr-queue queue-limit wrr-queue random-detect minthreshold wrr-queue random-detect min-threshold wrr-queue random-detect max-threshold wrr-queue random-detect max-threshold wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map wrr-queue cos-map macro description pd-test end Note that MAX thresholds should be configured before MIN thresholds. This will avoid errors where the new MIN threshold might be greater than the existing MAX threshold. 6. Monitoring QoS QoS configurations need to be validated to ensure packets are properly marked and monitored to ensure they are operating as designed. To validate configurations, grab a packet trace (or use netflow) at the end points and check that in/out bound DSCP markings are set correctly (SCCP = CS3; RTP = EF routing protocols = CS6, and all other set to 0).

20 Locations to check o o o Switch ports that connect to CMs and voic servers Switch ports that connect to gateways PC port on phone (random sampling). Config checkers should be used to validate that configurations are applied properly. Queue drops can be monitored per interface, per queue and per threshold. Excessively high drops in a particular queue could indicate the need to adjust queuing parameters. The CLI command 'show queuing interface <blah> detailed' provides per interface, per queue and threshold drops, however, the "detailed" portion of this command is only available in IOS ver x.x.x SXI. These are the OIDs: IOS CISCO-SWITCH-QOS-MIB, csqifstatsdroppkts CATOS CISCO-CATOS-ACL-QOS-MIB Here is an IOS example: SNMPv2-SMI::enterprises = Counter64: SNMPv2-SMI::enterprises = Counter64: 0 SNMPv2-SMI::enterprises = Counter64: 0 SNMPv2-SMI::enterprises = Counter64: 0 SNMPv2-SMI::enterprises = Counter64: 0 SNMPv2-SMI::enterprises = Counter64: 0 SNMPv2-SMI::enterprises = Counter64: 0 SNMPv2-SMI::enterprises = Counter64: 0 SNMPv2-SMI::enterprises = Counter64: 0 SNMPv2-SMI::enterprises = Counter64: SNMPv2-SMI::enterprises = csqifstatsdroppkts. The next number is the ifindex of the interface, followed by the direction (1=inbound, 2=outbound), then by the queue, and finally the threshold. We currently lack the ability to determine amount of traffic per COS/DSCP which could be useful for planning and troubleshooting purposes. This would be remedied if we ran netflow internally.

21 7. Telepresence (TP) and QoS at NCAR Here is a brief review of some key characteristics of Telepresence (TP): TP reserves /24 thru /24 for internal use but does not have a default gateway and therefore cannot route beyond the TP codec. TP appears as two SIP endpoints in CM, 1) the TP codec 2) 7975 IP phone Bandwidth requirements for TP: Mbps to Mbps per stream depending on configuration End to End Jitter target is < 10ms. Jitter greater than 20ms peak to peak will downgrade call quality. Jitter 40ms or greater call will terminate. Target packet loss is 0.5% Marks video traffic as CS4, voice as CS5, and signaling as CS3. CS4 marked traffic is to be assigned to the priority queue The SRND recommends a separate cluster for TP, unless there are no other video devices on the existing cluster There are many different ways to deploy TP, i.e. point-to-point, point-to-multipoint, intra-enterprise, inter-enterprise, etc. If TP were to be deployed on NCAR s campus, qos configurations would also have to be adjusted. Using the QoS configuration recommendations from above as a foundation, here is an overview of the changes that would be needed assuming an Intra-NCAR TP deployment: Input Queuing: No changes Classification and Marking: Extend trust boundary to TP equipment; configure CM to mark video traffic as CS4 Policing: No changes Transmit Queuing: Re-map CS4 marked traffic to the priority queue; possibly re-allocate buffers 8. Telepresence (TP) and QoS at the FRGP The FRGP uses DSCP markings to facilitate commodity policers at the FRGP. Inbound commodity traffic, destined for an FRGP/UPOP member not directly connected to the same router as the commodity provider, is marked as DSCP 1. To implement the policer strategy, QoS was globally enabled on the 6500s at the FRGP. QoS is NOT implemented at the FRGP to provide preference to specific traffic types i.e. voice, video, etc except for VOIP traffic destined to/from RAF. In this case, we trust the QoS markings along the path. Telepresence (or other video conferencing) capabilities appear to be on the horizon for FRGP member institutions either via NLR, between FRGP members, or a single FRGP member with multiple connections

22 to the FRGP. Given the strict latency, jitter, and packet loss requirements for TP, it s possible that member institutions will request preferential treatment of TP traffic transiting the FRGP routers. A single FRGP member requesting preferential treatment for video will impact all FRGP members in all but the simplest case (ports are on the same router) because some QoS policy would need to be applied on ports connecting l3-gw-1 <-> frgp-gw-2 <-> frgp-gw-3. At that point, one member s traffic is being preferred over all of the other member s traffic. In all the use cases mentioned above, the FRGP is the middleman transiting traffic from participant A to participant B. A generic path may look like this: particpanta s internal network <-> Tail circuit provider network<->frgp<-> Tail Circuit provider network<->participantb s internal network. In some cases, there may be a direct connection to a participant s network, eliminating the tail circuit provider network. Telepresence marks video CS4, voice CS5, and signaling as CS3 (this is configurable at the TP end). Both CS4 and CS5 are recommended to be placed in the priority queue. Ultimately we don t really care how Telepresence is deployed in the participant s networks, but we will need to know the COS/DSCP types and bandwidth requirements for each COS/DSCP marking. Should it be deemed necessary to implement QoS at the FRGP, a QoS policy will need to be created that clearly defines the number and types of queues and COS/DSCP mappings to those queues. When deploying QoS, coordination with participants and tail circuit vendors will be necessary to determine a mapping between the different queue structures. If they don t match, then traffic will need to be remarked either inbound and/or outbound to ensure similar treatment across different administrative boundaries. Participants will be responsible for re-marking. Using the QoS configuration recommendations from above as a foundation, here is an overview of the changes that would be needed to create an FRGP QoS policy recommendation: Input Queuing: No changes Classification and Marking: o Inbound Option 1: Extend trust boundaries to participant/tail circuit vendor. Any DSCP 1 marked traffic from a participant/tail circuit vendor will need to be remarked to DSCP 0 to avoid conflict with the commodity policer. Pros: easier to config and maintain, participants responsible for correctly marking traffic Cons: potential for abuse o Inbound Option 2: explicitly classify and mark (example: a policy statement that identifies SIP traffic and marks it as CS3) Pros: clearly identifies what traffic will be marked and at what priority Cons: more complex policy statements, FRGP engineers responsible for correctly marking traffic

23 Recommend Inbound Option 1 Note that for either option, DSCP values might have to be remarked on a per interface basis depending on how a participant s/tail circuit provider s queue strategy aligns with the FRGP s. Participants should be responsible for remarking where possible. o Outbound: Any packet marked DSCP 1 by the commodity policer will need to be remarked DSCP 0. If not there is a possibility that a participant/tail circuit vendor would deferentially treat this traffic as it is typically assigned to the Scavenger Class.. Policing: o Inbound Option 1: No policing of COS/DSCP marked traffic Pro: easier to admin Con: Potential for abuse i.e. mark all traffic as priority o Inbound Option 2: Place strict limits on COS/DSCP marked traffic Pro: Prevents potential abuse Con: more complicated policers Recommend Inbound Option 2 o Outbound: No change Transmit Queuing: o Option 1: Come up with FRGP QoS templates for services and how they will be mapped. Similar to QMOE QoS templates. Pro: limited set of configurations to maintain, easier to admin Con: less flexible o Option 2: Configure queuing on a per participant/tail circuit vendor basis. Pro: Very flexible Con: More configs to maintain, but not that many members o Its possible that neither options is viable. The QoS configuration on some cards is applied per ASIC (8 ports per ASIC on a 6148a) and not per port. More investigation is needed. Open questions: Charge members $$ for priority traffic??

24 9. References 1) Enterprise QoS SRND 2) Medianet Campus Qos Design ) Cisco Unified Communications SRND Based on Cisco Unified Communications Manager 6.x 4) Comparison of the Cisco Catalyst and Cisco IOS Operating Systems for the Cisco Catalyst 6500 Series Switch html 5) Implementing Quality of Service Policies with DSCP 6) Quality of Service - The Differentiated Services Model 1b36d.html 7) Understanding How Routing Updates and Layer 2 Control Packets Are Queued on an Interface with a QoS Service Policy 8) Understanding Quality of Service on Catalyst 6000 Family Switches igth 9) Cisco AutoQoS Data Sheet 10) Monitoring Voice over IP Quality of Service 11) Cisco TelePresence Network Systems 2.0 Design Guide