QoS in an AVVID-Enabled Wide-Area Network

Transcription

1 CHAPTER 4 QoS in an AVVID-Enabled Wide-Area Network This chapter provides information about implementing QoS in an AVVID-enabled wide-area network (WAN). It includes the following: Overview QoS Recommendations for WAN Aggregation Routers QoS Recommendations for Remote Branch Routers Verifying QoS Note This chapter contains references to other documents. These references are included as tips in the text. The URL for each referenced document is located in Appendix A, Reference Information. In some cases, an internal document is referenced. For copies of internal documents, please see your Cisco Systems representative. Overview A fundamental principle of economics states that the more scarce a resource, the more efficiently it should be managed. In an enterprise network infrastructure, bandwidth is the prime resource and it is scarcest over the WAN. Therefore, the case for efficient bandwidth optimization via QoS technologies is strongest over the WAN, especially for enterprises that are converging their voice, video, and data networks. This chapter provides design guidance for enabling QoS over the WAN. It is important to note that the recommendations put forward in this chapter are not autonomous. They are critically dependent on the recommendations discussed in Chapter 3, QoS in an AVVID-Enabled Campus Network. This chapter focuses strictly on the WAN components of the Cisco AVVID Network Infrastructure (as shown in Figure 4-1), specifically the: WAN aggregation routers Remote-branch routers WAN media For general information about using QoS for voice and video, see Chapter 1, Overview. 4-1

2 QoS Toolset Chapter 4 Figure 4-1 WAN Infrastructure IP IP IP IP IP IP phones Access Si Si Si Si Distribution Core WAN aggregator WAN IP IP IP Branch router Access IP phones QoS Toolset The challenges of packet loss, delay, and delay variation can be addressed through the application of various QoS tools. This section provides information about the use of QoS tools in WAN environments. For general information about the QoS Toolset, see the What is the Quality of Service Toolset? section on page Classification In a WAN environment, classification is generally DSCP-based. NBAR can also be used for classification within the WAN. For information on these two tools, see the Classification Tools section on page Provisioning Policers and Shapers With respect to provisioning tools in a WAN environment, special consideration should be given to the following: Policers and Shapers Link-Fragmentation and Interleaving TX Ring The principle drawback in strict traffic policing is that TCP will retransmit dropped packets and will throttle flows up and down, until the all the data is sent (or the connection times-out). Such TCP ramping behavior results in inefficient use of bandwidth (both over-utilizing and under-utilizing the WAN links). Because shaping usually delays packets rather than dropping them, it smooths flows and allows for more efficient use of WAN bandwidth. Therefore, shaping is more suitable in the WAN than policing. This is especially in the case with NBMA WAN media where physical access speed can vary between two endpoints, such as Frame Relay and ATM (as shown in Figure 4-2). 4-2

3 Chapter 4 QoS Toolset Figure 4-2 Varying Access Speeds in NBMA Networks Cause Delay and Drops Result: Buffering that will cause delay and, eventually, dropped packets 128 kbps 256 kbps Remote Sites 512 kbps 768 kbps Frame Relay, ATM T1 T1 Central Site Link-Fragmentation and Interleaving For slow-speed (768 kbps or below) WAN connections it is necessary to provide a mechanism for LFI. A data frame can only be sent to the physical wire at the serialization rate of the interface. This serialization rate is the size of the frame divided by the clocking speed of the interface. For example, a 1500 byte frame takes 214 ms to serialize on a 56 kbps circuit. If a delay-sensitive voice packet is behind a large data packet in the egress interface queue, the end-to-end delay budget of msec could be exceeded. Additionally, even relatively small frames can adversely affect overall voice quality by simply increasing the delay variation to a value greater than the size of the adaptive jitter buffer at the receiver. In WANs, two tools are available for LFI: MLP LFI (for point-to-point links), and FRF.12 (for Frame Relay links). Tip For more information on FRF.12, see Configuring FRF.12 Fragmentation on Switched PVCs. TX Ring On all PPP and MLP interfaces, the TX ring buffer size is automatically configured. These default buffer values can not be changed. On Frame Relay links, the TX ring is for the main interface, which all sub-interfaces use. The default value is 64 packets. This may need to be changed if the sub-interface is very small or there are many sub-interfaces. Otherwise, TX rings need to be adjusted on low-bandwidth ATM PVCs, where they should be set to a value of 3. Table 4-1 shows the default TX ring values for WAN Interfaces. Table 4-1 Default Tx Ring Values Media Default TX-Ring Buffer Sizing (packets) PPP 6 MLP 2 ATM 8192 (Must be changed for low speed VCs.) Frame Relay 64 (per main T1 interface) 4-3

4 QoS Recommendations for WAN Aggregation Routers Chapter 4 Modular QoS Command-Line Interface The Modular QoS CLI (MQC) was developed with the objective of making QoS configuration more consistent across platforms. MQC is a syntax structure for QoS policies, consisting of three parts: class-map statement classifies traffic policy-map statement defines the treatment for the various classes of traffic service-policy statement binds the policy to an interface and specifies direction MQC offers simplicity and consistency in policy definition as its main advantages. MQC also supports the ability to define policies within policies (hierarchical policies). Hierarchical policies are essential for deploying AVVID policies on distributed Frame Relay WAN aggregators. Also, MQC has associated with it an SNMPv2 MIB (CISCO-CLASS-BASED-QOS-MIB), which provides detailed, granular visibility for QoS monitoring and management. Tip For more information, see Modular Quality of Service Command Line Interface in the Cisco IOS Quality of Service Solutions Configuration Guide, Release For hierarchical policies, see the Traffic Policy as a QoS Policy (Hierarchical Traffic Policies) Example. And for the MIB, see the Cisco Class-Based QoS Configuration and Statistics MIB. QoS Recommendations for WAN Aggregation Routers This section contains information about using QoS in WAN aggregation routers. It covers the following: Classifying and Provisioning for Voice on the WAN Edge Classifying and Provisioning for Video on the WAN Edge Classifying and Provisioning for Data on the WAN Edge Link-Specific WAN QoS Recommendations Summary Configurations Classifying and Provisioning for Voice on the WAN Edge On the WAN edges (both at the WAN aggregator and the remote-branch), voice traffic needs to be assigned to an LLQ and voice control traffic needs a minimum bandwidth guarantee (CBWFQ). The bandwidth required for LLQ traffic can be expressed quantitatively (in absolute kbps) or it can be expressed relatively (as of IOS T) using the percent keyword. This keyword can enable increased modularity of configuration and simplification of management. Expressing the bandwidth reservation in absolute kbps or as a percentage is at the administrator's discretion. Note If the percent keyword is used for one LLQ provisioning, then it must be used for all other LLQ provisioning (as in the case of voice and video). Likewise, if an LLQ is provisioned in terms of absolute kbps, then any additional LLQs must also be provisioned in absolute kbps. 4-4

5 Chapter 4 QoS Recommendations for WAN Aggregation Routers Tip For more information on the percent keyword, see Low Latency Queueing with Priority Percentage Support. In the following configuration for voice only over a T1 link: Voice is assigned to an LLQ. Voice control traffic is guaranteed. All non-voice traffic is assigned to a default queue to receive weighted-fair queueing. class-map match-all VOICE match ip dscp ef class-map match-all VOICE-CONTROL match ip dscp af31 policy-map WAN-EDGE class VOICE priority percent 33 class VOICE-CONTROL bandwidth percent 2 class class-default fair-queue Or: class-map match-all VOICE match ip dscp ef class-map match-all VOICE-CONTROL match ip dscp af31 policy-map WAN-EDGE class VOICE priority 506 class VOICE-CONTROL bandwidth percent 2 or bandwidth 30 class class-default fair-queue DSCP keywords and decimal values are completely synonymous (DSCP EF is the exactly the same as DSCP 46). Both refer to a value of in the first 6 bits of the ToS byte. Note At the time of writing, IOS 12.2(8)T converts DSCP keywords to their decimal equivalents in the running configuration. Note Remember that this policy does not take effect until it is bound to an interface with a service-policy statement. Service-policy statements are discussed in the Link-Specific WAN QoS Recommendations section. For more information about LLQ and CBWFQ, see Scheduling Tools section on page

6 QoS Recommendations for WAN Aggregation Routers Chapter 4 Classifying and Provisioning for Video on the WAN Edge On the WAN edges (both at the WAN-aggregator and the remote-branch), video conferencing traffic should to be assigned to an LLQ. The video stream minimum bandwidth guarantee should be the size of the stream plus an additional 20%. Also, the LLQ burst parameter should be set to bytes per 384 kbps stream. Tip Additional details on bandwidth provisioning for video conferencing are explained in the IP Videoconferencing Solution Reference Network Design Guide. In the following configuration for video only over a T1 link: Video conferencing traffic is assigned to an LLQ. All non-video traffic is assigned to a default queue for weighted-fair queueing. class-map match-all VIDEO match ip dscp af41 policy-map WAN-EDGE class VIDEO priority class class-default fair-queue Note Remember that this policy does not take effect until it is bound to an interface with a service-policy statement. Service-policy statements are discussed in the Link-Specific WAN QoS Recommendations section. Classifying and Provisioning for Data on the WAN Edge Most enterprises have many applications that can be considered mission-critical (Gold). However, if too many applications are classified as mission-critical, they will contend amongst themselves for bandwidth and the result will be a dampening QoS effectiveness. A regular FIFO link (with no QoS) is scheduled in the exact same manner as a link where every application is provisioned as mission-critical. Therefore, it is recommended that you classify a maximum of three applications as mission-critical (Gold). Mission-critical applications should be marked with different AF drop-preference values to distinguish them from each other. These distinctions will provide more granular visibility in managing and monitoring application traffic and aid in provisioning for future requirements. Similar arguments can be made for having no more than three applications in a guaranteed-bandwidth (Silver) class of applications. You should also mark these applications with different AF drop-preference values. Default traffic is automatically marked as best effort (DSCP 0). However, non-critical bandwidth-intensive traffic could be marked differently, so that adverse policies could be applied to control such traffic. These types of traffic can be described as less-than best-effort, or scavenger traffic. For information on the recommended DSCP traffic classifications for data, see Table 1-3 on page

7 Chapter 4 QoS Recommendations for WAN Aggregation Routers It is imperative that DSCP classification be performed on all packets prior to their arriving at the WAN edges. This allows queueing and congestion-avoidance to be performed at the WAN edge based strictly on DSCP markings, which reduces WAN aggregator CPU overhead. Note The default class-map match setting is match-all. Therefore, when you attempt to classify mutually-exclusive traffic flows (such as differing DSCP values), it is important to explicitly use the match-any qualifier when defining the class-map. Example 4-1 shows how four classes of traffic (gold, silver, bronze, and less-than-best-effort) can be classified into three queues. Example 4-1 Four Classes in Three Queues ip cef For distributed platforms, use ip cef distributed. class-map match-any GOLD-DATA match ip dscp af21 match ip dscp af22 match ip dscp af23 class-map match-any SILVER-DATA match ip dscp af11 match ip dscp af12 match ip dscp af13 policy-map WAN-EDGE class GOLD-DATA bandwidth percent 25 random-detect dscp-based class SILVER-DATA bandwidth percent 15 random-detect dscp-based class class-default fair-queue random-detect dscp-based random-detect dscp The queue depth and thresholds are increased for the default random-detect dscp queue. This allows more traffic and increases the portability (to random-detect dscp Frame Relay Traffic-Shaping [FRTS] interfaces and VIP platforms) random-detect dscp of the configuration. The best effort traffic and the less-than-best-effort traffic (DSCP 2, 4, 6) share the default queue, but the random-detect thresholds have been adjusted such that less-than-best-effort traffic is dropped significantly sooner than regular best-effort traffic. Example 4-2 shows how four classes of traffic (gold, silver, bronze, and less-than-best-effort) can be classified into four queues. 4-7

8 QoS Recommendations for WAN Aggregation Routers Chapter 4 Example 4-2 Four-Classes in Four Queues ip cef class-map match-any GOLD-DATA match ip dscp af21 match ip dscp af22 match ip dscp af23 class-map match-any SILVER-DATA match ip dscp af11 match ip dscp af12 match ip dscp af13 class-map match-any <BE-DATA match ip dscp 2 match ip dscp 4 match ip dscp 6 policy-map WAN-EDGE class GOLD-DATA bandwidth percent 20 random-detect dscp-based class SILVER-DATA bandwidth percent 15 random-detect dscp-based class <BE-DATA bandwidth percent 5 random-detect dscp-based random-detect dscp random-detect dscp random-detect dscp class class-default fair-queue For distributed platforms, use ip cef distributed and use: class <BE-DATA bandwidth percent 5 random-detect dscp-based random-detect dscp random-detect dscp random-detect dscp Although it may appear contradictory that less-than-best-effort traffic is assigned a minimum bandwidth guarantee, the logic is based on the fact that CBWFQ will drop packets from a given class when the link is congested and the particular queue for the traffic class is full. Therefore, during times of congestion, all applications designated less-than-best-effort will receive a maximum of 5% of the pipe and then be dropped. Also, the random-detect thresholds have been adjusted to drop less-than-best-effort traffic in order of importance (6 before 4 before 2, which corresponds to the AFxy drop-preference model). This design is more efficient than strict policing, as these bandwidth-intensive, non-critical applications have the ability to make use of additional bandwidth if it is available. Note Remember that this policy does not take effect until it is bound to an interface with a service-policy statement. Service-policy statements are discussed in the Link-Specific WAN QoS Recommendations section. 4-8

9 Chapter 4 QoS Recommendations for WAN Aggregation Routers For more information about classification and provisioning tools, see Classification Tools section on page 1-12 and Provisioning Tools section on page Link-Specific WAN QoS Recommendations This section contains configuration recommendations for WAN QoS on a variety of high-speed (over 768 kbps) and slow-speed (768 kbps and less) links. It includes the recommendations for the following: High-Speed Point-to-Point Links Slow-Speed Point-to-Point Links High-Speed Frame Relay Links Distributed-Platform High-Speed Frame Relay Links Slow-Speed Frame Relay Links Distributed-Platform Slow-Speed Frame Relay Links High-Speed ATM Links Slow-Speed ATM Links ATM-to-Frame Relay Recommendations ISDN Recommendations High-Speed Point-to-Point Links High-speed point-to-point links can be configured as MLP links, PPP links, or HDLC links. An advantage of MLP is that future expansion is easier to manage as there are fewer configuration changes required if an additional link to a remote site is added. Note crtp can be used on high-speed point-to-point links, but do so with caution. CPU performance should first be evaluated when enabling crtp on high-speed links. For more information on crtp, see Provisioning Tools section on page 1-22 and When to Enable crtp section on page A-5. Example 4-3 illustrates how MQC policies can be applied to a high-speed MLP link. Example 4-3 High-Speed MLP Link interface Multilink40 description MLP Link to BRANCH#40 bandwidth 1536 ip address service-policy output WAN-EDGE This command applies the MQC policy to the interface. ppp multilink multilink-group

10 QoS Recommendations for WAN Aggregation Routers Chapter 4 interface Serial0/0 description MLP GROUP 40 Member bandwidth 1536 no ip address encapsulation ppp ppp multilink multilink-group 40 The following commands can be used to verify that a voice, video and data QoS policies have been applied correctly to a MLP interface. show policy show policy interface show interface show queue For more information, see the Verifying QoS section on page Example 4-4 illustrates how MQC policies can be applied to a high-speed PPP link. Example 4-4 High-Speed PPP Link interface Serial0/1 description PPP Link to BRANCH#40 bandwidth 1536 ip address encapsulation ppp service-policy output WAN-EDGE This command applies the MQC policy to the interface. Example 4-5 illustrates how MQC policies can be applied to a high-speed HDLC link. Example 4-5 High-Speed HDLC Link interface Serial0/1 description HDLC Link to BRANCH#40 ip address service-policy output WAN-EDGE This command applies the MQC policy to the interface. Slow-Speed Point-to-Point Links For slow-speed point-to-point WAN links, link-fragmentation and interleaving is required to minimize serialization delay. MLP must be used on PPP links below 768 kbps, as MLP LFI is the only mechanism for reducing serialization delay on point-to-point links. Enabling LFI on MLP links requires two additional commands, as shown in Example 4-6. Note Optionally, crtp can be implemented on slow point-to-point links. crtp is best utilized on slow-speed links after the CPU performance impact has been evaluated. For more information on crtp, see Provisioning Tools section on page 1-22 and When to Enable crtp section on page A-5. Provided the bandwidth savings and overall network design justify implementing crtp, the following commands can be used. 4-10

11 Chapter 4 QoS Recommendations for WAN Aggregation Routers Example 4-6 Slow-Speed MLP Links interface Multilink40 description MLP Link to BRANCH#40 bandwidth 768 ip address ip tcp header-compression iphc-format service-policy output WAN-EDGE ppp multilink ppp multilink fragment-delay 10 ppp multilink interleave multilink-group 40 ip rtp header-compression iphc-format interface Serial0/0 description MEMBER MLP GROUP 40 bandwidth 768 no ip address encapsulation ppp no fair-queue ppp multilink multilink-group 40 This command is automatically added when crtp is enabled. This command applies the MQC policy to the interface. This command sets the maximum delay to 10 milliseconds. This command enables LFI for MLP. This command enables crtp. High-Speed Frame Relay Links Frame Relay networks are the most popular WANs in use today because of the low costs associated with them. Frame Relay is an NBMA technology that uses oversubscription to achieve costs savings. To manage oversubscription, a traffic shaping mechanism must be used. FRTS is the shaping mechanism for Frame Relay WAN media. FRTS requires the following parameters to be defined: Committed Information Rate Committed Burst Rate Excess Burst Rate Minimum CIR Tip For more information, see Configuring Frame Relay Traffic Shaping. Committed Information Rate Recommendation: CIR set to PVC speed. In most Frame Relay networks, a central site uses a T1 link, or something faster, to terminate WAN connections from many remote offices. The central site sends data out at Mbps, while a remote site may have only a 56 kbps circuit. In addition, there is typically a many-to-one ratio of remote offices to central hubs. It is possible for all the remote sites to send traffic at a rate that can overwhelm the T1 at the hub. Both of these scenarios can cause frame buffering in the provider network that introduces loss, delay, and delay variation. The only solution is to use traffic shaping at both the central and remote routers and to define a consistent Committed Information Rate (CIR) at both ends of the Frame Relay DLCI. For high-speed (greater than 768 kbps) Frame Relay links, the CIR should be set to the Permanent Virtual Circuit (PVC) speed. 4-11

12 QoS Recommendations for WAN Aggregation Routers Chapter 4 Committed Burst Rate Recommendation: Bc set to CIR/100. With Frame Relay networks, you also need to consider the amount of data a node can transmit at any given time. A 56 kbps PVC can transmit a maximum of 56 kbps of traffic in 1 second. How this second is divided is called the interval (Tc). The amount of traffic a node can transmit during this interval is called the Committed Burst (Bc) rate. By default, Cisco IOS sets the Bc to CIR/8. The formula for calculating the Tc is: Tc = Bc / CIR For example, with a CIR of 56 kbps: Tc = 7000 / or 125 ms This example is illustrated in Figure 4-3. Figure 4-3 FRTS Delay with Default Bc (CIR/8) IMPORTANT By default, Bc is set to CIR/8 0 Tc = Bc CIR = 125ms ms When 7000 bits (Bc) of transmitted credits are exhausted then no more packets can be sent in that interval. The transmitting router must wait until the next interval to send another burst of 7000 bits. For a T1 line rate, this cycle will translate to 4.5 ms of transmission, followed by ms of silence. This cycle is repeated eight times in one second, by default. VoIP latency/jitter requirements cannot tolerate the interpacket delay introduced by the FRTS default Bc. In this example, after a router sends its allocated 7000 bits, it must wait ms before sending the next batch of traffic.while this is a good default value for data, it is a very bad choice for voice. By setting the Bc value to a much lower number, you can decrease the interval, which means the router will send traffic more frequently. The optimal configured value for Bc is CIR/100, which results in a 10 ms interval (Tc=Bc/CIR) Excess Burst Rate Recommendation: Be set to 0. If the router does not have enough traffic to send all of its Bc (1000 bits, for example), it can credit its account and send more traffic during a later interval. The maximum amount that can be credited to the router's traffic account is called the excess burst (Be) rate. The problem with the Be in Cisco AVVID networks is that this can create a potential for buffering delays within a Frame Relay network (because the receiving side can pull the traffic from a circuit only at the rate of Bc, not Bc + Be). Therefore, to remove this potential for buffering delays, it is recommended that you set Be to

13 Chapter 4 QoS Recommendations for WAN Aggregation Routers Minimum CIR High-Speed FRTS Parameter Summary Recommendation: mincir set to CIR. The minimum CIR is the transmit value a Frame Relay router will rate down to when Backward Explicit Congestion Notifications (BECNs) are received. By default, Cisco IOS sets the minimum CIR to CIR/2. However, to maintain consistent service-levels, it is recommended that you disable adaptive shaping and set the minimum CIR equal to the CIR (which means there is no rating down ). In summary: CIR is set to the PVC speed Bc is set to CIR/100 Be is set to 0 Minimum CIR is equal to the CIR No LFI mechanism is required on WAN links above 768 kbps. Note crtp can be used on high-speed Frame Relay links, but do so with caution on WAN aggregators that serve a large number of remote sites. CPU performance should first be evaluated when enabling crtp on high-speed links. For more information on crtp, see Provisioning Tools section on page 1-22 and When to Enable crtp section on page A-5. Example 4-7 High-Speed Frame Relay Link interface Serial0/1 description Parent FR Link no ip address encapsulation frame-relay frame-relay traffic-shaping This command enables FRTS on the main interface. interface Serial0/1.50 point-to-point description FR Link to BRANCH#50 bandwidth 1536 ip address frame-relay interface-dlci 150 class FRTS-1536kbps This command applies the FRTS map-class to the DLCI. map-class frame-relay FRTS-1536kbps frame-relay cir frame-relay bc frame-relay be 0 frame-relay mincir no frame-relay adaptive-shaping service-policy output WAN-EDGE This command applies the MQC policy to the FRTS map-class. 4-13

14 QoS Recommendations for WAN Aggregation Routers Chapter 4 Distributed-Platform High-Speed Frame Relay Links Most policies, when ported to distributed platforms (such as the Cisco 7500 VIP), require little more than ensuring that CEF is running in distributed mode. However, because FRTS is not supported in a distributed environment, another shaping tool is required. Distributed Traffic Shaping (dts) can be used in conjunction with hierarchical MQC policies to achieve a similar effect on traffic flows over distributed Frame Relay WAN links. However, the Cisco 7500 VIP requires the interval (Tc) to be defined in an increment of 4 ms. Because the target interval for all platforms is 10 ms, which is not evenly divisible by 4 ms, the recommendation for Cisco 7500 VIP is to use an interval of 8 ms. The interval can be set to 8 ms by defining the burst using the following formula: Bc = CIR/125 For example, for a T1 (1.536 Mbps), the Bc for an 8 ms interval is bits. As before, the Be is set to 0. Additionally, it is recommended that you increase the queue-limit from the default of 126 packets to reduce the potential for tail-drops from the shaping queue. To provision the shaping queue to allow for shaping for up to a 10 second period, use the following formula: Queue-limit = (CIR * 10) / (MTU * 8) For example, the queue-limit for a T1 circuit (with an MTU of 1500) would be calculated as: Queue-limit = ( *10)/(1500*8) or 1280 packets Tip For more information, see Configuring Distributed Traffic Shaping. Example 4-8 illustrates using dts and hierarchical MQC policies for a distributed-platform Frame Relay WAN aggregator. Example 4-8 Distributed-Platform High-Speed Frame Relay Link ip cef distributed policy-map dts-1536kbps class class-default shape average This command is the dts shaper (CIR at T1 rate, Bc=CIR/125). queue-limit 1280 This command increases the queue limit. service-policy WAN-EDGE This command applies the LLQ/CBWFQ policies within the dts policy. interface Serial1/0/1 bandwidth 1536 no ip address encapsulation frame-relay no fair-queue interface Serial1/0/1.50 point-to-point description FR Link to BRANCH#50 bandwidth 1536 ip address frame-relay interface-dlci 150 class dts-to-frts-1536kbps This command binds the Frame Relay map-class to the DLCI. 4-14

15 Chapter 4 QoS Recommendations for WAN Aggregation Routers Slow-Speed Frame Relay Links Fragment Sizes map-class frame-relay dts-to-frts-1536kbps no frame-relay adaptive-shaping service-policy output dts-1536kbps This command applies the dts policy to the FR map-class. The symbolic name of DTS-1536kbps represents distributed traffic shaping (DTS) and the link speed. The following commands can be used to verify that a voice, video and data QoS policies have been applied correctly to a Frame Relay link. show policy interface For more information, see the Verifying QoS section on page Similar to high-speed Frame Relay links, slow-speed Frame Relay links require the following parameters to be defined: Fragment Sizes Committed Information Rate and Burst Committed Information Rate and Burst As with all slow links, slow Frame Relay links require a mechanism for fragmentation and interleaving. In the Frame Relay environment, the tool for accomplishing this is FRF.12. Unlike MLP LFI, which takes the maximum serialization delay as a parameter, FRF.12 requires the actual fragment sizes to be defined. This requires some additional calculations, as the maximum fragment sizes vary by link speed. These fragment sizes can be calculated by dividing the recommended 10 ms of delay by 1 byte of traffic at the provisioned line-clocking speed: Fragment Size = (Maximum Allowed Jitter * Link Speed in kbps) / 8 For example, the calculation for the maximum fragment size for a 56 kbps circuit is: Fragment Size = (10 ms * 56) / 8 or 70 Bytes Fragment sizes that correspond to the recommended minimum serialization delay of 10 ms per link are shown in second column of Table 4-2. With FRTS, it is important to note that Frame Relay header flags and the CRC (which are added by the interface driver) are not taken into account in the shaping algorithm. When the flags and CRC are included in the CIR calculation, the value is determined by the following formula: CIR = (Link-Speed * Maximum-Frame-Size) / (Maximum-Frame-Size + 4) For high-speed links, this value works out to around 99%. For example, a on a T1 link: CIR = * 1500 / ( ) CIR = * CIR = which is 99.7% of the link speed. While this calculated value could be used, the difference is so negligible that for simplicity's sake the CIR could just as well be set to the link-speed (as recommended in the High-Speed Frame Relay Links section on page 4-11). 4-15

16 QoS Recommendations for WAN Aggregation Routers Chapter 4 Because a flag and CRC need to be added for every fragment, enabling fragmentation directly affects the CIR value. Therefore, the recommended CIR values as calculated by the formula above (and rounded down to the nearest byte-boundary) are shown in the third column of Table 4-2 and the Bc values (CIR/100, rounded) are shown in the fourth column. As before, Be is set to 0 and the minimum CIR is set to the CIR. Table 4-2 Recommended Fragment Sizes, CIRs and Bursts for Slow-Speed FR Links PVC Speed Maximum Fragment Size (for 10 ms delay) Recommended CIR Values Recommended Bc Values 56 kbps 70 Bytes bps 530 bits per Tc 64 kbps 80 Bytes bps 610 bits per Tc 128 kbps 160 Bytes bps 1250 bits per Tc 256 kbps 320 Bytes bps 2530 bits per Tc 512 kbps 640 Bytes bps 5090 bits per Tc 768 kbps 960 Bytes bps 7560 bits per Tc Note crtp can also be implemented on slow Frame Relay links. crtp is best utilized on slow-speed links after the CPU performance impact has been evaluated. For more information on crtp, see Provisioning Tools section on page 1-22 and When to Enable crtp section on page A-5. Example 4-9 illustrates a MQC policy applied to a slow-speed (512 kbps) Frame Relay link. Example 4-9 Slow-Speed Frame Relay Link interface Serial0/1 description Parent FR Link no ip address encapsulation frame-relay frame-relay traffic-shaping This command enables FRTS on the main interface. interface Serial0/1.50 point-to-point description FR Link to BRANCH#50 bandwidth 512 ip address frame-relay interface-dlci 150 class FRTS-512kbps This command applies the FRTS map-class to the DLCI. frame-relay ip rtp header-compression This command enables crtp. map-class frame-relay FRTS-512kbps frame-relay cir frame-relay bc 5090 frame-relay be 0 frame-relay mincir no frame-relay adaptive-shaping service-policy output WAN-EDGE This command applies the MQC policy to the FRTS map-class. frame-relay fragment 640 This command enables FRF

17 Chapter 4 QoS Recommendations for WAN Aggregation Routers The following commands can be used to verify that a voice, video and data QoS policies have been applied correctly to a Frame Relay link. show policy show frame-relay pvc For more information, see the Verifying QoS section on page Distributed-Platform Slow-Speed Frame Relay Links Slow-speed Frame Relay policies on a distributed platform require the following: CEF must be enabled in distributed mode dts policy (with nested queueing policies) Bc set to CIR/125 Be set to 0 Queue-limit can be increased to (CIR*10) / (MTU*8) FRF.12 must be enabled CIR values need to reflect fragmentation headers and CRCs CIR must be a multiple of 8000 Note crtp can be enabled. For more information on crtp, see Provisioning Tools section on page 1-22 and When to Enable crtp section on page A-5. One constraint of class-based shaping (which includes dts) is that the CIR must be a multiple of Therefore, the calculated FRTS CIR values from Table 4-2 must be rounded down to the nearest multiple of 8000 for dts. Because the Cisco 7500 requires that an interval (Tc) be defined in 4 ms increments, the FRTS optimal recommendation of Bc=CIR/100 (which sets the interval to equal 10 ms) cannot be used. The nearest (rounded-down) 4 ms interval is 8 ms. To set an 8 ms interval, the Bc value must be defined as CIR/125. Be is held at 0. Table 4-3 shows the recommended fragmentation, CIR, Bc and queue-limit values for distributed-platform, slow-speed Frame Relay links. Table 4-3 Distributed-Platform Fragmentation and (dts) Shaping Values PVC Speed Fragment Sizes CIR Bc dts Queue-Limits 56 kbps 70 Bytes bps 384 bits per Tc Default (126 packets) 64 kbps 80 Bytes bps 448 bits per Tc Default (126 packets) 128 kbps 160 Bytes bps 960 bits per Tc Default (126 packets) 256 kbps 320 Bytes bps 1984 bits per Tc 213 packets 512 kbps 640 Bytes bps 4032 bits per Tc 427 packets 768 kbps 960 Bytes bps 6080 bits per Tc 640 packets 4-17

18 QoS Recommendations for WAN Aggregation Routers Chapter 4 Example 4-10 illustrates a MQC policy applied to a slow-speed (512 kbps) Frame Relay link connecting to a distributed-platform. Example 4-10 Distributed-Platform Slow-Speed Frame Relay Link ip cef distributed policy-map dts-512kbps class class-default shape average This command sets the dts shaper. queue-limit 427 This command increases the queue limit. service-policy WAN-EDGE This command applies the LLQ/CBWFQ policies within the dts policy. interface Serial1/0/1 bandwidth 1536 no ip address encapsulation frame-relay no fair-queue interface Serial1/0/1.50 point-to-point description FR Link to BRANCH#50 bandwidth 512 ip address frame-relay interface-dlci 150 class dts-to-frts-512kbps This command binds the FR map-class to the DLCI. frame-relay ip rtp header-compression This command enables crtp. map-class frame-relay dts-to-frts-512kbps no frame-relay adaptive-shaping service-policy output dts-512kbps This command applies the dts policy to the FR map-class. frame-relay fragment 640 This command enables FR.12. High-Speed ATM Links Class-based policies are only recommended on ATM hardware that supports per-vc traffic-shaping (for example, ATM Enhanced Port Adaptors [PA-A3] for the Cisco 7200/7500 routers and OC3 Network Modules for Cisco 3600 routers). Example 4-11 illustrates how MQC policies can be applied directly to ATM PVCs. Example 4-11 High-Speed ATM Links ip cef In distributed platforms, use ip cef distributed. interface ATM4/0 description Parent ATM Link bandwidth 3000 no ip address no atm ilmi-keepalive 4-18

19 Chapter 4 QoS Recommendations for WAN Aggregation Routers interface ATM4/0.60 point-to-point description ATM Link to BRANCH#60 bandwidth 3000 ip address pvc BRANCH#60 0/60 vbr-nrt This command sets the high ATM AAL5 traffic contract. tx-ring-limit 3 This command reduces delay caused by the TX ring buffer. service-policy output WAN-EDGE This command applies the MQC policy to the ATM PVC. Tip For more information, see Configuring ATM. Slow-Speed ATM Links Prior to IOS 12.1(5)T, there was no mechanism for fragmenting-and-interleaving over low-speed ATM links. At that time, a new standard of applying MLP LFI over ATM provided the required minimization of serialization delay for slow ATM links. Additionally, MLP over ATM requires the MLP bundle to classify the outgoing packets before they are sent to the ATM VC. Note There are two virtual-access interfaces created for each virtual-template. One is the Layer 2 PPP-only virtual-access. The other is the bundle virtual-access interface. It is the bundle interface that takes on all of the properties of Layer 3 (the IP address, crtp if configured, etc.). CBWFQ can be applied only to the bundle interface. MLP uses the bundle virtual-access interface to reassemble packets received over individual links and to fragment packets sent out over individual links. The bundle interface inherits its configuration from the virtual-template specific to MLP. Example 4-12 illustrates how virtual-templates and MLP LFI can be applied in such a scenario. Example 4-12 Slow-Speed ATM Link Using Virtual-Templates ip cef In distributed platforms, use ip cef distributed. interface ATM4/0 bandwidth 768 no ip address no atm ilmi-keepalive interface ATM4/0.60 point-to-point pvc BRANCH#60 0/60 vbr-nrt This command sets the high ATM AAL5 traffic contract. tx-ring-limit 3 This command reduces the delay caused by the TX ring buffer. protocol ppp Virtual-Template60 This command binds the virtual template to the PVC. interface Virtual-Template60 bandwidth 768 ip address service-policy output WAN-EDGE This command sets the MQC policies for voice and data. ppp multilink ppp multilink fragment-delay 10 This command sets the maximum delay to 10 ms. ppp multilink interleave This command enables MLP LFI. 4-19

20 QoS Recommendations for WAN Aggregation Routers Chapter 4 Note crtp is only supported on ATM PVCs (via PPPoATM) as of IOS 12.2(2)T. For more information, see IP Header Compression Enhancement PPPoATM and PPPoFR Support. When using virtual-templates for low-speed ATM links, keep the following in mind: The dynamic nature of virtual-template interfaces may make network management unwieldy. MLP over ATM can only be supported on hardware that supports per-vc traffic shaping. An alternative option is to use VC-bundling, in which you have 2 (or more) PVCs with different traffic contracts. For example, one PVC (for voice) would have vbr-nrt, while another (for data) would be ubr. Example 4-13 Slow-Speed ATM Link Using VC-Bundling ip cef vc-class atm VOICE-VC-256 vbr-nrt tx-ring-limit 3 precedence 5 no bump traffic protect vc vc-class atm DATA-VC-512 ubr 512 tx-ring-limit 3 precedence other interface ATM3/0 no ip address no atm ilmi-keepalive interface ATM3/0.60 point-to-point ip address bundle BRANCH#60 pvc-bundle BRANCH60-DATA 0/60 class-vc DATA-VC-512 service-policy output WAN-EDGE-DATA pvc-bundle BRANCH60-VOICE 0/600 class-vc VOICE-VC-256 Only data policies are required. One drawback to the multiple PVC design is that data can never get access to the voice (or video) VCs, even if there is available bandwidth in them. This forces sub-optimal consumption of WAN bandwidth. The following commands can be used to verify that a voice, video and data policies have been applied correctly to an ATM PVC-bundle. show policy show atm bundle show policy interface atm show atm vc show atm pvc For more information, see the Verifying QoS section on page

21 Chapter 4 QoS Recommendations for WAN Aggregation Routers ATM-to-Frame Relay Recommendations Many enterprises are deploying AVVID over networks that use Frame Relay at the remote sites and ATM at the central location. The conversion is accomplished through ATM to Frame Relay Service Interworking (FRF.8) in the carrier network. FRF.12 cannot be used because currently no service provider supports FRF.12 termination in the Frame Relay cloud. In fact, there are no Cisco WAN switching devices that support FRF.12. Tunneling FRF.12 through the service provider's network will do no good because there is no FRF.12 standard on the ATM side. This is a problem because fragmentation is a requirement if any of the remote Frame Relay sites use a circuit speed of 768 kbps or below. However, MLP over ATM and Frame Relay provides an end-to-end, Layer 2, fragmentation and interleaving method for low-speed ATM to Frame Relay FRF.8 Service Interworking links. FRF.8 Service Interworking is a Frame Relay Forum standard for connecting Frame Relay networks with ATM network. Service Interworking provides a standards-based solution for service providers, enterprises, and end users. In Service Interworking translation mode, Frame Relay PVCs are mapped to ATM PVCs without the necessity for symmetric topologies; the paths can terminate on the ATM side. FRF.8 supports two modes of operation of the IWF for upper-layer user protocol encapsulation: Translation mode maps between ATM and Frame Relay encapsulation. It also supports interworking of routed or bridged protocols. Transparent mode does not map encapsulations but sends them unaltered. This mode is used when translation is impractical because encapsulation methods do not conform to the supported standards for Service Interworking. MLP for LFI on ATM and Frame Relay Service Interworking networks is supported for transparent mode VCs and translational mode VCs that support PPP translation (FRF 8.1). To make MLPoFR and MLPoATM interworking possible, the Interworking Switch must be configured in transparent mode and the end routers must be able to recognize both MLPoFR and MLPoATM headers. This is enabled with the frame-relay interface-dlci dlci ppp and protocol ppp commands for Frame Relay and ATM respectively. When a frame is sent from the Frame Relay side of an ATM to Frame Relay Service Interworking connection, the following should happen to make interworking possible: 1. A packet is encapsulated in MLPoFR header by the sending router 2. The Carrier Switch, in transparent mode, strips off the 2-byte Frame Relay DLCI field and sends the rest of the packet to its ATM interface 3. The receiving router examines the header of the received packet. If the first two bytes of the received packet are 0x03cf, it treats it as a legal MLPoATM packet and sends it to MLP layer for further processing. When an ATM cell is sent from ATM side of an ATM to Frame Relay Service Interworking connection, the following should happen to make interworking possible: 1. A packet is encapsulated in MLPoATM header by the sending router 2. The Carrier Switch, in transparent mode, prepends 2-byte Frame Relay DLCI field to the received packet and sends the packet to its Frame Relay interface 3. The receiving router examines the header of the received packet. If the first 4 bytes after the 2-byte DLCI field of the received packet is 0xfefe03cf, it treats it as a legal MLPoFR packet and sends it to MLP layer for further processing. Tip For more information, see Configuring Frame Relay-ATM Interworking. 4-21

22 QoS Recommendations for WAN Aggregation Routers Chapter 4 Optimizing Fragment Sizes A new ATM to Frame Relay Service Interworking standard, FRF.8.1, supports MLP over ATM and Frame Relay Service Interworking, but it can be years before all switches are updated to the new standard. When enabling MLP over ATM the fragment size should be optimized such that it fits into an integral number of cells. Otherwise, the bandwidth required could potentially double due to cell padding. For example, if a fragment size of 49 bytes is configured, then this fragment would require 2 cells to transmit (as ATM cells have 48 byte payloads). This would generate 57 bytes of overhead (2 cell headers plus 47 bytes of cell padding), which is more than double the fragment itself. Tip Currently, IOS does not automatically optimize the fragment sizes (which are indirectly defined by the MLP fragment-delay parameter). Arriving at such optimal sizes requires a fair bit of math, which is well documented in the Multilink PPP over Frame Relay and ATM white paper (internal). Table 4-4 provides a summary of the optimal fragment-delay parameters for MLP over ATM. Table 4-4 MLP over ATM Optimal Fragment-Delay Values PVC Speed Optimal Fragment Size ATM Cells (Rounded Up) ppp multilink fragment-delay value 56 kbps 84 bytes 2 12 ms 64 kbps 80 Bytes 2 10 ms 128 kbps 176 Bytes 4 11 ms 256 kbps 320 Bytes 7 10 ms 512 kbps 640 Bytes ms 768 kbps 960 Bytes ms Sample ATM-to-Frame Relay Configuration An ATM-to-Frame Relay configuration is shown below, in two parts: Example 4-14 shows the Central Site WAN Aggregator ATM configuration.the Central Site WAN Aggregator ATM configuration is identical to the configuration required for end-to-end low-speed ATM connections implementing MLP LFI. Example 4-15 shows the Remote-Branch Frame Relay Configuration. Example 4-14 Central Site WAN Aggregator ATM Configuration ip cef For distributed platforms, use ip cef distributed. interface ATM4/0 no ip address no atm ilmi-keepalive 4-22

23 Chapter 4 QoS Recommendations for WAN Aggregation Routers interface ATM4/0.60 point-to-point pvc BRANCH#60 0/60 vbr-nrt This command sets the high ATM AAL5 traffic contract. tx-ring-limit 3 This command reduces the delay caused by the TX ring buffer. protocol ppp Virtual-Template60 This command binds the virtual template to the PVC. interface Virtual-Template60 bandwidth 256 ip address service-policy output WAN-EDGE This command sets the MQC policy for voice and data. ppp multilink ppp multilink fragment-delay 10 This command sets the maximum delay. ppp multilink interleave This command enables MLP LFI. Example 4-15 Remote-Branch Router Frame Relay Configuration ip cef interface Serial6/0 description Parent FR Link for BRANCH#60 no ip address encapsulation frame-relay frame-relay traffic-shaping interface Serial6/0.60 point-to-point description FR Sub-Interface for BRANCH#60 bandwidth 256 frame-relay interface-dlci 60 ppp Virtual-Template60 class FRTS-256kbps interface Virtual-Template60 bandwidth 256 ip address service-policy output WAN-EDGE This command sets the MQC policies for voice and data. ppp multilink ppp multilink fragment-delay 10 This command sets the maximum delay. ppp multilink interleave This command enables MLP LFI. map-class frame-relay FRTS-256kbps frame-relay cir frame-relay bc 2530 frame-relay be 0 frame-relay mincir no frame-relay adaptive-shaping Considerations for MLPoATM and MLPoFR When using MLP over ATM and MLP over Frame Relay, keep the following in mind: MLPoATM can only be supported on platforms that support per-vc traffic shaping (such as ATM Enhanced Port Adaptors for the 7200/7500 and OC3 Network Modules for the 3600). MLPoATM requires the MLP bundle to classify the outgoing packets before they are sent to the ATM VC. It also requires that the per-vc queuing strategy for the ATM VC to be FIFO. MLP over Frame Relay relies on the FRTS engine to control the flow of packets from MLP bundle to FR VC. crtp is only supported over ATM links as of IOS 12.2(2)T 4-23

24 QoS Recommendations for WAN Aggregation Routers Chapter 4 ISDN Recommendations When designing VoIP over ISDN networks, remember the following: Link bandwidth varies as B channels are added or dropped. RTP packets may arrive out of order when transmitted across multiple B channels. CAC limitations with Call Manager locations-based CAC. Variable Bandwidth MLP Packet Reordering Considerations CallManager CAC Limitations ISDN allows B channels to be added or dropped in response to the demand for bandwidth. The fact that the bandwidth of a link varies over time presents a special challenge to the CBWFQ/LLQ queuing mechanisms of IOS. Prior to IOS 12.2(2)T, a policy-map implementing LLQ could only be assigned a fixed amount of bandwidth. On an ISDN interface, IOS assumes that only 64 kbps is available, even though the interface has the potential to provide 128 kbps, Mbps, or Mbps of bandwidth. By default, the maximum bandwidth assigned must be less than or equal to 75% of the available bandwidth. Hence, prior to IOS 12.2(2)T, only 75% of 64 kbps, or 48 kbps, could be allocated to an LLQ on any ISDN interface. If more was allocated, then an error message was generated when the policy-map was applied to the ISDN interface. This severely restricted the number of VoIP calls that could be carried. The solution to this problem was introduced in IOS 12.2(2)T with the introduction of the percent keyword to the priority statement. This keyword allows for the reservation of a variable bandwidth percentage to be assigned to the LLQ. MLP LFI is used for fragmentation and interleaving voice and data over ISDN links. LFI fragments large data packets into smaller fragments and transmits them in parallel across all the B channels in the bundle. At the same time, voice packets are interleaved in between the fragments, thereby reducing their delay. The interleaved packets are not subject to MLP encapsulation. They are encapsulated as regular PPP packets. Hence, they have no MLP sequence numbers and cannot be reordered should they arrive out of sequence. The need to reorder the packet is quite likely. The depth of the various link queues in the bundle may differ, causing RTP packets to overtake each other as a result of the difference in queuing delay. The various B channels may also take different paths through the ISDN network and end up with different transmission delays. This reordering of packets is not generally a problem for RTP packets. The buffers on the receiving VoIP devices will reorder the packets based on the RTP sequence numbers. However, reordering does become a problem if crtp is used. The crtp algorithm assumes that RTP packets are compressed and decompressed in the same order. If they get out of sequence, then decompression will not occur correctly. Therefore, today it is not safe to use crtp if there is more then one B channel in the MLP bundle. The same restriction applies if you are using MLP over ATM or Frame Relay, in which case crtp is not possible if there is more then one VC in the bundle. A solution to the reordering problem is offered by Multiclass Multilink PPP (MCMP). With MCMP, the interleaved packets are given a small header with a sequence number, which allows them to be reordered by the far end of the bundle before crtp decompression takes place. At the time of this writing, MCMP is not yet supported in IOS. IP telephony in branch networks are typically based on the centralized call-processing model and use locations-based CAC to limit the number of calls across the WAN. Locations-based CAC does not currently have any mechanism for tracking topology changes in the network. So if the primary link to a 4-24