Multicast: Conformance and Performance Testing

Transcription

1 White Paper Multicast: Conformance and Performance Testing Agoura Road, Calabasas, CA Tel: Fax:

2 Contents 1. Introduction 1 2. What Is Multicast? Multicast Taxonomy Historical Perspective/Evolution 3 3. How Does Multicast Work? Multicast Groups Multicast Addressing Multicast Scope/Time-To-Live (TTL) Multicast Routing Multicast Distribution Trees Multicast Distribution Trees Dynamic Registration 7 4. Multicast Protocols Multicast Group Management Protocols Multicast Routing Protocols 9 5. Multicast Testing Challenges Why Test for Multicast Conformance? Why Test for Multicast Scalability and Performance? Multicast Testing Challenges Why Test for Multicast Conformance? Why Test for Multicast Scalability and Performance? Test Solution Requirements Optimized Hardware Platform Routing Protocol Emulation Integrated Control and Data Plane Testing Traffic Generation Automation Ixia s Approach to Multicast Testing Multicast Scalability and Performance Testing Conclusion Appendix: Multicast Testing Example PIM-SM Conformance Test IPv4 Multicast Benchmarking Test Pv6 Multicast Listener Discovery (MLD) Performance Test PIM-SM Shortest-Path Tree Switchover Test Glossary of Terms Bibliography Acknowledgments 34 Copyright 2005 by Ixia All rights reserved IXIA West Agoura Road, Calabasas, CA (877) FOR-IXIA This Test Plan Primer contains a general outline for testing a particular technology. Not all the capabilities of Ixia technology have been exposed in this document. Please feel free to contact us if additional capabilities are required.

3 1. Introduction The sustained expansion of Internet communications continues to generate new services and network applications. At the same time, the exponential growth in the number of concurrent users who want to simultaneously access shared data in corporate intranets, as well as the global Internet, has generated the need for applications that provide data access while minimizing bandwidth requirements. The implementation strategy adopted by many of these emerging applications involves sending packets from one or more senders to a group of recipients in a single operation. This technique is referred to as IP Multicast. This paper presents a review of IP multicast, but given the immensity of the field and its continuing evolution, this review is by no means exhaustive. It also includes an overview of multicast test challenges and Ixia s approach to testing multicast technologies to determine the scalability and performance of a multicast system. Finally, this paper reviews the importance of validating the conformance of various multicast technologies to the growing number of Internet Engineering Task Force (IETF) Drafts and RFCs associated with multicast. Multicast testing before, as well as after, deployment of network equipment helps ensure a fully operational multicast system. 2. What is Multicast? IP multicast is defined in RFCs 966 and 988 as the transmission of an IP datagram to a host group a set of hosts identified by a single IP destination address. A multicast datagram is delivered to all members of its destination host group with the same besteffort reliability as regular unicast IP datagrams, i.e., the datagram is not guaranteed to arrive at all members of the destination group or in the same order relative to other datagrams. Multicast was developed as an efficient method of data delivery over an IP packet-switched network that allows a server to send a single data stream to a local multicast-capable router that then redistributes it to other local hosts. In multicast, each packet is sent from one sender to multiple receivers with a single "transmit" operation. 2.1 Multicast Taxonomy A multicast application can be characterized as one of three types: One-to-Many, Many-to-Many, or Many-to-One. One-to-Many applications (see Figure 1) are similar to standard television or radio broadcasts. Examples include: push media content distribution of news, sports, weather, etc.; security monitoring; distribution of stock market prices, manufacturing process information, schedule announcements, keys, and network times; and file distribution and caching. Figure 1: One-to-Many multicast transmission from a single host to all intended recipient hosts. The sender dispatches a multicast packet addressed to the multicast group of receivers. Multicast routes are indicated with the thicker, red arrows. Multicast: Conformance and Performance Testing Copyright Ixia,

4 An important distinction to be noted is that one-to-many Netcasting does NOT imply multicast. There are various applications that implement unicast Netcasting to provide one-to-many broadcast-like services. The one-to-many Netcasting applications distribute information using separate unicast IP streams for each user, providing broadcast-like services such as Real Audio and Real Video, as well as several servers for MP3 file distribution. Many-to-Many applications (see Figure 2) include interactive distance learning, interactive multi-player games, jam sessions, multimedia teleconferencing (voice/video phones and whiteboards), chat groups, shared editing and collaboration tools, parallel computing, as well as distributed interactive simulations. Figure 2: Many-to-Many multicast transmission from two senders to all intended recipient hosts. The senders dispatch multicast packets addressed to the multicast group of receivers. Multicast routes are indicated with the thicker, red arrows. Many-to-One applications (see figure 3) usually require a request and response mechanism. Examples include polling and data collection, resource and service location discovery (Anycast), on-line auctions, as well as various jukebox and real-time multimedia playing applications. Figure 3: Many-to-one multicast transmission from two senders to a single receiver. The senders dispatch multicast packets addressed to the multicast group of receivers. In the illustrated example above, the group consists of a single host. Multicast routes are indicated with the thicker, red arrows. 2 Copyright Ixia, 2005 Multicast: Conformance and Performance Testing

5 2.2 Historical Perspective/Evolution Early efforts in the 1980s to define a multicast-capable Internet resulted in a RFC 966, Host Groups: A Multicast Extension to the Internet Protocol (1985). IP multicast concepts evolved through additional RFCs (988 and 1054), resulting in the multicast standard, in RFC 1112, Host Extensions for IP Multicasting (1989). Additional work in the early 90s led to the creation of Virtual Internet Backbone for Multicast IP or MBone, which was an experimental test bed system for multicast application and protocol development. Mbone was first deployed in 1992 as a virtual network, with application-layer packet replication: one packet in, one or more packets out. From Mbone s flat, virtual set of networks in 1992, all under the same Autonomous System (AS) or domain, multicast routing has evolved from an intra-domain routing emphasis to a broader scope of inter-domain routing in the late 1990 s, supporting a hierarchical set of domains. [1] The earliest multicast routing protocol, Distance Vector Multicast Routing Protocol (DVMRP), was defined in RFC DVMRP is a distance-vector-routing protocol, similar to the RIP unicast routing protocol, but with DVMRP used exclusively for multicast. This type of multicast routing is termed Dense Mode. It operates by flooding multicast packets in all directions, based on the assumption that the network is densely populated with multicast receivers that want to receive these packets. In practice, however, many networks are not densely populated, and for this reason other multicast methods are preferred. Additionally, Dense Mode multicasting is subject to significant scaling problems. Alternatively, Sparse Mode protocols, such as Protocol Independent Multicast Sparse Mode (PIM-SM), are more efficient for transmitting to multicast groups whose members are thinly distributed. PIM-SM receivers must explicitly request membership in the multicast group. Due to its scalability, PIM-SM is currently the most commonly used multicast routing protocol. Figure 4 shows the differences in the data transmission between Dense Mode and Sparse Mode multicast protocols. Figure 4: Differences between Dense Mode and Sparse Mode multicast packet transmission. For Sparse Mode, multicast packets will be sent only to Receivers that have indicated interest. Multicast: Conformance and Performance Testing Copyright Ixia,

6 3. How Does Multicast Work? Multicast routing delivers a single stream of transmitted data packets to a multicast group IP address. However, in contrast with a broadcast operation, it only attempts to distribute the packet stream to the set of hosts that are part of the recipient group not to all hosts. Multicast differs from unicast technologies because it delivers a single stream of information to a potentially large group, with only one copy of the packet stream traveling over any individual link. This can result in significant traffic reduction, maximizing the use of network bandwidth. Another important difference between multicast and unicast is that while unicast applications can use TCP or UDP at the transport layer, the only available transport protocol for multicast applications is UDP. For this reason, the application layer must provide the reliability data transfer mechanisms, such as sequence numbers, timers, and retransmissions, for multicast operations. IP multicast involves both hosts and routers, with a multicast topology consisting of sources and groups of receivers. Usually, the source and destination receiver are hosts, while routers distribute multicast traffic across the network or set of networks that connects them. The multicast router must find multicast sources on the network, send out copies of packets on several interfaces, prevent routing loops, connect interested destinations with the proper source, and keep the flow of unwanted packets to a minimum. A multicast source transmits its data to a specific group of receivers, represented by a multicast group of IP addresses. To receive that data, receivers join the relevant group by explicit request. The source initially sends multicast data to the receiving group in a single stream of datagrams/packets. If a datagram reaches a point (router) on the network where some multicast receivers cannot be reached by continuing to use a single, common path, the router duplicates the datagram, and then streams it out over the fewest paths possible to the next hop routers in the network. Figure 5 shows an example of a group of four multicast receivers that are interested in receiving data, voice, and video information from the source. These hosts that form the receiver multicast group indicate their interest in the source s information by sending a message to the routers in the network. In order to deliver the data from the source to the receivers, the routers dynamically generate a multicast distribution tree. Figure 5: Multicast source, routers, and receiver group, showing the network segments that form the multicast path between them as black arrows Multicast Groups Multicast receivers are required to contact their local multicast router and to join, or request a connection to, a multicast group (or channel for Source-Specific Multicast /SSM). Group membership is dynamic, where hosts may join and leave groups at any time. A host need not be a member of a group to send datagrams to it, and a host can be a member of more than one group at a time. When a group has no members and no sources, the states associated with the group will cease to exist on the router after a prespecified timeout. There are no restrictions on location or number of member hosts per group, but membership in a group may be restricted to only those hosts possessing a private access key. 4 Copyright Ixia, 2005 Multicast: Conformance and Performance Testing

7 3.2. Multicast Addressing The IETF has divided the IPv4 address space into four classes. Classes A, B, and C are used for unicast traffic. Class D addresses are reserved for multicast traffic, and are allocated dynamically. Figure 6: Multicast Addressing Classification The Class D range of IP multicast addresses starts with 1110 for the high-end bits, and leaves 28 bits to identify the destination multicast group. Multicast addresses designate either permanent or transient multicast groups. A permanent multicast group is associated with an IP multicast address that is registered with the Internet Assigned Numbers Authority (IANA). Permanent special multicast groups are the all hosts group with address , which designates all hosts and all routers on a network, and the all routers group with address , which designates all routers on a network. Several protocols reserve permanent multicast groups. For example, the IP address is used by RIP2 protocol to designate all RIP2 routers on a network, and is the group of all routers that use the DVMRP multicast routing protocol. Addresses with the prefix /8 (GLOP addresses per RFC 3180) are reserved for owners of autonomous systems and are used as follows. The AS number of an autonomous system is converted to a 16-bit number, and these 16 bits are assigned to the second and third bytes of the IP address. Then each autonomous system can use the fourth byte to designate a set of 256 reserved multicast addresses. For example, if the AS number of an autonomous system is 0x8080, the autonomous system effectively owns the multicast addresses in the range from to All addresses that do not designate permanent multicast groups designate transient groups. Addresses for these groups are assigned dynamically. An application implicitly allocates a multicast address by starting to transmit to a transient group. The address is released implicitly when there is no sender and no receiver for this group. If two applications use the same multicast address, IP multicast treats the applications as a single multicast group, and packets sent by any of the applications are delivered to both applications. Multicast: Conformance and Performance Testing Copyright Ixia,

8 3.3. Multicast Scope/Time-To-Live (TTL) The Time-To-Live (TTL) field in IP multicast datagrams limits the scope of the IP datagram. The TTL, denoting the allowed number of hops between routers, can be a value from 0 to 255. The default value is 0, which means that all multicast packets are forwarded out the interface. By setting a TTL threshold for a given interface, only multicast packets with a TTL value greater than the threshold are forwarded out the interface. Setting the TTL to different values allows different applications to use the same IP multicast group address without interfering with each other. In this way, problems occur only if there is overlap between the scope of the IP datagrams transmitted by different applications that use the same multicast group. Packets with addresses in the range are sent with the TTL field set to one, and, therefore, the scope of the corresponding groups is limited to the local network. TTL Value Description 0 Restrict to the same host. No output by any interface. 1 Restricted to the same subnet. Not forwarded by a router. 15 Restricted to the same site, organization, or department. 63 Restricted to the same region. 127 Worldwide. 191 Worldwide; Limited Bandwidth. 255 Unrestricted in scope; Global. Table 1: Multicast Time-To-Live (TTL) values and descriptions 3.4. Multicast Routing Multicast routing protocols make possible the exchange of messages between routers, the assembly of distribution trees, and the forwarding of multicast packets. Multicast routing is different from standard IP routing in several aspects. First, the network focuses on routing traffic from the source host or hosts to the receiver group of hosts. Second, while all receiver hosts share the same multicast group IP address, they can be arbitrarily scattered throughout the network. Third, the receivers can join and leave multicast groups arbitrarily, as well. Figure 7: Comparison of Multiple Unicast transmissions and Multicast transmissions. 6 Copyright Ixia, 2005 Multicast: Conformance and Performance Testing

9 With multicast routing, a router or switch sends packets to a specific group of hosts without using broadcasts or multiple unicast transmissions (see Figure 7), and without routing loops or excess transmissions. Multicast destinations can include hosts that reside on a local LAN, hosts that reside on different sites within a private network, or hosts that are scattered throughout the Internet. Networks must be able to build distribution trees that connect sources to all receivers. A primary goal of these packet distribution trees is to ensure that each packet appears exists only one time on any given network link (that is, if there are multiple receivers on a given branch, there should only be one copy of the packets on that branch). The goal of multicast routing is to find a tree of links that connects all of the routers that have attached hosts belonging to the multicast group. Multicast packets will then be routed along this tree from the sender to all of the hosts belonging to the multicast tree. Of course, the tree may contain routers that do not have attached hosts belonging to the multicast group Multicast Distribution Trees There are two distinct methods used for creating the multicast distribution tree (MDT): the group-shared tree approach, and the source-based tree approach. In the group-shared tree approach, a single routing tree is constructed for the entire multicast group and is used to distribute the traffic from all senders in the group. The links that are not connected to those host members of the multicast group are not included in the tree. Note that the links are bi-directional, since packets can flow in either direction on a link. In a source-based tree approach, an individual, source-specific routing tree is constructed for each sender in the multicast group. In a multicast group with N hosts, N different routing trees will be constructed for that single multicast group. Packets will be routed to multicast group members in a source-specific manner, depending on which was the initiating source host. Note that not only are there different links than in the group-shared tree case, but that some links may also be used only in a single direction Dynamic Registration Identification and registration of the receivers is the initial step in multicast operations. In IPv4 networks, this is achieved between multicast-group member hosts and the corresponding local router via the Internet Group Management Protocol (IGMP). In IPv6 networks, receivers use Multicast Listener Discovery (MLD) to inform the network of their multicast group membership. These protocols are described below. Multicast: Conformance and Performance Testing Copyright Ixia,

10 4. Multicast Protocols There are a number of different routing protocols for intra-domain multicast routing, a number of protocols for inter-domain multicast routing, and some that are well suited to serve both intra- and inter-domain multicast routing implementations. The following sections describe several of the most widely used multicast routing protocols, including Protocol-Independent Multicast (PIM), Distance Vector Multicast Routing Protocol (DVMRP), and Multicast Open Shortest Path First Protocol (MOSPF). The task of these multicast routing protocols is to create efficient multicast delivery paths through the network, with each protocol using different algorithms to achieve this efficiency. Also discussed below are the multicast group membership discovery (group management) protocols, IGMP and MLD Multicast Group Management Protocols The multicast group management protocols are IGMP and MLD and are described in detail in the following sections Internet Group Management Protocol (IGMP) The Internet Group Management Protocol (IGMP) is used by IPv4 systems to report their IP multicast group memberships to neighboring multicast routers. Hosts use IGMP to register their dynamic multicast group membership, and the connected routers use IGMP to discover these group members. Hosts join groups by sending Join messages to the multicast routers on their LAN. These routers listen for IGMP messages and periodically send queries to find out which groups are active on particular subnets. IGMP is essentially an asymmetric protocol, specified from the point of view of a host. IGMP Version 1 is specified in RFC 1112 (August 1989), and Version 2 is specified in RFC 2236 (August 1997). IGMP Version 3, specified in RFC 3376 (August 2002), is interoperable with Versions 1 and 2. Version 3 adds support for "source filtering," which enables a system to report interest in receiving packets only from specific source addresses, as required to support Source-Specific Multicast (SSM), or from all but specific source addresses, sent to a particular multicast address. SSM is defined in draft-ietf-ssm-arch-xx, and uses a restricted set of multicast addresses Multicast Listener Discovery (MLD) The purpose of the Multicast Listener Discovery (MLD) protocol is to enable an IPv6 router to discover the presence of multicast listeners (receivers) on links directly attached to it, and to determine specifically which multicast addresses are of interest to which attached nodes. This protocol was originally defined by RFC Its first version, MLDv1, supports join and leave functions similar to IGMP. MLD Version 2, defined by RFC 3810 and designed to be interoperable with MLDv1, adds the ability for a node to report interest in listening to packets with a particular multicast group address but only those packets from specific source addresses, or from all sources except for specific source addresses. The added support for "source filtering" in MLDv2 thus supports Source- Specific Multicast (RFC3569) from specific source addresses, or from all BUT specific source addresses, sent to a particular multicast address. 8 Copyright Ixia, 2005 Multicast: Conformance and Performance Testing

11 4.2. Multicast Routing Protocols This section describes several Multicast Routing Protocols, including PIM-SM, SSM, MBGP, MSDP, and DVMRP Protocol Independent Multicast-Sparse Mode (PIM-SM) Protocol Independent Multicast Sparse Mode (PIM-SM) Version 2 is defined in RFC 2362 (and updated by other existing IETF drafts, designed to obsolete the current RFC 2362). Dense mode protocols present significant scaling problems since flooding packets in all directions (and periodically re-flooding to reach new receivers) is inherently inefficient, especially when there are only a few receivers. To address this shortcoming, sparse mode protocols such as Protocol-Independent Multicast Sparse Mode (PIM-SM) were developed. Sparse mode protocols make the more realistic assumption that the network will be sparsely populated with multicast receivers. Instead of flooding the transmission in all directions, a multicast router sends the transmission only to a specific Rendezvous Point (RP), which plays the role of a proxy for both receivers and senders and is available to all members of a multicast group for that transmission. Receivers must explicitly request membership in the multicast group. This is sometimes called an explicit join distribution method. Because sparse mode protocols generally provide more efficient use of the available bandwidth, they are the basis for most current multicast development and implementations. PIM-SM maintains the traditional IP multicast service model of receiver-initiated membership and uses explicit joins that propagate hop-by-hop from directly connected member routers along the distribution tree. PIM-SM is not dependent on a specific unicast routing protocol. With the PIM-SM protocol, there is one Rendezvous Point (RP) router per multicast domain, which serves as the root of a unidirectional shared distribution tree whose leaves are multicast receivers. The term upstream is used to indicate the direction toward the root of the tree; downstream indicates the direction away from the root of the tree. The address of the RP can be configured statically by an administrator, or configured via a Bootstrap router (BSR) mechanism. In addition, if the flow of data packets from the source(s) to the multicast group is great enough, PIM-SM can create an optional shortest-path tree for an individual source (where the source is the root). The robustness, flexibility, and scaling properties of this architecture make it well suited to large heterogeneous inter-networks. Figure 8: PIM-SM Example Diagram Multicast: Conformance and Performance Testing Copyright Ixia,

12 Source-Specific Multicast (SSM) The Source-specific multicast (SSM) protocol identifies multicast traffic by both source and group address. SSM is an extension that allows datagram traffic to be forwarded to receivers from only those multicast sources for which the receivers have explicitly expressed interest. Thus, SSM requires support in receiver hosts and applications to signal source specific membership. Based on IETF guidelines, successful deployment of SSM requires IGMP and an end-to-end multicast-enabled network, with applications that use an IGMPv3 stack. RFC 3569 provides an overview of SSM, and the more recent IETF draft-ietf-ssm-arch-06.txt defines an extension to the Internet network service that applies to datagrams sent to SSM addresses, and defines the host and router requirements to support the extension. An IP datagram is transmitted by a source S to an SSM destination address G, and receivers can receive this datagram by subscribing to channel (S,G). SSM provides host applications with a channel abstraction, in which each channel has exactly one source and any number of receivers Multiprotocol BGP (MBGP) Defined in RFC 2283, Multiprotocol BGP (MP-BGP or MBGP) is an extension to BGP-4 that enhances BGP, allowing it to carry IP multicast routes. MBGP enables multicast routing policy throughout the Internet and can be used to connect multicast topologies within and between BGP autonomous systems. (The acronym MBGP is often read as Multicast BGP, but the correct name is Multiprotocol BGP.) Although BGP is a unicast, exterior gateway protocol, its multicast counterpart recognizes that a multicast network need not be congruent with that of the unicast network due to differences caused by topology and/or policy constraints. MBGP is a simple way to distribute two sets of routes simultaneously one set for unicast routing and one set for multicast routing. The routes associated with multicast routing are used by the multicast routing protocols to build data distribution trees. With MBGP, both unicast and multicast routes are carried in the same session but in different routing tables. MBGP carries multicast routes by adding the Subsequent Address Family Identifier (SAFI) to either of two new path attributes, the Multiprotocol Reachable Network Layer Reachability Information (NLRI) and the Multiprotocol Unreachable NLRI. The SAFI specifies whether the forwarding information is unicast, multicast, or unicast/multicast. Because MBGP is an enhanced version of BGP-4, all of the familiar BGP policy and configuration tools are also available for multicast. The main advantages of MBGP are that an Internet can support non-congruent unicast and multicast topologies, and when the unicast and multicast topologies are congruent, they can support differing policies. Also, MBGP provides greater control over multicast paths, and it is valuable when having multi-homed scenarios with multiple multicast feeds. Another important advantage is that with MBGP, a router needs to know only its own internal topology and the path to each of the other ASs rather than the entire flat multicast topology, and that MBGP is backward compatible with BGP-4 (routers that do not understand the extended attributes simply ignore them.) The two main drawbacks to MBGP are the increased routing table size and the potential storage of redundant information, which can result in routers storing multiple sets of routes for the same prefixes Multicast Source Discovery Protocol (MSDP) Multicast Source Discovery Protocol (MSDP) is a mechanism used to communicate between routers passing information about multicast sources. Its two forms Comes - internal MSDP and external MSDP- are identical. External MSDP is spoken across the boundaries between PIM clouds, internal MSDP is spoken between RPs within a PIM cloud. Multicast Source Discovery Protocol (MSDP) is a mechanism used to connect multiple routing domains, generally PIM-SM domains. MSDP allows sources for a multicast group to be known to all of the rendezvous points (RPs) in different domains. Each MSDP router establishes adjacencies with internal and external MSDP peers, informing each other about active sources within the domain. When they detect active sources, the routers can send PIM sparse-mode explicit Join messages to the active source. When an RP in a PIM-SM domain learns of a new sender, for example, through PIM register messages, it constructs a "Source-Active" (SA) message and forwards it to all its MSDP peers. To discover multicast sources in other domains, RPs run MSDP over TCP and exchange source list messages to set paths to sources of interest, in other domains. Multicast data is delivered to receivers in those domains of interest, over the normal, source-tree building mechanism in PIM-SM. 10 Copyright Ixia, 2005 Multicast: Conformance and Performance Testing

13 Distance Vector Multicast Routing Protocol (DVMRP) The Distance Vector Multicast Routing Protocol (DVMRP) establishes multicast delivery paths over a series of routing devices. DVMRP is a distance-vector-routing protocol, based on the Bellman-Ford algorithm, implementing the Truncated Reverse Path Broadcasting algorithm. DVMRP establishes the appropriate paths (i.e., a distribution tree) between a multicast source and a receiver through a method called Reverse Path Forwarding (RPF). When a multicast router receives a packet, it sends the packet out in all paths except the one that leads back to the packet s source through a mechanism called flooding the packet. This method allows the data stream to reach all downstream LANs, possibly more than once. If a router is attached to a set of LANs with no receiving members for a particular Multicast transmission, the router sends a message back up the distribution tree to stop subsequent packets, thus pruning the branch of the tree beyond that router. Multicast routers exchange distance vector updates that contain lists of destinations and the distance in router hops to each destination. They maintain this information in a routing table. DVMRP uses the Internet Group Management Protocol (IGMP) to exchange routing datagrams. It is an Interior Gateway Protocol (IGP) suitable for use within a single autonomous system, but not between different autonomous systems. Similar to the Routing Information Protocol (RIP), DVMRP exclusively routes only multicast datagrams. One crucial difference between DVMRP and RIP is that while RIP operates in terms of routing and forwarding datagrams to a particular destination, the purpose of DVMRP is to keep track of the return paths to the source of multicast datagrams. Multicast: Conformance and Performance Testing Copyright Ixia,

14 5. Multicast Testing Challenges Multicast test tools must be able to perform a wide variety of functions to test and validate devices and systems adequately. For conformance testing, the test solution must be able to fully exercise the control plane of the device or system under test. Because the ability to receive multicast does not guarantee the ability to source this type of traffic for performance and scalability testing, the test solution must be able to emulate multicast routers at the control plane level and scale up for large capacity testing, checking both traffic being sent and traffic being received by the system under test. In addition, this solution must be able to drive traffic through the system at the data plane level to fully stress the device being tested Why Test for Multicast Conformance? Multicast standards and implementations are dynamic. There are typically multiple IETF drafts associated with multicast, in addition to scores of RFCs. In this constantly changing environment, maintaining compliance with standards is a significant challenge and a crucial one, since compliance is essential to the ongoing interoperability of network equipment from various vendors. Equipment vendors find themselves at the leading edge of these challenges as they repeatedly update their feature sets to conform to the latest standards and options, while at the same time improving performance and scalability. They must do this both to remain competitive in their markets and to meet the demands of their customers. Development test and quality assurance groups, therefore, need an efficient way to verify the suitability of their implementations. Formalized conformance testing against standards supplies this confidence. Beyond ensuring product interoperability and quality, conformance testing can also accelerate product development by detecting bugs or correcting design issues early in the development cycle, thereby reducing the product s time to market and hence increasing profitability. Multi-vendor environments are the reality today for both service providers and enterprise organizations. Such a reality is untenable without the assurance of equipment interoperability that is driven by standards-based implementations. Networks are also dynamic; so when upgrades are required, conformance and regression testing are crucial to ensure that new software releases do not break existing services. To achieve adequate test coverage for compliance to a standard, hundreds of conformance test cases are typically necessary to cover a given protocol, and these tests must be appropriately updated as necessary. To address these challenges, most vendors and service providers rely on conformance testing products that are maintained and supported by a dedicated third party Why Test for Multicast Scalability and Performance? Standards compliance and interoperability are prerequisites to a working multicast system, but the ultimate success of an implementation depends on its performance in a real network. Given the complexity of the protocols involved, scalability and performance are often genuine concerns. Equipment vendors typically test and publish the scalability and performance capabilities of their products to facilitate compliance assessment. End users will often validate the numbers while additionally testing specific network scenarios unique to their own deployment. Scalability is typically viewed as the biggest challenge in service provider networks today. Providers must understand the dynamics of growth in their networks as new customers are added, as well as the ultimate performance limits of their networks. Determination of routing capacity, verification of branch creation and proper traffic routing, measurement of the latency or the time it takes for multicast receivers to join a group, and how that time is affected by the group s size, are some of the metrics used to determine scalability and performance in a multicast system. Together, scalability and performance metrics can be competitive differentiators for equipment vendors. For service providers and network managers, they are key selection criteria between vendors. Characterizing these elements is critical, since they directly impact the service quality that can be delivered to the end customer. 12 Copyright Ixia, 2005 Multicast: Conformance and Performance Testing

15 6. Test Solution Requirements Multicast test tools must be able to perform a wide variety of functions to adequately test and validate multicast devices and systems. For conformance testing, the test solution must be able to fully exercise the control plane of the device or system under test. For performance and scalability testing, the test solution must be able to emulate a wide variety of topologies and hierarchies at the control plane level and scale up for large capacity testing. The solution must also be able to drive wire-speed traffic through the system at the data plane level to fully stress the device under test Optimized Hardware Platform While simple router emulations can be run on PCs or workstations, an optimized test system must be employed to provide complete testing capabilities and high levels of scalability. For example, to emulate a large network, a network interface on a test tool must support hundreds or even thousands of IP interfaces and MAC addresses requirements that standard off-the-shelf hardware cannot support. Purpose-built test hardware is required to provide the flexibility and scalability needed to adequately test multicast equipment Routing Protocol Emulation The test solution must be able to emulate the full range of routing protocols used in today s networks, including OSPF, IS-IS, RIP, BGP, and BGP+. These routing protocols are used to advertise the underlying network topologies over which the multicast system is established. In addition, the multicast testing solution supports the emulation of essential multicast protocols, such as IGMP, MLD, PIM-SM and PIM-SMv6, SSM, and MBGP. Testing equipment must be able to fully stress the dual-stack data planes of dual stack routers, generating and analyzing multicast IPv4 and IPv6 packets at wire-speed. 6.3 Integrated Control and Data Plane Testing Another significant requirement for the test tool is the ability to integrate control and data plane testing. This capability requires test tools to emulate large and complex topologies and networks, and then target data traffic over the advertised topology with an integrated traffic generator. The test tool must be able to run signaling protocols simultaneously with the routing protocols on the same network interface. The traffic is used to determine device and network performance, and to verify the proper operation of the control plane. Automated data plane traffic generation tied to the control plane emulation is necessary to eliminate the need to manually configure traffic to all destination networks. 6.4 Traffic Generation Once the network has been established and all connections signaled, the test tool must be able to inject multicast data traffic into the network topology, at speeds up to line rate, and it must be able to receive traffic as well. This means sending traffic over all of paths configured on the test interfaces. On the receiving side, the test tool must be able to gather statistics and capture the traffic for analysis. An increasingly common requirement for test tools is to emulate real enterprise applications over the network. This allows for the characterization of the network in terms the end user really cares about, namely, how their applications will perform. 6.5 Automation The complexity of multicast testing with intricate setup and analysis requirements, conditions tests to be repeatable and often automated. Scripting languages, such as Tcl (Tool Command Language), are used to provide automation in and to enable quality assurance and manufacturing environments to perform repeatable regression tests necessary to ensure product functionality and quality. Generally, setting up a large test bed consisting of a large number of multicast routers and receivers is not economically feasible. Instead, dedicated multicast test tools designed to emulate the same type of test topology are needed to solve the problem. Multicast: Conformance and Performance Testing Copyright Ixia,

16 7. Ixia s Approach to Multicast Testing 7.1. Multicast Conformance Testing Ixia has addressed the challenges of protocol conformance testing by developing IxANVL (Ixia Automated Network Validation Library), the industry standard conformance test suite. IxANVL IxANVL (Automated Network Validation Library) is a data network testing solution that validates the protocol implementations and operational robustness of networking devices. For protocol conformance testing, IxANVL supports over 30 protocols overall, and the multicast protocol conformance suite includes support for PIM-SM, PIM-DM, IGMP, MLD, and DVMRP. IxANVL provides positive as well as negative test cases against the RFCs that specify these standards. Negative tests help validate device response to "killer packets." IxANVL performs its tests as a dialogue: it sends packets to the router being tested, receives the packets sent in response, and then analyzes the response to determine the next action to take. This allows IxANVL to test complicated situations or reactions in a much deeper and flexible way than can be done by simple packet generation and capture devices. IxANVL can run on standalone workstations or via Ixia's optimized test platforms. IxANVL can be completely automated using a scripting interface. IxANVL source code is also available to users for customization, allowing a great degree of testing flexibility Multicast Scalability and Performance Testing Ixia s IP multicast emulation offers the most comprehensive tools for testing scalability and performance of multicast routers, including automation. The general methodology employed by Ixia for testing the scalability and performance of multicast routers involves first surrounding the device or system under test (DUT/SUT) with Ixia hardware test interfaces. The Ixia system then emulates everything else needed to test the device, including other multicast routers, IP route injection, host receivers, and traffic transmission. In this way, large and complex topologies can be simulated to test the DUT/SUT in realistic system environments with a minimum of hardware requirements. Ixia has developed two primary applications for Multicast performance testing, IxExplorer/IxRouter and IxScriptMate, each with a distinct testing focus. IxExplorer and IxRouter IxExplorer provides a high level of flexibility and functionality in protocol emulation, traffic generation, and analysis. IxExplorer is the primary controlling application for Ixia s purpose-built hardware test platform, allowing detailed configuration of protocols and analysis of test results. As part of IxExplorer, IxRouter provides protocol emulation capabilities, including a comprehensive set of IP multicast protocols, as well as other routing and signaling protocols such as OSPF, IS-IS, RIP, BGP, LDP, and RSVP-TE. IxRouter provides routing protocol emulation with layer 2 and 3 packet-generation and traffic analysis capabilities. IxRouter s GUI exposes many of the packet parameters so that complex IP unicast, and IP multicast packets can be created. IxRouter provides support for the emulation of versions 1, 2, and 3 of the Internet Group Membership Protocol (IGMP), and support Multicast Listener Discovery (MLD) Protocol version 1 and 2 for testing IPv4, IPv6, and dual-stack routers. These protocols are designed for Ixia s Load Modules, which integrate a CPU per port, offering unparalleled scalability and performance. IxRouter s spreadsheet GUI facilitates the entry of large and complex configurations, where, for example, displaying only the relevant protocol and stream parameters can customize a test environment. IxRouter can log and graph statistics, and can be run simultaneously on one or many ports, in conjunction with line-rate traffic, to simulate realistic Internet scenarios. Ixia s PIM-SM multicast routing emulation offers the capability to extensively test PIM-SM enabled routers. Designed for Ixia s Load Modules, Ixia s PIM-SM emulation offers unparalleled scalability and performance. Hundreds of PIM neighbors are supported per port. Also, any combination of PIM Register and Join/Prune Messages can be advertised from a port to simulate traffic from multicast sources and destinations. Besides scalability, Ixia s protocol emulations can simulate any number of topologies to validate a System under Test s (SUT s) ability to create unidirectional, shared trees rooted at an RP per group, and shortest-path trees per source. Ixia s PIM-SM multicast routing emulation can be run with any combination of IGP and signaling protocols. To properly test Reverse Path Forwarding functionality, routing protocol emulation, such as the emulation of OSPF, IS-IS, RIP, or RIPng, must be used to advertise the sender source addresses. Also, Ixia s IGMP and MLD multicast emulations can be used to replicate hosts joining/leaving multicast groups, and can be used in conjunction with PIM-SM to simulate a variety of multicast topologies. 14 Copyright Ixia, 2005 Multicast: Conformance and Performance Testing

17 IxScriptMate IxScriptMate provides a framework for running automated test scenarios. Numerous test suites have been developed within the IxScriptMate environment for testing Multicast traffic throughput performance, latency, delay of multicast group Joins or Leaves, group capacity, and scalability. IxScriptMate simplifies the configuration process by defining a configuration for the test and displaying the relevant parameters for user input. Tests then run automatically, and the results are presented to the user. Figure 9: Ixia Emulation of Multicast Protocols. Here Ixia is emulating IGMP, as well as MLD clients. Multicast: Conformance and Performance Testing Copyright Ixia,

18 8. Conclusion Multicast is a crucial component of the Internet. It enables a wide variety of applications that require sending packets from one or more senders to a group of recipients, in a single operation. The long list of applications based on Multicast includes content distribution of news, security monitoring, file distribution and caching, interactive multimedia games and distance learning, multimedia teleconferencing and whiteboards, chat groups, polling and data collection, on-line auctions, parallel and distributed computing, etc. As Multicast continues to expand its functionalities through ongoing development, it has already proven to be a complex technology, encompassing a wide range of services and applications. Equipment vendors, carriers and service providers, as well as enterprise customers depend on the interoperability, scalability, and performance of their network equipment to perform multiple services, critical to their communications and core infrastructures. Handling Multicast's dynamic complexity requires proficient testing tools and proper test methodologies to help network managers and Multicast equipment vendors in several critical areas: Conformance to standards, to ensure interoperability in a complex, multi-vendor environment. Network scalability, to understand network limitations. Performance characterization, to avoid bottlenecks. Ixia provides that solution with IxANVL, the industry standard conformance test suite. IxRouter and IxExplorer, providing flexibility and functionality in protocol emulation, traffic generation, and analysis. IxScriptMate, providing the efficiency of automated testing. Together, these tools enable the providers of next-generation Multicast products and services to ensure the reliability, performance, scalability, and profitability of their offerings. The following appendix section provides sample test plans that demonstrate several multicast testing methods and scenarios. 16 Copyright Ixia, 2005 Multicast: Conformance and Performance Testing

19 9. Appendix: Multicast Testing Example This section provides four test case examples for testing conformance and performance of networking equipment enabled for multicast forwarding. The first test case is an example of conformance testing. The second test case is an example of using automated scripts to quickly generate test results. The third demonstrates Ixia's Custom Engineering capability for creating a custom application for a specific technology. The last test case presents an example of utilizing a GUI test application to configure a complex test and take precise measurements PIM-SM Conformance Test Objective To characterize a router s protocol compliance with PIM-SM RFC standards. IxANVL offers conformance test suites to validate the vendor s protocol implementations of PIM-DM and PIM-SM. In this example, we will focus only on the PIM-SM test suites Test Setup The IxANVL software runs a Linux workstation and it supports Ethernet interfaces for Multicast conformance tests. A minimum of two test ports is required to run most tests. A third test port will be required if the full test suite is run. Optionally, IxANVL can also utilize Ixia s Ethernet Load Modules that have a CPU per port for the multicasting test suites. In this configuration, the IxANVL Linux workstation communicates with an Ixia chassis to take ownership of the respective ports and runs the respective test scripts through the port CPUs. For each test, IxANVL simulates a portion of a topology in order to test the DUT in various network configurations. Up to 17 simulated topologies are utilized by the test suite. For example, in Figure 10, IxANVL simulates two multicast-enabled routers connected to two DUT interfaces. One of these simulated routers is the BSR and RP; the other is downstream from the DUT. Figure 10: PIM-SM Conformance Sample Topology Multicast: Conformance and Performance Testing Copyright Ixia,

20 Parameters DUT Hostname IP First Unused Net IP Unused Net Mask IP Number Unused Nets Multicast group address Interface Entries The name of the device under test, a PIM router A starting IP address in dotted-decimal format. Specifies the network mask in dotted-decimal format. Specifies the number of networks that IxANVL can use for advertised routes. An IPv4 Multicast group address to be advertised by IxANVL. A number of layer 2 and 3 entries required to enable a particular interface. Table 2: PIM-SM Configuration Parameters The above entries are the minimum parameters necessary to run test cases manually. Ixia also provides additional entries to automate IxANVL testing. These entries specify capabilities (or limitations) of the specific DUT, or commands to configure the DUT for certain tests Methodology IxANVL runs a number of test cases against the DUT based on direct interpretation of the PIM-SM IETF draft-ietf-pim-sm-v2-new-07. Enter values for required configuration parameters in the IxANVL Configuration tab (see Figure 11). Configure each IxANVL network interface with the appropriate network parameters, including those of the DUT such as IP address, MAC address, gateway, etc. 1. Specify DUT capabilities and commands, typically via command scripts such as Expect scripts, if the test suite will be automated. 2. Select the set of IxANVL PIM-SM test cases to run during the test session (see Figure 12). The tree view allows selecting or deselecting a test case, an entire test branch, or the entire test suite. In addition, test cases can be filtered based on the RFC interpretation of MUST, Should, or May statements. Figure 11: PIM-SM Conformance Test Configuration 18 Copyright Ixia, 2005 Multicast: Conformance and Performance Testing