Managing LTE IP Transport Networks with Route Analytics

Managing LTE IP Transport Networks with Route Analytics WHITE PAPER

Table of Contents Executive Summary 3 LTE Core and Backhaul Transport Architectures 4 Why IP Networks Are Inherently Unpredictable 4 Traditional Network Management Many Points of View, No Big Picture 5 Route Analytics Seeing the Network from the Router s Point of View 6 Improving LTE Network Management with Route Analytics 7 Conclusion 10 Page 2 of 13

Executive Summary Mobile operators today confront a major evolution in their network architecture, service traffic, and economics. The explosion of smartphones, tablets and High Speed Packet Access (HSPA) mobile broadband traffic has driven the roll-out of Long Term Evolution (LTE) based on 3GPP standards. While mobile operators have relied for years on IP/MPLS networks for their mobile core backbone communications, LTE is driving a significant transformation of many mobile backhaul networks from statically engineered ATM over SONET architectures to Layer 3 IP/MPLS networks, particularly in the aggregation or High Radio Access Network (HRAN) layer. While the precise nature of this transformation varies dramatically depending on the legacy network assets and services, there s no question that the evolution to IP/MPLS requires a new approach to network Operations and Management (OAM). An inherent OAM challenge of IP is its dynamic nature. Unlike circuit-based TDM network architectures of the past, IP networks can continuously and automatically reroute traffic paths around link failures and other changes in the network infrastructure. The result is an intelligent but unpredictable network topology that can not only cause delays to sensitive voice and broadband traffic, but also make management visibility and operational processes much more challenging. Traditional network management tools don t provide visibility into IP network dynamics, without which it is difficult to reduce OPEX Key Performance Indicators (KPIs) such as Mean Time to Detection (MTTD) and Mean Time to Repair (MTTR). In addition, lack of insight into dynamic network behavior impedes accurate maintenance and capacity planning, leading to costly operations errors and CAPEX waste. Route analytics technology, which taps into the network s live routing protocol control plane to provide real-time, network-wide insight of the operational routing topology and the traffic flowing across all network paths and links, is a key OAM technology for LTE mobile core and backhaul IP networks. Deployed by hundreds of telecom and mobile operators today, route analytics transforms IP/MPLS network management processes, helping engineering and operations teams deliver optimal service performance, speed problem resolution, strengthen change management processes, proactively uncover network vulnerabilities, increase capacity planning efficiency and ensure network and service resilience. Effective integration of route analytics within the LTE mobile core and backhaul IP/MPLS network OAM portfolio can help mobile operators ensure higher service quality, leading to lower subscriber churn and reacquisition costs while reducing IP network OPEX and CAPEX. Page 3 of 13

LTE Core and Backhaul Transport Architectures Long Term Evolution (LTE) and System Architecture Evolution (SAE) as defined by the Third-Generation Partnership Project (3GPP) introduced significant architectural changes to mobile operator networks. Of greatest interest to those responsible for deploying and managing the underlying transport is the fact that unlike previous standards, LTE and SAE together introduce a completely IP-based communications paradigm: Evolved Packet System (EPS). EPS employs an IP-based Bearer concept, essentially IP packet flows with defined Quality of Service (QoS), as the communications channel between handsets and tablets (the User Equipment or UE in 3GPP parlance) and the Internet via core network (CN) gateways (see Figure 1). Figure 1: System Architecture Evolution/Evolved Packet Core (source: 4G Americas) 3G Mobile operators have for years operated IP/MPLS VPN backbone networks to interconnect their core nodes. With the onset of LTE, the much larger metro area backhaul networks that transport traffic from enodeb cell sites must transform to handle IP traffic flows. Some mobile operators are choosing to utilize IP-friendly Layer 2 networks based on carrier Ethernet for backhaul transport, but for most operators IP/ MPLS networks will be the transport network architecture of choice for LTE backhaul. There are a number of different approaches to designing these IP/MPLS backhaul networks. For example, due to the need to accommodate legacy ATM transport of 2.5G and 3G traffic, some operators are using point-to-point MPLS Layer 2 VPN tunnels to transport backhaul traffic between enodeb sites and core gateways. Others are choosing to utilize Layer 3 VPNs in the backhaul networks. While there are many architectural variants of IP/MPLS backhaul networks, they all must offer a high degree of resilience via Page 4 of 13

router, link and path redundancy, as well as automated traffic re-routing around failures via standardsbased OSPF or IS-IS protocols. In some cases, mobile operators will additionally implement MPLS Traffic Engineering (TE) via RSVP-TE tunnels to ensure bandwidth availability and fast re-route (FRR) for failure recovery. Given the size and the complexity of these networks, it is critical that mobile operators possess strong OAM capabilities based on next-generation technologies and tools that can address the dynamic nature of IP/MPLS networks and also provide visibility into their Layer 2 VPN tunnels. Why IP Networks Are Inherently Unpredictable LTE is just the latest example of the convergence of all types of communication over IP. A major reason that IP became the de facto worldwide standard for data communications networks is its automated resiliency based on intelligent IP routing protocols that control the traffic routing topology. But while IP s distributed routing intelligence makes it efficient and resilient, it also makes IP network behavior unpredictable and harder to manage. IP routing protocols automatically calculate traffic routes or paths from any point to any other point in the network based on the latest known state of network elements. Any change to those elements causes the routing topology to be recalculated dynamically. While this means highly resilient traffic delivery with low administrative overhead, it also creates endless variability in the active routing topology. Large networks with many redundant links can be in any one of millions of possible active routing topology states, which makes it much harder to understand and manage how traffic will be delivered (see Figure 2). The lack of network management visibility into dynamic network behavior can be seen in the timeconsuming process of correlating service problems to non device-specific network causes. For example, when a user reports a service performance problem that doesn t stem from an obvious hardware failure, pinpointing the root cause can be quite difficult because in a large, complex IP network, IT engineers have no way to know the route the traffic took through the network, the relevant links servicing the traffic, whether those links were congested at the time of the problem or even which devices were servicing the traffic. Without understanding these factors, troubleshooting processes become slow, inefficient guesswork games played by highly paid escalation engineers, increasing MTTR and raising OPEX. Change management processes suffer from the same problem, since engineers making planned configuration changes in the network have little or no idea of how the network-wide routing and traffic delivery behavior will change once the configuration change is made. This can lead to higher OPEX because changes must be rolled back or corrected, impacting service reliability and customer satisfaction. Page 5 of 13

Figure 2: The dynamic nature of IP routing presents a mathematically daunting challenge to network management. In the illustrated network above, there are only 4 core routers and 5 edge routers. If one assumes that traffic only enters the network from the edges, the routed topology (combination of routed paths) can be in any one of 55 (3,125) possible states, or 53 (125) probable states. As a network expands and the number of interconnected routers grows, the complexity of understanding the network s behavior increases exponentially. For relatively non-critical applications like email and web browsing, the impact of routing and traffic changes may be slight, but for mobile voice, SMS, data services, interactive gaming, and streaming media, which have sensitive latency requirements, the impact can be dire. Next-generation OAM approaches are needed to manage IP network unpredictability, prevent and mitigate the service impacts of routing and traffic changes, and lower costs. Traditional Network Management Many Points of View, No Big Picture Network management s purpose is to overcome the complexity inherent in a large network and provide better visibility to network operations and engineering. The overarching architectural principle of today s network management is to gather information on a vast number of different points in the network, and then correlate various point data to infer service conditions. The key mechanism for doing this is the Simple Network Management Protocol (SNMP), which gathers information from devices such as routers, Page 6 of 13

switches, security devices and servers, and their interfaces. The type of data gathered is: Device health: current status, CPU and memory utilization Fault indicators: up/down status, uptime, dropped packets, errors Traffic information: interface utilization: bytes in/out, packets in/out, configuration Service utilization information: utilization per class of service, threshold violations Having this point data is critical for example, an interface or device that fails, runs out of memory or is congested with traffic can have a direct impact on service traffic. However, the sum of all this point data often doesn t provide the level of understanding needed to reduce detection and repair times. Just knowing that an interface is full of traffic doesn t tell you why it is full. Where is the traffic coming from and where is it going? Is the traffic usually on this interface, or was there a change in the network or elsewhere that caused it to shift to this interface? If so - from where, when and for how long? Without answers to these questions, there is no real understanding of the behavior of the network as a whole, which robs the point data of much of its meaning. While there are correlation algorithms for deducing certain types of network conditions, the fact of the matter is that SNMP was never designed to explain the complexities within routed IP networks. SNMP s key limitation is that it is too periodic polling cycles from 30 seconds to several minutes simply cannot produce an accurate portrait of the network s routing state, with its sometimes rapid and high-volume state changes. Even speeding up the polling cycle say, to every five seconds would still miss many routing state changes and would generate so much management traffic overhead as to be impractical. What s needed is a network management approach that can complement traditional, polled, device data collection with real-time tracking of routing protocol and traffic flow message events. Routing protocols such as OSPF, IS-IS and BGP send event messages to notify their peers of routing changes, such as a routed link going down, or a new routed prefix being added to the network. Likewise, routers utilizing traffic flow technologies such as NetFlow, broadcast event messages carrying the volume of traffic in specific IP flows. It is critical to monitor and be able to analyze these routing and traffic events in order to understand and manage IP networks dynamic behavior. Route Analytics Seeing the Network from the Router s Point of View Route analytics technology, adopted globally by hundreds of service providers, mobile operators, cable MSOs, large enterprises and government agencies, provides a new level of network visibility. The technology exploits a different type of network visibility that is afforded by tapping into the routing protocols the source of intelligence that determines how IP/MPLS VPN networks deliver traffic. Page 7 of 13

Route analytics are made possible by a sophisticated data collection technique. Using a collector that acts like a passive router, peering with selected routers across a network, and using the routing protocols OSPF, IS-IS and MP-BGP the control messages that routers use to calculate how traffic will be sent across the network can be recorded (see Figure 3). By processing this information just the way routers do albeit in a more comprehensive fashion every Layer 3 routed path in the network can be calculated, from every host to every other host. Thus, a routing topology of the entire network can be created and maintained for operational and engineering analysis. Since routing protocols report changes to the topology within milliseconds, the topology map is continuously updated in real time and always reflects exactly the way the real network is operating. Figure 3: Route analytics technology passively peers with, listens to and analyzes routing protocols to provide a real-time, network-wide understanding of all IP/MPLS VPN topology changes. Route analytics technology integrates traffic information into this live topology map by collecting flow data (statistical information on unidirectional IP traffic streams generated by routers, such as IPFIX, NetFlow) from key traffic ingress points such as IP edge routers and Internet and roaming peering points. Using knowledge of the precise path that every flow takes at any time through the network, route analytics project the traffic data onto the component links of that path (see Figure 4). The result is a highly accurate, integrated routing and traffic map that shows the volume of class-of-service (CoS) traffic on every link in the network. Since both routing and traffic data are generated and recorded continuously into a database, it is possible to view the network conditions exactly as they were at a past moment in time. In addition, since the topology is algorithmically calculated, it is possible to model routing and traffic changes, and simulate changes in network-wide behavior. Page 8 of 13

Data Center Flow 1 Src Dest...... Netflow Data Internet Flow 1 Src Dest... Flow 2 Src Dest... Netflow Data Traffic Explorer User Figure 4: Route analytics integrate NetFlow traffic statistics information into the IP/MPLS VPN routing topology by mapping traffic flows onto their routed paths. The result is a real-time, integrated routing and traffic topology. Improving LTE Network Management with Route Analytics Route analytics technology provides new network management visibility to mobile operators who must manage their mobile core and backhaul IP networks to deliver excellent service quality. For the first time, engineers can understand the relationship between service delivery and network operations. The results are greatly improved accuracy and efficiency of key business processes, contained costs, increased subscriber loyalty and lower customer churn. Real-Time, Network-Wide Routing and Traffic Monitoring, with Alerting: Route analytics provide visibility into traffic flows on all internal and external links in the network. Network Operations can easily monitor critical traffic paths, MPLS VPNs and RSVP-TE tunnels to ensure that the network architecture maintains adherence to design specifications, and traffic remains under utilization thresholds across the entire network. Real-time alerts via SNMP, syslog or console views help reduce MTTD of service-impacting issues and enable more proactive customer experience management. For example, operators can monitor reachability to distributed enodeb s in the network, detect when any have lost reachability or have a path change that is sub-optimal. Route analytics also reveal IP signaling plane stability issues such as excessive overall Layer 3 network churn, problems like link flaps or the loss of routing redundancy to key Internet Autonomous Systems (AS) or peering partners. Page 9 of 13

Benefit: Network operations can detect and anticipate problems much faster, reducing and preventing service impacts, lowering MTTD for packet core network issues by up to 60%. Layer 2 VPN Visibility Extends Analytics Capabilities: By monitoring Layer 2 VPNs, the power of route analytics is expanded to include the same link level data on Virtual Leased Lines (VLL) as with other typical Layer 2 links. For example, traps and alerts can be generated for Virtual Private Wire Service (VPWS) VLL State, VLL Redundancy and Service availability. This enables mobile operators to monitor pseudo-wires being used for backhauling cell site data. Network engineers also can verify that the actual deployments of Layer 2 VPNs conform to design specifications (see Figure 5). Benefit: Service resiliency and redundancy can be confirmed and status monitored for quicker response to failures. Figure 5: Route analytics incorporate path information from transported Layer 2 VPNs (e.g. ATM, E-Line, Frame Relay, SONET/SDH and TDM) providing tools for managing mobile backhaul resiliency and redundancy. Rewindable Routing and Traffic History for Improved Troubleshooting: By continuously recording the state of routing and traffic over time, route analytics technology accurately portrays the network-wide state of all links, peerings, paths, and prefixes as well as all traffic flows at any point in time. Engineers can rewind the network topology to pinpoint the precise MPLS VPN, RSVP-TE tunnel and routed path that service traffic took through the network, as well as the utilization of the component links, at the time of a problem. For example, in the case of a suboptimal VLL path from the mobile core to an enodeb, engineers Page 10 of 13

can see exactly what precipitated the change at what time and visualize the resulting path and traffic levels along the path. Using these historical forensics, engineers can solve hard-to-find and intermittent problems in less time, increasing operational efficiency. Benefits: MTTR for packet core network issues lowered by 20-40%. Improved network and service quality, and customer service responsiveness, results in higher customer satisfaction and lower churn. Network Modeling for More Robust Change Management Processes: Route analytics provide powerful modeling capabilities that can be used to strengthen change management processes. Industry research shows that 80 percent of unplanned downtime is caused by people and process issues, including poor change management practices, while the remainder is caused by technology failures and disasters. Route analytics allow engineers to model and simulate planned IP, MPLS VPN, L2 VPN and RSVP-TE routing and traffic changes, and to accurately predict the effect on the entire network s behavior and on service levels. For example, highly accurate modeling can help ensure path resilience between enodeb and core gateway nodes. After making changes, engineers can use route analytics to validate correct network-wide routing and traffic behavior in real time. Benefit: Service impacts from change management errors reduced by 25% or more. Internet Routing and Analysis for Improved Service Performance: Mobile broadband service customers who access the Internet judge their mobile operator s service quality by how well they can access their favorite applications and websites. Route analytics for BGP Internet routing help network engineers and planners identify important sources of traffic for key customer groups. Then, using simulation and modeling, they can find ways to optimize routing between multiple Internet peerings to achieve the shortest number of AS hops from those sources and reduce traffic latency from those key sites. Real-time monitoring of BGP AS paths to critical external networks can alert network operators to a loss of redundancy so that they can take measures to ensure service delivery continuity. In addition, route analytics technology can be used to ensure acceptable external peering utilization levels and optimize transit and peering arrangements, which can significantly reduce mobile operator costs. Route analytics provide engineers with a set of capabilities, including the ability to monitor peering or transit traffic to ensure it is within contracted ranges, as well as analyze, identify and justify new peering relationships. Engineers can also accurately simulate proposed peering changes to project exactly how traffic would behave with the proposed changes, helping them to make more informed investment decisions. Whether moving traffic from paid transit to settlement-free peering, or balancing between multiple transit providers, route analytics provide the intelligence operators need to optimize their peering traffic and maximize their bottom line. Benefits: Improves customer service quality experience, increases customer loyalty and lowers customer churn. Reduces peering and transit operating costs by up to 20%. Network-Wide Routing Health Audits for Improved Service Continuity: One of the hardest challenges in the midst of complex network operations is to anticipate and avoid problems. There is often no Page 11 of 13

insight into the potential causes of service impacts. Route analytics provide a network-wide audit of routing health by systematically examining the network for problems and vulnerabilities, such as outof-policy asymmetric routes, routing black holes, lack of or potential loss of path diversity between critical service nodes, underutilized assets and potential redundancy failures. By proactively identifying misconfigurations and potential failure points, route analytics technology enables engineers to prioritize fixes, increase network quality and prevent service impacts. Benefit: Higher network quality, lower service impacts, and improved customer satisfaction. Network-Wide Capacity Planning for Reduced CapEx: One of the most important network management processes for capital-intensive mobile operator networks is accurate capacity planning. Unfortunately, network planners often lack accurate and comprehensive information about the network s traffic utilization over time, leading to inaccurate planning exercises, sub-optimally deployed resources and wasted capital expenditures. By maintaining an always-accurate model of the entire network s routing and traffic behavior, route analytics technology provides the basis for rapid, iterative and automated capacity planning and traffic trending. Benefit: Capital expenditure savings of up to 20% for evolved packet core network infrastructure. Conclusion This paper has established the need for insight into the dynamic behavior of LTE core and backhaul IP transport networks, and how route analytics technology meets this need. Mobile operators that leverage route analytics to increase the automation of IP/MPLS and Layer 2 VPN network operations and engineering will gain a sustained competitive advantage due to the ability to deliver more reliable and higher quality services while increasing profitability. Packet Design s Route Explorer System combines route analytics for all IGP and BGP protocols with pathaware traffic flow analytics. In a single code base, it offers: Real-time visibility into routing and traffic behavior plus intelligent alerts for proactive operational monitoring and more efficient triage of service interruptions DVR-like replay and analysis of routing events for faster troubleshooting of intermittent and hardto-find service delivery issues Interactive simulation of configuration changes for risk-free network maintenance Predictive analysis of new workloads for better capacity planning Page 12 of 13

To learn more about Packet Design and Route Explorer, please: Email us at info@packetdesign.com Visit Packet Design s web site at www.packetdesign.com Call us at +1.408.490.1000 Corporate Headquarters Packet Design 2455 Augustine Drive Santa Clara, CA 95054 Phone: 408.490.1000 Fax: 408.562.0080 Page 13 of 13