Wide-Area Traffic Management for. Cloud Services

Transcription

1 Wide-Area Traffic Management for Coud Services Joe Wenjie Jiang A Dissertation Presented to the Facuty of Princeton University in Candidacy for the Degree of Doctor of Phiosophy Recommended for Acceptance By the Department of Computer Science Adviser: Jennifer Rexford & Mung Chiang Apri 2012

2 Copyright by Joe Wenjie Jiang, A rights reserved.

3 Abstract Coud service providers (CSPs) need effective ways to distribute content across wide area networks. Providing arge-scae, geographicay-repicated onine services presents new opportunities for coordination between server seection (to match subscribers with servers), traffic engineering (to seect efficient paths for the traffic), and content pacement (to store content on specific servers). Traditiona designs isoate these probems, which degrades performance, scaabiity, reiabiity and responsiveness. We everage the theory of distributed optimization, cooperative game theory and approximation agorithms to provide soutions that jointy optimize these design decisions that are usuay controed by different institutions of a CSP. This dissertation proposes a set of wide-area traffic management soutions, which consists of the foowing three thrusts: (i) Sharing information: We deveop three cooperation modes with an increasing amount of information exchange between the ISP s (Internet Service Provider) traffic engineering and the CDN s (Content Distribution Network) server seection. We show that straightforward ways of sharing information can be quite sub-optima, and propose a Nash bargaining soution to reduce the efficiency oss. This work sheds ight on ways that different groups of a CSP can communicate to improve their performance. (ii) Joint contro: We propose a content distribution architecture by federating geographicay or administrativey separate groups of ast-mie CDN servers (e.g., nano data centers) ocated near end users. We design a set of mechanisms to sove a joint content pacement and request routing probem under this architecture, achieving both scaabiity and cost optimaity. This work demonstrates how to jointy contro mutipe traffic management decisions that may have different resoutions (e.g., inter vs. intra ISP), and may happen at different timescaes (e.g., minutes vs. severa times a day). (iii) Distributed impementation: Today s coud services are offered to a arge number of geographicay distributed cients, eading to the need for a decentraized traffic contro. We present DONAR, a distributed mapping service that outsources repica seection, whie providing a sufficienty expressive service interface for specifying mapping poicies based on performance, oad, and cost. Our soution runs on a set of distributed mapping nodes for directing oca cient requests, which ony requires a ightweight exchange of summary statistics for coordination between mapiii

4 ping nodes. This work exempifies a decentraized design that is simutaneousy scaabe, reiabe, and accurate. Coectivey, these soutions are combined to provide a synergistic traffic management system for CSPs who wish to offer better performance to their cients at a ower cost. The main contribution of this dissertation is to deveop new design techniques to make this process more systematic, automated and effective. iv

5 Acknowedgments I owe tremendous thanks to a great many peope that I am fortunate to meet, who hep buid my research and share their wisdom of ife. I am deepy gratefu to my advisors, Jennifer Rexford and Mung Chiang, for infuencing me by their pursuit of top quaity research. I am fortunate to have both of them advise me an unparaeed opportunity to earn appying theory in soving practica probems. Thanks to Jen for her sefess offerings throughout my entire graduate study: the freedom needed to pursue exciting ideas, the knowedge needed to sove compex probems, the optimism needed to chaenge the unknowns, and the skis needed for perfection on a aspects of research: writing, presentation, communication and teamwork. Thanks to Mung for his dedication that heps shape this thesis, his insightfu thoughts on both research and ife, his guidance on being a professiona researcher, and his passion that aways encourages me to expore new areas fearessy. I coud not ask for more from them. I woud ike to thank Mike Freedman, Andrea LaPaugh, and Augustin Chaintreau for serving on my thesis committee. Mike is aso the co-author and mentor of the DONAR project presented in Chapter 4. His taste of research and sharp comments have aways been a reiabe source for improving my work. Thanks to Andrea for enightening my presentation. Thanks to Augustin for being hands-on, which has made this thesis more soid and compete. I woud aso ike to thank the contributors of this thesis. Rui Zhang-Shen co-authored the work in Chapter 2. I am aso gratefu for her mentorship during the eary stage of my Ph.D. Thanks to Stratis Ioannidis, Laurent Massouie and Fabio Picconi for heping with the proofs and measurement data in Chapter 3. Thanks to Laurent and Christophe Diot for giving me the opportunity to intern at Technicoor Research Lab in Paris. Thanks to Stratis for demonstrating the quaity of a true theorist, and other researchers for making the internship an unforgettabe experience. Thanks to Patrick Wende, the gifted young undergrad, for a the system impementation work in Chapter 4. I enjoyed our brainstorming, deadine fights, and unch sandwiches. Specia thanks to my other coaborators, incuding S.-H. Gary Chan, Minghua Chen, Sangtae Ha, Tian Lan, Shao Liu, Srinivas Narayana, D. Tony Ren, Bin Wei, and Shaoquan Zhang, for their inputs to the work that I am not abe to present in this thesis. Thanks to Gary for hosting my summer visit in HKUST, and providing generous resource and support to conduct my research. v

6 Thanks to Mike Schansker, Yoshio Turner, and Jean Tourrihes for mentoring my internship at HP Labs, making it a productive and peasant winter in Pao Ato. I woud ike to thank a ong ist of members (past and present) from Cabernet group: Ioannis Avramopouos, Matthew Caesar, Aex Fabrikant, Sharon Godberg, Robert Harrison, Eiott Karpiovsky, Eric Keer, Changhoon Kim, Haakon Ringberg, Michae Schapira, Srinivas Narayana, Martin Suchara, Peng Sun, Yi Wang, Minan Yu, Rui Zhang-Shen, and Yaping Zhu; and from EDGE Lab: Ehsan Aryafar, Jiasi Chen, Amitabha Ghosh, Sangtae Ha, Prashanth Hande, Jiayue He, Jianwei Huang, Hazer Inatekin, Ioannis Kamitsos, Hongseok Kim, Haris Kremo, Tian Lan, Ying Li, Jiaping Liu, Shao Liu, Chris Leberknight, Soumya Sen, Chee Wei Tan, Feix Wong, Dahai Xu, and Yung Yi. Working in a mixed cuture of computer science and eectrica engineering, and with foks of versatie expertise, has given me a unique opportunity to broaden my scope of knowedge and ways of thinking. It was great to have worked with and earned from them. Specia thanks to Meissa Lawson for her years dedication in serving as the graduate coordinator. Friends have immensey enriched my ife at graduate schoo. I thank: Dain Shi, for being ike a brother; Yunzhou Wei, for beers and basketba, Yiyue Wu, for being the best roommate; Wei Yuan, for being a good od-schoo friend; Yinyin Yuan, for comparing Princeton and Cambridge, and introducing the art of wine; Pei Zhang, for offering fun and sefess hep; and many others from computer science department, for their support and making me fee home. I woud ike to thank John C.S. Lui for sharing his wisdom, and continuay encouraging me to foow the heart. I d ike to acknowedge Princeton University, Nationa Science Foundation, Air Force Office of Scientific Research, and Technicoor Labs for their financia support. Thanks to Yuanyuan for bringing me happiness and being part of my ife. Thanks to my parents for their enduring ove and support. This dissertation is dedicated to you. vi

7 Contents Abstract iii 1 Introduction An Overview of Today s Coud Service Providers Requirements of CSP Traffic Management The Need for Sharing Information The Need for a Joint Design The Need for a Decentraized Soution Design Approaches A Top-Down Design Optimization as a Design Language Optimization Decomposition for a Distributed Soution Contributions Cooperative Server Seection and Traffic Engineering in an ISP Network Introduction Traffic Engineering (TE) Mode Server Seection (SS) Modes Server Seection Probem Server Seection with End-to-end Info: Mode I Server Seection with Improved Visibiity: Mode II Anayzing TE-SS Interaction TE-SS Game and Nash Equiibrium vii

8 2.4.2 Goba Optimaity under Same Objective and Absence of Background Traffic Efficiency Loss The Paradox of Extra Information Pareto Optimaity and Iustration of Sub-Optimaity A Joint Design: Mode III Motivation Nash Bargaining Soution COTASK Agorithm Performance Evauation Simuation Setup Evauation Resuts Reated Work Summary Federating Content Distribution across Decentraized CDNs Introduction Probem Formuation and Soution Structure System Mode A Goba Optimization for Minimizing Costs System Architecture A Decentraized Soution to the Goba Probem Standard Dua Decomposition A Distributed Impementation Request Routing and Service Assignment Inter-Domain Request Routing Intra-Domain Service Assignment Optimaity of Uniform Sot Poicy Content Pacement Designated Sot Pacement An Agorithm Constructing a Designated Sot Pacement viii

9 3.6 Performance Evauation Uniform-sot service assignment Synthesized trace based simuation BitTorrent trace-based simuation Reated Work Summary DONAR: Decentraized Server Seection for Coud Services Introduction A Case for Outsourcing Repica Seection Decentraized Repica-Seection System Research Contributions and Roadmap Configurabe Mapping Poicies Customer Goas Appication Programming Interface Expressing Poicies with DONAR s API Repica Seection Agorithms Goba Repica-Seection Probem Distributed Mapping Service Decentraized Seection Agorithm DONAR s System Design Efficient Distributed Optimization Providing Fexibe Mapping Mechanisms Secure Registration and Dynamic Updates Reiabiity through Decentraization Impementation Evauation Trace-Based Simuation Predictabiity of Cient Request Rate Prototype Evauation ix

10 4.6 Reated Work Summary Concusion Summary of Contribution Synergizing Three Traffic Management Soutions Open Probems and Future Work Optimizing CSP Operationa Costs Traffic Management Within a CSP Backbone Long Term Server Pacement Traffic Management within Data Centers Concuding Remarks x

11 List of Figures 1.1 Four parties in the coud ecosystem: Internet Service Provider (ISP), Content Distribution Network (CDN), content provider, and cient CSP traffic management decisions The interaction between traffic engineering (TE) and server seection (SS) An Exampe of the Paradox of Extra Information A numerica exampe iustrating sub-optimaity ISP and CP cost functions The TE-SS tusse v.s. CP s traffic intensity (Abiene topoogy) TE and SS performance improvement of Mode II and III over Mode I. (a-b) Abiene network under ow traffic oad: moderate improvement; (c-d) Abiene network under high traffic oad: more significant improvement, but more information (in Mode II) does not necessariy benefit the CP and the ISP (the paradox of extra information) Performance evauation over different ISP topoogies. Abiene: sma cut graph; AT&T, Exodus: hub-and-spoke with shortcuts; Leve 3: compete mesh; Sprint: in between Decentraized soution to the goba probem GLOBAL The repacking poicy improves the resource utiization by aowing existing downoads to be migrated Pacement Agorithm xi

12 3.4 Characterization of rea-ife BitTorrent trace. (a) Cumuative counts of downoads/boxes. (b) Per-country counts of downoads/boxes. (c) Predictabiity of content demand in one hour interva over one month period Dropping probabiity decreases fast with uniform sot strategy. Simuation in a singe cass with a cataog size of C = Performance of fu soution with decentraized optimization, content pacement scheme and uniform-sot poicy, under the parameter settings C = 1000, D = 10, B = 1000, Ū = 3, M = Performance of different agorithms over a rea 30-day BitTorrent trace DONAR uses distributed mapping nodes for repica seection. Its agorithms can maintain a weighted spit of requests to a customer s repicas, whie preserving cient repica ocaity to the greatest extent possibe DONAR s Appication Programming Interface Muti-homed route contro versus wide-area repica seection Interactions on a DONAR node Software architecture of a DONAR node DONAR adapts to spit weight changes Network performance impications of repica-seection poicies Sensitivity anaysis of using toerance parameter ɛ i Stabiity of area code request rates Server request oads under cosest repica poicy Proportiona traffic distribution observed by DONAR (Top) and CoraCDN (Bottom), when an equa-spit poicy is enacted by DONAR. Horizonta gray ines represent the ɛ toerance ±2% around each spit rate Cient performance during equa spit xii

13 List of Tabes 2.1 Summary of resuts and engineering impications Summary of key notation Link capacities, ISP s and CP s ink cost functions in the exampe of Paradox of Extra Information Distributed agorithm for soving probem (2.12a) To cooperate or not: possibe strategies for content provider (CP) and network provider (ISP) Summary of key notations Operations performed by a cass tracker Summary of key notations Decentraized soution of server seection xiii

14 Chapter 1 Introduction The Internet is increasingy a patform for onine services such as Web search, socia networks, mutipayer games and video streaming distributed across mutipe ocations for better reiabiity and performance. In recent years, arge investments have been made in massive data centers supporting computing services, by Coud Service Providers (CSPs) such as Facebook, Googe, Microsoft, and Yahoo!. The significant investment in capita outay by these companies represents an ongoing trend of moving appications, e.g., for desktops or resource-constrained devices ike smartphones, into the coud. The trend toward geographicay-repicated onine services wi ony continue and increasingy incude sma enterprises, with the success of coud-computing patforms ike Amazon Web Services (AWS) [1]. CSPs usuay host a wide range of appications and onine services, and ease the computationa power, storage and bandwidth to their customers, for instance, the video subscription service Netfix. Each onine service has specific performance requirements. For exampe, Web search and mutipayer games need ow end-to-end atency; video streaming needs high throughput, whereas socia networks require a scaabe way to store and cache user data. Cients access these services from a wide variety of geographic ocations over access networks with widy different performance. CSPs undoubtedy care about the end-to-end performance their customers experience. For instance, even sma increases in round-trip times have significant impact on their revenue [2]. On the other hand, CSPs must consider operationa costs, such as the price they pay their upstream network providers for bandwidth, and the eectricity costs in their data centers. Large CSPs can easiy 1

15 send and receive petabytes of traffic a day [3], and spend tens of miions of doars per year on eectricity [4]. As such, CSPs increasingy need effective ways to manage their traffic in order to optimize cient performance and operationa costs. 1.1 An Overview of Today s Coud Service Providers Traditionay, traffic management over wide-area networks has been performed independenty by administrativey separated entities, which together make up today s coud ecosystem. As iustrated in Figure 1.1, there are four parties in the ecosystem: CDN (Data Center)! Server! CP!! Internet (ISPs)!! Cient Cient Cient Cient Figure 1.1: Four parties in the coud ecosystem: Internet Service Provider (ISP), Content Distribution Network (CDN), content provider, and cient. Internet Service Provider: Internet Service Providers (ISPs, a.k.a. network providers such as AT&T [5]) provide connectivity, or the bandwidth pipes to transport content simpy treating them as packets and thus are obivious of content sources. A traditiona ISP s primary roe is to depoy infrastructure, manage connectivity, and baance traffic oad inside its network by computing the network routing decisions. In particuar, an ISP soves the traffic engineering (TE) probem, i.e., adjusting the routing configuration to the prevaiing traffic. The goa of TE is to ensure efficient routing to minimize congestion, so that 2

16 users experience ow packet oss, high throughput, and ow atency, and that the network can gracefuy absorb bursty traffic. Content Distribution Network: Content Distribution Networks (CDNs, e.g., Akamai [6]) provide the infrastructure to repicate content across geographicay-diverse data centers (or servers). Today s CDNs strategicay pace servers across geographicay distributed ocations, and repicate content over a number of designated servers. CDNs sove a server seection (SS) probem, i.e., determining which servers shoud deiver content to each end user. The goa of server seection is to meet user demand, minimize network atency to reduce user waiting time, and baance server oad to increase throughput. Sometimes, the job of server seection can be outsourced to a third party, in addition to the CDN, to aow customers to specify their own high-eve poicies, based on performance, server and network oad, and cost. Content Provider: Content Providers (CPs, e.g., Netfix [7]), a.k.a. tenants of the coud service, produce, organize and deiver content to their cients by renting a set of CDN servers for better avaiabiity, performance and reiabiity. Content providers generate revenue by deivering content to cients. Increasingy, coud computing offers an attractive approach where the coud provider offers eastic server and network resources, whie aowing customers to design and impement their own services. Whie network routing and server seection are handed by individua CDNs and ISPs, such customers are eft argey to hande content pacement on their own, i.e., pacing and caching content on appropriate servers to meet cient demands and reduce end-to-end atency. Cient: Cients, who consume the content, choose their ISPs for Internet connectivity, and subscribe to various content providers for their services. Therefore, the user-perceived performance is affected by many factors that are infuenced by different ISPs, CDNs and CPs. As the Internet increasingy becomes a patform for onine services, the boundaries between these parties have been more burred than ever. We formay define a CSP as a service provider that pays two or mutipe roes in the coud ecosystem, for instance: ISP + CDN, e.g., AT&T, who depoys CDNs inside its own network. 3

17 Data Center! Cient Server! Mapping Node Cient Content!! Internet!! Mapping Node Cient Cient Data Center! Cient Server! Mapping Node Cient! Internet!! Mapping Node Cient Cient Data Center! Cient Server! CSP Backbone!! Cient! Internet!! Cient Cient (a) Content pacement (b) Server seection (c) Network routing Figure 1.2: CSP traffic management decisions CDN + CP, e.g., Youtube, who depoys many geographicay-distributed servers to stream videos. ISP + CDN + CP, e.g., Googe, who has its own data centers and the network backbone, provides onine services to its customers. In a of the above scenarios, CSPs have a unique opportunity to coordinate between trafficmanagement tasks that are previousy controed by different institutions. In particuar, today s CSPs optimize cient performance and operationa costs by controing (i) content pacement, i.e., pacing and caching content in servers (or data centers) that are cose to cients, (ii) server seection, i.e., directing cients across the wide area to an appropriate service ocation (or repica ), and (iii) network routing, i.e., seecting wide-area paths to cients, or intra-domain paths within a CSP s own backbone, as shown in Figure 1.2. To hande wide-area server-seection, CSPs usuay run DNS servers or HTTP proxies (front-end proxies) at mutipe ocations and have these nodes coordinate to distribute cient requests across the data centers. To have contro over wide-area path performance, arge CSPs often buid their own backbone network to inter-connect their data centers, or connect each data center to mutipe upstream ISPs when they do not have a backbone. Further, some CSPs depoy a arge number of nano data centers [8] and need to pace specific content in each server, or may additionay cache popuar content in the front-end proxies when they direct cient requests. In the rest of this chapter, we outine the detaied design requirements ( 1.2), introduce our design methodoogies ( 1.3), and summarize our main contributions ( 1.4). 4

18 1.2 Requirements of CSP Traffic Management Today s CSPs usuay do not make optima traffic management decisions and achieve the high performance and ow cost that they coud, due to (i) imited visibiity, (ii) independent contro, and (iii) poor scaabiity. To address these chaenges, we propose a compete set of networking soutions that promote information sharing, joint contro, and distributed impementation The Need for Sharing Information Traffic management decisions are made by administrativey separated groups of the CSP, or even affected by other institutions such as intermediate ISPs that transit traffic between cients and the CSP. They often have reativey poor visibiity into each other, eading to sub-optima decisions. Misaigned objectives ead to conficting decisions. Conventionay, ISPs and CDNs (or content providers) optimize different objectives. Typicay, ISPs adjust the routing configurations for a traffic inside their networks, in addition to the CDN traffic, in the hope of minimizing network congestion, achieving high throughput and ow atency, and reducing operationa costs such as traffic transit costs. On the other hand, CDNs direct cients to the cosest data center to reduce round-trip times, without regard to the resuting costs and network congestion. As a consequence, the network routing and server seection decisions are usuay at odds. For exampe, an ISP may prefer to route the CDN traffic on a onger path but with ower congestion, and CDN may direct a cient to a coser data center that is reached through an expensive provider path. Incompete visibiity eads to sub-optima decisions. In making server-seection decisions, a CDN needs to predict the cient round trip atency to a server, which depends on the widearea path performance. In practice, the CDN has imited visibiity into the underying network topoogy and routing, and therefore has imited abiity to predict cient performance in a timey and accurate manner. A CDN rates server performance by IP geoocation database [9, 10], or Internet path performance prediction toos [11], which are usuay oad obivious. Therefore, without information about ink oads and capacities, a CDN may direct excessive traffic to a geographicay coser server, eading to overoaded inks. 5

19 1.2.2 The Need for a Joint Design Traffic management decisions are made independenty by different institutions or different groups in the same company, yet they ceary infuence each other. As such, today s practice often achieves much ower performance or higher costs than a coordinated soution. Separate optimization does not achieve gobay optima performance. We observe that separate decision makings do not enabe a gobay optima performance, even given the compete visibiity into a participating systems. For exampe, separating server seection and traffic engineering, e.g., carefuy optimizing one decision on top of the other, eads to sub-optima equiibria, even when the CDN is given accurate and timey information about the network. In genera, such separate optimizations do not enabe a mutuay-optima performance, motivating a joint design for a coordinated soution. Decision making happens at different timescaes. Since traffic management decisions are made by different institutions, they usuay are optimized at different timescaes. For instance, the ISP runs traffic engineering at the timescae of hours, athough it coud run on a much smaer timescae. Server-seection is usuay optimized at a smaer timescae such as minutes to achieve an accurate oad-baancing across servers. Depending on content providers choices, how often the content pacement decision is updated can vary quite differenty, ranging from a few times a day to on-the-fy caching. These heterogeneities raise the need for a joint contro that is both accurate and practica The Need for a Decentraized Soution Traffic management decisions are often made in a centraized manner, eading to a high compexity and poor scaabiity. The functiona separation impied by today s architecture, and the arge number of network eements (e.g., data centers, servers, wide-area network paths, and cients), raise the need for a decentraized soution in our design. Functiona separation is practica and efficient. A joint design naturay eads to a centra coordinator for controing a traffic management decisions inside a CSP. However, we want a moduarized design by functionay separating these decisions, e.g., between the ISP and the CDN, yet achieving a jointy optima soution. Today s server seection, network routing, and content 6

20 pacement are themseves performed by arge distributed systems managed by separate groups in the same company, or even outsourced to third parties (e.g., Akamai running Bing s front-end servers, or DONAR [10]). Tighty couping these systems woud ead to a compex design that is difficut to administer. Instead, these systems shoud continue to operate separatey and run on existing infrastructures, with ightweight coordination to arrive at good coective decisions. Distributed impementation is scaabe and reiabe. The need for scaabiity and reiabiity shoud drive the design of our system, eading to a distributed soution that consists of a set of spatiay-distributed network nodes, such as mapping nodes (for server seection), backbone or edge routers (for networking routing), and nano data centers and proxy servers (for content pacement), to hande traffic management in an autonomous and coaborative manner. Whie a simpe approach by having a centra coordinator is straightforward, it introduces a singe point of faiure, as we as an attractive target for attackers trying to bring down the service. Further, it incurs significant overhead for the infrastructure nodes to interact with the controer, eading to excessive communication overhead. Finay, a centraized soution adds additiona deay, making the system ess responsive to sudden changes in cient demands (i.e., fash crowds). To overcome these imitations, we need a scaabe and reiabe distributed soution that runs on individua nodes whie sti attains a gobay optima performance. Meeting a the above requirements poses severa significant chaenges. First, each onine service runs at mutipe data centers at different ocations, which vary in their capacity (e.g., number of servers), their connectivity to the Internet (e.g., upstream ISPs for mutihoming, or bandwidth provisioning for individua nano data centers), and the proximity to their cients. These heterogeneities present many practica constraints in our design. Second, some probems, e.g., enabing the joint contro as an optimization probem, do not accept a simpe formuation that is computationay tractabe. We need advanced optimization and approximation techniques to make the soution computationay efficient and easy to impement, yet with provabe guarantee of optimaity. Further, as cient demands and network conditions are varying from time to time, our soution shoud be we adaptive to these changes. We address these chaenges in this dissertation. 7