DEMAND FORECAST, RESOURCE ALLOCATION AND PRICING FOR MULTIMEDIA DELIVERY FROM THE CLOUD

Transcription

1 DEMAND FORECAST, RESOURCE ALLOCATION AND PRICING FOR MULTIMEDIA DELIVERY FROM THE CLOUD by Di Niu A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy, Department of Electrical and Computer Engineering, at the University of Toronto. Copyright c 2013 by Di Niu. All Rights Reserved.

2 Demand Forecast, Resource Allocation and Pricing for Multimedia Delivery from the Cloud Doctor of Philosophy Thesis Edward S. Rogers Sr. Dept. of Electrical and Computer Engineering University of Toronto by Di Niu May 6, 2013 Abstract Video tra cconstitutesamajorpartoftheinternettra cnowadays. Yetmostvideode- livery services remain best-e ort, relying on server bandwidth over-provisioning to guarantee Quality of Service (QoS). Cloud computing is changing the way that video services are o ered, enabling elastic and e cient resource allocation through auto-scaling. In this thesis, we propose a new framework of cloud workload management for multimedia delivery services, incorporating demand forecast, predictive resource allocation and quality assurance, as well as resource pricing as inter-dependent components. Based on the trace analysis of a production Video-on-Demand (VoD) system, we propose time-series techniques to predict video bandwidth demand from online monitoring, and determine bandwidth reservations from multiple data centers and the related load direction policy. We further study how such quality-guaranteed cloud services should be priced, in both agametheoreticalmodelandanoptimizationmodel.particularly,whenmultiplevideo providers coexist to use cloud resources, we use pricing to control resource allocation ii

3 in order to maximize the aggregate network utility, which is a standard network utility maximization (NUM) problem with coupled objectives. We propose a novel class of iterative distributed solutions to such problems with a simple economic interpretation of pricing. The method proves to be more e cient than the conventional approach of dual decomposition and gradient methods for large-scale systems, both in theory and in trace-driven simulations. iii

4 Dedicated to my parents iv

5 Acknowledgments First and foremost, I would like to sincerely thank my thesis supervisor, Professor Baochun Li, for his invaluable guidance throughout my Ph.D. studies. I deeply appreciate his strict training and precious advice in all aspects of my academic development, which are life-long treasures for myself. I feel very fortunate to have Professor Li as my supervisor, and cherish all the opportunities to explore my favourite research topics and exert my own strengths. These would not have been possible without the unwavering support of Professor Li. The dedication, diligence and enthusiasm he has for an academic career was contagious and motivational for me, especially during tough times in my Ph.D. pursuit. I am also thankful for the excellent example Professor Li has provided as a successful scholar and professor in computer engineering and science. The members of the iqua research group have contributed immensely to both my personal and professional time at the University of Toronto. The group has been a source of good advice and collaboration as well as friendships. I am especially grateful for the valuable discussions and collaborations with Chen Feng, Hong Xu and Zimu Liu. Through the projects we worked on and the papers we coauthored, I very much appreciated their enthusiasm, intelligence, and amazing abilities in mathematical and experimental studies. I would like to thank all the other colleagues in the iqua research group, for precious friendships that have supported me throughout the past four years: Jin Jin, Wei Wang, Yuefei Zhu, Yiwei Pu, Yuan Feng, Junqi Yu, and Elias Kehdi. I would also like to acknowledge the visiting students in our group: Fangming Liu, Anh Tuan Nguyen, Zhi Wang, and Boyang Wang for the numerous nights we spent together in the lab. Other past group members that I have had the pleasure to work alongside of are graduate students Mea Wang, Chuan Wu, and Hassan Shajonia; visiting scholars v

6 Hui Wang and Ruixuan Li; and the numerous summer and rotation students who have come through the lab. Lastly, I would like to thank my parents for raising me with a passion for science and engineering, and for their love and encouragement in all my pursuits. Thank you! Di Niu University of Toronto May 6, 2013 vi

7 Contents Abstract iii iv Acknowledgments v List of Tables xii List of Figures xviii 1 Introduction Challenges in Multimedia Delivery Systems In the Era of Cloud Computing Thesis Scope Thesis Organization Related Work Measurement, Prediction and Learning in VoD Systems Cloud Workload Management and Resource Allocation Cloud Resource Pricing vii

8 CONTENTS CONTENTS 2.4 Algorithms for Distributed and Parallel Optimization Forecast in Video-on-Demand Systems Monitoring UUSee Video-on-Demand System Some Prediction Problems Modeling and Forecasting the Demand Population Prediction via Seasonal ARIMA Processes Forecasting the Aggregate Bandwidth Demand Inferring Initial Demand using Mixtures of Gaussians Predicting Peer Contribution Applications in a P2P System Summary Volatility Prediction and Dimension Reduction Modeling Demand Volatility Volatility Estimation and Resource Allocation Comparing Five Volatility Models Utilization as a Volatility Indicator Reducing Volatility through Diversification Dimension Reduction using PCA Summary Cloud Bandwidth Reservation for VoD Applications System Architecture Load Direction and Bandwidth Reservation The Optimal Load Direction viii

9 CONTENTS CONTENTS Suboptimal heuristics with Limited Replication Experiments based on Real-World Traces A Channel Interleaving Scheme Algorithms for Comparison Assumption Validation Predictive Auto-Scaling vs. Reactive Provisioning Theoretical Optimal vs. Replication-Limited Heuristics Summary The Bandwidth Reservation Market System Model The Cloud and Tenants The Broker and Load Direction Matrix Bandwidth Pricing in the Presence of a Broker Main Objectives Optimal Bandwidth Saving Discussions Pricing Region in Controlled Markets Broker Profit and Price Discounts in the Pricing Region Pricing in Free Markets: A Game Theoretical Analysis Nash Equilibrium: Existence and Uniqueness Market Properties and Equilibrium Bandwidth Costs Trace-Driven Simulations Summary ix

10 CONTENTS CONTENTS 7 Algorithmic Cloud Bandwidth Pricing A New Tenant-Cloud Agreement Pricing Towards Social Welfare Maximization An Equivalent Pricing Problem No Multiplexing vs. Multiplexing All Economic Implications Distributed Optimization with Coupled Objectives A Pricing Framework for Distributed Optimization Algorithms Dual Decomposition and Gradient Methods New Algorithms Relationship to the Nonlinear Jacobi Algorithm Convergence Conditions Contraction Mapping Convergence of Algorithm Convergence of Algorithm Convergence of Gradient Methods for Dual Problem Quadratic Programming: Convergence Speed Comparison Handling General Coupled Objectives Distributed Solutions to Cloud Pricing Equation Update Performance Evaluation Summary x

11 CONTENTS CONTENTS 8 Concluding Remarks Conclusions Future Directions Bibliography 185 xi

12 List of Tables 3.1 Frequently used notations (for a particular channel) Prediction of population at the initial stage based on mixtures of Gaussians for flash-crowd video channels The performance of five schemes averaged over each test period, in terms of QoS, resource utilization, and replication overhead xii

13 List of Figures 1.1 Bandwidth auto-scaling with quality assurance, as compared to provisioning for the peak demand and pay-as-you-go The evolution of several time series in a typical flash crowd video channel (with channel ID: A55F) since the channel was released. Each time unit represents 10 minutes. All rates are average quantities for the channel. The video bit rate is R =79.6 KB/s Population evolution in steady-state channels and various kinds of flash crowd channels. The samples of steady-state channels start from :51:58. The samples of flash-crowd channels start from the time that the corresponding video is released. The video release times are: channels 4872 and B830, before :51:58; channel F190, :21:03; channel FA5C, :47:20; channels 7A40 and E856, :57:34; channel 2E0B, :43:43; channel D55A, :03: The power spectrum of online population N t for various channels. To better view periodicity, the x axis is set to be the period measured in timestamps, which is the inverse of frequency xiii

14 LIST OF FIGURES LIST OF FIGURES 3.4 One day-ahead population prediction based on seasonal ARIMA models for three di erent channels. The model is trained based on the data of first few days. Prediction for N t only makes use of the observations {N : 0 < apple t 144} up to time t 144. The two flash-crowd channels have di erent daily decreasing of N t. The validation and training sets overlap for 1 day, since in a seasonal model, the predictor requires the first 144 validation samples to be the initial input The autocorrelation function (ACF) and partial autocorrelation function (PACF) of Ñ(t) =r 144 N t for the training data of channel B minutes-ahead prediction for the aggregate bandwidth demand D t in a popular video channel A55FF released at time period 264, compared against the trace data Prediction of initial demand using a mixture of Gaussians. The fitting function is trained with the first 360 population samples in channel 8331 released on :46:29. Test data 1 is from channel 57F7 released on :16:35, and test data 2 is from channel EDAF released on :49:39. R-square 1 =0.5864, R-square 2 =0.1374, RMSE 1 =74.44, RMSE 2 = Fitting AR(10) to {u t } of the first 9 days in channel A55F, and making 2.5 hour (15-step) ahead prediction for the rest of the days. The prediction for each u t is conditioned on {u ;0< apple t 15} xiv

15 LIST OF FIGURES LIST OF FIGURES 3.9 A fluid chart illustrating the dependency of c t on N t through the underlying system dynamics. The white arrows denote the transition rates, while the black arrows denote the dependencies. N c t is the number of cached peers and u c (t) istheaverageuploadrateofeachcachedpeerattimet The h-step prediction of average receiving rate c t and total receiving rate c t N t from cached peers in channel A55F. Prediction for c t or c t N t only makes use of the observations of N and c up to time t h. We choose d 1 = d 2 =0,p =54,q =0intheexperiments. Thevalidationandtraining sets overlap for 1 day, since in a seasonal model, the predictor requires the first 144 validation samples to be the initial input Progressive model learning and 2.5 hours ahead (15-step) prediction for the server bandwidth required by the peers in channel A55F over a 16-day period, compared with the real server bandwidth requirement calculated from the traces, and the actual server bandwidth used by the channel in the traces. The channel is released on :47:39, and the predicted values start from :47:39 (timestamp 181). The predictor is retrained with the most recent data on a daily basis Progressive model learning and the estimation for the average receiving rate r t of playing peers in channel A55F over a 12-day period. There is a 2.5-hour (15-step) delay in data collection. A comparison with the real r t and the server rate s t allocated to each peer is provided Online bandwidth demand monitoring and reservation. The reserved bandwidth should match the sum of the expected demand and a risk premium that tolerates volatility xv

16 LIST OF FIGURES LIST OF FIGURES 4.2 The ACFs of the disturbance Z t and Zt 2 obtained from fitting the seasonal ARIMA model (3.6) with d =0andp = q =20tobandwidthdemand {D t } in video channel A55FF One-step-ahead forecast errors for bandwidth demand D t in video channel A55FF, and predicted conditional standard deviations for the forecast errors The empirical CDF of the utilization of booked bandwidth and the ratio of time periods where the booked bandwidth is insu cient. Bandwidth reservation is performed in 169 popular video channels independently, each with a test period of 2 days Bandwidth reservation with probabilistic GARCH in channel A55FF, tested on a period of 3 days The mean and standard deviation of U when di erent numbers of video channels are randomly combined for bandwidth reservation. The achieved average e is less than 4% in all cases Bandwidth reservation with probabilistic GARCH for the mixed tra c of all the 93 channels in the 6-day period from time 264 to The first 3daysarethetrainingperiods,andtheother3days(time )are the test periods Bandwidth consumption time series of 5 representative channels over a 2.5-day period The first 3 principal components C 1 C 3 in bandwidth consumption series of 468 channels during 2 days via principal component analysis Variance explained in bandwidth consumption series of 468 channels in 2 days xvi

17 LIST OF FIGURES LIST OF FIGURES 4.11 Data projected onto the first 2 principal components. Each point is one of the 468 channels Root mean squared errors (RMSEs) of the two prediction methods over test period (1.25 days), compared to mean bandwidth consumption in each channel. The three largest channels are channels 295, 241 and minute-ahead conditional mean prediction in channel 172 over a test period of 1.25 days The departure of actual bandwidth consumption from its conditional mean forecast and the predicted standard deviation of bandwidth consumption in channel Q-Q plot of conditional mean forecast errors in channel 172 over the test period (1.25 days), in reference to Gaussian quantiles The system decides the bandwidth reservation from each data center and amatrixw =[w si ] every t minutes, where w si is the proportion of video channel i s requests directed to data center s. DC: data center Using demand correlation between channels, we can save the total bandwidth reservation, even within each 10-minute period, while still providing quality assurance to each channel. DC: data center The conditional mean demand prediction for virtual new channel 11, with atestperiodof1.5daysfromtime1585to QQ plot of innovations for t = vs. normal distribution xvii

18 LIST OF FIGURES LIST OF FIGURES 5.5 Predictive vs. reactive bandwidth provisioning for a typical peak period There are 35 data centers available, each with capacity 300 Mbps, and 91 channels, including 52 popular channels, 24 small channels, 15 non-zero new channels. K =2,k = Workload portfolio selection vs. random load direction for a typical peak usage period from time 1562 to K =2,k = A system of two cloud providers and two tenants. Random variables are labeled with r.v.. The data centers are possibly owned by di erent cloud providers The region of P i ( ) inagoodpricingpolicy{p i ( )}. P i ( ) isbetweenp i ( ) and P 0 i ( ), and satisfies P 0 i (1) P 0 i (1) The aggregate bandwidth P s A s(t)bookedbythebroker,comparedtothe aggregate bandwidth P i B i(t) needed if each channel books individually and the real aggregate demand P i D i(t) The histogram of payment discounts of all channels at all times in equilibrium, i.e., the histogram of 1 P i (1,t)/P 0 i (1,t)foralli and t Iterative updates of prices and guaranteed portions Behavior of Equation Update in a one dimensional case. o represents the starting point (w (0) i,k (0) i ) CDF of convergence iterations in all minute periods Final objective values SW(x )(expectedsocialwelfare)producedbydifferent algorithms The evolution of kw(t) w(t 1)k 1 in the first experiment xviii

19 Chapter 1 Introduction 1.1 Challenges in Multimedia Delivery Systems Video delivery systems, such as Youtube, Netflix, Hulu, etc., have gained unprecedented popularity on the Internet nowadays. According to Cisco Visual Networking Index: Forecast and Methodology, globally, Internet video tra c will be 55% of all consumer Internet tra cin2016,upfrom51%in2011. Thisdoesnotincludevideoexchanged through peer-to-peer (P2P) file sharing. Most Internet users watch live and on-demand videos through a web-browser and http-based video streaming, others may watch videos through client software downloadable from the video service provider. Other video delivery systems include on-line video gaming services such as OnLive [7]. Despite the popularity of Internet video and the ever-increasing demand for improved video quality, most Internet video services remain best-e ort systems. Since video flows are delay-sensitive, to guarantee the Quality of Experience (QoE) for an end-user, the video must be delivered from media servers to the end-user at a rate no less than the video bit rate (at least in the long run). However, QoE is usually not guaranteed in 1

20 1.2. IN THE ERA OF CLOUD COMPUTING 2 current Internet VoD systems, mainly due to the bounded egress bandwidth from video servers. 1 Most video service providers over-provision the bandwidth capacity of their streaming servers to provide quality assurance. However, over-provisioning is costly and even ine ective sometimes, since a large amount of server capacity is unused during nonpeak hours, whereas in the event of a flash crowd, when a large number users join the system, the provisioned capacity may not even be su cient. Certain VoD systems adopt a Peer-to-Peer (P2P) architecture (e.g., CoolStreaming [47], PPLive [8], UUSee [9]), where end-users can help the servers deliver video content to each other. While leveraging user upload bandwidth can alleviate the burden on media servers to some extent, the user resources are not dedicated and their contribution is not reliable. As a result, most P2P video systems are in fact peer-assisted systems, where the servers still play a major role in streaming, and thus face the same issue of how much server capacity should be provisioned. Because of the above reasons, in order to guarantee the QoE, what is missing from today s video delivery systems is a refined scheme to accurately predict the online user demand together with a flexible allocation mechanism that can economically vary the resource provisioning over time. 1.2 In the Era of Cloud Computing Cloud computing delivers Infrastructure as a Service (IaaS) that integrates computation, storage and network resources in a virtualized environment. It represents a new business model where applications as tenants of the cloud can dynamically reserve instances on 1 Due to the practice of over-provisioning, Quality-of-Service (QoS) is usually guaranteed at the core of the Internet, with very small losses and delays.

21 1.2. IN THE ERA OF CLOUD COMPUTING 3 demand. Cloud computing is changing the way that multimedia content providers operate their businesses, including video-on-demand (VoD) and online gaming companies. Traditionally, video companies invest in commodity servers as business grows and acquire monthly bandwidth deals from Internet Service Providers (ISPs). Nowadays, they can be freed from the complexity of hardware maintenance and network administration, relying on the cloud for service. One particular example is Netflix [6], which moved its streaming servers, encoding software, search engines, and huge data stores to Amazon Web Services (AWS) in 2010 [12, 13]. Moreover, many interactive gaming companies, e.g., OnLive [7], also depend on the cloud to stream their content to online players. One of the most important economic appeals of cloud computing is its elasticity and auto-scaling in resource provisioning. As has been pointed out, to accommodate the peak workload, over-provisioning is traditionally a common practice, leading to low resource utilization during non-peak hours. In contrast, in the cloud, the number of computing instances launched can be changed adaptively at a fine granularity with a lead time of hours or even minutes. This converts the up-front capital infrastructure investment to operating expenses charged by cloud providers. Since the cloud s autoscaling ability enhances resource utilization by closely matching supply with demand, overall expenses of the enterprise may be reduced. However, unlike web servers or scientific computing, VoD is a network-bound service with stringent bandwidth requirements. A major risk to the video applications using cloud services is that unlike CPU and memory, bandwidth is not guaranteed in currentgeneration cloud platforms (e.g., Amazon EC2), leading to unpredictable network performance [17, 63]. A lack of bandwidth guarantee to some degree impedes cloud adoption by

22 1.3. THESIS SCOPE 4 applications that require such guarantees, such as video-on-demand (VoD) applications [12] and transaction processing web applications [45]. The utility of tenants running these applications depends not only on the bandwidth usage, but more importantly on how many of their end-user requests are served with guaranteed performance. With an ever-increasing demand for performance predictability, a recent trend in networking research is to augment cloud computing to explicitly account for network resources. In fact, data center engineering techniques have been developed to expand the tenant-cloud interface to allow bandwidth reservation for tra cflowingfromavirtual machine (VM) in the cloud to the Internet [19, 39]. We envision that in future cloud platforms, bandwidth reservation will be a value-added feature that attracts tenants, such as video providers, who seek bandwidth guarantees. 1.3 Thesis Scope In this thesis, we aim to leverage the auto-scaling ability of the cloud to save the operational cost for multimedia delivery services, while providing quality assurance. Since bandwidth, as compared to CPU and memory, is a major factor that a ects the video delivery cost and quality, we focus on bandwidth auto-scaling. The benefits of bandwidth auto-scaling can be intuitively envisioned. As shown in Fig. 1.1(a), traditionally, a VoD provider acquires a monthly plan from ISPs, in which a fixed bandwidth capacity, e.g., 1 Gbps, is guaranteed to accommodate the anticipated peak demand. As a result, resource utilization is low during non-peak hours and demand troughs. Alternatively, a usagebased pay-as-you-go model is adopted by current cloud providers as shown in Fig. 1.1(b), where a VoD provider pays for the total amount of bytes transferred. However, the bandwidth capacity available to the VoD provider is subject to variation due to contention

23 1.3. THESIS SCOPE 5 Bandwidth Capacity Demand Capacity Demand Capacity Demand Days (a) Provisioning for peak demand Days Days (b) Pay as you go (c) Auto-scaling Figure 1.1: Bandwidth auto-scaling with quality assurance, as compared to provisioning for the peak demand and pay-as-you-go. from other applications, incurring unpredictable performance and QoS issues. Fig. 1.1(c) illustrates bandwidth auto-scaling and reservation to match demand with su cient resources, leading to both high resource utilization and quality guarantees. Apparently, the more frequently the rescaling happens, the more closely resource supply will match the demand. The topics that will be covered in this thesis include demand prediction, resource allocation based on prediction, as well as the economic issues related to cloud resource pricing for video providers. Our contributions can mainly be divided into the following four parts: Demand Forecast. Accurate demand forecast serves as the first step towards a flexible resource allocation scheme to replace traditional capacity over-provisioning. Targeting multimedia applications, we employ a set of established statistical learning and time series analysis techniques to automatically forecast demand and infer performance in large-scale VoD applications. Our work is backed up by 400 GB of operational traces collected from UUSee 2, a major commercial VoD provider based in China. Although many measurement studies of VoD systems exist in the literature, few practical workload 2 Although UUSee adopts a peer-assisted video streaming architecture, our collected data contains the download bandwidth of all online users sampled every 10 minutes, which reflects user bandwidth demands regardless of the service architecture adopted.

24 1.3. THESIS SCOPE 6 forecast mechanisms have been developed for these systems. We utilize the daily periodicity and trends inherent in historical workload data to predict the future population and bandwidth demand in video services, study the demand volatility, and use a factor model to reduce the dimension and complexity involved in statistical learning. Cloud Resource Allocation. Based on such demand forecast, we have developed resource allocation mechanisms for data centers in the cloud to satisfy video application requirements with the minimum cost. We study a new type of service where companies like Netflix can make reservations for (outgoing) bandwidth guarantees from the cloud. The objective is to reserve as little bandwidth as possible while still guaranteeing the video delivery quality. We have proposed an auto-scaling system that can adjust resource allocation for each cloud tenant (a video company or channel) to closely match its shortterm demand prediction, accommodating both the estimated demand expectation and unexpected demand variation. Borrowing insights from financial risk management, we have proposed a mean-variance optimization framework to statistically mix negatively correlated (anti-correlated) workloads to save the bandwidth reservation cost. When multiple data centers coexist in the cloud, we have derived the optimal direction policy of tenant demands to di erent data centers, proposed practical heuristic solutions and evaluated our methods through extensive trace-driven simulations. The Bandwidth Reservation Market. Based on demand forecast and the proposed workload management mechanisms, we further propose new pricing models for cloud bandwidth resources. The current usage-based cloud pricing or pay-as-you-go is insu cient for multimedia applications that require bandwidth guarantees. We study the pricing of cloud bandwidth reservations for this kind of tenant. Since tenants may lack expertise in demand estimation, we propose a new business model in which a tenant

25 1.3. THESIS SCOPE 7 just needs to specify a percentage of its demand, called the guaranteed portion, tobe satisfied with guaranteed service, while the cloud takes the responsibility to fulfill the guarantee. Leveraging abundant computing power and workload monitoring, the cloud can predict tenant demands and make corresponding reservations of actual bandwidth for the tenants. Under the assumptions that each tenant always chooses a guaranteed portion of 100% (i.e., each video provider as a cloud tenant always requires the cloud to guarantee all its end-user requests) and that pricing cannot a ect such choices, wehavemadethefollowing contributions. First, we characterize the unique Nash equilibrium in the bandwidth reservation market, where the price that each tenant is willing to pay is a function of its tra cburstinessanditsdemandcorrelationtotheaggregatemarketdemand. Second, when multiple cloud providers coexist, we propose a profitable cloud brokerage service similar to Groupon, which sells bandwidth guarantees to tenants and jointly reserves bandwidth from the clouds for the mixed demand while directing requests to di erent cloud providers. We analytically characterize the broker pricing region in which a profitdriven broker helps enhance cloud resource utilization and lower bandwidth prices. Third, we develop a proof-of-concept bandwidth trading system driven by real-world demand traces, incorporating demand estimation, workload management and pricing as the key components. Algorithmic Pricing for Cloud Resources. We further consider the case that atenant schoiceofguaranteedportionmaybea ected by the pricing policy and may not always be 100%. For example, a video provider may require the cloud to guarantee only 95% of all its end-user requests if such bandwidth guarantees are costly. In this model, our objective is to find the optimal bandwidth pricing policy to maximize the

26 1.4. THESIS ORGANIZATION 8 expected social welfare, that is the sum of the profit of all the entities in the system including the tenants and the cloud provider. The problem is related to network utility maximization (NUM) with a coupled objective function, which is traditionally solved through Lagrangian dual decomposition and gradient/subgradient methods. However, due to the large scale of our algorithmic pricing problem, the gradient method usually achieves slow convergence because of its over-sensitivity to stepsize choices. In order to e ciently solve such a large-scale distributed optimization problem within a small number of iterations, we propose a novel class of nonlinear distributed and iterative algorithms, which achieve faster convergence than gradient methods both in theory and in trace-driven simulations. Besides bandwidth, bu ering and jitter handling are also important in video flow management. With bu ering on the user side, more data blocks in a particular video flow can be prefetched and stored when bandwidth is abundant, so that the playback can rely on the bu er when bandwidth conditions degrade. For example, the applications (e.g., Netflix) that use HTML5, Microsoft Silverlight and HTTP-based streaming all perform bu ering. In this thesis, we do not consider these control mechanisms for individual video flows. Instead, we focus on satisfying all the end users of a video provider or a video channel as a whole in the long run instead of individual video flows. Nevertheless, any individual flow management and bu ering scheme can be combined with our cloud-based video workload management mechanism to achieve good video delivery performance. 1.4 Thesis Organization The remainder of this thesis is organized as follows. The related literature is reviewed in Chapter 2. We present e ective time-series methods for demand forecast in Chapter 3,

27 1.4. THESIS ORGANIZATION 9 and study the demand volatility estimation and the dimensionality reduction problem involved in the statistical learning in Chapter 4. In Chapter 5, we study the problem of cloud bandwidth allocation and reservation for video-on-demand applications based on prediction, together with the related load direction problem when multiple data centers coexist. We then study the bandwidth pricing issues in a broker-assisted cloud bandwidth reservation market in Chapter 6, under the assumption that pricing will not a ect tenant demands. In Chapter 7, we further study the algorithmic cloud bandwidth pricing problem when pricing can a ect tenant demands, and solve the underlying utility maximization problem with e cient new distributed algorithms.

28 Chapter 2 Related Work 2.1 Measurement, Prediction and Learning in VoD Systems Video-on-Demand systems have gained enormous popularity on today s Internet. Some examples of production systems include Netflix [6], Hulu [3], Justin.tv [4], Youtube [10], Metacafe [5], etc. Since the inception of using a mesh-like P2P architecture as a solution to live media streaming, exemplified in CoolStreaming [75], peer assistance has also been introduced into video-on-demand services to increase system scalability. Significant research e orts [35], [41], [42], [50], [52] have been devoted to the measurements of video streaming systems, with a focus on the user behavior, scalability and system performance. A number of coding and ad hoc optimization techniques [15], [31], [50] have also been proposed to enhance their performance. The importance of bandwidth demand estimation to capacity planning in Internet VoD systems has been recognized recently. It is shown that estimating time-varying 10

29 2.1. MEASUREMENT, PREDICTION AND LEARNING IN VOD SYSTEMS 11 demands in a large-scale IPTV network can help the system optimally place content on its geographically distributed servers [16]. Toward this goal, the recent demand history is used as an estimate of future demand in each video channel [16] and demands for previously released movies are used as the demand prediction for newly released movies. Apparently, this simple method does not yield accurate forecasts. Since measurements show that video workload demonstrates regular diurnal periodicity [16, 59, 70, 72], various techniques have recently been proposed to forecast large-scale VoD tra c. An Autoregressive Integrated Moving Average (ARIMA) model is introduced in [70] to predict the population evolution in video streaming systems. To account for diurnal e ects, seasonal ARIMA models have been introduced in our previous work [58, 59] to predict non-stationary demand evolution at a fine granularity. However, most existing forecast methods for video demand assume a constant forecast error variance, and thus provision a constant amount of resource cushion for quality assurance. In fact, our measurements of the UUSee system show that bandwidth demand is subject to rapid changes in some periods, while remaining tranquil and highly predictable in other periods. We therefore introduce Generalized Autoregressive Conditional Heteroskedastic (GARCH) models [24, 28, 33] from econometrics research to model the volatility persistence phenomenon the bandwidth demand at a certain time period tends to exhibit similar volatility as in recent time periods. Volatility reduction in the mixed tra c of multiple channels is similar to the idea of statistical multiplexing and resource overbooking [67] in shared hosting platforms, where the resources are booked to satisfy a certain percentile of demand in each application instead of its worst-case demand, so as to enhance resource utilization. However, the volatility reduction discussed here is novel in three aspects. First, we are concerned with

30 2.1. MEASUREMENT, PREDICTION AND LEARNING IN VOD SYSTEMS 12 forward-looking resource allocation and volatility forecasts for future demand, while in [67] the resource usage of each application is profiled in an o ine and fixed manner, ignoring the change of demand patterns over time. Second, our study focuses on large-scale VoD systems, where the concurrent number of users can ramp up by several hundreds or thousands in tens of minutes. In this scenario, any fixed resource usage profiling for small video channels (e.g., those with a user population of 20 in [67]) will be insu cient. Last but not least, we do not assume independence between the demands of channels. Instead, we accurately quantify the conditional demand variance in each channel, which enables the use of financial instruments such as hedging and diversification to achieve cost-e ective server management with service level guarantees. Principal Component Analysis (PCA) has been proposed in [40] to extract video demand evolution patterns over longer periods (of weeks or months) and forecast coarsegrained daily populations. In this thesis, we utilize an approach to find the common factors that drive the demand evolution of all coexisting video channels using PCA at afinegranularity. Wethenmakeforecastsforindividualvideochannelsasregressions from factor forecasts obtained from the seasonal ARIMA model. Such an approach combines the strengths of both PCA and the seasonal ARIMA model. Unlike [40], our approach makes short-term predictions with a lead time of 10 minutes, enabling finegrained autoscaling of resource allocation. Recently, there has been an increasing interest in applying statistical learning tools to diagnose, predict and improve the reliability of large-scale distributed systems. Bahl et al. [18] propose Sherlock, which discovers the dependencies among services, software and physical components in a large enterprise network based on network tra c, and leverages such dependencies to detect and localize problems. Chen et al. [30] further

31 2.2. CLOUD WORKLOAD MANAGEMENT AND RESOURCE ALLOCATION 13 study the performance and limitations of automated dependency discovery from network tra c, and propose Orion that discovers dependencies in an enterprise network, using the delay information between every pair of network flows. Mahimkar et al. [53] focus on characterizing performance issues in a large-scale IPTV network, and propose Giza, which applies multi-resolution data analysis to localize regions and physical components in the IPTV distribution hierarchy that are experiencing performance problems. This thesis represents one of the first attempts to characterize the dependencies among the demand statistics and performance metrics in a large-scale video delivery system. We introduce a systematic time-series analysis approach that learns the user behavior and system dynamics from online measurements, and utilizes the learned rules to predict demand and performance for the future. 2.2 Cloud Workload Management and Resource Allocation Cloud bandwidth reservation is becoming technically feasible. There have been proposals on data center tra cengineeringtoo er elastic bandwidth guarantees for egress tra c from virtual machines (VMs) [39]. The idea of virtual networks has also been proposed to connect the VMs of the same tenant in a virtual network with bandwidth guarantees [19, 39]. Further, explicit rate control has been proposed to apportion bandwidth according to flow deadlines [69]. Such research progress has made the cloud more attractive to bandwidth-intensive applications such as video-on-demand and MapReduce computations that rely on the network to transfer large amounts of data at high rates [73]. Netflix, as a major VoD provider, moved its data store, video encoding, and streaming

32 2.2. CLOUD WORKLOAD MANAGEMENT AND RESOURCE ALLOCATION 14 servers to Amazon AWS [2] in 2010 [12]. Virtualization techniques for supporting cloud-based IPTV services are also being developed by major U.S. VoD providers such as AT&T [14]. Novel flow control algorithms and improvements over TCP have been presented in [34] to ensure QoE-aware video delivery from cloud data centers. Furthermore, video demand forecasting techniques have been proposed, such as the non-stationary time series models introduced in [58, 59, 60], and video access pattern extraction via principal component analysis in [40]. All these recent advances will help the realization of video delivery from data centers, and will enable e cient and quality-assured management of video workload in the cloud. Predictive and dynamic resource provisioning has been proposed mostly for VMs and web applications with respect to CPU utilization [23,?, 36, 37, 66] and power consumption [46, 48]. VM consolidation with dynamic bandwidth demand has also been considered in [68]. Our work exploits the unique characteristics of VoD bandwidth demands and is distinct from the foregoing works in three aspects. First, our bandwidth workload consolidation is as simple as solving convex optimization for a load direction matrix. We leverage the fact that unlike VM, demand of a VoD channel can be fractionally split into video requests. Second, our system forecasts not only the expected demand but also the demand volatility, and thus can control the risk factors more accurately. In contrast, most previous works [37, 36] assume a constant demand variance. Third, we exploit the statistical correlation between bandwidth demands of di erent video channels to save resource reservation while previous work, such as [68], considers VM consolidation with independent random bandwidth demands. The idea of statistical multiplexing and resource overbooking has been empirically evaluated for a shared hosting platform in [67]. Our novelty is that we formulate the

33 2.3. CLOUD RESOURCE PRICING 15 quality-assured resource minimization problem using Value at Risk (VaR), a useful risk measure in financial asset management [54], with the aid of accurate demand correlation forecasts. We believe that our theoretically grounded approach provides stronger robustness against the demand volatility in practice. 2.3 Cloud Resource Pricing Cloud computing, e.g., Amazon EC2, is usually o ered with usage-based pricing (payas-you-go) [17, 38]. In addition, cloud resources can also be sold with auction type of pricing policies in a spot instance market [64, 74], to reflect the underlying demandsupply relationship in the computing instance market. Di erent from pay-as-you-go, resource reservation involves paying a negotiated cost to have the resource over a time period, whether or not the resource is used. Although suitable for delay-insensitive applications, pay-as-you-go is insu cient as a business model for bandwidth-intensive and quality-stringent applications like VoD, since no performance guarantees are provided in general. To support guaranteed cloud services, we need new policies to price not only the bandwidth usage but also bandwidth reservations. Cloud brokers, e.g., Zimory [11], have recently emerged as intermediaries connecting buyers and sellers of computing resources. The engineering aspects of using brokerage to interconnect clouds into a global cloud market have been discussed in [29]. We propose a new type of cloud brokerage that multiplexes bandwidth reservations to save cost while providing quality guarantees to customers. Amazon Cluster Compute (as of 2011) [1] allows tenants to reserve, at a high cost, adedicated10gbpsnetworkwithnomultiplexing. Insteadofover-provisioningafixed amount of capacity, our proposed broker dynamically books resources in response to

34 2.4. ALGORITHMS FOR DISTRIBUTED AND PARALLEL OPTIMIZATION 16 demand changes, exempting tenants from demand estimation, for which they have no expertise. We aim to find the pricing policies under which a selfish broker can also enhance cloud resource e ciency and save money for VoD providers. Di erent from usage-based pricing [17], our proposed pricing policy depends on demand statistics such as burstiness and correlation. Our bandwidth reservation pricing model is partly inspired by pricing electric power consumption and capacity reservation under demand uncertainty [62]. However, due to the computing capability and abundant workload data in the cloud, our bandwidth reservation pricing theory is essentially a distributed computational problem based on prediction. 2.4 Algorithms for Distributed and Parallel Optimization The optimal cloud pricing problem presented in Chapter 7 is related to network utility maximization (NUM), which has been extensively studied in the past [32, 61, 49]. A number of primal/dual decomposition methods have been proposed to solve such problems in a distributed way. Most NUM problems in the literature are concerned with uncoupled utilities, where the local variables of one network element do not a ect the utilities of other network elements. Unfortunately, this does not apply to systems with competition or cooperation, where utilities are often coupled, such as in our cloud resource reservation model in Chapter 7. Traditional distributed solutions to NUM problems with coupled objectives seek decomposition from Lagrangian dual problems with auxiliary variables, which are then

35 2.4. ALGORITHMS FOR DISTRIBUTED AND PARALLEL OPTIMIZATION 17 solved with gradient/subgradient methods. This approach has a simple economic interpretation of consistency pricing [61, 65]. However, gradient methods are sensitive to the choice of stepsizes, leading to slow and unstable convergence. Furthermore, existing faster numerical convex problem solvers, e.g., Newton s method [20, 27], can not be applied in a distributed way, since they involve a lot of message-passing beyond gradient information, which is hard to implement and justify physically in reality. In Chapter 7, we propose a new class of distributed algorithms to solve NUM problems with coupled objective functions in general. The new methods are e cient in terms of convergence performance without complicating the message-passing that is involved. The nonlinear Jacobi algorithm [21] solves convex optimization problems by optimizing the objective function for each variable in parallel while holding other variables unchanged, and converges under certain contraction conditions. Yet it cannot be applied to our pricing problem, since no network element has global information of all the utilities. However, we will show in Chapter 7 that the nonlinear Jacobi algorithm is a limiting case of our Algorithm 1 proposed in Sec. 7.5 (executed asynchronously), while our proposed Algorithm 1 can be understood as a distributed version of the non-linear Jacobi algorithm.

36 Chapter 3 Forecast in Video-on-Demand Systems Automated demand and performance prediction can help video services instantaneously estimate their operational cost, avoid performance issues, place servers optimally and allocate resources dynamically. In this chapter, we investigate the feasibility of automated forecasting in on-demand video streaming systems, by analyzing the operational traces that we have collected from UUSee Inc., one of the leading media content providers in China, during the 2008 Summer Olympic Games. The first step towards performance prediction is to understand the evolution of demand, which is not only a deciding factor for the resource consumption, but also has strong links with various system dynamics. From the real-world traces, we discover that the demand evolution for a video channel is highly predictable even in the long term as users exhibit diurnal behavior and regular habits that can be learned from measurements. To model the diurnal periodicity and trend in the demand evolution, we use seasonal Autoregressive Integrated Moving Average (ARIMA) models [25], which can 18

37 CHAPTER 3. FORECAST IN VIDEO-ON-DEMAND SYSTEMS 19 yield excellent prediction results for non-stationary demand evolution. Surprisingly, not only is the demand of a channel predictable based on the autocorrelation with its history, videos with similar properties, e.g., popular Olympics videos released around the same time of day, also demonstrate similar demand evolution patterns. This enables the inference of the initial demand of a newly released channel based on the statistics of similar channels that were released earlier. We propose a generative model based on the regression of mixtures of Gaussians to account for such a phenomenon. When it comes to the estimation of the demand for server bandwidth and performance at end users, the peer-assisted architecture adopted by many VoD services, including UUSee, has posed great challenges to such tasks. Unlike a client-server based system, apeer-assistedservicepartlyreliesontheuploadsfromendusers,whicharehighly dynamic, volatile and hard to predict. We adopt a general class of algorithms that apply non-linear transformations to the performance-related time series, and make predictions leveraging both the serial dependencies within each series and the interdependencies among di erent series. We also demonstrate the application of our time-series modeling techniques in the prediction of server bandwidth requirement and the estimation of peer playback performance at system run-time. In this chapter, we conduct extensive evaluation of the proposed schemes based on the traces of 40 channels (with simultaneously online peer population up to 8000) in a 21-day period spanning the entire 2008 Beijing Olympics, corroborating the feasibility of implementing forecasting functions in real systems.