Traffic-Aware Resource Provisioning for Distributed Clouds

Transcription

1 ENERGY EFFICIENCY Traffic-Aware Resource Provisioning for Distributed Clouds Dan Xu, AT&T Labs Xin Liu, University of California, Davis Athanasios V. Vasilakos, Lulea University of Technology, Sweden Examining important cloud traffic characteristics and optimizations produced fine-grained traffic-awareness approaches that can more efficiently reduce energy costs for distributed clouds with dynamic, diverse traffic. loud-computing-based traffic has been rapidly growing in recent years. Cisco forecasted that annual global datacenter IP traffic will reach 7.7 zettabytes by the end of 2017, with its cloud IP traffic reaching 5.3 zettabytes. 1 Correspondingly, the service providers, including Google, Microsoft, Facebook, and AT&T, are building and expanding their datacenters nationwide and worldwide. Such geographically distributed datacenters are often referred to as Internet datacenters (IDCs) and we use cloud as a general term that refers to an IDC s collection of hardware, software, and services. An IDC typically consumes many megawatts of power, which imposes a significant electricity cost to its operator. For example, Google s datacenters consume nearly 300 million watts annually. 2 To reduce energy costs, researchers have proposed load-aware server provisioning schemes, in which the number of active servers is controlled dynamically based on the load. 3 6 When the load is low, extra servers can be scheduled in sleeping mode. In this paradigm, obtaining traffic volume information is a challenging issue. As the Related Work in Resource Provisioning sidebar describes, many researchers have worked on traffic-aware cloud resource provisioning. However, considerable room for achieving a fine-grained traffic-awareness remains. In the load-aware server provisioning schemes, 4 7 researchers typically consider traffic dynamics in a large time scale only, such as tens of minutes, during which traffic demand (that is, input to the server provisioning algorithms) is usually assumed static given the current time interval for server provisioning. However, as we observed from a real 30 IEEE CLOUD COMPUTING PUBLISHED BY THE IEEE COMPUTER SOCIETY /15/$ IEEE

2 RELATED WORK IN RESOURCE PROVISIONING any researchers have explored traffic-aware cloud resource provisioning. Common approaches include prediction- or estimation-based approaches. For example, Gong Chen and his colleagues propose statistical models to forecast Microsoft servers load of connection-intensive requests (such as messenger login in), and design server provisioning schemes accordingly. 1 Minghong Lin and his colleagues propose basing their online server provisioning design on an estimate of the load size at the beginning of each time interval. 2 Other work gives workload as the inputs of the resource provisioning algorithms. 3,4 In those schemes, server provisioning is often jointly designed with load dispatching that is, by determining the load at each server. To guarantee service quality, some work explicitly considers a QoS constraint, such as a delay. 3 References 1. G. Chen et al., Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services, Proc. 5th USENIX Symp. Networked Systems Design and Implementation (NSDI 08), 2008, pp M. Lin et al., Dynamic Right-Sizing for Power-Proportional Data Centers, IEEE/ACM Trans. Networking, vol. 21, no. 5, 2013, pp L. Rao et al., Minimizing Electricity Cost: Optimization of Distributed Internet Data Centers in a Multi-Electricity-Market Environment, Proc. IEEE INFOCOM, 2010, pp J. Tu et al., Dynamic Provisioning in Next-Generation Data Centers with On-Site Power Production, Proc. 4th Int l Conf. Future Energy Systems (e-energy 13), 2013, pp datacenter traffic trace and as Theophilus Benson and his colleagues indicated 8 cloud traffic demand also varies over a small time scale, such as hundreds of milliseconds, and often shows burstiness. Here, we ll discuss the key challenges caused by traffic dynamics in a small time scale, such as server overloading. In considering traffic dynamics in both large and small time scales, we propose a framework of solutions that can guarantee service quality with a low cost for distributed clouds. Our solutions leverage joint server provisioning (dam building) for large time scales and cross-idc load shifting (water balancing) for small time scales, which are similar to water flood control. We discuss how cross-idc load shifting can be designed to efficiently and costeffectively alleviate traffic dynamic issues in a small time scale. Like existing work, 6,9 our cross-idc load-shifting schemes can leverage electricity price diversity to reduce energy costs. To achieve a desirable balance between costs and service quality, it s also important to exploit the heterogeneity of cloud jobs. Cloud jobs can be classified by two general categories: delay sensitive jobs (DSJs), such as interactive jobs, and delay tolerant jobs (DTJs), such as batch jobs and analytical jobs. Most existing work considers only interactive jobs. 4,6,7 Although some work considers both interactive and batch jobs, 10 joint resource provisioning for both DSJs and DTJs hasn t been studied in-depth. In this article, we further examine cloud traffic diversity and consider joint DSJ and DTJ provisioning. Cloud Traffic Dynamics and Diversity In this section, we examine traffic dynamics and diversity of clouds. We first show the traffic dynamics of an IDC in different time scales. Traffic Dynamics and Time Scales User request traffic at an IDC varies over time. For example, research shows that the login rate of Windows Live Messenger changes significantly at different hours. 4 Dynamic traffic leads to a time-varying load demand. The current traffic or load-aware server provisioning schemes often use load prediction or estimation methods to capture the variation. However, those schemes address only large-time-scale traffic variation, such as predicting the future load approximately every half hour. 4 Small-time-scale traffic variation is rarely considered. JANUARY/FEBRUARY 2015 IEEE CLOUD COMPUTING 31

3 ENERGY EFFICIENCY Traffiic size (Mbit) Time granularity:1 sec Time (in seconds of a three-minute snapshot) FIGURE 1. Datacenter traffic burstiness. Traffic trace is collected from the datacenter of a major US service provider. We analyze the traffic of an IDC operated by a major US service provider. We extract the traffic information from the IDC s Hadoop Distributed File System (HDFS) log and plot a one-second traffic cap size for a three-minute snapshot (see Figure 1). As the figure shows, the one-second traffic exhibits an obvious on/off pattern. Other research shows the on/off traffic pattern for a Microsoft datacenter, 8 in a time granularity of 15 and 100 milliseconds, respectively. Small-time-scale traffic burstiness is difficult to address for two reasons. First, it s hard to predict traffic variation in a small time scale. Second, it s impossible to perform dynamic server provisioning according to the small-time-scale traffic variation due to the large time delay in turning on servers. Thus, server provisioning can be performed only in a large time scale, such as every tens of minutes to hours. Traffic burstiness might incur both congestion and server idleness at different times. Sometimes, traffic spikes impose a load that s larger than the available capacity. This overloading degrades service performance, such as by incurring a large service delay. It shouldn t happen frequently because most Internet requests, such as search and Web browsing, tolerate only a small time latency. When traffic goes lower that is, it goes into off cycles servers become idle and power is wasted. To address these issues, we view the IDCs as rivers or lakes and overloading as flooding. We design joint server provisioning and cross-idc load shifting, with server provisioning performed in a large time scale and considered as dam building. In a small time scale, cross-idc load shifting is performed to reduce traffic burstiness such that a smaller number of servers (a lower dam) is needed at each IDC. Cross-IDC load shifting is like water balancing among rivers, lakes, and reservoirs. We discuss efficient load-shifting policies that can reduce total energy costs for distributed clouds. The traffic we consider here mainly refers to interactive jobs. In a cloud, many other types of jobs might require provisioning in different ways, as we now describe. Cloud Traffic Diversity As we noted earlier, IDC traffic comes from both DSJs and DTJs. Resource provisioning to DTJs should be different from DSJs. DSJs have a small service delay and thus must be served promptly. In contrast, DTJs can be buffered in queues. DSJs have a higher service priority than DTJs, so capacity resource for DSJs must be guaranteed first. The remaining server capacity, which varies instantaneously due to dynamic DSJ traffic demand, can then serve DTJs via dynamic speed scaling. This paradigm is called valley filling that is, using DTJs to fill the DSJ load-demand valley (see Figure 2). Our discussion of interactive job traffic refers to DSJ traffic. When traffic is low, some servers become idle. In this case, they can be allocated to process DTJs rather than shut down. The cross-idc load shifting for DSJs also fills the valley of some IDCs. But this load shifting doesn t cause valleys to disappear; there s still some remaining capacity to utilize. DTJs can also be shifted from one IDC to another, such as to exploit more efficient serv- 32 IEEE CLOUD COMPUTING

4 Total active capacity Load demand of DSJs DTJs (filling the valley) Overloading Overloading Overloading Overloading Overloading FIGURE 2. Delay sensitive jobs (DJBs) overload and fill valleys by delay tolerant jobs (DTJs). The DSJs load demand exceeds total available capacity at four time intervals. If the measurement period starts from the first interval and lasts to the last interval (that is, the period is 34 time intervals), the overloading probability is calculated as 4/34 =11.7 percent. Time ers or a lower electricity price in another location. Also, DSJs or DTJs can be further divided by different classes according to job types and service requirements. For example, search requests and Web browsing requests can be considered as two different DSJ classes, whereas user information backup and weblog analysis can be two different DTJ classes. Building Dams and Balancing Water Using water and damming as metaphors for IDC resource provisioning applies to DTJ provisioning, which is like filling a river valley: the water (of DSJs) is below the height of the dam (an IDC s total active server capacity). Efficient valley filling is challenging, however. On one hand, DTJ load demand is high, which often incurs a large energy cost. On the other hand, DTJs also desire as small a delay as possible. Moreover, valley filling must be jointly designed with DSJ provisioning. Here, we discuss joint server provisioning, cross-idc load shifting, and valley filling for distributed clouds. Our design goal is to minimize the total costs, including energy costs and load-shifting costs, while satisfying DSJ QoS constraints and assuring a desirable delay performance for DTJs. Given the problem s complexity, we first focus on DSJs only, discussing joint server provisioning and cross-idc load shifting; we then address how valley filling can jointly work with server provisioning and load shifting. Measuring Server Overloading As discussed earlier, the small-time-scale DSJ traffic burstiness can lead to overloading that is, a situation in which load demand exceeds the capacity provisioned. To measure overloading, we introduce a QoS metric called overloading probability, which is defined as the probability that the DSJ load demand is larger than the available server capacity. As Figure 2 shows, IDC overloading can be temporally monitored. We can set a small time interval, such as hundreds of milliseconds, to track overloading and then set a measurement period, such as hours, to calculate overloading probability. We can monitor overloading probability by first counting the number of time intervals with overloading, and then dividing them by the total number of small time intervals during the measurement period. In practice, we can use a slide window of measurement periods to update an IDC s overloading probability. We can directly translate overloading probability into a DSJ s QoS requirements (that is, its service latency requirements) because the DSJ load demand is implicitly based on a given service latency requirement. A smaller service latency requirement leads to a larger load demand given the same amount of traffic. We can consider overloading probability as the percentage of time that the service latency requirement can t be guaranteed. A small overloading probability requirement is desirable for guaranteeing DSJ service quality. To reduce the overloading probability, we can increase the available capacity by provisioning more active servers. This is like building a higher dam to prevent flooding. However, turning on many additional active servers isn t cost effective. To address this issue, traffic burstiness at each IDC must be reduced first so a lower capacity can meet the same overloading probability requirement. Cross-IDC Load Shifting To reduce traffic burstiness, we designed cross-idc load-shifting schemes to leverage traffic s statistical multiplexing gain. Although existing work has considered cross-idc load shifting for exploiting electricity price diversity, 9 how to use load shifting JANUARY/FEBRUARY 2015 IEEE CLOUD COMPUTING 33

5 ENERGY EFFICIENCY to efficiently reduce traffic burstiness and further achieve cost effectiveness hasn t been investigated. By load shifting, an IDC s excessive traffic can be processed by another IDC that has lower traffic demand. Thus, load demand at each IDC is smoothed. Here, we consider different policies for load shifting. First, we consider a policy in which each IDC instantaneously shifts its traffic to another one according to a fixed ratio. Consider D i as the traffic volume of IDC i, which is a time-varying random variable. IDC i shifts its traffic to IDC j according to the current D i and a fixed ratio r ij (that is, r ij D i ). Any pair of IDCs can shift traffic to each other. Thus, we call this scheme ratio-based load swapping. In RBLS, traffic shifting ratio r ij is a control variable to optimize. We must consider many factors when optimizing r ij. For example, if IDC j has a large maximum server capacity, r ij is likely to be large. If the current electricity price at IDC j is low, r ij might also be large to let IDC i shift more traffic to IDC j. Thus, RBLS can exploit various factors to reduce energy costs and explore server capacity more efficiently; nevertheless, it also uses statistical multiplexing gain to smooth traffic at each IDC. However, RBLS can be expensive to implement because the load-shifting numbers can be large that is, O(N 2 ), where N is an operator s number of IDCs. We thus consider offloading polices, in which each IDC shifts traffic only to a selected offloader. In offloading, each IDC can shift traffic based on a fixed ratio that is, ratio-based offloading. In RBO, distribution of the remaining traffic demand at each IDC still follows the same (scaled-down) distribution as the original load demand. Alternatively, each IDC can shift the load beyond the available capacity to the offloader; this is called threshold-based offloading. Intuitively, TBO is more efficient than RBO in terms of decreasing overloading probability and saving energy costs. This is because, in RBO, each IDC s load demand is still random. A large server capacity is still needed to constrain server overloading, especially when traffic demand at each IDC is heavy-tail distributed. Although in TBO, the excessive traffic of all IDCs is aggregated at the offloader, we can expect a higher statistical multiplexing gain. TBO is similar to discharging flood from rivers to a reservoir that is, only the amount of water (traffic) that is beyond the dam (maximum active server) capacity is offloaded. Server Provisioning and Load Shifting Cross-IDC load shifting is performed in a small time scale to capture the small time scale s traffic variation. In TBO, configuring load shifting such as determining traffic-splitting ratios in RBLS or determining each IDC s number of servers (that is, the threshold for offloading is coupled with server provisioning, which is performed in a large time scale. For a relatively large time interval, load shifting follows the same configuration. The joint server provisioning and load-shifting schemes need each IDC s traffic information. To calculate overloading probability, we need the DSJ traffic s mean and variance (before load shifting). Assuming a certain distribution of the DSJ traffic, we can calculate the overloading probability based on the DSJ traffic s mean and variance. We require an overloading probability constraint that is, it must be smaller than a predefined value to guarantee service quality. Our objective is to minimize all IDCs costs, including energy costs and load-shifting costs. An IDC s energy cost is the product of the electricity price and the total power consumption, which is proportional to that the servers consume. Further, server power consumption can be modeled by an increasing function of both the number of active servers and the average load. Problem 1 Joint sever provisioning and load shifting for DSJs can be formulated by the following optimization problem (Problem 1). The main inputs for Problem 1 are each IDC s traffic statistics (mean and variance). The objective is to minimize the total energy and load-shifting costs. Problem 1 has three constraints: An overloading probability constraint at each IDC. A server allocation constraint at each IDC (the number of active servers at each IDC is upper limited by the total number of servers). A load-shifting constraint between two IDCs (the traffic amount shifted is bounded by bandwidth between two IDCs). Problem 1 s outputs are the number of active servers at each IDC and the parameters of traffic shifting. Although DSJ traffic is dynamic, Problem 1 is a deterministic optimization problem based on traffic statistics information that remains static for a large time interval. The variation of traffic statistics rightly represents the large-time-scale traffic dynamics we discussed earlier. Thus, Problem 1 captures traffic dynamics in both the large and small time scales. We formulated Problem 1 with different cross- IDC load-shifting schemes RBLS, RBO, and TBO using sophisticated convex optimization models. 11,12 Figure 3 compares the performance of 34 IEEE CLOUD COMPUTING

6 Total cost ($) 1,600 1,400 1,200 1, Joint server provisioning with RBLS Joint server provisioning with RBO Joint server provisioning with TBO Server provisioning without cross-idc load shifting Overloading probability constraint at each IDC FIGURE 3. Simulation results of three joint server provisioning schemes based on convex optimization models: ratio-based load swapping (RBLS), ratio-based offloading (RBO), and threshold-based offloading (TBO). Costs at an Internet datacenter (IDC) are the product of its power consumption and electricity price. the different schemes, each of which jointly works with server provisioning. In the simulations, we consider 15 IDCs, each of which has a maximum of 100 servers. Each server s maximum CPU frequency is normalized as 1. Server capacity or load is defined as the total CPU frequency of all active servers. The number of active servers, constrained by 100, is a control variable to optimize. Load demand (before load shifting) at each IDC follows an exponential distribution, with a mean of 10. We randomly selected each IDC s electricity price from one to five. The figure s x-axis shows the overloading probability constraint at each IDC. Clearly, a smaller overloading probability constraint leads to more active servers by the proposed optimization model for each scheme, and thus a higher energy cost. In RBO and TBO, we selected one IDC as the offloader. As the figure shows, RBLS always leads to the smallest cost, because it can achieve traffic statistical multiplexing gain at every IDC. TBO is second. Although RBO performs the worst, it s still much better than no load shifting. Our results show that it s important to reduce traffic burstiness using an efficient cross-idc load-shifting scheme. In practice, with operation complexity considered, TBO is a desirable load-shifting scheme with dynamic DSJ traffic demand. Filling Valleys: Joint Resource Provisioning for DSJs and DTJs In terms of resource usage, DSJs have a higher priority than DTJs. When the current DSJ load demand is smaller than the available capacity, we provision the remaining capacity to DTJs, which is like filling a river valley. Beyond a Best-Effort Scheme Intuitively, it seems that valley filling provides besteffort provisioning to DTJs, because they can use only the capacity remaining. However, valley filling isn t simply best effort; DTJ provisioning must be carefully optimized together with DSJs. As we indicated earlier, DTJs contribute to energy costs and also require reasonable service latency. Thus, when performing server provisioning that is, when determining total capacity we can t consider only the DSJ overloading issue; we also must consider the DTJ performance delay. Otherwise, even DTJ queue stability can t be guaranteed. Thus, valley filling must be jointly configured with server provisioning in a large time scale to determine how much average capacity DTJs should receive. In a small time scale, following the configuration, instantaneous DTJ capacity allocation is performed according to current DSJ load demand. Figure 4 shows the joint server provisioning, cross-idc load shifting, and DSJ/DTJ capacity allocation scheme. Configuring Valley Filling As Figure 4 shows, without DTJs, server provisioning in Problem 1 is deterministic and performed independently for each (large) time interval. This is because the overloading probability constraint for DSJs must be satisfied within each large time interval. Otherwise, for some large intervals, DSJs will experience poor performance. With DTJs, joint server provisioning and valley-filling configuring aren t independent for different large time intervals. In this case, DTJ delay performance, such as queue stability, is measured over a long time period, which might cross many large time intervals. Also, DTJs JANUARY/FEBRUARY 2015 IEEE CLOUD COMPUTING 35

7 ENERGY EFFICIENCY Deterministic Independent of other time intervals Problem 1 Problem 2 Stochastic Correlated with other time intervals Configuring Cross-IDC load shifting Server provisioning Configuring valley filling Estimate DSJ traffic statistics Estimate DSJ traffic statistics Instantaneous load shifting Instantaneous valley filling Large time interval (scale) Small time interval (scale) FIGURE 4. Joint server provisioning, load shifting, and valley filling at different time scales. Without delay tolerant jobs (DTJs), server provisioning in Problem 1 is deterministic and performed independently for each (large) time interval. can be buffered in a long queue, whereas DSJs must be served in real time. Moreover, configuring valley filling must incorporate both load variation of DSJs and price temporal dynamics. For example, if the electricity price is potentially lower in the next large time interval, less capacity can be allocated to DTJs in the current large time interval. Historic information can also affect the current decision, such as by affecting the DTJ queue. Problem 2 We now consider Problem 2, a stochastic problem of joint server provisioning, load shifting, and valley filling, that we designed based on Problem 1. Here, we can shift the load of DTJs from one IDC to another. For DTJ load shifting, the purpose isn t to exploit statistical multiplexing gain, because DTJs don t have issues with small-scale traffic dynamics. For DTJs, traffic can be smoothed by buffering. DTJ load shifting can exploit a lower electricity price and a higher service rate in another IDC. Given their different purposes, the structures of DSJ and DTJ load shifting are also different. First, the objective of Problem 2 is to minimize the total costs, which also include energy costs contributed by DTJs. Unlike Problem 1, the total costs here aren t for each single (large) time interval, but rather the average costs over all large time intervals. Second, there are additional constraints. The sum of load demand of DSJs and DTJs must be smaller than the total available capacity, which is the capacity allocation constraint. To guarantee DTJ queue stability, the average queue length should be upper bounded. Or, alternatively, the average service rates for DTJs should be larger than their average arrival rates. We determine the service rate at each IDC for DTJs by the capacity allocated. Nevertheless, the overloading probability, server allocation, and load-shifting constraints also must be satisfied. Thus, Problem 1 s constraints constitute a subset of Problem 2 s constraints. Leveraging DTJ queue information. Problem 2 is difficult because DSJs and DTJs must be jointly considered, along with the traffic dynamics of DSJs and the stochastic properties of DTJ provisioning. We developed several different sophisticated algorithms that solve Problem 2. 13,14 Our proposed algorithms barely leverage system statistical information which is difficult to obtain in practice yet nonetheless guarantee the cost and delay bounds. One important difference among our schemes is whether DTJ queue backlog information at each IDC is leveraged or not; note that when a class of DTJs is shifted to another IDC, a sub-queue is created for that DTJ class in the destination IDC. We evaluate performance of a DTJ queue-based valley-filling scheme that leverages the current DTJ queue information to adjust capacity for DTJs. The DTJ queue-based scheme s basic idea is to achieve a tunable tradeoff between DTJ queue length and IDC energy costs. The scheme s objective function is to minimize the sum of two functions of the control variable vector that is, the server capacity assigned 36 IEEE CLOUD COMPUTING

8 Costs by service rate-based valley filling Costs by queue-based valley filling Delay by service rate-based valley filling Delay by queue-based valley filling Total cost ($) x 10, DTJ queue delay (mins) Percentage of DTJ load demand out of total load demand 0 FIGURE 5. Performance of valley filling that leverages queue information of delay tolerant jobs (DTJs). Here, five Internet datacenters (IDCs) are considered, with 15 classes of delay sensitive jobs (DSJs) and 10 classes of DTJs; load demand between the DSJs and DTJs is comparable. to each class of DTJs at each IDC. One function is increasing with the control variable vector, such as the energy cost function, which is weighted by a control parameter V. The other one is decreasing with the control variable vector, such as a weighted linear sum of server capacity for each DTJ class at each IDC, multiplied by 1, where the server capacity s weight for each class of DTJs at each IDC is the current queue length of the DTJ class. Clearly, when queue length of a DTJ class becomes large, the decreasing function has a large weight, and a high server capacity will be assigned to this DTJ class due to the minimization objective; this in turn reduces the queue length of the DTJ class. The control parameter V is to tune the tradeoff between energy costs and DTJ queue length. Setting a large value of V gives the increasing function a large weight, which leads to a small server capacity due to the objective of minimization, and thus reduces energy costs. When V is set small, the decreasing function has a large weight and thus a large server capacity will be assigned, which further reduces DTJ queue length. The scheme s constraints are the same as those in Problem 2. We compared the queue-based scheme to a service-rate based scheme that doesn t leverage any queue length information but does guarantee longterm queue stability. The service-rate based scheme almost achieves the minimum cost with DTJ queuestability assured. As Figure 5 shows, the queue-based scheme achieves a significantly smaller queue delay for a slightly larger cost. This result implies the importance of leveraging DTJ queue information in valley filling. Details of this study are available elsewhere. 13 The DTJ queue-based provisioning scheme intuits that capacity allocation for each class of DTJs at each IDC is correlated with the queue length of the class of DTJs. When a DTJ queue in an IDC is long, a larger server capacity is assigned to reduce queue length and vice versa. In practice, we can tune queue-based algorithms to achieve a desirable tradeoff between energy costs and DTJ queue delay. Coupling load shifting to capacity allocation. Intuitively, the classic stochastic optimization method back-pressure routing 15 can be a good candidate for DTJ load shifting among IDCs because it guarantees DTJ queue stability. Back-pressure routing s basic idea is to shift more of a DTJ class s traffic to an IDC, which has less traffic than the class of DTJs. However, we found that back-pressure routing isn t optimal in reducing energy costs in the distributed clouds scenario. This is because, in backpressure routing, the decision of cross-idc DTJ load shifting is mainly based on the difference between the amount of DTJ class traffic in both the original IDC and each destination IDC. Cross-IDC load shifting isn t directly correlated with capacity allocation in each IDC. Thus, back-pressure routing doesn t leverage electricity price location diversity well. To address this issue, we designed a cross-idc DTJ load-shifting scheme that s closely coupled to server capacity allocation at each IDC. The scheme s basic idea is that the amount of a DTJ class load JANUARY/FEBRUARY 2015 IEEE CLOUD COMPUTING 37

9 ENERGY EFFICIENCY Total cost ($) 95,000 90,000 85,000 80,000 75,000 Total costs (Dollar) DTJ queue delay (sec) 1,800 1,600 1,400 1,200 1, DTJ queue delay (sec) 70,000 Joint back-pressure routing based cross-idc DTJ load shifting and capacity allocations scheme Closely coupling cross-idc DTJ load shifting to capacity allocation 0 FIGURE 6. Simulation results comparing the cross-idc load-shifting scheme based on back-pressure routing with our scheme of closely coupling cross-idc delay tolerant job (DTJ) load shifting with capacity allocation. The simulations considered five IDCs and 10 classes of DTJs. The result shows the importance of closely coupling cross-idc DTJ load shifting with capacity allocation in cloud resource provisioning. that s shifted to another IDC depends only on how much server capacity in the destination IDC is assigned for the DTJ class. As Figure 6 shows, our scheme achieves a much smaller energy cost and a much smaller DTJ queue delay compared to the cross-idc DTJ load-shifting scheme based on backpressure routing. 15 any challenges and opportunities exist related to efficient cloud management: Traffic dynamics might need to be considered in different time granularities. Applications with different resource requirements such as CPU, I/O, disk, and memory must be better coordinated and provisioned. Multiplexing different applications by virtualization could further improve efficiency. Jobs with different QoS or service deadlines must be further prioritized. Accurately modeling server energy costs and load-shifting costs is challenging. We will consider all of these issues in our future work. References 1. Cisco Global Cloud Index: Forecast and Methodology, , white paper, Cisco Systems, J. Glanz, Power, Pollution and the Internet, New York Times, 22 Sept. 2012; -vast-amounts-of-energy-belying-industry-image.html. 3. Y.Y. Chen et al., Managing Server Energy and Operational Costs in Hosting Centers, Proc. ACM SIGMETRICS Int l Conf. Measurement and Modeling of Computer Systems, 2005, pp G. Chen et al., Energy-Aware Server Provisioning and Load Dispatching for Connection- Intensive Internet Services, Proc. 5th USENIX Symp. Networked Systems Design and Implementation (NSDI 08), 2008, pp M. Lin et al., Dynamic Right-Sizing for Power- Proportional Data Centers, IEEE/ACM Trans. Networking, vol. 21, no. 5, 2013, pp L. Rao et al., Minimizing Electricity Cost: Optimization of Distributed Internet Data Centers in a Multi-Electricity-Market Environment, Proc. IEEE INFOCOM, 2010, pp J. Tu et al., Dynamic Provisioning in Next- Generation Data Centers with On-Site Power Production, Proc. 4th Int l Conf. Future Energy Systems (e-energy 13), 2013, pp T. Benson et al., Understanding Data Center Traffic Characteristics, ACM SIGCOMM Computer Comm. Rev., vol. 40, no. 1, 2010, pp R. Stanojevic and R. Shorten, Distributed Dynamic Speed Scaling, Proc. IEEE INFOCOM, 2010, pp H. Xu and B. Li, Temperature Aware Workload Management in Geo-Distributed Datacenters, 38 IEEE CLOUD COMPUTING

10 IEEE Trans. Parallel and Distributed Systems, preprint, doi: /TPDS D. Xu, X. Liu, and B. Fan, Minimizing Energy Cost for Internet-Scale Datacenters with Dynamic Traffic, Proc. IEEE 19th Int l Workshop on Quality of Service (IWQoS), 2011, pp D. Xu, X. Liu, and B. Fan, Efficient Server Provisioning and Offloading Policies for Internet Datacenters with Dynamic Load Demand, IEEE Trans. Computers, vol. 64, no. 3, pp , Mar. 2014; doi: /tc D. Xu and X. Liu, Geographic Trough Filling for Internet Datacenters, Proc. IEEE INFOCOM, 2012, pp L. Georgiadis, M.J. Neely, and L. Tassiulas, Resource Allocation and Cross-Layer Control in Wireless Networks, Foundations and Trends in Networking, vol. 1, no. 1, pp , D. Xu, X. Liu, and Z. Niu, Joint Resource Provisioning for Internet Datacenters with Diverse and Dynamic Traffic, IEEE Trans. Cloud Computing, preprint, doi: /tcc DAN XU is a senior member of technical staff at AT&T Labs in California. His research interests include datacenter network resource provisioning and energy efficiency design, along with 3G/4G mobile networks performance modeling and analysis. Xu has a PhD in computer science from the University of California, Davis. Contact him at danxu@ucdavis.edu. XIN LIU is a professor in the Computer Science Department at the University of California, Davis. Her research interests include wireless communication networks. Liu has a PhD in electrical engineering from Purdue University. She s a Chancellor s Fellow and a member of IEEE. Contact her at liu@cs.ucdavis.edu. ATHANASIOS V. VASILAKOS is a professor in the Department of Computer Science, Electrical, and Space Engineering at the Lulea University of Technology, Sweden. His research interests include cloud computing, cyber physical systems, and green computing/ networking. He s a senior member of IEEE. Contact him at th.vasilako@gmail.com. ADVERTISER INFORMATION Advertising Personnel Marian Anderson: Sr. Advertising Coordinator manderson@computer.org Phone: Fax: Sandy Brown: Sr. Business Development Mgr. sbrown@computer.org Phone: Fax: Advertising Sales Representatives (display) Central, Northwest, Far East: Eric Kincaid e.kincaid@computer.org Phone: Fax: Northeast, Midwest, Europe, Middle East: Ann & David Schissler a.schissler@computer.org, d.schissler@computer.org Phone: Fax: Southwest, California: Mike Hughes mikehughes@computer.org Phone: Southeast: Heather Buonadies h.buonadies@computer.org Phone: Fax: Advertising Sales Representatives (Classified Line) Heather Buonadies h.buonadies@computer.org Phone: Fax: Advertising Sales Representatives (Jobs Board) Heather Buonadies h.buonadies@computer.org Phone: Fax: JANUARY/FEBRUARY 2015 IEEE CLOUD COMPUTING 39