How To Plan A Cloud Infrastructure

Transcription

1 Concrrent Placement, Capacity Provisioning, and Reqest Flow Control for a Distribted Clod Infrastrctre Shang Chen, Yanzhi Wang, Massod Pedram Department of Electrical Engineering University of Sothern California Los Angeles, USA {shangc, yanzhiwa, pedram}@sc.ed Abstract Clod compting and storage have attracted a lot of attention de to the ever increasing demand for reliable and cost-effective access to vast resorces and services available on the Internet. Clod services are typically hosted in a set of geographically distribted data centers, which we will call the clod infrastrctre. To minimize the total cost of ownership of this clod infrastrctre (which acconts for both the pfront capital cost and the operational cost of the infrastrctre resorces), the infrastrctre owners/operators mst do a carefl planning of data center locations in the targeted service area (for example the US territories), data center capacity provisioning (i.e., the total CPU cycles per second that can be provided in each data center). In addition, they mst have flow control policies that will distribte the incoming ser reqests to the available resorces in the clod infrastrctre. This paper presents an approach for solving the nified problem of data center placement and provisioning, and reqest flow control in one shot. The soltion techniqe is based on mathematical programming. Experimental reslts, sing Google clster data and placement/provisioning of p to eight data center sites demonstrate the cost savings of the proposed problem formlation and soltion approach. I. INTRODUCTION Clod compting has been envisioned as the nextgeneration compting paradigm for its major advantages in on-demand self-service, biqitos network access, location independent resorce pooling, and transference of risk [1]. In a clod compting system, comptation workloads are transferred from the sers to the clod, in which they are processed by data centers owned by the clod service providers. These data centers are sally geographically distribted and may be mirror to each other for latency and reliability concerns. Typically, a data center occpies tens of thosands of sqare meters of land space, and consists of hndreds of thosands of compter servers, along with power delivery infrastrctres, internal networking connections, and a cooling system. An overview of the architectre of a clod compting system can be fond in [2]. Given the specifications of data centers in a clod infrastrctre, the problem of reqest flow control and resorce allocation has been well stdied in a series of prior work. Athors of [3] developed a toolkit called ClodSim to simlate the clod compting system and to evalate the resorce provisioning algorithm. Varios models are sed to analyze the behavior in a clod compting system and to /DATE14/ c 2014 EDAA develop effective management schemes [4], [5]. On top of this resorce allocation problem, there is the design problem of the placement and capacity provisioning of the data centers. Undersizing of data centers reslts in an increase of response time, degradation in qality of service, and finally loss of market for the service provider. And oversizing of the data centers increases the capital and operational cost with little practical se. When placing data centers, every major aspect of the capital and operational cost shold be considered careflly becase it is sally location dependent. For instance, Microsoft s data center in Qincy, Washington consmes 48MW of electricity power [6]. The electricity cost of this data center will be arond 1.4 million dollars per month. However, to operate a data center with the same power consmption in Los Angeles, California, the electricity bill will increase to over 4 million dollars per month. Some prior work have addressed the placement and/or the capacity provisioning problem of data centers. For instance, in [7], the athors identified the main costs in data centers and stated that their placement and provisioning can have a significant impact on service profitability. The athors of [8] provided a generalized cost calclation method and considered the availability of the clod service. In [9], the athors addressed the problem of green compting and had a discssion on the tradeoff between the carbon footprint, the average cost, and the latency when different locations are selected to bild new data centers. While it is obvios that different placement and capacity provisioning schemes of the data centers reslt in different reqest flow control and resorce allocation schemes, the reqest flow control and resorce allocation algorithm can in trn affect how a placement and capacity provisioning performs. De to this interdependency, any placement and capacity decision of the data centers shold also be made based on realistic reqest roting and resorce allocation. Unfortnately, the prior work has only applied very simple roting models and barely considered the problem of resorce allocation inside a data center. In this paper, we propose a generalized joint optimization framework of both data center placement/capacity provisioning and reqest flow control/resorce allocation. The objective is to minimize the total average cost of the data centers in the network provided an average delay constraint. The reqest generation behavior of the clod sers is smmarized based on the Google clster data [10]. The problem of resorce

2 allocation is formlated sing the Generalized Processor Sharing (GPS) model [11], [12], in which the processor allocates a certain amont of its total comptation resorces to each reqest rnning on it. And the response time of a reqest is calclated accordingly. In the proposed model, we accont for all the major aspects of the capital and operational cost of a data center, which can be time dependent and/or location dependent. In addition to bilding new data centers, we also allow the option of pgrading existing data centers. We provide an example based on the geographical information of the United States, and the optimal offline soltion is presented. The rest of the paper is organized as follows: In Section II, we introdce the system model we se. The problem formlation and the soltion framework are presented in Section III. Section IV shows the experimental reslts. And the last section is the conclsion. II. SYSTEM MODEL A. User behavior Let U denote the set of clod sers. Please note that when the problem we are interested in is in the scale of a large contry sch as the United States, it is neither feasible nor necessary to address to each individal ser in the network. In sch cases, we divide the whole contry into small regions and aggregate the individal sers in each region into one ser node, corresponding to one element U. According to [13], the amont of ser activity can be for times higher in peak hors than in off-peak hors. This means that the nmber of service reqests may have significant flctation in one day. Therefore, a single reqest generation rate is not enogh to flly characterize the ser behavior. Generally, we can divide one day into a set of time periods, denoted by H. For each time period h H, the reqest generation rate of ser is denoted by λ h. The total lasting time of time period h in one day is denoted by l h. In the case that the sers behavior is similar and can be characterized by the same set of reqest generation rate in different time sections, we inclde these time sections into one time period for simplicity. In other words, each time period in or problem may be comprised of a set of disjoint time sections. A simple example wold be dividing a whole day into peak hors, h peak, and off-peak hors, h off peak. Apparently, shold be greater than λ h off peak λ h peak. B. Data center location and capacity Let D denote the set of available locations for data centers. For each location d D, let y d = 1 if a data center is going to be bilt at location d, and y d = 0 otherwise. For each location d D, let z d = 1 if a data center is already bilt at location d, and z d = 0 otherwise. We assme that the servers in the data centers are homogeneos, and then the size of a data center can be characterized by the nmber of servers installed. Let n d denote the nmber of servers in a new data center at location d if y d = 1 or the nmber of servers added to an existing server if z d = 1. De to some reasons, e.g. limited land space or available electricity generation, we cannot install infinite nmber of servers in a data center. Also, it is not practical to bild p a data center with jst a few servers. The minimm and maximm nmbers of servers that can be added to each location are denoted by N d,min and N d,max, respectively. Alternatively, we can express the size of a data center in terms of its amont of comptation resorces. If the processing speed of one server is µ s, then the total processing speed of a data center with n d severs, denoted by µ d, can be calclated as µ d = n d µ s. In spite of the discrete natre, n d and µ d can be assmed to be able to take continos vales in the case that the total nmber of servers in a data center is large. For existing data centers, we denote the original nmber of servers and the original processing speed by n 0,d and µ 0,d, respectively. For new data centers, we define n 0,d = 0 and µ 0,d = 0. C. Roting and delay modeling As described in Section II.A, the sers behave differently in different time periods. Therefore, the roting strategy shold be specified accordingly. Let x h,d = 1 if dring time period h, some of the reqests generated by ser are roted to the data center at location d, and x h,d = 0 otherwise. And let λh,d denote the rate that reqests are roted from ser to the data center at location d dring time period h when x h,d = 1. According to the GPS model, the average processing latency of these reqests, t h proc,,d, can be calclated as t h 1 proc,,d = φ h d, (µ d + µ 0,d ) λ h,d where φ h d, is the proportion of comptation resorces that data center at location d allocated to ser dring time period h. The eqivalent processing speed for reqests roted from ser to data center at location d, denoted by ψd, h, can be calclated as ψd, h = φ h d, (µ d + µ 0,d ) (2) It reflects the processing capability seen from a ser s side as if it is exclsively sed for the ser. Let t prop,,d and t prop,d, denote the propagation delay from ser to location d and the propagation delay from location d to ser, respectively. Then the average total delay experienced by a reqest roted from ser to the data center at location d dring time period h, denoted by t h total,,d, can be calclated as: (1) t h total,,d = t prop,,d + t h proc,,d + t prop,d, (3) where t h proc,,d is calclated as in (1). Generally, the propagation delays in the two directions are not necessarily the same becase the data packets may go throgh different paths in the network. However, since the analysis of all the roters behavior in the network is complicated and ot of the scope of this paper, we se the estimation of propagation delay based on Eclidean distance between the sorce and the destination, in which case the propagation delay in two directions will be the same. D. Power consmption modeling For a data center consisting of homogeneos servers, let P idle and P peak denote the power consmptions of a server at the tilization level of 0 and 100%, respectively. And let E sage,d denote the power sage effectiveness (PUE) [14] of a data center at location d. Then according to [15], the total power consmption of a data center at location d dring time period h, denoted by Ptotal,d h, can be calclated as P h total,d =(n d + n 0,d ) [P idle + (E sage,d 1) P peak ] + (n d + n 0,d ) (P peak P idle ) γ h d + ɛ (4)

3 where γd h is the tilization level of the data center dring time period h, which can be obtained by ( ) γd h = /(µ d + µ 0,d ) (5) x h,dλ h,d and ɛ is an empirical constant. If the data center treats all the servers in it fairly, then the power consmption corresponding to the newly installed servers, denoted by Pnew,d h, can be approximated as P h new,d ={n d [P idle + (E sage,d 1)P peak ] + n d (P peak P idle )γ h d + ɛ Please note that Ptotal,d h is eqal to P new,d h for a new data center since n 0,d = 0. Define P total,d as the maximm power consmption of the data center at location d. It can be calclated by applying γd h = 1 to (4) as (6) P total,d = (n d + n 0,d ) E sage,d P peak + ɛ (7) Similarly, we define P new,d the maximm power consmption of all the new servers. P new,d can be obtained by P new,d = n d E sage,d P peak + ɛ (8) These two parameters are sefl for the estimation of the cost of a data center, which is introdced in Section III.A. PUE is a parameter that characterize the power consmption of components other than the servers. It is related to the geographical location. For instance, PUE is sally higher in regions with higher temperatre or hmidity becase the cooling system consmes more energy in these regions. III. PROBLEM FORMULATION AND SOLUTION METHOD We formlate or problem as an optimization problem with the objective to minimize the average total cost of all the new and existing data centers in the network sbject to the constraint of a maximm allowable average latency for each ser. We first calclate the overall cost of a data center. A. Cost calclation The overall cost of a data center over its lifetime is comprised of varios aspects of capital cost and operational cost, each of which may depend on the time, the size of the data center and/or the location of the data center. We will address these aspects separately. Land cost. The land cost, denoted by C land,d, is the cost of bying enogh land space to bild p a data center, and can be calclated as follows C land,d = LandP rice(d) Area(d) (9) where LandP rice(d) is the cost of nit area at location d, and Area(d) is the area reqired for a data center at location d. According to [16], the reqired area of a data center can be estimated as linearly proportional to its maximm total power consmption. Since we do not need to accont for the land space occpied by the original part of an existing data center, P new,d is sed instead of P total,d. (9) can be rewritten as C land,d = LandP rice(d) k p a P new,d (10) where k p a is the ratio between the reqired area of a data center and its maximm total power consmption. Infrastrctre cost and server cost. The infrastrctre cost, denoted by C infra,d, is the cost of installing the power delivery network, the cooling system, and the internal networking within a data center. Similar to the land cost, the infrastrctre cost of a data center can be estimated as C infra,d = k p infra P new,d (11) where k p infra, sally in the nit of $/MW, is the infrastrctre cost per nit power consmption of the data center. The server cost, denoted by C serv,d, is the cost of bying new servers for a data center and can be straightforwardly calclated as C serv,d = n d ServerP rice (12) where ServerP rice is the price of a single server. Connection cost. The connection cost, denoted by C conn,d, is the cost to lay ot the optical fibers to the nearest Internet Backbone and the cost to lay ot power transmission lines to the existing electric power network, and can be calclated as C conn,d =T ranslinep rice DistP ower(d) + F iberp rice DistInternet(d) (13) where DistP ower(d) is the distance from the data center at location d to the nearest power plant or the existing transmission line, DistInternet(d) is the distance from the data center at location d to the nearest Internet backbone, and T ranslinep rice and F iberp rice are the prices of transmission line and optical fiber per nit length, respectively. For an existing data center, we do not have this part of cost since the connection is already bilt p along with the data center. Cooling cost. The cooling cost of a data center can be divided into two parts. One is the cost of electricity consmed by the compter room air conditioners (CRACs), which is characterized by the PUE and covered in the electricity cost. And the other is the cost of water wasted in the cooling circlation, which is specified in the Water cost. Electricity cost. The electricity cost, denoted by Celec,d h, is the cost of the electricity energy consmed while operating the data center. Since the total electricity power consmption of a data center can be obtained sing (4), the calclation of Celec,d h is straightforward. C h elec,d = ElectricityP rice(d) P h total,d (14) where ElectricityP rice(d) is the price of nit electricity energy at location d. Bandwidth cost. The bandwidth cost, denoted by C band,d, is the cost of acqiring sfficient bandwidth for commnications from the internet service providers (ISPs). If we allocate constant bandwidth to each server in the data center, then C band,d can be calclated as C band,d = BW P rice (n d + n 0,d ) BW Server (15) where BW P rice is the price of nit amont of bandwidth set by the ISPs, and BW Server is the amont of bandwidth allocated to each server. Water cost. As introdced in the description of the cooling cost, the water cost, denoted by C water,d, is the cost of the water wasted by the water chiller, which can be estimated

4 from the maximm total power consmption of the data center. C water,d is calclated as C water,d = W aterp rice(d) k p w P total,d (16) where W aterp rice(d) is the price of nit amont of water at location d, and k p w is the amont of water reqired by nit of maximm power consmption. Maintenance cost. The maintenance cost, denoted by C main,d, is the cost of hiring personnel to maintain and operate a data center and can be calclated as follows C main,d = k p main P total,d + Salary(d) (n d + n 0,d )/k s p (17) where the first term on the right-hand side is the maintenance cost and the second term on the right-hand size is the operational cost. k p main is the ratio between the maintenance cost and the maximm power consmption, Salary(d) is the salary to hire an administrator at location d, and k s p describes how many servers one administrator can manage. To obtain the overall cost of a data center in one day, the one-time investment inclding the land cost, the infrastrctre cost, the server cost, and the connection cost shold be amortized over the expected life time of the data center, a server, the transmission line, or the optical fiber. Therefore, if y d = 1, i.e. we will bild a new data center at location d, then the amortized overall cost per day of this data center, denoted by C total,d, can be calclated as C total,d = C land,d T dc + h + C infra,d T dc + C serv,d T serv + C conn,d T line l h C h elec,d + C band,d + C water,d + C main,d (18) where T dc, T serv, and T line are the expected lifetimes for the corresponding components. For an existing data center at location d with z d = 1, all the cost calclation is the same except that the connection cost, C conn,d, is set to zero. B. Problem formlation The average latency is calclated as the weighted average of the latency vales of reqests roted to each data center. Hence, in the case that the reqests from one ser node are roted to mltiple data centers with different average latencies, the overall average latency can still satisfy the average latency constraint even if a small portion of reqests roted to some data centers experience extremely long latency. However, we eliminate this possibility by enforcing the latency constraint on every roting path becase it can case serios nfairness otherwise. This is becase each ser node in or problem consists of a large nmber of individal sers, and the reqest from some individal sers may be consistently roted to the data centers with long latency de to the implemented roting scheme in reality. Based on the assmption of homogeneos servers, we can combine (4) and (5), and get Ptotal,d h =(n d + n 0,d ) [P idle + (E sage,d 1) P peak ] + µ s (P peak P idle ) x h,dλ h,d + ɛ (19) From (9) to (18), one can see that the total cost of a data center is a monotonically increasing fnction of the maximm power consmption of the data center. And from (19), one can see that the maximm power consmption is a nondecreasing fnction of the nmber of servers in the data center. Therefore, there is no reason to install extra servers to have to a data center with higher processing speed once the average latency constraint is satisfied. This being tre, we can transform the constraint of maximm average latency to the reqired eqivalent processing speed. Combining (1) with (3), we get ψ h d, = 1/(t h,max t prop,,d t prop,d, ) + λ h,d (20) where t h,max is the maximm allowable average latency for ser dring time period h. Given the vales of z d s, λ h s, n 0,d s, N d s, µ s, and all other information reqired dring the cost calclation, the optimization problem can be formlated as follows: Find x h,d, y d, n d, λ h,d, U, d D, h H Minimize (y d + z d )C total,d (21) d D Sbject to λ h,d = λ h,, h (22) d x h,dψd, h (n 0,d + n d )µ s, d, h (23) λ h,d λ h y d 1 z d, d (24) x h,d,, d, h (25) x h,d x h th,,d δ,, d, h (26) n d N d y d + z d, d (27) x h,d y d + z d,, d, h (28) x h,d {0, 1},, d, h (29) y d {0, 1}, d (30) n d [N d,min, N d,max ] N, d (31) λ h,d 0,, d, h (32) where C total,d is specified in (18), ψd, h is calclated in (20), δ is set to a small positive vale for the convenience of optimization, and x h th,,d is defined as x h th,,d = th,max t prop,,d t prop,d, t h,max + t prop,,d + t prop,d, + 1 (33) x h th,,d being less than or eqal to 1 means that the propagation delay between the ser and the data center is at least the maximm allowable average latency, and no matter how mch comptation resorce is allocated to this portion of reqest, the delay constraint will be violated. The objective fnction is the sm of the total cost of all the data centers over 24 hors of a day. (22) ensres that all the ser reqests at any time are roted to data centers for processing. (23) ensres that the allocated comptation resorces in each data center do not exceed its maximm amont of available comptation resorces. We have (24) becase the locations of existing data centers are not the candidate locations for bilding new data centers. We force

5 λ h,d to be 0 when we do not choose to rote any reqest from ser to the data center at location d dring time period h by (25). With (26), no reqests are roted to those data centers that cannot respond within the latency constraint. With (27), no servers are installed where no data center already exists or is to be bilt. And with (28), no reqests are roted to nonexisting data centers. (29)(30)(31)(32) specify the ranges of vale for variables x h,d, y d, n d, and λ h,d. C. Soltion method Generally, this is a mixed integer non-linear programming (MINLP) problem. Even thogh MINLP problems are NPhard, it is acceptable to solve the problem directly with some standard solvers, sch as CPLEX [17], regardless of the comptational complexity, since the problem is solved offline for only once. Nevertheless, some simplification can be made to the original problem. As is stated in Section II.B, since the nmber of servers in a data center is considerably large, we can assme that n d s can take continos vales with negligible error in cost and delay calclation, which redce the nmber of integer variables in the problem. Moreover, once the vales of x h,d s and y d s are given, the objective fnction and all the constraints are linear. Therefore, we can apply stochastic optimization methods sch as simlated annealing [18] or genetic algorithm [19] combined with linear programming to solve the problem. We also propose a greedy algorithm for sb-optimal soltions, which is applied in the following steps. Initially, we set all the y d s to 1 and all the n d s to 0. For each time period h, we first pick one ser and set the roting scheme to the one with the minimm cost increase among the roting schemes in which all the reqests are roted to a single data center. This process is repeated for every ser. When the roting for all the sers in every time period is done, we can find ot the minimm capacity for each data center, and the data center that is not sed will not be bilt. In the simple case with niform ser behavior over time, no existing data centers and ɛ in (4) eqal to 0, if the data center can be sized arbitrarily and all the candidate location can be connected to the power network and Internet with minimal cost, then the greedy algorithm can be proved to solve the problem optimally. IV. EXPERIMENTAL RESULTS In this section, we give an example on a service area of the United States. A. Experimental setp We set the ten most poplos cities in the U.S. as the ser nodes. The poplation information is obtained from the U.S. Censs Brea website 1. We select to bild the data centers among eight candidate locations, which are Astin, Bismarck, Los Angeles, New York City, Oklahoma City, Orlando, Seattle, and St. Lois. The World Geodetic System (WGS) coordinates of the ser nodes and the candidate bilding locations are obtained from the GeoNames database 2. The propagation delay between a data center and a ser node is set to be linearly proportional to the distance between the two locations. The propagation delay increases by 5ms for every 100km of distance. For each ser and time period, the maximm average latency, t h,max, is set to t MAX niformly Each day is divided into peak hors and off-peak hors. Throgh the Google clster data, we obtain the reqest arrival pattern by averaging the nmber of reqests arriving in each hor within a period of one month. We then divide the 24 hors of a day into 12 peak hors and 12 off-peak hors and calclate the ratio between the average reqest arrival rates of peak hors and off-peak hors. We define this ratio to be also the ratio of reqest generation rate in each ser node dring the peak hors and the off-peak hors. We assme a 1 reqest per second generation rate per 40,000 poplation in each ser node dring the peak hors. And the reqest generation rate dring the off-peak hors can be calclated accordingly. For each data center, the minimm and maximm nmber of servers are set to 1,000 and 50,000, respectively. We se Dell PowerEdge R610 as the server model, which has a peak power consmption of 260W and an idle power consmption of approximately 160W. Each server by itself is assmed to process reqests with an average response time of 100s. The empirical constant, ɛ, is set to 0. The PUE of a data center is modeled as a fnction of the temperatre as in [8]. Other parameters in the cost calclation are specified as follows. The land price for each location is calclated by averaging the prices looked p on real estate websites. k p a in (10) is set to 6,000 sf/mw as recommended in [16]. k p infra in (11) is set to $15/W [16]. Each server costs $2,000. The electric power grid map is obtained from the National Pblic Radio website 3 and each candidate city is covered by the existing power transmission lines. The Internet backbone maps can also be fond online 4, and the cost of optical fiber is set to $480,000 per mile. Each server is allocated 1Mbps of network bandwidth, and the price of bandwidth is set to $1/Mbps. The electric power prices in different states are obtained from the U.S. Energy Information Administration website 5. The water prices are obtained from the government websites of each city, and k p w in (16) is set to 24,000 gal/mw/day. The maintenance cost is set to $0.05/W/month [20]. The salary of each administrator is set to $100,000 per year, and k s p in (17) is set to 1,000. The expected lifetime of a data center, a server, and the optical fiber is set to 12 years, 4 years, and 12 years, respectively. B. Simlation reslts In this part, we will present the simlation reslts nder two different scenarios. In Scenario 1, there are no existing data centers, i.e. z d = 0, d. Two baselines are chosen by randomly place three data centers among the eight candidate locations. In Baseline 1, data centers are bilt in Los Angeles, New York City, and Seattle. In Baseline 2, data centers are bilt in Bismarck, Oklahoma City, and St. Lois. The relationship between the overall cost per day and t MAX is shown in Fig. 1. The cost of Baseline 2 when t MAX = 100ms is not shown becase the latency constraint cannot be satisfied in that case. Generally, the placement and capacity of the data centers are different with different t MAX in or reslt. For example, when t MAX = 200ms, data centers are bilt in Oklahoma City and St. Lois with 46,671 servers and 23,322 servers, respectively. When t MAX = 600ms, data centers are bilt in Astin, Oklahoma City, and Seattle with 15,741 servers,

6 maximm average latency for each service reqest. Simlation reslts are presented to show that the proposed scheme can achieve considerable cost saving over the baseline schemes. ACKNOWLEDGMENT This work is spported in part by the Software and Hardware Fondations program of the NSFs Directorate for Compter & Information Science & Engineering. Fig. 1. Simlation Reslt of Scenario 1 Fig. 2. Simlation Reslt of Scenario 2 38,094 servers, and 9,158 servers, respectively. One can make the observation that the overall cost is lower with higher t MAX, which is consistent with the fact that looser constraints reslt in better performance. It can also be seen from the reslt that the overall cost of the proposed schemes always lower than that of the baselines. When t MAX = 500ms, the proposed method saves $1.22M and $475k per month compared to Baseline 1 and Baseline 2, respectively. In Scenario 2, there already exists a data center in Seattle with 5,000 servers. The same baseline schemes are sed for comparison. The reslt is shown in Fig. 2. As can be seen from the figre, the reslt is similar to that of Scenario 1. V. CONCLUSION In this paper, we proposed a joint optimization framework of reqest flow control/resorce allocation and data center placement/capacity provisioning in a clod clod infrastrctre. The major aspects of the capital and operational cost of a data center are acconted for. An optimization problem is formlated with the objective to minimize the overall cost of all the data centers in the network and a constraint on the REFERENCES [1] B. Hayes, Clod compting, Commn. ACM, vol. 51, no. 7, pp. 9 11, Jl [2] R. Byya, C. S. Yeo, S. Vengopal, J. Broberg, and I. Brandic, Clod compting and emerging it platforms: Vision, hype, and reality for delivering compting as the 5th tility, Ftre Generation compter systems, vol. 25, no. 6, pp , [3] R. N. Calheiros, R. Ranjan, A. Beloglazov, C. A. De Rose, and R. Byya, Clodsim: a toolkit for modeling and simlation of clod compting environments and evalation of resorce provisioning algorithms, Software: Practice and Experience, vol. 41, no. 1, pp , [4] H. Godarzi and M. Pedram, Mlti-dimensional sla-based resorce allocation for mlti-tier clod compting systems, in Clod Compting (CLOUD), 2011 IEEE International Conference on. IEEE, 2011, pp [5] Y. Wang, S. Chen, H. Godarzi, and M. Pedram, Resorce allocation and consolidation in a mlti-core server clster sing a markov decision process model. in ISQED, 2013, pp [6] R. Katz, Tech titans bilding boom, Spectrm, IEEE, vol. 46, no. 2, pp , [7] A. Greenberg, J. Hamilton, D. A. Maltz, and P. Patel, The cost of a clod: research problems in data center networks, ACM SIGCOMM Compter Commnication Review, vol. 39, no. 1, pp , [8] I. Goiri, K. Le, J. Gitart, J. Torres, and R. Bianchini, Intelligent placement of datacenters for internet services, in Distribted Compting Systems (ICDCS), st International Conference on, 2011, pp [9] A.-H. Mohsenian-Rad and A. Leon-Garcia, Energy-information transmission tradeoff in green clod compting, Carbon, vol. 100, p. 200, [10] Google clster data. [Online]. Available: [11] A. K. Parekh and R. G. Gallager, A generalized processor sharing approach to flow control in integrated services networks: the single-node case, IEEE/ACM Transactions on Networking (TON), vol. 1, no. 3, pp , [12] Z.-L. Zhang, D. Towsley, and J. Krose, Statistical analysis of generalized processor sharing schedling discipline, in ACM SIGCOMM Compter Commnication Review, vol. 24, no. 4. ACM, 1994, pp [13] A. Qreshi, R. Weber, H. Balakrishnan, J. Gttag, and B. Maggs, Ctting the electric bill for internet-scale systems, ACM SIGCOMM Compter Commnication Review, vol. 39, no. 4, pp , [14] R. Brown et al., Report to congress on server and data center energy efficiency: Pblic law , [15] X. Fan, W.-D. Weber, and L. A. Barroso, Power provisioning for a warehose-sized compter, ACM SIGARCH Compter Architectre News, vol. 35, no. 2, pp , [16] W. P. Trner and J. H. Seader, Dollars per kw pls dollars per sqare foot are a better datacenter cost model than dollars per sqare foot alone, Uptime Institte White Paper, [17] I. CPLEX, 11.0 sers manal, ILOG SA, Gentilly, France, [18] P. J. Van Laarhoven and E. H. Aarts, Simlated annealing. Springer, [19] D. Whitley, A genetic algorithm ttorial, Statistics and compting, vol. 4, no. 2, pp , [20] L. A. Barroso and U. Hölzle, The datacenter as a compter: An introdction to the design of warehose-scale machines, Synthesis Lectres on Compter Architectre, vol. 4, no. 1, pp , 2009.