Algorithms for sustainable data centers


 Dustin Grant
 2 years ago
 Views:
Transcription
1 Algorithms for sustainable data centers Adam Wierman (Caltech) Minghong Lin (Caltech) Zhenhua Liu (Caltech) Lachlan Andrew (Swinburne) and many others
2 IT is an energy hog The electricity use of data centers is 23% of the US total, and it is growing 12% a year! Total US use grows ~1% a year! x 60 co 2 co 2
3 Huge push from academia & industry Lots of engineering improvements
4 Power Usage Effectiveness (PUE) = >3 <1.1 Power to building Power to computing lower PUE lower energy consumption more sustainable
5 Power Usage Effectiveness (PUE) = >3 <1.1 Power to building Power to computing The building is efficient, it s time to work on the algorithms
6 This talk: An overview of algorithmic questions & challenges 1. Two examples 2. A general model & some results 3. A case study
7 Example 1: Dynamic resizing in a data center Workload Highly nonstationary Idling servers use 3050% of peak power Active Servers Dynamic Controversial! resizing Goal: Adapt provisioning to minimize cost. Challenges: Switching is costly & prediction is hard.
8 Example 1: Dynamic resizing in a data center Workload (Interactive) Highly nonstationary Idling servers use 3050% of peak power Active Servers Dynamic resizing Workload (Batch) Large, delay tolerate jobs, typically with deadlines Deadlines are often on the order of hours Load shifting Valley filling
9 Example 1: Dynamic resizing in a data center Electricity prices Active Servers Solar availability PUE Dynamic resizing Load shifting PUE Valley filling
10 Example 2: Geographical load balancing Data centers: Diverse, time varying electricity prices, renewable availabilities, cooling efficiencies, etc. Solar availability Electricity prices Proxy/Mapping Nodes: Highly nonstationary workload
11 Example 2: Geographical load balancing Goal: Adapt routing & provisioning to minimize cost. Challenges: Switching is costly & prediction is hard.
12 Dynamic resizing Geographical load balancing [Chase et al. 01] [Pinheiro et al. 01] [Chen et al. 05] [Ghandi et al. 09] [Khargharia et al. 10] [Kansal et al. 10] [LWAT10] [Pakbaznia et al. 09] [Qureshi et al. 09] [Rao et al. 10] [Stanojevic et al. 10] [Wendell et al 10] [Le et al 2010] [LLWLA 11] [LLWAL11] [LAW12], Goal: Adapt routing & provisioning to minimize cost. Challenges: Switching is costly & prediction is hard.
13 A (familiar) model
14 x t λ t # of active servers 0 T ~10min workload time ~1month
15 Note: λ t and x t can be scalar or vector x t e.g. a homogeneous data center e.g. geographical load balancing λ t # of active servers 0 T workload time
16 Data center goal: min x t F (stability constraints) t (operating c t x t + cost) x+ t (switching x t 1 cost)
17 Data center goal: min x t F t stability constraints x t N N max x t 0 x t SLA(λ t ) (operating c t x t + cost) x+ t (switching x t 1 cost)
18 Data center goal: min x t F t (operating c t x t + cost) x+ t (switching x t 1 cost)
19 Data center goal: min x t F t c t x t + x t x t 1 E.g. A homogeneous data center E.g. Cost to switch server on/off β x t x t 1 c t x t ; λ t x t = min g t (λ t λ i ) it i=1 g t l (delay) d(l) + (energy p t e lcost) r + t
20 Data center goal: min x t F t c t x t convex + x t x t 1
21 Stochastic / Adversarial x 1, c 1, x 2, c 2, x 3, c 3, online min x t F t c t x t convex + x t x t 1 smoothed Smoothed online convex optimization (SOCO) Goal: Algorithms to minimize cost
22 c 1 c 1 (x 1 ) F x 1
23 c 2 c 1 c 2 (x 2 ) F x 1 x 2 x 2 x 1
24 SOCO comes up in many applications geographical load balancing dynamic capacity management video streaming electricity generation planning product/service selection portfolio management labor markets penalized estimation
25 x 1, c 1, x 2, c 2, x 3, c 3, online min x t F t c t x t convex + x t x t 1 smoothed Smoothed online convex optimization (SOCO) Goal: Algorithms to minimize cost 2 Communities Online Learning Online Algorithms
26 x 1, c 1, x 2, c 2, x 3, c 3, online min x t F t c t x t convex + x t x t 1 smoothed Smoothed online convex optimization (SOCO) Goal: Algorithms to minimize cost 2 Communities Online Learning Online Algorithms 2 Metrics Regret Competitive ratio
27 Regret(Alg) = Cost Alg Cost(Static_Opt) Competitive ratio(alg) = Cost Alg Cost Offline_Opt
28 Can an algorithm maintain sublinear regret? [KV02] [BBCM03] [Z03] [HAK07] [LRAMW12] Competitive ratio(alg) = Cost Alg Cost Offline_Opt
29 Can an algorithm maintain sublinear regret? [KV02] [BBCM03] [Z03] [HAK07] [LRAMW12] Can an algorithm maintain a constant competitive ratio? [BKRS92] [BLS92] [BB00] [LWAT11][LRAMW12] Can an algorithm maintain sublinear regret and a constant competitive ratio?
30 x 1, c 1, x 2, c 2, x 3, c 3, online min x t F t c t x t convex + x t x t 1 smoothed Smoothed online convex optimization (SOCO) Goal: Algorithms with sublinear regret 2 Communities Online Learning Online Algorithms 2 Metrics Regret Competitive ratio
31 x 1, c 1, x 2, c 2, x 3, c 3, online min x t F t c t x t convex + x t x t 1 smoothed Online convex optimization Goal: Algorithms with sublinear regret 2 Communities Online Learning Online Algorithms 2 Metrics Regret Competitive ratio
32 x 1, c 1, x 2, c 2, x 3, c 3, min x t F t c t c t x t 2 < C Online convex optimization Goal: Algorithms with sublinear regret Many algorithms achieve sublinear regret Online gradient descent (OGD) [Z03] x t+1 = Proj(x t η t c t x t )
33 Theorem [Z03] When η t = Θ(1/ t), online gradient descent has O T regret for OCO. Theorem [Z03,H06] Any algorithm for OCO must incur Ω on linear cost functions. T regret
34 x 1, c 1, x 2, c 2, x 3, c 3, min x t F t c t c t x t 2 < C Online convex optimization Goal: Algorithms with sublinear regret Many algorithms achieve sublinear regret  Online gradient descent (OGD) [Z03]  Multiplicative weights [AHK05] [FS99]  Newton s method based [HKKA06][HAK07]
35 x 1, c 1, x 2, c 2, x 3, c 3, min x t F t c t c t x t 2 < C Online convex optimization Goal: Algorithms with sublinear regret Do OCO algorithms maintain sublinear regret for SOCO?
36 x 1, c 1, x 2, c 2, x 3, c 3, min x t F t c t c t x t 2 < C + x t x t 1 smoothed Smoothed online convex optimization (SOCO) Goal: Algorithms with sublinear regret Do OCO algorithms maintain sublinear regret for SOCO?
37 Corollary: When η t = Θ(1/ t), online gradient descent still has O T regret for SOCO. Proof: t x t x t 1 M x t x t 1 2 (Equivalence of norms) M η t c t 1 (x t 1 ) 2 (Projection is nonexpansive) MC η t = O T (Bounded gradient)
38 Can an algorithm maintain sublinear regret? Yes: Online gradient descent has O T regret. Can an algorithm maintain a constant competitive ratio? [BKRS92] [BLS92] [BB00] [LWAT11][LRAMW12] Can an algorithm maintain sublinear regret and a constant competitive ratio?
39 x 1, c 1, x 2, c 2, x 3, c 3, online min x t F t c t x t convex + x t x t 1 smoothed Smoothed online convex optimization (SOCO) Goal: Algorithms with constant comp. ratio 2 Communities Online Learning Online Algorithms 2 Metrics Regret Competitive ratio
40 c 1, x 1, c 2, x 2, c 3, x 3, x 1, c 1, x 2, c 2, x 3, c 3, online min x t F t c t x t convex + x t x t 1 smoothed Metrical task system (MTS) Goal: Algorithms with constant comp. ratio 2 Communities Online Learning Online Algorithms 2 Metrics Regret Competitive ratio
41 c 1, x 1, c 2, x 2, c 3, x 3, online min x t F t c t x t convex + x t x t 1 smoothed Metrical task system (MTS) Goal: Algorithms with constant comp. ratio Is it possible to achieve a constant competitive ratio?
42 n is the size of the action set Theorem [BLS92] [BKRS92]: Any deterministic algorithm is Ω(n)competitive. Any randomized algorithm is Ω log n/log log n competitive
43 c 1, x 1, c 2, x 2, c 3, x 3, online min x t F t convex c t x t + x t x t 1 smoothed Metrical task system (MTS) Goal: Algorithms with constant comp. ratio Is it possible to achieve a constant competitive ratio?
44 c 1, x 1, c 2, x 2, c 3, x 3, online min x t F t convex c t x t + x t x t 1 scalar smoothed Smoothed online convex optimization (SOCO) Goal: Algorithms with constant comp. ratio Is it possible to achieve a constant competitive ratio?
45 Theorem: Lazy Capacity Provisioning (LCP) is 3competitive. Further Cost LCP Cost OPT + 2 SwitchingCost(OPT) Is it possible to achieve a constant competitive ratio?
46 t Minimize c s (x s t s=1 ) + s=1 1 (x s 1 x s ) + Subject to x s F x t U : last entry in Lazy Capacity Provisioning (LCP) time x t L : last entry in t Minimize c s (x s t s=1 ) + s=1 1 (x s x s 1 ) + Subject to x s F
47 Theorem: Lazy Capacity Provisioning (LCP) is 3competitive. Further Cost LCP Cost OPT + 2 SwitchingCost(OPT) Lemma: The offline optimal solution is lazy in reverse time.
48 t Minimize c s (x s t s=1 ) + s=1 1 (x s 1 x s ) + Subject to x s F x t U : last entry in Lazy Capacity Provisioning (LCP) time x t L : last entry in Optimal t Minimize c s (x s t s=1 ) + s=1 1 (x s x s 1 ) + Subject to x s F
49 c 1, x 1, c 2, x 2, c 3, x 3, online min x t F t convex c t x t + x t x t 1 scalar smoothed Smoothed online convex optimization (SOCO) Goal: Algorithms with constant comp. ratio Is it possible to achieve a constant competitive ratio?
50 c t x t min x t F, c t+1,, c t+w t convex c t x t predictions + x t x t 1 smoothed Smoothed online convex optimization (SOCO) Goal: Algorithms with constant comp. ratio Is it possible to achieve a constant competitive ratio?
51 Classic Approach: Receding Horizon Control Is it possible to achieve a constant competitive ratio?
52 Classic Approach: Receding Horizon Control, c t 1, c t, c t+1,, c t+w 1, c t+w, c t+w+1, x t 1 x t x t+1
53 Receding Horizon Control (RHC): Choose x t to minimize cost over t, t + w, given prediction window and x t 1., c t 1, c t, c t+1,, c t+w 1, c t+w, c t+w+1, x t 1 x t x t+1
54 Receding Horizon Control (RHC): Choose x t to minimize cost over t, t + w, given prediction window and x t 1. Theorem: For onedimensional SOCO, RHC is O(1 + 1 /w) competitive. But, in general, RHC is Ω 1 competitive. Does not improve as w grows!
55 , c t 1, c t, c t+1,, c t+w 1, c t+w, c t+w+1, RHC x t 1 x t FHC x t, x t+1,, x t+w AFHC x t is the average
56 Averaging Fixed Horizon Control (AFHC): Choose x t as the average of w + 1 fixed horizon control algorithms. AFHC x t is the average
57 Averaging Fixed Horizon Control (AFHC): Choose x t as the average of w + 1 fixed horizon control algorithms. Theorem: AFHC is O(1 + 1 /w) competitive
58 Can an algorithm maintain sublinear regret? 1 Yes: Online gradient descent has Θ T regret. Can an algorithm maintain a constant competitive 2 ratio? Yes. (in the scalar case): LCP is 3competitive. Yes. (in the vector case): AFHC is O( 1 /w)comp 3 Can an algorithm maintain sublinear regret and a constant competitive ratio? No!
59 Theorem: For arbitrary γ > 0 and any online algorithm A, there exist linear cost functions such that: Regret A CR A + T Deterministic or randomized γ
60 What now? Decide which metric you really care about?  Is εtregret good enough?  Is log log T  competitive good enough?  What about under stochastic assumptions?
61 Decide which metric you really care about?  Is εtregret good enough?  Is log log T  competitive good enough?  What about under stochastic assumptions? Theorem: Given a 1dimensional SOCO, for arbitrary, increasing f T, Randomly Biased Greedy (RBG) is: (1 + f(t)/ 1 )competitive with O(max {T/f(T), f(t)})regret
62 Randomly Biased Greedy (f(t)) x t = argmin w t (x t ) + x t r Where: 1 RBG = f(t). w t x = min{w t 1 y + c t y + x y RBG } y r = UnifRand 1 RBG, 1 RBG Theorem: Given a 1dimensional SOCO, for arbitrary, increasing f T, Randomly Biased Greedy (RBG) is: (1 + f(t)/ 1 )competitive with O(max {T/f(T), f(t)})regret
63 1 Can an algorithm maintain sublinear regret? Yes: Online gradient descent has Θ T regret. Can an algorithm maintain a constant competitive 2 ratio? Yes. (in the scalar case): LCP is 3competitive. Yes. (in the vector case): AFHC is O( 1 /w)comp 3 Can an algorithm maintain sublinear regret and a constant competitive ratio? No! But (in the scalar case): RBG is O(f T )competitive with O(max {T/f(T), f(t)}) regret.
64 Back to the applications
65 An implementation: Dynamic resizing Electricity prices Active Servers Solar availability PUE Dynamic resizing Load shifting PUE Valley filling
66 An implementation: Dynamic resizing WSJ ARTICLE SCREEN SCHOT HP collaborators: Yuan Chen, Cullen Bash, Martin Arlitt, Daniel Gmach, Zhikui Wang, Manish Marwah and Chris Hyser
67 An implementation: Dynamic resizing HP collaborators: Yuan Chen, Cullen Bash, Martin Arlitt, Daniel Gmach, Zhikui Wang, Manish Marwah and Chris Hyser
68 A case study: Geographical load balancing Data centers: Google data center locations Real traces for wind, solar, electricity price, etc. Solar availability Wind availability Proxy/Mapping Nodes: Scaled by population density
69 A case study: Geographical load balancing GLB: geographical load balancing & dynamic resizing vs. LOCAL: Route to the closest data center & dynamic resizing
70 # Servers ON The good Follow the renewables routing emerges. east coast west coast Day 1 Day 2
71 The good 8 Follow the renewables routing emerges. Huge reductions in grid usage become possible. Capacity needed to reduce grid usage by 85% 6 LOCAL 4 2 GLB Solar capacity
72 The good Follow the renewables routing emerges. Huge reductions in grid usage become possible. The bad GLB uses more energy if data centers don t have renewables available. GLB LOCAL Day 1 Day 2
73 The good Follow the renewables routing emerges. Huge reductions in grid usage become possible. The bad GLB uses more energy if data centers don t have renewables available. The ugly GLB uses dirtier grid electricity too!
74 Demand response can help! The ugly GLB uses dirtier grid electricity too!
75 Final thoughts
76 Dynamic resizing Geographical load balancing 𝑥1, 𝑐1, 𝑥 2, 𝑐 2, 𝑥 3, 𝑐 3, 𝑡 min 𝑥 𝑡 𝐹 𝑐 𝑥 𝑡 𝑡 convex 𝑡 + 𝑥 𝑥 online 𝑡 1 smoothed Smoothed online convex optimization Implementation
77 Lots of interesting theoretical questions remain: Stochastic models for cost functions better algorithms? Predictions are extremely important in practice Stochastic variations? The full cost function is not observed in practice Bandit versions? Can an algorithm balance regret and the competitive ratio? x 1, c 1, x 2, c 2, x 3, c 3, min x t F t c t x t convex online + x t x t 1 smoothed Smoothed online convex optimization
78 Algorithms for sustainable data centers Adam Wierman (Caltech) Minghong Lin (Caltech) Zhenhua Liu (Caltech) Lachlan Andrew (Swinburne) and many others Minghong Lin, Adam Wierman, Lachlan Andrew, and Eno Thereska. Dynamic rightsizing for power proportional data centers. Infocom, Best Paper award winner. Zhenhua Liu, Minghong Lin, Adam Wierman, Steven Low, and Lachlan Andrew. Greening geographical load balancing. Sigmetrics Zhenhua Liu, Minghong Lin, Adam Wierman, Steven Low, and Lachlan Andrew. Geographical load balancing with renewables. Greenmetrics, Best Student Paper award winner. Zhenhua Liu, Yuan Chen, Cullen Bash, Adam Wierman, et al. Renewable and cooling aware workload management for sustainable data centers. Sigmetrics Minghong Lin, Lachlan Andrew, and Adam Wierman. Online Algorithms for Geographical Load Balancing. Green Computing Conference, Best Paper award winner.
A few algorithmic issues in data centers Adam Wierman Caltech
A few algorithmic issues in data centers Adam Wierman Caltech A significant theory literature on green computing has emerged over the last decade BUT theory has yet to have significant impact in practice.
More informationOnline Dynamic Capacity Provisioning in Data Centers
Online Dynamic Capacity Provisioning in Data Centers Minghong Lin and Adam Wierman California Institute of Technology Lachlan L. H. Andrew Swinburne University of Technology Eno Thereska Microsoft Research
More informationAlgorithmic Challenges in Green Data Centers
Algorithmic Challenges in Green Data Centers Thesis by Minghong Lin In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California
More informationENERGY EFFICIENT AND REDUCTION OF POWER COST IN GEOGRAPHICALLY DISTRIBUTED DATA CARDS
ENERGY EFFICIENT AND REDUCTION OF POWER COST IN GEOGRAPHICALLY DISTRIBUTED DATA CARDS M Vishnu Kumar 1, E Vanitha 2 1 PG Student, 2 Assistant Professor, Department of Computer Science and Engineering,
More informationData centers & energy: Did we get it backwards? Adam Wierman, Caltech
Data centers & energy: Did we get it backwards? Adam Wierman, Caltech The typical story about energy & data centers: The typical story about energy & data centers: Sustainable data centers Remember: The
More informationOnline Algorithms for Geographical Load Balancing
Online Algorithms for Geographical Load Balancing Minghong Lin, Zhenhua Liu, Adam Wierman, Lachlan L. H. Andrew California Institute of Technology, Email: {mhlin,zhenhua,adamw}@caltech.edu Swinburne University
More informationOnline Algorithms for Geographical Load Balancing
Online Algorithms for Geographical Load Balancing Minghong Lin, Zhenhua Liu, Adam Wierman, Lachlan L. H. Andrew California Institute of Technology, Email: {mhlin,zhenhua,adamw}@caltech.edu Swinburne University
More informationDynamic Virtual Machine Allocation in Cloud Server Facility Systems with Renewable Energy Sources
Dynamic Virtual Machine Allocation in Cloud Server Facility Systems with Renewable Energy Sources Dimitris Hatzopoulos University of Thessaly, Greece Iordanis Koutsopoulos Athens University of Economics
More informationDynamic Scheduling and Pricing in Wireless Cloud Computing
Dynamic Scheduling and Pricing in Wireless Cloud Computing R.Saranya 1, G.Indra 2, Kalaivani.A 3 Assistant Professor, Dept. of CSE., R.M.K.College of Engineering and Technology, Puduvoyal, Chennai, India.
More informationImpact of workload and renewable prediction on the value of geographical workload management. Arizona State University
Impact of workload and renewable prediction on the value of geographical workload management Zahra Abbasi, Madhurima Pore, and Sandeep Gupta Arizona State University Funded in parts by NSF CNS grants and
More informationOnline Resource Management for Data Center with Energy Capping
Online Resource Management for Data Center with Energy Capping A. S. M. Hasan Mahmud Florida International University Shaolei Ren Florida International University Abstract The past few years have been
More informationRenewable and Cooling Aware Workload Management for Sustainable Data Centers
Renewable and Cooling Aware Workload Management for Sustainable Data Centers Zhenhua Liu, Yuan Chen 2, Cullen Bash 2, Adam Wierman, Daniel Gmach 2, Zhikui Wang 2, Manish Marwah 2, and Chris Hyser 2 Computing
More informationGeographical Load Balancing for Online Service Applications in Distributed Datacenters
Geographical Load Balancing for Online Service Applications in Distributed Datacenters Hadi Goudarzi and Massoud Pedram University of Southern California Department of Electrical Engineering  Systems
More informationGeographical Load Balancing for Online Service Applications in Distributed Datacenters
Geographical Load Balancing for Online Service Applications in Distributed Datacenters Hadi Goudarzi and Massoud Pedram University of Southern California Department of Electrical Engineering  Systems
More informationAdaptive Online Gradient Descent
Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650
More informationCompetitive Analysis of QoS Networks
Competitive Analysis of QoS Networks What is QoS? The art of performance analysis What is competitive analysis? Example: Scheduling with deadlines Example: Smoothing realtime streams Example: Overflow
More information17.3.1 Follow the Perturbed Leader
CS787: Advanced Algorithms Topic: Online Learning Presenters: David He, Chris Hopman 17.3.1 Follow the Perturbed Leader 17.3.1.1 Prediction Problem Recall the prediction problem that we discussed in class.
More informationForcedirected Geographical Load Balancing and Scheduling for Batch Jobs in Distributed Datacenters
Forcedirected Geographical Load Balancing and Scheduling for Batch Jobs in Distributed Datacenters Hadi Goudarzi and Massoud Pedram University of Southern California Department of Electrical Engineering
More informationProvablyEfficient Job Scheduling for Energy and Fairness in Geographically Distributed Data Centers
3nd IEEE International Conference on Distributed Computing Systems ProvablyEfficient Job Scheduling for Energy and Fairness in Geographically Distributed Data Centers Shaolei Ren Yuxiong He Fei Xu Electrical
More informationOnline Convex Optimization
E0 370 Statistical Learning heory Lecture 19 Oct 22, 2013 Online Convex Optimization Lecturer: Shivani Agarwal Scribe: Aadirupa 1 Introduction In this lecture we shall look at a fairly general setting
More informationROUTING ALGORITHM BASED COST MINIMIZATION FOR BIG DATA PROCESSING
ROUTING ALGORITHM BASED COST MINIMIZATION FOR BIG DATA PROCESSING D.Vinotha,PG Scholar,Department of CSE,RVS Technical Campus,vinothacse54@gmail.com Dr.Y.Baby Kalpana, Head of the Department, Department
More informationAlgorithmic Issues in Green Data Centers
Algorithmic Issues in Green Data Centers Thesis by Minghong Lin In Partial Fulfillment of the Requirements for the Degree of Master of Science California Institute of Technology Pasadena, California 2011
More informationSimple and Eective Dynamic Provisioning for Powerproportional Data Centers
Simple and Eective Dynamic Provisioning for Powerproportional Data Centers LU, Tan A Thesis Submitted in Partial Fullment of the Requirements for the Degree of Master of Philosophy in Information Engineering
More informationTrafficAware Resource Provisioning for Distributed Clouds
ENERGY EFFICIENCY TrafficAware Resource Provisioning for Distributed Clouds Dan Xu, AT&T Labs Xin Liu, University of California, Davis Athanasios V. Vasilakos, Lulea University of Technology, Sweden Examining
More informationRenewable and Cooling Aware Workload Management for Sustainable Data Centers
Renewable and Cooling Aware Workload Management for Sustainable Data Centers Zhenhua Liu, Yuan Chen 2, Cullen Bash 2, Adam Wierman, Daniel Gmach 2, Zhikui Wang 2, Manish Marwah 2, and Chris Hyser 2 California
More informationResourceDiversity Tolerant: Resource Allocation in the Cloud Infrastructure Services
IOSR Journal of Computer Engineering (IOSRJCE) eissn: 22780661,pISSN: 22788727, Volume 17, Issue 5, Ver. III (Sep. Oct. 2015), PP 1925 www.iosrjournals.org ResourceDiversity Tolerant: Resource Allocation
More informationGeographical load balancing
Geographical load balancing Jonathan Lukkien 31 mei 2013 1 / 27 Jonathan Lukkien Geographical load balancing Overview Introduction The Model Algorithms & Experiments Extensions Conclusion 2 / 27 Jonathan
More informationEnergy Efficient Geographical Load Balancing via Dynamic Deferral of Workload
2012 IEEE Fifth International Conference on Cloud Computing Energy Efficient Geographical Load Balancing via Dynamic Deferral of Workload Muhammad Abdullah Adnan, Ryo Sugihara and Rajesh K. Gupta University
More informationSociallyResponsible Load Scheduling Algorithms for Sustainable Data Centers over Smart Grid
SociallyResponsible Load Scheduling Algorithms for Sustainable Data Centers over Smart Grid Jian He, Xiang Deng, Dan Wu, Yonggang Wen, Di Wu Department of Computer Science, Sun YatSen University, Guangzhou,
More informationOnline Algorithms: Learning & Optimization with No Regret.
Online Algorithms: Learning & Optimization with No Regret. Daniel Golovin 1 The Setup Optimization: Model the problem (objective, constraints) Pick best decision from a feasible set. Learning: Model the
More informationExploring Smart Grid and Data Center Interactions for Electric Power Load Balancing
Exploring Smart Grid and Data Center Interactions for Electric Power Load Balancing Hao Wang, Jianwei Huang, Xiaojun Lin, Hamed MohsenianRad Department of Information Engineering, The Chinese University
More informationOnline Resource Management for Data Center with Energy Capping
Online Resource Management for Data Center with Energy Capping Hasan Mahmud and Shaolei Ren Florida International University 1 A massive data center Facebook's data center in Prineville, OR 2 Three pieces
More informationData Mining for Sustainable Data Centers
Data Mining for Sustainable Data Centers Manish Marwah Senior Research Scientist Hewlett Packard Laboratories manish.marwah@hp.com 2009 HewlettPackard Development Company, L.P. The information contained
More informationSimple and efficient online algorithms for real world applications
Simple and efficient online algorithms for real world applications Università degli Studi di Milano Milano, Italy Talk @ Centro de Visión por Computador Something about me PhD in Robotics at LIRALab,
More informationData Center Energy Cost Minimization: a SpatioTemporal Scheduling Approach
23 Proceedings IEEE INFOCOM Data Center Energy Cost Minimization: a SpatioTemporal Scheduling Approach Jianying Luo Dept. of Electrical Engineering Stanford University jyluo@stanford.edu Lei Rao, Xue
More informationProfit Maximization and Power Management of Green Data Centers Supporting Multiple SLAs
Profit Maximization and Power Management of Green Data Centers Supporting Multiple SLAs Mahdi Ghamkhari and Hamed MohsenianRad Department of Electrical Engineering University of California at Riverside,
More informationEnergy Management and Profit Maximization of Green Data Centers
Energy Management and Profit Maximization of Green Data Centers Seyed Mahdi Ghamkhari, B.S. A Thesis Submitted to Graduate Faculty of Texas Tech University in Partial Fulfilment of the Requirements for
More informationMinimizing the Operational Cost of Data Centers via Geographical Electricity Price Diversity
203 IEEE Sixth International Conference on Cloud Computing Minimizing the Operational Cost of Data Centers via Geographical Electricity Price Diversity Zichuan Xu Weifa Liang Research School of Computer
More informationBig Data Processing of Data Services in Geo Distributed Data Centers Using Cost Minimization Implementation
Big Data Processing of Data Services in Geo Distributed Data Centers Using Cost Minimization Implementation A. Dhineshkumar, M.Sakthivel Final Year MCA Student, VelTech HighTech Engineering College, Chennai,
More informationCostaware Workload Dispatching and Server Provisioning for Distributed Cloud Data Centers
, pp.5160 http://dx.doi.org/10.14257/ijgdc.2013.6.5.05 Costaware Workload Dispatching and Server Provisioning for Distributed Cloud Data Centers Weiwei Fang 1, Quan Zhou 1, Yuan An 2, Yangchun Li 3 and
More informationData Center Optimization Methodology to Maximize the Usage of Locally Produced Renewable Energy
Data Center Optimization Methodology to Maximize the Usage of Locally Produced Renewable Energy Tudor Cioara, Ionut Anghel, Marcel Antal, Sebastian Crisan, Ioan Salomie Computer Science Department Technical
More informationCSC2420 Fall 2012: Algorithm Design, Analysis and Theory
CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms
More informationMapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research
MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With
More informationLeveraging Renewable Energy in Data Centers: Present and Future
Leveraging Renewable Energy in Data Centers: Present and Future Ricardo Bianchini Department of Computer Science Collaborators: Josep L. Berral, Inigo Goiri, Jordi Guitart, Md. Haque, William Katsak, Kien
More informationCharacterizing Task Usage Shapes in Google s Compute Clusters
Characterizing Task Usage Shapes in Google s Compute Clusters Qi Zhang 1, Joseph L. Hellerstein 2, Raouf Boutaba 1 1 University of Waterloo, 2 Google Inc. Introduction Cloud computing is becoming a key
More informationIntroduction to Online Learning Theory
Introduction to Online Learning Theory Wojciech Kot lowski Institute of Computing Science, Poznań University of Technology IDSS, 04.06.2013 1 / 53 Outline 1 Example: Online (Stochastic) Gradient Descent
More informationJoint Optimization of Overlapping Phases in MapReduce
Joint Optimization of Overlapping Phases in MapReduce Minghong Lin, Li Zhang, Adam Wierman, Jian Tan Abstract MapReduce is a scalable parallel computing framework for big data processing. It exhibits multiple
More informationOnline Learning. 1 Online Learning and Mistake Bounds for Finite Hypothesis Classes
Advanced Course in Machine Learning Spring 2011 Online Learning Lecturer: Shai ShalevShwartz Scribe: Shai ShalevShwartz In this lecture we describe a different model of learning which is called online
More informationThe Answer Is Blowing in the Wind: Analysis of Powering Internet Data Centers with Wind Energy
The Answer Is Blowing in the Wind: Analysis of Powering Internet Data Centers with Wind Energy Yan Gao Accenture Technology Labs Zheng Zeng Apple Inc. Xue Liu McGill University P. R. Kumar Texas A&M University
More informationAn improved online algorithm for scheduling on two unrestrictive parallel batch processing machines
This is the PrePublished Version. An improved online algorithm for scheduling on two unrestrictive parallel batch processing machines Q.Q. Nong, T.C.E. Cheng, C.T. Ng Department of Mathematics, Ocean
More information2.3 Convex Constrained Optimization Problems
42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions
More informationHow to assess the risk of a large portfolio? How to estimate a large covariance matrix?
Chapter 3 Sparse Portfolio Allocation This chapter touches some practical aspects of portfolio allocation and risk assessment from a large pool of financial assets (e.g. stocks) How to assess the risk
More informationOnline Adwords Allocation
Online Adwords Allocation Shoshana Neuburger May 6, 2009 1 Overview Many search engines auction the advertising space alongside search results. When Google interviewed Amin Saberi in 2004, their advertisement
More informationDynamic Workload Management in Heterogeneous Cloud Computing Environments
Dynamic Workload Management in Heterogeneous Cloud Computing Environments Qi Zhang and Raouf Boutaba University of Waterloo IEEE/IFIP Network Operations and Management Symposium Krakow, Poland May 7, 2014
More informationMultiArmed Bandit Problems
Machine Learning Coms4771 MultiArmed Bandit Problems Lecture 20 Multiarmed Bandit Problems The Setting: K arms (or actions) Each time t, each arm i pays off a bounded realvalued reward x i (t), say
More informationOnline Learning and Competitive Analysis: a Unified Approach
Online Learning and Competitive Analysis: a Unified Approach Shahar Chen Online Learning and Competitive Analysis: a Unified Approach Research Thesis Submitted in partial fulfillment of the requirements
More informationLoad Balancing and Switch Scheduling
EE384Y Project Final Report Load Balancing and Switch Scheduling Xiangheng Liu Department of Electrical Engineering Stanford University, Stanford CA 94305 Email: liuxh@systems.stanford.edu Abstract Load
More informationModern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh
Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem
More informationDynamic TCP Acknowledgement: Penalizing Long Delays
Dynamic TCP Acknowledgement: Penalizing Long Delays Karousatou Christina Network Algorithms June 8, 2010 Karousatou Christina (Network Algorithms) Dynamic TCP Acknowledgement June 8, 2010 1 / 63 Layout
More informationComputing Load Aware and LongView Load Balancing for Cluster Storage Systems
215 IEEE International Conference on Big Data (Big Data) Computing Load Aware and LongView Load Balancing for Cluster Storage Systems Guoxin Liu and Haiying Shen and Haoyu Wang Department of Electrical
More information1 Introduction. 2 Prediction with Expert Advice. Online Learning 9.520 Lecture 09
1 Introduction Most of the course is concerned with the batch learning problem. In this lecture, however, we look at a different model, called online. Let us first compare and contrast the two. In batch
More informationOptimal Online MultiInstance Acquisition in IaaS Clouds
Optimal Online MultiInstance Acquisition in IaaS Clouds Wei Wang, Student Member, IEEE, Ben Liang, Senior Member, IEEE, and Baochun Li, Fellow, IEEE Abstract InfrastructureasaService (IaaS) clouds
More informationChapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling
Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NPhard problem. What should I do? A. Theory says you're unlikely to find a polytime algorithm. Must sacrifice one
More informationData Centers Power Reduction: A two Time Scale Approach for Delay Tolerant Workloads
Data Centers Power Reduction: A two Time Scale Approach for Delay Tolerant Workloads Yuan Yao, Longbo Huang, Abhihshek Sharma, Leana Golubchik and Michael Neely University of Southern California, Los Angeles,
More informationLecture Notes 12: Scheduling  Cont.
Online Algorithms 18.1.2012 Professor: Yossi Azar Lecture Notes 12: Scheduling  Cont. Scribe:Inna Kalp 1 Introduction In this Lecture we discuss 2 scheduling models. We review the scheduling over time
More informationForcedirected Geographical Load Balancing and Scheduling for Batch Jobs in Distributed Datacenters
Forcedirected Geographical Load Balancing and Scheduling for Batch Jobs in Distributed Datacenters Hadi Goudarzi and Massoud Pedram University of Southern California Department of Electrical Engineering
More informationOnline Scheduling with Bounded Migration
Online Scheduling with Bounded Migration Peter Sanders, Naveen Sivadasan, and Martin Skutella MaxPlanckInstitut für Informatik, Saarbrücken, Germany, {sanders,ns,skutella}@mpisb.mpg.de Abstract. Consider
More informationSetting deadlines and priorities to the tasks to improve energy efficiency in cloud computing
Setting deadlines and priorities to the tasks to improve energy efficiency in cloud computing Problem description Cloud computing is a technology used more and more every day, requiring an important amount
More informationLowCarbon Routing Algorithms For Cloud Computing Services in IPoverWDM Networks
LowCarbon Routing Algorithms For Cloud Computing Services in IPoverWDM Networks TREND Plenary Meeting, VolosGreece 0105/10/2012 TD3.3: Energyefficient service provisioning and content distribution
More informationOptimizing Cost and Performance for Content Multihoming
Optimizing Cost and Performance for Content Multihoming Hongqiang Harry Liu Ye Wang Yang Richard Yang Hao Wang Chen Tian Aug. 16, 2012 Yale LANS Content Multihoming is Widely Used Content Publisher Content
More informationA Truthful Incentive Mechanism for Emergency Demand Response in Colocation Data Centers. Linquan Zhang, Shaolei Ren Chuan Wu and Zongpeng Li
A Truthful Incentive Mechanism for Emergency Demand Response in Colocation Data Centers Linquan Zhang, Shaolei Ren Chuan Wu and Zongpeng Li 1 Demand vs Supply in Power Industry? Ideal: Supply = Demand
More informationGI01/M055 Supervised Learning Proximal Methods
GI01/M055 Supervised Learning Proximal Methods Massimiliano Pontil (based on notes by Luca Baldassarre) (UCL) Proximal Methods 1 / 20 Today s Plan Problem setting Convex analysis concepts Proximal operators
More information! Solve problem to optimality. ! Solve problem in polytime. ! Solve arbitrary instances of the problem. !approximation algorithm.
Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NPhard problem What should I do? A Theory says you're unlikely to find a polytime algorithm Must sacrifice one of
More informationADAPTIVE LOAD BALANCING ALGORITHM USING MODIFIED RESOURCE ALLOCATION STRATEGIES ON INFRASTRUCTURE AS A SERVICE CLOUD SYSTEMS
ADAPTIVE LOAD BALANCING ALGORITHM USING MODIFIED RESOURCE ALLOCATION STRATEGIES ON INFRASTRUCTURE AS A SERVICE CLOUD SYSTEMS Lavanya M., Sahana V., Swathi Rekha K. and Vaithiyanathan V. School of Computing,
More informationOnline Learning of Optimal Strategies in Unknown Environments
1 Online Learning of Optimal Strategies in Unknown Environments Santiago Paternain and Alejandro Ribeiro Abstract Define an environment as a set of convex constraint functions that vary arbitrarily over
More informationCutting Down the Energy Cost of Geographically Distributed Cloud Data Centers
Cutting Down the Energy Cost of Geographically Distributed Cloud Data Centers Huseyin Guler 1, B. Barla Cambazoglu 2 and Oznur Ozkasap 1 1 Koc University, Istanbul, Turkey 2 Yahoo! Research, Barcelona,
More informationEnergy Aware Consolidation for Cloud Computing
Abstract Energy Aware Consolidation for Cloud Computing Shekhar Srikantaiah Pennsylvania State University Consolidation of applications in cloud computing environments presents a significant opportunity
More informationSTORM: Stochastic Optimization Using Random Models Katya Scheinberg Lehigh University. (Joint work with R. Chen and M. Menickelly)
STORM: Stochastic Optimization Using Random Models Katya Scheinberg Lehigh University (Joint work with R. Chen and M. Menickelly) Outline Stochastic optimization problem black box gradient based Existing
More informationLatticeBased ThresholdChangeability for Standard Shamir SecretSharing Schemes
LatticeBased ThresholdChangeability for Standard Shamir SecretSharing Schemes Ron Steinfeld (Macquarie University, Australia) (email: rons@ics.mq.edu.au) Joint work with: Huaxiong Wang (Macquarie University)
More informationBig Data  Lecture 1 Optimization reminders
Big Data  Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Big Data  Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Schedule Introduction Major issues Examples Mathematics
More informationA o(n)competitive Deterministic Algorithm for Online Matching on a Line
A o(n)competitive Deterministic Algorithm for Online Matching on a Line Antonios Antoniadis 2, Neal Barcelo 1, Michael Nugent 1, Kirk Pruhs 1, and Michele Scquizzato 3 1 Department of Computer Science,
More informationDemand Response in Data Centers Feasibility and Proof of Concept Demonstration
Demand Response in Data Centers Feasibility and Proof of Concept Demonstration Jay Madden, SCE; Mukesh Khattar, EPRI; Henry Wong, Intel; Chris Battista, Calit2 CalPlug Workshop Series #7 University of
More informationLoad Balancing in Distributed Web Server Systems With Partial Document Replication
Load Balancing in Distributed Web Server Systems With Partial Document Replication Ling Zhuo, ChoLi Wang and Francis C. M. Lau Department of Computer Science and Information Systems The University of
More informationMultifaceted Resource Management for Dealing with Heterogeneous Workloads in Virtualized Data Centers
Multifaceted Resource Management for Dealing with Heterogeneous Workloads in Virtualized Data Centers Íñigo Goiri, J. Oriol Fitó, Ferran Julià, Ramón Nou, Josep Ll. Berral, Jordi Guitart and Jordi Torres
More informationOnline Learning and Online Convex Optimization. Contents
Foundations and Trends R in Machine Learning Vol. 4, No. 2 (2011) 107 194 c 2012 S. ShalevShwartz DOI: 10.1561/2200000018 Online Learning and Online Convex Optimization By Shai ShalevShwartz Contents
More informationOnline Convex Programming and Generalized Infinitesimal Gradient Ascent
Online Convex Programming and Generalized Infinitesimal Gradient Ascent Martin Zinkevich February 003 CMUCS03110 School of Computer Science Carnegie Mellon University Pittsburgh, PA 1513 Abstract Convex
More informationitesla Project Innovative Tools for Electrical System Security within Large Areas
itesla Project Innovative Tools for Electrical System Security within Large Areas Samir ISSAD RTE France samir.issad@rtefrance.com PSCC 2014 Panel Session 22/08/2014 Advanced datadriven modeling techniques
More informationOptimal Strategies and Minimax Lower Bounds for Online Convex Games
Optimal Strategies and Minimax Lower Bounds for Online Convex Games Jacob Abernethy UC Berkeley jake@csberkeleyedu Alexander Rakhlin UC Berkeley rakhlin@csberkeleyedu Abstract A number of learning problems
More informationCapacity Planning and Power Management to Exploit Sustainable Energy
Capacity Planning and Power Management to Exploit Sustainable Energy Daniel Gmach, Jerry Rolia, Cullen Bash, Yuan Chen, Tom Christian, Amip Shah, Ratnesh Sharma, Zhikui Wang HP Labs Palo Alto, CA, USA
More information14.1 Rentorbuy problem
CS787: Advanced Algorithms Lecture 14: Online algorithms We now shift focus to a different kind of algorithmic problem where we need to perform some optimization without knowing the input in advance. Algorithms
More informationSoSe 2014 Dozenten: Prof. Dr. Thomas Ludwig, Dr. Manuel Dolz Vorgetragen von Hakob Aridzanjan 03.06.2014
Total Cost of Ownership in High Performance Computing HPC datacenter energy efficient techniques: job scheduling, best programming practices, energysaving hardware/software mechanisms SoSe 2014 Dozenten:
More informationCURTAIL THE EXPENDITURE OF BIG DATA PROCESSING USING MIXED INTEGER NONLINEAR PROGRAMMING
Journal homepage: http://www.journalijar.com INTERNATIONAL JOURNAL OF ADVANCED RESEARCH RESEARCH ARTICLE CURTAIL THE EXPENDITURE OF BIG DATA PROCESSING USING MIXED INTEGER NONLINEAR PROGRAMMING R.Kohila
More informationOnline Learning with Switching Costs and Other Adaptive Adversaries
Online Learning with Switching Costs and Other Adaptive Adversaries Nicolò CesaBianchi Università degli Studi di Milano Italy Ofer Dekel Microsoft Research USA Ohad Shamir Microsoft Research and the Weizmann
More informationDynamic rightsizing for powerproportional data centers
Dynamic rightsizing for powerproportional data centers Minghong Lin and Adam Wierman California Institute of Technology Lachlan L. H. Andrew Swinburne University of Technology Eno Thereska Microsoft
More informationStochastic Inventory Control
Chapter 3 Stochastic Inventory Control 1 In this chapter, we consider in much greater details certain dynamic inventory control problems of the type already encountered in section 1.3. In addition to the
More informationSingle machine models: Maximum Lateness 12 Approximation ratio for EDD for problem 1 r j,d j < 0 L max. structure of a schedule Q...
Lecture 4 Scheduling 1 Single machine models: Maximum Lateness 12 Approximation ratio for EDD for problem 1 r j,d j < 0 L max structure of a schedule 0 Q 1100 11 00 11 000 111 0 0 1 1 00 11 00 11 00
More informationData Centers to Offer Ancillary Services
Data Centers to Offer Ancillary Services Mahdi Ghamkhari and Hamed MohsenianRad Department of Electrical Engineering, University of California at Riverside, Riverside, CA, USA emails: {ghamkhari, hamed}@ee.ucr.edu
More informationMobile Visual Analytics for Data Center Power and Cooling Management
Mobile Visual Analytics for Data Center Power and Cooling Management Ratnesh Sharma, Ming Hao, Ravigopal Vennelakanti, Manish Gupta, Umeshwar Dayal, Cullen Bash, Chandrakant Patel, Deepa Naik, A Jayakumar,
More informationLoad balancing of temporary tasks in the l p norm
Load balancing of temporary tasks in the l p norm Yossi Azar a,1, Amir Epstein a,2, Leah Epstein b,3 a School of Computer Science, Tel Aviv University, Tel Aviv, Israel. b School of Computer Science, The
More informationGreening Geographical Load Balancing
Greening Geographical Load Balancing Zhenhua Liu, Minghong Lin, Adam Wierman, Steven H. Low CMS, California Institute of Technology, Pasadena {zliu,mhlin,adamw,slow}@caltech.edu Lachlan L. H. Andrew Faculty
More information