Minimization of Costs and Energy Consumption in a Data Center by a Workload-based Capacity Management

Minimization of Costs and Energy Consumption in a Data Center by a Workload-based Capacity Management Georges Da Costa 1, Ariel Oleksiak 2,4, Wojciech Piatek 2, Jaume Salom 3, Laura Siso 3 1 IRIT, University of Toulouse 2 Poznan Supercomputing and Networking Center 3 IREC, Institut de Recerca en Energia de Catalunya 4 Poznan University of Technology E2DC, Cambridge, 10/06/14 1

Outline Data center model Workload-based dynamic power capping Workload-based dynamic power capping for variable power supply E2DC, Cambridge, 10/06/14 2

Problem and motivation Capacity management Finding such DC configuration that space, power and cooling capacity is maximized Additional goals Minimization of energy use, OPEX, CAPEX Issues Capacity management based on server nameplate leads to overprovisioning The approach Capacity management based on workload, tuned by dynamic power capping DC model that include both workload and cooling needed E2DC, Cambridge, 10/06/14 3

A holistic approach to simulate data center DATA CENTER MODEL E2DC, Cambridge, 10/06/14 4

Integrated analysis of software, IT equipment, and cooling Metrics Calculation CFD Simulation Power used 460 440 420 400 380 360 340 320 300 280 260 Daemon output Real output Linpack 4c 10:58 10:59 10:59 11:00 11:00 11:01 11:01 11:02 11:02 Date\nTime Workload & Resource Simulation Hardware & Software Modeling 5 E2DC, Cambridge, 10/06/14

Rack Node group (e.g. blade center) Node (server) Processor Core (if power and load are known) Power use modeling IT P RACK = ( n å i=1 P NODE_GROUP = å P NODE + åp FAN P NODE = å P CPU + P RAM + P NET i=1 P P X CPU (L) = P P X idle +(P P X max - P P X idle) L 100 P CPU = P idle + l P NODE_GROUP n L åp Ci P C = P C maxc i=1 l i=1 + c) /h PSU m j=1 m å j=1 100 E2DC, Cambridge, 10/06/14 6

Power use modeling P DATA_CENTER = n å P RACK i=1 + P FANSDC + P COOLING + P OTHERS P FANSDC = Dp*V air total h f h f - efficiency of fans P OTHERS = a * n å i=1 P RACK α percentage of power used by UPS, PDU, lighting, etc. E2DC, Cambridge, 10/06/14 7

Cooling models E2DC, Cambridge, 10/06/14 8

Power use modeling cooling Q P (t) = cooling chiller (t) EER(t) EER - Energy Efficiency Ratio for a chiller EER improves with higher inlet temp (T R_in ) EER(t) ~ T ev T ev = T R_in - DT h-ex EER improves with higher cooling capacity (Q cooling_rated ) EER(t) ~ 1 PLR(t) PLR(t) = Q cooling(t) Q cooling_ nom Q cooling_nom ~ Q cooling_rated E2DC, Cambridge, 10/06/14 9

An approach to reduced energy use, OPEX and CAPEX WORKLOAD-BASED DYNAMIC POWER CAPPING E2DC, Cambridge, 10/06/14 10

Power capping: ensuring that overall power use of a system does not exceed given thresholds Supported by hardware and software (DCIM) vendors (P- States and clock throttling) Various levels and types of capping (e.g. HP) Power capping E2DC, Cambridge, 10/06/14 11

Adaptation to workload by Dynamic power capping Cooling managemen t (temp.) Set power caps to Avoid increase of energy use by IT Keep mean completion time below threshold Workload-based dynamic power capping Theoretical peak power Actual peak power E2DC, Cambridge, 10/06/14 12

Minimizing energy consumption by power capping t ò 2 (t)- i excess E = max(0, IT i Pi t1 t 2 ò reserve E = max(0, IT i i t1 PC -P excess E < reserve i i E i PC )dt (t))dt E2DC, Cambridge, 10/06/14 13

Power capping algorithm E2DC, Cambridge, 10/06/14 14

Power capping algorithm E2DC, Cambridge, 10/06/14 15

Power capping algorithm E2DC, Cambridge, 10/06/14 16

Power capping algorithm E2DC, Cambridge, 10/06/14 17

Power capping algorithm E2DC, Cambridge, 10/06/14 18

Power capping algorithm E2DC, Cambridge, 10/06/14 19

Power capping algorithm, part2ś E2DC, Cambridge, 10/06/14 20

Power capping algorithm, part2 E2DC, Cambridge, 10/06/14 21

Power capping algorithm, part2 E2DC, Cambridge, 10/06/14 22

Power capping algorithm, part2 E2DC, Cambridge, 10/06/14 23

Simulation studies EXPERIMENTS AND RESULTS E2DC, Cambridge, 10/06/14 24

Simulation experiments Three cases: Experiment A: Load Balancing strategy, reference case Experiment B: Workload-based Power Capping, allowing server inlets up to 27 C (servers far from CRAC) Experiment C: Workload-based Power Capping, allowing server inlets up to 27 C (servers far from CRAC), Smaller cooling capacity used: 180[kW] Workload: Nr of tasks: 1280 batch rendering tasks Load: Mean ~ 25% [0% - 75%] Arrival rate: According to 8 different Poisson distributions Overall mean ~ 7s [1s 205s] E2DC, Cambridge, 10/06/14 25

Eight racks real server room 4414 cores Case based on rendering farm CFD simulations applied to check the CRAC outlet temp. increase Simulation experiment E2DC, Cambridge, 10/06/14 26

Simulation results [kwh] 100 Energy [kwh] 600 Energy 80 500 60 40 20 A B C 400 300 200 100 A B C 0 Total cooling device energy consumption 0 Total energy consumption 250 [kw] Power [s] 8000 Time 200 150 100 50 A B C 6000 4000 2000 A B C 0 Mean rack power Mean power Max rack power Max power 0 Mean completion time Mean task execution time E2DC, Cambridge, 10/06/14 27

Simulation results Metrics Cooling energy reduction by 38% PUE decrease by 5% Total energy use by 4% E2DC, Cambridge, 10/06/14 28

Cooling CAPEX reduction up to 25% Power infra CAPEX reduce by 10% Cooling + power infra up to 14% Total CAPEX reduction 4% / 7% Simulation results CAPEX E2DC, Cambridge, 10/06/14 29

Issues Limited reduction of energy use caused by: Chiller partial load characteristics (EER-PLR curve) Simplified model provides lower estimations of savings than real ones In the studied case cooling is relatively small part ~15% Need to run CFD to investigate detailed impact E2DC, Cambridge, 10/06/14 30

Reducing energy costs for variable power supply WORKLOAD-BASED POWER CAPPING FOR DEMAND-RESPONSE E2DC, Cambridge, 10/06/14 31

Power capping for DRM Demand-Response Management (DRM): Adaptation of DC configuration to changing demand and supply Changing prices of energy depending on a period and agreed power use limit Power capping as a technique to manage demand and minimize costs E2DC, Cambridge, 10/06/14 32

Application to demand-response management Regular price for energy: 0.0942/kWh Agreement: not exceed 200kW Otherwise: the cost of 1 kwh = 0.15/kWh Yearly savings of 45k euros [euros] 150 Total energy cost [euro] 0,15 Average energy price 8000 [s]mean completion time 100 0,1 6000 50 0 no power capping mix 0,05 0 no power capping mix 4000 2000 0 no power capping mix E2DC, Cambridge, 10/06/14 33

Conclusions Holistic model to DC modeling including workloads and cooling Along with simulations tools (DCworms) Workload-based dynamic power capping led to Up to 38% reduction of cooling energy and OPEX reduction (>4% of total) Up to 25% decrease of cooling and 14% of cooling and power infrastructure in CAPEX (7% of total) ~25% OPEX reduction for dynamic energy prices Next steps Model improvements, validation, other policies E2DC, Cambridge, 10/06/14 34

Questions? E2DC, Cambridge, 10/06/14 35