Towards Energy Efficient Workload Placement in Data Centers

Transcription

1 Towards Energy Efficient Workload Placement in Data Centers Rania Elnaggar, Portland State University Abstract. A new era of computing is being defined by a shift to aggregate computing resources into large-scale data centers (DCs) that are shared by a global pool of users. In this paradigm, DCs' operational energy costs are a rising concern as they continue an upward trend that is poised to surpass the capital cost of equipment in a typical lifetime usage model. A DC is a complex distributed system comprised of a hierarchy of numerous components; thus, power and energy management can be performed at many levels of granularity and through various techniques. We contend that the energy efficiency problem in DCs should be addressed through a holistic, endto-end approach that accounts for the many, and sometimes-conflicting, parameters. In this paper, we discuss workload placement strategies as a model for a holistic approach. Earlier research that addressed workload placement, capitalized on a maximumidle approach that seeks to maximize both spatial and temporal idleness. We show that the underlying concept for that approach does not hold as a basis for energy-efficient placement; we investigate current and future system power expenditure with respect to system load and analyze the contributing factors. We then utilize our analysis to introduce a framework for energy-efficient load placement strategies in DCs. Comparing our approach to maximum-idle-based placement shows gains in compute energy efficiency. Finally, we discuss how our new approach affects DC thermals and the energy required for cooling. 1 Introduction Energy efficiency of has always been a first-class design goal in the mobile and embedded fields due to battery-life limitations. However, until recently, it has been less of a concern for servers and data centers. We witnessed waves of interest in DC energy efficiency with the advent of new technologies such as WWW, clusters, grid, utility compute models. The interest is now renewed with the inception of Cloud Computing [2] [8], given the massive scale anticipated in future DCs. Cloud Computing is charcterized by a move to aggregate computing resources (in terms of hardware, software and services) into large-scale data centers that are shared by a global pool of users. Those data centers are typically owned and run by third-party entities and export a wide array of services and applications ranging from individual consumer-oriented services to enterprise-class offerings.

2 In this new paradigm, computing energy costs are a rising concern. This is especially the case in DCs where energy costs continue an upward trend that is poised to surpass the cost of the equipment itself in a typical lifetime usage model [6]. In 2005, DCs consumed an estimated total of about 50 billion kwh in the U.S., and around 130 billion kwh for the world. These figures accounted for 1.2% and 0.8% of the total electricity consumption of the U.S. and the world respectively [24]. The U.S. Environmental Protection Agency (EPA) estimates that if current energy-efficiency trends continue, DCs will consume more than 120 billion kwh in the U.S. alone [13]. While cutting down operational costs of data centers is a chief concern, the environmental impact of this spiraling computing energy expenditure is equivalently important. It is projected that improving DC efficiency beyond current trends can reduce carbon dioxide emissions by 15 to 47 million metric tons in 2011 [13]. A data center is a complex entity that can be partitioned into two inter-dependent systems; the IT system and the facilities system. The IT system is comprised of compute related parts such as servers, networks and management components. The facilities system delivers the power and cooling needed by IT equipment as well as other facilities overhead such as lighting. As servers performance grows, they generate increasing amounts of heat [3] which in turn demands progressively demand more cooling [6]. In fact, cooling and power delivery energy expenditure already surpasses compute energy use in a great percentage of data centers [6]. DCs are usually cooled using Computer Room Air Conditioning (CRAC) units, and are typically arranged in a hot-aisle, cold-aisle configuration [47]. In this paper we contend that achieving improved energy-efficiency for a DC should be driven by a holistic that takes into consideration all system components to achieve maximum synergetic energy savings. This strategy governs a set of local policies and protocols that effectively use existing power-saving features within each system component in a way that is proportional to the workload of the component and of the overall system. Though ultimately, we are interested in defining an end-to-end global energy optimization strategy over a generalized model of a multi-level distributed system, the focus of our short-term research effort is on defining such a strategy for DCs, as a key subset of the larger problem. We consider a DC workload that is a mix of online and offline workloads that are mostly characterized as massively parallel, or throughputoriented. In defining such strategies, we will initially examine just the compute-related part of the DC, thus excluding networking, cooling and power delivery overheads. While we recognize that these overheads are critically important, we also observe that, in many cases, they are proportional to computing power expenditure in modern data centers. Expanding our work to explicitly consider those other components is a topic for future research. The rest of this paper is organized as follows. In section 2 we present research background and review related work. In section 3 we present our hypotheses and outline an investigation methodology. In section 4 we present results and analysis. In section 5 we introduce a framework for energy efficient workload placement. In section 6 we conclude the paper and present direction for future research.

3 2 Background A DC s compute system is comprised of a hierarchy of components of different scale, where power management can be performed at many levels of that hierarchy, as shown in Fig. 1 below. Fig. 1.. Scale and hierarchy of power management in a DC We recognize three power-related attributes that affect the average power consumption for each component, as well as, the overall system; namely: peak power, idle power and dynamic power range. Peak power is the power consumed at the maximum workload of a component. Idle power is the power consumed when a component has no workload but is powered-on and active, thus has low-latency latency response to increasing workload. The dynamic power range defines the distance between peak power and idle power, and it is desirable for it to scale proportionally to the workload of the component [3]. When addressing the power consumption problem in a DC, we identify two main cost components: the capital cost of power provisioning in the infrastructure, and the operational power cost during the life span of the DC. The capital cost component is directly related to expected maximum power consumption and is a pressing issue as more DCs are built and typically amortized over an average of 15 years. Large DC operators such as Google are exploring ways to cut down on that cost through a workload mix that keeps maximum utilization within a decreased power envelope [13]. Decreasing operational power, however, is our area of interest and of a large body of other research. Approaches to decrease operational (also termed average) power have focused on the three power attributes we identified earlier. The solutions proposed can be broadly classified into three categories: scaling solutions that track utilization levels; sleep solutions that shut off parts of the system at low utilization; and hybrid solutions that combine the two former approaches. On the component level, Dynamic Voltage and Frequency Scaling (DVFS) has been widely researched and applied to CPUs in particular as they consume a large percentage of the overall system power. Also, clock gating and system sleep states have been introduced for various platform components. These techniques have been utilized in wide-spread commercial products such as those produced by Intel [22] and AMD [1]. The Advanced Configuration and Power Interface (ACPI) [20] was defined to standardize power management for different platforms. Use of heterogeneous cores and accelerators has also been explored to enhance energy efficiency [27], and the use of operating system timers has been scrutinized to enable longer periods of uninterrupted sleep [44]. Conserving disk drive related energy consumption has also been explored through scaled-speed and sleep modes [9]. In the computer networks domain, the concept of maximizing idleness through sleep states has also been the corner stone of a large body of research that includes

4 21. Hwang, C., & Allen, C.: A predictive system shutdown method for energy saving of event-driven computation. ACM Transactions on Design Automation of Electronic Systems (TODAES),vol. 5, pp (2000) 22. Intel Xeon Processor 7400 Series datasheet, ucts_xeon7000+tab_techdocs. (2008) 23. Keeton, K., Kelly, T., Merchant, A., Santos, C., Wiener, J., Zhu, X., et al.: Don t settle for less than the best: use optimization to make decisions. In proceedings of the 11th USENIX workshop on Hot topics in Operating Systems. USENIX Association Berkeley, CA, USA (2007) 24. Koomey, J.: Estimating total power consumption by servers in the US and the world. Final report. (2007) 25. Meisner, D., Gold, B., & Wenisch, T.: PowerNap: Eliminating Server Idle Power. SIGPLAN Notices, vol. 44, pp ACM, New York, USA. (2008) 26. Mishra, N., Chebrolu, K., Raman, B., & Pathak, A.: Wake-on-WLAN. In proceedings of the 15th international conference on World Wide Web, pp ACM New York, NY, USA (2006) 27. Mogul, J., Mudigonda, J., Binkert, N., Ranganathan, P., & Talwar, V.: Using Asymmetric Single-ISA CMPs to Save Energy on Operating Systems. IEEE Micro, vol. 28, pp IEEE Press (2008) 28. Moore, J., Chase, J., Farkas, K., & Ranganathan, P.: Data center workload monitoring, analysis, and emulation. In proceedings of Eighth Workshop on Computer Architecture Evaluation using Commercial Workloads (2005) 29. Moore, J., Chase, J., Ranganathan, P., & Sharma, R.: Making scheduling cool : Temperature-aware resource assignment in data centers. In proceedings of Proceedings of the USENIX Annual Technical Conference (2005) 30. Nathuji, R., & Schwan, K.: Reducing system level power consumption for mobile and embedded platforms. In proceedings of the International Conference on Architecture of Computing Systems (ARCS) (2005) 31. Nathuji, R., & Schwan, K.: Virtualpower: Coordinated power management in virtualized enterprise systems. ACM SIGOPS Operating Systems Review, vol. 21, no. 6, pp ACM New York, NY, USA. (2007) 32. Patel, C., Sharma, R., Bash, C., & Graupner, S.: Energy Aware Grid: Global Workload Placement based on Energy Efficiency. In proceesings of IMECE ( 2003) 33. Pinheiro, E., Bianchini, R., Carrera, E., & Heath, T.: Load balancing and unbalancing for power and performance in cluster-based systems. In processing of Workshop on Compilers and Operating Systems for Low Power, vol. 180, pp (2001) 34. Qiu, Q., & Pedram, M.: Dynamic power management based on continuous-time Markov decision processes. In proceedings of the 36th ACM/IEEE conference on Design automation, pp ACM New York, NY, USA (1999) 35. Raghavendra, R., Ranganathan, P., Talwar, V., Wang, Z., & Zhu, X.: No" power" struggles: coordinated multi-level power management for the data center. In proceedings of ASLOPS (2008) 36. Rajamani, K., Lefurgy, C., Res, I., & Austin, T.: On evaluating requestdistribution schemes for saving energy in server clusters. In proceedings of the

5 IEEE International Symposium on Performance Analysis of Systems and Software, pp (2003) 37. Ramakrishnan, L., Irwin, D., Grit, L., Yumerefendi, A., Iamnitchi, A., & Chase, J.: Toward a doctrine of containment: grid hosting with adaptive resource control. In proceedings of the ACM/IEEE conference on Supercomputing. ACM New York, NY, USA (2006) 38. Ranganathan, P., Leech, P., Irwin, D., & Chase, J.: Ensemble-level power management for dense blade servers. In proceedings of the 33rd International Symposium on Computer Architecture (ISCA), pp IEEE Computer Society Washington, DC, USA (2006) 39. Ranger, C., Raghuraman, R., Penmetsa, A., Bradski, G., & Kozyrakis, C.: Evaluating MapReduce for multi-core and multiprocessor systems. In proceedings of IEEE 13th International Symposium on High Performance Computer Architecture, pp IEEE Computer Society Washington, DC, USA (2007) 40. Ren, Z., Krogh, B., & Marculescu, R.: Hierarchical adaptive dynamic power management. IEEE Transactions on Computers, vol. 54, pp IEEE Press (2005) 41. Rivoire, S., Shah, M., Ranganathan, P., & Kozyrakis, C.: Joulesort: a balanced energy-efficiency benchmark. In proceedings of the 2007 ACM SIGMOD international conference on Management of data, pp ACM New York, NY, USA (2007) 42. Sharma, R., Bash, C., Patel, C., Friedrich, R., & Chase, J.: Balance of power: Dynamic thermal management for internet data centers. IEEE Internet Computing,vol. 9, pp IEEE Press (2005) 43. Sharma, V., Thomas, A., Abdelzaher, T., Skadron, K., & Lu, Z.: Power-aware QoS management in web servers. In proceedings of 24th IEEE Real-Time Systems Symposium, pp IEEE Press (2003) 44. Siddha, S., Pallipadi, V., & De, A. V.: Getting maximum mileage out of tickless. In proceedings of Linux Synposium, pp (2007). 45. Simunic, T., Benini, L., Glynn, P., & Micheli, G. D.: Event-driven power management. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 20, pp IEEE Press (2001) 46. Srivastava, M., Chandrakasan, A., & Brodersen, R.: Predictive system shutdown and other architectural techniques for energy efficient programmable computation. IEEE Transactions on Very Large Scale Integration (VLSI) Systems,vol. 4, pp IEEE Press (1996) 47. Sullivan, R.: Alternating cold and hot aisles provides more reliable cooling for server farms. Uptime Institute (2000) 48. Tang, Q., Gupta, S., & Varsamopoulos, G.: Energy-Efficient Thermal-Aware Task Scheduling for Homogeneous High-Performance Computing Data Centers: A Cyber-Physical Approach. IEEE Transactions on Parallel and Distributed Systems, vol. 19, pp IEEE Press (2008) 49. Tolia, N., Wang, Z., Marwah, M., Bash, C., Ranganathan, P., & Zhu, X.: Delivering Energy Proportionality with Non Energy-Proportional Systems-- Optimizing the Ensemble. In proceedings of the 1st Workshop on Power Aware Computing and Systems (HotPower) (2008)

6 50. VM Ware, Weissel, A., & Bellosa, F.: Dynamic thermal management for distributed systems. In proceedings of the First Workshop on Temperature-Aware Computer Systems (2004) 52. Xen, Yavatkar, R., & Krishnamurthy, L.: Method and apparatus for managing energy usage of processors while executing protocol state machines. US Patent App. 10/056,160 (2002)