Best Practices Server: Power Benchmark
Rising global energy costs and an increased energy consumption of 2.5 percent in 2011 is driving a real need for combating server sprawl via increased capacity and higher frequency memory modules to meet server needs for on-demand scaling at lower power. [1] Figure 1.World primary energy consumption [1] Figure 2. Typical power used by office equipment [2]
As shown in Figure 2, servers are typically the biggest power consuming computing platform in an organisation due to their increased processing performance compared to a standard desktop computer or portable computer. Server component configuration, therefore, plays an important role in reducing power consumption while still meeting increased client computing demands.[2] Figure 3. System average power consumption [3] Managing the power consumption of a server requires a component level breakdown as shown in Figure 3 by which means we can then identify the memory component being the third highest consumer of power. [3] To combat rising energy costs and reduced power allowances, companies are scrambling to consolidate servers to efficiently utilize their multi-core processor architecture and large memory addressing capabilities by operating servers at their peak performance 24 hours a day, 7 days a week and 365 days a year through virtualization. Balancing target memory allocation, host memory over- commitment per virtual machine versus the efficiency at which those resources are then utilized and most importantly, at what cost to the company, impacts the server Total Cost of Ownership (TCO) and overall Quality of Service (QoS) to clients. [4] By obeying the three rules below we can easily reduce power usage while increasing capacity to meet scaling demands in new or existing servers 1. Less DIMMs (Dual Inline Memory Module) use less power - when possible install the least amount of DIMMs to reach your application memory capacity needs 2. Quad-rank DIMM have a lower power usage per Gigabyte (GB) than any other DIMM type 3. Configure the server to drive memory frequency at the slowest permissible frequency for additional power savings
To understand exactly how the consumption of power scales on newer generation servers using DDR3 technology DRAM (Dynamic Random Access Memory) the following results have been compiled for analysis. Power consumed per memory bank In Figure 4, the total server power consumption of three scenarios using dual-rank memory configurations put under load with PassMark BurnInTest 7.1 Pro on an Intel Romley server platform are measured with the integrated Hewlett-Packard ilo Management Engine to verify the increased power consumption in Watt (W) using 1 DPC (DIMM per channel), 2 DPC and lastly, 3 DPC. [5] [6 Figure 4. Total server power consumption per DPC under full memory load* As anticipated, adding more DIMMs per channel increases the overall server power consumption, total memory capacity and removes any future upgrade potential since all memory sockets are now populated. The addition of a second bank of memory per processor (2 DPC) adds an additional ~10.5% need for power, while the third increases with ~5%. With each additional DPC upgrade to a server past the first DPC, the total power consumed and Total Cost of Ownership (TCO) increases proportionately. Once all three DPC are populated there are no further direct memory upgrade options available to meet scaling client demands. Power consumed - dual- versus quad-rank Alternatively to upgrading past the first DPC, replacing the entire memory configuration with quad-rank configured memory DIMMs running at 800MHz memory frequency enables access to a larger memory capacity, up to twice as much as the 2 DPC configuration in Figure 4, while automatically being switched to 1.35V and thus consuming only 2 Watts more electricity or ~4% less power under load compared to a 3 DPC populated system with 384GB of memory capacity as illustrated in Figure 5. Using quad-rank parts, not only do we get more addressable memory capacity per server but also reduced power usage under load and thus added power savings.
Figure 5. Total server power consumption using dual-rank versus quad-rank under full memory load* Power consumed at similar capacity The initial rollout of a server is possibly the most important as the server must be pre-configured for the anticipated workload and while some servers are best served with immediate upgrades to either maximum capacity or frequency to suite different workloads, anticipating the memory requirements can be difficult if the value of power savings is unknown. In Figure 6 we highlight the power consumption of a single server rollout with 256GB of DDR3-1600 Dual-rank and DDR3-1066 Quad-rank memory running at two frequencies, a higher performance 1600MHz and power efficient 1066MHz. In a quad-rank memory population the server has future direct-upgrade potential to 512GB to meet increased demand whereas a dual-rank memory population limits the direct-upgrade potential to only 384GB. Figure 6. Total server power consumption using dual-rank versus quad-rank under full memory load*
Initially configuring a 1U blade server using quad-rank instead of dual-rank memory modules up to 256GB reduces the system power consumption by 13 Watts under load, equivalent to 6% of the dual-rank memory module populated server total power consumption. Based on the current 2013 summer electric rate schedule from Pacific Gas and Electric Company to commercial customers in the State of California, USA, of ~21 /kwh (United States Cents per kilowatt-hour), saving a total of 13 Watts system power reduces the operating costs of the server under full load 24 hours a day from US$ 33.26 per month (5.28 kwh per day*30 days*21 ) to US$ 31.30 per month (4.968 kwh per day*30 days*21 ), a 6% cost saving per month per server! [7] Enough power savings in Watt that, if thirty two 1U servers were deployed in a standard 42U rack using only quad-rank memory modules running at full memory load, the difference in power usage could feed two of the included 1U servers or be re-allocated to the power budget, depending if the rack is located on a hot or cold aisle, to cooling. Power consumed while in idle In some scenarios not all servers fitted to a rack may be utilized to full load 24 hours a day and may, in fact, consume more power sitting in idle or processing a low volume of work only during certain parts of the day, possibly for load balancing in a failover cluster. In these cases the use of quad-rank memory can reduce power consumption down to 9 Watts less than a similar capacity dual-rank memory equipped server running at idle as shown below in Figure7. Figure 7. Total server power consumption under load and idle* At 21 /kwh, saving a total of 9 Watts system power reduces the operating costs of the server in idle 24 hours a day from US$ 23.59 per month (3.744 kwh per day*30 days*21 ) to US$ 22.23 per month (3.528 kwh per day*30 days*21 ), a 6% cost saving per month per server! [7] Calculating the projected electrical operating costs over a single server life-cycle at 3, 5 and 10 years using a static electric rate we can observe in figure 8 that the reduced server power consumption of using quad-rank memory modules can effectively pay for an additional server in multi-server deployment scenarios.
Figure 8. Projected server operating costs at 21 /kwh over a typical 3, 5 and 10 year server life-cycle* Conclusion Using the formula in Figure 9 and integrated server management tool the cost of any given server in an organisation can be calculated by multiplying the power consumed by the system at the power supply in Watts by the active time in hours per day(i.e. 0.5 hours for 30 minutes) and divide it by a value of 1000 (kilo) to establish the standard kilowatt-hour consumed by the system within the given time period (i.e. 30 minutes) during a day. Figure 9.kWh formula Multiplying this value by days in the month or year that the system is active, the long term operating costs of a system can be projected based on the current or anticipated future electric rate schedule as shown in Figure 10 and 11. Figure 10. Operating cost per month Figure 11. Operating cost per year
Based on the above results quad-rank memory modules are, on a per gigabyte basis, more power efficient and allow direct future memory upgrades to reduce the total system power consumed at the same capacity and higher than by using dual-rank memory modules. They are therefore a clear choice for organizations seeking to maximizememory capacity while minimizing power consumption. *Test system: SiSoftwareBurnInTest 7.1 Pro on Intel Romley platform HP Proliant ML350p Gen8 with two Intel Xeon E5 2650 processors and up to 256GB, 384GB or 512GB of memory (Dual-rank KTH-PL316/16G or quad-rank KTH-PL310QLV/32G) installed.tested in HP performance mode. Intel Hyper-threading technology disabled. References [1] Statistical Review 2012, BP p.l.c. http://www.bp.com/sectiongenericarticle800.do?categoryid=9037130&contentid=7068669 [2] Electricity Used by Office Equipment and Network Equipment in the U.S., University of California http://enduse.lbl.gov/info/lbnl-45917b.pdf [3] Power Management in Intel Architecture Servers, Intel Corporation http://download.intel.com/support/motherboards/server/sb/power_management_of_intel_architecture_servers.pdf [4] Understanding Memory Resource Management in VMware ESX Server, VMwareInc. http://www.vmware.com/files/pdf/perf-vsphere-memory_management.pdf [5] BurnInTest Professional edition V7.1,PassMark Software http://www.passmark.com/ [6] HP ilo (Integrated Lights-Out) 4, Hewlett-Packard http://h18000.www1.hp.com/products/quickspecs/14234_div/14234_div.pdf [7] A-1 Electric Rates schedule (commercial rate) 2013, Pacific Gas and Electric Company http://www.pge.com/nots/rates/tariffs/commercialcurrent.xls 2013 Kingston Technology Corporation, 17600 Newhope Street, Fountain Valley, CA 92708 USA. All rights reserved. All trademarks and registered trademarks are the property of their respective owners. MKF-555