The Importance of Cabinet-Level Power Monitoring in a Data Center Jeff Miller Major Account Manager Geist Manufacturing
What is metering/monitoring Hardware and software methods to identify power levels and usage. Track efficiency improvements to continuously track power usage effectiveness (PUE) and data center efficiency (DCE). Power usage effectiveness (PUE) = total facility power total IT equipment power Data center efficiency (DCE) = DCE = total IT equipment power total facility power 1 PUE
What is metering/monitoring Total facility power Total facility power is defined as the power measured at the utility meter that is dedicated solely to the data center. IT equipment power IT equipment power is defined as the effective power used by the equipment that manages, processes, stores, or routes data within the raised floor space. Source = Increasing Data Center Efficiency through Metering and Monitoring Power Usage, Intel, June 2009
Why is this important? If you cannot measure it, you cannot improve it! - Lord Kelvin The Right Honorable Lord Kelvin was the first President of the International Electrotechnical Commission (IEC), which was founded in 1906. Corollaries derived from Lord Kelvin s theorem - You can t manage what you can t measure - What gets measured gets improved - Increase efficiencies, reduce costs of your IT.
Location, Density, Power, Heat Data Center Trends Approximately 30% of a company s critical infrastructure assets are in unmonitored, suboptimal locations Power density in the data center has continued to grow, increasing the need for efficient heat dissipation measures
Market Trends Power to the Rack The market trend is for data center cabinets and racks to require more and more energy as the increasing demand put upon data centers drives the need for greater system performance and higher server density. Timeframe Average Watts/sq. ft. Average kw/rack 2003 40 2 2005 80 4 2007 240 15 Source: Surveys by 7x24 Exchange, Lawrence Berkeley Labs, Uptime Institute estimates based on gradual implementation of new server technology that is either already announced or currently on the market
Market Trends Blade Servers on the Rise Server consolidation, virtualization, and power savings are leading to a boom in blade server sales over the next four years. WW blade server shipments are expected to rise from 620,000 in 2006 to 2.4 million units in 2011 - a compound annual growth rate of 31.5% In 2006 blade servers represented only 7.9% of all servers sold By 2011 the sales of blade servers are expected to rise to approximately 21.6% of all servers sold Blade Servers 92.1% 7.9% Other Servers 78.4% 21.6% 2006 2011 Source: isuppli
Environmental Monitoring By design, blade servers concentrate computational power in a smaller space, leading to Higher electrical power usage More heat generated in a smaller space A rack of blade servers can generate as much as 20 kilowatts of heat output (about the amount of heat given off by three household electric ovens) Heat is a reliability issue that must be addressed For every increase of 10ºC above 20ºC, long term reliability is reduced by 50% Computer equipment ages faster when it gets hot!
Environmental Monitoring The average wiring closet contains $150,000 to $300,000 of equipment, with a typical small server room housing more than three times that amount Downtime costs for most companies can run anywhere from $10,000 to $100,000 an hour! Most companies spend 70%-90% of their infrastructure budgets maintaining existing networks
You Asked I have benefited from measuring the current draw in my deployment -- however, I would like to be able to monitor additional factors that affect the performance of my equipment, such as Temperature Airflow Humidity & Moisture Security
PDU Level Meter Options Local Ammeter Amps only or Scrolling Meter Can be added to almost any PDU Remote Ammeter Onboard AMP Meter Remote Monitoring at circuit, phase and/or receptacle levels SNMP Alarming and XML Scripting built in Remote Management/Control Remote Power and Environmental Monitoring Includes Remote Reboot/Management SNMP Traps and XML built in Monitor at outlet and circuit levels
Power Meter: Diagnostic Tool Measure phase loads & neutral loads for proper balance - Correcting unbalanced phase loads and minimizing the neutral load can result in electric utility bill savings Monitor breaker loads - Ensure that current draw is safely below breaker ratings to avoid unnecessary operational interruptions Measure current usage at each socket - Identify what equipment has high current demands and may be in need of service or replacement - Within an organization, departments can be billed by the IT department for equipment energy use
Why cabinet monitoring instead of RPP A Remote Power Panel (RPP) monitors the total power (current) of the branch circuit. Power draw of the equipment powered through the cable that is run from the circuit panel to the rack. A 30A PDU distributing to 15A or 20A receptacles must be broken down into either 15A or 20A independent circuits internally. By opting for 20A internal circuits, PDU circuit balance is less critical. One circuit may be loaded to >15A. This would not be possible if each breaker were rated at 15A.
Why cabinet monitoring instead of RPP An RPP does not know how much power is on each internal PDU circuit! This exposes you to load imbalance and unintended breaker overload Power Panel In Rack PDU Circuit A Circuit B Breaker A Breaker B 30-Amp breaker
Why cabinet monitoring instead of RPP Whenever a PDU has more than one breaker, an RPP is not be able to distinguish the power load difference between the PDU s breakers. Amps (not derated) Volts (listed) Volts (actual) # of Phases Watts (listed) Watts (derated to 80%) Kilowatts (V x A x 1000) Kilowatts (derated to 80%) Breakers 30A 125V 120V 1 3600W 2880W 3.6kW 2.9kW 2 30A 250V 208V 1 6240W 4992W 6.2kW 5.0kW 2 30A 120/208V 208V 3 10795W 8636W 10.8kW 8.6kW 3 (DP)
Single-Phase Power Single-phase power is like a 1-cylinder diesel engine. Power is produced once each cycle. Single Phase
3-Phase Power A 3-cylinder diesel engine has one cylinder firing with each 120 degree rotation of the crankshaft so that 3 power pulses are generated for each full cycle. Like the 3-cylinder diesel engine, a 3-phase power feed has 3 power lines that are offset from each other by 120 degrees. Three Phases
3-Phase Delta Configuration 3-Phase Delta A 3-phase delta power configuration consists of 4 conductors - 3 hot lines 1 ground line. In a delta configuration there is only one output voltage level available. 3-Phase 208V Delta Line 1 208V by wiring one hot line to another hot line (green arrows) 208V 208V Line 2 Ground 208V Line 3
3-Phase WYE Configuration 3-Phase WYE A 3-phase WYE power configuration consists of 5 conductors - 3 hot lines 1 ground line 1 neutral line The neutral line in the circuit is what makes the 3-phase WYE configuration flexible and increasingly popular, as it allows the output of both 208V and 120V from the same PDU without using a transformer or multiple whips. 3-Phase 120V/208V WYE Line 1 208V by wiring any hot line to another hot line 120V 208V 208V (green arrows) Line 2 120V by wiring any hot line to neutral (brown arrows) Ground 120V 120V Neutral 208V Line 3
3-Phase Power Makes Sense Why bring 3-phase power to the cabinet level? Less wire under the floor to block airflow Fewer whips for the electrician to pull, reducing installation cost Ability to power 120V and 208V loads from the same branch circuit Simpler to balance loads within a cabinet rather than across cabinets
The 3-Phase Advantage Equal Power Example 20A 120V 3-Phase System Power = Voltage x Current = [120V x 20A] x 3 phases = 2400W per phase x 3 phases = 7200W or 7.2kW 5 wires per whip x 1 whip = 5 wires 1 circuit = 1 whip WHY USE 3 WHIPS WHEN A SINGLE 3-PHASE WHIP CAN DO THE JOB? 20A 120V Single-Phase System Power = Voltage x Current = [120V x 20A] x 3 circuits = 2400W per circuit x 3 circuits = 7200W or 7.2kW 3 wires per whip x 3 whip = 9 wires 3 circuits = 3 whips
The 3-Phase Advantage More Power Less Current Example 7.2kW 20A 208V 3-Phase System Power = Voltage x Current x 1.73 = 208V x 20A x1.73 = 7197W or 7.2kW NO CIRCUIT BREAKERS REQUIRED 4.2kW 6.2kW 30A 208V Single-Phase System Power = Voltage x Current = 208V x 30A = 6240W or 6.2kW REQUIRES 2 2-POLE CIRCUIT BREAKERS 20A 208V 1-Phase 30A 208V 1-Phase 20A 208V 3-Phase 20A 208V Single-Phase System Power = Voltage x Current = 208V x 20A = 4160W or 4.2kW NO CIRCUIT BREAKERS REQUIRED 20A 3-Phase Benefits > 70% more power than 20A single-phase > 15% more power than 30A single-phase No circuit breakers required Requires only a 20A feed
3-Phase Power Guidance Examples of preferred arrangements for branch circuits For a medium density rack of up to 7.2kW a 20A 120V/208V 3-phase PDU serves the rack well. For a higher density rack of up to 10.8kW a 30A 120V/208V 3-phase PDU with circuit breakers can handle the larger load
What to do with the data Collected data needs help to become information Hardware aggregator Software interface Utilize a system that collects multiple protocols SNMP, Modbus, etc. Can now compare power draw and environmental data. Is everything as it is supposed to be? How do I fix one, without affecting the other? Load balance power and cooling, are my highest power draw cabinets the hotest, or coldest!
Questions
thank you