High Density in the Data Center

Transcription

1 High Density in the Data Center Choosing and specifying density High density in an existing environment High density in a new environment Presented by: Christopher Eko Business Development - ASEAN All content in this presentation is protected

2 High density is a key factor in the challenges facing data centers Uncertain long-term plans for capacity or density Increasing availability expectations Rapid changes in IT technology Energy and service cost control pressure High density blade server power/heat Dynamic power variation Regulatory requirements Server consolidation & virtualization APC can help simplify the complexities of data center high density challenges

3 Power and cooling: The new #1 data center issue Rising densities have led to an increase in power and cooling issues 2008 IDC study

4 Challenges to COOLING Example: COOLING requirement for an 20 kw rack = 2,100 cfm per rack Would require 7 vented floor tiles per rack (7x more than normally allocated) Floor tile 300 cfm Floor tile 300 cfm Floor tile 300 cfm Floor tile 300 cfm Floor tile 300 cfm Floor tile 300 cfm Floor tile 300 cfm Requires substantial increases in aisle width and spacing between racks Clogged raised floors compound the problem White paper 46 Raised floor, perimeter-cooled data centers face practical limitation of approximately 5 kw (average) per rack

5 Challenges to POWER Breaker chaos Insufficient breaker positions Cascading panels Mislabeled or unlabeled breakers Becomes worse with redundancy Adding new voltage levels & receptacle types to the rack Increases need for hot work Trend towards minimizing hot work Getting more power to the rack Avoid tripping breakers White papers COMPLEXITY increases likelihood of downtime

6 Challenges to SPACE and CABLING Floor loading of fully-loaded high-density rack Sub-floor weight bearing limitations Supplemental raised floor pedestals may be required Airflow limitations of front and rear doors Cable chaos Risk of blocked airflow and human error from proliferation of cables in the rack Without effective high-density implementation, rack and cable challenges proliferate

7 First we need a standardized way to specify density Specifying Density Traditional way Same 500 kw data center but different average density depending on how calculated: 746 watts/ft watts/ft watts/ft watts/ft 2 Ambiguous Include access area around racks? Include back-room area? Consider total mains power consumption? White paper 120 A better way 5 kw/rack = Total IT power # of racks Eliminates the ambiguities of watts/ft 2 Allows different densities for different areas of the data center

8 Density specification principles Specifying Density Specify at the ROW level Rack: Too detailed Not enough information up front Room: Not detailed enough No flexibility for variation ROW: Just right! Use watts/rack Unambiguous Environment Lab Average watts/rack Medium (4-10 kw/rack) Typical peak-to-average ratios High (up to 4) Establish peak-toaverage ratio of approximately 2 Too low impacts IT flexibility and causes inefficiency Too high leads to oversizing Server intensive (Typical data center) Storage intensive Co-location Supercomputing Medium to high (4-15 kw/rack) Low (< 4 kw/rack) High (10-15 kw/rack) High (10-15 kw/rack) Medium (approx 2) Low (< 2) Medium (2) Low (close to 1) These 3 basic principles simplify density implementation

9 Example of how to create a clear density specification Specifying Density 1 Room layout with rack locations 2 Row-by-row specification Inventory of planned IT Strategic placement of IT 3 Design tools to use the specification to size rack/power/cooling elements White paper Designer 120

10 Four cooling strategies for high density A spectrum of strategies, depending upon level of investment and planning High investment Most planning Whole-room high density 4 Low investment Least planning High-density pod 3 White paper Suplemental cooling 2 46 Spread the load 1

11 Cooling strategies: Option 1: Spread the load Advantages Extends life of near-end-of-life data center Essentially free band-aid approach Disadvantages 1 Spread the load 2 Supplemental cooling 3 High-density pod 4 Whole room IT equipment placement can be surprisingly complex One addition of IT equipment can have surprising effects on other existing loads Difficult to figure out where the limits to growth are Uses the most floor space Data cabling issues Spread out high density equipment in the room Assessment services can help plan implementation and determine the limits of this strategy

12 Cooling strategies: Option 2: Supplemental cooling 1 Spread the load 2 Supplemental cooling 3 High-density pod 4 Whole room Advantages Can target high density (brings the solution TO the hot spot) Defers capital cost Disadvantages Room constraints could limit deployment May be constrained to available bulk cooling Air removal unit Air distribution unit Single AC unit Supplemental cooling devices are available and can help with targeted high-density equipment

13 Cooling strategies: Option 3: High-density pods 1 Spread the load 2 Supplemental cooling 3 High-density pod 4 Whole room Low-density room Advantages Maximum density capability (30 kw/rack) High-efficiency design Optimal floor space utilization Hot/cool air circulation is localized within the zone High-density pod HEAT OUT To building s heat rejection system Allows for targeted availability Effective long-term strategy No need for raised floor Disadvantages Needs to be planned for in advance Requires grouping of high-density equipment White paper 134 Easy-to to-implement, cost effective, high-efficiency solution

14 Cooling strategies: Option 4: Whole-room high density 1 Spread the load 2 Supplemental cooling 3 High-density pod 4 Whole room Advantages Handles all high density scenarios Smallest footprint Go Watch video about this purpose-built high-density data center Disadvantages Highly specialized and custom built Extreme capex compared to other options Can result in extreme underutilization of cooling capacity if not correctly engineered Not a widely adopted approach requires significant engineering for unique scenarios

15 Deployment strategies for cooling EXISTING data centers 1 Spread the load 2 High density is very small fraction of load Supplemental cooling High density is small fraction of load Location placement of IT equipment is flexible Open U-space in existing racks No flexibility over the placement of IT equipment Limited capital budget Dedicated high-density pods 3 4 High density is moderate to high fraction of load Whole-room high-density cooling Sub-section of data center is available for pod(s) White paper 46

16 Deployment strategies for cooling NEW data centers 1 Spread the load 2 Supplemental cooling Perimeter cooled room specified at < 6 kw/rack High density is small fraction of load Limited capital budget Dedicated high-density pods 3 4 Whole-room high-density cooling White paper 46 High density is moderate to high fraction of load Cooling redundancy requirements vary by equipment Future rack density requirements uncertain Large farms of high density servers (i.e. HPC) Location of all racks and density are known in advance Expensive to implement

17 Cooling strategies: High-density pod explained 1 Spread the load 2 Supplemental cooling 3 High-density pod 4 Whole room A mini data center with its own cooling Contributes no heat to rest of data center Works alongside existing room-based cooling Hot/cool air circulation localized within the pod by short air paths and/or containment Achieves optimum efficiency

18 Cooling IN the row, close to the load 1 Spread the load 2 Supplemental cooling 3 High-density pod 4 Whole room Hot-aisle air enters from rear, preventing mixing of hot and cool air Heat is captured and transferred to heat rejection system Cold air is supplied to the cold aisle Cooling units Operates on hard floor or raised floor

19 Standardized modular multi-rack high-density pod 1 Spread the load 2 Supplemental cooling 3 High-density pod 4 Whole room Air conditioners return ambient room-temperature air Hot air is exhausted to the hot aisle and returns to the back of the air conditioners Integral row-based air conditioners

20 Major efficiency benefits of row-based approach vs traditional room cooling Less air mixing Fans move only the air required by the IT equipment, instead of mixing the room air: 60% reduction in fan power Variable fan speed Fan speed dynamically tracks the actual IT load, instead of running at full speed: 50% typical further reduction in fan power Higher return temperature Air is captured by the CRAC at higher temperature, easing the transfer of heat to the heat rejection systems: 10% reduction in chiller power No rehumidification Air is processed by the CRAC at higher temperature, eliminating the energy associated with dehumidification / rehumidification: 10% reduction in CRAC power More economizer use Return water temp to chiller is higher, increasing the operating hours of economizer modes: 10% typical reduction in chiller power

21 Is there an efficiency vs density tradeoff? High density and high efficiency are related Many people think high density makes efficiency worse Only true when trying to push existing data centers to cool high density High density enables high efficiency in a properly designed new data center or a pod in an existing data center Shorter pipe lengths Less pump power Shorter air flow lengths Less fan power Less air mixing Higher return air temperatures Less humidification/dehumidification power and chiller power The highest efficiency data centers will be high-density data centers

22 Raised floor or not? Traditional: Competing uses of raised-floor space Air plenum for perimeter air conditioners Cables and piping for data, power, and water Modern: High-density design With raised floor - Water distribution only, with data and power cables overhead No raised floor - Data, power, and water overhead

23 POWER: High-density power distribution 415 volt distribution (Elimination of PDU transformers) Increased energy efficiency Decreased copper Smaller footprint Lower electric bill Less weight More space for IT racks Modular power distribution Hot swappable No need to predict future rack power requirements Allows higher power densities in distribution products Auto-sensing of breaker size and location

24 MANAGEMENT: Adding intelligence Knowing what s going on, in real time Smart management can: Analyze the effect of proposed changes Suggest the best place for adding new servers Recognize overloads or trends in time for corrective action Identify stranded capacity Locations where there is available POWER, COOLING or SPACE but not enough of the other two White paper 150

25 High density Summary Choosing and specifying density Row-level specification eliminates stranded capacity, prevents hot spots, and provides flexibility for future deployments. This leads to a high efficiency data center. High density in an existing environment Assessment of existing cooling and power environment helps determine best strategy to implement high density. Cooling solutions range from spreading out the load to high density pods. High density in a new environment An optimal data center physical infrastructure design strategy includes dedicated high-density pods with close-coupled cooling, 415 V power distribution, modular power distribution, and capacity & change management software.