COMPARATIVE ANALYSIS OF WIDELY USED AIR CONTAINMENT SYSTEMS IN COMMERCIAL DATA CENTERS FOR MAXIMUM COOLING PERFORMANCE KASTURI RANGAN RAJAGOPALAN

Transcription

1 COMPARATIVE ANALYSIS OF WIDELY USED AIR CONTAINMENT SYSTEMS IN COMMERCIAL DATA CENTERS FOR MAXIMUM COOLING PERFORMANCE by KASTURI RANGAN RAJAGOPALAN Presented to the Faculty of the Graduate School of The University of Texas at Arlington in Partial Fulfillment of the Requirements for the Degree of MASTERS OF SCIENCE IN MECHANICAL ENGINEERING THE UNIVERSITY OF TEXAS AT ARLINGTON AUGUST 2013

2 Copyright by Kasturi Rangan Rajagopalan 2013 All Rights Reserved

3 ACKNOWLEDGEMENTS I am really thankful to my advisor Prof. Dereje Agonafer for his continuous guidance and support over the last two years of my research and study at the University of Texas at Arlington. It has been an awesome learning process for me under his expertise in terms of understanding the core concepts of engineering and in order to become a wonderful CFD engineer. His inspiration and motivation is totally an unforgettable experience. I would like to thank Prof. Haji Sheikh and Prof. Kent Lawrence for serving to be a part of my committee. I would also like to take the opportunity to thank Deepak Sivanandan CommScope Inc. and other Industry mentors for all their continuous support and feedback in the projects I have worked. I am obliged to Ms. Sally Thompson who has been of wonderful help in assisting me in all official procedures. I also want to thank the entire EMNSPC team, especially Betsegaw Gebrehiwot. Next I would like to thank Fahad Mirza, John Fernandez, Richard Eiland and Marianna Vallejo for all their guidance provided. I would also like to acknowledge the help and support extended by all my friends and colleagues who have made my stay at UTA a great venture. Finally, this acknowledgement would not be complete without mentioning my parents, Mr. Rajagopalan and Mrs. Mala Rajagopalan who have served as my motivation and inspiration. I am grateful to them for providing their continuous support to pursue my goals. July 08, 2013 iii

4 ABSTRACT COMPARATIVE ANALYSIS OF WIDELY USED AIR CONTAINMENT SYSTEMS IN COMMERCIAL DATA CENTERS FOR MAXIMUM COOLING PERFORMANCE Kasturi Rangan Rajagopalan, M.S. The University of Texas at Arlington, 2013 Supervising Professor: Dereje Agonafer Many data centers implement containment systems to improve cooling system performance. Literature review shows that Hot Aisle Containment System (HACS) have higher supply air temperature set point than Cold Aisle Containment Systems (CACS) [1]. CACS in data centers have claims to support higher density loads when compared to HACS systems [2]. Data center owners suggest that containment of a group of racks can be a very effective technique to improve cooling performance. On the other hand, others suggest that fewer area for containment is better than no containment in data centers that favours the partially contained architectures of HACS and CACS [3].The optimum containment architecture in terms of cooling and saving power is becoming imperative for every data center nowadays. In this study, a data center from published literature is modeled in a commercially available computational fluid dynamics (CFD) tool and simulation results of this model are validated with the published paper [4]. The data center is a hot aisle/cold aisle raised floor data center. Cooling is provided with one computer room air conditioning (CRAC) unit to the data center room. The raised floor data center is implemented with various containment architectures iv

5 ranging from partial containment to full containment of racks. There are eight configurations that are compared in terms of fan power and Inlet supply temperature for the servers. The objective of this study is to compare the cooling performances for these eight cases and determine the reasons behind them and also come up with new guidelines for the data center owners. v

6 TABLE OF CONTENTS ACKNOWLEDGEMENTS.....iii ABSTRACT... iv LIST OF ILLUSTRATIONS... viii LIST OF TABLES...xii Chapter Page 1. INTRODUCTION TO DATACENTER COOLING Modern Data Centers Types of Data Centers Literatures in Data Center Cooling PRIME CHALLENGES FACED IN THE DATACENTER Venturi Effect Recirculation of Airflow Bypass Airflow LITERATURE REVIEW ON CONTAINMENT TECHNIQUES Hot Aisle Containment System Cold Aisle Containment System Hot aisle / Cold aisle Partial Containment System Merits of the Containment System Reasons for Containment System vi

7 4. SELECTION OF AIR CONTAINMENT SYSTEMS Logical Selection of Air containment systems Conditions recommended for Ducted HACS Conditions recommended for CACS CFD PROBLEM SPECIFICATION Problem Description Over Heated Server Sets Fan power DATA CENTER CFD MODELING Cabinet Model Description CRAC Model Description Ceiling drop duct specification Air Containment Systems CFD Models RESULTS AND ANALYSIS Case 1 Results and Analysis Case 2 Results and Analysis Case 3 Results and Analysis Case 4 Results and Analysis Case 5 Results and Analysis Case 6 Results and Analysis Case 7 Results and Analysis Case 8 Results and Analysis Summary of Analyzed Results for Eight Cases CONCLUSION AND FUTURE WORK vii

8 REFERENCES BIOGRAPHICAL INFORMATION viii

9 LIST OF ILLUSTRATIONS Figure Page 1. A Sun Microsystems datacenter Airflow management in a datacenter Fluid flow shown in Venturi tube Pressure differential in the Venturi tube Recirculation of Airflow Bypass Airflow Leakage through cable outlets Under floor airflow path blocked by cables Hot aisle containment system Cold aisle containment system Partial Containment Air flow management Partially contained (End of Aisle) Selection of Air Containment Systems for the comparison study Pros and Corns of Air Containment Systems for the comparison study Ducted Hot Aisle Containment System Cold Aisle Containment System (CACS) Cabinet front and rear Equation for calculation of Fan power The side view of the CRAC unit CFD model Equation for flow rate through the racks Cabinet simple model ix

10 22. System resistance curve for the experimental server System resistance curve for the advanced resistance model Fan curve for the advanced resistance model Equation for the advanced resistance model Reduced equations for the advanced resistance model CRAC unit CFD model Ceiling drop duct in raised floor datacenter CFD model Isometric view of the data center room layout Case 1 Hot aisle / cold aisle datacenter Case 2 Data center contained with both the cold aisle Case 3 Data center contained with first cold aisle alone Case 4 Datacenter contained with second cold aisle alone Case 5 Data center contained with all the three hot aisles Case 6 Data center contained with first and second hot aisle alone Case 7 Data center contained with first and third hot aisle alone Case 8 Data center contained with second and third hot aisle alone Server sets which are considered as over heated The maximum temperature observed in server sets is 35 C The pressure surface plot for the cabinet showing maximum temperature The velocity surface plot for the cabinet showing maximum temperature Temperature surface plot X axis through midsection cabinets A3- D Cabinets which are considered to be over heated in case Temperature surface plot X axis through A3- D Temperature surface plot for the cabinet showing maximum temperature The pressure surface plot for the cabinet showing maximum temperature x

11 47. The velocity surface plot for the cabinet showing maximum temperature Cabinets which are considered to be over heated in Case Temperature surface plot X axis through cabinets A3 - D Temperature surface plot X axis through cabinets A1-D Temperature surface plot X axis through cabinets rows A3-D Temperature surface plot X axis through midsection cabinets A1-D The server sets in the cabinet which show higher temperature in case The temperature surface plot for the cabinet with maximum temperature The pressure surface plot for the cabinet exhibiting maximum temperature The velocity surface plot for the server set in cabinets showing 26ºC Temperature surface plot through X axis midsection cabinets through A3-D Temperature surface plot through X axis midsection cabinets through A1-D xi

12 LIST OF TABLES Table Page 1. Cabinet Model Specifications Simple model Cabinet Model Specifications - Advanced Resistance model Pressure drop at various fan speeds across the servers CRAC Model Specification Analyzed cooling performance for Case Analyzed cooling performance for Case Analyzed cooling performance for Case Analyzed cooling performance for Case Analyzed cooling performance for Case Analyzed cooling performance for Case Analyzed cooling performance for Case Analyzed cooling performance for Case Summary of analyzed results for all the eight cases...52 xii

13 CHAPTER 1 INTRODUCTION TO DATACENTER COOLING 1.1 Modern Data center Data centers have high concentrations of computers and digital equipment dedicated to housing websites and provides an essential service for new digital economy. These datacenters centralize and consolidate information technology (IT) sources, enabling the organizations to conduct business round the clock and around the world [5]. Data centers are mission criticafacilities and the nerve center for various successful business operations [6]. Figure 1 A Sun Micro systems datacenter [7] Data center is a facility used for housing the equipment responsible for storing and processing information. They typically consist of the servers used for transferring, storing and 1

14 managing data or information [8]. The computers used in datacenters are known as servers. Multiple servers are stored in these racks. These racks are placed in a raised floor or slab floor. The heat produced by the servers is removed by passing cold air through the cabinets. This cold air is extracted by the server fans and the hot air is returned through the rear of the servers. The uninterrupted power supply is placed away from the servers. There is a mixing of hot and cold air, bypass of hot air directly to the CRAC unit, as well as the recirculation of hot air in the data center. The air flow pattern inside the datacenter plays a vital role for the efficient performance of the data centers. 1.2 Types of Data Centers Most of the data centers fall into two major categories. a Private Data center (PDC) Private data centers are owned and operated by the private corporations with main focus on supporting data processing and web oriented services for their own organizations. Private data centers provide connectivity, application services and support to thousands of company employees working with in the building, campus [9]. b Internet Data center (IDC) Internet data centers are owned and operated by the traditional telecoms and they provide competitive service providers. These services include wide area communications, internet access, also web and application hosting [9]. 1.3 Literatures in Data Center cooling Kwok Wu compared Kool IT from Afco Systems, In Row cooling with Hot aisle containment from APC, Cold aisle containment by Knur and Overhead cooling by Liebert [10]. These systems were compared for effectiveness, efficiency, control, and robustness. The data center has 100 Cabinets,45 % open area ratio for tiles, 2 ft. raised floor, 12 ft. ceiling from the bottom of the raised floor plenum. 2

15 The goal of cold aisle containment is to maximize the utilization of air coming from the perforated floor [10].The energy consumed in the datacenter is mainly due to UPS (Uninterrupted Power supply), power distribution units, IT equipment; cooling equipment, lighting and power back up. A study by Boucher et.al [11] provides guidance for effective heat transfer and cooling performance in data centers by automated control using sensors. The control technique consists of a sensor which controls the supply temperature of the CRAC unit and also measures heat dissipation. The requirements that are used essentially in this technique are distributed sensor network, the ability to vary localized cooling and having insight of how each variable affect the conditions of the data center. Kukri et.al [8] focused on the techniques used for air flow distributions. He used a mathematical model to test their effectiveness. The air flow distribution techniques depends on the plenum height, area of the perforated tiles, position of the CRAC units and the presence of the under floor blockages. Baedeker et.al [13] studied the effect of altering the location of CRAC units in the raised floor datacenter. It was found that that the performance of the data center is influenced by the CRAC unit location. It was also found that it also depends on air flow entering the cabinets. Beaty et.al [14] has studied the cooling performance resulting from changing the location of CRAC units with respect to the racks. The final prediction of this study was that the racks which have clear path for hot air to the CRACs show lower inlet temperatures. The study suggested that CRACs can be placed facing the hot aisles rather than facing the cold aisles. Nakao et.al [15] modeled datacenters with four variations of datacenter cooling configuration in their computational study. The cooling models of the datacenters consisted of a supply through the under floor with exhaust through the ceiling, a supply through the under floor 3

16 with the horizontal exhaust, a supply over head with exhaust through the under floor and a overhead supply with exhaust through the ceiling. Figure 2 Airflow management in a data center Shrivastava et.al [16] studied different datacenter configurations. A CFD model were constructed to analyze the effectiveness of the configurations. The analysis study characterized the datacenter performance based on the average region Rack Inlet Temperature (RIT ) and mean region RIT. The results proved that, for a particular datacenter, under floor supply and ceiling return configuration were very efficient. The analysis study considered factors such 4

17 as, supply air flow fraction, the ceiling height and the location of the return vent. It was also found that the supply air flow fraction is highly influences rack inlet temperature in the configurations of the data center studied. Schmidt and Iyengar [16] discussed the suitability of air flow configurations for high density datacenters. The under floor and overhead configurations were used for this study. A CFD model was constructed for both the configurations to compare the air supply fraction and rack location along the height of the rack. High temperature gradients in the rack inlet temperatures were seen in these configurations [16]. It was also found that the rack inlet temperatures were more pronounced in under floor configuration compared with the overhead supply configuration. Roger et.al [17] studied the effect of the rack inlet temperature resulting from the chilled air that exits from the outlet of the perforated tiles. The tiles were situated in both the hot and cold aisles of the raised floor datacenter. A computational model [17] of the room was built with electronic equipment installed on the raised floor with a particular focus on effect of rack inlet temperature of IT equipment load, placing the air conditioning units and chiller flow rates. The results suggested that, increasing the air flow rate at the tiles decreases the inlet air flow temperature for the servers. It was also found that it was important to use chilled air at the CRAC units placed in the cold aisle in order for the air to enter the front of the racks [17]. Bhopte et.al [18] studied that the height of the under floor, ceiling height, and the cold aisle width affects the rack inlet air temperatures. Ceiling height, under floor height were maintained constant in this study. The lay out consisted of a raised floor platform with room return from the cabinets. Based on the findings, [18] a multivariable optimization methodology was proposed to decrease the server inlet temperature for the CFD model of the raised floor datacenter. The study utilized 5

18 the under floor height and ceiling height as the two design variables for this optimization technique. 6

19 CHAPTER 2 PRIME CHALLENGES FACED IN THE DATA CENTER The prime airflow management issues that the data center industry faces these days are the Venturi effect, recirculation of air flow and air by pass. The efficiency of the CRAC unit to cool the datacenter is affected by the above mentioned challenges. This affects the cooling power which is directly associated with the cooling cost spent in the datacenters. 2.1 Venturi Effect Venturi effect is the reduction of fluid pressure when the fluid passes through the narrow constricted section of pipe [47]. The fluid mechanic principle behind the Venturi effect is that the velocity of the fluid must increase through the constriction to satisfy the continuity equation whereas, its pressure must decrease to balance the conservation of energy. The gain in kinetic energy is balanced by the drop in pressure. The pressure drop as a result of Venturi effect may be derived from a combination of Bernoulli s principle and equation of continuity [47]. Figure 3 Fluid flow shown in Venturi tube [49] 7

20 Figure 4 Pressure differential in Venturi tube [49] The limitation of the Venturi effect occurs when a fluid reaches the state of choked flow where, the velocity of the fluid approaches the local speed of sound. In the case of choked flow, the mass flow rate remains constant and does not increase or decrease in the downstream pressure environment [49] 2.2 Recirculation of air flow Some portion of air that exits out of the outlet of the server exhaust, cycles back to the server inlet. This phenomenon in air flow is called recirculation of air flow [19]. In the modern datacenters, most of the hot air in aisle returns back to the CRAC unit [6]. But small portion of air may mix into the cold aisle from the top or bottom of the racks or from the side of the racks. The recirculation can also occur due to reversed air flow in certain IT equipments, whenever the internal fans of the IT equipment blows the hot exhaust air from hot aisle directly to cold aisle. Thus, the recirculated air is always a part of inlet air that flows to the racks in the raised floor data center [19]. a Effect of recirculated air [19] 1. The datacenter cooling efficiency is affected due to entropy change owing to recirculated air. 8

21 2. The hot spot are formed in the servers due to recirculation of air. Figure 5 Recirculation of Airflow [19] 3. The numbers of CRAC units operated must be increased to compensate the hike in temperature. b Measures to prevent recirculation of air [19] 1. Fitting of blanking plates in the cabinet in the locations where the blade servers are not fixed. 2. Gaps in the cabinet needs to be closed at the areas where the warm air can seep into the inlet. 3. An adequate supply of cold air should be ensured for the all the servers in the data center. 4. The air distribution at the server inlet should be made sure to reach all the servers in the cabinet. 5. The path for the return air from the cabinet should be checked for duct work if needed. 9

22 6. Cold air supply should be isolated from the hot air streams wherever needed. 7. Obstructions under the floor that restricts the cold air stream should be removed. 8. The proper routing for the cables through the under floor and at the back of the cabinets should be made sure to be away from the airflow path. 2.3 By pass Air flow The air that enters the perforated tiles returns back to the CRAC unit without cooling the servers. This is termed as Bypass Air flow. The supply air from the CRAC unit enters the under floor and penetrates the perforated tiles. This air directly moves to the CRAC or cooler return without entering the server. Figure 6 Bypass Airflow [20] a Effects of Bypass Air flow 1. The CRAC return temperature is increased because of the bypass Air flow. 2. The cold air reduces the static pressure resulting in poorly distributed airflow through the cold aisle. 10

23 3. The hot spots are present in high heatload areas which occurs due to Bypass Air Flow. Figure 7 Leakage through cable outlets [4] Figure 8 Under floor airflow path blocked by cables [4] 11

24 b Best practices to prevent bypass air in datacenter [19]. 1. Air gaps in the raised floor should be sealed with the foam brushes or pillows. 2. Floor grills if present at the hot aisles should be removed if cooling is not necessary. 3. The air flow from the floor grills should be optimized not to overshoot the cabinet top. 4. The back draft dampers need to installed on the idle CRAC units as they are the source of air by pass. 5. Duct extensions should be installed on the CRAC return air paths to have increased height for the air intake. 12

25 CHAPTER 3 LITERATURE REVIEW ON CONTAINMENT TECHNIQUES 3.1 Hot Aisle Containment System The hot aisle containment collects the server exhaust air and restricts its entry to the rest of the datacenter. In one of the hot aisle containment types, the hot aisle is sealed with doors, side walls or roofs [21]. The CRAC unit cools the servers and the rest of the remaining room without allowing them to get overheated. In other arrangements, the data center room have specialized cabinets with solid rear door and there is also an attached duct that leads to the false ceiling. The server cabinets collect the exhaust air off the servers and transfer it into the false ceiling. This exhaust air is finally driven back to the air conditioning unit. The setting above mentioned may or may not contain fans which may remove exhaust air from servers efficiently Figure 9 Hot aisle containment system [2] 13

26 3.2 Cold Aisle Containment System Cold aisle containment provides physical separation between the hot and cold air by isolating the cold aisle. This containment technique sets apart the cold air from the room. It uses the metal, curtain to concentrate the cold air at the server inlet. By this technique, the cold air compulsorily passes through the server racks cooling the racks before entering the rest of the room. The importance of the cold aisle containment is that, they are good for the existing datacenters without hot air return but, focused cooling on the IT equipment eliminating the rest of the room. They also support high density loads in datacenters. [22] The cold aisle containment provides optimum cold air to enter the IT equipment by aiding for the uniform inlet temperatures [6]. Figure 10 Cold aisle containment system [2] 3.3 Hot Aisle / Cold Aisle The Hot aisle / Cold aisle are design lay out for server racks and other computing equipment in the data centers. [24] In this layout, the servers are arranged to form a separate 14

27 hot and cold aisle. This helps to save energy by allowing air supply at lower operating temperatures than hot aisle containment: [25] Figure 11 Partial Containment Air flow management [26] 3.4 Partial Containment System ` End of Aisle is one of the preferred partial containment technique used in the datacenters. In this technique, the end of the cold aisle is contained with the doors on each ends. These doors allow for the stratification of the air and help to reduce the temperatures in the cold aisle. [27]. The partial containments system is said to have double or even triple the power density limits in the data center rooms. The partial containment products are advantageous in terms of reduced bypass airflow, increased usable cooling unit capacity and maximized efficiency for HVAC equipments [28].The essential reasons to implement the partial containment systems are, to improve CRAC efficiency, to aid in simple installation, to be effective at cold aisle proximity. [38] 15

28 3.5 Merits of the Containment systems [27] 1. Cooling systems can be set at higher supply temperatures that helps in increasing Figure 12 Partially contained (End of Aisle) [38] the cooling capacity with lesser frequent hot spot formation. 2. Amount of economization hours is increased since the cooling system is turned off for a longer period of time. 3. The humidification cost is increased since containment technique helps to eliminate the mixing of hot and cold air. 4. The containment aids in better infrastructure utilization and right sizing that has decreased the datacenter s dependency on required fan power. 3.6 Reason for implementation of Containment Systems [48] 1. The containment systems are essential to improve CRAC s efficiency. 2. The containment is highly effective for areas where CRAC units are close to the cold aisles. 3. They are proven for their effectiveness and simple installation of their frame work. 16

29 CHAPTER 4 SELECTION OF AIR CONTAINMENT SYSTEMS 4.1 Logical Selection of Air containment systems This chapter briefly describes the logical steps for deciding which airflow management solutions to implement starting with, assessing the datacenter facility, reviewing the possible solutions and selecting the right containment solutions. Figure 13 Selections of Air Containment Systems for the Comparison Study [29] The containment methods responsible for cooling are classified according to hot air and cold air and further classified according to their purposes especially for cooling the racks or cooling the servers [29]. 17

30 Figure 14 Pros and Corns for Air Containment Systems for the comparison study [29] 4.2 Conditions Recommended for Ducted HACS [30] 1. The racks and IT equipments need to be alligned in the hot / cold aisle configuration [48]. Figure 15 Ducted Hot Aisle Containment System [30] 18

31 2. When the drop ceiling is present for the Hot air return path [7]. 3. When Individual IT equipments are placed on the data center perimeter [31]. 4. When data center is frequently occupied by workers with less hotter enviroment [30]. Figure 16 Cold Aisle Containment System (CACS) [30] 4.3 Conditions Recommended for CACS [30] 1. When IT equipment and racks are aligned in a hot aisle / cold aisle pattern of air containment [29]. 2. When the data center makes use of raised floor platform and flooded return air distribution [32] technique. 3. When there are no individual separate IT devices on the perimeter of the data center room. 4. When the racks are fully packed and unable to draw enough cool air from the raised floor. 5. When the project of air containment has a shorter period of time for the implementation. 19

32 CHAPTER 5 CFD PROBLEM SPECIFICATION 5.1 Problem Description 1. The containment techniques modeled are essentially Hot Aisle / Cold Aisle, full Hot Aisle Containment System [29], full Cold Aisle Containment System, Partial hot aisle containment system and Partial cold aisle containment system which are basically from the earlier mentioned containment system. 2. The objective of this case study is to compare these containment techniques in separate CFD platform and find out which technique is the best in terms of maximum cooling performance at reduced fan power consumption. There are eight cases studied in CFD platform compared for this cooling performance study. 5.2 Overheated Server sets [33] This value is the number of servers in the cabinets that have server inlet temperatures over 77 F with air conditioning units at a set point of 64.4 F. Figure 17 Cabinet front and rear 20

33 It is essential to observe the amount of racks overheated by implementing containment configurations. This gives an idea of which configuration provides the most effective containment. The location of these racks in the cabinets in relation to the air conditioning units is very important. [33] 5.3 Fan power [34] The fan power or the pump power to drive the air throughout the data center depends on mass flow rate, the liquid density and the differential height. Figure 18 Equation for calculation of Fan power In the above figure, P is the pumping power in Watts, ρ is the fluid density in Kg/m³, and q is the flow rate in m³/s, h is the head difference in meter and g is the acceleration due to gravity which is m /s². a Pressure drop calculation: [35] Figure 19 The side view of the CRAC unit CFD model [35] A volume region using the FloVENT smart part of size (2.4*0.152*0.9) each below the supply and above the return side of the CRAC unit to calculate the average pressure in the 21

34 supply side and the average pressure in the return side of the CRAC unit. The averaged pressure drop is calculated from the values form the supply and return side of the CRAC unit. b Heat load calculation: [48] There are two main functions for a data center cooling system [48]. The first function is to remove heat from datacenter and second function is to effectively distribute this cool air to the data center equipment. c Flow rate through racks: [18] The heat removal rate is a function of the temperature difference across the server intake and exhaust and also function of volumetric flow rate. Figure 20 Equation for flow rate through the racks [36] In the above figure, ΔT is the temperature difference across the racks, Q is the heat removal rate and CFM is the measure of volumetric flow rate [48]. 22

35 CHAPTER 6 DATA CENTER CFD MODELING 6.1 Cabinet Model Description a Cabinet model description (Simple model) Supply side side Rack 1 Rack 2 Extract side Rack 3 Figure 21 Cabinet simple model [35] The cabinet assembly consists of an outer chassis with three racks. This allows for the heat load at lower, middle and higher levels. Three fans are placed at the rack supply to pull the air through the rack system. The model is also flexible to allow different flow rates through the three levels to coincide with the different sources. The sources attached inside the cabinet define equal heat on each thirds of the cabinet. The source also straightens the flow preventing 23

36 vertical flow. The front door is divided into three sections to allow analysis of the air temperatures at lower, middle and upper door sections. Table 1 Cabinet Model Specifications Simple model Cabinet Model Geometric Parameters Value/info Cabinet dimensions (0.9*2*2.4) m Number of racks per cabinet 3 Temperature difference across the cabinet 11 C Extract and supply dimensions (0.6*0.6) m Cabinet heat load 2.67 KW b Cabinet model description (Advanced) [35] The cabinet model considered for this study is advanced resistance model.this cabinet is specified with the system curve and fan curve input. The server used in this cabinet is an Intel Xeon 5500, 350 Watt processor. The experimental data of the system resistance curve of this Face book server has been implemented in the Flo-VENT code. Each rack of the cabinet is represented as a set of servers. Single rack accommodates nine servers. There are in total 27 servers in the cabinet model. Table 2 Cabinet Model Specifications - Advanced Resistance Model [35] Cabinet Model Geometric Parameters Value/info Cabinet dimensions [35] (0.9*2*2.4) m Number of server sets per cabinet 3 Number of servers in server set 9 Number of servers per cabinet 27 Server supply and extract dimensions (23*2.6) in Type of resistance [35] Advanced resistance model Heat load per server 296 W of 350 W Heat load per rack 2.67 KW Maximum heat load per cabinet [18] 8 KW 24

37 The system resistance curve is fed as a input by creating an advance resistance model that acts as a flow resistance. The fan curve is also specified to the server fan from the face book server. Static pressure Vs Flow rate Figure 22 System resistance curve for the experimental server [37] The server fan curve represents the increase in volumetric flow with the increase in the pressure drop. The maximum pressure drop for this fan is 80 pascals (Pa) and maximum volumetric flow across the server is 1150 cfm. c) Approach to advanced resistance model [35] In my study, the server is modeled using advanced resistance model feature from FloVENT software. The Intel XEON 5500 R server is simulated by using its fan curve and system resistance curve in the FloVENT software. 25

38 The server is represented as a 2D model. This procedure involved in creating the advanced resistance model consists of two major steps. a To import the system resistance curve. [35] Table 3 Pressure drop at various fan speeds across the servers Velocity (m/s) Pressure Drop (Pa) Pressure Vs. Velocity Figure 23 System resistance curve for the advanced resistance model [35] b To insert the fan curve. 26

39 The fan curve is a function of Pressure drop and volumetric flow rate as known from Pressure Vs Volumetric Flow. Figure 24 Fan curve for the advanced resistance model [35] We equate the equation of the trend line to the equation which we arrived for the advanced resistance model. The value of l is arbitarily set to 1 m and considerd. Figure 25 Equation for the advanced resistance model 27

40 Figure 26 Reduced equations for the advanced resistance model This gives the complete values to be entered in the dialogue box for the advanced resistance model, ρ is the density of the resistance, v is the air flow velocity across the server. ΔP is the Pressure difference across the server inlet and outlet. The resistance model is calculated with the equation above. Here a and b are the calculated by equating the υ and υ² coefficients from the above equation with the equation of the system resistance curve. This a and b coefficients are entered in the resistance model to represent the resistances for the servers. The fan curve is imported in the rack smart part along with resistance model to represent the server compact model. 6.2 CRAC Model Specification The CRAC unit provides use a cuboid to represent the body of the unit. A recirculation device represents the fans and coils. Figure 27 CRAC unit CFD model [35] 28

41 Table 4 CRAC Model Specification [35] CRAC Model Geometric Parameters CRAC dimensions Number of CRAC unit Flow rate Extract and supply dimensions Value/info (0.9*2*2.4) m One Cfm (0.9*2.4) m Supply Temperature set point 18 C CRAC unit capacity Direction of Flow 211 KW Normal 6.3 Ceiling drop duct specification The ceiling drop duct in the HACS is specified with 82% open area ratio for air from the room to the ceiling plenum. This is the preferred open are considered in most of the datacenters. Figure 28 Ceiling drop duct in raised floor datacenter CFD model 29

42 6.4 Air Containment Systems CFD Models [29] The air containment systems are configured in a common datacenter room lay out with fixed underfloor depth.the figure below shows the drop duct and the underfloor im the same lay out. Figure 29 Isometric view of the data center room layout [35] This Data center platform is validated first from a published paper [4]. The Ceiling drop duct, perforated floor and CRAC unit positions in this raised floor data center. Only half of the data center is modeled and simulated due to symmetry. The symmetry plane is shown in the left. The raised floor perforation is shown in blue. The air ducts in the ceiling are colored in yellow. Some of the feature might not appear in the actual model, depending on the configuration being analyzed. 30

43 a Case 1 Hot aisle / cold aisle datacenter [35] Figure 30 Case 1 Hot aisle / cold aisle datacenter [35] The case 1 is the datacenter is simple hot aisle /cold aisle datacenter. This does not have any blanking panels. The hot aisles are considered from 1 through 3 beginning from the CRAC unit The cold aisle are considered 1 and 2 beginning from the CRAC unit. The server cabinets are numbered from 1 through 3 in their respective rows. This layout represents the hot aisle / cold aisle lay- out referred common in most of the datacenter rooms. This case is expected to have more recirculation than the other cases since there is no isolation between the hot aisles and the cold aisles. 31

44 b Case 2 Data center contained with both the cold aisle Figure 31 Case 2 Data center contained with both the cold aisle [35] The above figure shows the datacenter with both the aisle contained. This is the CFD model. The blanking panel of neglible thickness is placed above the cabinet rows A and B. There are blanking panels that also covers the end of the aisle. Similarly the blanking panels are placed containing the second cold aisle for the cabinets rows C and D in the figure. This leaves the datacenter with both the cold aisles contained. This lay out is expected to have better cooling in the server sets compared to the case. This arrangement is referred from Cold aisle containment system [29]. 32

45 c Case 3 Data center contained with first cold aisle alone Figure 32 Case 3 Data center contained with first cold aisle alone [1] In this CFD model of the datacenter room lay out, the first cold aisle considered is contained leaving the second cold aisle uncontained. This model is similar to previous containment system excepting that the second cold aisle is left uncontained. A blanking panel is used to contain the roof of the cabinets in the second cold aisle. The blanking panels are also used to contain the end of the aisle. The cold air is contained due to the blanking panel leaving the hot air isolated from the cold air in this arrangement. 33

46 d Case 4 Data center contained with second cold aisle alone Figure 33 Case 4 Data center contained with second cold aisle alone [35] In this FloVENT model, the arrangement is again on the raised floor datacenter platform but second cold aisle formed far from the CRAC unit is contained by using the blanking panel of negligible thickness at the end of the aisle. One more blanking panel of negligible thickness is used at the top of the cabinets placed in rows C and D. 34

47 e Case 5 Data center contained with the all the three hot aisles [35] Figure 34 Case 5 Data center contained with all the three hot aisles [35] In this CFD model of the raised floor datacenter, the three hot aisles formed are contained by the blanking panel. The drop ceiling formed with ceiling ducts are used allow the hot air to be removed from the room. 35

48 f Case 6 Data center contained with first and second hot aisle alone [35] In this CFD model of the raised floor datacenter, the first and second hot aisles formed with respect to the CRAC unit location are contained by the blanking panel. The drop ceiling formed with ceiling ducts are used allow the hot air to be removed from the room. Figure 35 Case 6 Data center contained with first and second hot aisle alone [35] In this CFD model of the raised floor datacenter, the first and second hot aisles formed with respect to the CRAC unit location are contained by the blanking panel. The drop ceiling formed with ceiling ducts are used allow the hot air to be removed from the room. 36

49 g Case 7 Data center contained with second and third hot aisle alone [35] Figure 36 Case 7 Data center contained with first and third hot aisle alone [35] In this the raised floor datacenter platform the second and third hot aisles formed with respect to the CRAC unit location are contained by the blanking panel. The second hot aisle mentioned is the aisles in between row B and C. 37

50 g Case 8 Data center contained with second and third hot aisle alone [35] Figure 37 Case 8 Data center contained with second and third hot aisle alone [35] In this the raised floor datacenter platform the second and third hot aisles formed with respect to the CRAC unit location are contained by the blanking panel. The second hot aisle mentioned is the aisles in between row B and C. 38

51 CHAPTER 7 RESULTS AND ANALYSIS 7.1 Case 1 Results and Analysis From the previously mentioned configurations, the following results have been obtained by the computational analysis. Figure 38 Server Sets which are considered as over heated [35] The above figure shows the servers sets in various cabinets which had temperatures higher than the 25 C at the inlet. The results of temperature surface plot, pressure plot and the velocity surface plot for the server set in the cabinet which has maximum temperature compared with other server sets is shown. The surface plot for temperature is plotted for the server set which has been observed with maximum temperature at the server inlet. The pressure plot allows to understand the 39

52 resistance offered by the system. The occurrence of recirculation can be observed from velocity plot. Figure 39 The maximum temperature observed in server sets is 35 C Figure 40 The pressure surface plot for the cabinet showing maximum temperature 40

53 Figure 41 The velocity surface plot for the cabinet showing maximum temperature Table 5 Analysed cooling performance for Case 1 S.no Cooling performance parameters Values 1 Number of Overheated server sets 40 / 72 2 Fan Power 1.85 KW 3 Maximum Temperature exhibited by the server set 34.5 C 7.2 Case 2 Results and Analysis Figure 42 Temperature surface plot X axis through midsection cabinets A3- D3 41

54 In the datacenter contained with both the cold aisle, the surface plot, Fan power and the Maximum temperatures exhibited by the server set is tabulated. Table 6 Analyzed cooling performance for Case 2 S.No Cooling performance parameters Values 1 Number of Overheated Sets 0 2 Fan Power 5.16 KW 3 Maximum Temperature exhibited by the server set <25 C 7.3 Case 3 Results and Analysis In case 3, the location of the servers sets, the thermal surface plot, total number of servers sets over heated and the Fan power is shown below. Figure 43 Cabinets which are considered to be over heated in case 3 [35] 42

55 Figure 44 Temperature surface plot X axis through A3 D3 in Case 3 The temperature surface plot shows the distribuution of the temperatures through server sets though midesection A3- D3. Figure 45 Temperature surface plot for the cabinet showing maximum temperature The Cabinet was observed with temperatures 29 C at the inlet which is higher than the considered temperature of 25 C. 43

56 Figure 46 The pressure surface plot for the cabinet showing maximum temperature Figure 47 The velocity surface plot for the cabinet showing maximum temperature Table 7 Analyzed cooling performance for Case 3 S.No Cooling performance parameters Values 1 Number of Overheated Sets 20 / 72 2 Fan Power 2.01 KW 3 Maximum Temperature exhibited by the server set 28 C at D3 44

57 7.4 Case 4 - Results and Analysis In case 4, the location of the servers sets, the thermal surface plot, total number of servers sets over heated and the Fan power is shown below. Figure 48 Cabinets which are considered to be over heated in Case 4 [35] Table 8 Analyzed Cooling performance Table for Case 4 S.No Cooling performance parameters Values 1 Number of Overheated Sets 16 / 72 2 Fan Power 2.01 KW 3 Maximum Temperature exhibited by the server set 35 C at B3 7.5 Case 5 - Results and Analysis The thermal surface plot, total number of servers over heated and the fan power consumed in this configuration is shown below. Figure 49 Temperature surface plot X axis through cabinets A3-D3 45

58 Figure 50 Temperature surface plots X axis through cabinets A1-D1 Table 9 Analyzed cooling performance for Case 5 S.No Cooling performance parameters Values 1 Number of Overheated Sets 0 2 Fan Power 5.36 KW 3 Maximum Temperature exhibited by the server set <25 C It is observed that there are no server sets have temperatures higher than 25 C. 7.6 Case 6 - Results and Analysis Figure 51 Temperature surface plot X axis through cabinet rows A3-D3 Figure 52 Temperature surface plot X axis through cabinet rows A1 - D1 46

59 Table 10 Analyzed cooling performance for Case 6 S.No Cooling performance parameters Values 1 Number of Overheated Sets 0 2 Fan Power 2.4 KW 3 Maximum Temperature exhibited by the server set <25 C 7.7 Case 7 - Results and Analysis The location of the server set showing maximum temperature, the temperature surface plot, pressure surface plot and velocity surface plot is shown in the next three figures. Figure 53 The server sets in this cabinet shows temperature higher in case 7 [35] 47

60 There are three server sets that show temperatures higher than 25 C in the half symmetry model. The temperature surface plot is modified to observe temperature distribution in the server sets. The server sets placed closer to second aisle exhibit temperatures lower than 25 C at the inlet. Figure 54 The temperature surface plot for the cabinet maximum temperature The temperature surface plot shows the temperature distribution in the server inlet for the case 7 when the first and the third aisle is contained and the second aisle is left uncontained. 48

61 Figure 55 The pressure surface plot for the cabinet exhibiting maximum temperature Figure 56 The velocity surface plot for the server set in cabinets showing 26 C [35] 49

62 The velocity surface plot aids for understanding the recirculation that occurs in the server sets showing higher temperatures at the inlet. Table 11 Analyzed cooling performance for Case 7 S.No Cooling performance Parameters Values 1 Number of Overheated Sets 6 / 72 2 Fan Power 2.38 KW 3 Maximum Temperature exhibited by the server set 26 C Fan power is observed to 2.38 KW which is quite less compared with the data center contained with all the three hot aisles. 7.8 Case 8 - Results and Analysis The temperature surface plot results, total number of overheated server sets, the Fan power consumed in case 7 is illustrated below. The below two temperature surface plots shows higher temperature closer to CRAC unit since the first aisle is left uncontained. Figure 57 Temperature surface plot through X axis midsection cabinets through A3-D3 [35] It is well observed from the surface plot results that the temperature of the serves sets in the corner cabinets show lower temperatures. When the second and third hot aisle is contained and the fists aisle is left uncontained. 50

63 Figure 58 Temperature surface plots through X axis midsection cabinets through A1-D1 It is well observed from the tabulated results that there is no cabinet with servers sets showing temperatures higher than 25ºC in this case. Table 12 Analyzed cooling performance for Case 8 S.No Cooling performance Parameters Values 1 Number of Overheated Sets 0 2 Fan Power 0 3 Maximum Temperature exhibited by the server set 2.27 KW The Fan power value is slightly lower compared to the previous cases of partially contained datacenters 7.9 Summary of Analyzed Results for Eight Cases The next table shows the summarized results for all the eight cases of the air containment systems that has been compared. The predicted results of Fan power that is consumed in a day for each of the configurations is also tabulated in addition. 51

64 Table 13 Summary of analysed results for all the eight cases Air containment System [29] incorporated in the CFD problem Number of Server sets observed over heated out of 72 Maximum Temperature observed in an individual server set in ºC (above 25ºC) Fan Power (calculated ) KW Fan power spend for a day 24 hours KW Case1 Hot aisle / Cold Aisle datacenter Case 2 - Datacenter contained with both the cold aisle. Case 3 Data center contained with first cold aisle alone. Case 4- CACS [29] Second cold aisle alone contained Case 5-HACS [29] Fully contained Case 6 -HACS [29]- first and second hot aisle alone contained Case 7- HACS [29] First and third hot aisle alone contained < < < Case 8 - HACS [29] Second and third hot aisle alone contained 0 <

65 CHAPTER 8 CONCLUSION AND FUTURE WORK Containment is a fairly effective and relatively cheap approach to reduce the cost of energy in legacy as well design of new data centers. The hot aisle containment collects the server exhaust air and restricts its entry to the rest of the datacenter. Cold aisle containment provides physical separation between the hot and cold air by isolating the cold aisle. This containment technique sets apart the cold air from the room. In this study, eight different containment configurations of the datacenter were created using Flo-VENT. Based on reference from ASHRAE handbook TC , maximum temperature at which servers gets over heated is utilized as one matrix for evaluating the performance or effectiveness of the various containment systems discussed in this study. A second matrix in evaluating the performance of the containment in question is the fan power consumption which is dependent on the pressure drop at the CRAC unit. Primarily, the cold air containment fully contained with asymmetric CRAC unit location with raised floor datacenter performs better in cooling performance than the ducted hot aisle containment system [31] in terms of cooling performance. The reasons as observed from the simulation result are, the pressure drop observed at the CRAC supply and return is higher in the fully contained hot air containment system than the cold air containment system. It was also observed that, partial aisle containment system which is currently less frequent in practice consumes lower power to cool the server sets. Finally, we conclude that by containing the second and third hot aisle from the incorporated ducted hot aisle containment system technique (Hot air containment system approach [30]), we achieve maximum cooling performance since there is lesser recirculation is 53