NEURO-FUZZY NETWORK MODELING OF DATA CENTRE ENERGY PERFORMANCE

Transcription

1 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance CHAPTER 6 NEURO-FUZZY NETWORK MODELING OF DATA CENTRE ENERGY PERFORMANCE 6.1 Introduction An exploratory study of the whole-system based neuro-fuzzy network modeling for data centre energy performance is conducted as part of the efforts to understand an empirical energy use pattern of data centres in Singapore. The models developed integrates energy consuming constituents contributing to the energy consumptions of data centres, and analyzes various factors such as physical parameters of space and environment control facilities, design and operation strategies of HVAC, UPS, IT equipment and lighting systems. Neuro-fuzzy network presents itself a desired and applicable candidate of modeling energy performance of complex systems by taking advantages of mimicking human decision and self-training by existing data sets. A well structured and trained neuro-fuzzy model can serve as evaluation tool kits at both design and post-diagnostic stages of data centres. Energy saving potential can also be estimated quantitatively. FuzzyTech 5.5 is employed as the modeling software in this study. In this study, the neuro-fuzzy network models are established based on the energy performance benchmarking of the six data centres studied. Due to the small data base, these models cannot provide highly accurate evaluation of data centre energy performance and prediction of data centres energy saving potential. However, they can serve as efficient and effective tools, prior to the further and detailed measurement and verification (M&V) or energy simulation for the evaluation of data centre energy 146

2 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance performance and estimation of energy saving potential. 6.2 Application of neuro-fuzzy network in modeling data centre energy performance Fuzzy logic is a technology that enhances model-based system design by using both intuition and engineering heuristics. This provides an efficient way of designing, optimizing, and maintaining highly complex systems and results in robust and fault tolerant systems. The use of linguistic modeling, instead of mathematical modeling, greatly enhances system transparency and modification potential. It leads to quick development cycles, easy programming and accurate control. Fuzzy logic principle is especially advantageous for problems that cannot be easily represented by mathematical modeling because data is either unavailable, incomplete, or the process is too complex to use a traditional approach [FuzzyTech, 2001]. There are a number of reasons for neuro-fuzzy network to be employed as modeling approach in this study. Firstly, data centres energy consumption is a complex system. A typical data centre contains various energy consuming systems including IT equipment, UPS, HVAC system, lighting, office equipment, building control system, fire suppression system, current transformer and stand-by power generator. The interactions among energy consuming systems also make this system more complex. Energy performance of data centres is also affected significantly by physical variables such as space usage, indoor temperature, humidity, as well as the design and operation strategies. While it is possible to develop a deterministic--namely precise mathematical or empirical--model, the complexity of the data centre energy performance makes it 147

3 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance difficult to be modeled using conventional modeling approach such as multiple regression. Energy simulation, the popular tool of modeling building system and predicting energy consumption, is another possible solution for modeling data centre. However, simulation is highly time and cost consuming. By comparison, neuro-fuzzy network with the capability of modeling complex system can be a suitable solution in this study. Secondly, the term energy performance is by nature a fuzzy concept. It not only associates with the energy consumption and power demand, but also the operational energy efficiencies of various systems and total data centre. A precise definition of energy performance does not exist. Traditional indicators such as annual energy consumption (kwh/m 2 /year) and power demand density (W/m 2 ) per unit gross floor area are not sufficient for describing data centres energy performance. For example, the total data centre power demand densities of data centre 4 and data centre 6 are 348 W/m 2 and 231 W/m 2 respectively. This does not mean that data centre 6 is more energy efficient than data centre 4, as data centre 6 may have a sparse equipment distribution over a large area, and also resulting in significant amount of energy spent on space cooling. Besides, it is found that the UPS system in data centre 6 is the least efficient among all the data centres studied. It is evident that energy efficiencies of HVAC and UPS systems, area occupancy, over-designing of the supporting systems, designed power demands, and environmental setting put significant impact on data centres energy performance. For the reasons above, fuzzy logic which mimics human logic by combining various parameters or rules for making description or decision is suitable for the application of modeling data centre energy performance. 148

4 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance Thirdly, classification of energy performance involves the usage of fuzzy concept. For example, assuming the actual total data centre power density is categorized into 3 classes, which are less than 300 /m 2, 300 W/m 2 to 410 W/m 2 and higher than 410 W/m 2 respectively, a data centre of 301 W/m 2 will be classified under Class 2, and a data centre of 299 W/m 2 will be classified under Class 1. This could be an unreasonable classification since the difference of power demand between these two data centres is very slight. This reflects the fact that the defined power demand range of each class should not be a crisp range. There should be a buffer area between each class. In this case, fuzzy logic is particularly suited for addressing this issue. 6.3 Knowledge background of neuro-fuzzy network Structure of fuzzy logic model A typical fuzzy logic based approach involves three modules: a fuzzifier, a fuzzy interference engine, and a defuzzifier as illustrated in Figure 6.1. Input linguistic variables Fuzzy inference Output linguistic variables Linguistic Level Technical Level Fuzzification Input real variables Defuzzification Output real variables Figure 6.1 Typical structure of rule-based fuzzy logic model (Source: FuzzyTech, 2001; Gouda et al., 2001; Krarti, 2003) A fuzzifier is used to map input vectors into fuzzy sets defined by linguistic variables 149

5 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance such as Bad, Average and Good and membership functions. The inference engine associates fuzzy sets to other fuzzy sets using a rule library of IF-THEN fuzzy rules that highly intuitive and easily understood by human users. Finally the defuzzifier is used to map fuzzy sets to output vectors. Further information of fuzzy logic is provided following in section Principles of neural network For the purpose of enhancing fuzzy logic systems with learning capacities, neural network (NN) is introduced. The objective of a neural net is to process information in a manner that has been previously trained into the net. Training uses either sample data sets of inputs and corresponding outputs, or an expert who rates the performance of the neural net. A neural network can be any model in which the output variables are computed from the input variables by compositions of basic function or connections. However, one of the most commonly used neural network models is the multilayer perception [Krarti, 1998]. Figure 6.2 shows a schematic diagram of the structure of a neural network. Figure 6.2 Neural network schematic diagram (Source: Krarti, 2003) 150

6 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance The first and last layers of neurons are called input and output layers; between them are one or more hidden layers. The neuron depicted by the small circles is the fundamental building block of a network. Each element of the input set, INPUT i, is multiplied by a weight, W ij, and the products are summed to provide the output at the neuron, OUTPUT j : OUTPUT j = Σ i INPUT i W ij +B j Eqn (6.1) where, B j is the bias at the neuron j. The bias is the activation threshold for the neuron j. This bias avoids the tendency of an activation function to get stuck in the saturated, limiting value area of the activation function described below. After each OUTPUT j is calculated, an activation function is applied to modify it. The activation function is typically a bounded monotonic function. Sigmoid function is the most common one. ƒ(x)= 1/ (1+exp(-x)) Eqn (6.2) The weights W ij of the neural network are adjusted iteratively so that application of a set of inputs produces the desired set of outputs. If the computed outputs do not match the known values during network training, the NN model is in error. The error, E, is typically calculated as the sum of squared differences between computed and target values: E= Σ j (OUTPUT j -TARGET j ) 2 Eqn (6.3) 151

7 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance If the error E is large, then the neural network weights, W ij are adjusted to reduce this error. The most commonly used approach to adjust the weights is the gradient descent method. Using this approach, the weights is changed from W ij to (W ij a de/dw ij ), where parameter a is called the learning rate. This procedure of weight adjustment is called back-propagation. A simplified procedure for the learning process of neural network is summarized below: Provide the network with training data consisting of patterns of input variables and target outputs. Assess how closely the network output matches the target outputs. Adapt the connection strengths (i.e., weights) between the neurons so the network produces better approximations of the desired target outputs. Continue the process of adjusting the weights until some desired accuracy level is achieved. Analyse the results to assess the validity of the parametric relationships Neuro-fuzzy network [FuzzyTech, 2001] The key benefit of fuzzy logic is that it allows design consultant to describe desired system behaviour with simple IF-THEN relations. However, this is at the same time its major limitation. In many applications, knowledge that describes desired system behaviour is contained in data sets. The design consultant must derive the IF-THEN rules from the data sets manually. In this case, a neural network presents a solution since it can train the model itself from the data, while the fuzzy logic model provides 152

8 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance the rules which governs the paradigm relationships. On the other hand, neural network itself is limited by a number of reasons. Firstly, neural network solution stands as a black box. Design consultant can neither interpret what derive a certain behaviour, nor modify a neural network manually to influence a certain behaviour. Secondly, neural network require prohibitive computational effort, and numbers of sample data during the modeling process. Thirdly, selection of the appropriate net model and setting the parameters of the learning algorithm is still a tough job which requires extensive experience. Among the mentioned reasons, inadequacy of data base and experience, as well as the lack of an easy way to verify and optimize a neural network solution are probably the major limitations of neuro network. For this application, combined use of neural network and fuzzy logic might provide a powerful technique which compensates individual method s weakness. Neural network can learn from data sets, while fuzzy logic solutions are easy to verify and optimize. Neuro-fuzzy network combines the advantages of fuzzy systems, the explicit knowledge representation of knowledge, and the ability to cope with uncertainties with the learning power of neural network Method of neuro-fuzzy network modeling using FuzzyTech 5.5 FuzzyTech 5.5 is powerful software using neuro-fuzzy technology, with which models describing complex systems can be established easily and clearly. As shown in Figure 6.3, the basic neuro-fuzzy network model typically consists of input variables, rule blocks and output variables. 153

9 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance Fuzzification Fuzzy Inference Defuzzification Figure 6.3 Basic structure of fuzzy logic model Fuzzification The fuzzification interface consists of the following operations [Gouda et al., 2001]: 1) Perform a scale mapping that transfers the input variable ranges into a corresponding universe of discourse (quantization/ normalization). 2) Performs the fuzzification strategy that converts crisp input data into suitable linguistic variables, which may be viewed as labels of fuzzy sets. Firstly, in the fuzzification interface, real values are transformed into linguistic variables. In this study all the input and output variables are defined correspondingly to three classes within the full range of observed values. For example, Total data centre energy consumption is converted into fuzzy sets such as low, medium and high. 154

10 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance The degree to which the crisp value belongs to a set is represented by a value between 0 and 1, called the degree of membership µ. Fuzzy logic describes degree of membership by a function known as a membership function (MBF). Standard membership functions are used in most practical applications with types of Z, Lambda, Pi and S, as shown in Figure 6.4. Figure 6.4 Standard membership function types Figure 6.5 below shows the fuzzification of term total data centre energy consumption. Based on the data collected during the measurements in the 6 data centre studied, the details of the ranges of defined input and output variables are given in Appendix H. Degree of Membership Range of real value 4100 kwh/m 2 /year Figure 6.5 Fuzzification of variable 155

11 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance For example, energy consumption at 4100 kwh/m 2 /year is a member of the fuzzy sets for the terms: Low to the degree of 0.00 Medium to the degree of 0.30 High to the degree of Fuzzy inference Once all input variable values are translated into the respective linguistic variable values, the so-called fuzzy inference step evaluates the set of IF-THEN rules that define system behaviour. IF-THEN rules always describe the reaction to a certain situation as: IF <situation> THEN <action>. The inference is a calculus consisting of the steps: aggregation, composition and, if necessary, result aggregation. First, aggregation determines the degree to which the complete IF part of the rule is fulfilled. Special fuzzy operators are used to aggregate the degrees of support of the various preconditions. There are in Min operator, which corresponds with the linguistic AND, is most common and has been employed for the study. Equation below shows the principle of this method. Figure 6.6 shows the dialog of input fuzzy operator. AND: µ (A^B)=min {µ(a), µ(b)} Eqn (6.4) 156

12 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance Figure 6.6 Operator of input aggregation Each rule defines an action to be taken in the THEN part. The degree to which the action is valid is given by the adequacy of the rule to the current situation. This adequacy is computed by the aggregation as the degree of truth of the IF part. In the composition step, the degree of truth of the IF part of the rule is multiplied by a weighting factor, named Degree of Support (DoS). This factor represents the individual significance of the rule. By this way, rules themselves can be fuzzy, with a support between 0 and 1. The support of a consequence is calculated by linking the support of the entire condition with the DoS by composition operator. See the equation below. µ THEN =µ IF * DoS Eqn (6.5) 157

13 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance Figure 6.7 shows an example of the defined rules. Input variables Degree of support (DOS) Output variables Figure 6.7 Definition of fuzzy rules A result aggregation step determines the maximum degree of support for each consequence which is used for all further processing. Also shown in Figure 6.6, there are two methods of result aggregation, the maximum method (MAX) and the bounded sum method. If more than one rule produces the same consequence, an operation must aggregate the results of these rules. MAX operator takes the maximum of the Dos values as final result. The BSUM method takes the bounded sum. In this study, MAX method is employed to aggregate the results of these rules, as presented by the equation below. MAX result aggregation: µ RESULT = max i (µ THEN, RULEi ) Eqn (6.6) With the application of neuro-fuzzy network in this study, all the fuzzy rules were formulated automatically from the training data. Firstly, a full range of possible rules is generated. For example, say, there are two input variables and one output variable; and each variable has three classes low, medium, high. The total number of possible 158

14 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance rules is 3x3x3=27. After the neural network training, weight of each fuzzy rule is determined through Dos. The insignificant rules will be removed from the rule block. Further discussion of neuro-fuzzy training is provided in Section Defuzzification The result produced from the fuzzy rules are linguistic values, namely fuzzy, for the output variables. The defuzzification step translates the linguistic results into real values. There are several methods of defuzzification. Centre-of-Area (CoA), sometime also called Centre-of-Gravity (CoG) is the most frequently used defuzzification method. This method is based on taking the aggregate of the fuzzy outputs from each rule, weighted by their grades of fuzzy input set membership, see Figure 6.8. Figure 6.8 Deffuzification with Centre-of-Area The Centre-of-Maximum (CoM) is another commonly used method of defuzzification. It computes a crisp output as a weighted average of the term membership maxima, weighted by the inference results (Figure 6.9). The locations of the individual term membership maxima are indicated by the gray arrows and the inference result is shown 159

15 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance by the height of the black bar in the arrows. Figure 6.9 Deffuzification with Centre-of-Maximum Most CoA implementations are only approximations since they neglect overlapping and can be represented with methods such as CoM. Note that, since the quality of approximation has proved to be fair enough for the vast majority of applications, and CoA defuzzification is significantly slower than the approximated CoA methods, most hardware and software implementations of fuzzy logic algorithm only support CoM and others rather than the real CoA [FuzzyTech, 2001]. For this reason, CoM is employed in this study Neuro-fuzzy training i) Training of fuzzy rules The basic idea of neuro-fuzzy approach is to set up a complete fuzzy rule base, not by entering the rules, but by presenting a set of training samples and selecting the set of rules that represent the samples [FuzzyTech, 2001]. In the neuro-fuzzy model, neural network training function is used to evaluate the significances or weights of the 160

16 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance predefined fuzzy rules. Unlike early neuro-fuzzy network approaches, this study employs restricted training, which requires a given rule base, to restrict the number of rules used and save apriori knowledge that was entered into the system before training. This is due to the fact that if a huge unrestricted rule base does not provide a good match to the given sample distribution, the training may not converge. Configuration of neural training is shown in Figure The definitions of the terms in this dialog are shown in Appendix I. Figure 6.10 Configuration of neural training (Source: FuzzyTech, 2001) ii) Training of fuzzification and defuzzification In a fuzzy system, membership function of these parameters found in the fuzzification (input variables) and defuzzification (output variables) can also be trained. The manner in which neuro-fuzzy model treats the parameters of the membership functions of variables is similar to the training of fuzzy rules. Figure 6.11 shows an example of the 161

17 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance variable after training. Figure 6.11 Training of output variable Summary of settings of neuro-fuzzy network modeling Fuzzyification Membership function (MBF) Standard membership function Fuzzy inference Input aggregator Min operator (linguistic AND) Result aggregator Max operator (maximum of the degree of support (DoS) value) Defuzzification Method of defuzzification Centre of Maximum (CoM) Neuro-fuzzy training Learning method Batch Random 162

18 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance Winner neurons 1 Step Width (DoS) 0.1 Step Width (Term) 1% Stop criteria Max. Steps from 4000 to Max. Deviation from 50% to 1%, factor 0.75 Avg. Deviation 0.1% 6.4 Objectives of neuro-fuzzy network modeling of data centre energy performance Data centre energy performance involves a complex system of energy consuming components and elements. It comprises various energy consuming components and environmental and design variables. Neuro-fuzzy network combining the advantages of fuzzy systems, the explicit knowledge representation of knowledge, and the ability to cope with uncertainties with the learning power of neural network is a powerful technique for developing desired solution of integrating all these systems and variables into a whole system. The objectives of neuro-fuzzy network modeling in this study are as follows: a. To develop whole-system based models for data centre energy performance. b. To provide efficient and effective evaluation of data centre energy performance, and accurate estimation of energy-saving potential of data centres based on the models developed. 163

19 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance 6.5 Modeling methodology In this study, neuro-fuzzy network modeling consists of four steps. Figure 6.12 shows the systematic diagram of the modeling procedure. Parameter analysis Establish fuzzy logic model Neural network training Model verification Figure 6.12 Systematic diagram of modeling procedure Parameter analysis & establishment of fuzzy logic model Based on parameter analysis conducted in Chapter 5, variables that affect overall data centre energy performance are determined; linking those (input variables) to the key indicators (output variables) of data centre energy performance by fuzzy rule blocks, the structure of the model is determined. The results of parameter analysis are summarized as follows. The key indicators of total data centre energy performance are the total data centre energy consumption per unit gross floor area in term of kwh/m 2 /year and the energy consumption of IT equipment as a percentage of total data centre energy consumption. The significant energy consuming system of data centres are HVAC system and IT equipment. Physical variables gross floor area (m 2 ), IT footprint as a percentage of gross floor area, and total over-designing of data centre power demand plus energy consumptions of HVAC and IT equipment in terms of kwh/m 2 /year are set as the 164

20 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance significant variables of total data centre energy consumption. On the other hand, relative energy effectiveness of HVAC system which is denoted by the ratio of HVAC energy consumption to IT equipment energy consumption seems to be positively correlating to IT equipment energy consumption as a percentage of total data centre energy consumption. Based on the results of parameter analysis, model of total data centre energy performance is shown in Figure Figure 6.13 Model of overall data centre energy performance where, Grss_area is gross floor area of data centre (m 2 ) HVAC_energy is HVAC energy consumption per unit gross floor area (kwh/m 2 /year) IT_energy is IT equipment energy consumption per unit gross floor area (kwh/m 2 /year) ITfootprint is IT equipment footprint area as percentage of gross floor area (%) Total_overdesign is over-designing of total data centre power demand (-) Total_energy is total data centre energy consumption per unit gross floor area (kwh/m 2 /year) HVAC_Vs_IT is the ratio of HVAC energy consumption to IT equipment energy consumption (-) IT_Vs_Total is the IT equipment energy consumption as percentage of total data centre energy consumption (%) 165

21 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance As shown in section 5.2, affecting variable IT equipment energy consumption in terms of kwh/m 2 /year is affected by the over-designing of IT equipment power demand and IT equipment footprint area as percentage of gross floor area. These relationships are described in the model of IT equipment energy performance (Figure 6.14). Figure 6.14 Model of IT equipment energy performance where, IT_overdesign is the over-designing rate of IT equipment power demand Based on the results of parameter analysis in Section 5.4, variables such as energy consumptions of HVAC and the ratio of HVAC energy consumption to IT equipment energy consumption, which indicate energy performance of HVAC system, are also affected by other variables. Figure 6.15 depicts that HVAC energy consumption (kwh/m 2 /year) is affected by IT equipment energy consumption (kwh/m 2 /year), gross floor area (m 2 ), IT equipment footprint area as percentage of gross floor area and the over-designing of HVAC system. The ratio of HVAC energy consumption to IT equipment energy consumption is determined by the over-designing of HVAC system. 166

22 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance Figure 6.15 Model of HVAC system energy performance Moreover, as shown in Figure 5.12 and Figure 5.21, over-designing of HVAC system power demand and total data centre power demand are both affected by the over-designing of IT equipment power demand. Figure 6.16 Integrated model of total data centre energy performance 167

23 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance As stated earlier, neuro-fuzzy network is suitable for the purpose of modeling complex systems. Based on the models above, it is practical to integrate all the affecting variables and systems to a whole-system based model of total data centre energy performance (Figure 6.16). In this model, the preliminary input variables are gross floor area (m 2 ), over-designing of IT equipment power demand, and IT equipment footprint as a percentage of gross floor area. Rule block level 1 acts as the intermediate level collecting the information from input variables and transmit it to the next level defined as rule block level 2. On this step, the information that over-designing of IT equipment power demand determines the over-designing of HVAC cooling capacity and the over-designing of total data centre power demand is input to the model and passed to the next level of rule block. Rule block level 2 also acts as the intermediate level collecting the information from input variables and transmitting it to the next level defined as rule block level 3. Rule blocks 2and 4 (RB2 & 4) together illustrate the structure of HVAC system energy performance. Rule block 3 describes the significant variables that affect IT equipment energy consumption. Rule block level 3 is defined as the same as that in the previous model of total data centre energy performance (Figure 6.13). It receives the information transmitted from rule block level 2 and other independent significant variables including gross area, IT equipment footprint as percentage of gross floor area. Based on the information 168

24 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance received, the values of the output variables can be determined Neural network training Data obtained from investigations conducted in the data centres studied are then used to train the model. This training data consist of both the observed values of input variables and output variables. Based on the correlationship analysis conducted in Chapter four, data centre 6 seems often the disturbing point of the analysis. This can be explained by its significantly larger area than other facilities and the low occupancy rate of the floor area. For this reason, the data of data centre 6 is excluded from the training. However, this data centre will be used to as a new data to test the validity of the trained model and evaluate the energy saving potential. Appendix J shows the training data of the model Model verification After the training, the information of the data is learned and stored into the model, and the significance levels of the various fuzzy rules are determined. Those rules have minor influences on output variables that are removed from the rule blocks. The trained model can then be used to predict the energy performance of a data centre. Moreover, by changing one or more values of the input variables, the influences of that or those variables can be examined. 169

25 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance However, sometime the neural training may not converge, which may lead to inaccurate prediction and analysis. This is due to many reasons. The main reason is the constraint of sample size. In this study, only 6 sets of data obtained from the selected data centres are available for the training and testing of the model. Moreover, there are significant variations among data centres in terms of energy consumption, power demand and systems efficiencies. These can lead to a poor convergence of the model developed. Improper model structure and unrestricted and great number of fuzzy rules can also result in improper model. In this case, further re-examination of model structure and more sample data are needed. Figure 6.17 Training result in 3-D plot Figure 6.17 depicts the trained model of total data centre energy performance in a 3-Dimensional plot. Relationships between HVAC energy consumption (vertical axis) and Gross area and Over-designing of IT equipment power demand (horizontal axes) are depicted graphically. 170

26 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance 6.6 Evaluation of energy performance and prediction of energy saving potential using neuro-fuzzy network model Input values Output results Figure 6.18 Remote Control Unit of neuro-fuzzy model in Excel After training, information of training data is learned by the model. With the application of the Remote Control Unit (RCU) [FuzzyTech, 2001], the neuro-fuzzy model is linked to the database in Microsoft Excel, as shown in Figure By this way, the predicted output variables recording data centre energy performance are calculated. By comparing the predicted values with the real data obtained from measurement, the energy performance of each data centre is evaluated, and the energy saving potential is examined. 171

27 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance Table 6.1 Comparison between actual and predicted energy performances of data centres studied HVAC energy consumption (kwh/m2/year) IT equipment energy consumption (kwh/m2/year) Total data centre energy consumption (kwh/m2/year) HVAC/ IT IT/ Total Data centre Measured Predicted Deviation (%) % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % Table 6.1 shows the results of the comparison between the actual data and value predicted by the neuro-fuzzy network model, where HVAC/ IT is the ratio of HVAC energy consumption to IT equipment energy consumption and; IT/ Total is IT equipment energy consumption as percentage of total data centre energy consumption. The results are also shown graphically in the Figures following. 172

28 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance It shows that in some cases, the deviations between predicted and actual values are significant. This, however, points out the direction of improving energy performance, and indicates the high energy saving potential of these data centres. For example, in case of the data centre 2, the predicted HVAC energy consumption is approximately 56% less than the actual value. This indicates that based on the database of the data centres studied, the energy consumption of HVAC system of data centre 2 is far from the reasonable range and has the potential to be reduced by 56%. On the other hand, the predicted value of IT energy consumption as percentage of the total energy consumption is 42% higher than the actual data, while it indicates the actual poor energy performance of the data centre 2. The results of deviation between predicted values and the actual measures are in great agreement with the findings. Another good example is the data centre 4. In this case, the predicted ratio of HVAC energy consumption to IT equipment energy consumption is 19% higher than the actual consumption, which is the only positive deviation among the 6 data centres studied. This indeed represents the better energy performance of HVAC system in the data centre 4 than the others. In the meanwhile, the predicted IT energy consumption as percentage of the total consumption is 18% less than the real situation, which confirms the good energy performance of the data centre 4. These results are also consistent with the finding of measurement. It can also be observed that the prediction deviation of the data centre 6 is extremely high, as compared with other facilities. Firstly, this is due to that fact that the data centre 6 is not involved with training. Secondly, this confirms that energy performance 173

29 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance of data centre 6 varies significantly from the others. Deviation % 120% 100% 80% 60% 40% 20% 0% -20% -40% -60% -80% Data centre Figure 6.19 Deviations between predicted and actual HVAC energy consumptions per unit of gross floor area Figure 6.19 shows the deviations between predicted and actual measures of HVAC energy consumption per unit gross floor area. It shows that the predicted energy consumption of HVAC systems in the data centre 6 and 1 are far higher than the actual values, while in the data centre 2 the prediction is approximately 56% lower than the actual measure, which indicates a minimum energy saving potential of 56% of the current energy consumption. By comparison, the predicted and actual consumptions of the data centres 3, 4 and 5 are very close, which reflects that the current situations of HVAC energy consumption are within reasonable ranges in these data centres. However, these positive findings may be highly different when better practices and efficient data centres are involved in the database of this study. The predicted performances of data centres will be on a higher level. Further case studies are necessary in this case for developing robust 174

30 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance database of data centre energy performance, and developing best practices. Deviation % 160% 140% 120% 100% 80% 60% 40% 20% 0% -20% Data centre Figure 6.20 Deviations between predicted and actual IT energy consumptions per unit of gross floor area Figure 6.20 shows that the predicted IT equipment energy consumption of the data centre 6 is almost 136% higher than the real measure, which may indicate a very low IT equipment load on the gross floor in the data centre 6. Compared with the data centre 6, most of the other data centres have relatively good predictions, which means the current situation is within a reasonable range. However, it seems that data centre 1 still has nearly 30% capacity to house more IT equipment. Figure 6.21 shows that data centres 2 and 5 were consuming 28% and 20% extra energy for the whole data centre than the predicted value, which respectively indicates at least 28% and 20% energy saving potential in these two facilities. The predictions of data centres 3 and 4 shows good agreement with the actual situation; however, there are still many opportunities for pursuing better energy efficiency and significant energy saving in data centre 3, according to the field measurements. The high 175

31 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance deviation occur in data centre 1 and 6 can be due to the low occupancy of the floor area. 120% 100% 80% 60% 40% 20% 0% -20% % Figure 6.21 Deviations between predicted and actual total data centre energy consumptions per unit of gross floor area Deviation % 30% 20% 10% 0% -10% -20% -30% -40% -50% -60% -70% Data centre Figure 6.22 Deviations between predicted and actual HVAC/ IT equipment (energy consumption) The deviations of the predictions of the metric indicating relative energy efficiency of data centres are significant, as can be seen in Figure Data centres 1, 2, 3and 6 176

32 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance show inefficient operations of space cooling than the situations are expected. By comparison, data centre 4 shows an outstanding performance of HVAC system with its current conditions of gross area, IT equipment footprint as percentage of gross floor area and over-designing of IT equipment. 50% 40% 30% Deviation % 20% 10% 0% -10% -20% -30% Data centre Figure 6.23 Deviations between predicted and actual IT equipment/ total data centre (energy consumption) Figure 6.23 shows interesting results and provide great agreement to the real measurements. According to the curve, data centres 2, 3 and 6 have unexpected inefficient energy performance of the whole facilities. By comparison, data centres 1 and 5 appear to have reasonable energy performance currently; and data centre 4 presents the best overall energy performance, which is also consistent with the actual finding. Form this graph, it can determined that IT equipment as percentage of total data centre energy consumption is a promising indicator of overall data centre energy performance, and the neuro-fuzzy network model can predict and evaluate it well. Predictions above show great agreement with the findings of field measurements. 177

33 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance Using this model, it will be more meaningful to examine the influences of input variables on the total energy performance and determine the energy saving potential of data centres. Efforts were provided to estimate energy saving potential by reducing over-designing of IT equipment power demand of data centres. However, the influences are not significant after lessening half of the over-designing of IT equipment power demand (Table 6.2). Table 6.2 Estimation of energy saving potential by modifying over-designing of IT equipment power demands of data centres Data centre Actual IT/ Total Prediction Prediction 2 a a After decreasing over-designing of IT equipment power demand to 50% of the current value. This can be explained by the weak database and significant variations of design and operation among the six data centres studied. To establish a dynamic whole-system based and well-predicting model, a broader and robust database of data centre energy use pattern is a must. 6.7 Conclusions An exploratory study of the whole-system based modeling of data centre energy performance is presented in this chapter. It shows that neuro-fuzzy network is a desired and applicable candidate of modeling energy performance of data centres. This approach presents the advantages of mimicking human decision and self training by existing data sets, and the advantage of modeling complex relationships which could 178

34 Chapter 6: Neuro-fuzzy Network Modeling of Data Centre Energy Performance be non-linear. A well structured and trained neuro-fuzzy model can act as evaluation tool kits at both design stag and post-diagnostic stage of data centres. Energy potential can also be determined quantitatively and dynamically. However, small sample size has constrained the validities and accuracy of the model developed. Further benchmarking activities and collaborations with data centre designers and operators are needed. 179

35 Chapter 7: Conclusions CHAPTER 7 CONCLUSIONS Data centres are facilities with rapid rate of growth in modern knowledge economies. Owing to the nature of housing concentrated IT equipment and high cooling load required, data centres are characterized by significantly higher power demands and energy consumptions than those of normal residential or commercial office spaces. However, the hasty installations, absence of design standards and guidelines, few practical benchmarking and the lack of financial incentives of using energy efficient technologies financial often lead to poor design and inefficient energy use in data centres. Recently, due to the rising concerns of energy crisis and its related pollution issues, more research regarding high power demand and energy consumption, and improving energy performance of data centres are conducted in some Western countries. However, to date in Singapore and other countries in the Southeast Asian region under tropical climate, relevant studies are very limited. Hence, this study aimed at determining an empirical energy usage pattern of data centres in Singapore, is an exploratory and valuable effort. 7.1 Review and achievement of research objectives First objective 180

36 Chapter 7: Conclusions To determine an empirical energy usage pattern of data centres in Singapore under tropical weather conditions Case studies and field measurements are conducted in the six data centres in Singapore. Investigations involve energy consumptions and power demands of energy consuming systems, space usage of the gross floor area, indoor environmental conditions, the design and operation strategies, as well as energy saving potentials of data centres. i) Energy consumption and power demand It shows that data centres are highly energy-intensive facilities as compared to normal commercial office spaces in Singapore. Table 7.1 summarizes the measured total electricity energy consumptions and power demands of the six data centres studied, in comparison with normal office spaces in Singapore. Table 7.1 Summary of overall energy consumptions and power demands of data centres Data centre Total electricity energy consumption (kwh/m 2 /year) Total electrical power demand density (W/m 2 ) Normal office space in Singapore (Source: CTBP, 2003)

37 Chapter 7: Conclusions ii) Over-designing In this study, over-designing of infrastructure has been proven to be the universal problem of data centres. These include the over-designed initial power demands and the over-sizing of supporting systems such as HVAC and UPS, as well as the floor area. All these can result in excessive and wasteful capital investment, long term maintenance and replacement costs that are caused by the inefficient operation of supporting systems. It is found that initial power demands of IT equipment are over-designed by a factor of 3 to 16. iii) Energy efficiencies of supporting systems In this study, the term energy performance is a comprehensive evaluation of energy use in data centres, which include not only energy consumptions and power demands, but also operational energy efficiencies of the supporting systems such as HVAC system and uninterruptible power supply (UPS) units. For the purpose of evaluating HVAC efficiency, the ratio of HVAC systems energy consumption to IT equipment s energy consumption is adopted as an alternative indicator. The ratios range widely from approximately 0.8 to 2.7, which reflects a significant variation among data centres regarding the energy performance of HVAC systems. It demonstrates that low load factor often leads to poor efficiency of UPS. For example, loaded at 13% the UPS system of the data centre 6 has a poor efficiency of approximately 58%, nearly 46 kw power is wasted in UPS. An alternative and useful indicator of overall data centre energy performance is the 182

38 Chapter 7: Conclusions energy consumption of IT equipment as percentage of the total data centre energy consumption. Among the six data centres studied, the percentages vary from approximately 25% to 49%. These are relatively low numbers as compared to those of the data centres investigated in the US. iv) Opportunities of energy saving Opportunities of significant energy saving of the major energy consuming systems are observed in the six data centres studied. These systems include IT equipment, HVAC system, UPS and lighting. Based on the measurements conducted and the review of previous studies, recommendations on improving energy performance are provided for the six data centres studied and the design of future data centres. Second objective To develop whole-system based models of data centre energy performance. For the purpose of evaluating energy performance of data centre and determining energy saving potential from a systematic perspective, whole-system based models of total data centre energy performance are developed. These models consist of the energy consumption and power demand of the main energy consuming systems in the data centre and various factors of influencing data centre energy performance. These factors include physical parameters of space and environment conditions, and other strategies of design and operation of data centres. Neuro-fuzzy network characterized by the high capability of modeling complex system 183

39 Chapter 7: Conclusions is employed in this study. By using this approach, influences of various parameters on the energy performance of systems as well as total data centre facility are examined in a dynamic way. Furthermore, energy saving potential can be predicted quantitatively within an acceptable range of accuracy. However, the small sample size of the case studies and measurements, and the observed significant variation of energy performances among data centres may limit the validity of the models developed. To improve on the accuracy of the models, further investigations are needed. 7.2 Contributions of the study Firstly, the energy usage pattern presented in this study, which includes energy consumption in terms of kwh/m 2 /year, power demand density in terms of W/m 2, and the affecting variables such as space usage, environmental conditions, strategies of design and operation, as well as the discussion of applying energy efficient design and new technologies could aid designers of future data centres to make informed decisions about design and construction of data centres. Secondly, methods of the field measurement and evaluation of data centre energy performance shown in this study can be applied to the further study of similar facilities. Thirdly, since utility power supply has been deregulated in Singapore, energy contract will be an up and coming trend. For this reason, accurate estimations of initial power demands and actual energy consumption will be significantly important for data 184

40 Chapter 7: Conclusions centres. The output of this study could contribute significantly to the accurate and reasonable estimations of data centre energy consumption. Fourthly, the benchmarking of energy performance carried out among data centres provides building managers or owners with a better understanding of the position of his/her facilities. In this way, the building manager or owner may be inspired/ motivated to better the facility s efficiency. Lastly, the neuro-fuzzy network modeling is a desired tool for the evaluation of data centre energy performance, and examining the potential of energy saving of such facilities. This can be highly useful for the Energy Services Company (ESCO), when conducting diagnostic and retrofitting projects of data centres. Moreover, using similar approach of neuro-fuzzy network modeling, this can be used for a wider application such as the evaluation of energy performance of entire building. 7.3 Recommendations of future studies Future investigations and benchmarking exercises are greatly valuable for setting up robust database and developing best practices for better designs and operations of data centres. These include not only the corporate data centres, but also other types of data centres, such as out-sourcing data centres. Study of the application of neuro-fuzzy network modeling to evaluate energy performance of highly intense energy consuming facilities like data centres is just on the beginning stage. Future efforts in this direction are needed to examine the validity of such approach in building energy performance study. 185

41 Bibliography BIBLIOGRAPHY Abernethy, R. B., et al. (1973). Handbook Gas Turbine Measurement Uncertainty, AEDC. Aebischer, B., R. Frischknecht, C. Genoud, A. Huser and F. Varone. (2003). Energy-and eco-efficiency of data centres. Département de l intérieur, de l agriculture et de l environnement. Service cantonal de l énergie. Geneve. Switzerland. Angel, J. (2001). Emerging technology: energy consumption and the new economy. Network Magazine. Anderson, R. (2001). Wattage where it s needed. New York Times. ANSI/ASME PTC 19.1 (1985). Performance Test Code, Instruments and Apparatus, Part 1, Measurement Uncertainty, P.12. ASHRAE. (1975). ASHRAE STANDARD Beck, F. (2001). Energy smart data centers: applying energy efficient design and technology to the digital information sector (Report No.14). Renewable Energy Policy Project (REPP). Washington, DC. Brüniger, R. (2004) Energy efficiency of servers, Improving Energy Efficiency in Commercial Buildings "IEECB'04 Conference, Frankfurt, Germany. BSI. (1975). Guide to Statistical Interpretation of Data, BS 2846, Part 1. Routine analysis of quantitative data. British Standards Institution. Callsen, T. (2001). Internet hotel-a big draw. Transmission & Distribution World. Cook, J. (2000). Internet data gain is a major power drain on local utilities: Server farms voracious appetite for electricity sparks several concerns. Seattle Post-Intelligencer. CP 24. (1999). CODE OF PRACTICE FOR: Energy efficiency standard for building services and equipment. ISBN Singapore Productivity and Standard Board. 1 Science Park Drive, Singapore CTBP. (2003). Energy Performance Assessment and Classification of Commercial buildings in Singapore. (ongoing study). Project No.CRP01/12/17. Center of Total Building Performance (CTBP), Dept. of Building, National University of Singapore; Building Construction Agency (BCA). Singapore. 186

42 Bibliography Dieck, R. (1992). Measurement Uncertainty Methods and Applications. Instrument Society of America, Research Triangle Park, NC. Drecher, W. H. (2002). Copper in the new economy. Wired Magazine, Encyclopedia of the Economy. E-Source. (2001). Delivering Energy Services to Internet Hotels and Other High-Density Electronic Loads. Prospectus, E Source Multi-Client Study. Eubank, H., J. Swisher, C. Burns, J. Seal, and Emerson, B. (2003). High performance data centers. Rocky Mountain Institute (RMI). Colorado. FuzzyTech. (2001). FuzzyTECH Germany. 5.5 User s Manual. Inform Software Corp. Gouda, M. M., S. Danaher and C. P. Underwood. (2001). Thermal comfort based fuzzy logic controller. Building Serv. Eng. Res. Technol. 22, 4 (2001) pp Grahame, T. J. and D. Kathan. (2001). Internet data centers & electricity growth: Will the emerging power crisis burn your bottom line? Broadband Wireless Business Magazine. Hartkamp, F. (2002). Energy consumption of ICT in the Netherlands. Perspective to the year Presentation at Impact of ICT on the energy system. Workshop of the International Energy Agency, Paris. HostPro. (2001). HostPro spec sheet on cabinets. IBM. (2001). High-Powered Initiative for low-power Products and Services, Press Release. Kennet, S. (2000). Main features information technology: Net gains. Building Services Journal. Krarti, M, J. F. Kreider, D. Cohen and P. Curtiss. (1998). Estimation of Energy Saving for Building Retrofits Using Neural Networks. Journal of Solar Energy Engineering. Vol. 120/211. Krarti, M. (2003). An Overview of Artificial Intelligence-Based Methods for Building Energy Systems. Journal of Solar Energy Engineering. Vol. 125/ 331. Kunz, M. (1997). Energy consumption of electronic network components. Zurich: Bundesamt für Energiewirtschaft Forschungsprogramm Elektrizität, Basler & Hofman. Lazarus, D. (2001). Net complex a dilemma for San Jose / SERVER FARM: plant would tax grid. San Francisco Chronicle. 187

43 Bibliography LBNL. (2003). Data center website of Lawrence Berkeley National Laboratory. Lewis, S. M. (2001). Utilities cannot afford to become sometimes power & light. Transmission & Distribution World. Mahedy S., D. Cummins and D. Joe. (2000). Internet data centers: If built will they come? New York: Salomon Smith Barney. Manning, C. B. (2001). Comments from Charles B. Manning, Jr. General Manager, Austin Energy. The Model Data Center Energy/Design Meeting. Austin Energy (AE). Austin, Texas. Miller, R. (2001a). How much power? Survey shows split among facility operators. CarrierHotels.com. New York. Miller, R. (2001b). More developers adding power plants: calif. crisis, utilities' demands for large deposits driving trend. CarrierHotels.com. New York. Miller, R. (2001c). Smaller Servers, Larger Loads. CarrierHotels News online. Miller, R. (2001d). Is Internet Usage Surging? CarrierHotels News online. Mitchell-Jackson, J. D. (2001) Energy needs in an internet economy: a closer look at data centers. University of California, Berkeley. Mitchell-Jackson, J., J. G. Koomey, M. Blazek, and B. Nordman. (2002). National and regional implications of internet data center growth in the US. Resources, Conservation and Recycling, 36(3): Mitchell-Jackson, J., J. G. Koomey, B. Nordman, and M. Blazek. (2003). Data center power requirements: measurements from Silicon Valley. Energy, 28, Mott Green & Wall and Davis Langdon & Everest. (2001). Cost model: Data centres. Building Services Journal. Newave. (2003). PG&E. (2000). Energy solutions and supersymmetry, data center market research study. Presentation for PG&E Customer Energy Management. PG&E. California. Plant-TECH Associates. (2002). A technical and market assessment of premium power in the harverhill cyber-district. The Massachusetts Technology Collaborative. Westborough. RLX Technologies. (2002). Product FAC, The Woodland,TX. Robertson, C. and J. Romm. (2002). Data centers, power, and pollution prevention: design for business and environmental advantage. The Center for Energy & 188

44 Bibliography Climate Solutions. The Global Environment and Technology Foundation. SCDF. (2002). Code of Practice for Fire Precautions in Buildings 2002 Volume 4 - Purpose Group IV Office. Singapore Civil Defence Force. SP. (2004). Singapore Power: Snevely, R. (2002). Enterprise data center design and methodology. Sun Microsystems Press. Palo Alto, Calif. Sun H. S., W. Schafer and S.E. Lee. (2004). Accurate Energy Measurement in Data Centres of Commercial and Administrative Buildings. Improving Energy Efficiency in Commercial Buildings (IEECB 04). Frankfurt, Germany. Tadpole. (2001). Tadpole and Platform Computing Launch PowerBack to Reduce Energy Costs for Power-Hungry Data Centers. TC 9.9. (2004) ASHRAE Technical Commettee Tschudi, W., T. F. Xu, D. Sartor, and J. Staein. (2003a). High performance data centers: a 10 year research plan for data center buildings (LBNL-53483). Lawrence Berkeley National Laboratory. Berkeley. Tschudi, W., P. Sreedharan, T. F. Xu, D. Coup, and P. Roggensack, (2003b). Data Centers and Energy Use-Let s Look at the Data. ACEEE 2003 Paper #162. TUI. (2000). TUI. (2003). Website: Uptime. (2000). Heat density trends in data processing, computer systems, and telecommunications equipment. Webopedia. (2003). Wu, C. F. (2003). Design recommendations for high-performance data centers. The RADIANT Group, Los Alamos National Laboratory. 189

45 Appendix A: Common energy consuming components of data centre 190

46 Data centre energy consuming systems IT equipment UPS HVAC Lighting Other auxiliaries Servers Data storing devices Main frames Monitors Network equipment: Switches Routers Firewalls Others UPS Batteries CRAC: Water-cooled unit Air-cooled unit Other types of CRAC Central air condition FCU Fluorescent lights Current transformer Diesel generator Office equipment Building control and automation system Fire suppression Telecommunication system Appendix A: Common energy consuming components of data centres 191

47 Appendix B: Questionnaire and forms 192

48 Appendix B.1 Specifications of energy consuming system and equipment Energy consuming system of data centre Brand & Model Unit Power Rating (kw) Total number of units Number of units in standby Total kw Installed year Remarks Appendix B.2 Operation schedules Energy consuming system of data centre Weekday (Mon-Fri) Weekend (Sat) Weekend (Sun) Public Holiday Total operation time (hours/week) Remarks Appendix B.3 Design criteria of data centre Cooling Load (W/ m 2 ) IT equipment power density (W/ m 2 ) HVAC power density (W/ m 2 ) Total data centre power density (W/ m 2 ) Temperature ( o C) Relative Humidity (%) Lighting Level (Lux) 193

49 Appendix B4: Data centre background Building name: Building address: Owner of data centre: Contact persons/ division: Tel: Survey date: Investigators: Notice: 1. Please put NA if it is not applicable. 2. Essential questions must be answered. Other questions, if you don t know the answers, please leave them blank. Data Centre installed year (Essential): Data Centre Location (Essential): Data Centre Area (Essential): Data Centre Raised Floor Area (Essential): Type of the Data Centre: Corporate data centres Managed data centres Co-located server hosting facilities Internet Service Provider routers Data storage and Internet hosting facilities Telecommunication switches Others (please specify): 194

50 Type of HVAC system (Essential) Cooling System (multiple selections) Central cooling system with AHU/CRAC Water-cooled package system Small stand- alone split system Fan coil units (FCU) Air-cooled package system Others (please specify): Air Distribution System (multiple selections) Computer room air conditioning unit (CRAC): Directly blow cooling air through server racks: Others (please specify): Whether have data centre power management software, such as PowerBack*. (PowerBack is a technology which monitors and powers down unneeded servers, powering them back up when required.) No Yes (please specify): Number of worker/operator working regularly inside data centre (Essential): Their working schedule (Essential): Weekdays Saturday Sunday Public Holidays Start Time End Time During Hours 195

51 Appendix C: Introduction of TEM 1 196

52 TEM 1 True Energy Meter with Pulse Outputting Data Logger Dr. Wolfgang Schafer, Research Fellow, Energy Sustainability Unit, Centre of Total Building Performance, Dept. of Building, National University of Singapore The TEM1 multiplies current and voltage instantaneously which results to real power and totalizes this to pulses, which represent True Energy. Power factor and form factor (for non-sinusoid curves) are automatically taken into account by this method. Accuracy is higher than required by the standard IEC 1036 for electrical billing meters. Accuracy of the current clamp is 2 % of full scale, selectable to 0-30A, 0-100A and 0-300A.Higher current values will be measured with a current clamp for 600A. Voltage measurement and analog processing has accuracy of 0.3 %, the digital multiplying, totalizing and data logging by pulse count has not failures possible. The TEM1 uses the new true power chip ADE 7757 from Analog Devices Inc., released in October 2002, which has 2 analog/digital converters integrated with 12 bit resolution for voltage and current signals. It multiplies both digitally to obtain true real rms power values, which are then totalized (digitally counted-up), to energy pulses. For detailed information see website selecting ADE7757. Each pulse will be indicated by a green LED for measurement and a yellow LED for calibration respectively. A connected data logger sums-up these pulses during selectable time periods of e.g. 10 min., allowing to collect 4000 samples (55 days at 10min periods). Retrieval from data logger is possible in graph form with indicated time scale containing date and time, as well as in spreadsheet form with up to 4000 lines, for further calculation to obtain sums of energy per day or else and to calculate real power, averaged over the sampling period of e.g. 10min.. Good experiences were made with LASCAR data logger type EL-2 (8bit resolution), available through Farnell. This data logger is small; battery supplied and can clipped-on to the TEM1. Easy-to-use software allows to preset the data logger to different sample periods, e.g. 10min for measuring and 5s for calibration. Although this data logger has a resolution of 8 bits, so can store up to 255 pulses per period only, this is enough (less than 0.5 %) per period. But because the Chip in the TEM1 is continuously totalizating, the lower bits are not lost but added to the next period counts. Therefore the resolution after 2 periods is 9bit, after 3 periods is 10bit and so on. If the period is selected to 10min, the resolution of this short period is 8bit (255 pulses max.), but for the whole day of 24 hours is pulses max., representing a very high resolution. 197

53 TEM 1 True Energy Meter, Layout and Operating Functions with Data Logger The TEM1 multiplies Current and Voltage instantaneously which results to Real Power and totalizes this to Pulses, which represent True Energy. A Data Logger will be connected to TEM1 for sampling the pulse counts. It sums-up these pulses during selectable time periods of e.g. 10 min., allowing to collect 4000 samples (55 days at 10min periods). Retrieval from data logger is possible in graph form with indicated time scale containing date and time, as well as in spreadsheet form with up to 4000 lines, for further calculation to obtain sums of energy per day or else and to calculate real power, averaged over the sampling period of e.g. 10min. For calibration, the sampling period will be shortened from 10 min. to 5 s to save time. The TEM1 will be switched to calibration mode to generate more pulses per time unit and the data logger set to 5s sampling time. A micro trimmer with 13 turns allows trimming the voltage influence and therefore the power calculated. Accurate and stable voltage of 230VAC rms shall be supplied. But to avoid the inaccuracy of the current clamp, it will be removed and a voltage of 1 VAC rms, in phase with the 230V voltage, will be supplied to the TEM current signal input jacks. Then the trimmer will be adjusted, until the pulse count during 5s results in 246 pulses. When switched-back to measuring mode and 10 min. sampling time, each of the maximal 255 pulses (8 bit resolution) represent: Wh at 300A range / or Wh at 100A range / or Wh at 30 A range. The current range selection does not need to be calibrated because they are made of professional resistors with 0.1 % accuracy and max. temperature failure of 30ppm. Instruction for Calibration of True Energy Meter TEM1 with Data logger EL2 Signals for calibration: 1) An AC signal of 230V rms, 50Hz, with good sinusoid curve form and stable over a period of 10min to 30min, load 15 ma max. 2) A second AC signal of 1 V rms, 50Hz, with good sinusoid curve form, totally in phase with first signal of 230V, stable over a period of 10min to 30min, load 1 ma max. 198

54 Connections and Settings: 1) The 230V signal shall be supplied to the voltage probe connectors of the TEM1, neutral to the blue and Life to the red connector of the cables. 2) The 1V signal shall be supplied to the current measuring input jacks. These are the 2 plug input to the TEM1, which will be available after removing the current clamp from the TEM1. The polarity is meaningless. 3) The current range selector on the TEM1 shall be switched to 100A. 4) The mode switch on the TEM1 shall be set to Calibrate, that is to the right position as shown on the figure above. How to measure the energy: 1) The principal method of measuring the energy is to count the energy pulses from the TEM1 over a fixed period of time. This period should be shorter for calibration than for normal measurement in order to save time at calibration. This method is similar to calibrating normal mechanical electrical meters. 2) For counting-up the pulses the data logger EL-2 can preferably be used. The pulse output connector of the TEM1 will then be plugged to the EL-2. 3) The data logger EL-2 has to be preset by a PC with the software EL-Win to: Count mode, Sampling period of 20s, Starting by push the button to start. After this there are 22 hours available for calibration or else until the data logger is full and has to be reset with the PC and Software again (8000 store values max.). Because the data logger EL-2 is battery-supplied, the battery should be checked for sufficient capacity or exchanged by a new one (type Lithium 3.6V size ½ AA ). Calibration Procedure: 1) After all connections made as listed above, the data logger will be started by pushing its button located under its LCD display. The display will change indicating from ---- to 0. 2) After 20s sampling time, the data logger will display the figure, which represents the number of pulses it has summed-up during the previous sampling period. It continues to sampling for the next period of 20s and displays then the result of the new period. The figures displayed will be around 245 and maximal

55 3) To change the count figure of around 245, the Calibration Trimmer in the middle of the TEM1 (see figure above) have to be turned by a small screw driver of 2mm width. Turning to right (clock-wise) will increase the count number, to left will decrease. The Calibration Trimmer allows 13 turns max. 4) The count number to calibrate to as the goal can be freely selected. The energy value per pulse when switched to measuring mode will have a calculated value, related to the calibrated count figure. However it is recommended to calibrate to the count figure of 246, which uses the facilities optimally. Then the energy per pulse in measuring mode is, depending on the selected current range: 300A current range : kwh ( 50Wh) 100A current range : kwh ( 17Wh) 30A current range : kwh ( 5Wh), all for a single-phase system. 5) Fine tuning of calibration is possible by using more than one sampling period of 5s each. The fine tuning will require, that several periods in sequence all show the same count sum, e.g. 246 as preferred. If 3 subsequent periods will show the same figure 246, then the sum during 3 x 5s will be 3 x 246, which is 3-times more accurate than if only one period will show the figure 246. The reason is, that although the data logger can store figures up to 8bit length, i.e. 255 only, the resolution of the TEM1 is much higher. The higher resolution will be made use of by extending the total sampling time to several periods of the data logger, overcoming its barrier of 8bit resolution per one period. Accuracies required Inaccuracies occur in the analog parts only, the digital part is fully accurate. The analog parts excluding the current clamp will be fully dependent on the accuracy of the calibrator devices, mainly the accuracy pf the 2 AC signals and its stability over the sampling periods. The inaccuracies of the calibrator signals, curve form and synchronized phase shall not exceed the value of 0.1 %. 200

56 Appendix D: Samples of measured data centre power demands 201

57 Appendix D.1: Sample of measured data of data centre power demands Measured power demands (kw) of data centre 2 Time Date Total data HVAC UPS input Lighitng Others centre 1 0:00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/

58 44 7:10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/

59 91 15:00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/

60 138 22:50:47 12/3/ :00:47 12/3/ :10:47 12/3/ :20:47 12/3/ :30:47 12/3/ :40:47 12/3/ :50:47 12/3/ :00:

61 Appendix D.2 UPS power demand recorded by UPS monitoring system Month: July 2002 Voltage= 230 v Date Time: am Time: 3.00 pm Remarks I1(A) I2(A) I3(A) I1(A) I2(A) I3(A) (KVA) 1 (Mon) (Tue) (Wed) (Thur) (Fri) (Sat) (Sun) (Mon) (Tue) (Wed) (Thur) (Fri) (Sat) (Sun) (Mon) (Tue) (Wed) (Thur) (Fri) (Sat) (Sun) (Mon) (Tue) (Wed) (Thur) (Fri) (Sat) (Sun) (Mon) (Tue) (Wed) Average

62 Appendix E: Specifications of HVAC and UPS 207

63 Appendix E: Specifications of HVAC and UPS systems Data centre Energy consuming system Brand & model Description Rated power per unit Total number of units Number of units stand-by Total power (kw) Remarks Water cooling package CANATAL (CCV)- 3 Nos. CAV TB5-1B 3 compressors 11.5 kw Cooling tower unknown 11.0 Estimated rated power by data centre operators Condense water pump unknown UPS ABB 60 kva Power factor 0.87 UPS MGE: Galaxy 20 kva Water-cooled package RC Condizionatori/DXW.U.1 5.2E sensible cooling load capacity 56.4 kw 14.9 kw Air-cooled package RC DXA.U.20.E1.6 (aircooled) sensible cooling load capacity 24.5 kw 20 kw Designed for stand-by of water cooled package, however, since water cooled package was out of order during the measurement, it worked as main cooling supplier. 2 Cooling tower Kuken/SKB-POR 900 Condense water pump- CCWP Ajax/ 150 x / Back-pull out / end 1-type capacity, two fans (5.5 kw each) 11 kw type capacity 18.5 kw Shared by data centre 2 and other two smaller computer room in the building. Calculation of rated power of cooling tower and condense water pump proportionally according to data centre areas. Old main UPS Silcon DP330BC 30 kva Power factor 0.8 New main UPS Small UPS Concept Power 30 kva ( catridge slot type) Aline Power System 9000s 30 kva Power factor 0.8, input 3x400V+N, Inpur current 45A, Output 3x400V+N, output freq 50Hz. Circuit breaker 63A. 9 kva Power factor

64 Data centre Energy consuming system Brand & model Description Air-cooled chiller TRANE_Koolman500 _CGAK1007DARGR NA-00 Cooling load capacity 25.7 kw. Rated power per unit Total number of units Number of units stand-by Total power (kw) 9.6 kw Remarks Fan coil unit (FCU) unknown 4 kw UPS iwatec-m kva Single phase, power factor 0.8 Water-cooled package Liebert system 3. PAC12-1&12-2. Cooling load capacity 17.3 kw kw Cooling tower NSCooling Tower (Japan) 20 Ton 0.3 kw Condenser water pump EBARA (Japan), RW m 3 / min. 2.2 kw UPS MGE_Galaxy kva Power factor 0.9 Air-cooled package STULZ CompTrol kw UPS Merlin Gerin Galaxy- SN:UU kva Power factor 0.8, battery capacity 100Ah, rated autonomy 15 mins, 400V/50Hz. 209

65 Data centre 6 Energy consuming system Brand & model Description Rated power per unit Total number of units Number of units stand-by Total power (kw) Remarks Chiller Carrier-- 30HG320 Cooling capacity: 671kW. 160 kw Cooling Tower Shinwa SDCu250ASS 7.5 kw Condense water pump ITT BELL GOSSETT VCS6x6x kw Chill water pump ITT BELL GOSSETT VCS6x8x9x34HL 7.5 kw CRAC units Citec-Denco Uc/Dc kw fans per unit, 4 kw/ fan. FCU unknown 21.4 kw UPS ABB 350 kva Power factor 0.7~

66 Appendix F: Correlation between outdoor air temperature and HVAC system power demand 211

67 y = 0.161x R 2 = Outdoor dry-bulb temperature (oc) y = x R 2 = Outdoor dry-bulb temperature (oc) HVAC power demand (kw) HVAC power demand (kw) Appendix F.1 Data centre 2 Appendix F.2 Data centre y = x R 2 = y = x R 2 = Outdoor dry-bulb temperature (oc) Outdoor dry-bulb temperature (oc) HVAC power demand (kw) HVAC power demand (kw) Appendix F.3 Data centre 5 Appendix F.4 Data centre 6 212

68 Appendix G: Specification of electrical power supply circuits of HVAC and UPS systems 213

69 Appendix G: Specifications of power supply circuit capacities Data centre Power supply circuit capacity (HVAC) Power supply circuit capacity (IT equipment) 1 415V/100A/3 phases 415V/250A/3 phases 2 415V/90A/3 phases 415V/25A/3 phases 3 415V/170A/3 phases 415V/75A/3 phases 4 415V/130A/3 phases 415V/200A/3 phases 5 415V/100A/3 phases 415V/150A/3 phases 6 415V/1230A/3 phases 415V/1900A/3 phases 214

70 Appendix H: Monitored indoor environmental conditions 215

71 Dry-bulb temperature (oc) and RH (%) RH % Temperature oc 14:10 15:40 17:10 18:40 20:10 21:40 23:10 0:40 2:10 3:40 Time 5:10 6:40 8:10 9:40 11:10 12:40 14:10 Appendix H.1 Indoor environmental conditions of data centre 2 Temperature (oc) and RH (%) RH % Temperature oc 5/8/2003 5/8/2003 6/8/2003 6/8/2003 6/8/2003 6/8/2003 6/8/2003 7/8/2003 7/8/2003 7/8/2003 8/8/2003 Date 8/8/2003 9/8/2003 9/8/2003 9/8/ /8/ /8/ /8/ /8/2003 Appendix H.2 Indoor environmental conditions of data centre 6 216

72 Appendix I: Configuration and terms definitions of neuro-fuzzy training 217

73 Appendix I: Neuro-Fuzzy configuration and terms definitions Contents: The Neuro-fuzzy configuration dialog consists of sections to select Neuro-fuzzy Learn Parameter The training is controlled in the learn parameters section. [Stepwidth(DoS)], [Stepwidth(Term)], and [Winner neurons] determine the update policy of the training iteration. [Stepwidth(DoS)] sets the value for the learn rate used for updating the DoS of fuzzy rules. Note that this value is used differently depending on the selected learn method. DoS values are normalized, thus the learn rate is entered as an absolute value. [Stepwidth(Term)] sets the value for the learn rate used for updating the position of terms of linguistic variables. The value is used differently depending on the selected learn method. The value is entered as a percentage of the base variable range. [Winner neurons] determines the number of open parameter, which are updated one at a time by the training algorithm. Neuro-fuzzy Stop Conditions The Neuro-fuzzy module includes different stop conditions that can be separately enabled or disabled. [max. Steps] This criteria stops the training after a fixed number of Iterations. [Avg. Deviation] The Avg. Deviation criteria is fulfilled if the average of the errors occurring during a complete iteration is less than an error threshold. The average error is computed by adding the errors for each sample divided by the number of samples. [Max. Deviation] The Max. Deviation criteria compares the error of the worst sample with an error threshold. Samples under the threshold are skipped within the training procedure. Training one sample can increase the error of other samples. Therefore, the training results are automatically performed after every complete iteration. If the training is stopped before an iteration is completed or if you step through the training procedure, the results can be verified by using the [Perform] button of the Neuro Control Window Toolbar. [From] [To] [Factor] Max deviation can be used as a Dynamic criteria. 218

74 Neuro-fuzzy Learn Methods The learn methods determine the content of learning and the usage of the learn parameter. [RealMethod] Real method uses a single selected sample to find the best terms and rules to be changed. The changes to membership functions and fuzzy rules are computed by using the constant StepWidth(Term) to change terms, and the constant StepWidth(DoS) to change rules. [RandomMethod] Like RealMethod only using random steps from the equipartioned interval [0 StepWidth(LV)] to change terms, and random steps from the equipartioned interval [0 StepWidth(DoS)] to change rules. [Batch_Learn] Batch method computes a batch in which all samples are used to find the best terms and rules to be changed. The changes to membership functions and fuzzy rules are computed by using the constant StepWidth(Term) to change terms, and the constant StepWidth(DoS) to change rules. [Batch_Random] Like Batch_Learn method only using random steps from the equipartioned interval [0 StepWidth(LV)] to change terms, and random steps from the equipartioned interval [0 StepWidth(DoS)] to change rules. [User Defined] Learn methods are separated in a dynamic link library. User-defined methods can be added. Please refer to the manual to see how a learning method is structured and how a user-defined method must be compiled to be integrated into fuzzytech. Neuro-fuzzy Selection Mode Training is based on sample data. The training success depends on the sequence in which the samples are selected. The selection mode determines whether samples are used sequentially from a file, if they are used in random sequence, or if a user-defined method is selected. Neuro-fuzzy Save Best Project Neuro-fuzzy Training usually is a non-greedy training method. Thus a derived solution can also get worse when training is continued. Therefore, the Neuro-fuzzy module delivers the option to save the best projects due to the selected criteria. The files automatically saved using default names. 219

75 Appendix J: Training data of neuro-fuzzy network model and defined ranges of input and output variables 220

76 Appendix J1: Defined ranges of input and output variables HVAC_energy Gross_area (m 2 ) IT_overdesign ITfootprint (%) (kwh/m 2 /year) IT_energy (kwh/m 2 /year) Total_energy (kwh/m 2 HVAC/ IT IT/ Total (%) /year) Min Max *although data centre 6 was excluded from the source of training data, the defined ranges of variables are based on the data measured from 6 data centres all. Appendix J2: Training data of neuro-fuzzy network model HVAC_energy DC Gross_area (m 2 ) IT_overdesign ITfootprint (%) (kwh/m 2 /year) IT_energy (kwh/m 2 /year) Total_energy (kwh/m 2 HVAC/ IT IT/ Total (%) /year) # # # # # #