Influence of Availability on the Cost Analysis of Solar Powered Data Centers with AC and DC Architecture and Mirroring Functionality

Transcription

1 Influence of Availability on the Cost Analysis of Solar Powered Data Centers with and Architecture and Mirroring Functionality Domagoj Talapko, Ph.D. Candidate Emerson Network Power Selska 93, Zagreb, Croatia Prof. Sejid Tesnjak, Ph.D. Faculty of Electrical Engineering and Computing Unska 3, Zagreb, Croatia Abstract - This paper addresses availability of power infrastructure in data centers and associated costs for downtime. Special attention is given to modeling of solar power sources in terms of availability as well as their implementation into critical infrastructure and the justification for their implementation from cost perspective. Influence on overall electrical reliability and availability of infrastructure is shown in different topologies. Architecture of the topologies is based on both alternating current () and direct current () 400V distributions. Results of reliability and availability calculations derive from models based on Dynamic Fault Tree Analysis (DFTA). Keywords-Solar, Data Center, Availability, DFTA, Cost topologies, what will enable good result differentiation resolution. Highest levels of availability for I are met not only when the infrastructure is duplicated within the facility, but when facilities are duplicated and placed on distance of at least several hundred kilometers and each facility acts as a back-up for other facility, working in a Mirroring mode. From that perspective, all facilities that are dedicated to serving the same data to same end user form an I eco-system. It is a very good question how will this mode of operation ultimately reflect onto availability of the energy needed for data, especially when considering that one or two of the Is from the eco-system can be solar powered. I. INTRODUCTION Energy efficiency can be seen as an imperative of sustainable development what is also confirmed by European Commission DG INFSO in their study from 2008 [1]. The goal of reducing electrical power consumption by 20% from power plants and therefore reduce carbon footprint by 20% by implementing alternative power sources increasing their share by 20% by the year 2020 will certainly require nonconventional approach to powering of many facilities among which facilities belonging to telecom industry have a fair share. This trend is also present outside of Europe; in USA the Department of Energy has taken the Microgrid Initiative [2] to develop commercial-scale microgrid system by the year 2020 and also in Japan who has taken the lead in technology development and deployment not only in Japan but also worldwide through leading technology companies [3]. Internet Data Center (I) industry axiom calls for high reliability and availability level allocated to electrical infrastructure in I facilities. When using traditional energy sources such as commercial grid in conjunction with diesel generators and battery back-up these levels are always met, but due to the global usage increase of alternative power supplies it is a question how implementation of these sources reflects on overall infrastructure availability. Presented models will consist of models of solar and diesel power generation systems implemented into different electrical II. CRITICAL FILITY MODELS For the purpose of this paper electrical models are proposed to reflect a powering of 200kW nominal data center load composed primarily of servers. Powering of cooling equipment and other mechanical loads is not covered in this paper. Servers as the main loads in I can be considered as constant loads and their power consumption does not very significantly during the day or year. Proposed models and s are based on European, USA and Japanese voltage levels and frequency. A. Electrical Architecture in I Facility Total system availability is not determined simply by the reliability of individual components or sub-systems, but also by system topology and how sub-systems interact together. Typically, mission critical facilities are powered from more than one power source. This can include separate feeds from independent utilities or substations often in conjunction with local diesel generator(s) on site. Distribution of energy is done via discrete wiring or bus-duct systems connected to distribution cabinets, from which the critical load is fed. Depending on proposed system topology, transfer switch gear like automatic transfer switch (ATS) is often used to enable transition between two sources. B. Electrical Equipment in I Facility State of the art UPS systems can be deployed in several different modes of operation. Modular construction not only /15/$ IEEE 365

2 provides for scalability but also by placing a static switch in every module eliminates a single point of failure. In both cases the operation modes can be on-line, line interactive or in eco mode (power supplied to the load via static switch bypass directly from the utility). Line interactive and eco mode operation depends heavily on switching between line and UPS and high reliability of static switch is crucial. Fig. 1. Implementation of solar power in Fig. 2. Implementation of solar power in Modern modular and scalable based power systems used in telecom applications can be easily expanded and designed with redundancy / availability built into the system based on the ease of use of modular approach. In the applied models all systems under study have two strings of batteries sized to supply the full load during period of 15 minutes enabling the succesful start of diesel generator. Batteries used in UPS have different configuration (number of cells) than those used in UPS due to different voltage requirements. The UPS uses a battery string with 40 blocks (240 cells) with 120Ah capacity while the battery string for UPS consist 28 blocks (156 cells) with 170Ah capacity. Ultimately, the number of cells impacts availability of battery. Estimated time between failures for battery system used in system is 12500h [4] and for the battery in system, 19231h. Proposed MTBF for battery strings in systems is derived from reference [4] while the figure for systems is derived from interpolation of data provided in [4] and is valid only during first two years of batteries exploitation. Availability of commercial grid is set to 99,9%. In the European distribution model a transformer-less power distribution units (PDU) from UPS to the load is used for the calculations. In the USA and Japan, PDUs would typically be equipped with a step down transformer what is ultimately decreasing the availability of an distribution model compared to European version. The modular UPS consists of eight (8) 30kVA modules placed in two cabinets (four modules per cabinet) for a total system rating of 240kW. Since modules can operate with cos φ 1, seven modules are sufficient to provide power to the load (210kW) and one module is redundant. This is assuming perfectly balanced load. Typically 3 phase systems are derated by the safety factor of 0.8 to accommodate for potential unbalance. This is not the case for as there is only a single circuit. Each module has additional capacity built-in for recharging of the batteries. All key components (rectifier, inverter and static switch) are present in each module. In the case of failure of any element of the module (controller board included), the whole module is pronounced as faulty. Modular UPS also supports eco mode or in on-line mode operating regimes, achieved on module level. In the case of more than one module failure, the system will run out of the power capacity and the load will be transferred on direct feed from mains by means of static by-pass switch. Power System model incorporates 16 modules connected in parallel, where each module is rated 15kW, enabling 240kW of installed power. Modules are placed in two cabinets so that in each cabinet 8 modules are placed. The system is dimensioned according to telecom application logic (N+1), what is achieved by 14 modules to power the load (210kW) with two additional modules of 15kW for battery recharging and redundancy. Output of the system is achieved via circuit breakers rated for 100% of load. The system controller is not subjected to analysis since its failure will not influence overall performance of the system. When mains is available the System is able to supply the load without any interruptions even if two out of the sixteen modules fail as that will simply mean that there is no more redundancy and no capacity to charge the battery. However, there will still be enough capacity to supply the load. In case if more rectifier modules go down, battery reserve will backup for lack of energy that should come from rectifiers. The battery discharge, considering low power drain would last for hours, so the maintenance time is very long and considering short MTTR the system s capacity can be easily restored without impact on critical bus. This is a fundamental difference between modular and modular battery in is on line while in is coupled through inverter, so technically in modular 2 rectifiers can fail if there are spares on site. 366

3 In order to increase the efficiency level of the solar system, maximum power point tracking (MPPT) modules are used [5] based on /. Each module is 15kW and modeled system consists of 16 modules in placed in two cabinets where 15 modules are used to enhance the efficiency of the power flow from solar panels and one more module is used to achieve N+1 specific for telecom sites. It is considered that all loads (servers) are based on dual cord power supply and that it is possible to set the server to take the energy mainly from one input and use other as a back-up line and in that way server will intake the energy from B branch by default and use the A branch as an auxiliary. C. Models of Power Sources Diesel Generators are most well-known standby power source of energy not only in telecom infrastructure but also in nuclear facilities, hospitals, airports and may other types of critical infrastructures. Diesel generators are capable to take the full load in about 10 seconds after they go online and can operate typically hours without major failures [6]. In size, usually they can be up to 7MW. They are easy to parallel and in that way form more flexible, scalable and reliable power supply systems. TABLE I INSTALLED POWER WITH SOLAR PANELS France Japan USA 1650 kw 1260 kw 1125 kw 1620 kw 1230 kw 1080 kw According to [7] two out of top ten data centers outages worldwide in year 2012 are associated to diesel generator failure. Since the diesel generators operate mainly as back-up systems, their failure to start can be potentially huge problem. For that reason, regular maintenance schedules must be followed. One of the weakest elements in the whole diesel system is actually an internal starter battery; therefore regular maintenance checks must also include that element. In this paper, model is based on single diesel system of 250kWe. Reliability model incorporate failure to start of diesel generator since it is modeled in a cold standby mode and not under maintenance in moment when it needs to operate. Unlike power sources that are constantly available and ready for operation, solar power generation systems significantly depend on availability of the prime energy sources the Sun s irradiance. One more element is crucial in this kind of operations to enable higher availability and that is battery back-up that will accumulate all excessive energy produced from solar power systems and be utilized during hours with low or no availability of the Sun. In proposed models, three locations worldwide are selected for implementation of alternative energy supply; dimensioning of data center with solar system is done according to parameters specific to Marseille France, Kofu Japan and Phoenix USA. All locations are selected intentionally since they have very constant amounts solar irradiance. Solar system is treated (modeled) as an add-on (side B) to the existing power source from commercial mains (side A). Solar system consists of 12V solar cells with maximum (peek) power 250W. Cells are assembled in panels with 2x3 configuration enabling voltage level of 72V and 10 of these panels are forming one solar system operating at 720V nominal. Total installed number of solar panels is determined by worst case scenario the month in year with least Sun irradiance available. Sufficient amount of power also needs to be reserved not only for the powering of the load but also for battery recharge. As indicated in Table I, it can be observed that number of panels depends on location of implementation and also on the ( and ) implying different efficiency levels. Battery dimensioning is based on 3000Ah 2V cells as main building blocks. As a first step a battery sizing optimization was performed in a way that depth of discharge (DoD) is compared to the number of cycles available for the individual battery with the goal of minimizing the total cost of ownership allocated to the battery. It is assumed that every day one cycle of charging and discharging will occur. Three scenarios are developed for the period of 20 years: two, three or four battery sets to be used. The case with three battery sets for the period of 20 years is indicated as optimal in terms of minimizing the TCO resulting with DoD in area of approximately 40%. For the calculation an inflation rate of 1% was considered per every year and lead price variations are not considered. According to reliability testing of solar panels of different manufacturers presented in [8], failure rates that derive from accelerated testing protocol indicate that failure rates of various solar panels are rather high: 31% in damp heat test, 14% humidity freeze test and 12-13% during static load, termination and 200 thermal cycling tests. However, these results need to be taken with caution when building a model, since these testing methods are developed for purpose of product qualification on the market. According to [9]-[11], in real life solar panels failures should be seen in terms of degradation of the individual cell depending on climate conditions such as rise of temperature and moisture. For the purpose of modeling, MTBF value of 600 years with lifetime of 30 years for individual 12V solar cell was considered. In all models potential impacts on reliability of equipment associated with power quality issues, such as harmonics, are omitted. III. AVAILABILITY AND COST CALCULATIONS A. Reliability and Availability Calculations Reliability of the system is defined as probability that system operates within normal parameters under certain 367

4 conditions during a given period of time and should be used as a measurement of system complexity, as per Equation (1). Availability is a statistical probability that the system will perform in random time t in the future [12], as per Equations (2) and (3). TABLE II SYSTEM AVAILABILITY WITH CORRESPONDING DOWNTIME Number of Nines Downtime per Year 99,0 90h 99,9 9h 99,99 0,9h 99,999 5min 99,9999 0,5min 99, sec Fig. 3. FTA of side A Fig. 5. FTA of France side B with solar system Fig. 4. FTA of side A In all of the calculations an assumption is taken that all components have exponential probability density of failure. R(t) = exp(-t/mtbf) (1) A = Uptime / (Uptime + Downtime) (2) U = Downtime / (Uptime + Downtime) (3) Downtime incorporates mean times when component is under corrective maintenance or under preventive maintenance that requires its shutdown. Fig. 6. FTA of France side B with solar system 368

5 Results derive from Dynamic Fault Tree Analysis method that is based on intuitive and graphical methods of failure analysis. DFTA is a top-down deductive approach to failure analysis that starts by definition of undesired event and proclaiming it to be the Top event [13]. Connections of top and lower events are created by the gates based on Boolean algebra. DFTA calculations in this paper are based on exact calculation method. The models discussed in this paper do not incorporate influences that are subjected to dynamic events, such as recovery actions during the certain failures of redundant system elements. Actions of preventive maintenance are also not considered. In the models with solar power, assumption is made that all accumulated energy with solar panels can be presented as is solar panels are operating at 100% of their capacity during 20% of time during one day in France, 22% in Japan and 24% in USA, respectively. In addition to already mentioned references and reliability indicators mentioned, some of the MTBF and MTTR figures (diesel generator, circuit breakers, etc.) used in this paper derive from [14]. B. Cost Calculations Costing models are achieved by benchmarking of two separate sub models. First sub model is based on existence of only A side (powering through commercial grid) where the main cost contributor is the energy purchase price according to [15] for the I. Second sub model is assumes that the electrical energy production facility based on solar systems is added as a side B to the I infrastructure with goal of improving total availability and also reducing costs related to purchasing of electrical energy. Costs for the second sub model (Side B) comprise of costs for solar systems implemented in commercial facilities according to [16], optimized costs for batteries, maintenance of 2% annually and also for the energy that still needs to be bought from commercial grid (side A) when the solar facility (side B) is unavailable. These costs are reduced for the price that can be achieved when solar facility is selling excessive energy back to the commercial grid. According to [17] costs for solar modules have reduced by more than half during last five years with strong tendency of decreasing. Fig. 7. Reliability results of every side in both s TABLE III RELATIVE AVAILABILITY FOR EH BRANCH Side A Side B France Side B Japan Side B USA 0, , , , , , , , TABLE IV AVAILABILITY ON FILITY LEVEL Site France Site Japan Site USA 0, , , , , , TABLE V AVAILABILITY IN MIRRORING MODE OF ANY TWO SITES Number of nines IV. RESULTS All results in this study derive from reliability assessment software Windchill Quality Solution Results of reliability assessment are presented in Figure 7, availability in figures 8 and 9 and tables III and IV. Electrical infrastructure availability results of mirroring of any of the two Is are presented in table V. In Figures 10 and 11 cost comparisons are presented. Fig. 8. Availability results of every side in both s 369

6 energy where as in USA that is not the case due to rather high investment costs and very low costs of commercial energy compared to France and Japan. Availability results of two mirroring sites indicate extremely high level of availability allocated to electrical infrastructure of the I eco system, however in further work more detailed analysis for the complete infrastructure needs to be performed. Fig. 9. Overall facility availability when both A and B side are present Fig. 10. Comparison of costs related to the I s Solar production plant and sell out of the excessive energy from Solar facility in USD Fig. 11. Comparison of TCO in USD of the I s Side B solar production facility and total amount of energy that would be bought through Side A if the Side B would not be present V. CONCLUSION Results of availability assessment indicate that solar photovoltaic system in conjunction with batteries yields slightly lower availability level compared to the conventional powering from commercial grid backed-up with diesel generator. Also, based topologies enable higher availability compared to the traditional based topologies. Another significant observation is that based topologies require smaller number of solar panels due to the efficiency savings made by inverter exclusion. As in regards to the total cost of ownership, simulated sites in France and Japan justify the implementation of solar REFERENCES [1] A. Beton, C. Abbayes, S. Iyama, L. Stobbe, S. Gallehr, L. Günter Scheidt, Impacts of Information and Communication Technologies on Energy Efficiency, Final report Executive summary, European Commission DG INFSO, September 2008 [2] M. Smith, D. Ton, Key Connections IEEE, Power & Energy, Volume 11, Number 4, July/August 2013 [3] A.P. Ai Ling, S. Kokichi, M. Masao, The Japanese Smart Grid Initiatives, Investments, and Collaborations, International Journal of Advanced Computer Science and Applications, Vol.3, No.7, 2012 [4] UPSonNet NewsLetter, Impact of Backup Batteries on UPS Reliability, Newsletter April 2010, [5] Shaanxi XinTong Intelligent Technology: MPPT Control System, XinTong Solar 2011, Shaanxi XinTong Intelligent Technology Co.,Ltd, China, [6] T. Loehlein, Maintenance is one key to diesel generator set reliability, published on October 4, 2013 in Latest EPSS News, [7] R. Miller, The Year in Downtime: Top 10 Outages of 2012, Data Center Knowledge, web published, (2012) [8] G. TamizhMani, Testing the reliability and safety of photovoltaic modules: failure rates and temperature effects, TÜV Rheinland PTL & Arizona State University, Tempe, Arizona, USA, tuv. com/media/usa/aboutus_1/pressreleases/ptl_magazine_article. pdf (consultado 18 de Agosto de 2011) [9] A. Realini, E. Bura, N. Cereghetti, D. Chianese, S. Rezzonico, Study of 20-year old PV plant (MTBF Project), 17th EPVSEC, Munich, Germany, [10] M. Vasquez, I. Rey-Stolle, Photovoltaic module reliability model based on field degradation studies, Progress in Photovoltaics: Research and Applications, Volume 16, Issue 5, pages , John Wiley & Sons, Ltd., August 2008 [11] A. Realini, MTBF PVm Mean Time Before Failure of Photovoltaic Modules, Federal Office for Education and Science BBW, Final report BBW , June 2003 [12] Department of Defence USA, Reliability, Availability, Maintainability, and Cost Rationale Report Manual, Office of the Secretary of Defence, Washington,, 2009 [13] E.O. Schweitzer III, B. Fleming, T.J. Lee, P.M. Anderson. Reliability analysis of transmission protection using fault tree methods, Proceedings of the 24th Annual Western Protective Relay Conference, pp. 1-17, 1997 [14] Headquarters Department of the Army, Survey of Reliability and Availability Information for Power Distribution, Power Generation, Heating, Ventilating and Air Conditioning (HV) Components for Commercial, Industrial and Utility Installations, Technical Manual, TM 5-69sol8-5, Washington,, 2006 [15] Shrinkthatfootprint, Average electricity prices around the world: $/kwh, July 2 nd 2015 [16] International Energy Agency, Technology Roadmap, Solar Photovoltaic Energy, IEA, 2014 Edition, France, 2014 [17] International Renewable Energy Agency, Solar Photovoltaics, Renewable Energy Technologies: Cost Analysis Series, Volume 1: Power Sector, Issue 4/5, IRENA, UAE,