OPTIMIZING POWER SYSTEM PROTECTION FOR MODERN DATA CENTERS

Transcription

1 OPTIMIZING POWER SYSTEM PROTECTION FOR MODERN DATA CENTERS Data center power systems are constructed with a premium on reliability at a significant cost. The return on this investment is realized if it provides many years of trouble free operation. Power system protection is designed to assure two seemingly contradictory purposes: 1. Maintain continuous (uninterrupted) flow of power 2. Act rapidly to isolate power system faults The most prevalent and at times insidious type of fault is the ground fault. This article discusses the fundamental issues pertaining to ground fault application on (Uninterruptible Power Supply) equipment. Examples are provided to illustrate the problem. A solution methodology is proposed. Data center power systems can be solidly grounded or high resistance grounded. Each method has its merits and disadvantages. The focus of this article is on solid grounding. Where solid grounding is employed, a low impedance return path is provided for ground currents. National Electrical Code (NEC) article requires ground fault protection on feeders 1000 amps or more on solidly grounded 4-wire system between 150 and 600 volts to ground. NEC requires the same on service entrance equipment. Figure 1 depicts a modern microprocessor based ground fault protection system. This figure illustrates a radial system - power can flow only in one (radial) direction. Under normal conditions, the currents in the three phases add up to zero. If a ground fault exists, the sum of the three phase currents will be equal to the ground current (or the zero sequence current). The sensor measures this current by summing the individual phase currents. The microprocessor will trip the circuit breaker when the zero sequence current exceeds its ground fault setting. The power distribution in a data center, however, is typically not radial. Multiple sources of power are tied together to increase system redundancy. This is very apparent in (Uninterruptible Power Supply) output switchgear equipment. Figure 2 depicts a parallel redundant architecture. Multiple modules are paralleled to increase both capacity and redundancy. A static bypass switch provides a path for power to flow around the modules when the modules can not handle the load. A manual bypass system is provided to bypass the entire system for maintenance. The modules are separately derived sources and are grounded per the NEC. The other power sources the utility source and the generator- are also grounded per the NEC. Figure 2 shows the utility and the generator with separate ground connections. This is acceptable as long as the neutral conductors are not extended (3 wire system). Where 4 wire systems are used, the transfer mechanism must be four pole or the generator must share the same grounding electrode conductor as the utility main. For short periods of time during power transfer (about 8 to 10 cycles) the static switch will run in parallel with the modules. This condition creates a path for zero sequence circulating currents to flow. Figure 3 illustrates this condition. One module is removed to simplify the diagram. A question may arise as to why these circulating currents exist. The modules and the utility source (or the generator source) are voltage sources. In a parallel condition, if two voltage sources are not exactly identical, a net voltage difference will exist. This net voltage could exist due to harmonics (voltage distortion) and the different characteristics of the sources. Additionally, the modules are controlled to lead the utility source by a few electrical degrees. If a low impedance closed path is provided, a circulating current will flow. Solid grounding provides a very low impedance closed path for all zero sequence currents. IEEE standard 142 (The Green Book) provides a discussion of circulating currents in parallel sources.

2 The ground fault relays on circuit breakers (Figure 3) recognize these circulating currents as ground fault currents. If the magnitude of the circulating current is higher than the setting of the ground fault protection relay and the time delay on the ground fault trip is set very low, the circuit breaker will trip, sacrificing the critical load. Simplistic Designs Increase the Damage One simplistic solution is to eliminate the ground fault protection on the output switchgear equipment. This practice will actually reduce the reliability of the system. Consider a ground fault on the System output bus (Figure 4). Under normal conditions the static switch is open and the power is supplied through the module(s). Ground faults often start as low magnitude arcing faults and sometimes they progress slowly. Because ground fault protection is eliminated, this fault will continue to burn until the magnitude of the fault is higher than the overload setting (typically 300% of full load amperes) of the system. The ideal condition would be to detect this fault quickly and isolate it before it causes extensive damage. This has an effect on the equipment Mean Time To Repair (MTTR). Quick identification and isolation of faults will reduce damage. This translates to shorter repair time and the data center can be put back in service a lot faster. Comment [MM1]: Say: This has a effect on Power System Protection, An Introduction to Differential Systems The ideal protection system is the differential system. Zones of protection are the essential elements of differential systems. Figure 5 shows one zone of protection. Any current that enters a differential zone, must also leave that zone through one of the protected circuit breakers. Otherwise all the circuit breakers on the boundary of the zone will trip. Therefore, a fault inside the zone of protection will cause tripping of the circuit breakers that are on the boundary of that zone. But a circulating current will not set off a differential scheme. If a differential system were employed, the fault on the System output bus would be seen only by circuit breakers MOD1, MOD2 and MODS. These circuit breakers would open without much delay and isolate the fault. Defining the zones of protection is the task of the consulting engineer. The degree of protection specified is a function of the reliability expectations of the system. Figure 6 illustrates one method of defining the zones of protection. Comment [MM2]: For those of us who are not intimately familiar with differential zoning, an example drawing would help. To further illustrate the differential protection principle, consider a ground fault in zone 2 (Figure 6). Only circuit breakers BP1 and BP2 see this fault. Circuit breakers MAIN and will not see this fault. The system is selective irrespective of the ground fault settings on the circuit breakers. So it is not necessary to increase the setting on the main utility circuit breaker to obtain coordination between circuit breakers MAIN, BP1 and BP2. This system is far more advanced than zone selective interlocking. Zone selective interlocking does not resolve the circulating current nuisance tripping problem and does not provide zones of protection. It is not Necessary to Increase the Equipment Cost Differential relays are expensive and costly to implement. Therefore, the ideal system will not include differential relays. A properly designed low cost differential ground fault system can be developed with the following features: Comment [MM3]: Say: Therefore, the ideal differential ground fault system would have the following features: 2

3 1. The system would utilize the ground fault protection circuitry in the circuit breakers to create the differential scheme. Costly addition of differential relays is not desirable or necessary Each bus would be protected independently. A fault in one zone of protection would only trip the circuit breakers that are connected to that bus. 3. The system would be impervious to any and all circulating currents that can cause nuisance tripping of the circuit breakers 4. The system would operate correctly under any arrangement of the circuit breakers. With any and all interlocks (mechanical and electrical) removed, the system would sense ground faults correctly and trip only the correct circuit breakers. The above four rules can be incorporated into almost any project specifications. With such systems, it is no longer necessary to artificially set the ground fault relays to high levels in order to avoid nuisance tripping. Ground fault settings can be set low enough to provide maximum protection and minimize the damage at the point of fault. Comment [MM4]: The system would utilitze Comment [MM5]: desirable or necessary. Comment [MM6]: would be Comment [MM7]: would Comment [MM8]: are connected to Comment [MM9]: would be Comment [MM10]: would Comment [MM11]: would Comment [MM12]: almost any Challenge the Manufacturer Competent protection engineers can design the system referenced above. Some manufacturers may employ differential ground fault schemes in their products, but many do not. The system design engineer should be aware of the circulating current phenomenon and challenge the equipment manufacturer to develop appropriate differential schemes that accommodate it.. It is not necessary to degrade system reliability to satisfy the NEC and provide ground fault protection. Authors: Mike Mosman, P.E. Reza Tajali, P.E. Comment [MM13]: may employ differential schemes in their products, but many do not. Comment [MM14]: Say: The system Deigning Engineer should be aware of the circulating current phenomenon and challenge the Equipment Manufacturers to develop appropriate differential schemes that accommodate it. Michael Mosman is Vice President and Director of Electrical Engineering for CCG Facilities Integration Incorporated. CCG is an Engineering/Architectural firm located in Baltimore, Maryland, dedicated to the programming, design and commissioning of high-technology and mission-critical facilities. Mr. Mosman is a graduate of Washington State University in Electrical Power Engineering, a registered Professional Engineer in twenty states and a licensed master electrician. CCG Facilities Integration, Inc 1500 S. Edgewood St. Baltimore, MD [email protected] Reza Tajali, a registered electrical engineer in California and Tennessee, is a staff engineer for Square D s Power Systems Engineering group in Nashville, Tennessee. He has more than 20 years experience with electrical power distribution and control and holds two United States patents on switchgear products. He is responsible for performing power quality audits on data centers and other high reliability facilities. That work includes measurement, analysis and simulation of power systems. 1 In the proposed system, ground fault sensors will have to be wired in a special configuration. A full testing and commissioning program must be included as part of the specifications. 3

4 Square D Power Systems Engineering 1010 Airpark Center Drive Nashville, TN [email protected] Figure 1 Microprocessor Based Ground Fault Protection System Microprocessor Trip Circuit Electronic Ground Fault Sensing Unit 4

5 Figure 2 The Parallel Redundant Architecture G MAIN BP1 MOD11 MOD2 MODS BP2 To Load Bank 5

6 Figure 3: Circulating Currents and Nuisance Tripping 6

7 Figure 4: Perpetuating Bad Ideas Removing the Ground Fault Protection from the Output Switchgear Ground Fault 7

8 Figure 5: Differential Protection for the Modules Bus G MAIN BP1 MOD1 MOD2 MODS BP2 To Load Bank Zone of Protection 8

9 Figure 6: Defining the Zones of Protection G MAIN BP1 MOD1 MOD2 MODS Zone 2 BP2 To Load Bank 9