1 Real Time Test Bed Development for Power System Operation, Control and Cyber Security Ram Mohan Reddi, Student Member, IEEE, and Anurag K Srivastava, Senior Member, IEEE Abstract--With ongoing smart grid initiative, there is a considerable need for developing new algorithmic solutions and validating at laboratory level before they can be successfully applied in the power grid. This research work addresses the development of a real time test bed by integrating several hardware s including the Allen Bradley Programmable Logic Controllers (PLC), National Instruments PXI (NI-PXI) controller, Real Time Digital Simulator (RTDS), and Schweitzer Engineering Lab (SEL) devices. This work also integrates the OSIsoft PI Server system and the establishment of data channel over Ethernet/IP for communication and control. The developed test bed is evaluated by simulating a simple test case in RTDS and executing the control logic in PLC. The test bed provides a user friendly Human Machine Interface (HMI) for monitoring and control at different levels of the system along with capabilities for storing the power system data which includes Synchrophasor data for forensic analysis. This test bed will be essentially used for modeling and study of different power system operation control algorithms as well as to investigate cyber vulnerability and mitigation. Index Terms LabVIEW, NI-PXI, PI Server, PLC, Real Time Test Bed, RTDS, HMI I. INTRODUCTION S the electric power system is moving towards the Smart AGrid (SG) development for improved reliable, secure, and economic operation, implementation of such a system requires enhanced testing and validation [1], [2]. Most of the control action schemes mainly rely on extensive offline studies using hypothetical scenarios and models that possibly include errors [3]. Current developments in control schemes are also more often theoretical and non-real time model based scenarios which are rarely evaluated. There is a need for testing and validating these Real Time Monitoring and Control techniques involving different hardware equipments to achieve flexibility, ease of operation, interoperability, control validation, and more importantly redundancy of the control schemes [4]. Research works in the field of real time control and data acquisition for power systems have been reported in [5], [6] and [7], but either it depends on software based analysis or limited hardware tests. Requirement for modern power system automation has been reported and Ram Mohan Reddi is with Department of Electrical and Computer Engineering, Mississippi State University, Mississippi State, MS 39762. (rr370@msstate.edu, 662-312-0232). Anurag K. Srivastava is with Department of Electrical and Computer Engineering, Mississippi State University, Mississippi State, MS 39762. (srivastava@ece.msstate.edu, 662-325-5838). existing from long time [8]. The test bed developed here is one-of-a-kind platform including simulated power grid interfacing with sensor network and control center type setup with data storage and different level of control. Developed test bed provides a good working platform to test and validate different protection and control schemes on several hardware equipments varying from simple relays to PLC applications and complex algorithms in the power system. The interoperability of different hardware devices for power system automation can be also tested. This research work mainly focuses on key aspects of real time modeling, simulation, remote monitoring, cyber security, and control of power system operation by employing different hardware features and software algorithms. The Developed power system test bed utilizes the RTDS, PLC, NI-PXI controller, and SEL devices. Corresponding software suites are employed to model, monitor, and control the system in real time. II. OVERVIEW OF HARDWARE AND SOFTWARE EMPLOYED This section discusses some basic details about the hardware and software employed in the development of the test bed for real time studies. A. Real Time Digital Simulator Real time digital simulator (RTDS) is a power system simulator which simulates power system programs in real time. This is unique in the sense that it works on the parallel processing technique of digital signal processors and executes the program developed on its processors and produces output both graphically and through the output interface cards incorporated into the system. The power system programs are developed using RSCAD user interface which is specially designed for RTDS and is used for both development of the different power system scenarios and also for viewing and studying the results graphically [9]. The RTDS present in the research lab at Mississippi state university consists of two racks with eight triple processor cards and two Giga processor cards, and additionally it contains several input and output interface cards for sending and receiving analog and digital signals. One of the important interface cards used in this work is Digital to Analog Converter Card (DDAC) from which 12 signals can be sent out of the RTDS, and also the front panel inputs are used to send the digital control signals into the RTDS to control the elements in the simulated power system. Figure 1 shows the Real Time Digital Simulator at MSU.
2 time will vary based on the complexity and the length of the Ladder Logic program. The PLC (Fig 2.) present in the research lab at MSU is supplied by Rockwell Automation, and it belongs to Compact Logix family of systems and consists of an L35E processor with 128 I/O [10]. Fig. 1. Real Time Digital Simulator (RTDS) B. Programmable Logic Controller A Programmable Logic Controller (PLC) is a standalone system which is capable of continuously executing logic and making decisions though the program written into it using the real world analog and digital signals as inputs. These devices are remote terminal units employed in many industries and in a power system infrastructure for controlling the elements of the corresponding systems. In this work, an Allen Bradley manufactured Control Logix PLC is used in the test bed for controlling a simulated breaker in the power system running on the RTDS. C. NI-PXI System National Instruments PCI extension Interface system (NI- PXI) is a real time embedded controller from National Instruments used for real time testing purposes. Built on PXI architecture which is an open PC based platform for test measurement and control [11], this system is a low cost high performance model used in various technologies. The NI-PXI system at the MSU research lab (Fig 3.) is a standard 8 slot chassis consisting of a 1084Q Embedded Controller and two I/O cards 6608 and 6251 responsible for sending and receiving analog and digital signals in and out of the controller. The controller is a stand-alone system running a program written in LabVIEW software. Fig. 3. NI-PXI 8 slot system at MSU Fig. 2. Programmable Logic Controller (PLC) This PLC is capable of Ethernet communication, and an identical PLC is also used in the test bed for testing the remote control center operation and cyber security analysis. The control logic is written by the RSLogix 5000 programming software suite provided by Rockwell Automation for the operation of the Compact Logix PLC and is a very flexible and powerful tool for writing the programs into the PLC. The current logic is developed in Ladder Logic (LL) using RSLogix and runs continuously in the PLC. The PLC executes the program through a procedure called scan cycle, where ladder logic is scanned step by step by reading the inputs variables and making the corresponding changes in the output elements. This is repeated continuously and the simulation D. LabVIEW LabVIEW is a graphical programming tool for test, measurement, and automation and is widely used as a virtual instrumentation tool. It allows user to develop sophisticated measurement, test, and control systems using intuitive graphical icons and wires that resemble a flowchart [11]. This tool is increasingly being used in development of laboratory test applications as well as industry standard working applications which are user friendly, and the test output can be checked instantly. One of the major advantages of LabVIEW, apart from being simple to use, is the ability to work with a number of hardware interfaces using real world analog and digital signals. LabVIEW programs work as simulation or data acquisition applications depending on the requirement using custom built hardware from National Instruments. This consists of two windows, a block diagram window where the actual graphical code is written and the front panel where the output can be visualized. In the current research, LabVIEW is used to develop a program which will acquire data from RTDS and runs custom built programs as well as to send and receive signals from the PLC. The LabVIEW program developed runs in real time on the NI-PXI embedded controller through Ethernet communication and displays the results in the front panel of the LabVIEW program.
3 E. PI System The PI system or Plant Information system, delivered by OSIsoft is one of the highly scalable and secure infrastructure for the management of real time data and events [12]. It includes several software interfaces for real time management and data base creation and investigative studies. The PI system is widely used among the industries, including power systems for real time data management and visualization. The PI system installed at MSU consists of a PI sever system and a PI process book along with OPC and C37.118 interfaces. The PI server system is a software tool which acquires, analyzes, stores, and routes data in real time and is the core of the PI system [12]. The PI server acquires the data from the developed test bed and stores the data over a custom time period in real time along with time stamping at flexible scanning cycles. The acquired data is visualized in the PI process book interface where the data is displayed as plots relative to time and also is capable of alerting the operator through alarms which act on the real time data. The data acquisition is done through the OPC, and C37.118 interfaces provided by OSIsoft and are capable of acquiring data from third party devices. The overall system described above is managed by the PI system manager in secure environment, as only a selected group will have rights to view or modify the real time data. F. Ethernet/IP communication Ethernet/IP is one of the commonly used communication protocol, due to its simple operation and the ability to incorporate number of devices by providing a fast communication medium. Ethernet communication has the advantages of higher transfer speeds, full duplex and collision free operation, fiber optic interface, remote monitoring and diagnostic support which make it viable for power system operation and control [13]. A 100 Mbps Ethernet network is used in this work with devices connected through a switch on private IP configuration. The only exception is a second PLC employed in the remote control center which communicates through wireless Ethernet over radio using antennas. This is specifically employed for testing the security standards of the developed system. III. TEST BED ARCHITECTURE AND OPERATION The integration of various hardware devices is done through Ethernet or hard wired connection. A simple power system test case is developed in RSCAD and is executed on the RTDS; the scaled signals from the simulated system are sent to the PLC which is integrated with RTDS using the NI- PXI system. The NI-PXI system is used here as there is no direct connection procedure devised to connect Allen Bradley compact logix PLC with RTDS, and moreover the NI system provides an intermediate level for monitoring the data. The block diagram of the developed test bed architecture is shown in fig 4. Fig. 4. Test Bed Communication Architecture As seen from the figure, a remote control center is installed with compact logix PLC and HMI for remotely monitoring the data and to sending the control commands over the wireless Ethernet. RTDS signals are hardwired to the SEL devices such as SEL-421 relay for monitoring the system and fault protection. In similar manner, it is connected to the NI- PXI through the DDAC cards on the RTDS and the DAQ cards on the PXI system. The control logic is written using, the RSLogix 5000 tool for both the local and remote PLC s. The NI-PXI uses Ethernet/IP suite available in LabVIEW to write and read data from the PLC data tags by directing the system to PLC IP address. The HMI shown in fig. 4 are the corresponding software suites needed for the hardware operation. One more important part of the test bed is the PI server system which is connected to the same Ethernet switch and acquires the system data in real time. The data acquisition is done by the OPC interface where the RTDS data is accessed from the LabVIEW using OPC protocol, and the PMU (fig. 4) data can be accessed by C37.118 interface installed on one of the systems. The operation of the test bed is as follows: firstly, the power system developed using RSCAD is simulated in RTDS and the required signals are sent out using the DDAC interface to the NI-PXI system which is running a real time LabVIEW program, which acquires the signals from RTDS. The program displays the data graphically in the front panel and also writes the data to local PLC tags. The PLC has a control logic which tests for the component outage caused by faults and generates digital control signals for possible remedial scheme, which are read back by the LabVIEW program and sent to the RTDS.
4 Fig. 5. RSCAD power system model The SEL-421 device is also hardwired to the RTDS which continuously monitors the power system data for faults. The local PLC is connected to the PLC in the remote control center which is also capable of running the control logic and monitoring operations. Since all the communication architecture is on Ethernet, the PI system is also incorporated into the system which continuously acquires the power system data from LabVIEW using OPC interface and acts as a data repository and monitoring client. The real time data can be collected over any period ranging from minutes to days or more, only limited by the amount of space allocated for the data storage. Thus, all the devices operate in real time making it possible to run several tests on a smaller power system network at laboratory level. provision to manual induce a fault into the system causing opening of one, or all breaker operations by relay leading to possible unbalanced operation. B. Test case scenario The test case scenario comprises of inducing a fault manually leading to generator outage. Corresponding corrective action has to be taken by the monitoring and control devices such that they shed one of the loads to balance the system. IV. TEST CASE AND NORMAL OPERATION This section describes the small basic power system test case employed in the research work and its results. Note that there are several other applications are possible, and has been completed but the simple one is presented here to have focus on development of test bed instead of the application. A. RSCAD power system model The sample power system developed in RSCAD is a 3 phase four bus system with two 250 MW generators supplying the power to two dynamic loads balancing the system. The rms values of the currents are taken into account as the monitoring parameters which are monitored at the generator bus. The program also includes a DDAC block for sending the analog signals from RTDS (fig. 5) for the complete power system model monitoring. The model also includes a digital input block to receive signals into the RTDS to operate on the breakers or relays in the model. The run time window of the program displays the monitoring currents and corresponding changes along with the load breaker status. It also has a Fig. 6. RSCAD front panel for no fault condition of the power system The system should also monitor the failed generator status since it has to bring the load back into the system if the fault with generator is rectified. All the above operation should be done in real time, and the PI system, LabVIEW, and the PLC should be monitoring the system in order for the system to operate successfully. Figure 6 shows the normal condition of
5 the sample power system being monitored through the RSCAD front panel. This includes the three phase currents and voltages of the two generators and the breaker status indicators for the generators and the manual breaker controls. As shown in figure 6, all breakers are closed and voltage/ currents are balanced. When the system is operating normally the current drawn by the loads are 1.05 ka each, and the rms value being monitored at 0.761 ka. V. SIMULATION RESULTS The first possible option to observe the system signals is the RSCAD monitoring interface, to identify the fault condition, as shown in fig. 7. It can be seen that one of the phases of the generator, the phase 3 of generator one, has a fault which is manually operated (can also be operated by relay) and the breaker status is indicating the fault. It can be seen from the other plots in fig. 7, that the system is unbalanced as the current imbalances can be observed with the generator one phase currents being dropped to 0.659 ka rms with phase B being zero. At generator two, the corresponding phase B current increased to 2.12 ka rms with others being increased to 1.4 ka which clearly indicate the unbalanced state of the system. Fig. 8. LabVIEW front panel with indicators System control will keep on sensing the status change and should immediately bring the load back when generator comes back in service for normal operation of the power system. System control will work regardless of the number of times the fault occurs and is cleared; the system senses the change in real time and will take the necessary control actions. Figure 9 shows the status of the system when it is recovered from the fault. This can be concluded from the fact that the monitoring parameter values observed in here coincide with previous value of 0.7614 ka rms phase currents for normal operation of the system. Fig 7. Simulated system with fault on phase C generator 1 The LabVIEW program also recognizes this fault and breaker opening and sends the same signal to the local PLC. It also indicates the generator status using the LED indicators, thus alerting the operator of failure. This also acts as an additional medium to monitor and record the data, and the graphs provided will show the real time variation of the monitoring signals. This program will send the data to the OPC server from where the PI system will pick up the data for recording and storage. An immediate action is taken here such that load one shed from the system so as to balance the system with only one generator in operation. This load shedding is indicated by the LED load status indicators present in RSCAD as well LabVIEW front panel. Figure 8 shows status of the LabVIEW front panel which is monitoring the real time rms three phase currents which are of the same magnitude as observed in the RSCAD interface and any change in the system operation is reflected here, and when the fault occurs, also can be seen is the change in indicator color. Fig. 9. Recovered system from the fault Fig. 10. PI process book interface The PI system also simultaneously monitors the changes taking place in the system, and the PI server tracks these changes in real time and stores the data and is capable of sending it to the third party clients for further calculations, if
6 needed. The PI process book is very useful in monitoring and displaying the data, and also it is capable of indicating faults with alarms, and the data plots are drawn over time by which the system changes can be tracked over for longer periods. The PI process book interface is shown in figure 10. The interface developed in fig. 10 consists of six different plots updating in real time over Ethernet the status of the generator currents over time. Real time data update the plots as well as store the three phase rms current data and interface also consist of visual indicators for alerting the operator of any faults. Here, the plots are monitoring the data for 3 hrs but are modified to show data plots for 60 min to observe the changes clearly. The results explained above indicate the successful integration of different hardware at laboratory level for testing and validation of a power system network. The current activities of this research also focus on PMU and PDC testing by including them in the test bed developed for further research activities. VI. SUMMARY The objective of this work is to develop a real time automated power system network and control test bed at the laboratory level to enhance the power system testing and validation schemes. This is achieved by integrating different hardware devices and developing communication interface between those devices. Test case was developed and validated for fault detection, breaker control, outage sensing, and automated corrective action. A simple power system test case is developed in RSCAD and simulated using RTDS to imitate virtual power system. The NI-PXI system and the PLC work on the monitoring and controlling part of the system. The remote control center also provides added advantages and opportunity for identifying cyber security vulnerabilities. The PI system brings the added advantage of data repository, monitoring and event analysis features for the overall system. Simulating a larger power system and incorporating the additional devices like Phasor Measurement Units (PMU) and the Phasor data to the PI system are some of the future goals of this research work. VII. ACKNOWLEDGMENT The authors gratefully acknowledge the continuous support of the National Instruments, OSI Soft, RTDS Technologies Inc, SEL Inc, and Control Systems Inc for their valuable suggestions on the device installation and operation. We would also like to thank the Department of Energy and Pacific Gas and Electric Company for their financial support. A special thanks to MSU Computer Science Engineering department for providing us with valuable hardware and additional expertise. [2] U.S Department of Energy, Grid 2030, A National Vision for Electricity s Second 100 years, Office of Electric Transmission and Distribution, July 2003. [3] Jay Giri, E. David Sun, and Rene Avila-Rosales, "Wanted: A More Intelligent Grid," IEEE Power & Energy Magazine, vol. 7, no. 2, March/April 2009. [4] A. Monti and F. Ponci, Power Grids of the Future: Why Smart Means Complex, presented at Complexity in Engineering (COMPENG), 2010. [5] K.S. Swarup and P. Uma Mahesh, Computerized data acquisition for power system automation, presented at IEEE Power India Conference, 2006. [6] S. P. Carullo and C. O. Nwankpa, Interconnected power systems laboratory: a computer automated instructional facility for power system experiments, IEEE Transactions on Power Systems, vol. 17, no. 2, pp. 215-222, 2002. [7] A. Golder, K. Miu, C.O Nwankpa, S. Carullo, Remote hardware power system loading studies over the World Wide Web, presented at the 37th Annual North American Power Symposium, 2005. [8] T. E. Dy Liacco, Real-time computer control of power systems, Proceedings of the IEEE, vol. 62, no. 7, pp. 884-891, 1974. [9] P.G. McLaren, R. Kuffel, R. Wierckx, J. Giesbrecht, "A Real Time Digital Simulator for Testing Relays," IEEE Trans. Power Delivery, vol. 7, no.1, January 1992. [10] http://www.rockwellautomation.com/ [11] http://www.ni.com/ [12] http://www.osisoft.com/ [13] V. Skendzic and A. Guzma, Enhancing Power System Automation Through the Use of Real-Time Ethernet, presented at Power Systems Conference: Advanced Metering, Protection, Control, Communication, and Distributed Resources, 2006. [14] http://www.selinc.com IX. BIOGRAPHIES Ram Mohan, Reddi is pursuing his master s degree in Electrical and Computer Engineering at Mississippi State University (MSU). He received B.Tech. Degree from Regency Institute of Technology (RIT) affiliated to Pondicherry University, Pondicherry, India in 2007. He is an active member of IEEE and Power and Energy Society. He is the recipient of Outstanding Academic Achievement award for 2003-2007 at RIT. His fields of interest include Power System Automation, Measurement and Instrumentation, and Control systems. Anurag K. Srivastava received his Ph.D. degree from Illinois Institute of Technology (IIT), Chicago, in 2005, M. Tech. from Institute of Technology, India in 1999 and B. Tech. in Electrical Engineering from Harcourt Butler Technological Institute, India in 1997. He is working as Assistant Research Professor at Mississippi State University since September 2005. Before that, he worked as research assistant and teaching assistant at IIT, Chicago, USA and as Senior Research Associate at Electrical Engineering Department at the Indian Institute of Technology, Kanpur, India as well as Research Fellow at Asian Institute of Technology, Bangkok, Thailand. His research interest includes real time simulation, power system modeling, power system security, power system deregulation and artificial intelligent application in power system. Dr. Srivastava is senior member of IEEE, and member of IET, Power and Energy Society, Sigma Xi and Eta Kappa Nu. He serves as vice-chair of IEEE PES career promotion committee and secretary of IEEE PES student activities committee. He is recipient of several awards and serves as reviewer for IEEE Transactions, international journals and conferences VIII. REFERENCES [1] U.S Department of Energy, The Smart Grid: An Introduction, Office of Electricity Delivery and Energy Reliability, 2008, [Online]. Available at:http://www.oe.energy.gov/documentsandmedia/doe_sg_book_sin gle_pages%281%29.pdf