Validation of ehs FPGA Reconfigurable Low-Latency Electric and Power Electronic Circuit Solver

Validation of ehs FPGA Reconfigurable Low-Latency Electric and Power Electronic Circuit Solver Jean Bélanger, Amine Yamane, Andy Yen, Sébastien Cense, Pierre-Yves Robert OPAL-RT Technologies 1751 Richardson, Suite 2525, Montréal, Québec, H3K 1G6, Canada jean.belanger@opal-rt.com Abstract This paper discusses the validation process and example of power electronic circuits simulated with a general purpose solver implemented on FPGA chips. The Electric Hardware Solver or ehs presented in this paper has the goal to facilitate the usage of FPGA for high-fidelity Hardware-In-the- Loop simulation with sub-microsecond time step by avoiding the difficulties associated with the coding of FPGA devices. Several examples, from very simple to more complex using one or several FPGA boards, are presented and results are compared with traditional simulation software such as SimPowerSystems and PLECS. It will be demonstrated that FPGA-based simulation is now accessible to control and simulation system specialists without requiring any FPGA programming skills. In fact, preparation of FPGA simulation requires only the use of PLECS or SimPowerSystems schematic user interface. I. INTRODUCTION Most fast power electronic systems and controllers are still designed and tested using physical setups and actual controller hardware since that traditional simulation software are too slow to implement accurate hardware-in-the-loop (HIL) real-time simulators. Making hardware-in-the-loop (HIL) tests of fast power electronic systems is very challenging for many reasons, which have restricted the use of HIL simulators for the design and tests of slower power electronic systems found in highvoltage power grids using line commutated converters. As discussed in [1] requirements for Hardware-In-the-Loop (HIL) testing of motor drives and other power electronic converters are getting more stringent as the performance of the devices increases. It is not uncommon today to have motor drive controllers with sample times below 25µs and PWM frequencies well above 20 khz. Multi-levels converter topologies with high-frequency PWM are now contemplated to interface photo-voltaic cells with distribution systems [8] in order to meet harmonic levels and economical constrains. New active filters, energy storage systems, electronic transformers, plug-in hybrid vehicles or very fast power electronic converters are considered for <more electrical> aircraft generation, distribution and active load systems. Implementing accurate HIL simulators for fast converters is required to decrease development and testing costs, to decrease time to market and to perform tests that are difficult to perform with physical test benches. Ideally, HIL simulators using virtual plant models interfaced with prototype or actual controllers should be used concurrently with physical benches. Most R&D work should be done by several engineering teams using accurate HIL simulators while the physical benches should be used to test the final design and validate the numerical models. Both tools should be used optimally according to their respective advantages and limitations. The benefits of using accurate simulators are well recognized by several power electronic manufacturers and research organizations. Consequently several HIL simulator manufacturers and universities are developing fast simulators using Field Programmable Gate Array (FPGA) chips or other specialized processor technologies to achieve the required time step values often below 500 ns. However, implementing accurate HIL fully numerical simulators for fast power electronic systems is very challenging. It requires: very low-sample time to simulate fast electromagnetic transients caused by power device switching, the necessity to sample the IGBT gate signals with high resolution much better than one microsecond and to minimize the latency between the firing pulse signals generated by the controller under test and the voltage and current signals send back to the controller. Such delay should ideally be below one microsecond to not influence the overall system dynamic for fast power electronic systems. Furthermore, HIL simulators should be powerful and scalable a) to simulate several interconnected and fast power electronic systems to analyze their interaction and b) to simulate the interactions with power grids including cables, lines, generators and transformer saturation characteristics. The difficulties and solutions to implement an easy-to-use general purpose electrical solver on FPGA chips such as ehs developed by OPAL-RT have been reported in [1]. These difficulties are related to

the long time to generate the FPGA code, which can reach several hours, the fixed point processing of basic FPGA operators, which may lead to over and under flow, the complexity of implementing mathematical algorithm in FPGA taking into consideration the lowlevel language and parallel processing feature of FPGA chips to achieve low latency and the limited resources of the FPGA chips, which force the used of simplified solvers. For these reasons, OPAL-RT is putting many efforts into developing an FPGA electric general purpose EMT hardware solver (ehs). Similar to a classic differential equation solver, the ehs is an electric circuit solver designed to accept and solve a variety of simulation problems on the FPGA. The main objective of this development is to hide the complexity of FPGA programing to control and simulation specialists by providing them an easy-to-use but very powerful and accurate simulation tool. The second objective is to facilitate the interface with real-time solvers executed on multicore processors to simulate complex power grid. This paper discusses the specifications of the ehs and the method used for its validation using several power electronic circuit examples. II. EHS REQUIREMENTS, FEATURES AND DESIGN FLOW A. ehs Solver Requirement Difficulties mentioned in the previous section are taken into account today in the development of a new solver approach. The solver requirements are: Must solve switched electric systems with minimal knowledge of FPGA coding methods. Must provide effective and maximum user interactivity by enabling on-the-fly modification of the circuit topology, component parameters and the simulation of several types of faults at different locations. Such flexibility prevent generating the FPGA bitstream at each modification, which save several hours of waiting time Must enable users to define the circuit topology and parameters using graphical circuit editor interface, which eliminates the needs to learn complex FPGA programing language. Must solve systems with several motor drives using filters and multi-levels inverters, which may require the use of several interconnected FPGA chips Must enable the simulation of the interaction between several power electronic sub-systems interfaced with power grids as found in PV and wind farms as well for the integration of plug-in hybrid vehicles and microgrids. Such requirements involve the interface between fast power electronic systems simulated on FPGA and power grid components simulated on classical multicore real-time simulators [11]. B. Key Characteristics As described in detail in [1], the key characteristics of the ehs FPGA solver are: Uses nodal method with fixed admittance matrix [5] to solve switched electric circuits. Use the Backward Euler method to provide very good accuracy and stability with time step below 1 microsecond. Is not limited by the number of switches in the simulated circuit because of the use of constant admittance matrix approach. Time step is variable depending on the circuit size and between 100 nanoseconds to 1 microsecond. Can be interfaced with RT-LAB multi-core power grid simulator with time step below 10 µs to analyze the interaction between the power electronic and grid components [11]. C. ehs Design Flow The ehs is designed to facilitate the design cycle of complex circuit simulation on the FPGA by allowing a gradual simulation integration set-up from off-line simulation to FPGA on-chip simulation. This simulation integration methodology is described in Fig. 1 Fig. 1. Simulation integration methodology of ehs The most important objective of this methodology is to allow a gradual verification method for the circuit that is required to be simulated. At the beginning of this process, a graphical interface is used such as the one from SimPowerSystems, EMTP-RV, PLECS, PSIM or SPICE, allowing the verification of the functionality of the circuit before its implementation on the FPGA. These software applications can generate simple and readable net lists. At the time of writing this article, only SPS

and PSIM are supported but implementing a new interface is rather easy. The ehs uses these readable net lists to compute all the matrices and data required by the ehs electrical solver implemented in the FPGA. At this stage, the simulation package used to make the net list can be used to cross-validate the FPGA simulation results. The next stage is to convert the circuit into data compatible with the ehs solvers implemented in the FPGA firmware, which are stored inside MAT files. This step is performed automatically by ehs, and the MAT files have two uses: Formatting the ehs data for upload to the FPGA at the beginning of the simulation. Simulating off-line, if necessary, the ehs circuits with the same equations as the ones that are implemented in the FPGA solvers. M-file scripting of the FPGA simulation provides at least a 100 times increase in simulation speed over real Xilinx System Generator so it can help pinpoint some problem before FPGA on-chip simulation, the last step of the process. The time from starting from the circuit schematics to start the simulation takes a few minutes III. ACHIEVED MILESTONES Many circuits have been implanted with the ehs FPGA solver. The tests circuits presented in this paper are: A three-phase-to-three-phase matrix converter feeding an R-L load. Two-level diode rectifiers with input filter feeding an R-L load. A three-level inverter feeding inductive load. A boost shopper feeding an R-L load. An AC-DC-AC system feeding a Permanent Magnet Synchronous Machine (PMSM). The converter is a two-level IGBT rectifier connected to a three-level IGBT inverter, with LC input and output filters. Zig-Zag-Connected chopper converter feeding resistive load. For each case, the results obtained with the ehs FPGA solver are compared with the results obtained with SimPowerSystems [9] or PLECS [10] using a very small time step of 1 µs or lower. This paper presents some results of the AC-DC-AC system. Besides achieving high accuracy because of the small time step, such FPGA-based simulation tool is very convenient to develop and optimize control systems to accelerate the simulation in off-line mode or to test real controller hardware in HIL mode. Users can modify circuit parameter values and connection within a few seconds without regenerating FPGA code, which saves hours during system optimization. Of course, ehs is designed to let users implement their models with the minimum training. Such a feature is required by several manufacturers who want to protect their IP and consequently prefer not to ask third-party organizations to develop their power models. However, pre-made validated models with modifiable parameters can be delivered to specific departments who need to concentrate their work only on software controller tests without modifying the models. Such low-cost test systems can be implemented by the R&D department in charge of supporting testing divisions or by a third-party such as OPAL-RT depending on the organization IP protection politic. A. Matrix Converter Fig. 2. shows the power circuit of a matrix converter driving a three phase R-L load. The matrix converter is connected to the grid throw an input L-C filter. The converter protection is insured by the clamp circuit as shown in Fig. 2. Fig. 2. Power circuit of a three-phase to three-phase matrix converter This matrix converter circuit runs on FPGA with a timestep of 500ns. It can be interfaced with complex power grids on each side simulated on standard Intel CPU cores. The next figure shows the matrix converter output current in case of a step up on the controller frequency set point input. Initially the controller frequency set point was set to be equal to the grid frequency (50 Hz). At time 1 second the frequency set point was changed to three times the grid frequency 150 Hz. Fig. 3. Real time matrix converter output current for a frequency reference step 1) Real-Time Simulation Setup The matrix drive including matrix converter, the L-C input filter and the clamp circuit are running all on the ML605EX1 -

Xilinx Virtex-6 FPGA processor of the RT-LAB efpgasim simulator. The system voltage source, the three phases R-L load and the matrix converter controller are running on only one of the 12 dual-xeon-based, 3.33 GHz cores also included in the OP5600 RT-LAB simulator. The communication between the CPU cores and the FPGA card simulator is performed internally by the RT-LAB simulation engine using PCI Express bus link, enabling multirate real-time simulation. Fast power electronic subsystems are simulated with sub-microsecond time step using ehs with the VIRTEX 6 processor while other electrical circuits such as transformer, lines, cables, generators and loads are simulated with a time step of 10 us or larger on standard INTEL multicore processors. SimPowerSystem and ARTEMIS-SSN parallel and real-time solver [3] are used for electrical circuit while Simulink is used for the control systems. B. Three-Level-Three-Phase inverter The next figure shows a three-level-three-phase IGBT inverter driving a three phases R-L load simulated on the same efpgasim VIRTEX 6 simulator described above Fig. 5. 3-phase diode rectifier with source inductance and load (Ts=170 ns). The possibility to implement complex circuit with a time step as low as 170 nanoseconds with large number of switches is an important aspect of the nodal method with fixed admittance: it does not pre-compute of all system matrices corresponding to each switch status, which would take too much low-latency static memory exceeding the capability of FPGA chips. D. DC to DC converter (boost type) Fig. 6 shows a DC to DC converter (boost type), feeding a fixed resistive load. Fig. 6. DC to DC converter (boost type) Fig. 4. Power circuit of the Three-Level inverter with a RL load The inverter output voltage and current obtained in real time simulation using ehs solver with a time step of 500 nanoseconds were, compared to the results obtained using the SimPowerSystems offline simulation with Tustin solver and a time step of 500 nanoseconds. This comparison shows that the real time results and the reference results are exactly the same. The above circuit uses less than 50% of the Xilinx Virtex-6 FPGA. C. 3-Phase Diode Rectifier The 3-phase diode rectifier circuit is depicted in Fig. 5. The simulation of such a rectifier on FPGA is required when the fundamental frequency of the generators is very high, as expected in future aircrafts and other applications where space and weight are limited. Such circuit can be simulated with a time-step of 170ns with the Xilinx Virtex-6 FPGA. Such low time-step enables to use high-frequency PWM converters. E. AC-DC-AC converter Fig. 7 shows a AC-DC-AC converter, feeding a Permanent Magnet Synchronous Motor (PMSM). It consists of a 6.6 kv, 60 HZ three-phase power source connected to a 6.6 kv/440v, 60 Hz Y/D transformer. The 440V, 60 Hz voltage obtained at the secondary of the transformer is first rectified by a six pulse IGBT bridge. Then, the filtered DC voltage is applied to the IGBT Three-Level Inverter. The IGBT inverter uses Pulse Width Modulation (PWM) with an 8-kHz carrier frequency. The rectifier Pulse Width Modulation (PWM) frequency is 4 khz.

Fig. 7. AC-DC-AC Converter driving a PMSM 1) Real-Time Simulation and Results The circuit is simulated as follow: The Rectifier including the first L-C filter are running on FPGA and discretized with a time step of 400ns. The inverter including the second L-C filter are running on FPGA and discretized with a time step of 690 ns The FPGA PMSM motor [6] model is discretized using a time step of 100ns The AC source and the transformer are simulated il the main Intel CPU at 15 microseconds. More complex power grid can be simulated. The controller is implemented in a separate Intel core with Simulink. 2) Sinusoidal Motor Power Variation Test During this test, the inverter voltage reference angle is varied from 0 to 360 degrees over 5 seconds to vary the motor power. Fig. 8 and 9 present the results obtained using the real time ehs model compared to the results obtained using the SPS offline model with a time step of 500 nanoseconds. The first Fig. 8 presents zoomed waveforms around 4.5s while Fig. 9 presents the motor power during the complete power swing. One can observe that the motor current and power calculated by the real-time ehs models compare very well with the results obtained in off-line mode with SPS. 3) Motor Speed Variation Test In the following test cases, the motor power is firstly adjusted to zero for a motor speed of 100 Hz. Then the motor speed is varied from 100 Hz to 200 Hz over 5 seconds by fixing the speed on the rotor shaft. The angle between the motor back EMF and the reference voltage of the inverter is kept constant to zero but the amplitude of the inverter voltage varies proportionally to the motor speed. The next figures show the different motor waveforms during the speed variation test. As expected, the motor current decrease with the motor speed, but the power remains at zero during the test. Results obtained with SimPowerSystems and ehs FPGA solver are the same but of course, the FPGA simulation achieves real-time speed. Fig. 10. Motor speed Fig. 8. Motor current with SPS and ehs (RT) Fig. 11. Motor current Fig. 9. Motor power with SPS and ehs (RT)

Fig. 12. Motor power The above circuit is rather complex and demonstrate the capabilities of the ehs solvers. More complex AC grid can be simulated on the main processors and several AC-DC-AC converters can be simulated using OPAL-RT OP7000 multi- FPGA platform. One must realize that such an AC-DC-AC motor drive connected to the grid is a rather complex circuit. The simulation of such circuit with conventional off-line simulation software can take several minutes to simulate a 10-s test. On the other hand, this circuit can be simulated in real-time with a single Virtex-6 FPGA including the simulation of the PMSM motor based on Finite-Element Analysis [7]. Furthermore, more complex AC grids can be simulated using several Intel processor cores and more drives can be simulated using several FPGA processor boards [1] included in the OP7000 multi-fpga efpgasim platform [4]. F. Photo-Voltaic Inverter The next Fig. 13 presents a typical multi-level DC to AC inverter used to couple photo-voltaic cells with the grid. It is composed of several two-level DC-AC converters connected in series to achieve minimum losses and harmonic and to reduce space and cost. Fig. 13. Zig-Zag connected converter power circuit [8] The above circuit can be simulated using 50% of one Virtex 6 FPGA with a time step of about 600 ns. IV. CONCLUSION This paper has presented the structure of the ehs FPGAbased electrical system simulation tool. The ehs objectives are 1) to increase the simulation accuracy of complex and fast electric circuit and drives by achieving very small model time step update and 2) to facilitate the usage of FPGA chips to enable control and system engineers to take advantage of FPGA-based simulation. The ehs electrical solver module allows users to program the FPGA to simulate circuits with variable topologies and parameters by simply reloading models parameter. The FPGA code does not need to be regenerated, which save a lot of time. The structure of the ehs tools enable the use of schematic capture graphical interface provides with popular simulation software such as SimPowerSystem, EMTP-RV and SPICE. Such a feature facilitates the use of FPGA-bases simulator and the result validation. The ehs solver module notably uses a Fixed Admittance Matrix Nodal method that alleviates switch count and memory limitation problems found in other approaches. The ehs electrical simulation tool includes a library of premade machine models and custom blocks such as inverters models as well as a library of ready-to-used and reconfigurable models as presented in this paper. It is compatible with OPAL- RT OP5600 and OP7000 real-time simulator platforms supporting INTEL multi-core standard processors and multi- FPGA chips. REFERENCES [1] C. Dufour, S. Cense, T. Ould-Bachir, LA. J. Belanger, General-purpose reconfigurable low-latency electriccircuit and motor drive solver on FPGA J. IECON 2012-38th Annual Conference on IEEE Industrial Electronics Society, 3073-3081,2012. [2] P. Pejovic and D. Maksimovic, A method for fast time-domain simulation of networks with switches, IEEE Transactions on Power Electronics, vol. 9, no. 4, pp. 449 456, July 1994. [3] C. Dufour, J. Mahseredjian, J. Bélanger, and J. L. Naredo, "An advanced real-time electro-magnetic simulator for power systems with a simultaneous state-space nodal solver," in IEEE/PES Transmission and Distribution Conference and Exposition: Latin America, São Paulo, Brazil, 2010. [4] RT-LAB OP7000 Multi-FPGA simulation platform : Opal-RT Website, www.opal-rt.com [5] Ould Bachir, T. and Dufour, C. and Bélanger, J. and Mahseredjian, J. and David, J.P. A fully automated reconfigurable calculation engine dedicated to the real-time simulation of high switching frequency power electronic circuits, Partial acceptance for Special Issue of Elsevier journal on Mathematics and Computers in Simulation (MATCOM), 2012. [6] C. Dufour, S. Cense, T. Yamada, R. Imamura, J. Bélanger, FPGA Permanent Magnet Synchronous Motor Floating-Point Models with Variable-DQ and Spatial Harmonic Finite-Element Analysis Solvers 15th Int. Power Electronics and Motion Control Conference, EPE- PEMC 2012, Novi Sad, Serbia, Sept. 4-6, 2012 [7] C. Dufour, H. Blanchette, J. Bélanger, Very-high Speed Control of an FPGA-based Finite-Element-Analysis Permanent Magnet Synchronous Virtual Motor Drive System, 34th Conference of the IEEE Industrial Electronics Society(IECON-08), Orlando, Florida, USA, November 10-13, 2008 [8] Yamane A, W. Wang, L-A. Gregoire, Wei Li, The Real-time Simulation of a Power Grid Integrating Solar Energies Using a Novel Power Conditioner, IEEE Workshop on Complexity in Engineering (COMPENG 2012), 1-4, 2012 [9] SimPowerSystems, http://www.mathworks.com/products/simpower/ [10] PLECS, www.plexim.com [11] RT-LAB emegasim and efpgasim: Opal-RT Website, www.opalrt.com