How can I. implement a high-availability system? Develop your project. System Technical Guide High Availability solutions

Transcription

1 How can I implement a high-availability system? System Technical Guide High Availability solutions Develop your project

2

3 Disclaimer This document is not comprehensive for any systems using the given architecture and does not absolve users of their duty to uphold the safety requirements for the equipment used in their systems or compliance with both national or international safety laws and regulations. Readers are considered to already know how to use the products described in this document. This document does not replace any specific product documentation. 3

4 The STG Collection System Technical Guides (STG) are designed to help project engineers and Alliance System Integrators during the development of a project. The STGs support users during the architecture selection and the project execution (design, configuration, implementation and operation) phases, with an introduction to the system operating modes. Each STG is a starter kit that provides users with: Technical documentation Application examples Object libraries Each STG addresses one or several customer challenges within the proposed solution using the offer from Schneider Electric. All explanations and applications have been developed by both Schneider Electric experts and system integrators in our solution labs. The contributions from the system integrators help the kit s content address the needs of our users. All STGs are illustrated with industry-specific applications to give more concrete examples of the methodology. The STGs are not intended to be used as substitutes for the technical documentation related to the individual components, but rather to complement these materials and training. Development Environment Each STG has been developed in one of our solution platform labs using a typical PlantStruxure architecture. PlantStruxure, the process automation system from Schneider Electric, is a collaborative system that allows industrial and infrastructure companies to assess their automation needs while at the same time addressing their growing energy management requirements. In a single environment, measured energy and process data can be analyzed to yield a holistically optimized plant. 4

5 Table of Contents 1. Introduction Purpose Customer Challenges Prerequisites Project Methodology Selection Redundancy Basics Operational Principles Selection Criteria Selected Architecture Conclusion Design Introduction SCADA System Premium Hot-Standby System Quantum Hot-Standby System Configuration SCADA Control and Field Network Premium Hot-Standby PAC Station Quantum Hot-Standby PAC Station Advantys STB Ethernet I/Os Implementation Premium PAC

6 5.2. Quantum PAC Conclusion Performance Performance test protocols Premium PAC Architecture Quantum PAC Architecture

7 1-Introduction 1. Introduction 1.1. Purpose The purpose of this document is to provide recommendations, guidelines and examples to help implement a high availability automation system. A System Technical Note: How can I increase the Availability of a system detailing the theoretical basics of high-availability has already been issued. This guide focuses on the different ways to increase availability through redundancy at various layers of the automation system and proposes a methodology to implement an efficient high-availability automation system. Each step of the implementation of the solution, from the selection to the operation, is broadly described and illustrated with examples. The solutions described in this STG are fully part of the PlantStruxure control system. The High availability System Technical Guide is structured in two parts: Part 1: Redundant architecture with single attachment. Part 2: Complex architecture including dual network attachment and dual ring Customer Challenges For customers in industries where availability and reliability are major concerns, the challenges are: Provide a high level of availability. Attain a high level of reliability. Short recovery following system unavailability. Maintainability facilitated by redundancy. Switchover time adapted to critical processes. This STG suggests approaches to these challenges and highlights specific areas using SoCollaborative Engineering products in line with PlantStruxure Control System. 7

8 1-Introduction 1.3. Prerequisites We recommend the reader have knowledge of the following SoCollaborative software: Unity Pro Vijeo Citect We also recommend the reader become familiar with the System Technical Note: How can I increase the Availability of a system and to have knowledge of the Premium/Quantum/M340 Schneider PACs Project Methodology This STG describes the project methodology and includes the following phases: Selection, Design, Configuration, Implementation and Performance. This guide is illustrated using 2 architectures (Premium and Quantum). Their features are described in the Selection phase. Each architecture shows a specific feature necessary of Hot-Standby application. Beginning with process analysis and user requirements, we identify and develop common functionalities for all the architectures. These key functions are explained in the Design, Configuration and Implementation phases. Finally, the Performance phase summarizes the results of different tests performed on the 2 architectures. Here are the phases described in this document: I. Selection: In this phase, the selection procedure to define a redundant architecture is presented: Basic of redundancy Operational Principles Description of architectures II. Design: This phase covers the operational principles of the different components of high availability architecture: SCADA system Network PAC Station Quantum and Premium 8

9 1-Introduction Specifications and constraints I/O system DFB library III. Configuration: In this phase, the configuration information of the different components of the architectures is detailed. IV. Implementation: Using the same architectures and components, information about final customization to address the project requirements is provided. V. Performance: A summary of the architectures performance in response to simulated events is presented in this phase. 9

10 2-Selection 10

11 2-Selection 2. Selection During the selection phase, an optimal architecture is chosen as well as the most appropriate components of the project, according to your specific requirements. Several architectures and systems are presented in this chapter in order to address a wide range of functions and needs. Also, the way to select among these architectures given project needs is presented. The following illustration summarizes our project development approach: 11

12 2-Selection 2.1. Redundancy Basics This chapter describes redundancy general principles and its application in an automation system. The following PlantStruxure architecture is a representative example to illustrate the different layers where redundancy can be implemented. SCADA Clients Ethernet Data Servers Ethernet Control Network PACs Station PAC PAC PAC Field Network Ethernet Profibus DP Ethernet Field Devices The diagram also represents a wide range of hardware setups demonstrating the ability to achieve various redundancy levels Redundancy Layers Availability can be increased in an automation system at different layers: SCADA system: The SCADA system has to handle data acquisition, graphics, events, alarms, trends, and reports. SCADA server redundancy enhances the likelihood these services will continue to operate without loss of data in case of system interruption. Different software and hardware configurations allow different levels of availability. Control Network: A well defined topology and management of the control network increase network availability and reliability. Thus, in turn, makes communication between the SCADA system and the PAC stations more reliable. Several network topologies and network 12

13 2-Selection protocols are available to achieve the optimal level of availability and to fit the whole system needs. PAC station: According to your needs in terms of I/O number and topology, you can choose among a Quantum or a Premium Hot-Standby PAC system. The field network type is also an element to consider before choosing the PAC station, as well as the I/O system (local or distributed). Field Network: Redundancy can also be applied to the field network. As was the case for the control network, a well defined topology and management of the network increase field network availability. Device redundancy is also implemented to increase availability of the field equipment Redundancy level We can differentiate several levels of redundancy according to their performances in term of availability. A summary of these levels is presented in the following table: Redundancy Level State of the standby system Switchover performance No redundancy No standby system Not applicable Cold Standby Warm Standby Hot-Standby The standby system is only powered up if the default system becomes inoperative. The standby system switches from normal to backup mode. The Standby system runs together with the default system. Several minutes Large amount of lost data Several seconds Small amount of lost data Several milliseconds No lost data 13

14 2-Selection 2.2. Operational Principles Depending on the level of availability required, redundancy is applied in various ways. This chapter discusses Hot-Standby applications and describes the basics of redundancy in each layer of an automation system. Moreover the chapter describes the various options available in terms of redundancy and availability. The selected architectures used in the following parts of this STG are described in the Chapter SCADA The main operating principles of the different SCADA servers are described in the following paragraphs. General Principles The different servers of the SCADA system (Alarm, Report, Trends, and I/O servers) can either be installed on the same computer or on different computers allowing for more reliability. For a redundant configuration, each server (Primary) is associated with its redundant server (Standby) installed on a different computer. For example, the picture below describes servers installed on redundant computers (Primary and Standby) 14

15 2-Selection I/O Server redundancy In a redundant SCADA system, an I/O device is associated with the Primary and Standby servers. The Primary server accesses periodically the I/O device to read and write tags. The Standby server only checks the communication with the I/O device. At startup, if the Primary I/O server can not establish a connection with the I/O device, the SCADA system switches to the Standby I/O server. During operation, if the Primary I/O server stops communicating with the I/O devices, the system then switches to the Standby I/O server. The following diagrams illustrate 2 cases: a broken network cable and a server that has stopped communicating. When the I/O server defined as Primary returns to operational state, the SCADA system returns control back to the Primary server. Alarms / Trends / Reports Servers (ATR) redundancy The management of the Alarms, Trends and Report servers (ATR servers) by the SCADA system follows the steps listed below: If the Primary ATR server stops operating, the system switches to the Standby ATR server. When the ATR server defined as Primary returns to operational state, any clients connected to the Standby ATR server remain connected to the Standby server. 15

16 2-Selection For example, the following picture describes a server reconfiguration initiated by a switchover: I/O, Trends and Reports servers are working on the Primary SCADA server, and the Alarms server is working on Standby SCADA server. 16

17 2-Selection Network Various topologies and protocols are used to increase the availability of the network (control or field network). The principle is to create different paths to access devices. In case of a network element on the main path stops functioning, another path is used. The following table illustrates the main network topologies. Architecture Limitations Advantages Disadvantages Bus The traffic must flow serially, Cost-effective solution If a switch becomes therefore the bandwidth is not inoperative, used efficiently. communication is lost. Star Efficient use of the If the main switch bandwidth, as the traffic is becomes inoperative, spread across the star. communication is lost. Tree Cable ways and distances Preferred topology when there is no need for redundancy. Ring Auto-configuration if used The availability of auto- with self-healing protocol. configuration depends Possible to couple other on the protocol used. Dual Ring Behavior similar to Bus. rings for increasing redundancy. Ring topologies are mainly used to increase the level of network availability. Network redundancy management protocols, such as Hiper-ring or MRP, are used for network recovery in case of part of the network cease to function. 17

18 2-Selection PAC Station Hot-Standby Definition A Hot-Standby system is used when downtime cannot be tolerated. It delivers high availability through redundancy and always consists of two units with identical configurations. One of the two units acts as the Primary CPU controller, and the other acts as the Standby CPU controller. One controller must be set in the Primary CPU state and the other must be in the Standby CPU state or offline. The redundant unit takes the control when the main one encounters an anomaly. The Primary PAC updates inputs, manages Hot-Standby, runs the program while transferring data to the Standby PAC and updates outputs. Thus, the switchover between the Primary and the Standby PAC occurs without any loss of data. As described on the diagrams above, for each execution cycle, the outputs update only takes place when the data transfer AND the program execution are completed. Therefore, it is important to properly define the amount of data to be transferred from the Primary to the Standby PAC to minimize the wait time induced by a data transfer longer than the execution time of the program execution. On the diagram on the left, the cycle execution is optimized: the data transfer is performed faster than the program execution. On the diagram on the right, the longer data transfer induces a wait time that slows down the cycle execution. 18

19 2-Selection Primary and Standby PACs Assuming that the configuration of the system is correct, the first PAC to be powered up is automatically recognized as the Primary one. Therefore, you can define the PACs role by controlling the sequence order in which they are powered up. When two redundant CPU PACs are switched on simultaneously, the firmware automatically affects the Primary status according to the MAC address. The PAC with the lower MAC address is defined as the PAC A, that is the Primary at the powering up of the system. 19

20 2-Selection Hot-Standby System Programming Elements This paragraph describes programming basics, useful to know when implementing a Hot-Standby system. System Words %SW60: Command Register The command register defines the operating parameters of a Hot-Standby application for both the Primary and Standby CPU. The System Word %SW60 can be used to read and write the command register of Hot-Standby System. The diagram below illustrates the Quantum System Word %SW60: Disables LCD Invalidate Keypad - bit0 = 0 Enables LCD Invalidate Keypad - bit0 = 1 Sets Controller A to OFFLINE mode - bit1 = 0 Sets Controller A to RUN mode - bit1 = 1 Sets Controller B to OFFLINE mode - bit2 = 0 Sets Controller B to RUN mode - bit2 = 1 Forces Standby offline if there is a logic mismatch - bit3 = 0 Does not force Standby offline if there is a logic mismatch - bit3 = 1 Allows exec upgrade only after application stops - bit4 = 0 Allows exec upgrade without stopping application - bit4 = 1 MSB LSB bit5 = 0 - No application program transfer bit5 = 1 - Application program transfer requested bit8 = 0 - Swaps Modbus port 1 adress during switchover bit8 = 1 - Does not swap Modbus port 1 adress on a switchover bit9 = 0 - Swaps Modbus port 2 adress during switchover bit9 = 1 - Does not swap Modbus port 2 adress on a switchover bit10 = 0 - Swaps Modbus port 3 adress during switchover bit10 = 1 - Does not swap Modbus port 3 adress on a switchover The diagram below illustrates the Premium System Word %SW60: Sets Controller A to OFFLINE mode - bit1 = 0 Sets Controller A to RUN mode - bit1 = 1 Sets Controller B to OFFLINE mode - bit2 = 0 Sets Controller B to RUN mode - bit2 = 1 OS versions Mismatch (this bit can be used to permit temporary differences between the firmware versions on the respective Hot-Standby PACs) MSB LSB 20

21 2-Selection %SW61: Status Register The Hot-Standby Status Register is a readable register located at system word %SW61 and is used to monitor the current status of the Primary CPU and Standby CPU. The following diagram illustrates the Quantum System Word %SW61: This PAC in OFFLINE mode - bit1= 0. bit0= 1 This PAC running in primary CPU mode - bit1= 1. bit0= 0 This PAC running in standby CPU mode - bit1= 1. bit0= 1 Other PAC in OFFLINE mode - bit3= 0. bit2= 1 Other PAC running in primary CPU mode - bit3= 1. bit2= 0 Other PAC running in standby CPU mode - bit3= 1. bit2= 1 The remote PAC is not accessible - bit3= 0. bit2= 0 PACs have matching logic - bit4 = 0 PACs do not have matching logic - bit4 = 1 This PAC's switch set to A - bit5 = 0 This PAC's switch set to B - bit5 = 1 MSB LSB bit7 = 0 - Same PAC OS version bit7 = 1 - Different PAC OS version bit8 = 0 - Same copro OS version bit8 = 1 - Different copro OS version bit12 = 0 - Information given by bit13 is not relevant bit12 = 1 - Information given by bit13 is valid bit13 = 0 - NOE address set to IP bit13 = 1 - NOE address set to IP+1 bit15 = 0 - The hot standby has not been actived bit15 = 1 - The hot standby is active 21

22 2-Selection The following diagram illustrates the Premium System Word %SW61: This PAC in OFFLINE mode - bit1= 0. bit0= 1 This PAC running in primary CPU mode - bit1= 1. bit0= 0 This PAC running in standby CPU mode - bit1= 1. bit0= 1 Other PAC in OFFLINE mode - bit3= 0. bit2= 1 Other PAC running in primary CPU mode - bit3= 1. bit2= 0 Other PAC running in standby CPU mode - bit3= 1. bit2= 1 The remote PAC is not accessible - bit3= 0. bit2= 0 No application Program or Unity Pro configuration Checksum mismatch beetween Remote PAC - bit4 = 0 Application Program or Unity Pro configuration Checksum mismatch beetween Remote PAC - bit4 = 1 This PAC set as Unit A - bit5 = 0 This PAC set as Unit B - bit5 = 1 CPU-sync link OK - bit6 = 0 CPU-sync link NOK - bit6 = 1 No processor OS version mismatch - bit7 = 0 Main processor OS version mismatch - bit7 = 1 No Copro OS version mismatch - bit8 = 0 Copro OS version mismatch - bit8 = 1 MSB LSB bit10 = 0 - No monitored ETY OS version mismatch bit10 = 1 - Monitored ETY OS version mismatch bit13 = 0 - Configured IP or Modbus Adress bit13 = 1 - Configured IP or Modbus Adress +1 bit15 = 0 - The hot standby has not been actived bit15 = 1 - The hot standby is active bit9 = 0 - All in-rack (Monitored and no-monitored) ETY modules have the minimun version bit9 = 1 - At least one ETY does not have the minimun version 22

23 2-Selection %SW62 65: Reverse Register: System Words %SW62/63/64/65 are reverse registers reserved for the reverse transfer process. The reverse registers can be written in the application program (first section) of the Standby CPU controller and are transferred at each scan to the Primary CPU controller. Non-Transfer Area The Non Transfer Area is a defined memory zone which is not transferred during the update of the Standby CPU controller. The Premium PAC has a 100 words predefined zone (%MW0 to %MW99) whereas the Quantum PAC the size of the zone is defined by the user (%MW1 to %MWx). First Section (section 0) In a PAC redundancy system, the execution of the application program is different according to the PAC in which the execution takes place. The main difference is that the whole application program is executed in the Primary PAC whereas the Standby PAC only executes the first section (section 0). This point is very important as many settings of the system are defined in the section 0. Hot-Standby Management Switchover Conditions A Hot-Standby system is designed to provide uninterrupted service. This feature requires continuous monitoring of different equipment. Note: Concerning both Premium and Quantum, the switchover is performed only if the Standby PAC is operational and ready to take over control from the Primary PAC. A Hot-Standby system continuously monitors the key components in order to detect any stoppage in operation. Additional monitoring is performed by the application for more specific requirements. The monitoring by the system initiates a switchover on the following occurrences: Premium system Fault on power supply Fault CPU (firmware, hardware) Halt, Stop, Offline CPU Fault Monitored ETY module (firmware, hardware) 23

24 2-Selection Quantum System Fault power supply Fault CPU (firmware, hardware) Halt, Stop, Offline CPU Fault CRP module For the Premium and Quantum Hot-Standby architectures presented in this guide, additional equipment (Network Controller and Device Network) are monitored by the application in order to increase the availability. To perform this specific monitoring, we need to develop Derived Function Blocks (DFBs) that monitor the system, control and process the anomalies, and handle the switchover. DFBs Libraries The Unity Pro Quantum system library offers EFBs to manage a Hot-Standby system. These EFBs allow the handling of command (%SW60), status (%SW61) and reverse (%SW62 to 65) registers. The Unity Pro Premium library does not include pre-designed EFBs. Consequently, we have developed a user-defined Hot-Standby DFBs library. This library is described in the next chapter. Network monitoring is integrated in our architectures. This functionality is handled neither by the Premium PAC nor by the Quantum PAC (except for the Monitored ETY module on a Premium system). Therefore, we have developed a specific Hot-Standby DFBs library for each configuration: Premium ETY_Monitor (Ethernet) Quantum NOE_Monitor (Ethernet) PTQ_Monitor (Profibus) We have developed an events synthesis block that processes the output of these DFBs, while offering the possibility to mask some defaults, and a switchover management block which controls the availability of the Standby PAC before initiating a switchover. 24

25 2-Selection 2.3. Selection Criteria Various levels of performance can be attained with different architectures and using different components. It is crucial to select the right configuration that most closely fits your needs in terms of availability, cost and maintenance. Before implementing a high-availability system, consider the following points: What is the general availability level to reach for the whole system? How many devices can stop functioning, yet have the system remain operational? What is the maximum allowed downtime for the entire system? Are there any constraints (existing system, topology) that imply the use of specific tools and equipment? What is the size of the system and are additional extensions planned? What is the topology? Centralized? Distributed? Are there some areas of the process with priority needs? 2.4. Selected Architecture We have selected two representative architectures to illustrate in this guide, one for Quantum PAC and one for Premium PAC. All the other layers are common for the two architectures. These architectures are intended to represent a medium range automation system with high availability needs in terms of process control and medium availability in terms of SCADA and network control. Each layers in the system can tolerate one non-operating device and still remain operational. The implementation of the different layers is described in the following chapters SCADA The resources used by our application are moderate, so all the servers (Alarms, Trends, Reports and I/O) can be installed on one computer. In order to withstand one non-functioning device, the redundant servers are installed on a second computer. Two clients connected to the network allow the control and monitoring of the process. This configuration is adapted to our needs in terms of performance and redundancy level. 25

26 2-Selection Control Network The ring architecture is chosen for its redundancy capability. Four ConneXium switches handle the ring architecture. One switch or cable segment can cease operating with no impact on the communication through the network. The MRP redundancy management protocol is chosen for its performance in recovery time. 26

27 2-Selection PAC station The Hot-Standby architecture allows that one PAC stops operating without loss of data. As was the case with the control network, if the Primary PAC ceases to operate, the Standby PAC takes over from the Primary. The following diagram sums up the selection of the PAC station according to specific requirements of the application. Time Critical Application? N Y In-Rack I/O Stations only! Application requiring multiple and/or scattered I/O Stations? N In-Rack and/or distributed I/O system Y Remote I/O Stations Local I/O Station Redundant I/O Modules required? Y N Premium or Quantum HSBY Quantum HSBY Premium or Quantum HSBY Premium HSBY The selected architectures, Premium and Quantum, are detailed in the following paragraphs. Note: The number of In-Rack I/Os in the process is decisive for dimensioning the system. As a Premium Hot-Standby system does not handle extension racks, the use of In-Rack I/Os is limited. This means that, beyond a given number of In-Rack I/Os, a Quantum PAC that handles Remote I/Os will be used instead of a Premium PAC. Note: Only the Quantum PAC station provides redundancy solutions for a Profibus network. 27

28 2-Selection Premium The chosen redundant PAC station is a Premium Hot-Standby architecture with redundant analog and digital inputs and outputs. The 2 units are synchronized via an Ethernet link. From Control Network PAC A IP: MASK: PAC B IP: MASK: Sync-link IP: MASK: IP: MASK: JM Concept Modules Analog Output Analog Input ABE7 Connection Blocks ABE7 ABE7 Digital Output Digital Input To Ethernet Field Network 28

29 2-Selection Quantum The chosen redundant PAC station is a Quantum Hot-Standby architecture with a shared Remote I/Os module. The 2 units are synchronized via an optical fiber link (Sync link) Field Network Profibus DP The first part of the field network is composed of a Profibus DP daisy chain managed by 2 redundant Profibus Master modules. Each extremity of the daisy chain is connected to a redundant Quantum PAC. The control of the device on the chain is possible even if one of the PACS ceases to operate. The Standby Profibus master PAC then handles the control of the chain. 29

30 2-Selection Ethernet The second part of the field network is built around an Ethernet ring to bring redundancy to the field devices. The Ethernet ring is built in the same manner than for the control network. 3 Connexium switches run MRP on the ring and connect different devices connected on Ethernet. 30

31 2-Selection 2.5. Conclusion The 2 following diagrams present the whole Quantum and Premium architectures from the SCADA to the field network. These architectures will be used subsequently in the document to illustrate redundancy runtime principles and performance reviews. The second part of this document will use the same architectures, but including dual ring structures and dual attachment. 31

32 2-Selection Redundant Premium Architecture SERVER 1 SERVER 2 IP: Client 1 IP: IP: SW1 SW2 Manager IP: Client 2 SW3 SW4 PAC A IP: MASK: PAC B IP: MASK: Sync-link IP: MASK: IP: MASK: JM Concept Modules Analog Output Analog Input ABE7 Connection Blocks ABE7 ABE7 Digital Output Digital Input IP: IP: SW10 Manager SW12 SW11 IP: IP: IP: IP: IP:

33 2-Selection Redundant Quantum Architecture SERVER 1 SERVER 2 IP: IP: IP: IP: Client 1 SW1 SW2 Manager Client 2 SW3 SW4 PAC A IP: MASK: PAC B IP: MASK: Remote I/O IP: MASK: IP: MASK: IP: SW10 Manager SW12 SW11 IP: IP: IP: IP: IP: IP:

34 2-Selection 34

35 3-Design 3. Design 3.1. Introduction The design part of the STG covers the operational principles of the different components of high availability architecture. After a short review of the SCADA set up, the following points concerning the PAC station will be detailed: What is a Hot-Standby System (Premium and Quantum) Parts and tools of a Hot-Standby System (Premium and Quantum) Specifications and constraints of a Hot-Standby System (Premium and Quantum) Distributed and In-Rack I/Os (Premium and Quantum) We will also describe the DFBs used in our Hot-Standby library and why they have been developed. The SCADA and Network parts are more detailed in the Configuration chapter 3.2. SCADA System Architecture Presentation The architecture is composed of 2 redundant Vijeo Citect servers, 2 clients and 2 Hot- Standby PACs (Quantum or Premium). The communication between these components is achieved through an Ethernet ring. Each Vijeo Citect server handles I/O, Alarms, Trends and report server functionalities. In our hardware configuration, we choose to install the IO and ATR (Alarm, Trends, and Reports) servers on one computer. The performance of these computers is sufficient in terms of CPU and disk space to handle our application. Only one cluster is configured to manage all servers. 35

36 3-Design 3.3. Premium Hot-Standby System This chapter describes the different features and specifications of a redundant Premium system Premium PAC Specifications Primary and Standby PACs The Primary PAC executes the application program, controls Ethernet network and In-Rack I/Os and synchronizes the Standby PAC at the beginning of each program cycle. The Standby PAC does not run the whole program but only the first section (section 0). Moreover, it does not handle the redundant In-Rack and Ethernet I/Os but just checks the state of the Primary PAC. In case of an anomaly, the Standby PAC takes over the control from the Primary PAC (see switchover time measurements in Performance chapter). Primary and Standby PACs permanently exchange data in order to check the system integrity via the synchronization link. A Premium Hot-Standby system necessarily comprises Monitored ETY modules (one in each rack). These modules handle the diagnosis of Premium CPU redundancy configuration status. This diagnosis is achieved through Sync ETY link. Note: Sync ETY and Synchronization link are different and are not used for the same purpose. 36

37 3-Design Monitored ETY modules As for the CPUs, the position in the rack and the firmware version of the Ethernet modules must be identical. Note: A firmware version 4.0 or earlier is required. The monitored ETY module allows the swap of the Ethernet services as well as the automatic permutation of the IP addresses between Primary and Standby TSX ETY. ETY modules are linked with Ethernet switches (one switch per ETY) or via an Ethernet crossover cable. An optical connection is also possible in the case of a long distance communication. Sync ETY link also allows handling of Ethernet I/O devices with the proper Ethernet I/O Scanning service configuration. In order to initiate a switchover when a Sync ETY link stops operating on the Primary PAC, Ethernet I/O Scanning service must be configured on the monitored ETY module. In addition to the service activation, an I/O Scanning line must also be declared. If the service is not configured in the monitored ETY module, a switchover will not occur if a Sync ETY ceases to operate. In case a monitored ETY module ceases to function, the CPU sends a status modification command to all the configured ETY modules populating the X-Bus and the monitored ETY module populating the Standby PAC to switch their IP addresses. 37

38 3-Design Hardware Constraints The following table lists the only modules that can be used in a Premium Hot-Standby configuration: Power Supply Rack Ethernet Communication All available power supply modules Non-expendable Racks only TSX ETY4103 or TSX ETY5103 (firmware version v4.0 or earlier) - Modbus communication module TSX SCY21601 (firmware version 2.3 or earlier) equipped with multiprotocol communication board TSX SCP114 (firmware version 1.7 or earlier) (slave or master) Modbus Communication Digital I/Os Analog I/Os - Modbus communication module TSX SCY21601 (firmware version 1.1 or earlier) (Master Modbus only) Note: The TSX SCY associated with multiprotocol communication board TSX SCP114 allows the redundant Premium PAC systems to run as Modbus Slave or Master. This configuration allows using Modbus Masters from other suppliers. TSX SCY module can only be used in Modbus Master. No restrictions apply No restrictions apply 38

39 3-Design Software Constraints The following constraints apply at the application level The use of event tasks is not recommended. An event might be lost if it occurs just before or during the switchover. The use of FAST tasks handling dedicated outputs is not recommended as output status modifications might be lost during the switchover. The use of counting modules is not recommended. Following the frequency, some pulses might be lost during the switchover. The use of fronts is not recommended. They might not be accounted during the switchover. The use of the SAVE_PARAM function is not recommended in a CPU redundancy application. This function erases the initial value of a module parameter saved in the program code. This code is not transferred from the Primary PAC to the Standby. More generally, explicit instructions like WRITE_CMD and WRITE_PARAM must be well defined before use. Initial values declared with a recorded attribute (for example DFB variables) can not be replaced with actual values: Do not use%s94 bit. Following inherited functions blocks can not be used: PL7_COUNTER PL7_DRUM PL7_MONOSTABLE PL7_REGISTER_32 PL7_REGISTER_255 PL7_TOF, PL7_TON, PL7_TP PL7_3_TIMER The use of TON, TOFF and TP blocks is not allowed in the first section 39

40 3-Design Premium Hot-Standby DFBs Library The following table summarizes the different DFBs created for our application. DFB FUNCTION HSBY_RD Reading Command word (%SW60) hot-standby system SYSTEM HSBY_WR Writing Command word (%SW60) hot-standby system HSBY_ST Reading Status word (%SW61) hot-standby system ETHERNET ETY_MONITOR Monitoring ETY Ethernet Module SYNTH_FAULT Synthesis Fault monitored elements SYNTHESIS SYNTH_OR_ETY Synthesis Fault ETY module (Logic OR) SYNTH_AND_ETY Synthesis Fault ETY module (Logic AND) SWITCHOVER SWITCH_MANG Switchover Managment System DFBs In order to manage the different registers of a Premium Hot-Standby system, we have created blocks that allow reading and writing registers %SW60 and %SW61. HSBY_RD_P: Read the command register %SW60 HSBY_RD_P PLCA_RUN PLCB_RUN Offline_if_OS_Mismatch BOOL BOOL BOOL Run Mode Controller A Run Mode Controller B OS Versions Mismatch HSBY_WR_P: Write the command register %SW60 Manual Control BOOL Manual_Control_Enable Command Run Mode Controller A BOOL PLCA_RUN Command Run Mode Controller B BOOL PLCB_RUN HSBY_WR_P Forced Command OS no Mismatch BOOL Offline_if_OS_Mismatch This block allows sending switch commands from the program (PLCA_RUN, PLCB_RUN), also, in order to be able to update the CPU OS, the ETY module or the coprocessor, it allows to set the OS mismatch bit to 1 to avoid switching in offline mode. A dedicated input allows sending switch orders for example, during maintenance activities. 40

41 3-Design HSBY_ST_P: Hot-Standby system, status check This block allows to process data from register %SW61. It gives information about each PAC role (Primary, Standby, and Offline), OS version, and so on. HSBY_ST_P HSBY_Active THIS_ISA THIS_ISB THIS_OFF THIS_PRI THIS_Sby REMT_UNDEF REMT_OFF REMT_PRI REMT_SBY LOGIC_OK CPU_SyncLink_OK CPU_OS_OK Copro_OS_OK ETY_minVersion Mon_ETY_OS_OK BOOL BOOL BOOL BOOL BOOL BOOL BOOL BOOL BOOL BOOL BOOL BOOL BOOL BOOL BOOL BOOL Hot-Standby System active This Pac is PAC A This Pac is PAC B This Pac is Offline This Pac is Primary This Pac is Standby Remote state Pac undefined Remote Pac is Offiline Remote Pac is Primary Remote Pac is Standby Identical Logic Pac A et Pac B CPUs synchronized Same CPUs OS Same Copro OS ETY version ok Monitored ETY OS Mismatch 41

42 3-Design Ethernet link monitoring DFBs ETY_Monitor: Ethernet module monitoring ETY_Monitor External default, Ethernet cable unplugged BOOL BLK Fault BOOL Module Fault Module Error BOOL MOD_ERROR Command Run Mode Controller A (T_COM_X103) IODDT COM_ETY5103 COM_ETY5103 IODDT Monitoring Rate value INT Monitoring_Rate Enable BOOL Reading Pulse READ_STS function Pulse computer in the Standby Section BOOL Pulse Monitoring Rate current value INT RateEt RateEt INT The ETY_Monitor DFB monitors the status of the Ethernet link provided by the TSX ETY 5104 (or TSX ETY 4103). We use as inputs the BLK and MOD_ERROR information from IODDT T_GEN_MOD. BLK: external default, Ethernet cable unplugged MOD_ERROR: Module error The IODDT T_GEN_MOD is updated by the READ_STS function. This function reads the status word of a ETY module. The execution rate is controlled by the Monitoring Rate parameter configured by the user (see Chapter 5: Implementation). ETY_Monitor BOOL BLK Fault BOOL BOOL MOD_ERROR IODDT COM_ETY5103 COM_ETY5103 IODDT READ_STS INT Monitoring_Rate Enable BOOL Reading Pulse READ_STS function EN ENO BOOL Pulse %CHx.X.MOD CH INT RateEt RateEt INT The structure of the IODDT T_GEN_MOD is detailed in the table on the next page: 42

43 3-Design ETY3_State T_GEN_MOD + MOD_ERROR BOOL Module error EXCH_STS INT Exchange status STS_IN_PROGR BOOL Status parameter read in progress EXCH_RPT INT Channel report STS_ERR BOOL Error while reading module status MOD_FLT INT Module Faults MOD_FAIL BOOL Internal fault: Module failure CH_FLT BOOL Faulty channel(s) BLK BOOL External fault: Terminal Block CONF_FLT BOOL Hardware or software configuration fault NO_MOD BOOL Module absent or power down EXT_MOD_FLT BOOL FIPIO extension module fault MOD_FAIL_EXT BOOL Internal fault: Module failure (only FIPIO extension) CH_FLT_EXT BOOL Faulty channel(s) (only FIPIO extension) BLK_EXT BOOL External fault: Terminal Block (only FIPIO extension) CONF_FLT_EXT BOOL Hardware or software configuration fault (only FIPIO extension) NO_MOD_EXT BOOL Module absent or power down (only FIPIO extension) The MOD_ERROR bit is set to 1 when an ETY module ceases operation. One frequent cause is the cessation of communication of a device on the I/O Scanning which, in our case, should not initiate a switchover. Therefore, in order to filter this occurrence, we use the T_COM_X103 function to monitor the I/O Scanning status and validate the MOD_ERROR value. When implementing a Hot-Standby system, this block is used once for each ETY module in the configuration. 43

44 3-Design Switchover Management SYNTH_FAULT: Performs the defaults synthesis SYNTH_FAULT Synthesis Fault ETY Module BOOL Faulty_ETY Synthesis Fault SCY Module BOOL Faulty_SCY Synthesis Fault Scada BOOL Faulty_SCADA Fault Mask word WORD Fault_Mask Fault_Synth INT Synthesis Fault Word Fault BOOL OS Versions Mismatch This block aims at processing the faults that would lead to a switchover. We find in input the results of the ETY and SCY modules failure detection. Faulty_SCADA is an input pin in the case of the communication between the SCADA and the PAC is monitored. This DFB also processes: Battery faults %S67 = application memory card battery %S68 = processor battery %S75 = data storage memory card battery CPU fault %S12 = CPU running General In-Rack I/O fault %S119 = fault of one or several I/O modules in the rack Slots 3 to 10 fault %SW160 = operating status of Premium modules installed on station 1 The faults processing is performed using the mask value set on the input pin Fault_Mask. This mask allows to select which fault to take into account according the configuration and to the user s settings. 44

45 3-Design Each fault corresponds to one bit of the Fault_Synthesis word: BIT Element monitored Bit 0 Battery Fault Bit 1 Fault CPU Bit 2 General In-Rack I/O fault Bit 3 Fault on Slot 3 Bit 4 Fault on Slot 4 Bit 5 Fault on Slot 5 Bit 6 Fault on Slot 6 Bit 7 Fault on Slot 7 Bit 8 Fault on Slot 8 Bit 9 Fault on Slot 9 Bit 10 Fault on Slot 10 Bit 11 Ethernet Adapter(s) ETY Fault Bit 12 MODBUS Adapter(s) SCY Fault Bit 13 SCADA Fault The result of this synthesis is saved in a word and set as an output on the Fault_Synth_Plc pin. If there is at least one fault, the output pin Fault is set to 1. During the implementation of the system, this block is used twice: once for the Primary PAC and once for the Standby PAC. In order to be able to compute the status of several ETY modules, logical OR and AND processing DFBs have been created: SYNTH_AND_ETY BOOL FLT_ETY_1 FAULT_ETY BOOL BOOL FLT_ETY_2 BOOL FLT_ETY_3 SYNTH_OR_ETY BOOL FLT_ETY_1 FAULT_ETY BOOL BOOL FLT_ETY_2 BOOL FLT_ETY_3 45

46 3-Design SWITCH_MANAG: Approve or deny a switchover Synthesis Fault word Primary INT PRIM_DIAG Synthesis Fault word Standby INT STBY_DIAG SWITCH_MANAG Switchover Number Reset BOOL SWITCH_NB_Reset SWITCH_NB UNIT Switchover request Manual Switchover BOOL FORCE FORCE BOOL Manual Switchover The Switch_Manag DFB manages and counts switchover queries. The switchover approval is computed from the Primary and Standby PACs diagnosis coming from the Fault_Synthesis DFBs as seen above. A switchover is allowed if: The Standby PAC diagnosis is OK. More than 30s have elapsed since the previous switchover. Note: The time delay before the switchover takes place can be adjusted using variables of the DFB (Delay_Time_Before_Switchover). This delay is set to 1s by default. The switchover counter can be reset using the input pin Switch_N_Reset. For maintenance reasons, the input pin FORCE allows a manual switchover of the system. During the implementation, this block is used only once. Switchover Time Switch_Over_Time Remote Pac is Primary BOOL Remote_is_Primary Sw_Timer TIME Switchover Time This Pac is Primary BOOL This_is_Prima The time gap during the switchover is a very important feature of the Hot-Standby system. A DFB has been defined to measure this time. The principle is based on the measurement of the time when the Primary PAC loses its Primary status and when the Standby turns Primary. This block, placed in the section 0, processes the system word %SW61 information and uses the ITCNTRL block function which allows event time measurements. The accuracy of the switchover time depends on the PAC scan time, for more accuracy, other measurement can be performed as described in the performance chapter. 46

47 3-Design In-Rack I/O System This paragraph describes the management of the I/Os populating the main rack. Inputs acquisition is performed locally by both Primary and Standby PACs, whereas the Primary PAC outputs are mirrored on the Standby PAC (provided that there is no specific action programmed in the section 0). Redundant Digital I/Os Implementation Digital input and output signals are connected to the PAC through an ABE7 connection block. These signals are multiplexed/de-multiplexed by a Telefast connection device as seen on the above diagram (ABE7 ACC11 for the inputs and ABE7 ACC10 for the outputs). Exceptions detected on Digital inputs cannot initiate a switchover. The digital I/Os implementation is illustrated on the diagram below. Digital Outputs in the section 0 As the Standby PAC executes the first section (section 0) of the application program and then applies the object image %Q received from the Primary PAC, it is important not to modify the redundant output status in this section. A modification of the output bits in the section 0 can lead to an inconsistent status of the outputs as they are modified twice in the same MAST task. 47

48 3-Design Digital Outputs fall-back mode In general, the outputs fall-back mode must be similar to their current mode in order to avoid an operation discrepancy during the switchover. Pulse Triggered Actuators Digital output redundancy implies a distortion of the command signal. Output modules are connected in parallel of the physical output, via a connection block. The result of a command is based on the length of the pulse and the delay after which the pulse is applied on the Standby PAC. These mechanisms have to be taken into account in order to handle In-Rack digital output redundancy. Positive pulse trigger As shown on the above diagram, the length of the output is longer than the Pulse Time. This does not have any impact on the device behavior. In the case where the delay is greater than the pulse length and using an actuator with a low response time, the signal received by the actuator might be composed of 2 commands, as shown on the diagram below: 48

49 3-Design Negative pulse trigger As shown on the above diagram, the length of the output is shorter than the Pulse Time. This does not have any impact on the device behavior unless it cannot handle a shorter command. The following diagram presents the case where the delay is greater than the pulse time of the output signal: Because the delay is greater than the Pulse Time, the device will not receive any command. 49

50 3-Design Analog Inputs implementation Analog signals are connected to the PACs through a signal duplicator. For our application, we use a JMConcept TELIS9000U2 module (which replaces reference JK3000N2) The table below describes the signal range handled by the TELIS9000U2: INPUTS Current (continuous) Voltage (continuous) Probe Thermocouple Potentiometer Resistance Sensor Power Supply Standard scales 0/1mA; 0/10mA; 4/20mA; +/-1mA; +/-10mA; +/-20mA User defined scales from -22mA to 22mA Standard scales 0/100mV; 0/1V; 0/5V; 1/5V; 0/10V; 2/10V; 2/10V; 0/50V 0/100V; 0/200V User defined scales from -110mV to 110mV; from 2V to 11V; from -200V to 220V PT100; PT1000 Ni100; Ni1000 J, K, R, S, T, E, B, N, W3, W5, NiMo from 100Ω to 100kΩ 0/200Ω; 0/1kΩ; 0/10kΩ; 2 or 3 wires, 24V - 29mA max OUTPUTS Output 1 Current Output 1 Voltage Output 2 Current Output 2 Voltage Digital Output Relay Output 0/20mA; 4/20mA; from 0 to 20mA 0/10V; +/-10V - from 0 to 10V 0/20mA; 4/20mA; from 0 to 20mA 0/10V; +/-10V - from 0 to 10V USB connector in front panel RS485 Modbus Jbus isolated from input and output 1 Relay: 1RT; 2RT; 3RT; 4T; 1RT & 1T As seen on the next figure the signal from the process is duplicated and wired on both PACs thanks to the JMConcept module. 50

51 3-Design Analog Input Analog Outputs implementation The implementation is handled by a low-level commutation interface, in our case a JMConcept GK3000D1 module. The principle is to select the output coming from the Primary PAC. This selection is performed by 2 relays controlled by a PAC digital output. The management of the relays can be performed either by 1 non-redundant output or by 2 redundant outputs to increase reliability. In our architecture, we choose to manage the relays with 2 redundant digital outputs. The following table describes the GK3000D1 interface relays logical operation: Relay A Relay B Output Channel Analog Input 1 Analog Input 2 last correct channel Analog Input 1 Analog Input 2 4/20 ma 4/20 ma INPUTS Digital Input 1 Digital Input 2 Analog Output Digital Output on optocoupler 30V max on optocoupler 30V max OUTPUTS 4/20 ma RS 485 isolated from input Modbus, Jbus Digital link allows programing of the module Digital link allows acquisition of the measurements 51

52 3-Design Analog Outputs controlled by non-redundant Digital Outputs The 2 PACs analog outputs are connected to a GK3000 D1 module. This module, controlled by digital signals, routes to its output, using 2 relays, one of its 2 inputs. In our case, the digital signals that control the GK3000D1 module are 2 digital outputs of the PACs. The main benefit of this solution is that it only uses 1 digital output to route analog outputs. A wiring example is presented in the diagram below: Analog Output Digital Output The Primary PAC must set to 1 the digital output that controls the relay in order to route its own analog output to the output channel of the commutation interface. The Standby PAC then sets to 0 the digital output that controls the other relay. Note: This logical operating principle must be coded in section 0. The fall-back mode of the digital output module must be set to 0. For example, according to the diagram above, if PAC_A is the Primary PAC, the digital output connected on the relay A is set to 1 while PAC_B digital output connected to relay B is set to 0. This leads to route the analog signal A on the output of the communication interface. Note: This solution is not used in our architecture. 52

53 3-Design Analog Outputs controlled by redundant Digital Outputs Analog Output Digital Output An analog output redundancy is performed, thanks to a GK3000D1 communication interface and redundant digital outputs, using 2 digital outputs per PAC. Each relay of the GK3000D1 interface is connected to a digital output. If PAC A is Primary PAC B is Primary Then Digital output number 0 is set to 1 (relay A) Digital output number 1 is set to 0 (relay B) Digital output number 0 is set to 0 (relay A) Digital output number 1 is set to 1 (relay B) Note: This logical operating principle must not be coded in the first section (section 0). For example, according to the diagram above, if PAC_A is the Primary PAC, the digital output No 0 (relay A) is set to 1 and the digital output No 1 (relay B) is set to 0. Thus, the analog signal ANA_ A is routed to the output of the communication interface. Note: This solution is used in our architecture. 53

54 3-Design 3.4. Quantum Hot-Standby System This chapter describes the different features and specifications of a redundant Quantum PAC system Quantum PAC Specifications Primary and Standby PACs Primary PAC runs the whole application program including the first section. It handles remote I/Os and updates the redundant PAC after each program cycle. If the Primary PAC stops, the Standby PAC takes over the control from the Primary PAC in one cycle. Standby PAC only runs the first section of the application program, checks the CPU and CRP modules availability and does not handle remote I/Os. Note: CRP modules are needed in a Quantum Hot-Standby configuration, even if the system does not use remote I/Os. In this case, the two CRP modules are linked for monitoring purposes. Local I/Os can be configured and used in a Hot-Standby Quantum system. However, they are not saved on the Standby system. They can be written in the Standby system with different values using the first section of the program (section 0). Moreover, the output modules management can only be performed using variables that are not transferred from Primary to Standby PAC. Hardware Constraints Non-compatible modules: The following modules can not be used in a Quantum Hot-Standby configuration: 140N WM N OC Note: Modules 140NOE77100 and 140NOE77110 are not available anymore. Local I/Os: The term local I/Os refers to the I/O modules located in the local rack, which do not take part in the redundancy system. When no programming has been specified, the Standby CPU outputs state is imposed by the Primary CPU. Standby CPU outputs can be programmed in the section 0 (which is the only section executed in the Standby CPU) and they take the specified status unless they are forced in the Primary CPU. 54

55 3-Design Standby CPU inputs status (%l readable in the application and from the animation table) does not reflect the hardware Standby CPU inputs status, but instead is related to the Primary CPU inputs status. The Standby module byte status indicates: - the Primary module status in the case of a mixed module - the Standby module status in the case of an output module (DDO) It is important to manage the locals I/Os in the section 0 only by using the %MW words of the non transfer area. Remote I/Os: Communication modules cannot be used in a Remote I/O rack within a Hot-Standby application. Software Constraints We recommend not using TIMER events as they are not synchronized in Quantum Hot-Standby applications Quantum Hot-Standby DFBs Library The following table summarizes the different DFBs created for the Quantum application. DFB FUNCTION ETHERNET NOE_MONITOR Monitoring NOE Ethernet Module PROFIBUS PTQ_MONITOR Monitoring PTQ Ethernet Module SYNTH_FAULT Synthesis Fault monitored elements SYNTH_OR_NOE Synthesis Fault NOE module (Logic OR) SYNTHESIS SYNTH_AND_NOE Synthesis Fault NOE module (Logic AND) SYNTH_OR_PTQ Synthesis Fault PTQ module (Logic OR) SYNTH_AND_PTQ Synthesis Fault PTQ module (Logic AND) SWITCHOVER SWITCH_MANG Switchover Managment 55

56 3-Design Ethernet link monitoring DFBs NOE_Monitor NOE_Monitor BOOL MSTR_ACTIVE MSTR_ACTIVE BOOL BOOL MSTR_DONE MSTR_DONE BOOL BOOL MSTR_ERROR MSTR_ERROR BOOL INT RateEt RateEt INT INT Error_Count Error_Count INT ARRAY[0..9]OF INT MSTR_Control MSTR_Control MSTR_DataBuf BYTE Slot Data_TCP DIAG_TCP INT MonitoringRate LED_APPL BOOL INT Retries LED_LINK BOOL BOOL Pulse LED_RUN BOOL NOE_Failure BOOL CFG_PORT BOOL ETH_100M BOOL OPTICFIB BOOL FULL_DUP BOOL FAULT BOOL EQUP_TYP UINT ARRAY[1..9]OF INT ARRAY[1..37]OF WORD The NOE_Monitor DFB monitors the Ethernet link hosted by the NOE module. The monitoring is managed by the MBP_MSTR block function integrated in the NOE_Monitor DFB. The MBP_MSTR block allows performing operations on communication networks, for example, the extraction of the local statistics of the NOE module. IP_AD_1 IP_AD_2 IP_AD_3 IP_AD_4 MAC_ADD_1 MAC_ADD_2 MAC_ADD_3 MAC_ADD_4 MAC_ADD_5 MAC_ADD_6 INT INT INT INT WORD WORD WORD WORD WORD WORD MBP_MSTR ENABLE ACTIVE ABORT ERROR SUCCESS CONTROL DATABUF This extraction rate is controlled by the Monitor Rate parameter configured by the user. Extracted data is available on output pins as shown on the diagram on the left. 56

57 3-Design NOE Local statistics In order to diagnose the health status of the NOE module, we check the status of the module and the link thanks to respectively Led RUN and Led Link. The number of faults is given by the MPB_MSTR. This number is compared to the retries value, if the number of faults is greater or equal than the retries value the NOE_Failure is set to 1. 57

58 3-Design The block implementation is associated to a data structure NOE_Monit. NOE_Monit Struct + MSTR_Active BOOL MSTR_Done BOOL MSTR_Error BOOL MSTR_RateEt INT MSTR_Error_Count INT Failure BOOL Fault BOOL Led_Appl BOOL Led_Cfg_Port BOOL Led_Eth_100M BOOL Led_Full_Dup BOOL Led_Link BOOL Led_Ooptic_Fib BOOL Led_Run BOOL EQUP_TYP BOOL MAC_ADD_1 WORD MAC_ADD_2 WORD MAC_ADD_3 WORD MAC_ADD_4 WORD MAC_ADD_5 WORD MAC_ADD_6 WORD IP_AD_1 INT IP_AD_2 INT IP_AD_3 INT IP_AD_4 INT EQUIP_TYPE UINT + MSTR_Control ARRAY[1..9] OF INT + MSTR_DataBuf ARRAY[0..37] OF WORD + MSTR_Data Diag_TCP During the implementation the block is used as many times as the number of NOE modules. 58

59 3-Design Profibus link monitoring DFBs PTQ_Monitor PTQ_Monitor Fault Timeout TIME Fault_Timeout PTQ_FAULT BOOL General PTQ fault Master PTQ Status WORD Master_Status Faulty_Active BOOL Active PTQ fault Master PTQ Operating WORD Master_Operating_State Active PTQ Status BYTE Active_Status Faulty_Passive BOOL Passive PTQ fault Nb slaves seen by active PTQ BYTE Active_NbSlave Passive PTQ Status BYTE Passive_Status Faulty_Passive_UDP BOOL Passive PTQ fault via UDP Nb slaves seen by passive PTQ BYTE Passive_NbSlave Passive PTQ Status via UDP BYTE PassiveP_Status_UDP Nb slaves seen by passive PTQ via UDP BYTE Passive_NbSlave_UDP A Quantum Hot-Standby system does not handle automatically the PTQ-PDPMV1 module redundancy. However, this module provides enough information about the health status of the Active and Passive Master (Active = Primary and Passive = Standby) to be able to develop a DFB to manage redundancy. The health status information circulates through the Profibus network. That means that, if the Profibus link is lost, the modules are not able to communicate their health status. Therefore, we use the Ethernet UDP capability ( PTQ Link Message ) to set up the PTQ module redundancy function. An Ethernet crossover cable is needed to link the two modules. Thus, the Primary module can access the health status of the Standby module even if the communication link is lost. Note: The link between the 2 PTQ modules can also be established using an Ethernet switch. In that case, the user can transfer the module configuration without unplugging the cable. This solution is also used when the distance between the 2 PACs is too important. 59

60 3-Design Data used by the PTQ_Monitor DFB 1. Data circulating through the Profibus network ProfibusCRC32: Profibus Master configuration This word describes the configuration of the Profibus parameters and of the slave devices. PTQModuleCRC32: PTQ-DPPMV1 configuration. This word describes the configuration of the PTQ module (mapping, and so on). Profibus Master Operating State This word describes the status of the master module. 0x0000 0x4000 0x8000 0xC000 Offline Stop Clear Operate ProfibusMasterModuleStatus: Profibus master module s operating status Bit 2 Application Status Bit 8 Data exchange Bit 9 Slave input frozen/cleared Bit 12 Reset 0 - Application Stopped 1 - Application Running 0 - There is no data exchange with any of the assigned slaves 1 - There is Data Exchange with at least one of the assigned slaves 0 - A slaves inputs in the IN area are cleared in a slave is not in Data Exchange 1 - A slave's inputs in the IN area are frozen if a slave is not in Data Exchange 0 - No action 1 - A reset is requested by the PROFIBUS Master module because a new database has been downloaded 60

61 3-Design HSBY Passive Status (Byte) from Profibus interface Bit 0 = PA This bit indicates the state of the local master. Bit 1 = SO This bit indicates if the local master recognizes any of its assigned slaves as "offline". Bit 2 = CE This bit indicates if the local master has recognized an exception response. Bit 3 = DB This bit indicates if the local master has detected a database mismatch. Bit 4 = OD This bit indicates when the data in the output data area of the DPRAM is updated after a switch over. 0 - Active master. Master is controlled by the Primary PAC 1 - Passive master. Master controlled by the Standby PAC 0 - At least one slave is "offline" 1 - All slaves OK 0 - No exception response active 1 - At least one exception response active 0 - Database OK 1 - Database mismatch 0 - Output data is not updated 1 - Output data is updated (Once this bit is set, it remains set for the remaining session until the any bus is either reset or HSBY state changes to "Not Connected") Bit 5 6 = not used Bit 7 = COM This bit indicates if the counterpart is present 0 - Counterpart not present 1 - Counterpart is present HSBY Passive number of slaves (Byte) - from PROFIBUS interface Slave number seen by the Passive module HSBY Active Status (Byte) - from PROFIBUS interface Bit 7 = HS This bit indicates that the Hotstandby functionality is enabled. 0 HSBY disabled. Module operates as "stand alone" master or HSBY state equals "Not connected". 1 HSBY enabled Bit 0 to 6 are identical as for HSBY Passive Status (as seen above) 61

62 3-Design 2. Data circulating through UDP mailer The data available via UDP mailer only apply to the Passive module. It is identical to the data found on Profibus network: HSBY Passive Status UDP - from UDP HSBY Server HSBY Passive number of slaves UDP - from UDP HSBY Server HSBY Passive PROFIBUS CRC32 UDP - from UDP HSBY Server HSBY Passive User Configuration CRC32 UDP - from UDP HSBY Server 3. Operation The Active PTQ module is declared non-operational in the following cases: exception response application discrepancy inactive Hot-Standby system module in Run state loss of communication with the slave devices Master Operating State equals 0xC000=Operate The Passive PTQ module is declared non-operationel in the following cases: Exception response (Profibus and UDP) and Passive_number_of_slaves_UDP equals 0 missing Counter part (Profibus and UDP) and Passive_number_of_slaves_UDP equals 0 module in Active mode and Passive_number_of_slaves_UDP equals 0 The Ethernet link which supports the UDP service is also monitored. If the status word PTQ_Passive_UDP equals 0, we consider the link as non-operational In short, our DFB asks for a switchover when the Active module is non-operational and the Passive one is operating normally. While implementing the Hot-Standby system, this DFB is used as many times as there are PTQ modules. 62

63 3-Design Switchover Management Defaults Synthesis SYNTH_FAULT Synthesis Fault NOE Module BOOL Faulty_NOE Synthesis Fault PTQ module BOOL Faulty_PTQ Synthesis Fault Scada BOOL Faulty_SCADA Fault Mask word WORD Fault_Mask Fault_Synth INT Synthesis Fault Word Fault BOOL OS Versions Mismatch This block aims at processing the faults that would lead to a switchover. We find in inputs the results of the NOE and PTQ modules failure detection. Faulty_SCADA is an input pin in the case of the communication between the SCADA and the PAC is monitored. This DFB also processes: Battery events %S67 = application memory card battery %S68 = processor battery %S75 = data storage memory card battery CPU non-operating %S12 = CPU running General In-Rack I/O non-operating %S119 = event of one or several I/O modules in the rack Slots 3 to 10 non-operating %SW180 = operating status of Quantum modules installed on station 1 The faults processing is performed using the mask value set on the input pin Fault_Mask. This mask allows to select which event to take into account according to the configuration and the user s settings. 63

64 3-Design Each exception corresponds to one bit of the Fault_Synthesis word: BIT Element monitored Bit 0 Battery Exception Bit 1 CPU Exception Bit 2 General In-Rack I/O Exception Bit 3 Exception on Slot 3 Bit 4 Exception on Slot 4 Bit 5 Exception on Slot 5 Bit 6 Exception on Slot 6 Bit 7 Exception on Slot 7 Bit 8 Exception on Slot 8 Bit 9 Exception on Slot 9 Bit 10 Exception on Slot 10 Bit 11 Ethernet Adapter(s) NOE Exception Bit 12 PROFIBUS Adapter(s) PTQ Exception Bit 13 SCADA Exception The result of this synthesis is saved in a word and set as an output on the Fault_Synth_Plc pin. If there is at least one exception response, the output pin Fault is set to 1. During the implementation of the system, this block is used twice: one for the Primary PAC and one for the Standby PAC. In order to be able to compute the status of several NOE or PTQ modules, logical OR and AND processing DFBs have been created: SYNTH_AND_NOE BOOL FLT_NOE_1 FAULT_NOE BOOL BOOL FLT_NOE_2 BOOL FLT_NOE_3 BOOL FLT_NOE_4 BOOL FLT_NOE_5 BOOL FLT_NOE_6 SYNTH_AND_PTQ BOOL FLT_PTQ_1 FAULT_NOE BOOL BOOL FLT_PTQ_2 BOOL FLT_PTQ_3 BOOL FLT_PTQ_4 BOOL FLT_PTQ_5 BOOL FLT_PTQ_6 SYNTH_OR_NOE BOOL FLT_NOE_1 FAULT_NOE BOOL BOOL FLT_NOE_2 BOOL FLT_NOE_3 BOOL FLT_NOE_4 BOOL FLT_NOE_5 BOOL FLT_NOE_6 SYNTH_OR_PTQ BOOL FLT_PTQ_1 FAULT_NOE BOOL BOOL FLT_PTQ_2 BOOL FLT_PTQ_3 BOOL FLT_PTQ_4 BOOL FLT_PTQ_5 BOOL FLT_PTQ_6 64

65 3-Design Switch Management SWITCH_MANAG Synthesis Fault word Primary INT PRIM_DIAG Synthesis Fault word Standby INT STBY_DIAG Switchover Number Reset BOOL SWITCH_NB_Reset SWITCH_NB UNIT Switchover request Manual Switchover BOOL FORCE FORCE BOOL Manual Switchover The Switch_Manag DFB manages and counts switchover queries. The switchover approval is computed from the Primary and Standby PACs diagnosis coming from the Fault_Synthesis DFBs as seen above. A switchover is allowed if: The Standby PAC diagnosis is OK. More than 30s elapsed since last switchover. Note: The time delay before the switchover takes place can be adjusted using variables of the DFB (Delay_Time_Before_Switchover). This delay is set to 1s by default. The switchover counter can be reset using the input pin Switch_N_Reset. For maintenance reasons, the input pin FORCE allows a manual switchover of the system. During the implementation of a Quantum Hot-Standby system, this block is used only once. Remote I/Os The use of Remote I/Os in a Hot-Standby system allows to work with redundant I/Os. It is important to configure the drop hold up time according to the cycle time and to the application. This parameter is the time during which I/O values are maintained while a switchover occurs. The Remote I/O stations are monitored using the following system words %SW535 This word stores the start-up error code. This word is always set to 0 when the system is running; in the event of error, the PLC does not start up, but generates a stop status code. %SW536 to %SW538 Communication error words on cable A. %SW539 to %SW541 Communication error words on cable B 65

66 3-Design %SW542 to %SW544 %SW545 to %SW640 These words are the global communication error words. Dedicated to the global station. That means these words can refer to Primary PAC as well as Standby PAC. These words are used to describe the status of the decentralized stations. Three status words are used for each station. Our Remote I/Os are configured on the Drop2. We therefore use the following system words: %SW545.0 to 7 = retry totalizer counter %SW545.8 to 11 = lost communications counter %SW548: displays the global communication status for station 2 %SW = 1, communication on cable B operating correctly %SW = 1, communication on cable A operating correctly %SW = 1, communication operating correctly %SW549: global event totals for cable A station 2 %SW550: global event totals for cable B station 2 most significant bit: counts the errors detected least significant bit: counts "non responses". most significant bit: counts the errors detected least significant bit: counts "non responses". 66

67 3-Design Switchover Time Switch_Over_Time Remote Pac is Primary BOOL Remote_is_Primary Sw_Timer TIME Switchover Time This Pac is Primary BOOL This_is_Prima The time gap during the switchover is a very important feature of the Hot-Standby system. A DFB has been defined to measure this time. The principle is based on the measurement of the time when the Primary PAC loses its Primary status and when the Standby turns Primary. This block, placed in the section 0, processes the system word %SW61 information and uses the ITCNTRL block function which allows event time measurements. The accuracy of the switchover time depends on the PAC scan time, for more accuracy, other measurement can be performed as described in the performance chapter. 67

68 3-Design 68

69 4-Configuration 4. Configuration 4.1. SCADA The configuration of the different components of our high-availability system is described in this chapter. In the first part, the configuration of the redundant SCADA system, using MODNET and OFS communication protocols, is detailed We next describe the set-up of the Premium and Quantum Hot-Standby CPU and corresponding modules. The Ethernet configuration is also addressed, in particular, the configuration of the manageable Ethernet switches, main component of the Control and Field Network. The case of the Profibus network managed by a PTQ module in a Quantum Hot- Standby system will also be described. 69

70 4-Configuration Servers Configuration We use Citect Project Editor to perform the following different server configuration steps: Creation of a cluster Servers Mapping Creation of the I/O and ATR servers Clusters From the Servers menu, Click on Clusters and create a cluster called HighAv Servers As described in the Design chapter, we use a Primary and a Standby server. Therefore, still from the Servers menu, click on Network Addresses and create 2 servers with the following parameters: Server1 (Primary Server): IP Address: SN Mask:

71 4-Configuration Server2 (Standby Server): IP Address: SN Mask: Alarm / Trend / Report Servers Once server1 and server2 are created, we can create Primary and Standby ATR servers. Each ATR server is related to a cluster, a network address (Server1 or Server2) and an operational mode (Primary or Standby). From the Servers menu, select Alarm Servers. Alarm1 (Primary Server): 71

72 4-Configuration Alarm2 (Standby Server): Proceed the same way for the Reports and Trends Servers (Report1, Report2, Trend1 and Trend2). I/O Servers Like ATR servers, I/O Servers also are related to a Cluster and a network address (Server1 or Server2). The operational mode (Primary or Standby) is linked to the machine mode (Server1 or Server2). From the Servers menu, select I/O Servers. IOServer1 (Primary I/O Server) IOServer2 (Standby I/O Server) 72

73 4-Configuration Communication Configuration Communication configuration consists of Boards, Ports and I/O Devices set up. The above diagram illustrates the communication principle between the SCADA system and an I/O Devices. Cluster: HiAv I/O Server #1 IOServer1 Eth Addr #1 Server BOARD BOARD1 TCPIP Address 0 I/O Server #2 IOServer2 Eth Addr #1 Server BOARD BOARD1 TCPIP Address 0 TCP/IP PORT #1 PORT1_BOARD1 -I P0 -T TCP/IP PORT #1 PORT1_BOARD1 -I P0 -T Eth module #1 Eth module #1 Primary Priority 1 Primary Priority 2 I/O Device PAC In order to make this set up easier, you can use the Express Wizard accessed from the Communication menu. 73

74 4-Configuration Express Communications Wizard This tool helps to configure the I/O Devices that communicate with the SCADA system. For a redundant SCADA system, a single I/O Device is configured twice: Once for each server (Primary and Standby). Step Action 1 Start the Express Wizard and click on Next 2 Select IOServer1 and click on Next Create the I/O Device PAC 3 Select External I/O Device 4 74

75 4-Configuration If using Modnet, apply step 5 and 6. If using OFS, go to step 7. This step consists of selecting the I/O Device reference and the communication protocol. Here is the configuration for Modnet: If using a Premium PAC, select Modbus/TCP(Ethernet) SpeedLink Capable from the Premium list and click on Next. 5 If using a Quantum PAC, select Modbus/TCP(Ethernet) SpeedLink Capable from the Quantum list and click on Next. Note: Modbus/TCP SpeedLink is chosen instead of Modbus/TCP because it supports the importing of PAC variables directly from the program file.stu 75

76 4-Configuration Configure the IP address of your I/O Device. The diagram below shows the IP address of the Quantum Hot-Standby system 6 Click on Next and go to step 9. If using OPC: The software OPC Factory Server (OFS) acts as a gateway between I/O Devices and the SCADA system. Select OPC from the OPC Factory Server list and click on Next 7 76

77 4-Configuration Configure the OPC server address. You can use the default address: Schneider-Aut.OFS. 8 Click on Next. The step setting up the link between the I/O Device and an external database will be detailed in the Implementation Chapter. Click on Next. 77

78 4-Configuration The communication configuration between the Server1 (Primary) and the I/O Device is completed. Click on Finish. 9 To configure the communication between the Server2 (Standby) and the I/O Device, follow the same steps using IOServer2 during step 2. The I/O Device name is the same (PAC) as in the previous configuration. We will now check the components (Boards, Ports, I/O Devices) that have been configured by the Express Communications Wizard. The following dialogs are accessed from the Communication Menu 78

79 4-Configuration Boards Configuration The component Board is used to declare the communication type (TCPIP, OPC and so on) used by the network components of the machine-server. Concerning Modnet protocol, the wizard has created a board named BOARD1, using TCPIP, at address 0, on IOServer1 and IOServer2: Concerning OPC protocol, the wizard has created a board named BOARD1, using OPC, at address 0, on IOServer1 and IOServer2: Ports Configuration The component Port represents the link between the SCADA system and the I/O Device. The wizard has configured a first port on the IOServer1 named PORT1_BOARD1 which represents the link between the I/O Device IP (the Quantum PAC in our case) and IOServer1, and a second one associated with IOServer2. 79

80 4-Configuration I/O Devices Configuration In the I/O Devices configuration window, we find the device name, its number, the port to which it is associated and its communication protocol. The Startup Mode and Priority fields (press F2 key to get the complete window) are blank. In an Hot-Standby architecture, these fields have to be configured in order to define the Primary and Standby I/O Devices as well as their priority. For our architecture, we have configured the I/O Device related to IOServer1 in Primary mode with a Priority set to 1. The I/O Device related to IOServer2 has been set in Standby mode with a Priority of 2 I/O Device MODNET Configuration 80

81 4-Configuration I/O Device OPC Configuration 81

82 4-Configuration 4.2. Control and Field Network The aim of this chapter is to describe the configuration of the switches of the Control Network and Field Network using MRP (Media Redundancy Protocol) as redundancy management protocol. The MRP principle is to have one switch of the ring defined as the Redundancy Manager (Media Redundancy Manager). The Redundancy Manager handles the response of the non-operational ring devices or network segments. 82

83 4-Configuration Switch Configuration The configuration of a switch is done via its embedded web server (ConneXium TCESSM Web Server) accessed by typing its IP address in the address bar of an Internet browser. The IP address of the switch has been set using the Ethernet Switch software provided with the switch. The following Login and Password are required to log in to the web server: Login: admin Password: private 83

84 4-Configuration Once logged in, the system page opens, presenting the visual aspect of the switch and its name. The different configuration tools of the server are accessible via the menu on the left. The configuration of the Control Network switches is detailed through the following screenshots. In our configuration, switch #2 is set as the Redundancy Manager. From the Redundancy menu, click on Ring Redundancy to access the MRP configuration of the switch. 84

85 4-Configuration Note: To avoid loops during the switch configuration, do not connect the redundant path until you have completed the Ring Redundancy configuration. So it is important to unplug cable from port 2 and connect the computer on the port 3. Also, set the dipswitches on Ethernet switch front panel, labeled RM and Stand, to the ON (rightmost) position to enable software ring configuration (via Web Interface). The following table describes each step of the switch #2 configuration: Step Action Select the type of redundancy protocol - HIPER-Ring / MRP 1 Select the MRP radio button. Selection of Ring Ports Enter Port numbers corresponding to the ports assigned to the ring connection, namely 1 and 2, respectively in Ring Port 1 and 2 areas 2 Note: When the ring is operational, the Port Status is displayed. At this stage no information is presented. Port status values include the following: forwarding: port is switched on and hosts a link. inactive: port is blocked and hosts a link active : port is operational disabled: port is blocked switched off not-connected: port has no link Enable Redundancy Manager 3 Select the On radio button in Redundancy Manager area. 85

86 4-Configuration Validate Advanced Mode for fast switching time 4 Click on Advanced Mode check box in Configuration Redundancy Manager area Switch on operation Validate the On radio button in Operation area to allow the validation of the modifications. 5 Validate 200 ms Ring Recovery Ring Recovery group box presents 2 selections: Standard Recovery (500 ms) or Accelerated Recovery (200 ms) for the switch activated as the Redundancy Manager. 6 Select the accelerated recovery 200 ms radio button. Disable VLAN Assignments on ring ports 7 Assuming no VLAN is required, set VLAN ID 0 in VLAN area 8 Validation of the configuration Click on Set button for configuration changes 86

87 4-Configuration Configuration saving The modified configuration is only present in the switch #2 dynamic memory. To preserve these changes in the event of a power cycle, the configuration must be saved: 9 Click on menu entry Basic Settings, then on Load/Save entry Select to Device radio button in Save area. Then click on the Save button. The configuration of switch # 2 is now completed. To configure the other switches, the procedure is the same as above except that the Redundancy Manager parameter must be set to Off and the Advanced Mode must be de-selected. A summary of the configuration is shown in the screenshot below: 87

88 4-Configuration 4.3. Premium Hot-Standby PAC Station This chapter describes the configuration from Unity Pro of the key parameters of Premium Hot-Standby system. The configuration of an Ethernet ring dedicated to field devices is also illustrated Architecture The figure below illustrates the redundant PAC architecture composed of Premium Hot-Standby PACs. This paragraph describes: how to configure a Premium Hot-Standby system how to implement the management of digital and analogic I/Os how to manage 2 Ethernet rings using Ethernet ETY 5103 modules Redundant Premium Architecture PAC A IP: MASK: PAC B IP: MASK: Sync-link IP: MASK: IP: MASK: JM Concept Modules Analog Output Analog Input ABE7 Connection Blocks ABE7 ABE7 Digital Output Digital Input IP: IP: SW10 Manager SW12 SW11 IP: IP: IP: IP: IP:

89 4-Configuration Hardware Configuration The hardware setup used in this guide is illustrated below. It is composed of a Hot- Standby CPU, digital and analog I/Os modules, and 2 Ethernet TSX ETY 5103 communication modules for the management of the control and field networks. At least one Ethernet ETY module is needed to allow Hot-Standby capability CPU Configuration In Unity Pro, concerning the CPU, from the Configuration tab, the address range used for the application is defined (State of global address fields). 89

90 4-Configuration Hot-Standby Configuration Also from the CPU, in the Hot-Standby tab, we set the Hot-Standby runtime parameters. Monitored ETY module As seen in the Design Chapter, at least one monitored ETY module is required in a Hot-Standby system. In the Topological address of the monitored Ethernet module, choose a ETY module that will be declared as Monitored. Select Rack Slot

91 4-Configuration Logic Mismatch This parameter defines the PAC mode if a program mismatch is detected between the Primary and the Standby. Select Offline Non-Transfer Area The non-transfer area cannot be defined by the user on a Premium Hot-Standby system. An area of 101 word is set by default (%MW0-%MW100). The words located in this area are not transferred to the Standby PAC Ethernet Modules TSX ETY 5103 This part describes the ETY modules configuration to set up the communication with the SCADA system and the Ethernet devices. Controller Ethernet Network Only the IP configuration needs to be configured in order to set up the communication between the SCADA system and the ETY module. No specific service is required. IP Configuration IP address: Subnetwork mask: The IP configuration is summarized on the screenshot on the next page. 91

92 4-Configuration Devices Ethernet Network This module is dedicated to communicate with I/O Devices such as ATV71, TesysT, Advantys STB, Quantum, M340 and so on. IO Scanning service is used to communicate between the controller and the devices. The ETY module communicates with the I/O Devices via an Ethernet ring managed by MRP. The configuration of the ring is the same as the one described for the Control Network (see Chapter 4.2) 92

93 4-Configuration IP Configuration IP address: Subnetwork mask: Module Utilities The I/O Scanning service drives the communication between the module and the I/O devices. From Module Utilities area, select Yes for IO Scanning. IO Scanning Set up of the I/O Scanning lines associated to I/O Devices: - Switch SW10: ATV71: STB: ETG100: Switch SW11: ATV71: STB: Switch SW12: Quantum: M340: The I/O Scanning configuration is presented on the screenshot on the next page. 93

94 4-Configuration Redundant Digital I/O Redundant digital I/Os implementation is performed, as described in the Design chapter, thanks to Telefast connection equipments (ABE7 ACC11 for inputs, ABE7 ACC10 for outputs). Digital I/Os management does not require any specific configuration. Nevertheless, it is important to choose the proper output fallback mode according to the process. For this STG, it has been decided to maintain the outputs state before the default. 94

95 4-Configuration Redundant Analog I/O Redundant analog I/Os implementation is performed, as described in the Design chapter, thanks to JM Concept connection equipments This paragraph describes the analog I/Os and JM Concept modules configuration in order to set up redundant analog inputs and outputs. Analog Input The measurement input is a voltage signal +/-10V. The original signal is duplicated and converted in 0-10V by a JMC module and these 2 signals are connected to both PACs (Primary and Standby) via TSXAEY414 modules. TELIS 9000U2 (JMConcept) Configuration. Set the configuration switches according to the input signal type: Voltage 0-10V. INPUT SWITCH Input 1 - Current Input 2 - Current Voltage input < 10V Thermocouple Voltage input > 10V PT100- PT Ni100 - Ni1000 Sensor Power supply Resistance Potentiometer On the module, from the menu, and using the user guide set the input signal type, then the output signal type. ANALOG Input Mode : 1 channel Type : Voltage Scale : +/-10v ANALOG Output Scale : 0-10V Minimum Display : 0 Maximum Display : Decimal Point : «00000» Minimum scale : 0 Maximum scale : Resolution : 1 Pt 95

96 4-Configuration TSX AEY414 Configuration In Unity Pro, from the TSXAEY414 module configuration tab, set a voltage input 0-10V. select a 0..10V range on the channel 0 select a scale value of Analog Output The output signal generated by the PAC via a TSXASY410 module is a 4..20mA current signal. This type of signal is used because it is the only one accepted by the GK 3000 D1 communication interface for inputs. The signal is sent for outputs on the channel A or B according to the communication relays. The set up of the commutation relays is detailed in the Implementation chapter. 96

97 4-Configuration TSX ASY410 Configuration In Unity Pro, from the TSXASY410 module configuration tab, set an ma output current. Select a 4..20mA range on the channel 0 The signal has to be maintained on a default so, de-select Fallback The scale value is locked to GK 3000 D1 (JMConcept) Configuration. On the module, from the menu, only the high and low limits of the input signal can be defined. 97

98 4-Configuration 4.4. Quantum Hot-Standby PAC Station Architecture This chapter describes the configuration from Unity Pro of the key parameters of Quantum Hot-Standby system. The configuration of an Ethernet ring dedicated to field devices and Profibus network is also illustrated. As seen in the previous chapters, our architecture comprises a Quantum Hot-Standby system linked to: 2 Ethernet rings via NOE modules a Profibus network via a PTQ module a Remote I/O station 98

99 4-Configuration Hardware Configuration The hardware setup used in this guide is illustrated below. It is composed of a Hot- Standby CPU, 2 Ethernet modules 140 NOE for the management of the control and field networks. A 140 PTQ PDP MV1 module handles the Profibus network. The control of the Remote I/O station is performed by RIO Drop Head 140 CRP and RIO Drop End communicator 140 CRA modules CPU Configuration In Unity Pro, from the Configuration tab, we define the address range used for the application (State RAM). For a Hot-Standby application, it is recommended to check the Online modification in RUN option. This allows, while staying online, to add or to delete discrete or analog modules, and parameters modification. Note: This option is supported only with a firmware version v2.0 ir12 or earlier for the CRP module and with a firmware version v2.0 ir6 or earlier for the CRA module. 99

100 4-Configuration The following screenshot sums up the CPU configuration: Hot-Standby Configuration In the Hot-Standby tab of the CPU configuration, we set the Hot-Standby runtime parameters. 100

101 4-Configuration CPUs Run Mode In the Run Mode area, we define which PAC will be the Primary at the system power up. If the 2 PACs are declared Online", the PAC with the lower MAC address takes the Primary role. Controller A, select Online Controller B, select Online Logic Mismatch This parameter defines the PAC mode if a program mismatch is detected between the Primary and the Standby. Select Offline Keypad The Invalidate Keypad parameter allows inhibiting keypad commands sent from the Hot-Standby menu. Do not select the option Swap Address This parameter allows CPU memory swapping in case of a switchover. 101

102 4-Configuration Non-Transfer Area This area is defined by the user. Words located in this area will not be transferred to the Standby PAC. This area is used for specific operations performed by the Primary and must not impact the Standby. For our application, we set a 2000 words zone from %MW100. HSBY Configuration Options on RIO bus The implementation of a Quantum Hot-Standby system implies the use of CRP modules. Therefore, if I/O devices over Ethernet are used, the No RIO drop option has to be selected. Once a RIO drop is declared, the No RIO drop is automatically grayed 102

103 4-Configuration RIO Configuration It is important to set up properly the Drop hold up time parameter. This is the maximum time the RIO bus can be offline before triggering an exception event (1200 ms by default). In configuration part, right click on Remote IO Quantum Drop and select Open. 103

104 4-Configuration Ethernet Modules 140 NOE This part describes the NOE modules configuration to set up the communication with the SCADA system and the Ethernet devices. Controller Ethernet Network Only the IP configuration has to be set in order to set up the communication between the SCADA system and the NOE module. No specific service is required. IP Configuration IP address: Subnetwork mask:

105 4-Configuration Devices Ethernet Network This module is dedicated to communicate with I/O devices such as ATV71, TesysT, Advantys STB, Quantum, M340 and so on. The I/O scanning service is used for I/O device communication. The NOE module communicates with the I/O devices via an Ethernet ring managed with MRP. The configuration of the ring is the same as the one described for the control network (see chapter 4.2) IP Configuration IP address: Subnetwork mask: Module Utilities The I/O Scanning service drives the communication between the module and the I/O devices. From Module Utilities area, select Yes for IO Scanning. 105

106 4-Configuration IO Scanning Set up of the I/O Scanning lines associated with I/O devices: - Switch SW10: ATV71: STB: ETG100: Switch SW11: ATV71: STB: Switch SW12: Quantum: M340: A summary of the I/O Scanning configuration is presented on the following screenshot: 106

107 4-Configuration Profibus Module 140 PTQ-PDPMV1 PTQ-PDPMV1 is a Profibus DP communication module. Its configuration is made through the Prosoft configuration software and transferred into the module via a serial or Ethernet link. This software is used to configure the Profibus master/slaves and also to map the I/O variables. It also allows importing variables and structures associated with the project. Prosoft Configuration Builder (PCB) Creation of a new project: from PCB, Click on File and Select New right Click on Default Module and select Choose Module Type in Product Line Filter area, select PTQ choose module type PTQ-PDPMV1 select Enable Hot-Standby click on OK 107

108 4-Configuration PTQ Profibus Master DPV1 configuration From PTQ module created, in the tree list, right click on PTQ Profibus Master DPV1 and select Configure Type 7 in the text zone Slot Number. This value corresponds to the slot number in which the PTQ module is located on the Quantum rack. Mapping of Unity Pro I/O variables: We fill in the fields Output Start Register and Input Start Register with the address values used in Unity Pro: Intput Start Register: 1025 (%IW) (size: 768) Output Start Register: 4097(%MW) (size: 768) Ethernet port speed is set to 100MB/full-duplex. Note: In our Hot-Standby configuration, we connect the 2 PTQ modules with an Ethernet cross cable. This link allows network and modules monitoring via a UDP mailer. 108

109 4-Configuration In the case of the PTQ modules linked using an Ethernet switch, the Duplex/Speed Code parameter must be set to Auto-negotiate. In the case of the PTQ modules linked using an Ethernet crossover cable, the Duplex/Speed Code parameter must be set to 100Mb/full-duplex. 109

110 4-Configuration Profibus DP configuration The following table describes the configuration of the Baud Rate and the Host Delay time of the bus. Step Action From Profibus DP, right click on Configure. 1 Click on Configure Profibus Button 2 From the Bus Configuration Windows, Right Click on the picture Master HSBY and select Object properties 3 110

111 4-Configuration From the tab PROFIBUS, after having configured the master address, configure the Baud Rate, Profile and Host Delay Time. - Baud Rate: 1500kBit/sec (en fonction de la technologie des esclaves configurés sur bus) - Profile: User defined - Host Delay Time: 300ms 4 Click on OK 5 The setup is now complete Variables and Variable Structures export From the PDPMV1 Profibus Master Setup window, the mapping of the variables can be displayed and export files of these variables can be created. This file has to be imported in Unity Pro later. Step Action In the Processor Network Memory Map area, Click on Show Unity Map

112 4-Configuration From Unity Memory Map Window, click on Export Processor Files, and save the.xsy file in the directory of your choice. 2 3 Close the windows Unity Memory Map and PDPMV1 Profibus Master Setup. Loading of the Profibus configuration into the PTQ module The serial port of the module is used to perform the transfer, the procedure is detailed in the table below: Step Action Right Click on PTQ-PDPMV1-HSBY and select Download From PC to Device

113 4-Configuration 2 Connect the serial cross cable on the PC and the PTQ module. 3 Select the connection type (Ethernet or Serial). The transfer is performed only if the Quantum CPU is in Stop mode. Therefore, set the PAC in Stop mode using the keyboard on the CPU and click on DOWNLOAD. 4 Once the transfer is complete in the first module, a message box prompts you to connect the cable on the second module. Click on OK when the cable is properly plugged. Note: Set back the configured PAC in Run Mode and set the PAC to be configured in Stop mode (See step 4)

114 4-Configuration The transfer is now complete. Click on OK. Save and Close the PCB application. 6 You can now import in Unity Pro the variables file (.xsy) you created with PCB. 4.5 Advantys STB Ethernet I/Os It is necessary to set up the digital outputs in a redundant system when using Ethernet I/Os. During a switchover, outputs must maintain their current state. The Holdup Time parameter is the time during which the outputs state is not modified during a switchover. If, after this time, the outputs did not receive any commands, they assume the Fallback mode status previously defined by the user. In our case, the Holdup Time is set to 5000ms (the setting can be made via the NIP module webserver). 114

115 5-Implementation 5. Implementation 5.1. Premium PAC This chapter describes the way to define a program for a Hot-Standby system with Unity Pro. We will then detail the monitoring DFBs implementation presented in the Design chapter. The finalization of the I/O devices implementation with Vijeo Citect will also be described in a last part. This paragraph details the implementation of the monitoring elements in both Premium Primary and Standby sections as well as the switchover management. The management of redundant analog outputs is also described in a last paragraph Monitoring Elements in the Primary Section Ethernet Control Ring Monitoring The Ethernet Control Ring is managed by a TSX ETY5103 Ethernet communication module. The DFB ETY_Monitor, associated to the function READ_STS, that allows to read the module status words, performs the monitoring of the Ethernet link. The DFB and the function are both presented in the Design Chapter. DFB ETY_Monitor implementation Prim_ETY_Ring_1 ETY_Monitor ETY1_State.BLK BLK Fault Prim_ETY1_Fault ETY1_State.MOD_ERROR MOD_ERROR ETY1 COM_ETY5103 COM_ETY5103 READ_STS 2 Monitoring_Rate Enable EN ENO pulse Pulse ETY1_State CH Prim_ETY1_RateEt RateEt RateEt Prim_ETY1_RateEt Step Action 1 Instantiate the READ_STS function 115

116 5-Implementation Create and connect the variable ETY1_State on the pin CH of the DFB. 2 : 3 Instantiate the DFB ETY_Monitor under the name Prim_ETY_Ring_1. 4 Create and connect the variable ETY1 on the pin COM_ETY5103 of the DFB With the previously created structure ETY1_State, connect the bit ETY1_State.BLK on the pin BLK, and the ETY1_State.MOD_ERROR on the pin MOD_ERROR On the Pulse pin, connect the variable pulse computed at the beginning of the primary section (see next paragraph). This pulse indicates each cycle beginning. For the pin named Monitoring Rate, enter the value 2, This means that the rate at which the local statistics are extracted from the READ_STS function is every 2 cycles. Connect a variable Prim_ETY1_RateEt (INT) located in the non-transfer area, on the pins RateEt. The execution of the function READ_STS is performed at the rate defined by the Monitoring_Rate of the DFB. Activate the execution option of the function READ_STS. Right click on the block and select Properties. Tick the box Show EN/ENO. This enable a pin EN as input of the block and a pin ENO as output 9 116

117 5-Implementation On the pin EN, connect the output pin ENABLE of the DFB. This pin is updated at the Monitoring Rate. On the output pin Fault, connect the variable Prim_ETY1_Fault (bool) located in the non transfer area. This one will be used for the default synthesis. Devices Ethernet Ring Monitoring The Devices Ethernet Ring is also managed by a TSX ETY5103 Ethernet communication module. The same manner as for the Ethernet Control Ring, the DFB ETY_Monitor performs the monitoring of the Ethernet link. DFB ETY_Monitor implementation Prim_ETY_Devices ETY_Monitor ETY2_State.BLK BLK Fault Prim_ETY2_Fault ETY2_State.MOD_ERROR MOD_ERROR ETY1 COM_ETY5103 COM_ETY5103 READ_STS 2 Monitoring_Rate Enable EN ENO pulse Pulse ETY2_State CH Prim_ETY1_RateEt RateEt RateEt Prim_ETY2_RateEt Step Action 1 Instantiate the READ_STS function 2 Create and connect the variable ETY2_State on the pin CH of the DFB. : 3 Instantiate the DFB ETY_Monitor under the name Prim_ETY_Devices. 4 Create and connect the variable ETY2 on the pin COM_ETY5103 of the DFB. 5 With the previously created structure ETY2_State, connect the bit ETY2_State.BLK on the pin BLK, and the ETY2_State.MOD_ERROR on the pin MOD_ERROR 6 For the pin named Monitoring Rate, enter the value 2, 117

118 5-Implementation 7 8 Connect a variable Prim_ETY2_RateEt (INT) located in the non-transfer area, on the pins RateEt. Activate the execution option of the function READ_STS. Right click on the block and select Properties. 9 Tick the box Show EN/ENO On the pin EN, connect the output pin ENABLE du DFB. This pin is updated at the Monitoring Rate. On the output pin Fault, connect the variable Prim_ETY2_Fault (bool) located in the non transfer area. This one will be used for the default synthesis. Monitoring Rate Pulse HSBY_ST EN R_TRIG ENO CLK Q Pulse HSBY THIS_OFF HSBY_CONF_OK THIS_OFFLINE INC GT THIS_PRY THIS_PRIMARY EN ENO EN ENO MOVE THIS_SBY THIS_STANDBY CycleNb INOUT INOUT CycleNb CycleNb IN1 OUT EN ENO REMT_OFF REMOTE_OFFLINE 1 IN2 0 CLK Q CycleNb REMT_PRY REMOTE_PRIMARY REMT_SBY REMOTE_STANDBY LOGIC_OK THIS_ISA THIS_ISB In order to control the ETY_Monitor block monitoring rate (execution of the READ_STS function), a pulse signal is implemented. This pulse signal is a trigger at the beginning of each program cycle. Therefore the monitoring rate varies accordingly with the cycle length of the application. 118

119 5-Implementation Fault Synthesis The fault synthesis comprises 3 parts. In a first time, a synthesis of the ETY modules faults is performed. Then, the same operation is done with the SCY modules in the next STG. Finally, a global faults synthesis is performed using a mask allowing the suppression of specific defaults. To implement this synthesis, we use the DFBs presented in the Design chapter: SYNTH_OR_ETY and SYNTH_FAULT. PRIM_ETY_FAULT_SYNTH SYNTH_OR_ETY PRIM_SYNTHESE_FAULT SYNTH_FAULT Prim_ETY1_Fault FLT_ETY_1 FAULT_ETY Faulty_ETY Prim_ETY2_Fault FLT_ETY_2 False Faulty_SCY False FLT_ETY_3 False Faulty_SCADA 2#0000_1110_1110_1011 Fault_Mask Fault_Synth Fault PRIM_SYNTH_FLT_PLC PRIM_FAULT_PLC Part 1: ETY Synthesis Step Action Instantiate the DFB SYNTH_OR_ETY under the name PRIM_ETY_FAULT_SYNTH. Connect the variable Prim_ETY1_Fault previously computed by the PRIM_ETY_Ring_1 DFB on the pin FLT_ETY_1. Connect the variable Prim_ETY2_Fault previously computed by the Prim_ETY_Devices DFB on the pin FLT_ETY_2. 4 Connect a False variable (unlocated) on all the other pins. 119

120 5-Implementation Part 2: FAULT Synthesis Step 1 2 Action Instantiate the DFB SYNTH_FAULT under the name PRIM_SYNTHESE_FAULT. Link the input pin Fault_ETY to the output pin Fault_ETY of the PRIM_ETY_FAULT_SYNTH DFB. 3 On the Fault_SCY pin, connect a False variable. 4 On the Fault_Scada pin, connect a False variable. Our application does not include blocks dedicated to SCADA system monitoring. We now have to define which fault will initiate a switchover. The table below is used to compose the mask and define the monitoring filter to apply as an input of the DFB. 5 Bit Elements Fault_Mask Definition Bit 0 Battery fault 1 Bit 1 Fault CPU 1 Bit 2 General I/O Rack fault 0 Bit 3 Fault on slot 3 1 Bit 4 Fault on slot 4 1 Bit 5 Fault on slot 5 0 Bit 6 Fault on slot 6 1 Bit 7 Fault on slot 7 1 Bit 8 Fault on slot 8 0 Bit 9 Fault on slot 9 1 Bit 10 Fault on slot 10 1 Bit 11 Ethernet Adapter(s) ETY Fault 1 Bit 12 Modbus Adapter(s) SCY Fault 0 Bit 13 SCADA Fault 0 Bit 14-0 Bit 15-0 The mask value to be set on the Fault_Mask pin is 2# On the Fault_Synth_Plc pin, connect the variable PRIM_SYNTH_FLT_PLC located in non-transfer area. This word represents the Primary configuration fault synthesis after the Fault_Mask filter. It will be used in the determination of the switchover. On the Fault pin, connect the variable PRIM_FAULT_PLC located in nontransfer area. This boolean information indicates a fault detection after the Fault_Mask filter. 120

121 5-Implementation Monitoring Elements in the Standby Section In this section, which is the only one executed by the Standby PAC, we find the same elements as for the Primary section. ETY_Monitor DFB implementation DFB ETY_Monitor implementation Stby_ETY_Ring_1 ETY_Monitor ETY1_State.BLK BLK Fault Stby_ETY1_Fault ETY1_State.MOD_ERROR MOD_ERROR ETY1 COM_ETY5103 COM_ETY5103 READ_STS 2 Monitoring_Rate Enable EN ENO pulse Pulse ETY1_State CH Stby_ETY1_RateEt RateEt RateEt Stby_ETY1_RateEt Step Action 1 Instantiate the READ_STS function 2 Connect the previously created variable ETY1_State, on the CH pin of the DFB. 3 Instantiate the DFB ETY_Monitor under the name Stby_ETY_Ring_ Connect the previously created variable ETY1, on the COM_ETY5103 pin of the DFB. With the previously created structure ETY1_State, connect the bit ETY1_State.BLK on the pin BLK, and the ETY1_State.MOD_ERROR on the pin MOD_ERROR. 6 For the pin named Monitoring Rate, enter the value Connect a variable Stby_ETY1_RateEt (INT) located in the non-transfer area, on the pins RateEt. Activate the execution option of the function READ_STS. Right click on the block and select Properties. 9 Select the box Show EN/ENO 10 On the pin EN, connect the output pin ENABLE of the DFB. 11 On the output pin Fault, connect the variable Stby_ETY1_Fault (bool) located in the non transfer area. This one will be used for the default synthesis. 121

122 5-Implementation 12 To verify that only the Standby PAC executes the Stby_ETY_Ring_1 DFB, add a condition on its execution. Right click on the block and select Properties. Select the box Show EN/ENO. This enable a pin EN as input of the DFB and a pin ENO as output On the pin EN, connect the bit 0 of the status register %SW61. This bit indicates whether the PAC is Primary or Standby (see the Design chapter). Devices Ethernet Ring Monitoring DFB ETY_Monitor implementation Stby_ETY_Devices ETY_Monitor ETY2_State.BLK BLK Fault Stby_ETY2_Fault ETY2_State.MOD_ERROR MOD_ERROR ETY1 COM_ETY5103 COM_ETY5103 READ_STS 2 Monitoring_Rate Enable EN ENO pulse Pulse ETY2_State CH Stby_ETY1_RateEt RateEt RateEt Stby_ETY2_RateEt 122

123 5-Implementation Step Action 1 Instantiate the READ_STS function 2 Connect the previously created variable ETY2_State on the pin CH of the DFB. 3 Instantiate the DFB ETY_Monitor under the name Stby_ETY_Devices. 4 5 Create and connect the previously created variable ETY2 on the pin COM_ETY5103 of the DFB. With the previously created structure ETY2_State, connect the bit ETY2_State.BLK on the pin BLK, and the ETY2_State.MOD_ERROR on the pin MOD_ERROR. 6 For the pin named Monitoring Rate, enter the value Connect a variable Stby_ETY2_RateEt (INT) located in the non-transfer area, on the pins RateEt.. Activate the execution option of the function READ_STS. Right click on the block and select Properties. 9 Select the box Show EN/ENO. 10 On the pin EN, connect the output pin ENABLE du DFB. 11 On the output pin Fault, connect the variable Stby_ETY1_Fault (bool) located in the non transfer area. This one will be used for the default synthesis. 12 To verify that only the Standby PAC executes the Stby_ETY_Devices DFB, add a condition on its execution. Right click on the block and select Properties. 13 Select the box Show EN/ENO. 14 On the pin EN, connect the bit 0 of the status register %SW

124 5-Implementation Fault Synthesis The fault synthesis comprises 3 parts. Initially, a synthesis of the ETY modules faults is performed. Next, the same operation is performed with the SCY modules. This will be developed in a following release of the STG. Finally, a global events synthesis is performed using a mask allowing the suppression of specific defaults. To implement this synthesis, we use the DFBs presented in the Design Chapter: SYNTH_OR_ETY, and SYNTH_FAULT. STBY_ETY_FAULT_SYNTH SYNTH_OR_ETY STBY_SYNTHESE_FAULT SYNTH_FAULT Stby_ETY1_Fault FLT_ETY_1 FAULT_ETY Faulty_ETY Stby_ETY2_Fault FLT_ETY_2 False Faulty_SCY False FLT_ETY_3 False Faulty_SCADA 2#0000_1110_1110_1011 Fault_Mask Fault_Synth Fault Stby_SYNTH_FLT_PLC Stby_FAULT_PLC Part 1: ETY Synthesis Step Action Instantiate the DFB SYNTH_OR_ETY under the name STBY_ETY_FAULT_SYNTH. Connect the variable Stby_ETY1_Fault previously computed by the STBY_ETY_Ring_1 DFB on the pin FLT_ETY_1. Connect the variable Prim_ETY2_Fault previously computed by the Stby_ETY_Devices DFB on the pin FLT_ETY_2. 4 Connect a False variable (unlocated) on all the other pins. 124

125 5-Implementation Part 2: FAULT Synthesis Step 1 2 Action Instantiate the DFB SYNTH_FAULT under the name STBY_SYNTHESE_FAULT. Link the input pin Fault_ETY to the output pin Fault_ETY of the STBY_ETY_FAULT_SYNTH DFB. 3 On the Fault_SCY pin, connect a False variable On the Fault_Scada pin, connect a False variable. Our application does not include blocks dedicated to SCADA system monitoring. Set the Fault_Mask value with the same value as for the Primary section, which is 2# On the Fault_Synth_Plc pin, connect the variable STBY_SYNTH_FLT_PLC located in non-transfer area. On the Fault pin, connect the variable STBY_FAULT_PLC located in nontransfer area. This boolean information indicates a fault detection after the Fault_Mask filter Switchover Management Once the monitoring DFB implemented and the fault synthesis computed in the Primary and Standby sections, it is necessary to process the obtained data and to set the switchover management rules. To set the management rules, we use the DFB Switch_Manag previously described in the Chapter Design. This DFB is instantiated in the Primary section. 125

126 5-Implementation DFB Switch_Manag implementation HSBY_SWITCH SWITCH_MANAG PRIM_SYNTH_FLT_PLC STBY_SYNTH_FLT_PLC PRIM_DIAG STBY_DIAG Switch_Nb_Reset SWITCH_NB_Reset SWITCH_NB Switch_Nb Force_Switchover FORCE FORCE Force_Switchover Step Action 1 Instantiate the DFB SWITCH_MANAG under the name HSBY_SWITCH Connect the variable PRIM_SYNTH_FLT_PLC (Primary PAC Fault synthesis) on the pin PRIM_DIAG. Connect the variable STBY_SYNTH_FLT_PLC (Standby PAC Fault synthesis) on the pin STBY_DIAG. Connect a variable Switch_Nb_Reset, located in the non-transfer area, on the Switch_Nb_Reset pin. 5 Connect a variable Force_Switchover on the pin FORCE, 6 On the output pin Switch_NB, connect a variable Switch_NB located in nontransfer area. The Ethernet redundant links management of the Premium Hot-Standby system is now complete. 126

127 5-Implementation Redundancy Analog output Management In the Primary section, it is necessary to implement the code that handles the switchover of the analog signals communication interface. The principle is to drive the communication interface in order to set, as output, the signal coming from the Primary PAC. The implementation is performed with the HSBY_ST_P block that provides the status of the Hot-Standby system. It is then possible to compute which PAC is the Primary (THIS_ISA or THIS_ISB). Then, depending on the result, the Primary PAC controls either relay A or B. Step Action 1 Instantiate the DFB HSBY_ST_P under the name HSBY_ST2 2 3 Instantiate a first AND block. Link the pins THIS_ISA and THIS_PRIM of the HSBY_ST2 block. Instantiate a second AND bloc. Link the pins THIS_ISB and THIS_PRIM of the HSBY_ST2 block. Instantiate a RS block under the name CMD_RELAY_A. 4 Link the output of the first AND block on the input pin S. Link the output of the second AND block on the input pin R1. 5 Instantiate a RS block under the name CMD_RELAY_B. Link the output of the first AND block on the input pin R1. Link the output of the second AND block on the input pin S. 6 Connect a PAC digital output with the name JMC_GK_CH_A on the output pin Q1 of the block CMD_RELAY_A. 7 Connect a PAC digital output with the name JMC_GK_CH_B on the output pin Q1 of the block CMD_RELAY_B. The diagram of the HSBY_ST_P block is presented on the next page. 127

128 5-Implementation HSBY_ST2 HSBY_ST_P HSBY_Active THIS_ISA THIS_ISB THIS_OFF THIS_PRI THIS_Sby REMT_UNDEF REMT_OFF REMT_PRI REMT_SBY AND CMD_RELAY_A IN1 OUT S Q1 JMC_GK_CH_A IN2 R1 AND CMD_RELAY_B IN1 OUT S Q1 JMC_GK_CH_B IN2 R1 RS RS LOGIC_OK CPU_SyncLink_OK CPU_OS_OK Copro_OS_OK ETY_minVersion Mon_ETY_OS_OK 128

129 5-Implementation 5.2. Quantum PAC This sub-section details the implementation of the monitoring elements in both Quantum Primary and Standby sections as well as the switchover management Monitoring Elements in the Primary Section Ethernet Control Ring Monitoring The Ethernet control ring is managed by a 140 NOE Ethernet communication module. The DFB NOE_Monitor, using the variables structure NOE_Monit, performs the monitoring of the Ethernet link. The DFB and the variables structure are both presented in the Design chapter. 129

130 5-Implementation NOE_Monitor DFB implementation PRIM_NOE_RING1 NOE_Monitor PRIM_NOE_CTRL.MSTR_active MSTR_ACTIVE MSTR_ACTIVE PRIM_NOE_CTRL.MSTR_active PRIM_NOE_CTRL.MSTR_done MSTR_DONE MSTR_DONE PRIM_NOE_CTRL.MSTR_done PRIM_NOE_CTRL.MSTR_error MSTR_ERROR MSTR_ERROR PRIM_NOE_CTRL.MSTR_error PRIM_NOE_CTRL.MSTR_RateEt RateEt RateEt PRIM_NOE_CTRL.MSTR_RateEt PRIM_NOE_CTRL.MSTR_ErrorCount Error_Count Error_Count PRIM_NOE_CTRL.MSTR_ErrorCount PRIM_NOE_CTRL.MSTR_Control MSTR_Control MSTR_Control MSTR_DataBuf PRIM_NOE_CTRL.MSTR_Control PRIM_NOE_CTRL.MSTR_Databuf 4 Slot Data_TCP PRIM_NOE_CTRL.MSTR_data 2 MonitoringRate LED_APPL PRIM_NOE_CTRL.Led_Appl 2 Retries LED_LINK PRIM_NOE_CTRL.Led_Link Pulse Pulse LED_RUN PRIM_NOE_CTRL.Led_Run NOE_Failure PRIM_NOE_CTRL.Failure CFG_PORT PRIM_NOE_CTRL.Led_Cfg_Port ETH_100M PRIM_NOE_CTRL.Led_Eth_100M OPTICFIB PRIM_NOE_CTRL.Led_OpticFib FULL_DUP PRIM_NOE_CTRL.Led_Full_Dup FAULT PRIM_NOE_CTRL.Fault EQUP_TYP PRIM_NOE_CTRL.EQUIP_TYPE IP_AD_1 IP_AD_2 IP_AD_3 IP_AD_4 MAC_ADD_1 MAC_ADD_2 MAC_ADD_3 MAC_ADD_4 MAC_ADD_5 MAC_ADD_6 PRIM_NOE_CTRL.IP_AD_1 PRIM_NOE_CTRL.IP_AD_2 PRIM_NOE_CTRL.IP_AD_3 PRIM_NOE_CTRL.IP_AD_4 PRIM_NOE_CTRL.MAC_AD_1 PRIM_NOE_CTRL.MAC_AD_2 PRIM_NOE_CTRL.MAC_AD_3 PRIM_NOE_CTRL.MAC_AD_4 PRIM_NOE_CTRL.MAC_AD_5 PRIM_NOE_CTRL.MAC_AD_6 Step Action 1 Instantiate the DFB NOE_Monitor under the name PRIM_NOE_RING_1. 2 Instantiate the variables structure NOE_Monit under the name PRIM_NOE_CTRL. 3 Connect the variables of the structure on the DFB pins. 4 5 For the pin named Slot, enter the value 4. This corresponds to the NOE module slot number in the hardware configuration. On the Pulse pin, connect the variable pulse computed at the beginning of the primary section (see next paragraph). This pulse indicates each cycle beginning. For the pin named Monitoring Rate, enter the value 2, This means that the rate at which the local statistics are extracted from the MBP_MSTR function is every 2 cycles. 130

131 5-Implementation 6 For the pin named Retries, enter the value 2, this corresponds to the maximum number of unsuccessful attempts to extract the local statistics from the MBP_MSTR before issuing an exception response. Devices Ethernet Ring Monitoring The devices Ethernet ring is also managed by a 140 NOE Ethernet communication module. The same manner as for the Ethernet control ring, the DFB NOE_Monitor, using the variables structure NOE_Monit, performs the monitoring of the Ethernet link. DFB NOE_Monitor implementation PRIM_NOE_DEV NOE_Monitor PRIM_NOE_DEVICES.MSTR_active MSTR_ACTIVE MSTR_ACTIVE PRIM_NOE_DEVICES.MSTR_active PRIM_NOE_DEVICES.MSTR_done MSTR_DONE MSTR_DONE PRIM_NOE_DEVICES.MSTR_done PRIM_NOE_DEVICES.MSTR_error MSTR_ERROR MSTR_ERROR PRIM_NOE_DEVICES.MSTR_error PRIM_NOE_DEVICES.MSTR_RateEt RateEt RateEt PRIM_NOE_DEVICES.MSTR_RateEt PRIM_NOE_DEVICES.MSTR_ErrorCount Error_Count Error_Count PRIM_NOE_DEVICES.MSTR_ErrorCount PRIM_NOE_DEVICES.MSTR_Control MSTR_Control MSTR_Control MSTR_DataBuf PRIM_NOE_DEVICES.MSTR_Control PRIM_NOE_DEVICES.MSTR_Databuf 6 Slot Data_TCP PRIM_NOE_DEVICES.MSTR_data 2 MonitoringRate LED_APPL PRIM_NOE_DEVICES.Led_Appl 2 Retries LED_LINK PRIM_NOE_DEVICES.Led_Link Pulse Pulse LED_RUN PRIM_NOE_DEVICES.Led_Run NOE_Failure PRIM_NOE_DEVICES.Failure CFG_PORT PRIM_NOE_DEVICES.Led_Cfg_Port ETH_100M PRIM_NOE_DEVICES.Led_Eth_100M OPTICFIB PRIM_NOE_DEVICES.Led_OpticFib FULL_DUP PRIM_NOE_DEVICES.Led_Full_Dup FAULT PRIM_NOE_DEVICES.Fault EQUP_TYP PRIM_NOE_DEVICES.EQUIP_TYPE IP_AD_1 IP_AD_2 IP_AD_3 IP_AD_4 MAC_ADD_1 MAC_ADD_2 MAC_ADD_3 MAC_ADD_4 MAC_ADD_5 MAC_ADD_6 PRIM_NOE_DEVICES.IP_AD_1 PRIM_NOE_DEVICES.IP_AD_2 PRIM_NOE_DEVICES.IP_AD_3 PRIM_NOE_DEVICES.IP_AD_4 PRIM_NOE_DEVICES.MAC_AD_1 PRIM_NOE_DEVICES.MAC_AD_2 PRIM_NOE_DEVICES.MAC_AD_3 PRIM_NOE_DEVICES.MAC_AD_4 PRIM_NOE_DEVICES.MAC_AD_5 PRIM_NOE_DEVICES.MAC_AD_6 Step Action 1 Instantiate the DFB NOE_Monitor under the name PRIM_NOE_DEV. 2 Instantiate the variables structure NOE_Monit under the name PRIM_NOE_DEVICES located in the non-transfer area. 131

132 5-Implementation 3 Connect the variables from the structure on the DFB pins. 4 For the pin named Slot, enter the value 6. 5 For the pin named Monitoring Rate, enter the value 2. 6 For the pin named Retries, enter the value 2. To manage the Hot-Standby system, the FAULT output of this DFB is used as an input in the Fault synthesis. This operation is described further. Monitoring Rate Pulse HSBY_ST EN R_TRIG ENO CLK Q Pulse HSBY THIS_OFF HSBY_CONF_OK THIS_OFFLINE INC GT THIS_PRY THIS_PRIMARY EN ENO EN ENO MOVE THIS_SBY THIS_STANDBY CycleNb INOUT INOUT CycleNb CycleNb IN1 OUT EN ENO REMT_OFF REMOTE_OFFLINE 1 IN2 0 CLK Q CycleNb REMT_PRY REMOTE_PRIMARY REMT_SBY REMOTE_STANDBY LOGIC_OK THIS_ISA THIS_ISB In order to control the NOE_Monitor block monitoring rate (execution of the MBP_MSTR function), a pulse signal is implemented. This pulse signal is a trigger at the beginning of each program cycle. Therefore the monitoring rate varies accordingly with the cycle length of the application. Profibus Fieldbus Monitoring The Profibus network is managed by a PTQ PDP MV1 communication module. The monitoring of the fieldbus is performed using the DFB PTQ_Monitor described in the Design Chapter. DFB PTQ_Monitor implementation PTQ_PRIMARY PTQ_Monitor t#300ms Fault_Timeout PTQ_FAULT BOOL General PTQ fault PTQPDPMV1HSBY_StatIn.ModuleStatus_ProfibusMasterModuleStatus Master_Status Faulty_Active BOOL Active PTQ fault PTQPDPMV1HSBY_StatIn.ModuleStatus_ProfibusMasterOperatingState Master_Operating_State PTQPDPMV1HSBY_StatIn.ModuleStatus_HSBYActiveStatus Active_Status Faulty_Passive BOOL Passive PTQ fault PTQPDPMV1HSBY_StatIn.ModuleStatus_HSBYActivenumberofslaves Active_NbSlave PTQPDPMV1HSBY_StatIn.ModuleStatus_HSBYPassiveStatus Passive_Status Faulty_Passive_UDP BOOL Passive PTQ fault via UDP PTQPDPMV1HSBY_StatIn.ModuleStatus_HSBYPassivenumberofslaves Passive_NbSlave PTQPDPMV1HSBY_StatIn.ModuleStatus_HSBYPassiveStatusUDP PTQPDPMV1HSBY_StatIn.ModuleStatus_HSBYPassivenumberofslavesUDP PassiveP_Status_UDP Passive_NbSlave_UDP 132

133 5-Implementation Step Action 1 Instantiate the DFB PTQ_Monitor under the name PTQ_Primary Connect the variables coming from the PTQPDPMV1HSBY_StatIn structure on the input pins. This structure is imported during the PTQ module configuration. For the pin Fault_Timeout enter a Time formatted value, for our application, 300ms. This value is the time length after which the module will declare a PTQ_FAULT in response to an exception. For the output pins, connect the variables PTQ_FAULT, Prim_PTQ_Fault, Stby_PTQ_Fault and UDP_PTQ_Fault located in the non-transfer area. To manage the Hot-Standby system, the PTQ_FAULT output of this DFB is used as an input in the Fault synthesis during the switchover determination. The Stby_PTQ_Fault is used in the Fault synthesis in the Standby section. Fault Synthesis The fault synthesis comprises 3 parts. In a first time, a synthesis of the NOE modules faults is performed. Then, the same operation is done with the PTQ modules. Finally, a global faults synthesis is performed using a mask allowing to inhibit specific defaults. To implement this synthesis, we use the DFBs presented in the Design Chapter: SYNTH_OR_PTQ, SYNTH_OR_NOE and SYNTH_FAULT. PRIM_NOE_FAULT_SYNTH SYNTH_OR_NOE PRIM_SYNTH_FAULT SYNTH_FAULT PRIM_NOE_CTRL_Fault FLT_NOE_1 FAULT_NOE Faulty_NOE PRIM_NOE_DEVICES_Fault FLT_NOE_2 Faulty_PTQ False FLT_NOE_3 False Faulty_SCADA False FLT_NOE_4 2#0001_1100_1101_1111 Fault_Mask False FLT_NOE_5 Fault_Synth PRIM_SYNTH_FLT_PLC False FLT_NOE_6 Fault PRIM_FAULT_PLC PRIM_PTQ_FAULT_SYNTH SYNTH_OR_PTQ PTQ_Fault False False False False False FLT_PTQ_1 FLT_PTQ_2 FLT_PTQ_3 FLT_PTQ_4 FLT_PTQ_5 FLT_PTQ_6 FAULT_NOE 133

134 5-Implementation Part 1: NOE Synthesis Step Action Instantiate the DFB SYNTH_OR_NOE under the name PRIM_NOE_FAULT_SYNTH. Connect the variable PRIM_NOE_CTRL_FAULT previously computed by the PRIM_NOE_RING1 DFB on the pin FLT_NOE_1. Connect the variable PRIM_NOE_DEVICES_FAULT previously computed by the PRIM_NOE_DEV DFB on the pin FLT_NOE_2. 4 Connect a False variable (unlocated) on all the other pins. Part 2: PTQ Synthesis Step 1 2 Action Instantiate the DFB SYNTH_OR_PTQ under the name PRIM_PTQ_FAULT_SYNTH. On the FLT_PTQ_1 pin, connect the variable PTQ_FAULT previously computed by the PTQ_PRIMARY DFB. 3 Connect a False variable (unlocated) on all the other pins. Part 3: FAULT Synthesis Step Action 1 Instantiate the DFB SYNTH_FAULT under the name PRIM_SYNTH_FAULT Link the input pin Fault_NOE to the output pin Fault_NOE of the PRIM_NOE_FAULT_SYNTH DFB. Link the input pin Fault_PTQ to the output pin Fault_PTQ of the PRIM_PTQ_FAULT_SYNTH DFB. On the Fault_Scada pin, connect a False variable. Our application does not include blocks dedicated to SCADA system monitoring. 134

135 5-Implementation We now have to define which event will initiate a switchover. The table below is used to compose the mask and define the monitoring filter to apply as an input of the DFB. 5 Bit Elements Fault_Mask Definition Bit 0 Battery fault 1 Bit 1 Fault CPU 1 Bit 2 General I/O Rack fault 1 Bit 3 Fault on slot 3 1 Bit 4 Fault on slot 4 1 Bit 5 Fault on slot 5 0 Bit 6 Fault on slot 6 1 Bit 7 Fault on slot 7 1 Bit 8 Fault on slot 8 0 Bit 9 Fault on slot 9 1 Bit 10 Fault on slot 10 1 Bit 11 Ethernet Adapter(s) NOE Fault 1 Bit 12 Profibus DP Adapter(s) Fault 0 Bit 13 SCADA Fault 0 Bit 14-0 Bit 15-0 The mask value to be set on the Fault_Mask pin is 2# On the Fault_Synth_Plc pin, connect the variable PRIM_SYNTH_FLT_PLC located in non-transfer area. This word represents the Primary configuration fault synthesis after the Fault_Mask filter. It will be used in the determination of the switchover. On the Fault pin, connect the variable PRIM_FAULT_PLC located in nontransfer area. This boolean information indicates a fault detection after the Fault_Mask filter. 135

136 5-Implementation Monitoring Elements in the Standby Section In this section, which is the only one executed by the Standby PAC, we find the same elements as for the Primary Section except for the DFB in charge of the PTQ module monitoring. Indeed, the PTQ module is active only if the PAC on which it is installed is the Primary. Therefore, during the Standby Section fault synthesis, we use the passive module status information computed from the PTQ_Monitor DFB in the Primary section. Controler Ethernet Ring Monitoring DFB NOE_Monitor implementation STBY_NOE_RING1 NOE_Monitor STBY_NOE_CTRL.MSTR_active MSTR_ACTIVE MSTR_ACTIVE STBY_NOE_CTRL.MSTR_active STBY_NOE_CTRL.MSTR_done MSTR_DONE MSTR_DONE STBY_NOE_CTRL.MSTR_done STBY_NOE_CTRL.MSTR_error MSTR_ERROR MSTR_ERROR STBY_NOE_CTRL.MSTR_error STBY_NOE_CTRL.MSTR_RateEt RateEt RateEt STBY_NOE_CTRL.MSTR_RateEt STBY_NOE_CTRL.MSTR_ErrorCount Error_Count Error_Count STBY_NOE_CTRL.MSTR_ErrorCount STBY_NOE_CTRL.MSTR_Control MSTR_Control MSTR_Control MSTR_DataBuf STBY_NOE_CTRL.MSTR_Control STBY_NOE_CTRL.MSTR_Databuf 4 Slot Data_TCP STBY_NOE_CTRL.MSTR_data 2 MonitoringRate LED_APPL STBY_NOE_CTRL.Led_Appl 2 Retries LED_LINK STBY_NOE_CTRL.Led_Link Pulse Pulse LED_RUN STBY_NOE_CTRL.Led_Run NOE_Failure STBY_NOE_CTRL.Failure CFG_PORT STBY_NOE_CTRL.Led_Cfg_Port ETH_100M STBY_NOE_CTRL.Led_Eth_100M OPTICFIB STBY_NOE_CTRL.Led_OpticFib FULL_DUP STBY_NOE_CTRL.Led_Full_Dup FAULT STBY_NOE_CTRL.Fault EQUP_TYP STBY_NOE_CTRL.EQUIP_TYPE IP_AD_1 IP_AD_2 IP_AD_3 IP_AD_4 MAC_ADD_1 MAC_ADD_2 MAC_ADD_3 MAC_ADD_4 MAC_ADD_5 MAC_ADD_6 STBY_NOE_CTRL.IP_AD_1 STBY_NOE_CTRL.IP_AD_2 STBY_NOE_CTRL.IP_AD_3 STBY_NOE_CTRL.IP_AD_4 STBY_NOE_CTRL.MAC_AD_1 STBY_NOE_CTRL.MAC_AD_2 STBY_NOE_CTRL.MAC_AD_3 STBY_NOE_CTRL.MAC_AD_4 STBY_NOE_CTRL.MAC_AD_5 STBY_NOE_CTRL.MAC_AD_6 Step Action 1 Instantiate the DFB NOE_Monitor under the name STDBY_NOE_RING_1. 2 Instantiate a new structure NOE_Monit under the name STBY_NOE_CTRL. 3 Connect the structure variables on the pins of the DFB. 4 On the pin Slot, enter the value 4 (position of the NOE module on the rack) 136

137 5-Implementation 5 On the pin Monitoring Rate, enter the value 2. 6 On the pin Retries, enter the value 2. 7 To verify that only the Standby PAC executes this block, we add a condition on its execution. Right click on the block and select Properties. Select the box Show EN/ENO. This enable a pin EN as input of the DFB and a pin ENO as output. 8 9 On the pin EN, connect the bit 0 of the status register %SW61. This bit indicates whether the PAC is Primary or Standby (see the Design chapter). 137

138 5-Implementation Devices Ethernet Ring Monitoring DFB NOE_Monitor implementation STBY_NOE_DEV NOE_Monitor STBY_NOE_DEVICES.MSTR_active MSTR_ACTIVE MSTR_ACTIVE STBY_NOE_DEVICES.MSTR_active STBY_NOE_DEVICES.MSTR_done MSTR_DONE MSTR_DONE STBY_NOE_DEVICES.MSTR_done STBY_NOE_DEVICES.MSTR_error MSTR_ERROR MSTR_ERROR STBY_NOE_DEVICES.MSTR_error STBY_NOE_DEVICES.MSTR_RateEt RateEt RateEt STBY_NOE_DEVICES.MSTR_RateEt STBY_NOE_DEVICES.MSTR_ErrorCount Error_Count Error_Count STBY_NOE_DEVICES.MSTR_ErrorCount STBY_NOE_DEVICES.MSTR_Control MSTR_Control MSTR_Control MSTR_DataBuf STBY_NOE_DEVICES.MSTR_Control STBY_NOE_DEVICES.MSTR_Databuf 6 Slot Data_TCP STBY_NOE_DEVICES.MSTR_data 2 MonitoringRate LED_APPL STBY_NOE_DEVICES.Led_Appl 2 Retries LED_LINK STBY_NOE_DEVICES.Led_Link Pulse Pulse LED_RUN STBY_NOE_DEVICES.Led_Run NOE_Failure STBY_NOE_DEVICES.Failure CFG_PORT STBY_NOE_DEVICES.Led_Cfg_Port ETH_100M STBY_NOE_DEVICES.Led_Eth_100M OPTICFIB STBY_NOE_DEVICES.Led_OpticFib FULL_DUP STBY_NOE_DEVICES.Led_Full_Dup FAULT STBY_NOE_DEVICES.Fault EQUP_TYP STBY_NOE_DEVICES.EQUIP_TYPE IP_AD_1 IP_AD_2 IP_AD_3 IP_AD_4 MAC_ADD_1 MAC_ADD_2 MAC_ADD_3 MAC_ADD_4 MAC_ADD_5 MAC_ADD_6 STBY_NOE_DEVICES.IP_AD_1 STBY_NOE_DEVICES.IP_AD_2 STBY_NOE_DEVICES.IP_AD_3 STBY_NOE_DEVICES.IP_AD_4 STBY_NOE_DEVICES.MAC_AD_1 STBY_NOE_DEVICES.MAC_AD_2 STBY_NOE_DEVICES.MAC_AD_3 STBY_NOE_DEVICES.MAC_AD_4 STBY_NOE_DEVICES.MAC_AD_5 STBY_NOE_DEVICES.MAC_AD_6 Step Action 1 Instantiate the DFB NOE_Monitor under the name STDBY_NOE_DEV 2 Instantiate a new structure NOE_Monit under the name STBY_NOE_DEVICES located in the non-transfer area. 3 Connect the structure variables on the pins of the DFB. 4 On the pin Slot, enter the value 6. 5 On the pin Monitoring Rate, enter the value 2. 6 On the pin Retries, enter the value 2. 7 Repeat the preceding steps for the block execution and connect the bit 0 of the status register %SW61 on the pin EN. As for the Primary Section, the FAULT output of this DFB is used as an input in the Fault synthesis. 138

139 5-Implementation Fault Synthesis STBY_NOE_FAULT_SYNTH SYNTH_OR_NOE STBY_SYNTH_FAULT SYNTH_FAULT PRIM_NOE_CTRL_Fault FLT_NOE_1 FAULT_NOE Faulty_NOE PRIM_NOE_DEVICES_Fault FLT_NOE_2 Faulty_PTQ False FLT_NOE_3 False Faulty_SCADA False FLT_NOE_4 2#0001_1100_1101_1111 Fault_Mask False FLT_NOE_5 Fault_Synth STBY_SYNTH_FLT_PLC False FLT_NOE_6 Fault STBY_FAULT_PLC STBY_PTQ_FAULT_SYNTH SYNTH_OR_PTQ STBY_PTQ_Fault False False False False False FLT_PTQ_1 FLT_PTQ_2 FLT_PTQ_3 FLT_PTQ_4 FLT_PTQ_5 FLT_PTQ_6 FAULT_NOE Part 1 : NOE Synthesis Step Action Instantiate the DFB SYNTH_OR_NOE under the name STDBY_NOE_FAULT_SYNTH. On the pin FLT_NOE_1, connect the variable STBY_NOE_CTRL_FAULT previously computed by the DFB STBY _NOE_RING1. On the pin FLT_NOE_2, connect the variable STBY _NOE_DEVICES_FAULT previously computed by the DFB STBY_NOE_DEV. 4 Connect a False variable (unlocated) on all the other pins. Part 2 : PTQ Synthesis Step 1 2 Action Instantiate the DFB SYNTH_OR_PTQ under the name STDBY_PTQ_FAULT_SYNTH. On the pin FLT_PTQ_1, connect the variable Stby_PTQ_Fault previously computed by the DFB PTQ_PRIMARY in the Primary section. 3 Connect a False variable on all the other pins. 139

140 5-Implementation Part 3 : FAULT Synthesis Step Action Instantiate the DFB SYNTH_FAULT under the name STDBY_SYNTH_FAULT. Link the input pin Fault_NOE to the output pin Fault_NOE of the DFB STDBY _NOE_FAULT_SYNTH. Link the input pin Fault_PTQ to the output pin Fault_PTQ of the DFB STDBY_PTQ_FAULT_SYNTH. On the Fault_Scada pin, connect a False variable. Our application does not include blocks dedicated to SCADA system monitoring. Set the Fault_Mask value with the same value as for the Primary section, which is 2# On the Fault_Synth_Plc output pin, connect the variable STBY_SYNTH_FLT_PLC. This word represents the Standby configuration fault synthesis after the Fault_Mask filter. It will be used in the determination of the switchover. Therefore, this variable is located on the reverse register %SW62 that allows writing from Standby to Primary. On the Fault pin, connect the variable STBY_FAULT_PLC located in nontransfer area. This boolean information indicates a fault detection after the Fault_Mask filter. For each of these blocks, activate the execution option and connect the bit 0 of the status register %SW61 on the pin EN. 140

141 5-Implementation Switchover Management Once the monitoring DFB is implemented and the fault synthesis is computed in the Primary and Standby sections, it is necessary to process the obtained data and to set the switchover management rules. To set the management rules, we use the DFB Switch_Manag previously described in the Design chapter. This DFB is instantiated in the Primary section. DFB Switch_Manag implementation HSBY_SWITCH SWITCH_MANAG PRIM_SYNTH_FLT_PLC STBY_SYNTH_FLT_PLC PRIM_DIAG STBY_DIAG Switch_Nb_Reset SWITCH_NB_Reset SWITCH_NB Switch_Nb Force_Switchover FORCE FORCE Force_Switchover Step Action 1 Instantiate the DFB SWITCH_MANAG under the name HSBY_SWITCH Connect the variable PRIM_SYNTH_FLT_PLC (Primary PAC Fault synthesis) on the pin PRIM_DIAG. Connect the variable STBY_SYNTH_FLT_PLC (Standby PAC Fault synthesis) on the pin STBY_DIAG. Connect a variable Switch_Nb_Reset, located in the non-transfer area, on the Switch_Nb_Reset pin. 5 Connect a variable Force_Switchover on the pin FORCE, 6 On the output pin Switch_NB, connect a variable Switch_NB located in nontransfer area. The Ethernet and Profibus redundant links management of the Quantum Hot-Standby system is now complete. 141

142 5-Implementation 5.3. Conclusion The implementation of all the elements for the Hot-Standby system is now complete. The system is ready to be used and tested with the provided Vijeo Citect application. This application enables monitoring of the status of the PAC station and the Vijeo Citect servers. In Green, the Primary PAC Counter incremented every In White, the Standby PAC seconds to monitor the system activity Name of the active I/O Server 142

143 6-Performance 6. Performance Once the systems configured and implemented, performance tests are performed on both architectures (Premium and Quantum) in order to measure the actual switchover times. To test a wide range of cases that could be encountered in industry, different operating exceptions have been simulated Performance test protocols The tests have been performed at two different layers of the system architecture: between PAC and field device (test 1) between SCADA Server and PAC (test 2) PAC field device To measure the performance of the system when an event occurs between the PAC and a field device, different tests have been performed: Stop CPU Crash CPU CRP fault (for Quantum only) Broken Ethernet link (Control network) The Ethernet cable is unplugged between the switch SW3 and the PAC A. Broken Ethernet link (Device network). The Ethernet cable is unplugged between the PAC A and the Advantys STB island with IP address The system is designed to initiate a switchover on each of these cases, so the parameter measured is the switchover time, defined by the time length between the start of the simulation event and when the system becomes operational again. The criterion to declare the system operational is when a communication is established between the PAC and the STB Island with IP address sets of parameters (scan time and volume of data) are used in order to quantify the performance with different loads of the system: Scan time 40ms Data exchanged 130 kb for Quantum and 100 kb for Premium Scan time 130ms Data exchanged 130 kb for Quantum and 100 kb for Premium 143

144 6-Performance The establishment of the communication between the PAC and the Advantys STB Island is detected with the wireshark software ( which is a network protocol analyzer. We are thus able to monitor the switchover time very accurately by measuring the IOScanning communication between the PAC and the Advantys STB Island connected directly on the network SCADA Server PAC In this part we measure the performance of the system when an event occurs between the PAC and an IO Server of the SCADA system. Different tests have been performed. Stop CPU Crash CPU CRP fault (for Quantum only) Broken Ethernet link (Control network) The Ethernet cable is unplugged between the switch SW3 and the PAC A. Broken Ethernet link (Device network). The Ethernet cable is unplugged between the PAC A and the Advantys STB island with IP address In this case, the criterion to declare the system operational is the establishment of the communication between the IOServer1 and the PAC. We performed tests on OFS and Modnet between I/O server and I/O device (100 tags) on the Quantum architecture and tests on Modnet between I/O server and I/O device (100 tags) on the Premium architecture (Results for OFS will be included in the next issue of the document). Wireshark software is also used to perform these tests as indicated on the next diagrams. 144

145 6-Performance 6.2. Premium PAC Architecture This paragraph describes the results of the tests performed on the Premium architecture represented on the figure below ABE7 ABE7 145

146 6-Performance Test 1: Measurement of the connection time between the PAC and a field device (STB) after a switchover. Scan Time : 40ms Exchanged Data : 100kbytes Scan Time : 130ms Exchanged Data : 100kbytes Swithover System Stop Primary PAC Unexpected stop of the Primary PAC (Crash) 115ms 743ms 325ms 943ms Switchover by application Disconnection Ethernet Link NOE Scada ring Disconnection Ethernet Link NOE Devices ring 112ms 491ms 331ms 653ms Test 2: Measurement of the connection time between the IOServer 1 and the PAC after a switchover. Scan Time : 40ms Modnet Driver OFS Swithover System Stop Primary PAC Unexpected stop of the Primary PAC (Crash) 3,374s X 2,866s X Switchover by application Disconnection Ethernet Link NOE Scada ring Disconnection Ethernet Link NOE Devices ring 4,416s X 4,324s X 146

147 6-Performance 6.3. Quantum PAC Architecture This paragraph describes the results of the tests performed on the Quantum architecture represented on the figure below. SERVER 1 SERVER 2 Wireshark Test 2 IP: IP: IP: IP: Client 1 SW1 SW2 Manager Client 2 SW3 SW4 PAC A IP: MASK: PAC B IP: MASK: Remote I/O IP: MASK: IP: MASK: Wireshark Test 1 IP: SW10 Manager SW12 SW11 IP: IP: IP: IP: IP: IP:

148 6-Performance Test 1: Measurement of the connection time between the PAC and a field device (STB) after a switchover. Scan Time : 40ms Exchanged Data : 130kbytes Scan Time : 130ms Exchanged Data : 130kbytes Stop Primary PAC 819ms 849ms Swithover System Unexpected stop of the Primary PAC (Crash) 367ms 343ms Primary CRP Fault 473ms 661ms Switchover by application Disconnection Ethernet Link NOE Scada ring Disconnection Ethernet Link NOE Devices ring 679ms 545ms 313ms 878ms Test 2: Measurement of the connection time between the IOServer 1 and the PAC after a switchover. Scan Time : 40ms Modnet Driver OFS Swithover System Stop Primary PAC Unexpected stop of the Primary PAC (Crash) Primary CRP Fault 3s 1,6s 3,6s 1,7s 3,9s 1,9s Switchover by application Disconnection Ethernet Link NOE Scada ring Disconnection Ethernet Link NOE Devices ring 4,1s 3,1s 2,3s 2s 148

149

150 Schneider Electric Industries SAS Head Office 89, bd Franklin Roosvelt Rueil-Malmaison Cedex FRANCE Due to evolution of standards and equipment, characteristics indicated in texts and images in this document are binding only after confirmation by our departments Print: Version