, pp.233-242 http://dx.doi.org/10.14257/ijseia.2014.8.4.24 Methods of Software Qualification for a Safety-grade Optical Modem to be used Core Protection Calculator (CPC) in Korea Standard Nuclear Power Plant (KSNP) Jangyeol Kim 1 and Soongohn Kim 2* 1 Instrumentation and Control & Human Factors Division/Korea Atomic Energy Research Institute (KAERI) 989-111 Daedeok-daero, Yuseong-gu, Daejeon, Republic of Korea 305-353 2 Division of Computer and Game Science, Joongbu University, 101 Daehakro, Chubu-Meon, GumsanGun, Chungnam, 312-702, Korea 1 jykim@kaeri.re.kr, 2 sgkim@joongbu.ac.kr Abstract This paper describes safety-critical software qualification methods and its approach, which corresponds to well-structured qualification organization, methods and results, software quality assurance, and software configuration management. This paper also describes the verification test environment, test components and items, a traceability analysis, and system tests as a result of software qualification based on Software Requirement Specifications (SRS) for a safety-grade optical modem of a Core Protection Calculator (CPC) in a Korea Standard Nuclear Power Plant (KSNP), and Software Design Specifications (SDS) for a safety-grade optical modem of a CPC in a KSNP. We believe that we achieve the functionality, performance, reliability, and safety, which are the software qualification objective goals of safety-critical systems. Keywords: Software Qualification, Software Quality Assurance, Verification and Validation, Software Configuration Management, Integration & System Test, Safety-grade Optical Modem, Core Protection Calculator, Korea Standard Nuclear Power Plant 1. Introduction To qualify as safety-critical software, the defined responsibilities among the assurance organization are very important. The development team is responsible for producing a design output during the software life cycle. The Software Verification & Validation (SVV) and Software Safety Analysis (SSA) are used for a safety qualification on the produced design output by the development team. First, prior to use, a Commercial Off-The Shelf (COTS) software tool should be dedicated by the quality assurance organization. The Software Configuration Management under Software Quality Assurance is responsible for the configuration identification, which are the status accounting, and the revision control on all of the design output and its verification results. The suggested well-structured organization described in this paper is as shown in Figure 1. * Corresponding Author ISSN: 1738-9984 IJSEIA Copyright c 2014 SERSC
(SPM : Software Program Manual, SDP: Software Development Plan, SQAP: Software Quality Assurance Plan, SVVP: Software Verification and Validation Plan, SCMP : Software Configuration Management Plan, SO&MP : Software Operation and Maintenance Plan, SSPD : Software Safety Plan Description, CDP: Commercial Off the Shelf Dedication Plan, COTS: Commercial Off The Shelf Software, SQA: Software Quality Assurance, SVV: Software Verification and Validation, SCM: Software Configuration Management, SR: Software Review, SA: Safety Analysis, FCA: Functional Configuration Audit, PCA: Physical Configuration Audit) Figure 1. Well-structured Qualification Organization for Safety-Critical Systems This paper describes the test environment, test components and items, a traceability analysis, and system tests as a result of system verification and validation based on Software Requirement Specifications (SRS) for a safety-grade optical modem of a Core Protection Calculator (CPC) in a Korea Standard Nuclear Power Plant (KSNP), and Software Design Specifications (SDS) for a safety-grade optical modem of a CPC in a KSNP. 2. Methods and Results The methods used for software qualification are requirement traceability, software verification testing, software quality assurance and software configuration management. In this section, the test methods and results are described. Above all, in the case of safetycritical systems, it is important whether system test results on the host environment are satisfied with the target board have been performed or not. Functional tests, performance tests, event tests and scenario tests for safety-grade optical modem has been tested successfully. Coverage of the range value, boundary value, and equivalent value were also measured. 2.1. Traceability Analysis A traceability analysis is performed through the Code & Standard criteria, requirements, test identifier (ID) and object code of target board of completeness, correctness, and consistency. The accuracy of a definition, input/output accuracy, 234 Copyright c 2014 SERSC
accuracy of software behavioral characteristics, and accuracy of an interface of the software function are very important. Traceability analyses are classified by basis reviews & comments and the latter details the requirement traceability management, as shown in Figure 2. 2.2 Verification test environment Figure 2. Requirement Traceability Analysis Software testing consists of a component test, an integration test, and a system test. These test executions produce a test plan generation, a test design generation, a test case generation, and a test procedure generation according to the software test life cycle, as shown in Figure 3. A test plan should be prepared to satisfy the functional requirements and performance requirements, event test requirements, and scenario test requirements of the safety-critical system. Figure 3. Testing Process by Test Life Cycle The system test performs functionality, performance, event tests, and scenario tests including error detection and diagnosis, scan time violation, deterministic communication, and communication independence. Application firmware of the safety-grade optical modem to be used the Core Protection Calculator (CPC), in a Korea Standard Nuclear Power Plant (KSNP) was developed under GNU /Linux Ubuntu 11.10 of AMD64 environments. To build a system test with the host environment, firmware was ported in the target board of a safety-grade optical modem using Copyright c 2014 SERSC 235
USBISP. To measure the embedded software of a safety-grade optical modem, an AVR USBISP V3.0 and avrdude 5.10 utility were used as shown in Figure 4. Figure 4. Verification Test Environments of Safety-Grade Optical Modem 2.3. Test Components and Test Items Test components and items are as shown in Table 1. Table 1. Test Components and Items for Safety-Grade Optical Modem of CPC NO Category Test Test Items Components 1. Functional test Initial setup Variable of Hardware and Software - - Optical Modem - - LED - - Timer - - WDT etc. Optical signal translation Voltage-Optical signal Optical signal - Voltage Data communication Status indication Sending Only (Unidirectional) Receiving Only (Unidirectional) POWER TX RX FAULT Setup Protocol Gain, Offset Protocol Analysis (Packet) CRC8 2. Performance test Accuracy Accuracy ±0.05% Communication speed - 4ms - 57600bps 3. Event test Fault injection Power Fail, Abnormal State - - Signal short - - CRC - - Timeout - - Frame Error - - Buffer overflow 4. Scenario test Continuous operation test About three months burn-in test 236 Copyright c 2014 SERSC
In particular, the performance requirements listed above should be satisfied for the purchase order requirements of Korea Hydro and Nuclear Power Co. Ltd (KHNP) as follows. - The response time should be less than 4ms. - Full Range Accuracy within ± 0.05% or better should be satisfied. - Unidirectional buffering and deterministic communication should be satisfied. 2.4. Test Results The initialization setup, optical signal conversion capabilities, communication capabilities and accuracy, display status indication, parameter setup, and protocol were set up in the functional tests. The Performance tests were carried out as follows: - Response time : 4ms - Accuracy of ± 0.05% - 57600 bps transfer rate - Communication time between the ADC (Analog Digital Converter) and MCU (Main Control Unit) - Communication time between the MCU and DAC (Digital to Analog Converter) - Optical modem transmitter offset - Gain adjustment between the MCU and DAC - TWI (Two Wire Interface) communication as an optical transmitter - TWI communication as an optical receiver - Communication between the MCU of the optical modem sender and external clock - Communication between the MCU of the optical modem receiver and external clock - Status of communication tracking between the MCU of the optical modem receiver and optical receiver s component The verification results of the performance test for a 57600 bps transfer rate and response time (4ms) among several performance tests are shown in Figure 5 and Figure 6, respectively. Figure 5. Transfer Rate of 57600bps Copyright c 2014 SERSC 237
Figure 6. 4ms Response Time Event tests were performed based on the error injection; in particular, a signal short-circuit among several error injections was tested successfully. The signal source of making a triangular wave under the verification test oracle equipment was used as shown in Figure 7. A scenario based burn-in test was carried out during three months and two weeks continuously. Figure 7. Continuous Tests by Triangular Wave 3. Software Quality Assurance Measurement 3.1. Software Quality Assurance After the system test is completed, an inside peer review was performed in the same way as that of the integration test and system test. The defects, problems and corrective actions list should be submitted to the Software Quality Assurance (SQA) organization for an Anomaly Report (ANR). A review meeting should be held according to the quality assurance procedure. It should focus on an objective quality goal as quality management in the system test phase. After a review of the system s test results, the SQA should inspect and follow-up the test results with a checklist as to whether a system test will achieve enough of the original objectives. It should also determine whether the system test level was proper for the physical configuration audit and performance based audit by the SQA. After comparing the expected value and the resulting value by the SQA, it is necessary that they take the required actions. We have used a confirmation of the results of the SQA for the system test results. One can 238 Copyright c 2014 SERSC
use the software qualification results for the physical configuration audit and the performance based audit by the SQA. Software Quality Assurance activities have been performed during the software life cycle, as shown in Table 2. SWLC Table 2. Software Quality Activities under a System QA Program Software Development Baselines Requirements SQAP, SVVP, SRS. SRS review. Recommended Software Quality Assurance Activities. In-process audit. Software Test Plan Review. Managerial Review Design Implementation Preliminary Design Description Software Design Description Code Listing, Other Documentation(Code Implementation Specification). Preliminary Design Review. In-process Audit. Detailed Design Review. In-Process Audit Test Test Documentation. Functional Configuration Audit Installation and Checkout Deliverable items, Installation Report etc., SVVR User Documentation. Physical Configuration Audit. Performance-based Audit. User Documentation Review Notes : SWLC : Software Development Life Cycle, SVVR : Software Verification and Validation Report. The division of responsibilities among the System-Level Quality Assurance and Software- Level Quality Assurance (SQA) are that the former primarily focuses on quality assurance criteria and quality assurance procedure in the upper level, whereas the latter implements a review and audit from the technical perspective in the lower level. The classification in the audit consists of a Functional Configuration Audit, Physical Configuration Audit, and In- Process Audit. A Functional Configuration Audit compiles safety-grade optical modem software, whether or not it meets the requirements in testing level from functional entities (logical viewpoint). Although an In-Process Audit is largely applicable in all of the software lifecycle, it is applied in the design and implementation phases. Physical Configuration Audit and Performance-based Audit are planned to be applied in the release phase (final phase). A review performs a requirement phase, design phase, and user documentation to ensure the quality assurance criteria and quality assurance procedure. Copyright c 2014 SERSC 239
3.2. Software Configuration Management Software Configuration Management (SCM) under a software quality assurance policy is performed during the entire software lifecycle. Inconsistencies among the software configuration items were found to be the date and revision number of the software documents and source codes. Some of the reported anomalies have been resolved throughout the software configuration management process. Figure 7 shows an example of a SCM process. 4. Conclusions Figure 7. Software Configuration Management by NuSCM Tool Our safety-critical software qualification methodology through the projects was well established. The toolsets used was a self-developed one and a commercially available one. The technique has taken advantage of qualification techniques that use Code and Standard criteria, and requirement traceability analysis with verification testing techniques. We investigated software qualification processes, i.e., the requirements, design, implementation, and test phase, using the proposed well-structured qualification organization and qualification methodologies. Our major qualification techniques are a requirement traceability analysis, formal verification testing as a verifier, software configuration management under the quality assurance system including the integration, and a system test. The applied qualification methodology satisfies the SRP/BTP-14 criteria for a safety-critical system in nuclear power plants. All tests were performed according to the test plan and test procedures. Functional testing, performance testing, event testing, and scenario based testing for a safety-grade optical modem of a Core Protection Calculator in a Korea Standard Nuclear Power Plant as a thirtyparty verifier were successfully performed. We confirmed that the coverage criterion for a safety-grade optical modem of a Core Protection Calculator was satisfactory using a traceability analysis matrix between high-level requirements and a lower-level system test case data set. The qualification methodology and our experience are going to be continually upgrading in upcoming projects. 240 Copyright c 2014 SERSC
Acknowledgements This work, described herein, is being performed for Development of the Suitability Assessment on nuclear I&C, Surveillance and Early Response as part of the Korea Atomic Energy Research Institute (KAERI) projects and funded by the Korean Ministry of Science and Technology since on January the 1 st, 2012. References [1] J.-Y. Kim and S.-G. Kim, Software Qualification Approach for Safety-critical Software of the Embedded System, The 2012 International Conference on Future Generation Communication and Networking (FGCN), Kangwondo Korea, (2012) December 16-19. [2] J. Y. Kim and K.-C. Kwon, The Commercial Off The Shelf(COTS) Dedication of QNX Real Time Operating System(RTOS), International Conference on Reliability, Safety and Hazard-2010, Mumbai India, (2010) December 14-16. [3] J. Y. Kim, S. W. Cheon, J. S. Lee, Y. J. Lee, K. H. Cha and K.-C. Kwon, Software V&V Methods for a Safety Grade Programmable Logic Controller, International Conference on Reliability, Safety and Hazard- 2005, Mumbai India, (2005), December. 1-3. [4] K. H. Cha, J. Y. Kim, S. W. Cheon, J. S. Lee, Y. J. Lee and K.-C. Kwon, Software Qualificaiton of a Programmable Logic Controller for Nuclear Instrumentation and Control Applications, 2006 WSEAS International Conferences(ISCGAV 06), Crete, (2006) August. [5] 10CFR 50 Appendix A,4/94, General Design Criteria. [6] ASME NQA-1-1997, Quality Assurance Requirements for Nuclear Facility Applications. [7] USNRC Reg. Guide 1.152, Rev. 02, 2006, Criteria for Programmable Digital Computers System Software in Safety Related Systems of Nuclear Power Plants. [8] USNRC Reg. Guide 1.168 Rev.1 Feb. 2004, Verification, Validation, Reviews, And Audits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants. [9] USNRC Reg. Guide 1.169 Rev.0 Sep. 1997, Configuration Management Plans for Digital Computer Software Used in Safety Systems of Nuclear Power Plants. [10] USNRC Reg. Guide 1.170 Rev.0 Sep. 1997, Software Test Documentation for Digital Computer Software Used in Safety Systems of Nuclear Power Plants. [11] USNRC Reg. Guide 1.171 Rev.0 Sep. 1997, Software Unit Testing for Digital Computer Software Used in Safety Systems of Nuclear Power Plant. [12] USNRC Reg. Guide 1.172 Rev.0 Sep. 1997, Software Requirements Specification for Digital Computer Software Used in Safety Systems of Nuclear Power Plants. [13] USNRC Reg. Guide 1.173 Rev.0 Sep. 1997, Developing Software Life Cycle Processes for Digital Computer Software Used in Safety Systems of Nuclear Power Plants. [14] USNRC IEEE Std. 7-4.3.2-2003, Standard Criteria for Digital Computers in Safety System of Nuclear Power Generating Stations. [15] IEEE Std. 829-1998, IEEE Standard for Software Test Documentation. [16] IEEE Std. 1008-1987, IEEE Standard for Software Unit Testing. [17] IEEE Std. 1012-1998, IEEE Standard for Software verification and validation. [18] IEEE Std. 323-83Ed., R96, Standard for Qualifying Class 1E Equipment for Nuclear Power Generating Stations. [19] IEEE Std. 828-1998, Standard for Configuration Management in Systems and Software Engineering. [20] IEEE Std. 829-1998, Standard for Software Test Documentation. [21] IEEE Std. 1008-1987, R1993, Standard for Software Unit Testing. [22] IEEE Std. 1012-1998, Standard for Software Verification and Validation. [23] IEEE Std. 1028-1997, Standard for Software Reviews. [24] IEEE Std. 1058-1998, Standard for Software Project Management Plans. [25] IEEE Std. 1074-1997, Standard for Developing Software Life Cycle Processes. [26] IEEE Std. 1228-1994, Standard for Software Safety Plans. Copyright c 2014 SERSC 241
Authors Jangyeol Kim, he received Ph.D. degrees from University of CHUNG-ANG, Seoul, Korea, in Computer Science in 1994. He has been working as an Principal Researcherr in Korea Atomic Energy Research Institute (KAERI) from March 1985. His research interests include Distributed Operating System, System Programming, Embedded System, Safety-critical Software Engineering, Software Qualification, Hardware Qualification, Commercial Off-the-Shelf Software Dedication, Quality Assurance, and Safety Analysis, and so on. SoonGohn Kim, he received Ph.D. degrees from Chonbuk National University, Seoul Korea, in Computer Engineering in 1999. He has been working as a Professor in Joongbu University from March 1995. His research interests include Ubiquitous Computing, Distributed Computing, Database Integrity, Cryptographic Protocol, A methodology of Software Development, Software Evaluation, Networks, and so on. 242 Copyright c 2014 SERSC