1(60) Testing of safety-critical software some principles Emerging Trends in Software Testing: autumn 2012 Matti Vuori, Tampere University of Technology 27.11.2012
Contents 1/4 Topics of this lecture 6 What does "safety critical" mean? 7 What kind of system types do we think of? 8 Goals of testing 9 How safe is safe: SIL Safety Integrity Level 10 (Safety) standards are important 11 (Safety) standards are important 22 12 Understanding the overall system 13 Central quality characteristics 14 Long life cycle of machinery and industrial systems 17 Some factors affecting the ways and performance of testing. 18 Development is demanding 19 Sense of responsibility of the hazards 20
Contents 2/4 Organizational environment 21 Development life cycle processes 22 Quality policy and safety policy 23 Process maturity 24 Rigorous quality & safety assurance 26 Requirement specifications 27 Basis for development and testing (a bit of a caricature) 28 Architecture is important 29 Design analysis 31 Tracing 32 Testing is only one means of assessing the system 33 What do the standards require? 34 But first a look to the IEC 61508 V&V process context 35
Contents 3/4 IEC-61508-3 Failure Analysis 36 IEC-61508-3 Modelling 37 IEC-61508-3 Semi-formal methods 38 IEC-61508-3 Dynamic analysis and testing 39 IEC-61508-3 Functional and black box testing 40 IEC-61508-3 Performance testing 41 IEC-61508-3 Static analysis 42 IEC-61508-3 Overall 43 Quite heavy at higher SIL levels? 44 Sources of test conditions and test cases 45 Negative testing emphasised 47 Risk-based testing 48 Realistic traditional test automation 49
Contents 4/4 Model-based testing for robustness 50 Exploratory testing? 51 Diversity in testing practices and methods 52 Documentation 53 Competence 54 Suitable independence 55...and co-operation and collaboration between designers and testers 56 Top 10 points of conclusion 57 More information 58 References 59
6(60) Topics of this lecture i What is safety critical software and what is essential about them? What is their testing like and should be like? What do safety standards require of testing them? It is assumed that the audience knows about testing. We don't talk about details or basics (hopefully), but try to reflect the environment of safety-critical development so people can transfer and adjust their knowledge and skills to it.
7(60) What does "safety critical" mean? The system has hazards potential to cause accidents, losses to property, money. Safety is an essential requirement. Software has an influence on safety. There are formal requirements for safety regulations, standards. There are specific actions for assuring safety and those actions are expected and required. It would be a risk to the manufacturer also, if safety was not sufficient (accidents, lawsuits). Safety-critical software can be such in itself when it provides information used in situations where a failure would be dangerous, or when it is used to control something (like a machine) or when it is used as a safety system that does something when something else does not work correctly.
What kind of system types do we think of? 8(60) The actual systems could hospital systems, automobiles, nuclear powerplants To have a somewhat similar picture in our heads, let's think of "machines" in factories or on wheels construction or forests.
9(60) Goals of testing Find any problems that could cause hazards. Verify that any safety controls work as intended and reliably in case of a malfunction or human error. Provide proof that development of the system has been done as required by laws and safety standards. Validate that the overall system meets its safety requirements and is as safe as can be expected based on that. Nothing is ever absolutely safe. Produce information to management about the system's risk level and acceptability. Produce documentation to show all 9(60) that later on if needed. A lawsuit after an accident, for example.
Safety management How safe is safe: SIL Safety Integrity Level 10(60) A key term for SIL - Safety Integrity Level http://en.wikipedia.org/wiki/safety_integrity_level. "How reliably safety functions must work". Range from 1 to 4. In practice, for example, machines are often SIL2-level, the railway systems SIL3, nuclear power could 4 etc... Defined for the overall system and derived from that, to the software In standards, the SIL level defines, what kinds of development methods, design solutions, validation methods etc. must be used. 1 and 2 are easy. At the higher levels for example model-based development and testing may be needed SIL 4 is a very challenging. Note: SIL is just one of such measure. Many domains and standards have alternative measures for safety levels.
Safety management 11(60) (Safety) standards are important 1/3 Standards give high requirements for the development process. Process phases, development and deployment lifecycle. Methods and tools. Quality assurance. Verification. Validation. People's competence need to be trained. The requirements depend on the level of safety integrity level. Provide technical requirements of the design solutions. IEC 61508-series a good example. A generic standard series for development of safety critical systems. Own standard parts for different levels of abstraction of the system (overall system, programmable electronics, software). Vocabulary. Descriptions of the methods required or recommended.
Safety management 12(60) (Safety) standards are important 22 IEC 61508 parts: IEC TR 61508-0 Functional safety and IEC 61508. IEC 61508-1 General requirements. IEC 61508-2 Requirements for E/E/PE safety-related systems. IEC 61508-3 Software requirements. IEC 61508-4 Definitions and abbreviations. IEC 61508-5 Examples and methods for the determination of safety integrity levels. IEC 61508-6 Guidelines on the application of IEC 61508-2 and IEC 61508-3. IEC 61508-7 Overview of techniques and measures. 61508 is used for example in automation systems development. Some domains (like health care system and auto industry) have similar, domain-specific standards.
Understanding the system 13(60) Understanding the overall system Where the program is used, in which environment, in which devices. How is the system used. Task analyses how systems are operated. Process modelling. Requirements flow top-down. Safety of all things, and their acceptability, assessed at the level of the overall system. Safety analyses, human error analysis, reliability analyses etc needed. One needs good context knowledge, good information. One must understand the role of ones own doing and its effects How testing of something relates to the whole.
Understanding the system 14(60) Central quality characteristics 1/3 Safety. Of the overall system. No compromises. Information security. Correct data. Sabotage. Relevant in expanding the systems a machine as part of an information system, a part of a logistic system. Currently not considered enough. Robustness and reliability. The system can handle "anything" in varying environments. Don't assume anything about how other components work. Problem-free action in long durations of use.
Understanding the system 15(60) Central quality characteristics 2/3 Usability. Guiding the user to safe operation. Handling of human errors. Testability. In order for verification and validation to be easy and successful. In order to be able to test with a number of techniques, for example with model-based techniques. Standardised interfaces, ability to simulate etc. Analysability. Analysis of plans, designs is important. Modularity. Packaged modules. Changeability, replaceability during a long life cycle of machines. Redundancy, diversity duplication of sub-systems using different implementations.
Understanding the system 16(60) Central quality characteristics 3/3 Extensibility. New things that are added during a long life cycle. But plug-ins are dangerous need to know and control what the system contains. Limited functionality. Everything in system is relevant, validated and verified. No vague extras.
Understanding the system Long life cycle of machinery and industrial systems 17(60) Dozens of years of use. How to renew technology after ten years? And ensure that the revised whole is ok. How to keep the configuration in good shape (adding, changing things what are we developing and testing for?). How to test, if the only device in some configuration has been build years ago, is located at customer and even the components are not available anymore for testing. Simulators, standards Systems need to done right from the beginning.
About the development Some factors affecting the ways and performance of testing. 18(60) The status of software in the organisational culture Arrangements, environments and tools skills for testing The role of software development in product development projects Factors affecting the performance of software testing Professionalism of testing culture and testers Resources for software development The competences and maturity of software development units
About the development 19(60) Development is demanding Need to understand the overall system. Validation and certification in the context of the overall system. Safety must not be compromised. Quality may not be compromised. The overall process is a demanding. Technologies, methods. Plenty of process requirements. We need competence and experience. Hazards cause responsibility. In machine systems, simulation is important. Testing with actual experiment is difficult to arrange there may be only one, and that is being build at the same time.
About the development 20(60) Sense of responsibility of the hazards The safety-critical software is a critical part of the overall system, which may be dangerous. Leading to injuries and accidents. The software can be located operational systems of safety system. Danger causes responsibility. Understanding of what can be followed, unless the whole systems work, is important. Safety culture needs to be highly conscious and uncompromising. Tester's ethics and motivation are important.
About the development 21(60) Organizational environment Not perhaps a mature software development. Naturally, since all this is a relatively new. Machine manufacturers have machine engineering culture. Nature of the software is not understood. Software development is not understood or managed. Software development process is poorly integrated into the overall product development process. This challenge has been understood in many companies. Many companies are developing rapidly. On the positive side: a healthy engineering culture is good for quality. Systematic action is appreciated. Processes, checklists, testing.
About the development 22(60) Development life cycle processes Historically waterfall or V model. Standards are based on V model. Agile methods becoming. Need re-thinking. The standards do not give much help. For example, Scrum requires adding many missing process parts that do not belong in the "Scrum culture". Industry needs guidance see one example of that: Vuori, M. 2011. Agile Development of Safety-Critical Software. Tampere University of Technology. Department of Software Systems. Report 14. 95 p. http://urn.fi/urn:nbn:fi:tty-2011061414702 Some researchers try to apply Scrum directly, but that has builtin problems for V&V.
Quality and quality management 23(60) Quality policy and safety policy Good quality and safety cultures are a requirement for good testing. A critical issue for both are the top management's true values, Influences the resources for testing and the development of testing Influences the possible compromises in the pressures of the projects. A quality policy is a written statement of how the top management sees the concept of quality, the importance of good quality and with what principles the company aims to reach it. ISO 9001 requires this. Such a policy should be present for safety and risk management, too.
Quality and quality management 24(60) Process maturity 1/2 Safety-critical development is by nature a somewhat processheavy and systematic activity. This is important, because the process requires co-operation between many parties and needs to be very auditable. And all this applies to testing within the process too. Development process is usually guided by safety standards and quality management system standards (like the ISO 9000 series [Wikipedia. ISO 9001.] and especially ISO 90003 for software development). Testing practices can also benefit from some external guidance, like the various testing maturity models, such as TMMi [The Test Maturity Model Integrated 2011] and TPI [Test Process Improvement 2009].
Quality and quality management 25(60) Process maturity 2/2 The syllabi of the global certification system ISTQB [2011] presents an approach to testing which is in many ways quite compatible with safety-critical development. Systematic. Process oriented. Clarity of roles and principles.
Quality and quality management 26(60) Rigorous quality & safety assurance Validation. Validation of safety requirements. Validation at the level of the overall system. Verification. Testing. Analyses (design, reliability, hazard). Review. Quality of the process important. Strictness of using methods. Standards give requirements. Safety lifecycle and methods used. Depends on the safety integrity levels (SIL). Verification and validation always for a well defined, particular configuration.
Development process elements 27(60) Requirement specifications This product is validated and "certified" against its requirements. Planned use. Associated hazards. Imaginable misuse. Hazards, risk analysis, reliability analysis give input data. Creation of safety requirements. Through the levels of the system: the overall system, electronics, software. When requirements change, "everything" must be reassessed.
Development process elements Basis for development and testing (a bit of a caricature) 28(60) Traditional development Requirement specification Quality plan / master test plan Testing plans Safety critical development Business requirement specification (what) Risk assessment (what) Safety requirement specification (what) Safety regulations and standards (how) Safety validation plan Verification plan Testing plans
Development process elements 29(60) Architecture is important 1/2 A variety of architectures. Hardware. Operational software architecture. Independent safety system architecture. Design requirements. Redundancy doubled, tripled elements. Diversity different design principles and implementation all don't fail in a same situation algorithms, languages, compilers. Fail safe a "blue screens of death" not allowed. => Plenty of things to test!
Development process elements 30(60) Architecture is important 2/2 As safety related systems have special requirements for their testing one must be careful with the systems so the subsystems are independent. Makes a difference in validation work.
Development process elements 31(60) Design analysis Analysis of how safe and reliable a design is. How it meets the safety requirements, how it can fail. FMEA / FMECA often used. http://en.wikipedia.org/wiki/failure_mode_and_effects_analysis Testers should participate, understand the techniques. For new designs and for changes. After the analysis, verification. Testing. Review.
Development process elements 32(60) Tracing Practically everything must be traced / linked forward and backward. From requirements to tests and back. To a given configuration / version. Supports decisions on what must be verified / validated when something changes. Supports finding out how some issues have been tackled. Need good tools to do that especially when using agile practices. Polarion ALM is one modern tool used in industry.
Development process elements Testing is only one means of assessing the system 33(60) In many contexts, testing can have a too large role in determining the quality and acceptability of a system. Analytic methods need to be used in assessing the system: Reviews. Failure and reliability analyses. Architecture assessment. Usability assessment and human error analysis. Formal verification. Etc Many of these are required by the safety standards. They all form the whole of project's quality and safety management. Not separate things, but closely connected and need to be utilised more together. For example, failure analysis should not just be a check of the system, but a task that produces new knowledge for testing.
IEC 61508 34(60) What do the standards require? Let's take a look at what kind of testing related actions IEC 61508-3 requires. Notes on tables R = Recommended HR = Highly recommended = practically mandatory On some level one must also do the things required for lower levels The standard series also gives requirements & guidance to how the methods should be used
IEC 61508-3 V&V process But first a look to the IEC 61508 V&V process context 35(60) E/E/PE system safety requirements specification Software safety requirements specification Validation Validated software Validation testing Software architecture Integration testing (components, subsystems and programmable electronics) E/E/PE system architecture Software system design Integration testing (module) Module design Module testing Verification relations are drawn with dashed lines -.-.- Coding
IEC 61508-3 requirements 36(60) IEC-61508-3 Failure Analysis Technique SIL 1 SIL 2 SIL 3 SIL 4 Cause consequence diagrams R R R R Event tree analysis R R R R Fault tree analysis R R R R Software functional failure analysis R R R R Count SIL 1 SIL 2 SIL 3 SIL 4 - R 4 4 4 4 HR
IEC 61508-3 requirements 37(60) IEC-61508-3 Modelling Technique SIL 1 SIL 2 SIL 3 SIL 4 Data flow diagrams R R R R Finite state machines R HR HR Formal methods R R HR Time Petri nets R HR HR Performance modelling R HR HR HR Prototyping/animation R R R R Structure diagrams R R R HR Count SIL 1 SIL 2 SIL 3 SIL 4-3 R 4 6 4 2 HR 1 3 6
IEC 61508-3 requirements 38(60) IEC-61508-3 Semi-formal methods Technique SIL 1 SIL 2 SIL 3 SIL 4 Logic/function block diagrams R R HR HR Sequence diagrams R R HR HR Data flow diagrams R R R R Finite state machines/state transition diagrams R R HR HR Time Petri nets R R HR HR Entity-relationship-attribute data models R R R R Message sequence charts R R R R Decision/truth tables R R HR HR UML R R R R Count SIL 1 SIL 2 SIL 3 SIL 4 - R 9 9 4 4 HR 5 5
IEC 61508-3 requirements IEC-61508-3 Dynamic analysis and testing 39(60) Technique SIL 1 SIL 2 SIL 3 SIL 4 Test case execution from boundary value analysis R HR HR HR Test case execution from error guessing R R R R Test case execution from error seeding R R R Test case execution from model-based test case generation R R HR HR Performance modelling R R R HR Equivalence classes and input partition testing R R R HR Structural test coverage (entry points) 100 HR HR HR HR Structural test coverage (statements) 100 R HR HR HR Structural test coverage (branches) 100 R R HR HR Structural test coverage (conditions, MC/DC) 100 R R R HR Count SIL 1 SIL 2 SIL 3 SIL 4-1 R 8 7 5 2 HR 1 3 5 8
IEC 61508-3 requirements IEC-61508-3 Functional and black box testing 40(60) Technique SIL 1 SIL 2 SIL 3 SIL 4 Test case execution from cause consequence diagrams R R Test case execution from model-based test case generation R R HR HR Prototyping/animation R R Equivalence classes and input partition testing, including boundary value analysis R HR HR HR Process simulation R R R R Count SIL 1 SIL 2 SIL 3 SIL 4-2 2 R 3 2 3 3 HR 1 2 2
IEC 61508-3 requirements IEC-61508-3 Performance testing 41(60) Technique SIL 1 SIL 2 SIL 3 SIL 4 Avalanche/stress testing R R HR HR Response timings and memory constraints HR HR HR HR Performance requirements HR HR HR HR Count SIL 1 SIL 2 SIL 3 SIL 4 - R 1 1 HR 2 2 3 3
IEC 61508-3 requirements 42(60) IEC-61508-3 Static analysis Technique SIL 1 SIL 2 SIL 3 SIL 4 Boundary value analysis R R HR HR Checklists R R R R Control flow analysis R HR HR HR Data flow analysis R HR HR HR Error guessing R R R R Formal inspections, including specific criteria R R HR HR Walk-through (software) R R R R Symbolic execution R R Design review HR HR HR HR Static analysis of run time error behaviour R R R HR Worst-case execution time analysis R R R R Count SIL 1 SIL 2 SIL 3 SIL 4-1 1 R 9 7 6 5 HR 1 3 5 6
IEC 61508-3 requirements IEC-61508-3 Overall 43(60) Area SIL 1 SIL 2 SIL 3 SIL 4 Failure analysis (R / HR) 4/- 4/- 4/- 4/- Modelling 4/- 6/1 4/3 2/6 Semi-formal methods 9/- 9/- 4/5 4/5 Dynamic analysis and testing 8/1 7/3 5/5 2/8 Functional and black-box testing 3/- 2/1 3/2 3/2 Performance testing 1/2 1/2 -/3 -/3 Static analysis 9/1 7/3 6/5 5/6 Sum: R + HR 42 46 49 50 R 38 36 26 20 HR 4 10 23 30
IEC 61508-3 requirements 44(60) Quite heavy at higher SIL levels? Yes. A company has a long road to SIL3 level processes. Lessons: When there is a possibility to choose which standards to adhere to, think carefully Other standards may be easier to live with than IEC 61508. By design choices in the overall system one can keep the SIL levels down and thus keep the testing requirements (and software and electronics requirements) manageable. Put emphasis on architecture so that the safety-critical system is as small and simple as possible and thus easier to develop and test. When developing new testing standards they should allow for riskbased tailoring of how things are done. IEEE 829 (2008) does that, the coming ISO/IEC 29119 perhaps not.
Testing Sources of test conditions and test cases 1/2 45(60) There needs to be various sources for the test conditions and test cases as shown in. It is a generally accepted fact that no requirement specification can be complete or sufficient if it were, it would be too large to be usable and require too much resources to create and to maintain. Thus, many of the sources of test conditions and test cases come from many other sources and are often informal. Companies have experiences from developing similar systems and testers know what kinds of issues need to be tested. Design analyses during the development bring many factors to light. Creating good tests is a continuous experience and restricting test to just any set of formal requirements would be a serious error.
Testing Sources of test conditions and test cases 2/2 46(60) Requirements specified in the project Hazards and risks Safety assessment Generic requirements for the particular type of system and software Some sources of test conditions and test cases Standards Real, observed behaviour Exploratory testing Failure analysis and other design analysis Experience Tester skill and know-how
Testing 47(60) Negative testing emphasised The importance of negative tests needs to be emphasised. Accidents obviously happen when something goes wrong. Safety-critical system need to be able to handle any disturbances. System elements should never assume anything, but be ready to handle everything. Testing must be harsh try to break the software. Need very good coverage with conditions. As systems become larger and more complicated, this will become more and more important.
Testing 48(60) Risk-based testing Risk-based testing emphasises testing more thoroughly those system functions that have the biggest risk. That approach is an obvious choice for safety-critical systems. Testing should start on those functions that most contribute to safety. Assigning risk levels on requirements is needed for this a general SIL number is not sufficient. Testing should begin with the highest risk items and then move to items that have a lower risk level. Of course, this implies that the high risk items should be designed and implemented before others, to be testable first. But everything that contributes to a risk must be validated in some way (if the risk is not seen "acceptable").
Testing 49(60) Realistic traditional test automation Test automation can be important in making testing more efficient. But one should never rely too much on test automation. It needs always to be combined with good manual testing. Test automation is most at home in the basic regression testing where after any changes it can be verified that there are no adverse effects from the changes to the rest of the system. Model-based testing is an exception.
Testing 50(60) Model-based testing for robustness Required by latest standards for high-risk systems. Model-based testing gives the opportunity to test all variations of system behaviour good coverage. Great for producing a robust system a necessity for safety. Allows for randomness. Good for simulations of interactions between system elements. Due to high abstraction level suits systems with diversity 6 redundance in implementation. A new thing to most companies
Testing 51(60) Exploratory testing? Standards require a plan and documentation of what has been tested. It may depend on the consultants doing a certification whether exploratory testing is sufficient for validation. For "development testing" certainly a great thing if done properly. Supports learning of how the system really behaves & finding new requirements. Would need tailored methods. Some approaches would need cultural adaptation ("tourist tours" should perhaps not be used in serious development). Test logs are important anyway need to show that something has been tested and how.
Testing Diversity in testing practices and methods 52(60) Good testing is a rich collection of different approaches and methods. In the field of hazard and reliability analysis it has long ago been noted that one single method and approach can produce only so much information and therefore, multiple approaches are needed. The same applies to testing. One should practice many approaches and be wary of limiting itself to any "school of thought". Systematic testing, agile testing, test automation and manual testing, model-based testing and others can find a place in the same project. Variations in test design and execution. More than one tester with varying educational and testing course background is a good thing.
Testing 53(60) Documentation Safety standards require documentation of test designs and plans. But what counts as such? Do we need traditional documents at all levels? Meeting minutes? Detail plans? It may depend on the assessor Flipcharts or photographed whiteboards with metadata (date, authors) can count as valid documentation. Hand-drawn, team produced mind maps are more reliable than carefully edited ones... Note: Good documentation is risk management for testers, developers and managers at it demonstrates that they have considered issues.
Testers 54(60) Competence In safety-related work one must be "qualified". Professional skills. Training in safety matters. Ability to participate in design analyses FMEA etc. Training in the methods used. Understanding or reliability. Understanding of human errors and workers' / users' behaviour. Understanding of all the diverse technology. Due to diversity in technology, a tester's personal "toolbox" needs to be large and versatile. Experience is considered very important. Sense of responsibility. Systematic approach. "Strong character" no weak compromises in safety matters.
Testers 55(60) Suitable independence Testers and the testing process need to have a level of independence from development so that there are not any adverse psychological influences on the test designs or interpretation of test results. This is an important issue already in verification testing and obviously critical on validation testing. Standards give requirements for personnel doing validation testing. Alternatives include: Independent person from same unit. Independent person from another company unit. Independent person from another company. So, we need to find a balance with testing that is integrated with development and "official" validation testing.
Testers...and co-operation and collaboration between designers and testers 56(60) Independence should not mean lack of co-operation. It should be clear that developers and testers need to do cooperation in all development process phases and work in close collaboration in some. For example, failure analysis is a task where all input from varying sources is critical.
Finally 57(60) Top 10 points of conclusion 1. Safety first need to know the risks. 2. Must understand the overall system. 3. Software developers and testers must have all information. 4. High-risk systems require highly advanced verification and validation. 5. There is plenty of work. 6. Keep systems simple. 7. Diversity is a plus in finding bugs too. 8. Testers must be trained properly and must gain experience before trusted to do validation testing. 9. Standards can be challenging so see if there are alternatives for those. 10.Mostly this is just good testing with more strictness than usually.
58(60) More information The Ohjelmaturva publication "Safety Process Patterns In the Context of IEC 61508-3" [Vuori et al. 2011] includes many testing-related patterns. The publication "Agile Development of Safety-Critical Software" [Vuori 2011] contains discussion about testing when agile software development methods are used.
59(60) References 1/2 Vuori, M. 2011. Agile Development of Safety-Critical Software. Tampere University of Technology. Department of Software Systems. Report 14. 95 p. http://urn.fi/urn:nbn:fi:tty-2011061414702 Vuori, M., Virtanen, H., Koskinen, J. & Katara, M. 2011. Safety Process Patterns In the Context of IEC 61508-3. Tampere University of Technology. Department of Software Systems. Report 15. 128 p. http://urn.fi/urn:nbn:fi:tty-2011061414701 ISTQB. 2011. International Software Testing Qualifications Board. [Referenced 2011-05-30]. http://istqb.org/display/istqb/home
60(60) References 2/2 Test Process Improvement. 2009. A Step-by-step guide for improving your test process. http://www.sogeti.ie/documents/publications/sogeti_testing_test_process_impro vement_tpi_step_by_step_guide_150609.pdf Wikipedia. ISO 9001. [Referenced 2011-05-30]. http://en.wikipedia.org/wiki/iso_9000 The Test Maturity Model Integrated (TMMi). 2011. TMMi Foundation. [Referenced 2011-05-30] http://www.tmmifoundation.org/html/tmmiref.html Wikipedia. [Referenced 2012-11-26] Failure mode and effects analysis http://en.wikipedia.org/wiki/failure_mode_and_effects_analysis