1 1(60) Testing of safety-critical software some principles Emerging Trends in Software Testing: autumn 2012 Matti Vuori, Tampere University of Technology
2 Contents 1/4 Topics of this lecture 6 What does "safety critical" mean? 7 What kind of system types do we think of? 8 Goals of testing 9 How safe is safe: SIL Safety Integrity Level 10 (Safety) standards are important 11 (Safety) standards are important Understanding the overall system 13 Central quality characteristics 14 Long life cycle of machinery and industrial systems 17 Some factors affecting the ways and performance of testing. 18 Development is demanding 19 Sense of responsibility of the hazards 20
3 Contents 2/4 Organizational environment 21 Development life cycle processes 22 Quality policy and safety policy 23 Process maturity 24 Rigorous quality & safety assurance 26 Requirement specifications 27 Basis for development and testing (a bit of a caricature) 28 Architecture is important 29 Design analysis 31 Tracing 32 Testing is only one means of assessing the system 33 What do the standards require? 34 But first a look to the IEC V&V process context 35
4 Contents 3/4 IEC Failure Analysis 36 IEC Modelling 37 IEC Semi-formal methods 38 IEC Dynamic analysis and testing 39 IEC Functional and black box testing 40 IEC Performance testing 41 IEC Static analysis 42 IEC Overall 43 Quite heavy at higher SIL levels? 44 Sources of test conditions and test cases 45 Negative testing emphasised 47 Risk-based testing 48 Realistic traditional test automation 49
5 Contents 4/4 Model-based testing for robustness 50 Exploratory testing? 51 Diversity in testing practices and methods 52 Documentation 53 Competence 54 Suitable independence 55...and co-operation and collaboration between designers and testers 56 Top 10 points of conclusion 57 More information 58 References 59
6 6(60) Topics of this lecture i What is safety critical software and what is essential about them? What is their testing like and should be like? What do safety standards require of testing them? It is assumed that the audience knows about testing. We don't talk about details or basics (hopefully), but try to reflect the environment of safety-critical development so people can transfer and adjust their knowledge and skills to it.
7 7(60) What does "safety critical" mean? The system has hazards potential to cause accidents, losses to property, money. Safety is an essential requirement. Software has an influence on safety. There are formal requirements for safety regulations, standards. There are specific actions for assuring safety and those actions are expected and required. It would be a risk to the manufacturer also, if safety was not sufficient (accidents, lawsuits). Safety-critical software can be such in itself when it provides information used in situations where a failure would be dangerous, or when it is used to control something (like a machine) or when it is used as a safety system that does something when something else does not work correctly.
8 What kind of system types do we think of? 8(60) The actual systems could hospital systems, automobiles, nuclear powerplants To have a somewhat similar picture in our heads, let's think of "machines" in factories or on wheels construction or forests.
9 9(60) Goals of testing Find any problems that could cause hazards. Verify that any safety controls work as intended and reliably in case of a malfunction or human error. Provide proof that development of the system has been done as required by laws and safety standards. Validate that the overall system meets its safety requirements and is as safe as can be expected based on that. Nothing is ever absolutely safe. Produce information to management about the system's risk level and acceptability. Produce documentation to show all 9(60) that later on if needed. A lawsuit after an accident, for example.
10 Safety management How safe is safe: SIL Safety Integrity Level 10(60) A key term for SIL - Safety Integrity Level "How reliably safety functions must work". Range from 1 to 4. In practice, for example, machines are often SIL2-level, the railway systems SIL3, nuclear power could 4 etc... Defined for the overall system and derived from that, to the software In standards, the SIL level defines, what kinds of development methods, design solutions, validation methods etc. must be used. 1 and 2 are easy. At the higher levels for example model-based development and testing may be needed SIL 4 is a very challenging. Note: SIL is just one of such measure. Many domains and standards have alternative measures for safety levels.
11 Safety management 11(60) (Safety) standards are important 1/3 Standards give high requirements for the development process. Process phases, development and deployment lifecycle. Methods and tools. Quality assurance. Verification. Validation. People's competence need to be trained. The requirements depend on the level of safety integrity level. Provide technical requirements of the design solutions. IEC series a good example. A generic standard series for development of safety critical systems. Own standard parts for different levels of abstraction of the system (overall system, programmable electronics, software). Vocabulary. Descriptions of the methods required or recommended.
12 Safety management 12(60) (Safety) standards are important 22 IEC parts: IEC TR Functional safety and IEC IEC General requirements. IEC Requirements for E/E/PE safety-related systems. IEC Software requirements. IEC Definitions and abbreviations. IEC Examples and methods for the determination of safety integrity levels. IEC Guidelines on the application of IEC and IEC IEC Overview of techniques and measures is used for example in automation systems development. Some domains (like health care system and auto industry) have similar, domain-specific standards.
13 Understanding the system 13(60) Understanding the overall system Where the program is used, in which environment, in which devices. How is the system used. Task analyses how systems are operated. Process modelling. Requirements flow top-down. Safety of all things, and their acceptability, assessed at the level of the overall system. Safety analyses, human error analysis, reliability analyses etc needed. One needs good context knowledge, good information. One must understand the role of ones own doing and its effects How testing of something relates to the whole.
14 Understanding the system 14(60) Central quality characteristics 1/3 Safety. Of the overall system. No compromises. Information security. Correct data. Sabotage. Relevant in expanding the systems a machine as part of an information system, a part of a logistic system. Currently not considered enough. Robustness and reliability. The system can handle "anything" in varying environments. Don't assume anything about how other components work. Problem-free action in long durations of use.
15 Understanding the system 15(60) Central quality characteristics 2/3 Usability. Guiding the user to safe operation. Handling of human errors. Testability. In order for verification and validation to be easy and successful. In order to be able to test with a number of techniques, for example with model-based techniques. Standardised interfaces, ability to simulate etc. Analysability. Analysis of plans, designs is important. Modularity. Packaged modules. Changeability, replaceability during a long life cycle of machines. Redundancy, diversity duplication of sub-systems using different implementations.
16 Understanding the system 16(60) Central quality characteristics 3/3 Extensibility. New things that are added during a long life cycle. But plug-ins are dangerous need to know and control what the system contains. Limited functionality. Everything in system is relevant, validated and verified. No vague extras.
17 Understanding the system Long life cycle of machinery and industrial systems 17(60) Dozens of years of use. How to renew technology after ten years? And ensure that the revised whole is ok. How to keep the configuration in good shape (adding, changing things what are we developing and testing for?). How to test, if the only device in some configuration has been build years ago, is located at customer and even the components are not available anymore for testing. Simulators, standards Systems need to done right from the beginning.
18 About the development Some factors affecting the ways and performance of testing. 18(60) The status of software in the organisational culture Arrangements, environments and tools skills for testing The role of software development in product development projects Factors affecting the performance of software testing Professionalism of testing culture and testers Resources for software development The competences and maturity of software development units
19 About the development 19(60) Development is demanding Need to understand the overall system. Validation and certification in the context of the overall system. Safety must not be compromised. Quality may not be compromised. The overall process is a demanding. Technologies, methods. Plenty of process requirements. We need competence and experience. Hazards cause responsibility. In machine systems, simulation is important. Testing with actual experiment is difficult to arrange there may be only one, and that is being build at the same time.
20 About the development 20(60) Sense of responsibility of the hazards The safety-critical software is a critical part of the overall system, which may be dangerous. Leading to injuries and accidents. The software can be located operational systems of safety system. Danger causes responsibility. Understanding of what can be followed, unless the whole systems work, is important. Safety culture needs to be highly conscious and uncompromising. Tester's ethics and motivation are important.
21 About the development 21(60) Organizational environment Not perhaps a mature software development. Naturally, since all this is a relatively new. Machine manufacturers have machine engineering culture. Nature of the software is not understood. Software development is not understood or managed. Software development process is poorly integrated into the overall product development process. This challenge has been understood in many companies. Many companies are developing rapidly. On the positive side: a healthy engineering culture is good for quality. Systematic action is appreciated. Processes, checklists, testing.
22 About the development 22(60) Development life cycle processes Historically waterfall or V model. Standards are based on V model. Agile methods becoming. Need re-thinking. The standards do not give much help. For example, Scrum requires adding many missing process parts that do not belong in the "Scrum culture". Industry needs guidance see one example of that: Vuori, M Agile Development of Safety-Critical Software. Tampere University of Technology. Department of Software Systems. Report p. Some researchers try to apply Scrum directly, but that has builtin problems for V&V.
23 Quality and quality management 23(60) Quality policy and safety policy Good quality and safety cultures are a requirement for good testing. A critical issue for both are the top management's true values, Influences the resources for testing and the development of testing Influences the possible compromises in the pressures of the projects. A quality policy is a written statement of how the top management sees the concept of quality, the importance of good quality and with what principles the company aims to reach it. ISO 9001 requires this. Such a policy should be present for safety and risk management, too.
24 Quality and quality management 24(60) Process maturity 1/2 Safety-critical development is by nature a somewhat processheavy and systematic activity. This is important, because the process requires co-operation between many parties and needs to be very auditable. And all this applies to testing within the process too. Development process is usually guided by safety standards and quality management system standards (like the ISO 9000 series [Wikipedia. ISO 9001.] and especially ISO for software development). Testing practices can also benefit from some external guidance, like the various testing maturity models, such as TMMi [The Test Maturity Model Integrated 2011] and TPI [Test Process Improvement 2009].
25 Quality and quality management 25(60) Process maturity 2/2 The syllabi of the global certification system ISTQB  presents an approach to testing which is in many ways quite compatible with safety-critical development. Systematic. Process oriented. Clarity of roles and principles.
26 Quality and quality management 26(60) Rigorous quality & safety assurance Validation. Validation of safety requirements. Validation at the level of the overall system. Verification. Testing. Analyses (design, reliability, hazard). Review. Quality of the process important. Strictness of using methods. Standards give requirements. Safety lifecycle and methods used. Depends on the safety integrity levels (SIL). Verification and validation always for a well defined, particular configuration.
27 Development process elements 27(60) Requirement specifications This product is validated and "certified" against its requirements. Planned use. Associated hazards. Imaginable misuse. Hazards, risk analysis, reliability analysis give input data. Creation of safety requirements. Through the levels of the system: the overall system, electronics, software. When requirements change, "everything" must be reassessed.
28 Development process elements Basis for development and testing (a bit of a caricature) 28(60) Traditional development Requirement specification Quality plan / master test plan Testing plans Safety critical development Business requirement specification (what) Risk assessment (what) Safety requirement specification (what) Safety regulations and standards (how) Safety validation plan Verification plan Testing plans
29 Development process elements 29(60) Architecture is important 1/2 A variety of architectures. Hardware. Operational software architecture. Independent safety system architecture. Design requirements. Redundancy doubled, tripled elements. Diversity different design principles and implementation all don't fail in a same situation algorithms, languages, compilers. Fail safe a "blue screens of death" not allowed. => Plenty of things to test!
30 Development process elements 30(60) Architecture is important 2/2 As safety related systems have special requirements for their testing one must be careful with the systems so the subsystems are independent. Makes a difference in validation work.
31 Development process elements 31(60) Design analysis Analysis of how safe and reliable a design is. How it meets the safety requirements, how it can fail. FMEA / FMECA often used. Testers should participate, understand the techniques. For new designs and for changes. After the analysis, verification. Testing. Review.
32 Development process elements 32(60) Tracing Practically everything must be traced / linked forward and backward. From requirements to tests and back. To a given configuration / version. Supports decisions on what must be verified / validated when something changes. Supports finding out how some issues have been tackled. Need good tools to do that especially when using agile practices. Polarion ALM is one modern tool used in industry.
33 Development process elements Testing is only one means of assessing the system 33(60) In many contexts, testing can have a too large role in determining the quality and acceptability of a system. Analytic methods need to be used in assessing the system: Reviews. Failure and reliability analyses. Architecture assessment. Usability assessment and human error analysis. Formal verification. Etc Many of these are required by the safety standards. They all form the whole of project's quality and safety management. Not separate things, but closely connected and need to be utilised more together. For example, failure analysis should not just be a check of the system, but a task that produces new knowledge for testing.
34 IEC (60) What do the standards require? Let's take a look at what kind of testing related actions IEC requires. Notes on tables R = Recommended HR = Highly recommended = practically mandatory On some level one must also do the things required for lower levels The standard series also gives requirements & guidance to how the methods should be used
35 IEC V&V process But first a look to the IEC V&V process context 35(60) E/E/PE system safety requirements specification Software safety requirements specification Validation Validated software Validation testing Software architecture Integration testing (components, subsystems and programmable electronics) E/E/PE system architecture Software system design Integration testing (module) Module design Module testing Verification relations are drawn with dashed lines Coding
36 IEC requirements 36(60) IEC Failure Analysis Technique SIL 1 SIL 2 SIL 3 SIL 4 Cause consequence diagrams R R R R Event tree analysis R R R R Fault tree analysis R R R R Software functional failure analysis R R R R Count SIL 1 SIL 2 SIL 3 SIL 4 - R HR
37 IEC requirements 37(60) IEC Modelling Technique SIL 1 SIL 2 SIL 3 SIL 4 Data flow diagrams R R R R Finite state machines R HR HR Formal methods R R HR Time Petri nets R HR HR Performance modelling R HR HR HR Prototyping/animation R R R R Structure diagrams R R R HR Count SIL 1 SIL 2 SIL 3 SIL 4-3 R HR 1 3 6
38 IEC requirements 38(60) IEC Semi-formal methods Technique SIL 1 SIL 2 SIL 3 SIL 4 Logic/function block diagrams R R HR HR Sequence diagrams R R HR HR Data flow diagrams R R R R Finite state machines/state transition diagrams R R HR HR Time Petri nets R R HR HR Entity-relationship-attribute data models R R R R Message sequence charts R R R R Decision/truth tables R R HR HR UML R R R R Count SIL 1 SIL 2 SIL 3 SIL 4 - R HR 5 5
39 IEC requirements IEC Dynamic analysis and testing 39(60) Technique SIL 1 SIL 2 SIL 3 SIL 4 Test case execution from boundary value analysis R HR HR HR Test case execution from error guessing R R R R Test case execution from error seeding R R R Test case execution from model-based test case generation R R HR HR Performance modelling R R R HR Equivalence classes and input partition testing R R R HR Structural test coverage (entry points) 100 HR HR HR HR Structural test coverage (statements) 100 R HR HR HR Structural test coverage (branches) 100 R R HR HR Structural test coverage (conditions, MC/DC) 100 R R R HR Count SIL 1 SIL 2 SIL 3 SIL 4-1 R HR
40 IEC requirements IEC Functional and black box testing 40(60) Technique SIL 1 SIL 2 SIL 3 SIL 4 Test case execution from cause consequence diagrams R R Test case execution from model-based test case generation R R HR HR Prototyping/animation R R Equivalence classes and input partition testing, including boundary value analysis R HR HR HR Process simulation R R R R Count SIL 1 SIL 2 SIL 3 SIL R HR 1 2 2
41 IEC requirements IEC Performance testing 41(60) Technique SIL 1 SIL 2 SIL 3 SIL 4 Avalanche/stress testing R R HR HR Response timings and memory constraints HR HR HR HR Performance requirements HR HR HR HR Count SIL 1 SIL 2 SIL 3 SIL 4 - R 1 1 HR
42 IEC requirements 42(60) IEC Static analysis Technique SIL 1 SIL 2 SIL 3 SIL 4 Boundary value analysis R R HR HR Checklists R R R R Control flow analysis R HR HR HR Data flow analysis R HR HR HR Error guessing R R R R Formal inspections, including specific criteria R R HR HR Walk-through (software) R R R R Symbolic execution R R Design review HR HR HR HR Static analysis of run time error behaviour R R R HR Worst-case execution time analysis R R R R Count SIL 1 SIL 2 SIL 3 SIL R HR
43 IEC requirements IEC Overall 43(60) Area SIL 1 SIL 2 SIL 3 SIL 4 Failure analysis (R / HR) 4/- 4/- 4/- 4/- Modelling 4/- 6/1 4/3 2/6 Semi-formal methods 9/- 9/- 4/5 4/5 Dynamic analysis and testing 8/1 7/3 5/5 2/8 Functional and black-box testing 3/- 2/1 3/2 3/2 Performance testing 1/2 1/2 -/3 -/3 Static analysis 9/1 7/3 6/5 5/6 Sum: R + HR R HR
44 IEC requirements 44(60) Quite heavy at higher SIL levels? Yes. A company has a long road to SIL3 level processes. Lessons: When there is a possibility to choose which standards to adhere to, think carefully Other standards may be easier to live with than IEC By design choices in the overall system one can keep the SIL levels down and thus keep the testing requirements (and software and electronics requirements) manageable. Put emphasis on architecture so that the safety-critical system is as small and simple as possible and thus easier to develop and test. When developing new testing standards they should allow for riskbased tailoring of how things are done. IEEE 829 (2008) does that, the coming ISO/IEC perhaps not.
45 Testing Sources of test conditions and test cases 1/2 45(60) There needs to be various sources for the test conditions and test cases as shown in. It is a generally accepted fact that no requirement specification can be complete or sufficient if it were, it would be too large to be usable and require too much resources to create and to maintain. Thus, many of the sources of test conditions and test cases come from many other sources and are often informal. Companies have experiences from developing similar systems and testers know what kinds of issues need to be tested. Design analyses during the development bring many factors to light. Creating good tests is a continuous experience and restricting test to just any set of formal requirements would be a serious error.
46 Testing Sources of test conditions and test cases 2/2 46(60) Requirements specified in the project Hazards and risks Safety assessment Generic requirements for the particular type of system and software Some sources of test conditions and test cases Standards Real, observed behaviour Exploratory testing Failure analysis and other design analysis Experience Tester skill and know-how
47 Testing 47(60) Negative testing emphasised The importance of negative tests needs to be emphasised. Accidents obviously happen when something goes wrong. Safety-critical system need to be able to handle any disturbances. System elements should never assume anything, but be ready to handle everything. Testing must be harsh try to break the software. Need very good coverage with conditions. As systems become larger and more complicated, this will become more and more important.
48 Testing 48(60) Risk-based testing Risk-based testing emphasises testing more thoroughly those system functions that have the biggest risk. That approach is an obvious choice for safety-critical systems. Testing should start on those functions that most contribute to safety. Assigning risk levels on requirements is needed for this a general SIL number is not sufficient. Testing should begin with the highest risk items and then move to items that have a lower risk level. Of course, this implies that the high risk items should be designed and implemented before others, to be testable first. But everything that contributes to a risk must be validated in some way (if the risk is not seen "acceptable").
49 Testing 49(60) Realistic traditional test automation Test automation can be important in making testing more efficient. But one should never rely too much on test automation. It needs always to be combined with good manual testing. Test automation is most at home in the basic regression testing where after any changes it can be verified that there are no adverse effects from the changes to the rest of the system. Model-based testing is an exception.
50 Testing 50(60) Model-based testing for robustness Required by latest standards for high-risk systems. Model-based testing gives the opportunity to test all variations of system behaviour good coverage. Great for producing a robust system a necessity for safety. Allows for randomness. Good for simulations of interactions between system elements. Due to high abstraction level suits systems with diversity 6 redundance in implementation. A new thing to most companies
51 Testing 51(60) Exploratory testing? Standards require a plan and documentation of what has been tested. It may depend on the consultants doing a certification whether exploratory testing is sufficient for validation. For "development testing" certainly a great thing if done properly. Supports learning of how the system really behaves & finding new requirements. Would need tailored methods. Some approaches would need cultural adaptation ("tourist tours" should perhaps not be used in serious development). Test logs are important anyway need to show that something has been tested and how.
52 Testing Diversity in testing practices and methods 52(60) Good testing is a rich collection of different approaches and methods. In the field of hazard and reliability analysis it has long ago been noted that one single method and approach can produce only so much information and therefore, multiple approaches are needed. The same applies to testing. One should practice many approaches and be wary of limiting itself to any "school of thought". Systematic testing, agile testing, test automation and manual testing, model-based testing and others can find a place in the same project. Variations in test design and execution. More than one tester with varying educational and testing course background is a good thing.
53 Testing 53(60) Documentation Safety standards require documentation of test designs and plans. But what counts as such? Do we need traditional documents at all levels? Meeting minutes? Detail plans? It may depend on the assessor Flipcharts or photographed whiteboards with metadata (date, authors) can count as valid documentation. Hand-drawn, team produced mind maps are more reliable than carefully edited ones... Note: Good documentation is risk management for testers, developers and managers at it demonstrates that they have considered issues.
54 Testers 54(60) Competence In safety-related work one must be "qualified". Professional skills. Training in safety matters. Ability to participate in design analyses FMEA etc. Training in the methods used. Understanding or reliability. Understanding of human errors and workers' / users' behaviour. Understanding of all the diverse technology. Due to diversity in technology, a tester's personal "toolbox" needs to be large and versatile. Experience is considered very important. Sense of responsibility. Systematic approach. "Strong character" no weak compromises in safety matters.
55 Testers 55(60) Suitable independence Testers and the testing process need to have a level of independence from development so that there are not any adverse psychological influences on the test designs or interpretation of test results. This is an important issue already in verification testing and obviously critical on validation testing. Standards give requirements for personnel doing validation testing. Alternatives include: Independent person from same unit. Independent person from another company unit. Independent person from another company. So, we need to find a balance with testing that is integrated with development and "official" validation testing.
56 Testers...and co-operation and collaboration between designers and testers 56(60) Independence should not mean lack of co-operation. It should be clear that developers and testers need to do cooperation in all development process phases and work in close collaboration in some. For example, failure analysis is a task where all input from varying sources is critical.
57 Finally 57(60) Top 10 points of conclusion 1. Safety first need to know the risks. 2. Must understand the overall system. 3. Software developers and testers must have all information. 4. High-risk systems require highly advanced verification and validation. 5. There is plenty of work. 6. Keep systems simple. 7. Diversity is a plus in finding bugs too. 8. Testers must be trained properly and must gain experience before trusted to do validation testing. 9. Standards can be challenging so see if there are alternatives for those. 10.Mostly this is just good testing with more strictness than usually.
58 58(60) More information The Ohjelmaturva publication "Safety Process Patterns In the Context of IEC " [Vuori et al. 2011] includes many testing-related patterns. The publication "Agile Development of Safety-Critical Software" [Vuori 2011] contains discussion about testing when agile software development methods are used.
59 59(60) References 1/2 Vuori, M Agile Development of Safety-Critical Software. Tampere University of Technology. Department of Software Systems. Report p. Vuori, M., Virtanen, H., Koskinen, J. & Katara, M Safety Process Patterns In the Context of IEC Tampere University of Technology. Department of Software Systems. Report p. ISTQB International Software Testing Qualifications Board. [Referenced ].
60 60(60) References 2/2 Test Process Improvement A Step-by-step guide for improving your test process. vement_tpi_step_by_step_guide_ pdf Wikipedia. ISO [Referenced ]. The Test Maturity Model Integrated (TMMi) TMMi Foundation. [Referenced ] Wikipedia. [Referenced ] Failure mode and effects analysis
1(37) Agile development of safety-critical software while meetings standards' requirements Matti Vuori, Tampere University of Technology 2011-11-04 Contents 1/2 A study in Ohjelmaturva 4 Tendency to be
Software Engineering Prof. N. L. Sarda Computer Science & Engineering Indian Institute of Technology, Bombay Lecture - 2 Introduction to Software Engineering Challenges, Process Models etc (Part 2) This
Model-Based Testing and Formal Verification in IEC 61508-3 ed2.0 Mika Katara Tampere University of Technology Department of Software Systems 2 Outline Motivation IEC 61508: Verification & Validation How
T-76.5613 Software Testing and Quality Assurance Lecture 2, 13.9.2006 Basic Testing Concepts and Terminology Juha Itkonen SoberIT Contents Realities and principles of Testing terminology and basic concepts
IEC 61508 Overview Report A Summary of the IEC 61508 Standard for Functional Safety of Electrical/Electronic/Programmable Electronic Safety-Related Systems exida Sellersville, PA 18960, USA +1-215-453-1720
II.2 Life Cycle and Safety Safety Life Cycle: The necessary activities involving safety-related systems, occurring during a period of time that starts at the concept phase of a project and finishes when
Software testing cmsc435-1 Objectives To discuss the distinctions between validation testing and defect testing To describe the principles of system and component testing To describe strategies for generating
61508-3 ª IEC: 1997 1 Version 12.0 05/12/97 COMMISSION CEI ELECTROTECHNIQUE IEC INTERNATIONALE 61508-3 INTERNATIONAL ELECTROTECHNICAL COMMISSION Functional safety of electrical/electronic/ programmable
Risk Analysis VI 513 Design of automatic testing tool for railway signalling systems software safety assessment J.-G. Hwang 1, H.-J. Jo 1 & H.-S. Kim 2 1 Train Control Research Team, Korea Railroad Research
Version 2012 Copyright Notice This document may be copied in its entirety, or extracts made, if the source is acknowledged. Copyright (hereinafter called ISTQB ). Advanced Level Working Group: Mike Smith
ATAC Model-based testing in modern agile software development How to integrate it into the development process? Project number: ITEA 2 10037 Edited by: Matti Vuori, TUT Date: 2014/03/18 Document version
Reduce Medical Device Compliance Costs with Best Practices firstname.lastname@example.org 1 Agenda Medical Software Certification How new is Critical Software Certification? What do we need to do? What Best Practises
Software Process for QA Basic approaches & alternatives CIS 610, W98 / M Young 1/7/98 1 This introduction and overview is intended to provide some basic background on software process (sometimes called
Advanced Test Manager E-learning Course Outline General Description This course provides test managers with advanced skills in test estimation, test planning, test monitoring, and test control. Attendees
1. Overview 1. Overview...1 1.1 Total programme structure...1 1.2 Topics covered in module...2 1.3 Examples of SW eng. practice in some industrial sectors...4 1.3.1 European Space Agency (ESA), software
Preprint for conference proceedings for The Second World Congress on Software Quality, Yokohama, Sept 25.- 29, 2000. http://www.calpoly.edu/~pmcquaid/2wcsq Safety critical software and development productivity
TS Lockhart, Director of Engineering Moore Industries-International, Inc. Vetting Smart Instruments for the Nuclear Industry Moore Industries-International, Inc. is a world leader in the design and manufacture
Controlling Risks Safety Lifecycle Objective Introduce the concept of a safety lifecycle and the applicability and context in safety systems. Lifecycle Management A risk based management plan for a system
Certified Software Quality Engineer (CSQE) Body of Knowledge The topics in this Body of Knowledge include additional detail in the form of subtext explanations and the cognitive level at which the questions
PART A Questions 1. Mention any two software process models. 2. Define risk management. 3. What is a module? 4. What do you mean by requirement process? 5. Define integration testing. 6. State the main
SDLA-312 ISA Security Compliance Institute Security Development Lifecycle Assurance - Security Development Lifecycle Assessment v3.0 Lifecycle Phases Number Phase Name Description PH1 Security Management
Chapter 5 29 Verification and Validation of Software Components and Component Based Christina Wallin Industrial Information Technology Software Engineering Processes ABB Corporate Research email@example.com
Test Engineering Foundation Course Outline General Description This course provides test engineers and test managers with the essential ideas, processes, tools and skills they need in order to set themselves
Standard for Software Component Testing Working Draft 3.4 Date: 27 April 2001 produced by the British Computer Society Specialist Interest Group in Software Testing (BCS SIGIST) Copyright Notice This document
Lecture 8 About Quality and Quality Management Systems Kari Systä 10.03.2014 10.03.2014 TIE-21100/21106; K.Systä 1 Content of today s lecture Two weeks ago we discussed about testing and inspections, that
Module 10 Coding and Testing Lesson 26 Debugging, Integration and System Testing Specific Instructional Objectives At the end of this lesson the student would be able to: Explain why debugging is needed.
What is a life cycle model? Framework under which a software product is going to be developed. Defines the phases that the product under development will go through. Identifies activities involved in each
Best-Practice Software Engineering: Software Processes to Support Project Success Dietmar Winkler Vienna University of Technology Institute of Software Technology and Interactive Systems Dietmar.Winkler@qse.ifs.tuwien.ac.at
ASSESSMENT OF THE ISO 26262 STANDARD, ROAD VEHICLES FUNCTIONAL SAFETY Dr. Qi Van Eikema Hommes SAE 2012 Government/Industry Meeting January 25, 2012 1 Outline ISO 26262 Overview Scope of the Assessment
ISTQB Certified Tester Foundation Level Version 2015 American Copyright Notice This document may be copied in its entirety, or extracts made, if the source is acknowledged. #1 When test cases are designed
CS 1632 SOFTWARE QUALITY ASSURANCE 2 Marks Sample Questions and Answers 1. Define quality. Quality is the degree of goodness of a product or service or perceived by the customer. Quality concept is the
6. Software Lifecycle Models A software lifecycle model is a standardised format for planning organising, and running a new development project. Hundreds of different kinds of models are known and used.
Formal Software Testing Terri Grenda, CSTE IV&V Testing Solutions, LLC www.ivvts.com Scope of Testing Find defects early Remove defects prior to production Identify Risks Unbiased opinion When Should Testing
SOFTWARE DEVELOPMENT STANDARD FOR SPACECRAFT Mar 31, 2014 Japan Aerospace Exploration Agency This is an English translation of JERG-2-610. Whenever there is anything ambiguous in this document, the original
0305203 0305280 0305301 0305302 Software Engineering/Courses Description Introduction to Software Engineering Prerequisite: 0306211(Computer Programming 2). This course introduces students to the problems
THE ONLY BOOK CAN SIMPLY LEARN SOFTWARE TESTING! Page 1 Contents ABOUT THE AUTHOR... 3 1. Introduction To Software Testing... 4 2. What is Software Quality Assurance?... 7 3. What Is Software Testing?...
Software Testing Massimo Felici JCMB-1402 0131 650 5899 1BP-G04 0131 650 4408 firstname.lastname@example.org What is Software Testing? Software Testing is the design and implementation of a special kind of software
Safety for the manufacturing industry Functional Safety Services The modular service package for safe, efficient machines Industrial Technologies Machine safety is one of the key factors in ensuring that
(STP) Template Items that are intended to stay in as part of your document are in bold; explanatory comments are in italic text. Plain text is used where you might insert wording about your project. This
Management of Comments on this report are gratefully received by Johan Hedberg at SP Swedish National Testing and Research Institute mailto:email@example.com Quoting of this report is allowed but please
Learning Unit 2 Requirements engineering Contents Introduction............................................... 21 2.1 Important concepts........................................ 21 2.1.1 Stakeholders and
Hardware safety integrity Comments on this report are gratefully received by Johan Hedberg at SP Swedish National Testing and Research Institute mailto:firstname.lastname@example.org Quoting of this report is allowed
Software Engineering, Lecture 4 Decomposition into suitable parts Cross cutting concerns Design patterns I will also give an example scenario that you are supposed to analyse and make synthesis from The
The Role of Information Technology Studies in Software Product Quality Improvement RUDITE CEVERE, Dr.sc.comp., Professor Faculty of Information Technologies SANDRA SPROGE, Dr.sc.ing., Head of Department
VAIL-Plant Asset Integrity Management System Software Development Process Document Number: VAIL/SDP/2008/008 Engineering For a Safer World P u b l i c Approved by : Ijaz Ul Karim Rao Revision: 0 Page:2-of-15
Certification of a Scade 6 compiler F-X Fornari Esterel Technologies 1 Introduction Topic : What does mean developping a certified software? In particular, using embedded sofware development rules! What
Intland s Medical Template Traceability Browser Risk Management & FMEA Medical Wiki Supports compliance with IEC 62304, FDA Title 21 CFR Part 11, ISO 14971, IEC 60601 and more INTLAND codebeamer ALM is
JOURNAL OF OBJECT TECHNOLOGY Online at www.jot.fm. Published by ETH Zurich, Chair of Software Engineering JOT, 2006 Vol. 5. No. 8, November-December 2006 Requirements Engineering Tasks Donald Firesmith,
Automated Module Testing of Embedded Software Systems Master s Thesis Fredrik Olsson Henrik Lundberg Supervisors Thomas Thelin, LTH Michael Rosenberg, EMP Nicklas Olofsson, EMP II Abstract When designing
PAGE 24 EB AUTOMOTIVE Industrialized integration and validation of TargetLink models for series production Continuous Software Production The complexity of software systems in vehicles is increasing at
Industry-Driven Testing: Past, Present, and Future Activities at Simula Shaukat Ali Research Scientist Certus Software V & V Center Simula Research Lab Myself Affiliated with Simula since 2007 Have been
1 Open-Source Components in Safety Critical Systems S.Suomalainen 50575U, Mipro Oy, Kunnanmäki 9, 50100 Mikkeli Abstract Faults in control software of the airplanes have caused airplane to crash and many
functional Safety UL Functional Safety Mark Program UL Functional Safety Mark Program With the advent and evolution of functional safety standards in North America and Europe, UL is now offering a UL Functional
Software Engineering CS / COE 1530 Testing How does software fail? Wrong requirement: not what the customer wants Missing requirement Requirement impossible to implement Faulty design Faulty code Improperly
Exploratory Testing in an Agile Context A guide to using Exploratory Testing on Agile software development teams. Elisabeth Hendrickson 2 Exploratory Testing. So you bang on the keyboard randomly, right?
Certified Tester Expert Level Modules Overview Version 1.1, 12th April 2013 Copyright Notice This document may be copied in its entirety, or extracts made, if the source is acknowledged. Version 1.1 Page
Fundamental Principles of Software Safety Assurance Tim Kelly email@example.com Context Lack of agreement in the details of requirements of software safety assurance standards has long been recognised
Edwin Lindsay Principal Consultant, Tel: + 44 (0) 7917134922 E-Mail: firstname.lastname@example.org There were no guidelines/ regulations There was no training No Procedures No Inspectors Inform All staff of
Pathways to Digital Growth Course Outlines IT Service Management This course will help individuals understand the disciplines and processes that help service management staff to deliver and support quality
1 APPROACHES TO SOFTWARE TESTING PROGRAM VERIFICATION AND VALIDATION Validation: Are we building the right product? Does program meet expectations of user? Verification: Are we building the product right?
Factory Acceptance Testing Comments on this report are gratefully received by Johan Hedberg at SP Swedish National Testing and Research Institute mailto:email@example.com -1- Summary According to the
Presentation to TMF Testing the Internet of Things Test and Verification Solutions Delivering Tailored Solutions for Hardware Verification and Software Testing What is the IoT? Wikipedia The Internet of
MEHARI 2007 Overview Methods Commission Mehari is a trademark registered by the Clusif CLUB DE LA SECURITE DE L INFORMATION FRANÇAIS 30, rue Pierre Semard, 75009 PARIS Tél.: +33 153 25 08 80 - Fax: +33
Software Quality Assurance: VI Standards Room E 3.165 Tel. 60-3321 Email: firstname.lastname@example.org Outline I Introduction II Software Life Cycle III Quality Control IV Infrastructure V Management VI Standards VII Conclusion
Standard Glossary of Terms Used in Software Testing Version 3.01 Terms Used in the Advanced Level - Test Manager Syllabus International Software Testing Qualifications Board Copyright International Software
Certification Authorities Software Team (CAST) Position Paper CAST-26 VERIFICATION INDEPENDENCE COMPLETED January 2006 (Rev 0) NOTE: This position paper has been coordinated among the software specialists
Safety-critical software Developing software which should never compromise the overall safety of a system Ian Sommerville 1995 Software Engineering, 5th edition. Chapter 21 Slide 1 Objectives To introduce
Software Development Life Cycle (SDLC) Supriyo Bhattacharjee MOF Capability Maturity Model (CMM) A bench-mark for measuring the maturity of an organization s software process CMM defines 5 levels of process
Quality Management Lecture 12 Software quality management doc.dr.sc. Marko Jurčević prof.dr.sc. Roman Malarić University of Zagreb Faculty of Electrical Engineering and Computing Department of Fundamentals
Preface xvi Part I Introduction and System Engineering 1 Chapter 1 Introduction 2 1.1 What Is Software Engineering? 2 1.2 Why Software Engineering? 3 1.3 Software Life-Cycle Activities 4 1.3.1 Software
Central Page 284 of 296 Application of software product quality international standards through software development life cycle Mladen Hosni, Valentina Kirinić Faculty of Organization and Informatics University
Driving Quality Improvement and Reducing Technical Debt with the Definition of Done Noopur Davis Principal, Davis Systems Pittsburgh, PA NDavis@DavisSys.com Abstract This paper describes our experiences
Functional Safety Management: As Easy As (SIL) 1, 2, 3 Abstract This paper outlines the need for planning in functional safety management. Recent events such as the Montara blowout and the Deepwater Horizon