It s Only Software! MGA-855 Certification des systèmes embarqués d aéronefs. Maîtrise en génie : Concentration en génie aérospatial

Transcription

1 MGA-855 Certification des systèmes embarqués d aéronefs Maîtrise en génie : Concentration en génie aérospatial It s Only Software! 1

2 MGA-855 Certification des systèmes embarqués d aéronefs Maîtrise en génie : Concentration en génie aérospatial Chapitre 3.2 and 3.3 RTCA/DO-178B LIFE CYCLE PROCESSES, TOOLS AND TRANSITION CRITERIA présenté par : Maxence Vandevivere cellmaxence@gmail.com Professeur responsable : René Jr. Landry Poste : 8506 Porte : rlandry@ele.etsmtl.ca Site web : 2

3 This module is designed to show the application of the certification principles contained in the earlier chapters. As such, it touches upon many aspects of the certification process, but the material is not complete, comprehensive, or necessarily current with the latest regulations and guidance materials. For these reasons, the analysis contained in the following slides should not be used as the basis of a certification program for the system under investigation. 3

4 Overview 3.2, 3.3 RTCA/DO-178B Life cycle Integral processes Planning Design Summary 4

5 Where are we? (Fig 2-1, DO-178B) 5

6 3.2.1 Life cycle process 6

7 3.2.1 Life cycle process What is a software life cycle? 1. a series of stages in the development of an application and is often used in Software Engineering. 2. The phases a software product goes through between when it is conceived and when it is no longer available for use. The software lifecycle typically includes the following: requirements analysis, design, construction, testing ( validation), installation, operation, maintenance, and retirement. ( Management and support from birth to death of the software. 7

8 3.2.1 Life cycle process (cont d) COTS (Commercial off-the-shelf) software doesn t last long Released: September 14, Replaced on (RIP): October 2001 by Windows XP. Shelf-life: Approximately 1 year. 8

9 3.2.1 Life cycle process (cont d) How long does software in a certified system last? Boeing 757 First delivery: 1983 Last aircraft pulled from service:? Boeing 787 First delivery: 2011(???) Last aircraft pulled from service:??? 9

10 3.2.1 Life cycle process (cont d) We cannot certify software like other physical components. DO-178B is one method of compliance. DO-178B outlines objectives for us to meet. Meeting DO-178B objectives increases likelihood of creating software that performs its intended function with a level of confidence in safety that complies with airworthiness requirements (DO-178B, section 1.1) How do we meet these objectives? By using a defined process - a software development life cycle. 10

11 3.2.1 Life cycle process (cont d) As software criticality increases, process rigor increases: Validation and verification (V&V) is more detailed Requirement for independence increases (verification / auditing the process, and outputs of the process) Also,» Schedule increases,» Program costs increase,» Complexity of process increases,» Etc. 11

12 3.2.1 Life cycle process (cont d) 12

13 3.2.1 Life cycle process (cont d) Objectives of a life cycle process for certified software: Meet objectives from DO-178B guidelines Create a pedigree, or history of the software from its start to current state The items (data) created are called artifacts Ensure manageability of software development, and allow maintenance using processes that:» Are compliant with industry best practices,» Repeatable,» Controllable,» Fault tolerant i.e.: eliminate (mitigate) personnel single point failures,» Provide legal backup evidence of due diligence 13

14 3.2.1 Life cycle process (cont d) Influences on the software life cycle process: Software criticality (Level A vs. Level E process) System & software requirements System & software design Existing processes (ISO, RUP, best practices, expert consultation, etc.) Contractor processes or requirements Customer processes or requirements 14

15 3.2.2 Phases of a life cycle process What does DO-178B say about which process to follow? Nothing. Process is defined by the entity responsible for the software development. The process can be anything as long as it meets the DO- 178B objectives. The objectives are listed in DO-178B in specific documents. (Note: These specific documents don t need to be used, but there isn t much point re-inventing the wheel and taking away familiarity of existing documents.) 15

16 3.2.2 Phases of a life cycle process Many life cycle models exist, for example: Waterfall, Iterative, RUP, RAD, Spiral, Scrum Etc. The following phases must exist in some form: Integral processes Planning Development 16

17 Example life-cycle process Phase 1, Inception Phase 2, Elaboration Phase 2, Construction Phase 4, Testing Phase 5, Release Create plans and standards Create software requirements from system requirements Create SDD including : software low-level requirements, software design and architecture Source code development, Test case development, and development testing Perform Test for score Release to customer (internal or external) Review documents, and traceability from system to software requirements Review documents and traceability from software high-level to low-level Review code and traceability from lowlevel to code, code to test cases Review testing output, test cases to test output Conformity review 17

18 3.2.2 Integral processes DO-178B includes the following on-going integral processes that must be active in the life cycle process: Configuration management (CM) Quality assurance (QA) Verification and validation (V&V) Certification liason 18

19 3.2.2 Integral processes An integral process: Ensures the correctness, control, and confidence of the software life cycle processes and their outputs (DO-178B, 3.2 para 3). Is an activity that continues throughout the software life cycle Keeps the software life cycle process on track 19

20 A simple example: Integral processes Definition system level requirements, and other input Integral processes Transition criteria & Timing of Integral processes Planning PSAC, coding standards, etc. Configuration management Elaboration SDD Quality assurance Construction code, test cases Testing verification and validation Verification and validation Release Certification liaison Maintenance input from customers, service bullitins, etc. 20

21 3.2.2 Integral processes - CMP The Configuration Management Plan describes how data will be managed in the life cycle. In DO-178B terms : it answers how all of the CM objectives will be satisfied Should include: What environment, tools, procedures, standards, org. chart (responsibilties) CM will be run in. Detailed description of all of the different CM activities, such as:» What types of data (documents, reports, source code) will be identified» How the data items above will be traced/connected to each other» When baselines occur» Problem reporting, change control, change review 21

22 3.2.2 Integral processes CMP (cont d) Should also include :» How all of the above activities are tracked and monitored» How all of the data will be stored, released, etc.» How software builds/loads are created, controlled, and how it s kept safe» How all of the tools used the project will be controlled, stored, backed up, etc. (e.g.: what happens if a compiler you use is no longer available on the market to purchase?)» Life cycle data controls- CC1, CC2 Transition criteria : When and how the CM process starts List of all data the CM process produces e.g.: SCI, SECI, CRs, PRs, change history, release records, archive records, etc. Supplier control (if applicable) how suppliers will work with this CM process 22

23 Life-cycle data controls 23

24 3.2.2 Integral processes - QAP The Quality Assurance Plan describes the methods and processes used to ensure the life cycle is kept on track. In DO-178B terms : it answers how all of the QA objectives will be satisfied Should include a detailed description of : The QA environment that will be used, including:» tools, procedures, standards,» QA scope» Organizational responsibilties Authority what authority QA has, their independence, approval authority 24

25 3.2.2 Integral processes QAP (cont d) Should also include: QA activities that will occur, including:» Audits, reviews, reports, inspections, monitoring of the life cycle processes» Problem reporting & change request and tracking and monitoring» Software conformity review (SCR) what occurs, what will be done for the SCR Transition criteria: When does the SQA process begin? Timing and sequence of the SQA process and activities, specifically in relation to other life cycle processes List of all data the SQA process produces e.g.: meeting minutes, review records, audit reports, records of authorized process deviations, SCR records. Supplier control (if applicable) how suppliers will work with this QA process 25

26 3.2.2 Integral processes - VP The Verification Plan describes what will occur for software verification In DO-178B terms : it answers how all of the verification objectives will be satisfied Should include: Organizational responsibilties who does what to whom in the organization & and with respect to other life cycle processes If independence is required, how is established? What methods will be used for verification?» What methods will be used for reviews, analysis, and testing?» Will checklists be used? If so, which?» For analysis, need to include how traceability and structural coverage analysis will be done 26

27 3.2.2 Integral processes - VP Should also include: What methods will be used for verification? Cont d :» For testing: test procedures, test data to be produced, test case selection process guidelines Environment: Tools, hardware, procedure and guidelines for using them for verification. Transition criteria : When does software verification begin? If partitioning is used, how will its integrity be verified? Assumptions about the correctness of the compiler and compiler tools selected How will reverification be done? When will it be done? How will the process ensure that when a change is made it doesn t break other code? 27

28 3.2.2 Integral processes - VP Should also include: Previously developed software (PDS): If any PDS is used, and it did not use the same verification process as defined in the current VP, close the gap between the two. Multiple version dissimilar software: If used, how will it be verified? 28

29 3.2.2 Integral processes Cert Liason Certification Liason is the process of communicating and agreement with with the certification authority for your project Intended to: Communicate - make sure no one is surprised Resolve any issues in PSAC, including means of compliance Agree on PSAC Support certification authority reviews/audits of evidence of compliance (including issue resolution) Specified in the PSAC, SAS, and SCI These three documents are the deliverables to the certification authority 29

30 3.2.2 Integral processes Records To an auditor, or reviewer: unless a record exists of an activity occuring, it did not happen. If you cannot identify what the record is in reference to, it is useless. You cannot get credit with missing evidence. The moral of the story: Keep records, even meeting minutes, any reviews, audits, completed checklists, etc. Ensure records all identify the who, what, when, where (the how is the review record itself). For example:» Having a code review state only the module name is useless.» Instead: abc.c, v1.4, from source tree aaa/bbb/ccc/trunk/abc.c in project abc cert project in SVN. 30

31 3.2.3 Planning 31

32 3.2.3 Planning Planning: Plans:» Plan for software aspects of certification (PSAC)» Software development plan (SDP)» Software verification plan (SVP)» Quality assurance plan (QAP)» Configuration management plan (CMP) Standards:» Software requirement standard» Software coding standard» Software design standard 32

33 3.2.3 Planning PSAC The top-level plan for the software development program. Defines how the software will be certified. Once approved by certification authority, the PSAC is the contract and agreement of what will be done in the program between the applicant and certifcation authority. Deviations from the PSAC must be dealt with. PSAC is one of three documents delivered to the certification authority 33

34 3.2.3 Planning PSAC Includes a high-level system & software overview (familiarize the reader) Defines or points to every aspect of the program including: Certification basis means of compliance, software level, justified. High-level summary of life cycle process that will be used and how the objectives will be satisfied (details in SDP) Life cycle data to be produced also what will be delivered to certification authority, and the responsibilties of the certification liason process Schedule Additional considerations i.e.: anything out of the ordinary. 34

35 3.2.3 Software criticality level Level of Software Failure Conditions Probability Level E No Effect < 1 x10-5 Level D Minor > 1 x10-5 Level C Major 1 x 10-5 < 1 x 10-7 Level B Hazardous/Severe-Major 1 x 10-9 < 1 x 10-7 Level A Catastrophic < 1 x

36 3.2.3 Planning SDP Software development plan how the software will be developed. Can be included in the PSAC instead of a separate document. Standards identifies the following standards: Software Requirements Standards, Software Design Standards and Software Code Standards If necessary, references to the standards for previously developed software, including COTS software 36

37 3.2.3 Planning SDP Software life cycle Describes the software life cycle processes, and its sequence Includes the transition criteria for the software development processes Must be detailed enough to actually use the process Software development environment What hardware and software development environment will be used to develop the project (not just the software)?» Requirement development methods and tools to be used» Design methods and tools (if any)» Programming languages, coding tools, compilers, linkage editors, loaders to be used» Hardware platforms for the tools to be used 37

38 3.2.3 Planning SRS Software requirement standard defines how requirements will be constructed and developed Should include: The procedure of developing requirements, how tools to be used or not used How requirements will be managed, dealt with Describes any specification languages e.g.: UML How derived requirements will be dealt with with respect to the overall system process 38

39 3.2.3 Planning SCS Software coding standard defines how code will be written. Should take software criticality into account. Should include: Defines what programming language(s) will be used, what methods, rules, and tools (if any) will be used to create the software. Limitations on features which may or may not be used Should include things like:» Coding standards for code comments, documentation, style, etc.» Naming conventions for variables, procedures, functions, modules, etc.» Coding conventions 39

40 3.2.3 Planning SDS Software design standard defines how the software will be designed, what procedures and tools should be used. Should include: Methods to be used, including any conditions on the methods and justification for use if necessary (e.g.: scheduling, interrupts, exception handling) Naming conventions to be used Design constraints on design constructs, code structure, etc. Procedures and/or constraints in the use of design tools 40

41 3.2.4 Design 41

42 3.2.4 Design Design: Software Requirements Document (SRD) Software design document (SDD) Software code creation Software verification and test procedures Test cases Configuration index document (SCID & SECID) 42

43 3.2.4 Design SRD Software requirements document defines all high-level software requirements. Should include: Traceability to system requirements especially safety requirements, and potential failure conditions Software requirements that deal with the software:» Functionality, operation (espcially in each mode of operation, if multiple modes exist),» Performance (precision, accuracy),» limitations,» Timing, memory,» Hardware & software interfaces (protocols, formats, input/output frequency)» Safety monitoring, fault detection» Partitioning requirements 43

44 3.2.4 Design SDD Software design document defines the low-level detailed design of the software, its structure. Intent is that the SDD breaks down high-level SRD requirements into low-level requirements. Low-level requirements should be detailed enough to write code without ambiguity. Should include requirements and detail for: Algorithms to be used, data structure Software structure/architecture Inputs & outputs, including a data dictionary Data and control flows 44

45 3.2.4 Design SDD Should include requirements and detail for, cont d: Resource limitations (including measurement) Scheduling, processor tasking/use, IPC, interrupts Design methods for data loading, user-modifiable software, multipleversion dissimilar software Partitioning & how it will work Software components (whether they are new, PDS, etc.) Derived requirements If deactivated code is intended, need to explain how it cannot be activated Design decision rationale for safety-related system requirements 45

46 3.2.4 Design SVCP Software verfication cases and procedures details what will be done in the software verification activity, Including: additional information on analysis and review procedures from the SVP Test cases, purpose, input/output, conditions, expected results, coverage criteria, pass/fail criteria Test case process how each test case should be run, evaluated, test environment operational procedures 46

47 3.2.4 Design SVR Software verification results is a record of all verification activities. SVR should include: Identification of the item reviewed/tested/analysed Listing of all tests/reviews/analysis Results for each review/analysis/test performed, including the final test results Structural coverage and traceability results and analysis 47

48 3.2.4 Design SECI Software environment configuration index captures the complete configuration of the software life cycle environment. Intent is to help reproduce the environment, software, hardware, for software reverification, rebuilding, modification, etc. Often included with the SCI. 48

49 3.2.4 Design SECI SECI should describe and detail: Life cycle environment, and operating system Compilers, tools, linkers, data integrity tools, used Test environment used, including test tools Any qualified tools and associated tool qualification data 49

50 3.2.4 Design SCI Software configuration index captures the complete configuration of the software. Intent is exactly identify every component that makes up the software. Should identify: The software and executable name, version, size, date, etc. Everything that identifies the software and binary. All of the components that make up the software, including version, where to find it in the CM system, etc. Any PDS, if used. All software life cycle data, including where to find it The archive and release media 50

51 3.2.4 Design SCI Should identify (cont d) : Build instructions a readme, or detailed instructions) Procedures to use for modification, reverification Reference to the appropriate SECI Any data integrity checks to use with the binary (if applicable) 51

52 3.2.4 Design SAS Software accomplishment summary is a summary of everything that happened in the life cycle. This should be almost a copy-paste from the PSAC The PSAC says what you will do The SAS says what you actually did note the change in verb tense Most of the elements contained are identical to the PSAC The difference is: clarification must be given for any differences from the plan to state what was actually was done during the software life cycle. 52

53 3.2.4 Design SAS The SAS should include the following (PSAC elements noted with *): *System and software overview *Certification considerations Software characteristics describe the binary object size, timing and memory requirements, resource limitations, and how each characteristic is measured *Software life cycle & life cycle data Additional considerations summarize any issues which should be noted, extra data items necessary, such as issue papers, or special conditions. Software identification part number, version Change history of the software from the previous certified version especially any issues relating to safety 53

54 3.2.4 Design SAS The SAS should include the following (PSAC elements noted with *) (cont d) : Software status:» Summary of open/unresolved problem reports, change requests, at the time of certification» List of functional limitations due to unresolved issues Compliance statement:» statement of compliance with DO-178B» summary of methods used to demonstrate compliance with respect to criteria in plans» Address and additional deviations from plans, standards, and DO- 178B not already addressed. 54

55 55

56 Summary There is no such thing as a small software cert. project. Ultimately, the intent is to demonstrate & collect evidence that: The software was built using some methodology/process Process appropriate to the intended function and criticality of the software Approach agreed on with the cert authorities Software meets its intended function and requirements PSAC is the project contract with the certification authorities Ultimately you must comply with the PSAC (not DO-178B) Integral processes are the glue that holds everything together. 56

57 Summary (cont d) The reason we do all of this? How do you want the accident report to read? Or put another way: When you are on the witness stand in court, what would you like to be able to say to the judge, families, reporters, lawyers, etc.? 57

58 Questions? 58

59 References RTCA/DO178B 59