Advisory Circular. Susan J. M. Cabler Acting Manager, Design, Manufacturing, & Airworthiness Division Aircraft Certification Service DRAFT

Transcription

1 DRAFT U.S. Department of Transportation Federal Aviation Administration Advisory Circular Subject: Airborne Software Assurance Topics Date: mm/dd/yy Initiated By: AIR-134 AC No: 20-Software Topics On December 13, 2011, RTCA, Inc. (RTCA) published RTCA Document (DO)-178C, Software Considerations in Airborne Systems and Equipment Certification, along with three related supplements (RTCA DO-331, DO-332, and DO-333) and RTCA DO-330, Software Tool Qualification Considerations. On July 19, 2013, the FAA published Advisory Circular (AC) C, Airborne Software Assurance, recognizing these RTCA documents as an acceptable means, but not the only means, for showing compliance with the applicable airworthiness regulations for the software aspects of airborne systems and equipment certification. AC C also addresses how applicants can continue to use RTCA/DO-178B while they transition their existing DO-178B software assurance processes to DO-178C. This AC addresses software-related topics that are in addition to the guidance provided in DO- 178C, DO-178B, and AC C. This AC includes guidance that has been extracted from FAA Order change 1, Software Approval Guidelines, and from issue papers that have been applied to specific type certification programs. If you have any suggestions for improvements or changes, you may use the template provided in Appendix D of this AC. Susan J. M. Cabler Acting Manager, Design, Manufacturing, & Airworthiness Division Aircraft Certification Service

2 xx/xx/xxxx DRAFT AC 20.Software Topics Paragraph CONTENTS Page CHAPTER 1. General Information Purpose of this Advisory Circular (AC) To Whom this AC Applies CHAPTER 2. Field-Loadable Software (FLS) When to Apply this Chapter Addressing Field-Loadable Software FLS Installation Considerations Maintenance and Part Marking Considerations CHAPTER 3. User-Modifiable Software (UMS) When to Apply this Chapter Considerations for UMS Data Requirements CHAPTER 4. Software Problem Reporting When to Apply this Chapter Managing Software Problem Reporting Managing and Summarizing Unresolved Problem Reports (UPR) CHAPTER 5. Reverse Engineering When to Apply this Chapter Background Guidance for Reverse Engineering CHAPTER 6. Assuring Parameter Data Items (PDI) When to Apply this Chapter Background Assuring PDI Using DO-178C CHAPTER 7. Addressing Extraneous Code When to Apply this Chapter Addressing Extraneous Code ii

3 xx/xx/xxxx DRAFT AC 20.Software Topics CONTENTS (CONTINUED) Paragraph Page CHAPTER 8. Qualification of Software Tools Using DO-178B When to Apply this Chapter Software Tool Qualification CHAPTER 9. Applying DO-178B Criteria to Level D Software When to Apply this Chapter Background Assuring Level D software CHAPTER 10. Software Structural Coverage at the Object Code Level When to Apply this Chapter Background Object Code Structural Coverage Analysis CHAPTER 11. Object-Oriented Technology (OOT) When to Apply this Chapter Background Using OOT in DO-178B Processes CHAPTER 12. Model-Based Development (MBD) When to Apply this Chapter Background Using MBD in DO-178B Processes CHAPTER 13. Formal Methods (FM) and Associated Tools When to Apply this Chapter Background Using FM in DO-178B Processes Appendix A. Related Regulatory, Advisory, and Industry Material... A-1 A.1 14 CFR Applicable Sections.... A-1 A.2 FAA ACs.... A-1 A.3 Industry Documents.... A-1 A.4 Where to Get Referenced Documents... A-2 iii

4 xx/xx/xxxx DRAFT AC 20.Software Topics CONTENTS (CONTINUED) Paragraph Page Appendix B. Definitions...B-1 B.1 For purposes of this AC, the following definitions apply:...b-1 Appendix C. Acronyms...C-1 C.1 The following is a list of acronyms used in this AC:...C-1 Appendix D. Advisory Circular Feedback Information... D-1 iv

5 CHAPTER 1. GENERAL INFORMATION 1.1 Purpose of this Advisory Circular (AC) On July 19, 2013, we, the Federal Aviation Administration (FAA), published AC C, Airborne Software Assurance, which provides the following guidance: Recognizes RTCA, Inc. documents (hereafter referred to as DO in this AC) DO-178C, DO-330, DO-331, DO-332, and DO-333 for airborne software assurance Describes the use of supplements, including the use of multiple supplements, for software development and tool qualification Establishes guidance for transitioning to DO-178C when making modifications to software previously approved using DO-178, DO-178A, or DO-178B Explains under what conditions you may use software tools that were previously used with DO-178 or DO-178A, or qualified using DO-178B in software projects where you may or may not intend to declare compliance with DO-178C This AC provides guidance to help you comply with the software aspects of the applicable airworthiness regulations and is in addition to the assurance processes described in DO-178C and DO-178B. Most of the guidance contained in this AC has been extracted from FAA Order change 1, Software Approval Guidelines, and from issue papers that have been applied to specific type certification programs. If you are using DO-178C or DO-178B (hereafter referred to as DO-178C/B) for software assurance, you should determine which chapters apply to your project, and use the guidance contained therein Chapters 2 through 5 of this AC apply to both DO-178C and DO-178B. If you are using a DO-178B process, you may benefit from the referenced sections of DO-178C because of clarifications and wording improvements. We recommend you review DO-178C, appendix A, section 3, for a summary of differences between DO-178C and DO-178B Chapter 6 applies to DO-178C and addresses parameter data items Chapters 7 through 13 are intended for applicants who are using DO-178B for software assurance. These topics are sufficiently addressed in DO-178C Since some of the guidance in this AC originated in project-specific issue papers, use of this AC as a means of compliance may eliminate the need for the applicable subject issue paper(s). 1-1

6 1.1.4 References to DO-178C in this AC include use of DO-330 and supplements DO-331, DO-332, and DO-333, as applicable If you use the means described in this AC as a means of compliance, you must follow it entirely. 1.2 To Whom this AC Applies This AC applies to applicants, manufacturers, and developers of airborne systems and equipment containing software that are installed on all type certificated aircraft, engines, and propellers Although we recognize the type certificate (TC) applicant is ultimately responsible for complying with the airworthiness regulations, we also recognize supporting roles occur at various levels in the certification or Technical Standard Order (TSO) authorization processes. Your participation in the software development and product certification processes determines your roles and responsibilities. For example, the term you could apply to A TC applicant, A supplier to a TC applicant (refer to the definition of supplier in appendix B to this AC), An applicant for a TSO authorization, A supplier to an applicant for a TSO authorization, A supplier to a supplier (sub-tier supplier), The installer of a TSO authorized article into a (type-certificated) product, or The installer of a non-tso authorized piece of equipment into a (type-certificated) product. 1-2

7 CHAPTER 2. FIELD-LOADABLE SOFTWARE (FLS) 2.1 When to Apply this Chapter. This chapter supplements DO-178C, section 2.5.5, and DO-178B, section 2.5. You should use the guidance in this chapter in addition to the applicable guidance in DO-178C/B when you intend to use FLS. 2.2 Addressing Field-Loadable Software Your Plan for Software Aspects of Certification (PSAC) should identify your intent to use FLS and describe how you will satisfy the guidance of DO-178C/B and this chapter The following subparagraphs identify specific items that should be addressed during FLS development, verification, and installation: The configuration of the software load installed in the product should be identical to the configuration that was approved, and the product s target hardware configuration should be compatible with the FLS verification environment hardware A process should be established to ensure the software loaded has not been corrupted. This includes security concerns, such as FLS that is transferred over a network. The data integrity method(s) used should be capable of detecting transfer and corruption errors to an integrity level appropriate for the software level of the FLS being loaded. An aircraft-level cyber security plan should address FLS The FLS part number loaded in the airborne equipment should be verifiable with onboard equipment, carry-on equipment, or other appropriate means. For TSO approved articles, the appropriate part marking data pursuant to Title 14 of the Code of Federal Regulations (14 CFR), part 45, section15, Markings requirements for PMA articles, TSO articles, and Critical parts, must be verifiable on the ground at any geographic location An aircraft-level configuration management process should ensure the installation configuration is correct, controlled, and consistent with what is approved If redundant parts on the aircraft, engine, or propeller are field-loadable, you should address the following concerns The acceptability of intermixing different software loads, How partially successful loads should be handled, and How unsuccessful loads on redundant parts affect the ability to dispatch the aircraft. 2-1

8 Protection mechanisms should be implemented to inhibit loading of FLS during flight Any future changes to FLS should undergo a change impact analysis (refer to DO-178C/B, section 12.1) Data supporting the installation considerations of paragraph 2.3 of this AC should be provided to the FLS installer, as applicable The Software Accomplishment Summary (SAS) should explain how all applicable guidance is satisfied when implementing FLS features. The installation and loading procedures should be identified in the Software Configuration Index (SCI) and aircraft maintenance manual or appropriate system-level documentation. 2.3 FLS Installation Considerations. Approved FLS may be installed on the aircraft via service bulletin, engineering change request, or other acceptable means. The following elements should be addressed as applicable The applicability of the aircraft and hardware so the FLS is installed into an approved configuration; Considerations and limitations regarding the intermixing of software versions loaded into redundant systems Verification procedures that ensure the software is correctly loaded into an approved and compatible target computer or memory device(s); Any required post-load verification and test procedures; Actions to be taken in the event of a partially successful load or an unsuccessful load, such as prohibiting dispatch of the aircraft; Approved loading procedure; Maintenance record entry procedures required for maintaining configuration control; and Reference to the aircraft flight manual, aircraft flight manual supplement, or operator s manual, as appropriate. 2.4 Maintenance and Part Marking Considerations. Maintenance and part marking requirements are specified in part 43, Maintenance, Preventive Maintenance, Rebuilding, and Alteration, and part 45, Identification and Registration Marking. Additional considerations that apply specifically to FLS are 2-2

9 2.4.1 The aircraft maintenance manual or Instructions for Continued Airworthiness (ICA) should include the procedures to be followed when conducting maintenance on the airborne system or equipment implementing FLS. The FLS part number configuration should be verified before and after maintenance is performed. If you are unable to verify the software load, the system should not be considered functional and the aircraft should not be dispatched unless Minimum Equipment List (MEL) procedures allow dispatch with the equipment disabled Processes and procedures should be in place to ensure an approved FLS part number is loaded into the airborne equipment, verified to be the correct part number, and recorded in the appropriate maintenance logs before the equipment or aircraft is returned to service. For airborne equipment having separate part numbers for hardware and software, the software part numbers need not be displayed on the outside of the unit as long as they can be verified through an electronic query For airborne equipment with only one part number that represents an approved configuration of software and hardware, the identification on the nameplate should be changed when new software is loaded. The software part number stored in the target computer should be verified electronically after data loading. The electronic software part number and the equipment part number displayed on the nameplate should be verified to be an approved configuration before returning the equipment or aircraft to service If you want to use electronic part marking for FLS in equipment approved under TSO authorization or using the parts manufacturer approval process, the FLS must meet the part marking requirements of Changes to the FLS part number, version, or operational characteristics should be reflected in the operator s manual, Aircraft Flight Manual, or other document(s) as applicable. 2-3

10 CHAPTER 3. USER-MODIFIABLE SOFTWARE (UMS) 3.1 When to Apply this Chapter. This chapter supplements DO-178C, sections and 5.2.3, and DO-178B, sections 2.4 and You should use the guidance in this chapter in addition to the applicable guidance in DO-178C/B when you intend to use UMS. 3.2 Considerations for UMS Modification to UMS by the user should have no adverse impact on aircraft safety margins, aircraft operational capabilities, or flight crew workload and may require an FAA operational approval or acceptance The modifiable component should be developed to the software level corresponding to the worse-case failure condition of the component If the non-modifiable component is protected from the modifiable component by software, the protection software should be developed to the same software level as the non-modifiable component If tools will be used to enforce the protection of non-modifiable software components from modifiable components, your plans and procedures should provide for Controlling tool version, Controlling tool usage, Qualifying the tool to the appropriate software level, Using and maintaining the tool, and Modifying the tool, including re-qualification if required. 3.3 Data Requirements Your PSAC should describe How you will comply with this chapter and the applicable sections of DO- 178C/B, The mechanisms used to protect the non-modifiable components, and The means of ensuring the integrity of the protection mechanisms If software tools will be used for the modification, your PSAC should identify tool qualification plans or verification procedures that ensure the tool will modify the UMS 3-1

11 to approved procedures and constraints, and that it will not affect the non-modifiable software or protection mechanisms The Software Development Plan and design data should specify the design methods and details of the implementation for ensuring protection from user modifications The SCI should identify the approved procedures, methods, and tools for modifying the UMS, including tool qualification data, if applicable The SAS should summarize the development and verification of the non-modifiable software components, UMS component(s), protection mechanism, and modification procedures and tools, including tool qualification, if applicable You should consider implementing a system to track UMS modifications so the certification and continued airworthiness aspects of the modifications may be reviewed by the cognizant authorities Modification procedures, constraints, and tools should be documented or referenced in the operator s manual and maintenance procedures, as appropriate for the UMS component(s). 3-2

12 CHAPTER 4. SOFTWARE PROBLEM REPORTING 4.1 When to Apply this Chapter. As the applicant, you are responsible for managing problems detected during the development and verification processes of airborne systems implemented with software, including problems detected by your suppliers (including sub-tier suppliers). This chapter supplements DO-178C and DO-178B, sections through 7.2.6, and provides guidance for managing and overseeing the problem reporting processes so problems discovered during software development, verification, and in service are properly identified, reported, and resolved. 4.2 Managing Software Problem Reporting. You should describe in your Software Configuration Management Plan or other planning documents how you will manage the software problem reporting process. This applies to you and your suppliers problem reporting processes Describe the problem reporting processes and how you will ensure all problems will be reported, assessed, resolved, implemented, re-verified (regression testing and analysis), closed, and controlled. This includes problems related to software, parameter data item files, databases, software subcomponents (such as a real-time operating system), qualified tools, data items, and electronic files used in airborne systems and equipment installed on the aircraft Your suppliers may enter problem reports into their problem reporting system and then transfer them to your problem reporting system where they are validated to ensure that any translation is done correctly. Your plans and your supplier s plans should describe how this is accomplished. If you allow your suppliers to enter problems directly into your problem reporting system, describe how you will maintain configuration control Describe any tools that you or your suppliers will use for recording action items or observations for review and approval before entering them into any problem reporting system Describe how problem reports that you determine to have an impact on your suppliers will be conveyed to and coordinated with them Describe how problems that may influence other applications, or that may have system-wide influence, are made visible to the appropriate disciplines and suppliers Describe how problem reports will be categorized so each problem report will be classified based on the potential impact on the system and the software Categories should identify problems with a potential impact on safety, functionality, performance, operation, or design assurance. 4-1

13 Categories should aid in prioritizing the resolution of problems, such as problems that should be resolved immediately, prior to flight test, prior to approval or certification, or that may be deferred until after software approval. Define the criteria for determining when it is acceptable to defer a problem DO-248C, Supporting Information for DO-178C and DO-278A, Discussion Paper No. 9, describes typical categories used to classify problem reports. You should consider implementing these or similar categories into your problem reporting system You should establish a problem report review board consisting of individuals with expertise in the functions or disciplines affected by the problem report (e.g., flight test, human factors, systems, software, and other appropriate disciplines) to, at minimum Establish the criteria for determining if unresolved problem reports may be deferred beyond software approval or certification Participate in supplier problem report review boards and change control boards as appropriate Confirm that problem reports are categorized properly Determine the potential impact of each unresolved problem report on safety, functionality, and operation Your plans should establish a schedule for resolving problem reports that are deferred beyond software approval or certification. Unresolved problems related to mandatory corrections or conditions, such as airworthiness directives, may not be deferred. 4.3 Managing and Summarizing Unresolved Problem Reports (UPR). If you or your suppliers intend to defer the resolution of problem reports until after system approval or certification, the following guidance applies Each problem report should be properly categorized based on the potential impact of the problem to the software and the system All problems for the current release should be identified and evaluated. This includes new problems discovered since the previously approved release You should assess each proposed UPR and verify The assigned classification is appropriate The impact on the aircraft (if known) or system is identified (such as functional limitations and operational restrictions). 4-2

14 The likelihood of occurrence Deviation(s) from software plans or standards are identified Interrelationships with other UPRs are identified Any required safety mitigations are described The UPR is justified for remaining unresolved The root cause of the problem has been determined, if applicable The anticipated resolution schedule is identified, if known The person(s) responsible for system-level functionality are in agreement with the proposed UPRs The number of UPRs should be controlled to a level that the interactions of the UPRs can be assessed You should address the following concerns based on UPR categorization when assessing UPRs UPRs determined to have a safety impact should be corrected before software approval. If not corrected, define appropriate mitigation means, such as operational restrictions, so there are no adverse safety effects at the aircraft level. In some cases, such as a TSO authorized article developed for various installations, the safety impact may not be determined until installation. In this case, the installer may need to implement appropriate mitigation means and seek support from the TSO authorization holder as necessary. If the aircraft safety impact is at least hazardous and cannot be mitigated, the equipment should not be installed Functional failure UPRs should be assessed at the system or equipment level. Define any appropriate restrictions or limitations to ensure safety. An assessment of no safety impact should be justified Determination of UPRs as non-functional fault should justify why the problem cannot cause a functional failure. For simple cases, engineering judgment may be acceptable. In other cases, additional validation or verification activities may be required UPRs categorized as deviations from plans or standards should be assessed to determine if the deviations may contribute to incomplete software assurance activities or incomplete satisfaction of objectives. An analysis should show that the deviations do not result in unacceptable risk. If they do result in unacceptable risk, the deviations should not be allowed, or 4-3

15 mitigations should be provided, such as additional software life cycle activities UPRs categorized as all other types of problems do not require further assessment beyond the type classification and justification The root cause of each UPR (where applicable) and the expected schedule for its resolution should be documented in the appropriate software life cycle data, such as Problem Reports, software configuration management records, or software quality assurance records. You should identify the software life cycle process deficiencies that led to the error and document the actions to be taken to prevent a future occurrence In accordance with the requirements stated in DO-178C/B, section 11.20, Software Status, your UPR summary should include the following information Identification of each UPR, including configuration identification number (UPR number), A brief description of each UPR, and UPR assessment results including Classification of each UPR, The impact of the UPR at the aircraft level (if known) and system level and any operational limitations and changes to the intended function of the system, An assessment of when the problem is likely to be encountered and the corresponding trigger conditions, Deviation(s) from software plans or standards, Interrelationships with other UPRs, if applicable, Safety mitigations, and Justification for why the system can be approved with the UPR The SAS should include or reference a listing of other UPRs that might impact the airborne software, such as commercial-off-the-shelf (COTS) component UPRs. These related UPRs should be evaluated to ensure they do not negatively impact safety, performance, functional or operational characteristics, software assurance, or compliance For each UPR proposed to remain unresolved at type certification, the impact on the aircraft and any mitigations should be assessed and agreed upon by the installer. 4-4

16 CHAPTER 5. REVERSE ENGINEERING 5.1 When to Apply this Chapter. Use the guidance of this chapter when you establish reverse engineering processes to develop compliance evidence for software that does not satisfy the applicable DO- 178C/B objectives or that may have residual development errors. 5.2 Background When developing software to be used in an airborne system, some of the artifacts needed to satisfy the applicable DO-178C/B objectives may be inadequate or missing, especially if containing commercial-off-the shelf (COTS) components. Although some of the DO-178C/B guidance may imply a forward engineering life cycle sequence, a well-defined and structured reverse-engineering approach may be used to regenerate the missing software life cycle data when it is desired to obtain approval to DO-178C/B Forward engineering processes start with abstract representations of an implementation, defined as the more abstract layer (MAL), and refine those representations into more detailed representations, defined as the less abstract layer (LAL). For example, high-level requirements are refined into the software architecture and low-level requirements Reverse engineering processes start with detailed representations of an implementation, or LAL, and apply various techniques to produce less detailed representations, or MAL. An example would be generating low-level requirements from source code. The goal of reverse engineering is to create an MAL that can be used to understand and reason about the structure and the intent of the LAL In the software development process, the most abstract layer would be represented by high-level requirements, which are traced to the system requirements; the least abstract layer would be represented by executable object code. 5.3 Guidance for Reverse Engineering Section 5 of DO-178C/B discusses the software development processes and the associated objectives and activities. Although the guidance in section 5 is based on a forward engineering approach, most of the objectives are worded in a manner independent of a forward or reverse engineering development approach. However, two of the objectives imply a forward approach DO-178C/B, section a, states The software architecture and low-level requirements are developed from the high-level requirements. For a reverse-engineering approach, this objective should be interpreted as stated in annex A, table A-2: low-level requirements are developed. 5-1

17 DO-178C, section a, states Source Code is developed from the lowlevel requirements. For a reverse-engineering approach, this objective should be interpreted as stated in annex A, table A-2: Source Code is developed For a software product that has demonstrated an unsatisfactory usage history, reverse engineering may be used to discover causes of unexpected behavior resulting from inconsistencies between layers of abstraction. For example, source code could be analyzed and resulting artifacts compared against the forward-developed artifacts to determine if anomalous functionality was inadvertently implemented Subject matter experts (SME) from multiple disciplines should participate in the systems and software development processes to ensure the behavior of the software meets system requirements. SMEs should provide expertise in areas such as aircraft systems, engine systems, operating systems, and software engineering, as required by the project. SMEs should liaison with other SMEs and specialists so collectively they have an overall understanding of the system behavior and intermediate representations of the product Other specialists used in reverse engineering processes may need unique skills in addition to those necessary for a typical forward engineering approach, for example A requirements developer should possess the ability to develop more abstract requirements from less abstract requirements or design, such as developing low-level requirements from source code, or high-level requirements from low-level requirements An architecture developer should possess the ability to develop architectural descriptions from some detailed implementation representation, such as developing the architecture description from the source code A source code developer should possess the ability to develop source code, for which the low-level requirements and architectural descriptions may be incomplete or nonexistent, from existing executable object code Configuration management of life cycle data is critical during reverse engineering processes as the products are base-lined or versioned in reverse order from a forward engineering process. An effective problem reporting system should be structured to accommodate issues that may be unique to a reverse-engineering approach. For example, a problem report may be created to form a record of discussion between a SME and the development team in resolving a disagreement about how a function should be implemented and the associated requirements for the function Quality assurance records should document in-process audits conducted to ensure process plans and standards are being followed. While this is not specific to reverse engineering, it may be more difficult to track the information flow, therefore requiring additional attention. 5-2

18 5.3.7 In a reverse engineering process, the standards for both an existing LAL artifact and the associated MAL artifact to be developed should be complete before development of the MAL is started. This will enable the engineer developing the traceability between abstraction layers to verify that each representation complies with the applicable standard. The standards used for a reverse engineering process will typically be the same as those used for a forward engineering process Your PSAC and other planning documents should describe the reverse engineering processes and how they will satisfy the applicable DO-178C/B objectives. This includes a description of the development and integral processes and how sequencing will be accomplished, including transition criteria. You should describe the qualifications of SMEs that will be used and how they will participate in the project The process description should include the activities that transform the inputs to the outputs, and include defined entry and exit criteria An input to the process is the LAL artifact. This may be a complete artifact or part of an artifact. The input may also be a COTS component, such as a math library source file with functions required by the software. The LAL artifact should be identified and versioned, and controlled in a configuration management system Outputs from the process include the developed MAL artifact and LAL artifact that satisfy the applicable objectives. The reverse engineering life cycle process may handle development of an MAL artifact in more than one manner, and depends on how the process is defined. Two examples follow A complete set of low-level requirements for a library of functions may exist before the rest of the software low-level requirements are developed. Provided this partially completed MAL artifact is configuration managed, it can be further developed to the next level MAL. As additional portions of low-level requirements are developed, they are added to the configuration-controlled MAL. The completed MAL product would be the aggregate of partial MAL artifacts, which should be appropriately identified and recorded in the SCI An iterative life cycle process produces an artifact that, when complete, contains all of the low-level requirements. Each iteration accumulates requirements that have been developed. The artifact should be configuration managed, so only authorized changes are incorporated for each iteration. The final MAL product should be appropriately identified and recorded in the SCI Entry criteria should be defined to determine when a process may be entered or re-entered. For example, when developing low-level requirements from 5-3

19 source code, the transition criteria for developing the low-level requirements require that the source code be under configuration control and verified to comply with the coding standards Exit criteria will depend on the process that is established for development of the MAL artifact. For example, if developing low-level requirements from source code for a math library, the partial artifact could be the lowlevel requirements for a single math function. Another partial artifact could be the low-level requirements for a different math function from the same math library. If only complete baselines are used, all of the code could be reviewed against the coding standard, followed by the development of all of the low-level requirements for that code, followed by the development of high-level requirements for those low-level requirements. This requires careful configuration control as well as problem report tracking, because an iteration of the MAL product could contain many updates The completion criteria of the artifact depend on how the artifact is organized and managed. If the artifact is a single entity, such as a complete low-level requirements document, the completion criteria should include the change history of all changes made to the document showing how each change was dispositioned. If the artifact is organized hierarchically, the completion criteria should also be organized hierarchically. For example, if all of the functions in a math library have had low-level requirements developed, the low-level requirements representing the math library itself can be completed, because at that point its completion can be confirmed The difference cited between abstraction layers should demonstrate a thorough understanding of the two representations being traced and verified. A reverse engineered higher layer representation should be able to be transformed to an equivalent lower level representation. Following are two examples that illustrate potential verification issues Testing based on low-level requirements represented as pseudocode that is a simple restatement of the source code will only exercise the implementation and not the expected behavior and functionality. The ability of low-level testing to detect incorrect functionality, missing functionality, and unintended functions will be severely compromised. 5-4

20 If the gap between high-level requirements and low-level requirements is too great, it will be difficult to verify that the high-level requirements are a correct representation of the lowlevel requirements, and derived requirements may be missed. In this case, links could be established to provide traceability and annotated with additional information to help with understanding and verification of the relationships. The result should be equivalent to a forward engineering process where high-level requirements would be developed into equivalent low-level requirements using sound design decisions Verification, in general, is independent of the order of development. For example, the verification process is not sensitive to whether source code was developed from low-level requirements or low-level requirements were developed from source code, assuming completeness is assured at either end. Each artifact produced is evaluated for correctness. Two-way traceability ensures that the artifacts are a faithful representation of each other In forward engineering, each individual development process uses an artifact assumed to be correct as input. Errors introduced in the process can be detected by analysis, reviews, or tests. For example, where source code is developed from low-level requirements, the low-level requirements are assumed to be correct, and discrepancies between the low-level requirements and source code will result in changes to the source code In a reverse engineering process, the correctness of the input to the process may not be known. For example, having only the source code for math libraries does not ensure the source code properly implements the math libraries, and the math functions that are implemented may not be known. Therefore, there should be a means of establishing the correctness and completeness of the final MAL and LAL product If any system requirements are found to be inadequate or incorrect, the reverse engineering process should capture the issues and refer them to the system processes for resolution. The appropriate SME should be involved in liaising with the system processes The inputs to the verification process include the highest level of software requirements, which should be traced to a set of system requirements. This may be a complete or partial artifact. In the case of an iterative development process, only a specific subset of the reverse engineered highlevel requirements may be available for a particular iteration When reviewing each reverse-engineered artifact, you should determine If there is more functionality in the LAL than is required by the MAL, and if all of the LAL elements are necessary, 5-5

21 If there is less functionality in the LAL than is required by the MAL, and if the LAL elements are sufficient, If the translation between the LAL and the MAL is correct, and if errors may have been introduced in the LAL to MAL translation Problem reports should identify problems discovered throughout the reverse engineering process, as well as issues raised by developers because of incomplete information or to confirm specific assumptions. Problem reports should also document discrepancies revealed by the SME s assessment of the reverse-engineered MAL artifact. 5-6

22 CHAPTER 6. ASSURING PARAMETER DATA ITEMS (PDI) 6.1 When to Apply this Chapter. This chapter addresses PDI and supplements the guidance in DO-178C. You should follow the guidance of this chapter when you use PDI in your project, along with the applicable guidance in DO-178C. 6.2 Background DO-178C introduces the terms PDI and PDI file. PDI influences the behavior of the software without modifying the EOC, and is managed as a separate configuration item. A PDI file is an instantiation of the PDI where each data element has a defined value. A framework for addressing PDI and PDI files has been incorporated throughout DO-178C, and several objectives and activities are identified as applicable to PDI. Objectives A5-8 and A5-9 (refer to DO-178C, annex A, table A-5) apply if verification of PDI can be accomplished separately from the EOC (refer to DO-178C, section 6.6). Additional information on PDI is provided in DO-248C, Discussion Paper #20. DO-178C does not address databases that do not fit the definition of PDI. 6.3 Assuring PDI Using DO-178C DO-178C states that a PDI is a separately managed configuration item. Therefore, PDI life cycle data should be managed accordingly and applicable DO-178C objectives satisfied. This can present challenges when PDI files are created and verified after the initial software development process is completed, such as during production or after an article or product incorporating the PDI has been in service. These challenges include the following The PDI life cycle data may not be under the control of the EOC software developer The entity responsible for approval, configuration control, and maintenance of the PDI life cycle data may not have the appropriate software assurance infrastructure and processes in place Future changes to the PDI may require access to archived PDI life cycle data, and the changes may not be accomplished by the originator or the maintainer of the life cycle data Your plans should address the following, as applicable and in addition to that required by DO-178C, when you plan to use PDI in your project How PDI life cycle data will be approved and managed if it is to be accomplished outside of your software development assurance processes. You should describe how all applicable objectives will be satisfied. In some 6-1

23 cases, it may not be appropriate to rely on external processes to accomplish software assurance activities for PDI How you will ensure compatibility between combinations of PDI files and EOC versions (refer to DO-248C, section ) When a range of valid PDI element values are defined, the effect of implementing incorrect element values that are within the acceptable range for a particular application, and how the effect will be mitigated Changes to structure, attributes, or values of a PDI or the software component using it should be verified as applicable (refer to DO-248C, section ) How corrupted PDI files will be handled (refer to DO-248C, section ) How any tools used to create a PDI file will be qualified per DO-178C, section The proposed tool criteria and qualification level should be justified according to The robustness of the EOC regarding the PDI structure and attributes, The failure effects of incorrect element values created by the tool, The software level of the EOC, How objectives A5-8 and A5-9 are satisfied, and Whether or not the tool automates verification of compliance to objectives A5-8 and A

24 CHAPTER 7. ADDRESSING EXTRANEOUS CODE 7.1 When to Apply this Chapter. DO-178B addresses dead code, which is a subset of extraneous code, a term introduced in DO-178C. You should follow the guidance of this chapter if you are using a DO-178B process, and your software includes extraneous code. 7.2 Addressing Extraneous Code. While the definition of dead code is limited to executable object code, extraneous code includes code found at the source or object code level that may or may not be accessed and is not traceable to a software or system requirement. DO-178B states that dead code should be removed prior to software approval. However, extraneous code may be discovered late in the verification process and removal of such code may not be feasible for the current software release. If you are using a DO-178B process, you should refer to DO-178C guidance on extraneous code. 7-1

25 CHAPTER 8. QUALIFICATION OF SOFTWARE TOOLS USING DO-178B 8.1 When to Apply this Chapter. You should follow the guidance of this chapter in addition to DO-178B, section 12.2, when your software development or verification tool meets the criteria requiring tool qualification. 8.2 Software Tool Qualification Your PSAC should include a listing of all software tools and justification for why each tool does or does not require qualification DO-178B, section 12.2, provides guidelines for data required to support tool qualification. Table 8-1 summarizes the qualification data requirements for each type of tool. Table 8-1. Tool Qualification Data Data Item Verification Tool Qualification Applicability Development RTCA/DO-178B Reference(s) Plan for Software Aspects of Certification (PSAC) Required Reference Tool Qualification Plan 12.2, a, & Tool Qualification Plan Optional Required Tool Operational Requirements Required Required a(1), , & c(2) & Software Accomplishment Summary (SAS) Required Reference Tool Accomplishment Summary Tool Accomplishment Summary Optional Required c(3) & Tool Verification Records (test cases, procedures, and results) Required Required Tool Qualification Development data (requirements, design, and code) Not Applicable Required

26 8.2.3 You should document the following information for verification and development of Tool Operational Requirements, in addition to that described in DO-178B, section The tool s functionality in terms of specific requirements verified as part of the tool s qualification tests A description of the tool s operational environment, including operating system and any other considerations (for example, an analysis of what the tool will not do and what is required to cover that shortage, such as extensions to checklists or test cases, and any specialized hardware requirements, such as processors, special test equipment, or interfaces) Any other information necessary for the tool s installation or operation. In some cases the user s manual or other documentation may contain the needed information. Where additional information is included over and above the required information, the required information should be clearly identified. In the case where there is insufficient information from the tool supplier, you should provide the missing information DO-178B, section , states that for verification tools, verification under only the normal operating conditions is required. You should ensure those features or portions of the verification tool that are not used have no adverse effect on the features and portions being used. If additional features are used later, additional verification will be required DO-178B, section 12.2, states that only deterministic tools may be qualified, where the same input data results in the same output when operating in the same environment. However, if verification of the output demonstrates that all possible variations of the output from a given input are correct, the tool may be considered deterministic and may be qualified. This applies to tools where variations of the output do not adversely affect the intended use of the output, and correctness of the output is demonstrated. However, this does not apply to tools that have an effect on the final executable image embedded into the airborne system. In this case, generation of the final executable image should not vary for a given input Qualified tools should be kept under configuration management. DO-178B, section b, specifies that the control category for development tool qualification data should be CC1, and the control category for verification tool qualification data should be CC2. However, you may apply the same control categories to your development tool qualification data that is required for airborne software of the same level. Refer to the Control Category columns in the DO-178B annex A tables for the CC1 and CC2 applicability to your development tool qualification data. All verification tool qualification data may be categorized as CC A tool change impact analysis should be conducted for all changes to qualified tools. Your analysis should be thorough enough to assess the impact of the tool change on the 8-2

27 product, as well as other tools influenced by the change. Your tool change process should include conducting regression testing to verify all changes were correctly implemented and ensure there are no adverse effects on tool operation. 8-3

28 CHAPTER 9. APPLYING DO-178B CRITERIA TO LEVEL D SOFTWARE 9.1 When to Apply this Chapter. You should follow the guidance of this chapter when applying DO-178B to level D software. This guidance does not apply to other software levels or when using DO- 178C. 9.2 Background. The primary intent of the Level D software objectives is to provide a thorough investigation of the functional behavior of the software and to provide the necessary configuration control. However, experience has shown that some of the DO-178B objectives may have been misinterpreted when applied to Level D software. This chapter aligns the DO-178B level D objectives with the applicable objectives when using DO-178C. 9.3 Assuring Level D software. For a project involving Level D software, you should use the following guidance Ensure software plans exist and have been followed, including PSAC, Software Development Plan, Software Configuration Management Plan, Software Quality Assurance Plan, and SVP Ensure the plans enable compliance to the DO-178B objectives applicable to Level D software. However, objectives 4, 5, and 6 in DO-178B, table A-2, do not need to be satisfied Ensure low-level requirements, software architecture, derived low-level requirements, and source code are defined and exist for Level D software. If Level D software is partitioned from Level E software, ensure that an architecture description sufficiently ensures that partitioning integrity can be confirmed. At your discretion, evaluation of objectives 4, 5, and 6 in DO-178B, table A-2, is optional Confirm the safety assessment, system architecture, and software level determination, and verify a failure condition or malfunction of the Level D software can cause or contribute to no worse than a minor failure condition. 9-1