Certification Cost Estimates for Future Communication Radio Platforms

Transcription

1 Certification Cost Estimates for Future Communication Radio Platforms NOTICE: Price indications in this document are based on information from negotiated prices. Therefore, such prices are indicative and they may not constitute a negotiation base between parties. Edition Number : 1.1 Edition Date : Status : Issue 1 Intended for : EUROCONTROL Edition Number: 1.1 Issue 1 Page: C-i

2 DOCUMENT CHARACTERISTICS TITLE Certification Cost Estimates for Future Communication Radio Platforms Document Identifier Edition Number: 1.1 D3 Edition Date: Abstract This document presents detailed information about certification cost estimation for future communication radio platforms to be utilised for Air Traffic Control data communications beyond Certification cost estimations are provided for certification levels D to A as no failure mode and hazard analysis for the operational use of the future radios have taken place yet. The study addresses different certification aspects and respective costs for different platform options: 1. Pure COTS products 2. SDR Forum based developments (SCA) 3. FPGA based developments 4. Reprogrammable processors Keywords Certification COTS ED-12B / DO-178B ED-14E / DO-160E ED-80 / DO-254 FPGA IEEE802.16e MAC layer Physical Layer SDR Contact Person(s) Tel / Unit Eric THOMAS +33 (0) Rockwell Collins France ethomas@rockwellcollins.com Filename: ELECTRONIC SOURCE RCF_CertCostEstimate_D3-v00.doc Host System Software Size Windows XP Microsoft Word 2003 SP Edition Number: 1.1 Issue 1 Page: ii

3 DOCUMENT APPROVAL The following table identifies all management authorities who have successively approved the present issue of this document. AUTHORITY NAME AND SIGNATURE DATE Edition Number: 1.1 Issue 1 Page: iii

4 DOCUMENT CHANGE RECORD The following table records the complete history of the successive editions of the present document. EDITION NUMBER EDITION DATE INFOCENTRE REFERENCE REASON FOR CHANGE PAGES AFFECTED First working draft all 0.2 dd Second working draft all 0.3 dd Third working draft all 0.4 dd th working draft including: - comments from EUROCONTROL - COTS information added - MBD information added - Supplement on RTOS added Deliverable 1 version, includes: - comments from EUROCONTROL - DO-254 additional information - COTS and Reprogrammable solutions - Summary and Conclusion Deliverable 3 version, includes: - comments from EUROCONTROL - Summary and Conclusion reworked per D2 presentation all all all Edition Number: 1.1 Issue 1 Page: iv

5 CONTENTS DOCUMENT CHARACTERISTICS...ii DOCUMENT APPROVAL...iii DOCUMENT CHANGE RECORD...iv CONTENTS...v EXECUTIVE SUMMARY...ix 1. SCOPE APPLICABLE DOCUMENTS REFERENCE DOCUMENTS GLOSSARY OF TERMS BACKGROUND Avionics Certification EUROCAE ED-12B / RTCA DO-178B Overview Software Life Cycle Processes Special Considerations Use of Previously Developed Software Upgrading a Development Baseline Tool Qualification Alternative Methods RTOS in Certification Breakdown of DO-178B Objectives EUROCAE ED-12C / RTCA DO-178C Model-Based Development Impact on certification Pros and cons Costs EUROCAE ED-80 / RTCA DO-254 Overview Hardware Life Cycle Processes...19 Edition Number: 1.1 Issue 1 Page: v

6 5.3.2 Tools D0254 cost versus D D0254 and COTS D0254 and embedded IP Conclusion on DO 254 Costs (general) EUROCAE ED-14E / RTCA DO-160E Overview Major ED-14E / DO-160E Activities and Artefacts Qualification Test Plan Qualification Test Campaign Qualification Test Report Metrics Avionics Equipment development process life cycle VDLmode4 example VDLmode2 example Internal vs External DER Costs Security Aspects Background Status for Aeronautical Data Links Security Assurance Levels versus Development Assurance Levels CERTIFICATION COST ESTIMATES Generic Avionics Transceiver Functional Overview Baseband Module Front End Module Other Modules and Features Development Work Breakdown Structure IEEE e Background IEEE802.16e Physical Layer IEEE802.16e MAC Layer SDR-based Development Platform Overview RTOS and BSP CORBA Middleware Layer Core Framework Radio Devices Radio Services Certification Cost Estimates Matrix...48 Edition Number: 1.1 Issue 1 Page: vi

7 Considerations Matrix COTS Products-based Development COTS RF Chipsets COTS Baseband Chipsets Platform Overview Identified issues Pros and cons Certification Cost Estimates Matrix Alternative COTS Platform Overview FPGA-based Development Platform Overview Certification Cost Estimates Matrix Reprogrammable Processor-based Development Platform Overview Certification Cost Estimates Matrix CERTIFICATION COST ESTIMATES SUMMARY Summary Conclusions Future Standards and Processes Platforms and Certification Costs Estimates Recommendations...68 APPENDIX APPENDIX 1: SDR DATA Terminology APPENDIX 2: COTS PRODUCTS DATA RF Modules: TI s TRF Baseband Modules: Intel s Baseband Modules: Sequan s SQN APPENDIX 3: FPGA DATA...7 Edition Number: 1.1 Issue 1 Page: vii

8 FPGA: Xilinx s Virtex FPGA: Xilinx s Virtex4-QPro APPENDIX 4: REPROGRAMMABLE PROCESSOR DATA Reprogrammable Processors: picochip s PC APPENDIX 5: TOOLS and COTS Software FPGA Design: Mentor Graphics ModelSim Designer Model-Based Development: Esterel s SCADE Suite ORB: OIS ORBexpress (FPGA version) OS: LynuxWorks LynxOS Edition Number: 1.1 Issue 1 Page: viii

9 EXECUTIVE SUMMARY This document presents detailed information about certification cost estimation for future communication radio platforms to be utilised for Air Traffic Control data communications beyond Certification costs are based on the IEEE e (Mobile WiMAX) standard, as: IEEE e has been selected as the Future Communication Infrastructure (FCI) technology to be used for airport surface operations. The FCI radios will operate in the C-band between and GHz. IEEE e presents many elements and functionalities that could be used for L-band Digital Aeronautical Communication System -1 (LDACS-1) radios yet to be standardized. These FCI radios will be used for in-flight operations, and will operate in the L-band. Certification cost estimations are provided for certification levels D to A as no failure mode and hazard analysis for the new radio components of the FCI have taken place yet. The study is aiming at providing EUROCONTROL with a clear picture in different certification aspects and respective costs for different platform options: 1. Pure COTS products 2. SDR Forum based developments (SCA) 3. FPGA based developments 4. Reprogrammable processors Edition Number: 1.1 Issue 1 Page: ix

10 THIS PAGE INTENTIONALLY LEFT BLANK. Edition Number: 1.1 Issue 1 Page: x

11 1. SCOPE This study is undertaken in response to EUROCONTROL Technical Specification for Certification Cost Estimation for Future Communication Radio Platforms (Price Enquiry N E). The study will: Provide background information on software and hardware certification, processes, methods and standards Determine metrics for the estimation of certification costs for Design Assurance Level (DAL) D to DAL A designs Establish a baseline through the definition of a generic IEEE802.16e transceiver Derive certification costs for different implementations of the baseline transceiver Draw conclusions and recommendations In order to accomplish these objectives: Chapter 5 provides background information on avionics certification, and describes software development standards (ED-12B/DO-178B) as well as hardware development standards (ED-80/DO-254), and environmental conditions qualification. This chapter also discusses ongoing activities on ED-12C/DO-178C and model-based development. Security aspects and their relation with safety criticality are approached. Chapter 6 provides the certification estimates for 4 different architectures, i.e. software defined radio, commercial off the shelf, FPGA and finally reprogrammable processors. Cost matrices provide costs estimations based on effort (person.day), and prices of necessary tools are given for information. Indeed, prices are always negotiated between their provider and the customer, and may differ from one to the other. Chapter 7 summarizes the information collected during the study. Finally, appendices gather data sheets on main products discussed in the study. Edition Number: 1.1 Issue 1 Page: 1

12 2. APPLICABLE DOCUMENTS Document Number Date Document Title PE N E 05-Mar-08 Technical Specification, Certification Cost Estimation for Future Communication Radio Platforms EUROCONTROL 3. REFERENCE DOCUMENTS Document Number Date Document Title [AVI-HDBK] 2000 The Avionics Handbook Cary R. Spitzer [Part 21] Part 21 Certification procedures for products and parts [CS 23] Certification Specifications for Airworthiness of Normal, Utility, Aerobatic, and. Commuter Category Aeroplanes [CS 25] Certification Specifications for Airworthiness of Large Aeroplanes [IEEE802.16e] 2005 Air Interface for Fixed Broadband Wireless Access Systems [RTCA DO-160E] IEEE Task Group, Feb Environmental Conditions and Test Procedures for Airborne Equipment [RTCA DO-178B] 1992 Software Considerations in Airborne Systems and Equipment Certification RTCA SC-167 / EUROCAE WG-12 [RTCA DO-254] Design Assurance Guidance for Airborne Electronic Hardware [RTCA DO-278] Guidelines for Communications, Navigation, Surveillance, and Air Traffic Management (CNS/ATM) Systems Software Integrity Assurance [SAE ARP-4754] 1996 Certification Considerations for Highly-Integrated [SAE ARP-4761] or Complex Aircraft Systems Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment. IEEE e Task Group (Mobile WirelessMAN ) Edition Number: 1.1 Issue 1 Page: 2

13 4. GLOSSARY OF TERMS Term 16QAM 64QAM ABD-100/200 AC AGC ALC ANSP ASIC ATC ATN BER BS BSP CF CMM CMMI CORBA COTS DAL DER DFE DFE EAL EMC EMI FCI FEC FFPA FFT FHA FPGA GPP HIRF ICAO IF IPS JAA JTRS LDACS LMS MAC MASPS MBD MHAL MIMO MOPS NLOS OE ORB Meaning 16-Quadrature Amplitude Modulation 64-Quadrature Amplitude Modulation Advisory Circular Automatic Gain Control Automatic Level Control Air Navigation Service Provider Application Specific Integrated Circuit Air Traffic Control Aeronautical Telecommunication Network Bit Error Rate Base Station Board Support Package Core Framework Capability Maturity Model Capability Maturity Model Integration Common Object Request Broker Architecture Commercial Off The Shelf Design Assurance Level Designated Engineering Representative Digital Front End Decision Feedback Equalizer Evaluation Assurance Level Electro-Magnetic Compatibility Electro-Magnetic Interference Future Communication Infrastructure Forward Error Correction Functional Failure Paths Analysis Fast Fourier Transform Functional Hazard Analysis Field-Programmable Gate Array General-Purpose Processor High Intensity Radiated Field International Civil Aviation Organization Intermediate Frequency Internet Protocol Suite Joint Aviation Authority Joint Tactical Radio System L-band Digital Aeronautical Communication System Least Mean Square Media Access Control Minimum Aviation System Performance Standard Model-Based Development Modem Hardware Abstraction Layer Multiple Input Multiple Output Minimum Operational Performance Specifications Non Line of Sight Operating Environment Object Request Broker Edition Number: 1.1 Issue 1 Page: 3

14 Term OS PHAC PLD POSIX PSAC QoS QPSK RD RF RLS RS RSS RTOS SAL SCA SDR SLOC SS SWaP-C VoIP WiMAX WRC Meaning Operating System Plan for Hardware Aspects of Certification Programmable Logic Device Portable Operating System Interface (UNIX) Plan for Software Aspects of Certification Quality of Service Quaternary Phase Shift Keying Radio Devices Radio Frequency Recursive Least Squares Radio Services Radio Security Services Real-Time Operating System Security Assurance Level Software Communications Architecture Software Defined Radio Software Line Of Code Subscriber Station Size, Weight and Power - Cost Voice over IP Worldwide Interoperability for Microwave Access World Radio Conference Edition Number: 1.1 Issue 1 Page: 4

15 5. BACKGROUND 5.1 Avionics Certification Most if not all aspects of design, production, maintenance and operation of civil aircraft are subject to extensive regulation by governments. Certification is a critical element in the safety-conscious culture on which aviation is based [AVI-HDBK]. The purpose of avionics certification is to document a regulatory judgement that an airborne system meets all applicable regulatory requirements, can be manufactured properly and finally installed safely onboard an aircraft. Certification authorities will show a great interest for four engineering topics: 1. System requirements: Requirements capture is often considered by avionics developers as the most important technical activity. The resulting system specification will indeed describe normal and abnormal operations, performance, safety, functional testing, training and maintenance procedures. 2. Safety assessment 1 : As early as possible in the project, developers should address the aircraft-level hazards associated with their proposed system, as there is an explicit correlation between the severity of a system s hazard and the scrutiny to which the system will be subjected. Initial considerations of hazards are formalized in a Functional Hazard Assessment (FHA). 3. Environmental qualification: Environmental qualification is required for avionics to determine the performance of the equipment/system to its applicable environmental conditions. It is the responsibility of the applicant to identify the environmental conditions to which their systems may be exposed and to specify the applicable environmental and EMI/EMC requirements and the appropriate test methods (e.g. applicable environmental test conditions and test procedures for temperature, altitude, humidity, vibrations, susceptibility to radio frequencies, etc, as specified by RTCA DO-160E). The applicable environmental qualification levels (applicable RTCA DO-160E categories) represent the most severe environment to which the equipment/system is expected to be regularly exposed during its service life. Environmental testing must be performed on test units representative of the final products, and may be witnessed by specialists designated by certification authorities. 4. Software assurance: Since software has become increasingly important in avionics development, it is frequently the dominant consideration in certification planning. Regulatory compliance for software will show conformance to guidelines as described in EUROCAE ED-12B / RTCA DO-178B. These are assurance standards and not development standards for software, and remain 1 Safety assessment will not be discussed further in this study. As no safety analysis was done for the FCI components, this study addresses Level D (i.e. minor failure condition) up to Level A (i.e catastrophic failure condition). This way the outcome of this study is indicative and representative of the expected costs for all FCI new radio developments (including the L-band radio platforms). Edition Number: 1.1 Issue 1 Page: 5

16 neutral with respect to development methods. Though free to choose their own methods, developers will have to demonstrate satisfaction of assurance criteria in the areas of planning, requirement definition, design and coding, integration, verification and validation, configuration management and quality assurance. In recent years, certification authorities have paid growing attention to programmable logic devices (PLD) such as application-specific integrated circuits and field-programmable gate arrays (FPGA). Findings of compliance were often handled by designated software specialists from certification authorities, and industries and governments have specified acceptable practices for assurance of all avionics hardware design processes through guidelines provided by RTCA DO-254. SAE ARP-4754 is an overarching safety standard dealing with the systems development process of aviation systems and is intended to provide guidance directed toward highly-integrated or complex systems for demonstrating their compliance with applicable airworthiness requirements. It references RTCA DO-178B for software and RTCA DO-254 for hardware, as well as SAE ARP for safety engineering techniques. Nevertheless, SAE ARP-4754 has not been adopted by the industry which rather refers to aircraft manufacturer own procedures (e.g. ADB-100/200) or Part 21 and relevant CS 2 (e.g. CS 23 and CS 25). As any other development activity, certification can be straightforward when properly managed, and applicants hold early discussions with appropriate certification authorities designated personnel: At the very beginning of the project when the applicant and the regulators jointly define their expectations on both sides. During development when open communication is maintained among suppliers, customers and regulators. Evidence of compliance with regulatory requirements will then be produced with little incremental effort as a side-effect of good engineering practices. Certification should follow in the end, as the complete demonstration of compliance will result from the joint efforts cumulated along the project. An overall system development process is depicted Figure 1. 2 Unlike Standards or Guidelines developed by EUROCAE, RTCA or SAE, members of which include industry, academies, service providers and regulators, CS are developed by EASA based on previous work (e.g. JAR) from certification authorities. Edition Number: 1.1 Issue 1 Page: 6

17 Figure 1: Overall System Development Process 5.2 EUROCAE ED-12B / RTCA DO-178B Overview EUROCAE ED-12B / RTCA DO-178B, provides the international aviation community with guidelines for determining, in a consistent manner and with an acceptable level of confidence, that the software aspects of airborne systems and equipment comply with airworthiness requirements [RTCA DO-178B]. These guidelines for the production of software for airborne system: Discuss the objectives for software life cycle processes Describe activities and design considerations for achieving those objectives Describe the evidence that indicate that the objectives have been satisfied DO-178B defines five software levels based upon the contribution of software to potential failure conditions as determined by the system safety assessment process. The software level implies that the level of effort required to show compliance with certification requirements varies with the failure condition category: 1. Level A: Software whose anomalous behaviour would cause or contribute to a failure of system function resulting in a catastrophic failure condition, resulting in many fatalities including the crew. A level A failure might occur 1 in 1 billion flights. 2. Level B: Software whose anomalous behaviour would cause or contribute to a failure of system function resulting in a hazardous/severe-major failure condition, i.e. would reduce the Edition Number: 1.1 Issue 1 Page: 7

18 capability of the aircraft or its crew to cope with adverse operating conditions potentially leading to fatal injuries to a small number of occupants. A level B failure might occur 1 in 10 million flights 3. Level C: Software whose anomalous behaviour would cause or contribute to a failure of system function resulting in a major failure condition, i.e. would reduce the capability of the aircraft or its crew to cope with adverse operating conditions potentially leading to discomfort and possibly injuries to occupants. A level C failure might occur 1 in 100,000 flights. 4. Level D: Software whose anomalous behaviour would cause or contribute to a failure of system function resulting in a minor failure condition, i.e. would not reduce significantly aircraft safety but potentially leading to inconvenience to occupants 5. Level E: Software whose anomalous behaviour would cause or contribute to a failure of system function resulting in a no-effect failure condition, i.e. would not affect the operational capability of the aircraft or increase crew workload. No guidelines apply once software has been confirmed as level E by the certification authority Software Life Cycle Processes Software life cycle processes include: 1. The software planning process: definition and coordination of activities undertaken during the software development and integral processes for a project, i.e. definition of means for producing software which will satisfy system requirements and provide a level of confidence consistent with airworthiness requirements. The software planning process should also consider particular features of the coding language and compiler (e.g. code performance optimization). 2. The software development processes: production of software products, including sub-processes such as the software requirement process, the software design process, the software coding process and the integration process, i.e. definition and refinement of requirements allocated to software from the system requirements, implementation of source code and generation of an executable object code. 3. The integral processes: assurance of the correctness, control and confidence of the software life cycle processes and outputs, including sub-processes such as the software verification process, the software configuration management process, the software quality assurance process and the certification liaison process Artefacts produced during the software life cycle depend on the activities to be undertaken in regard to the selected software level. Figure 2 depicts main software development activities and outcomes. Edition Number: 1.1 Issue 1 Page: 8

19 Figure 2: Overview of DO-178B Activities and Outcomes Major differences in the software life cycle processes depending on the selected software level lie in the thoroughness and rigor of activities (and associated artefact) for testing and integration, but above all verification processes. These have a significant impact on the development effort, hence cost, in particular for Level A software requiring tests for robustness and independence for all verification activities to be performed Special Considerations Use of Previously Developed Software Modification to previously developed software already compliant with DO- 178B may result from requirements changes, detection of errors or software enhancements. Impacts of the modifications shall be thoroughly analysed as they may have consequences on the software architecture, other requirements and/or coupling between other components Upgrading a Development Baseline DO-178B provides guidance for the acceptance of COTS software, or airborne software developed using other guidance or a lower software level. In brief, software life cycle data, including software aspects of certification, from the previous development shall be thoroughly analysed and evaluated to ensure that the new software verification process objectives are satisfied. If deficiencies exist in the software life cycle data, the data should be augmented to satisfy the objectives of the new application. Edition Number: 1.1 Issue 1 Page: 9

20 Tool Qualification DO-178B requires qualification for tools meant to eliminate, reduce or automate any software development process without its output being verified. These tools basically are software development tools (e.g. compilers) and software verification tools (e.g. static analyzer). The objective of the tool qualification process is to ensure that the tool provides confidence equivalent to the processes being eliminated, reduced or automated. DO-178B provides guidelines for tool qualification, which are similar to the qualification of the software application being developed using that tool Alternative Methods DO-178B discusses a set of alternative methods that can facilitate software development and still fulfil the objectives of the development processes. Formal methods used to improve software specification and verification, exhaustive input testing used as a substitute for software verification, or, product service history which can grant some credit to certification, are alternative methods that can be considered, presented in the PSAC documentation and discussed with certification authorities RTOS in Certification Certification of Real Time Operating System (RTOS) is always an issue for users having developed (or integrated) an RTOS used in airborne systems and wanting it to be certified under DO-178B. DO-178B is much more than just a certification artefact checklist; it is a process that starts the same day as the development project itself. Hence, it is difficult if not nearly impossible to take an existing application that was not developed in cooperation with EASA and get it certified for any DO-178B levels, and definitely impossible for Level A applications (even though there are some guidance in DO-178B for upgrading COTS software, ref ). At the very beginning of a DO-178B development project, a document called Plans for Software Aspects of Certification is supposed to be created and supplied to the certification authority for approval. This plan describes the process and tools and procedures that the developer will utilize during the development and verification of the avionics software application. This plan also describes the application and its use in terms of safety impact on the aircraft. The developer and the certification authority will decide what level of safety criticality the application should be developed to and certified to. This determines the objectives and deliverables that must be met and produced during the development and verification phases of the project. An entire development effort would be necessary to take an existing application like a general purpose RTOS - not to produce the actual code itself, but to go back and develop all of the additional material necessary to support the certification, and get it certified under DO-178B. For example, DO- 178B Level A requires that verification testing include sufficient tests for 100% MCDC level object code coverage, and all of the tests are required to be Edition Number: 1.1 Issue 1 Page: 10

21 traceable back to specific software requirements that the code was designed to satisfy. There are a lot of objects to be met and deliverables required to be produced and sent or made available to the certification authority (source T- VEC). Note: It indeed took Rockwell Collins over 40 person years to certify LynxOS to level A, to become LynxOS Breakdown of DO-178B Objectives DO-178B Annex A provides a levelling tabulation of the objectives, and illustrates which data item should include the objective evidence, how the evidence must be controlled and where independence is required in achieving the objective. Annex A may be viewed as a checklist for applicants adopting DO-178B for certification purposes. [AVI-HDBK] DO-178B contains objectives for the entire software development lifecycle which are summarized in Annex A tables. Figure 3 depicts the repartition of objectives per DAL over the software development lifecycle. Figure 3: Breakdown of DO-178B Objectives As can be seen on Figure 3, the main differentiator between levels is related to Verification activities, with respectively 8 objectives (out of 28) for DAL D, 27 (out of 57) for DAL C, 39 (out of 65) for DAL B and 40 (out of 66) for DAL A. The following table lists the different assurance activities to be undertaken per level. DAL Assurance Activity Level D Planning (28 objectives) Configuration Management Quality Assurance High Level Requirement Coverage High Level Requirement Robustness Certification Liaison Tool Qualification Edition Number: 1.1 Issue 1 Page: 11

22 DAL Level C (57 objectives) Level B (65 objectives) Level A (66 objectives) Assurance Activity Level D, plus: More planning Verification Requirements Design and Integration Process Low Level Requirements Coverage Verification Plan, Procedures & Results Statement, Data & Control Coverage Level C, plus: Traceability Verifiability Independence Decision Coverage Transition Level B, plus: MCDC Coverage More Independence Source to Object Verification Table 1: Assurance Activities per DAL EUROCAE ED-12C / RTCA DO-178C As can be read above, ED-12B / DO-178B is process and requirements oriented. These guidelines, mainly developed for development of handwritten code and released in 1992, have not revealed major safety flaws. Nevertheless, software engineering has greatly evolved since, and lessons learned from its application by industry and certification authorities on one hand and by the EUROCAE ED-94B / RTCA DO-248 Clarification Group have led EUROCAE WG-71 / RTCA SC-205 to undertake modifications for a new version of ED-12B / DO-178B, ED-12C / DO-178C. Among the issue listed in Oct during RTCA Ad-hoc SW meeting (source QinetiQ MISRA Nov 2007), were: Model-based development (MBD) Development tool qualification criteria Separate documentation for CNS/ATM (EUROCAE ED-109 / RTCA DO-278) Security Guidance EUROCAE WG-71 / RTCA SC-205 sub-groups are addressing: Software modelling and the ability to use modelling to supplant some verification techniques Object oriented software and the conditions under which it can be used Software tools and their qualification Use of formal methods in software specification and design Work started March 2005, aiming at delivering EUROCAE ED-12C / RTCA DO-178C Guidance supplemented by EUROCAE ED-94C / RTCA DO-248C Edition Number: 1.1 Issue 1 Page: 12

23 Guidelines by December The delivery to industry for comment is now foreseen not before early Within DO178C work, the MDB topic is presently one of the less mature as no consensus is being reached especially on verification and tests, and the certification authorities have lots of comments. If EASA are confident with the use of qualified code generators and are convinced of its merits, this will reduce more than significantly the verification costs, FAA and part of the industry are less confident in auto coding, and less convinced with the benefits of using MDB. Indeed, these tools are seen as software development tools, i.e. formal languages (e.g. UML) or schematics (e.g. Simulink), and authorities as well as customer will still require high level requirement specifications in textual forms. Industry nevertheless would expect MDB to be used earlier in the development process, starting at customer requirement capture phases and system design, these tools being able to accommodate rapid prototyping, hence early validation of high level requirements. Moreover, suites like Esterel s SCADE or MathWorks Simulink include C code as well as VHDL autocoders, hence are suited not only for software development, but also system development as a whole. Similarly, Unified Modelling Language (UML) has been transformed into System Modelling Language (SysML) to cope with model based system design. A paper addressing the tool qualification aspects should be issued shortly. It seems DO-178-like recommendations for software development tools is on its way. This document is aiming at simplifying or at least clarifying the objectives to be achieved for verification, suppressing MCDC but keeping DC activities for high end software development tools. Tools will be categorized, and a matrix will indicate what category of tool is required for each software criticality level, i.e. tool category x DAL Model-Based Development A model-based (software) development (MBD) is based on use of formal method of modelling for software specification (definition of High Level Requirements through a specification language with rigorous semantics). The Model-Based Design models (used as Low Level Requirements) are then derived from these High Level Requirements for design purpose. The design models are the input artefacts for the software code generation, which is performed either by using a qualified automatic code generator tool or by manual coding with software developers. The typical waterfall scheme of a model-based approach is described hereunder: 3 [Note EUROCONTROL]: ESA has already an MDB based software (ADA) development environment in place since mid 80 for safety related software development for on board satellite equipment. Software had been developed according to PSS-01guidelines. Edition Number: 1.1 Issue 1 Page: 13

24 System Requirements Process Software Requirements Process (Formal Model) High Level Requirements Software Design Process (Model-Based Design) Low Level Requirements & Architecture Software Coding Process (Automatic Code Generation) Source Code Software Integration Process Figure 4: Waterfall scheme using modelling approach Today, several development suites offer modelling capability, but only few of them have qualified tools. The main one is Safety Critical Application Development Environment (SCADE) Suite ( and has a fully qualified chain of tools: Kernel Code Generator (KCG): automatic code generator, Compiler Verification Kit (CVK): check consistency between object code and C code, Model Test Coverage (MTC): check coverage analysis at model level. MathWorks Simulink Suite ( also offers a modelling approach but has no qualified tool for code generation. Nevertheless, gateways between Simulink models and SCADE exist, that would allow to use SCADE s qualified code generator on Simulink models. Initiatives launched recently with huge budgets aim at developing qualified code generation tools as well as qualified verification tools, foreseen to support efficiently the production of certification artefacts (if not eliminate the need to have them produced), hence are expected to reduce the overall expensive certification labour. The Toolkit in Open Source for Critical Applications & System Development (TOPCASED) ( initiative has been launched in 2004 in French AerospaceValley in order to develop a model-driven integrated development environment (IDE). Main partners on the project include industry (Airbus, Atos Origin, EADS, Rockwell Collins, THALES, Siemens VDO ), laboratories (ONERA, LAAS, CNRS ) and academies (ENSEEIHT, INSA, IRIT, INRIA, UPS ). Open source Edition Number: 1.1 Issue 1 Page: 14

25 versions are available for download and evaluation of a qualifiable code generator 4. All applications cannot be modelled as easily with an MBD such as SCADE or Simulink. These are suited for algorithms such as filters, mathematics with iterations, logical systems and state machines. Complex algorithms including many loops, or dealing with a large number of inputs or a large amount of data are not easily modelled. The result in that case may be a large size generated code, unless experienced developers build the models in order to achieve a 20-30% overhead compared to hand-coding Impact on certification Qualified tools of model-based development are the key components to lighten effort for software development activities and associated verification tasks for certification. For example, the use of qualified KCG tool avoids Code Reviews required for DO-178B, whereas using the qualified CVK tool reduces the Object Code Analysis effort required for DO-178B. Traceability is ensured through the use of SCADE Suite from System Requirements level to System Verification level. The impact on certification of using qualified tool for structural coverage analysis is a major debate today. It is not defensible for certification purpose to argue that structural coverage analysis (required at code level) could be simplified or even suppressed thanks to the use of the MTC tool: this tool only intervenes at model level and not at code level. Some proponents of modelbased development easily tend to do this shortage, yet to be approved by certification authorities. It is today difficult to have a clear view on return on investment of modelbased development. Too few avionics programs using this approach have been certified yet (e.g. Boeing s B787 Landing Gear System and Braking System Control Unit are qualified Level A by FAA, Dassault s F7X Full Digital Flight Control are qualified Level A by FAA and EASA, Airbus A380 Flight Control, Flight Warning System, Braking and Steering System and Cockpit Display System have been developed with SCADE as well). However, the suitability of SCADE Suite for certification constraints has been presented to FAA and judged DO-178B compliant by the FAA. A model-based development is estimated to bring a cost savings of approximately15-20% on the software application verification (which represents the main cost driver of certification). 4 The overall budget of the TOPCASED initiative is estimated to approx M and is partially covered through French R&D funding. The TOPCASED products cover a set of features (IDE modules) provided by individual partners of this open source consortium. Individual budgets are estimated to approx. 2-4 M each for the development of qualifiable tools. The initiative concerning the development of an automatic code generator is estimated to approx. 2 M. It is foreseen similar budgets will be necessary to actually qualify each tool. Edition Number: 1.1 Issue 1 Page: 15

26 5.2.8 Pros and cons There are pros and cons in using model-based development, as summarized hereafter. Pros Cons Easy prototyping for final user demonstration and requirement capture Models easier to review than code Facilitated debugging tests, validation on host rather than on target platform Simplified verification activities though the use of qualified tools Easier management of late specification changes (requirements upgrade) Cost savings for software development Greater development process automation (cascade of tools) High cost of qualified tools Automatic code generator 5 generates non-optimized (i.e. slightly larger) number of code lines (SLOC) for complex functions Complex functions (loops, large amount of data, sophisticated algorithms) are not easy to model Open debate on simplification of structural coverage analysis activities Too few FAA or EASA stamped certifications to assess real cost savings Table 2: Model Based Development Pros and Cons The following picture illustrates the expected benefits of using a fully qualified and automated MBD on the development process. As stated above, the process is approved by EASA, but FAA still need to be convinced. 5 Qualified automatic code generators usually provide the developer with C code, i.e. replace hand writing. This code will need to be either compiled by a qualified compiler specific to the target processor, or a basic compiler (e.g. GNU) provided the generated object code is verified (e.g. use of SCADE s CVK module). The Integrated Development Environment (IDE) would include the MDB tool, automatic code generator and the compiler. Edition Number: 1.1 Issue 1 Page: 16

27 Figure 5: Expected Benefits from Using MBD on the Development process Costs The following table lists orders of magnitude for SCADE Suite products: Product Unit Price ( ) SCADE Advanced Modeler per seat SCADE Simulink Gateway SCADE Qualification Package (KGC, Kit, CVK) SCADE MTC Yearly Maintenance: SCADE Advanced Modeler per seat SCADE Simulink Gateway SCADE Qualification Package SCADE MTC Investments are presently high to setup a fully automated and qualified MBD, along with the adequate process, for the first time on a product. Also, trade-off between investing in these suites and outsourcing software verification to India or China is still considered. The main gains on costs will be on future evolutions of the product line, once the baseline is established. It is foreseen MBD will then facilitate the handling of the software changes as well as reusability of previously developed modules in the product line or across product lines, and result in cost efficiencies - especially for large software developments. Edition Number: 1.1 Issue 1 Page: 17

28 5.3 EUROCAE ED-80 / RTCA DO-254 Overview EUROCAE ED-80 / RTCA DO-254 provide guidance for design assurance in airborne electronic hardware to ensure safe functions. It provides a framework of considerations for certification across the entire hardware engineering lifecycle. Alike DO-178B, DO-254 defines criticality or failure conditions which are determined using the system safety assessment process. The standard defines and assigns the exact same levels as DO-178B, ranging from Level A (catastrophic failure) to Level E (no effect). DO-254 also differentiates the rigor of the processes based on whether hardware is: Simple: when a set of comprehensive tests can be created to exhaustively determine correct functionality under all operating conditions Complex: when hardware cannot be categorized simple Simple hardware categorised items do not require extensive documentation of the design process. The supporting processes of verification and configuration management need to be performed and documented, but not extensively. There is thus a reduced overhead in designing a simple hardware item to comply with the DO-254. The main impact of the DO-254 is intended to be on the design of complex hardware items. The following figure (source HighRely) depicts where to apply DO-178B and DO-254 respectively. Figure 6: Scope of DO-178B & DO-254 Edition Number: 1.1 Issue 1 Page: 18

29 As figure 6 indicates, DO-178 is applicable for software running on a CPU from external memory. In case microcodes are used (running internal in the device) than DO-254 is applicable Hardware Life Cycle Processes Hardware life cycle processes include: 4. The planning process: definition and coordination of activities undertaken during the hardware design and support processes for a project, i.e. definition of means for producing hardware which will satisfy system requirements and provide a level of confidence consistent with airworthiness requirements. 5. The hardware design processes: design of hardware products, including sub-processes such as the requirement capture process, the conceptual and detailed design processes, the implementation process and the product transition process, i.e. definition and refinement of requirements allocated to hardware from the system requirements, production of the hardware device and preparation for the replication (manufacturing) of the device. 6. The support processes: assurance of the correctness, control and confidence of the hardware life cycle processes and outputs, including sub-processes such as the validation and verification processes, the configuration management process, the quality assurance process and the certification liaison process Artefacts produced during the hardware life cycle depend on the activities to be undertaken in regard to the selected software level. Figure 7 depicts hardware design activities and outcomes. Edition Number: 1.1 Issue 1 Page: 19

30 Figure 7: Overview of DO-254 Activities and Artefacts Tools Major differences in the hardware life cycle processes depending on the selected hardware level are: Hardware implementing Levels A and B functions requires to use advanced design assurance method(s) such as a systematic Functional Failure Paths Analysis (FFPA is a structured, top-down, iterative analysis used to determine that the hardware architecture and implementation complies with the safety requirements), and/or another advanced methods acceptable to the certification authority. The use of such design assurance methods is optional for hardware implementing Level C or lower level functions for which the design assurance may be provided using only the standard guidance provided by the DO-254 (section 3 to section 11). Traceability should be consistent with the design assurance level of the function performed by the hardware. For Levels C and D functions, only the traceability data from requirements to test is needed. Requirements, design, validation/verification, and archive Standard are required only for Levels A and B. Independent verification: all verification of Levels A and B functions should be independent. Level C and lower functions do not require independent verification. One key aspect of the DO-254 process is to determine that the tools used to create and verify designs are working properly. The process to ensure this is called tool assessment (though it is often mistakenly called tool qualification ). Tool qualification is one method of tool assessment. Edition Number: 1.1 Issue 1 Page: 20

31 The purpose of tool assessment (and potentially tool qualification) is to ensure that tools that automate, minimize, or replace manual processes for hardware design and/or verification perform to an acceptable level of confidence on the target project. Tools are classified as either design tools or verification tools, depending on which design flow processes they automate. Likewise, as mentioned previously, designs are designated with a criticality level that corresponds to the resulting severity of failure. The rigor of the tool assessment process depends on both the tool classification as well as the criticality level of the designated project. The tool assessment and qualification process takes one of three forms: 7. Independent Output Assessment: another independent tool or method must validate the results of the tool. 8. Relevant History: the tool has been previously used and has been shown to provide acceptable results. 9. Tool Qualification: a plan has been established and executed to confirm that the tool produces correct outputs for its intended application. Regardless of these classifications, the task of tool assessment falls upon the airborne applicant or airborne integrator (not the tool vendor). The applicant or integrator proposes the method of tool assessment as part of the DO-254 planning and documentation. The certification agency or its representative (DER) will determine if the proposed method of compliance to this requirement is adequate for the development process D0254 cost versus D0178 Presently, DO-254 is much more recent in its application than DO-178, and relatively less precise in the recommendations than DO-178 requiring more rigorous justifications, demonstrations and analysis. This fuzziness leaves flexibleness to audits which are, for the time being, less stringent than they would be for DO-178. This state of things leads to reduced DO-254 certification costs of approximately 20-30% for a DAL A development of a single software hosted on a single 6 DO-254-ruled component, when compared to equivalent software on a DO-178-ruled component. Emerging components such as Xilinx s Virtex 4 or Virtex 5 ( embed FPGA, DSP core and PowerPC core. These features are those commonly used in wireless communication systems, and may be the basis for Software Defined Radios (SDR) as discussed 6.2. There is little experience on use of such chips in avionics systems yet, but should their use spread in relevant system designs, it could be proposed with present status in DO-178C and DO-254 to certify these components as a single board hosting the split elements would, i.e. an FPGA, a DSP and a PowerPC: The DSP and the PowerPC cores would be certified through the software they run, i.e. through DO-178C 6 Reference is made here of single software on single FPGA (hence DO-254) to allow comparison with same software function on a processor (hence DO178). When multiple FPGA enter the design, the artifact to be submitted will be slightly more complex as the overall design will have to be submitted as well as each individual FPGA design. Edition Number: 1.1 Issue 1 Page: 21

32 The FPGA and interfaces through DO-254 Single mode of failure could be a safety concern, as if the chip fails, then the whole radio fails. This would also be the case with a split component design, i.e. it is likely that if either the DSP or the PowerPC fails, than the whole radio would in turn fail. Using these integrated chips will possibly simplify the design of the radio (hence lower the costs of certification related to design artefacts, but this has little impact on the overall costs). Whether they will be actually used in the future other than for reduced SWaP-C (size, weight and power, as well as cost) considerations (i.e. provided their cost is less than the sum of the costs of equivalent FPGA, DSP and PowerPC) will need further investigation. Indeed, reliability and maintainability of the equipment also need to be considered during the design phases D0254 and COTS FAA s Advisory Circular AC , 30-May-05, stipulates if you [manufacturers] choose to follow the guidance in RTCA/DO-254 for your level D devices, we do not need to review the life cycle data for that device. [ ] This AC recognizes the guidance in RTCA/DO-254 applies specifically to complex custom micro-coded components with hardware design assurance levels of A, B, and C, such as ASICs, PLDs, and FPGAs. Also, We [FAA] recognize that the hardware life cycle data for commercial-off-the-shelf (COTS) microprocessors may not be available to satisfy the objectives of RTCA/DO-254. Therefore, we don t intend that you apply RTCA/DO-254 to COTS microprocessors 7. There are alternative methods 8 or processes to ensure that COTS microprocessors perform their intended functions and meet airworthiness requirements. Coordinate your plans for alternative methods or processes with us early in the certification project. Components and/or computer boards manufacturers interpret the AC as massively deployed COTS products can be certified DO254 Level D by the fact that if these products would show defects, then they would have rapidly been revealed (hence corrected). Position of EASA 9 is presently similar; DO-254 may not apply to COTS for DAL D. Indeed, Certification Review Items (CRI) are being put in place between EASA and airframers (i.e. Airbus) for aircraft under development (e.g. A380 and A350), addressing applicability of DO-254 for DAL C and 7 [Note-RCF]: COTS microprocessors include processors (e.g. Intel s Pentium4 family, or Freescale s PowerPC family) and digital signal processors (DSP) (e.g. Texas Instruments TMS320 family) currently used in certified avionics designs. Other complex micro-coded components (e.g. Picochip s PC6530) are considered as ASICS. 8 [Note RCF]: COTS microprocessors are certified through a) justification (basically In Service Experience) and b) through DO-178 certification of the software they run. 9 An avionics manufacturer chooses to submit its application to FAA if the equipment is to fly in the US, or to EASA if the equipment is to fly in Europe. Fortunately, there are equivalences between the two organizations which facilitate the certification by one for equipment already certified by the other. Edition Number: 1.1 Issue 1 Page: 22