ISO Qualification of Compilers using SuperTest. Marcel Beemster ACE Associated Compiler Experts bv

Size: px

Start display at page:

Download "ISO Qualification of Compilers using SuperTest. Marcel Beemster ACE Associated Compiler Experts bv"

Rebecca Carson
7 years ago
Views:

1 ISO Qualification of Compilers using SuperTest Marcel Beemster ACE Associated Compiler Experts bv

2 Software Complexity in Automotive Model Based/ DSL C Assembly Use Cases

3 ISO is a Functional Safety Standard for Automotive Covers electrical and electronic systems in series production passenger cars Integral safety concept, like other functional safety standards (IEC 61508, railways, aviation, nuclear) One cannot just bolt safe components together, one has to consider the actual use case Interior light bulb versus indicator light bulb

4 Compiler Tools in ISO Functional safety: Absence of unreasonable risk (1.136) due to hazards (1.57) caused by malfunctioning behavior (1.73) of E/E systems (1.31) About Process, Design, System, Hardware, Software, Operation Part 8: Supporting processes Part 8, Clause 11: Confidence in the use of software tools This is where compilers fit As a developer of a car, you have to qualify the compilers used for safety critical systems

5 Quality Safety Example from our CoSy compiler development system: Quality Safety Compiler generates incorrect code for some programs Compiler does not work at all 30-days support obligation 3 days support obligation High risk No risk

6 About Compilers Compilers are complex pieces of software Easily grow to 1-2 million lines of code Development process stretches decades Overall architecture: Front-ends (e.g. C/C99/C++/Embedded-C) Optimizations (with many options) Code-generation Functionality very well defined by language specification Generated code is highly dependent on program source and option settings Compiler Use Cases Covered by Test-Suite

7 Types of Compiler Defects Compiler runtime errors, compiler crashes Compiler generation defects Compiler does not warn about incorrect source code (outside the language specification) Compiler does not warn about unintended source code, such as for example unreached code Compiler (silently) produces incorrect code Producing incorrect code is a safety critical defect in the compiler!

8 A Model of the Compiler Option Settings Use Cases Language Specification

9 ISO Compiler Qualification Variants Confidence of use But compilers are often updated Development process evaluation But the development process may span decades Development process according to standard Needs access to the development Validation Code review, but have to take care of 2 million lines of code Static analysis, at best proves absence of certain errors Testing for conformance against the language standard Language specification is stable for more than 20 years Recommended

10 SuperTest ü Over 3,000,000 C and C++ language conformance, correctness and quality checks ü 30+ years of experience ü Powerful and flexible test generator ü Advanced loop generators ü Depth generator for arithmetic testing ü Positive and negative testing ü ABI (calling conventions) testing ü Recently used in ISO qualification processes

11 SuperTest compiler test and validation suite SuperTest is your guide to qualification Over 50 man-years of experience in SuperTest, and many more man-years in building compiler technology Naturally it covers language constructs according to ISO C (C90, C99, C++, DSP/Embedded-C) standards Taxonomy (use-cases) by standard chapters, but also Test lists for different taxonomy and configuring known failures Configurable for target-defined behavior (type sizes, accuracy, rounding) with generated Depth Suite Tempest Template Expander Zoom in on problems (if specific use-cases covered by compiler are known to be fragile) Trade-off: Small test-case vs. size of context (for sake of and debugging and identifying issues)

12 Three Recommended Roads to Compiler Qualification 1. End-user performs qualification Gets unqualified compiler from vendor (or even public domain) Determines use-cases, acquires SuperTest, performs testing 2. Compiler vendor performs qualification Compiler vendor determines use-cases, gets SuperTest, performs testing Compiler vendor gets 3 rd party to verify process Compiler vendor creates Safety Manual, listing use-cases 3. Compiler vendor prepares qualification, end-user sets usecases Using ACEs SuperTest qualification suite

13 SuperTest Qualification Suite New product by ACE based on SuperTest Specifically designed to build ISO tool confidence Providing high compiler code coverage Test suite to be supplied with tool SDKs Tailored towards architecture and selected use-cases Currently being integrated and supplied by industrial partners (OEMs) for qualification in e.g. automotive sector

14 Concluding ISO requires developers of safety critical applications to qualify the compiler With SuperTest you have 3 methods to do this: All by yourself By the compiler vendor Most flexible Through a supplied qualification kit Most convenient Flexible and easy SuperTest includes 30 years of experience in compiler validation, offering high confidence of use

15 ACE Associated Compiler Experts Based in the Amsterdam, the Netherlands Founded in 1975 Proven track record RTOS kernels and language design World s first UNIX port for Motorola MC68000 Key consultant to the X/Open group Leading compilers for C, F77, Pascal, Modula2, HPF 30 Years experience in validation of compilers Home of the CoSy compiler development system Contact: marianne@ace.nl,

Introduction of ISO/DIS 26262 (ISO 26262) Parts of ISO 26262 ASIL Levels Part 6 : Product Development Software Level

ISO 26262 the Emerging Automotive Safety Standard Agenda Introduction of ISO/DIS 26262 (ISO 26262) Parts of ISO 26262 ASIL Levels Part 4 : Product Development System Level Part 6 : Product Development