ISO 26262 Qualification of Compilers using SuperTest Marcel Beemster ACE Associated Compiler Experts bv
Software Complexity in Automotive Model Based/ DSL C Assembly Use Cases
ISO 26262 is a Functional Safety Standard for Automotive Covers electrical and electronic systems in series production passenger cars Integral safety concept, like other functional safety standards (IEC 61508, railways, aviation, nuclear) One cannot just bolt safe components together, one has to consider the actual use case Interior light bulb versus indicator light bulb
Compiler Tools in ISO 26262 Functional safety: Absence of unreasonable risk (1.136) due to hazards (1.57) caused by malfunctioning behavior (1.73) of E/E systems (1.31) About Process, Design, System, Hardware, Software, Operation Part 8: Supporting processes Part 8, Clause 11: Confidence in the use of software tools This is where compilers fit As a developer of a car, you have to qualify the compilers used for safety critical systems
Quality Safety Example from our CoSy compiler development system: Quality Safety Compiler generates incorrect code for some programs Compiler does not work at all 30-days support obligation 3 days support obligation High risk No risk
About Compilers Compilers are complex pieces of software Easily grow to 1-2 million lines of code Development process stretches decades Overall architecture: Front-ends (e.g. C/C99/C++/Embedded-C) Optimizations (with many options) Code-generation Functionality very well defined by language specification Generated code is highly dependent on program source and option settings Compiler Use Cases Covered by Test-Suite
Types of Compiler Defects Compiler runtime errors, compiler crashes Compiler generation defects Compiler does not warn about incorrect source code (outside the language specification) Compiler does not warn about unintended source code, such as for example unreached code Compiler (silently) produces incorrect code Producing incorrect code is a safety critical defect in the compiler!
A Model of the Compiler Option Settings Use Cases Language Specification
ISO 26262 Compiler Qualification Variants Confidence of use But compilers are often updated Development process evaluation But the development process may span decades Development process according to standard Needs access to the development Validation Code review, but have to take care of 2 million lines of code Static analysis, at best proves absence of certain errors Testing for conformance against the language standard Language specification is stable for more than 20 years Recommended
SuperTest ü Over 3,000,000 C and C++ language conformance, correctness and quality checks ü 30+ years of experience ü Powerful and flexible test generator ü Advanced loop generators ü Depth generator for arithmetic testing ü Positive and negative testing ü ABI (calling conventions) testing ü Recently used in ISO 26262 qualification processes
SuperTest compiler test and validation suite SuperTest is your guide to qualification Over 50 man-years of experience in SuperTest, and many more man-years in building compiler technology Naturally it covers language constructs according to ISO C (C90, C99, C++, DSP/Embedded-C) standards Taxonomy (use-cases) by standard chapters, but also Test lists for different taxonomy and configuring known failures Configurable for target-defined behavior (type sizes, accuracy, rounding) with generated Depth Suite Tempest Template Expander Zoom in on problems (if specific use-cases covered by compiler are known to be fragile) Trade-off: Small test-case vs. size of context (for sake of and debugging and identifying issues)
Three Recommended Roads to Compiler Qualification 1. End-user performs qualification Gets unqualified compiler from vendor (or even public domain) Determines use-cases, acquires SuperTest, performs testing 2. Compiler vendor performs qualification Compiler vendor determines use-cases, gets SuperTest, performs testing Compiler vendor gets 3 rd party to verify process Compiler vendor creates Safety Manual, listing use-cases 3. Compiler vendor prepares qualification, end-user sets usecases Using ACEs SuperTest qualification suite
SuperTest Qualification Suite New product by ACE based on SuperTest Specifically designed to build ISO 26262 tool confidence Providing high compiler code coverage Test suite to be supplied with tool SDKs Tailored towards architecture and selected use-cases Currently being integrated and supplied by industrial partners (OEMs) for qualification in e.g. automotive sector
Concluding ISO 26262 requires developers of safety critical applications to qualify the compiler With SuperTest you have 3 methods to do this: All by yourself By the compiler vendor Most flexible Through a supplied qualification kit Most convenient Flexible and easy SuperTest includes 30 years of experience in compiler validation, offering high confidence of use
ACE Associated Compiler Experts Based in the Amsterdam, the Netherlands Founded in 1975 Proven track record RTOS kernels and language design World s first UNIX port for Motorola MC68000 Key consultant to the X/Open group Leading compilers for C, F77, Pascal, Modula2, HPF 30 Years experience in validation of compilers Home of the CoSy compiler development system Contact: marianne@ace.nl, www.ace.nl!