Software Module Test for an Electronic Steering Lock

Transcription

1 Software Module Test for an Electronic Steering Lock Wolfgang Beer, Dr. Peter Jüttner, Daniel Simonis (external subcontractor), Siemens VDO Automotive AG Siemensstr Regensburg, Germany Tel.: +49(0) Siemens VDO Automotive AG 2005 Supplying value

2 in ELV Project Feb-05 W. Beer, P. Jüttner, D. Simonis 2

3 (1) ELV = Elektronische Lenkradverriegelung Basic functions: Lock, unlock (with cryptology) Additional functions: Diagnosis, Learning, Operation Modes, Communication, Memory for Data, Errors, and Measured Values. 8Bit Micro, 24k ROM Embedded SW development with cross compiler Object-oriented approach for design in C with mapping rules OO -> C Layer model in SW (OS, Application) Development mainly from scratch Feb-05 W. Beer, P. Jüttner, D. Simonis 3

4 (2) Team 6 Team members 2 dedicated SW testers, one mainly for SW Module test, one for SW Integration and SW Validation Project start 2003 Project end 2005 Several car lines with slightly different features Feb-05 W. Beer, P. Jüttner, D. Simonis 4

5 (3) Term. 30z Ignition Switch separate separate Supply Supply Serial Serial Interface Interface NEC NEC 78K0 78K0 Q1 K1 "Lock" Sensor for Lock Q3 IN µc Oscillator Oscillator "Unlock" M Term. 30 Term. 31 Powersupply Powersupply K2 IN µc Sensor for Unlock Feb-05 W. Beer, P. Jüttner, D. Simonis 5

6 (4) Vehicle Integration ELV Master Bidirectional communication line ELV CAN Engine Control Feb-05 W. Beer, P. Jüttner, D. Simonis 6

7 (1) SW Requirements test against SW Validation SW Design test against SW Integration Test test against Coding Feb-05 W. Beer, P. Jüttner, D. Simonis 7

8 (2) Test of single SW modules in isolation Test of Classes (implemented in C), i.e. test of C functions Black-Box and White Box test strategies Test against SW design (Black-Box) Test against structure (White-Box) Test completeness criteria 100% coverage of module requirements and 100% statement coverage Black-Box and White Box tests in combination Introduced in SV C BC SW development process by CMMI improvement project Feb-05 W. Beer, P. Jüttner, D. Simonis 8

9 (3) Benefits Early error detection (errors become cheaper) Less bugs in following development steps Improvement of reuse Regression test from PC to target Feb-05 W. Beer, P. Jüttner, D. Simonis 9

10 (4) Drawbacks and prejudices in Embedded SW development SW modules are not always executable in target or PC environment Test harnesses are needed Runtime behavior cannot be tested on PC Test with non target compiler on PC Bugs in target compiler and on HW are not found Behavior of SW with different compilers is different Bugs are found anyway in later test phases Feb-05 W. Beer, P. Jüttner, D. Simonis 10

11 (5) BUT Testing SW is the goal, not testing compiler or HW There is no guarantee that later tests find the bugs Later bug fix is more expensive No code coverage in later test phases -> risk of untested SW Feb-05 W. Beer, P. Jüttner, D. Simonis 11

12 (1) Test tool Rational Test Realtime (RTR) supports Automatic test harness generation Automatic pass/fail decision Structured tests Test case generation Comprehensive Test protocol and report Black-Box and White Box Test in parallel Regression tests Feb-05 W. Beer, P. Jüttner, D. Simonis 12

13 (2) Source Code Test Harness Generator PTU File PTU File Manual Rework Test Harness "Compiler" C Code C Compiler & Linker executable Test Harness Feb-05 W. Beer, P. Jüttner, D. Simonis 13

14 (3) Service Name Construct_Add_Evaluate_Clear Service Type NOT INFORMED Status Passed Tests Passed 6 Tests Failed Tests 6 0 Variable Status Init Value Expected Value Obtained Value k list next Passed Passed 0? 0 &list H list prev Passed? &list H node[0].prev Passed NIL NIL 00H Root Functions Functions and exits Statement blocks 100.0% (12/12) 100.0% (25/25) 100.0% (14/14) Decisions Loops 100.0% (15/15) Feb-05 W. Beer, P. Jüttner, D. Simonis 14 none

15 (4) Feb-05 W. Beer, P. Jüttner, D. Simonis 15

16 Lessons Learned (1) Introduction in Test Process and Test Tool took 2 weeks Total Effort for Module Test ~ 6 MM (~ = coding effort) test cases for a single C function test cases in total High effort in stub development Many bugs found, most of coding bugs Bugs in target compiler found (by using a PC compiler) Split in responsibility (1 SW designer/coder, 1 SW module tester) reduced timing problems Implicit design review Coding and testing mainly in parallel saves time Feb-05 W. Beer, P. Jüttner, D. Simonis 16

17 Lessons Learned (2) SW should be designed in ANSI C HW abstraction is useful Proper SW architecture and detailed design required (e.g. encapsulation, lean interfaces, no global variables) Simple control structures in algorithms reduce effort Test automation is essential 100% code coverage is necessary but not sufficient Each test phase has its focus (no "competition" between test phases) Integration test could also be done partly on PC with RTR based on existing stubs Feb-05 W. Beer, P. Jüttner, D. Simonis 17

18 By SW module test most of the bugs and inconsistencies in design and implementation of the ELV SW could be found and fixed in an early development phase The effort of module testing is acceptable if automated tests using appropriate test tools (RTR) are used Successful bug fix can be checked automatically by regression of automated tests The separation between designer/coder and tester reduces bottlenecks in the development phase especially in tight schedules Feb-05 W. Beer, P. Jüttner, D. Simonis 18