Debugging and Bug Tracking. Slides provided by Prof. Andreas Zeller, Universität des Saarlands

Transcription

1 Debugging and Bug Tracking Slides provided by Prof. Andreas Zeller, Universität des Saarlands

2 Learning goals 1. part: Debugging Terminology o Defect o Infection o Failure Debugging with the Scientific Method TRAFFIC Principle 2. part: Tracking Problems Classifying problems Severity and Priority Problem Lifecycle MSE SEA Prof. Peter Sommerlad 2

3 How Failures Come to be Andreas Zeller

4 An F-16 (northern hemisphere) 4

5 An F-16 (southern hemisphere) 5

6 F-16 Landing Gear 6

7 The First Bug September 9,

8 More Bugs 8

9 Facts on Debugging Software bugs are costing ~60 bln US$/yr Improvements could reduce cost by 30% Validation (including debugging) can easily take up to 50-75% of the development time When debugging, some people are three times as efficient than others 9

10 A Sample Program $ sample Output: $ sample Output:

11 How to Debug (Sommerville 2004) Locate error Design error repair Repair error Re-test program 11

12 The Traffic Principle T R A F F I C rack the problem eproduce utomate ind Origins ocus solate orrect 12

13 The Traffic Principle T R A F F I C rack the problem eproduce utomate ind Origins ocus solate orrect 13

14 From Defect to Failure 1. The programmer creates a defect an error in the code. 2. When executed, the defect creates an infection an error in the state. Variables 3. The infection propagates. 4. The infection causes a failure. This infection chain must be traced back and broken. t 14

15 The Curse of Testing Not every defect causes a failure! Testing can only show the presence of errors not their absence. (Dijkstra 1972) Variables 15

16 Debugging Every failure can be traced back to some infection, and every infection is caused by some defect. Variables Debugging means to relate a given failure to the defect and to remove the defect. 16

17 The Scientific Method The scientific method is a general pattern of how to find a theory that explains (and predicts) some aspect of the universe Called scientific method because it s supposed to summarize the way that (experimental) scientists work 17

18 The Scientific Method 1. Observe some aspect of the universe. 2. Invent a hypothesis that is consistent with the observation. 3. Use the hypothesis to make predictions. 4. Tests the predictions by experiments or observations and modify the hypothesis. 5. Repeat 3 and 4 to refine the hypothesis. 18

19 A Theory When the hypothesis explains all experiments and observations, the hypothesis becomes a theory. A theory is a hypothesis that explains earlier observations predicts further observations In our context, a theory is called a diagnosis (Contrast to popular usage, where a theory is a vague guess) 19

20 Mastermind A Mastermind game is a typical example of applying the scientific method. Create hypotheses until the theory predicts the secret. 20

21 Scientific Method of Debugging Problem Report Hypothesis is supported: Code refine hypothesis Hypothesis Prediction Experiment Observation + Conclusion Run Hypothesis is rejected: create new hypothesis More Runs Diagnosis 21

22 Search in Space + Time variables " time?! 22

23 The Defect variables "! time! 23

24 A Program State 24

25 A Sample Program $ sample Output: $ sample Output:

26 Initial Hypothesis Hypothesis Prediction Experiment Observation Conclusion sample works. Output is Run sample as above. Output is 0 11 Hypothesis is rejected. 26

27 int main(int argc, char *argv[]) { int *a; int i; a = (int *)malloc((argc - 1) * sizeof(int)); for (i = 0; i < argc - 1; i++) a[i] = atoi(argv[i + 1]); shell_sort(a, argc); printf("output: "); for (i = 0; i < argc - 1; i++) printf("%d ", a[i]); printf("\n"); free(a); } return 0; 27

28 Find Origins variables! The 0 printed is the value of a[0]. Where does it come from? time!!!! Basic idea: Track or deduce value origins Separates relevant from irrelevant values! We can trace back a[0] to shell_sort 28

29 Hypothesis 1: a[] Hypothesis Prediction Experiment Observation Conclusion The execution causes a[0] = 0 At Line 37, a[0] = 0 should hold. Observe a[0] at Line 37. a[0] = 0 holds as predicted. Hypothesis is confirmed. 29

30 static void shell_sort(int a[], int size) { int i, j; int h = 1; do { h = h * 3 + 1; } while (h <= size); do { h /= 3; for (i = h; i < size; i++) { int v = a[i]; for (j = i; j >= h && a[j - h] > v; j -= h) a[j] = a[j - h]; if (i!= j) a[j] = v; } } while (h!= 1); } 30

31 Hypothesis 2:shell_sort() Hypothesis Prediction Experiment Observation Conclusion The infection does not take place until shell_sort. At Line 6, a[] = [11, 14]; size = 2 Observe a[] and size at Line 6. a[] = [11, 14, 0]; size = 3. Hypothesis is rejected. 31

32 Search in Time variables In shell_sort, the state must have time "!! become infected. Basic idea: Observe a transition from sane to infected. 32

33 Observing a Run variables argc argv [0] argv [1] a [0] a a [1] [2] i size h a = malloc(...) 3 "11""14"???? 0 i = 0 a[i] = atoi(argv[i + 1]) time i++ a[i] = atoi(argv[i + 1]) i++ shell_sort(a, argc) 3? 3 "11""14" 0 11? 2 return 0 33

34 Hypothesis 3: size Hypothesis Prediction Experiment Observation Conclusion size = 3 causes the failure. Changing size to 2 should make the output correct. Set size = 2 using a debugger. As predicted. Hypothesis is confirmed. 34

35 Specific Observation static void shell_sort(int a[], int size) { fprintf(stderr, At shell_sort ); for (i = 0; i < size; i++) fprintf(stderr, a[%d] = %d\n, i, a[i]); fprintf(stderr, size = %d\n, size); int i, j; $ sample int h = 1; a[0] = a[1] = 14 } a[2] = 0 size = 3 Output: 0 11 The state is infected at the call of shell_sort! 35

36 Fixing the Program int main(int argc, char *argv[]) { int *a; int i; a = (int *)malloc((argc - 1) * sizeof(int)); for (i = 0; i < argc - 1; i++) a[i] = atoi(argv[i + 1]); } shell_sort(a, argc); - 1);... $ sample Output:

37 Hypothesis 4: argc Hypothesis Prediction Experiment Observation Conclusion Invocation of shell_sort with size = argc causes the failure. Changing argc to argc - 1 should make the run successful. Change argc to argc - 1 and recompile. As predicted. Hypothesis is confirmed. 37

38 The Diagnosis Cause is Invoking shell_sort() with argc Proven by two experiments: Invoked with argc, the failure occurs; Invoked with argc - 1, it does not. Side-effect: we have a fix (Note that we don t have correctness but take my word) 38

39 Finding Causes Infected state Sane state The difference causes the failure 39

40 Search in Space Infected state Sane state argc = 3 Test? Mixed state 40

41 Search in Time Failing run Passing run argc = 3 argc = 3 Transition from argc to a[2] a[2] = 0 t 41

42 int main(int argc, char *argv[]) { int *a; // Input array a = (int *)malloc((argc - 1) * sizeof(int)); for (int i = 0; i < argc - 1; i++) a[i] = atoi(argv[i + 1]); // Sort array shell_sort(a, argc); Should be argc - 1 // Output array printf("output: "); for (int i = 0; i < argc - 1; i++) printf("%d ", a[i]); printf("\n"); } free(a); return 0; 42

43 Concepts A failure comes to be in three stages: 1. The programmer creates a defect 2. The defect causes an infection 3. The infection causes a failure -- an externally visible error. Not every defect results in an infection, and not every infection results in a failure. 43

44 Concepts (2) To debug a program, proceed in 7 steps: T R A F F I C rack the problem eproduce utomate ind Origins ocus solate orrect 44

45 Sources of Hypotheses Experimentation n controlled runs Induction n runs Observation 1 run Deduction 0 runs 45

46 Concepts of Debugging A cause of any event ( effect ) is a preceding event without which the effect would not have occurred. To isolate a failure cause, use the scientific method. Make the problem and its solution explicit. 46