Transcription

1 Test Case Generation for Ultimately Periodic Paths Joint work with Saddek Bensalem Hongyang Qu Stavros Tripakis Lenore Zuck Accepted to HVC 2007

2 How to find the condition to execute a path? (weakest precondition wp(true)) Put true at end of path. Propagate path backwards. On assignment, relativize (backwards assignment). On yes edge of decision, add decision as conjunction. On no edge, add negation of decision as conjunction. A>1/\B=0 no yes X:=X/A yes A=2\/X>1 no X:=X+1 true

3 How to find the condition to execute a path? Put true at end of path. Propagate path backwards. On assignment, relativize (backwards assignment). On yes edge of decision, add decision as conjunction. On no edge, add negation of decision as conjunction. A>1/\B=0 no yes X:=X/A A 2/\X 1 yes A=2\/X>1 no X:=X+1 true

4 How to find the condition to execute a path? Put true at end of path. Propagate path backwards. On assignment, relativize (backwards assignment). On yes edge of decision, add decision as conjunction. On no edge, add negation of decision as conjunction. A>1/\B=0 no yes A 2/\X/A 1 X:=X/A A 2/\X 1 A=2\/X>1 yes no X:=X+1 true

5 How to find the condition to execute a path? Put true at end of path. Propagate path backwards. On assignment, relativize (backwards assignment). On yes edge of decision, add decision as conjunction. On no edge, add negation of decision as conjunction. A>1/\B=0 no yes A 2/\X 1 A=2\/X>1 no A 2/\X/A 1/\ A>1/\B=0 yes X:=X/A X:=X+1 A 2/\X/A 1 true

6 How to find the transformation of a graph tr(x)? Start with the set of variables that participate. a,b,x A>1/\B=0 no yes X:=X/A A=2\/X>1 no yes X:=X+1

7 How to find the transformation of a graph? Start with the set of variables that participate. Apply transformation syntactically forwards. a,b,x A>1/\B=0 no yes X:=X/A A=2\/X>1 no yes X:=X+1 a,b,(x/a)+1

8 How to find the transformation of a graph? Start with the set of variables that participate. Apply transformation syntactically forwards. a,b,x A>1/\B=0 no yes X:=X/A yes A=2\/X>1 no X:=X+1 a,b,x/a a,b,(x/a)+1

9 How to calculate a path condition for an ultimately periodic path? This is the subject of this work! Of course in general this is an undecidable problem.

10 Test case generation based on LTL specification LTL Aut Compiler Flow chart Model Checker Path Path condition calculation Transitions First order instantiator Test monitoring

11 Goals Verification of software. Compositional verification. Use only a unit of code instead of the whole code. Parameterized verification. Verifies a procedure with any value of parameters in one shot Generating test cases via path conditions: A truth assignment satisfying the path condition. Helps derive the demonstration of errors. Generating appropriate values to missing parameters.

12 Spec: at l 2 U (at l 2 /\ x y /\ ο( at l 2 /\( at l 2 U at l 2 /\ x 2 y ))) Automatic translation of LTL formula into an automaton [Gerth et all] LTL is interpreted over finite sequences. Can use other (linear) specification. Property specifies the path we want to find (SPIN: never claim), not the property that must hold for all paths (for this, take the negation). at l 2 Observation: each node has conjunctions of predicates onprogram variables and program at l 2 /\ counters x y at l 2 at l 2 /\ x 2 y

13 Divide and Conquer Intersect property automaton with the flow chart, regardless of the statements and program variables expressions. Add assertions from the property automaton to further restrict the path condition. Calculate path conditions for sequences found in the intersection. Calculate path conditions on-the-fly. Backtrack when condition is false. Thus, advantage to forward calculation of path conditions (incrementally).

14 Spec: (only program counters here) at l 2 U (at l 2 /\ο at l 2 /\( at l 2 U at l 2 )) at l 2 l 2 :x:=x+z l 2 :x:=x+z at l 2 X at l 2 l 3 :x<t = at l 2 l 3 :x<t Either all executions of a path satisfy the formula or none. at l 2 at l 2 l 1 : at l 2 l 2 :x:=x+z Sifts away paths not satisfying formula. Then calculate path condition.

15 Spec: at l 2 U (at l 2 /\ x y /\ at l 2 ο( at l 2 /\( at l 2 U at l 2 /\ x 2 y ))) x y X l 2 :x:=x+z l 2 :x:=x+z at l 2 /\ x y l 3 :x<t = l 3 :x<t Only some executions of path may satisfy formula at l 2 at l 2 /\ x 2 y l 1 : l 2 :x:=x+z x 2 y Modify calculation of path condition to incorporate property

16 Calculating the intersection of the property automaton and flow graph (abstract variables away). a a s 1 s 2 a q 1 <> a a Acceptance is determined by property automaton. a s 3 q 2 s 1,q 1 a s 2,q 1 s 3,q 2 a a

17 How to generate test cases Take the intersection of an LTL automaton (for a never claim) with the flow graph. Some paths would be eliminated for not satisfying the assertions on the program counters. Seeing same flow chart node does not mean a loop: program variables may value. Use iterative deepening. For each initial path calculate the path condition. Backtrack if condition simplifies to false. Report path condition based on flow graph path+ltl assertions. Always simplify conditions!

18 How the LTL formula directs the search Consider (x=4)u (x=5/\o ) x=4 x=5 false x<5 true y:=7 x:=x+1

19 How the LTL formula directs the search Consider x=4u (x=5/\o ) x=4 x=5 false x<5 true y:=7 x:=x+1

20 How the LTL formula directs the search Consider x=4u (x=5/\o ) x=4 X=4 x=5 false x<5 true y:=7 x:=x+1

21 How the LTL formula directs the search Consider x=4u (x=5/\o ) x=4 X=4 x=5 false x<5 true X=4 y:=7 x:=x+1

22 How the LTL formula directs the search Consider x=4u (x=5/\o ) x=4 X=4 x=5 y:=7 false x<5 true X=4 x:=x+1 This is a contradiction X=4 x<5 true

23 How the LTL formula directs the search Consider x=4u (x=5/\o ) x=4 X=5 x=5 false x<5 true X=4 y:=7 x:=x+1

24 How the LTL formula directs the search Consider x=4u (x=5/\o ) x=4 X=5 x=5 false x<5 true X=4 y:=7 x:=x+1

25 Why use Temporal specification Temporal specification for sequential software? Deadlock? Liveness? No! Captures the tester s intuition about the location of an error: I think a problem may occur when the program runs through the main while loop twice, then the if condition holds, while t>17.

26 Example: GCD a>0/\b>0/\at l 0 /\ at l 7 l 0 l 1 :x:=a l 2 :y:=b l 3 :z:=x rem y at l 0 /\ a>0/\ b>0 l 4 :y:=z l 5 :x:=y at l 7 no l 6 :z=0? yes l 7

27 Example: GCD l 0 l 1 :x:=a l 2 :y:=b a>0/\b>0/\at l 0 /\ at l 7 l 3 :z:=x rem y Path 1: l 0 l 1 l 2 l 3 l 4 l 5 l 6 l 7 a>0/\b>0/\a rem b=0 l 4 :y:=z Path 2: l 0 l 1 l 2 l 3 l 4 l 5 l 6 l 3 l 4 l 5 l 6 l 7 a>0/\b>0/\a rem b 0 no l 5 :x:=y l 6 :z=0? yes l 7

28 Potential explosion Bad point: potential explosion Good point: may be chopped on-the-fly

29 Again: How to deal with the ultimately periodic case?

30 The Equality Method We are looking for the condition to execute a loop indefinitely, after a finite prefix, where in each iteration, the variables obtain the same values. Executing the periodic part ρ once when wp ρ /\X=tr ρ (X). Executing it after the prefix σ is when wp σρ /\wp σ (wp ρ /\X=tr ρ (X)). Simplifying: wp σρ /\wp σ (X=tr ρ (X)).

31 Example σ: z:=z-1; while ρ: x>0 do begin y:=x; x:=(x*2+z+y)/3 End wp σρ =x>0. tr ρ (x,y,z)=( (x*2+z+x)/3,x,z) x=(x*2+z+x)/3/\y=x, or equivalently, z=0/\y=x. wp σ (z=0/\y=x) is z=1/\y=x. Overall: x>0/\z=1/\y=x.

32 The Monotonicity Method It is sufficient to find a loop invariant such that I wp ρ (I) The weakest such invariant I is I= wp ρ (true). Proof: I true for each I. By monotonicity of wp, wp ρ (I) wp ρ (true). Since I wp ρ (I), it holds that I wp ρ (true), independently of I.

33 Deriving an ultimately periodic condition We set I= wp ρ (true) in the implication I wp ρ (I), obtaining wp ρ (true) wp ρ (wp ρ (true)). This can be rewritten as wp ρ (true) wp ρ (true)[tr ρ (X)/X]. Applying the wp of the prefix, we obtain wp σ (wp ρ (true)) wp σ (wp ρ (true)[tr ρ (X)/X]). The next slide will deal with the 2 nd bullet (and then we need to remember to apply the 3 rd ).

34 The case where wp ρ (true) is e 0 (or e>0). Set e =e[tr ρ (X)/X]. Bullet 2 from previous slide becomes e 0 e 0. A sufficient condition is e e. Other cases: when we have a condition wp ρ (true)=g f, we take e=g-f. In case wp ρ (true)=g 0/\f 0 we have condition g g /\f f. In case wp ρ (true)=g 0\/f 0 it is sufficient that we strengthen to either g g or f f. An equality can be transformed into two inequalities and the disjunction case is applied.

35 Some Mixed and not completely Ultimately Periodic paths While x>1 do begin if PowerTwo(x-1) then x:=4*(x-1) else x:=x-1 end. Example:

36 Conclusions An approach for generating test cases automatically. Also: verification of infinite state systems. Path by path verification rather than state by state. Challenge: the weakest precondition for ultimately periodic sequences in infinite state systems. We suggested several methods (e.g., the equality and monotonicity methods, etc.) Not all of the infinite executions are ultimately periodic.