Regression Verification: Status Report Presentation by Dennis Felsing within the Projektgruppe Formale Methoden der Softwareentwicklung 2013-12-11 1/22
Introduction How to prevent regressions in software development? 2/22
Introduction Formal Verification Formally prove correctness of software Requires formal specification Regression Testing Discover new bugs by testing for them Requires test cases 3/22
Introduction Formal Verification Formally prove correctness of software Requires formal specification Regression Testing Discover new bugs by testing for them Requires test cases Regression Verification Formally prove there are no new bugs 3/22
Project Objectives 1 Develop a tool for Regression Verification for recursive programs in a simple imperative programming language 2 Case study to evaluate how well our approaches work for different examples in comparison to other systems 3 Extend the tool to work with more programs and to be more general 4/22
Preliminary Considerations I Unbounded Integers vs Bit Vectors Unbounded Integers don t overflow Bit Vectors can be limited to simplify the problem Solution: Support both: Proofs are supposed to be over unbounded Integers For comparison Bit Vectors can also be used 5/22
Preliminary Considerations II Division by 0 In Z3, division by zero is allowed, but the result is not specified. Division is not a partial function. Actually, in Z3 all functions are total, although the result may be underspecified in some cases like division by zero. Possible Solutions: Check that there are no divisions by 0 It could be verified that the result is independent of the result of division by 0 6/22
Preliminary Considerations III Array Access over Boundaries Arrays have infinite size in Z3 Possibility: Check array boundaries on every access Programs can be proven to honor array boundaries Solution: Assume programs have been proven to honor array boundaries 7/22
Tool for Regression Verification Overview Function f (val n; ret r) Function f without recursions Equivalent? Uninterpreted Functions Function g (val x; ret y) Function g without recursions Static Single Assignment S f Static Single Assignment S g (n = x S f S g ) r = y SMT Solver Valid / Invalid 8/22
Tool for Regression Verification Formally prove there are no new bugs Goal: Proving the equivalence of two closely related programs No formal specification or test cases required Instead use old program version Make use of similarity between programs 9/22
Formally prove there are no new bugs Tool for Regression Verification Goal: Proving the equivalence of two closely related programs No formal specification or test cases required Instead use old program version Make use of similarity between programs int gcd1( int a, int b) { int g = 0; if (b == 0) { g = a; } else { a = a % b; g = gcd1(b, a); } return g; } int gcd2( int x, int y) { int z = x; if (y > 0) { z = gcd2(y, z % y); } return z; } 9/22
Uninterpreted Functions Overview Function f (val n; ret r) Function f without recursions Equivalent? Uninterpreted Functions Function g (val x; ret y) Function g without recursions Static Single Assignment S f Static Single Assignment S g (n = x S f S g ) r = y SMT Solver Valid / Invalid 10/22
Uninterpreted Functions Given the same inputs an Uninterpreted Function always returns the same outputs. Motivation: Proof by Induction, to prove f(n) = g(n) assume f(n 1) = g(n 1) int gcd1( int a, int b) { int g = 0; if (b == 0) { g = a; } else { a = a % b; g = U (b, a); } return g; } int gcd2( int x, int y) { int z = x; if (y > 0) { z = U (y, z % y); } return z; } 11/22
Conversion of Programs to Formulae Overview Function f (val n; ret r) Function f without recursions Equivalent? Uninterpreted Functions Function g (val x; ret y) Function g without recursions Static Single Assignment S f Static Single Assignment S g (n = x S f S g ) r = y Valid / Invalid SMT Solver 12/22
General idea Conversion of Programs to Formulae I Walk Abstract Syntax Tree of both programs Convert every SimPL construct to SMT formula: int x = y; declare fun x 0 () Int assert (x 0 = y i ) if (y) { x = b; } else { x = c; }. assert ( x i = b) assert (x ( i+1) = c) assert (x ( i+2) = ( ite y x i x ( i +1))) ; Phi node Use new variable for every assignment 13/22
Conversion of Programs to Formulae II Regression Verification Uninterpreted Functions: assert ( forall ((u Int ) (v Int ) ((gcd1 u v) = (gcd2 u v)))) Proof f = g: assert (not ( gcd1 result = gcd2 result )) check sat get model exit Objective Regression Verification proofs : Done 14/22
Case Study Done Collect examples: Papers, Refactoring Rules,... 51 program pairs so far Planned Framework for testing them Check how well extensions work More (interesting) examples Objective Case Study : Work in Progress 15/22
Convert Loops to Recursions Idea Convert every loop to a new recursive function Handling multiple loop variables: Return a tuple while (x < 10) { y = y + x; x = x 1; } Initial work (x,y) = loop(x,y); Added tuples to SimPL grammar and AST. tuple loop(int x, int y) { if (x < 10) { y = y + x; x = x 1; (x,y) = loop(x,y); } return (x,y); } 16/22
Idea Specify how often a function call is inlined: y = f (x) inline 3; Same for loops (converted to functions): while (x < y) inline 5 { z; } Possibility later: Inlining strategies Function Inlining Initial work Modified grammar to support inlining 17/22
Abstraction Refinement I Recursive Functions are the main problem Two ways of dealing with them: Most general abstraction Classical Regression Verification approach Uninterpreted functions x : f(x) = g(x) No further information about the functions Only works when the function bodies are equivalent 18/22
No abstraction Give recursive definition: Abstraction Refinement II forall x. f (n) = let r0 = 0 r1 = n r2 = f (n 1) r3 = n + r2 r4 = ite (n <= 1, r1, r3) in r4 Experiments for a few simple functions Only works when the function bodies differ for finite number of inputs 19/22
Abstraction Refinement III Problem: Find an abstraction inbetween CEGAR Loop Counter Example Guided Abstraction Refinement Start with simple over-approximation Extract patterns from counter examples Refine Abstraction Repeat if proof still fails 20/22
Abstraction Refinement IV Problem: Find an abstraction inbetween Horn Clauses (p q t) u Postcondition PC is true after recursive call r = f(n) PC(n,r) Solver figures out Postcondition on its own (e.g. using CEGAR) 21/22
Summary Regression Verification Prove that two similar programs are equivalent Better chance of being adopted than Formal Verification More powerful than Regression Testing Project Status 1 Develop Regression Verification tool: Basic tool: Done Loops to Recursions: WIP Function Inlining: WIP 2 Case study to compare approaches: WIP 3 Extend tool: Planning and Experimentation 22/22