An introduction to static analysis

Transcription

1 1 An introduction to static analysis Eric Goubault Modelisation and Analysis of Systems in Interaction Laboratory 8th of april 2008

2 Roadmap Validation, verication, compilation? A short demo with INTERPROC/APRON to see what we want to get! Formalisation for simple safety properties: collecting semantics resolution of the semantic equations (a bit of order theory) Abstractions for practical calculations an example: sign analysis simple formalisation by Galois connections To go further interval analysis relational analyses... short demo with INTERPROC/APRON 2

3 Compilation and verication 3 [egoubault@is Cours07]> more f.c int f(int i) { int j,k; k = i+j; j = 2; return k; } [egoubault@is Cours07]> gcc -c f.c -O -Wall f.c: In function `f': f.c:3: warning: `j' might be used uninitialized in this function

4 But... 4 [egoubault@is Cours07]> more g.c int g() { int j[10]; return j[12]; } [egoubault@is Cours07]> gcc -c g.c -O -Wall [egoubault@is Cours07]> The verications done by gcc are very basic... complementary tools for validation!

5 Extensions of these methods for guaranteed correct software: formal methods 5 Formal specications (B, Z etc.) Proof (essentially, partial correctness proof): ex. Hoare Logics (interaction with the user) Model-Checking: Verication of rened temporal properties on parallel programs Static Analysis: fully automatic method to nd bugs or synthetise proof terms

6 Example of validation 6 fft(complex_array_ref a,int n) { complex_array_ref b[n/2], c[n/2]; if (n > 2) { for (i=0 ; i < n ; i=i+2) { b[i/2] = a[i]; c[i/2] = a[i+1]; } fft(b,n/2); fft(c,n/2); for (i=0 ; i < n ; i=i+1) a[i] = F1(n)*b[i] + F2(n)*c[i]; } else { a[0] = g*a[0] + d*a[1]; a[1] = a[0] - 2*d*a[1]; } }

7 Validate? 7 Mathematical proof, not test; but of what? No bug at execution time (division by zero, out of bounds array dereference, null dereference, overows etc.) Parseval? a [i] 2 = i i a[i] 2 Equality very hard to achieve... because of oating-point numbers ( real numbers). Precision drift? Functional proof, in general not achievable...

8 Expected results 8 Find in an automatic and guaranteed manner: (1) Out of bounds array dereference (in the else statement) only for arrays whose size is not a power of 2! [should crash, or give an incoherent result; Astrée, Polyspace, typically] (2) Under conditions on the input signal, no overow (Astrée, Polyspace, typically; also FLUCTUAT) (3) Under conditions, to give bounds for the dierence between a real number calculation and the oating-point computation... (see our analyser FLUCTUAT) characterisation of reachable values for all variables (invariants) at each control point. [there are other general properties than these safety properties: liveness properties]

9 In order to do this (1)... 9 dim a=n,n=2^k,k>0 fft(a,n) { cplx b[n/2], c[n/2]; if (n > 2) { for (i=0;i<n;i=i+2) { b[i/2]=a[i]; i+1=2j+1<=n and n=2^k c[i/2]=a[i+1]; } implies i+1<n fft(b,n/2); fft(c,n/2); for (i=0;i<n;i=i+1) a[i]=f1(n)*b[i]+f2(n)*c[i];} else { a[0]=g*a[0]+d*a[1]; n=2 (n=2^j, 0<=j<=k) a[1]=a[0]-2*d*a[1]; } }

10 A short demo of INTERPROC/APRON 10...

11 Collecting semantics - informally We construct the control ow graph of a program, as being the transition system (S, i, E, Tran): S is the set of control points of the program (one by syntactic line for us - indicated in the text by [i]) i is the initial control point E = {x = expr Aexp} Bexp We dene a certain number of rules for the transitions 11

12 Example void main() { // [0] int x=[-100,50]; // [1] while [2] (x<100) { [3] x=x+1; // [4] } // [5] } Executions of the program correspond to dipaths on the graph (orbits of this discrete dynamical system), on which the environmnent (σ : Loc Z) is modied 12

13 Collecting semantics - informally 13 The environments now collect the values, for all paths, at each control point An environment is an element of (Loc Z) At each state s S, we want to compute such an environment σ

14 Collecting semantics - intuitively 14 We start from the control ow graph (expressed as a transition system T = (S, i, E, Tran)) We associate to each control point s i of T an equation in associated variables S i, which represent the sets of values that program variables can take, at control points s i : S i = [[t]]s j s j S (s j,t,s i ) Tran where [[t]] is the interpretation of the transition t seen as a function from the set of values of variables to itself The solutions S i to these equations are called the invariants at control points s i (it is a property which is always true for all possible executions)

15 Example 15 void main() { // [0] int x=[-100,50]; // [1] while [2] (x<100) { // [3] x=x+1; // [4] } // [5] } x 0 = x 1 = [ 100, 50] x 2 = x 1 x 4 x 3 = ], 99] x 2 x 4 = x 3 + [1, 1] x 5 = [100, + [ x 2

16 Partial orders, lattices, completeness etc. 16 How can we give a meaning to xed point equations? The fundamental notion is the one of successive approximations, ordered by a partial order

17 Partial order 17 A partial order (P, ) is composed of: a set P and a binary relation P P such that is reexive: p P, p p is transitive: p, q, r P, p q&q r is anti-symetric: p, q P, p q&q r = p r = p = q Ex.: ( (S), ) Notice: isomorphic to the boolean lattice of rst-order propositional logics (with atoms being predicates on one integer value). is =!

18 Upper bounds etc. 18 For X P: p is an upper bound of X if q X, q p p is a (the!) least upper bound (lub, sup etc.) if: p is an upper bound of X for all upper bounds q of X, p q the least upper bound, if it exists, is denoted X similarly for the notion of lower bound, the greatest lower bound (glb, inf etc.), is denoted X A lattice is a partial order admitting a lub and a glb for all X containing two elements (hence for any non empty nite set X ) Ex.: in ( (S), ), all X admit a lub (the classical set-theoretic union) and a glb (the classical set-theoretic intersection). It is a particular kind of lattice.

19 Chains, completeness A chain of P is p 0 p 1... p n An ω-chain of P if p 0 p 1... p n... A partial order is a cpo (Complete Partial Order) if for all ω-chains P, P admits a lub in the partial order In general, we suppose that a cpo also has a minimal element, denoted, i.e. p P, p A lattice is a complete lattice if all subsets admit a lub (and hence a glb). Ex.: ( (S), ) is a complete lattice ( =, = S). 19

20 Functionals 20 A function f : D E between two cpos D et E is increasing i d, d D, d d f (d) f (d ) f increasing is continuous i for all chains d 0 d 1... d n... of D, we have: ( ) f (d n ) = f d n n N n N

21 Fixed points - resolution of equations 21 Let f : D D increasing on a partial order D. A xed point of f is an element d of D such that f (d) = d. A post-xed point of f is an element d of D such that f (d) d A pre-xed point of f is an element d of D such that d f (d)

22 Resolution of equations 22 (1) Let f : D D be an increasing function on a complete lattice D. Then f admits at least a xed point. Furthermore, the set of xed points of f is a complete lattice, thus there is a least xed point (noted lfp(f )) and a greatest xed point (noted gfp(f )). (2) Let f : D D be a continuous function on a cpo D (with a element). Then, x(f ) = f n ( ) n N is the smallest xed point of f, lfp(f ) [Kleene iteration].

23 Resolution of the semantic equations for our example 23 We compute the Kleene (x) iteration for the functional: F x 0 x 1 x 2 x 3 x 4 x 5 = [ 100, 50] x 1 x 4 ], 99] x 2 x 3 + [1, 1] [100, + [ x 2 where we wrote x i as denoting the set of possible values at control point i.

24 Resolution of semantic equations 24 We begin with x i = (1 i 5) We compute the Kleene iteration... slightly ameliorated (chaotic - similar to Gauss-Seidl versus Jacobi)

25 Iteration 1 25 x 0 = x 1 = [ 100, 50] x 2 = x 1 x 4 x 3 = ], 99] x 2 x 4 = x 3 + [1, 1] x 5 = [100, + [ x 2 x0 1 = x1 1 = [ 100, 50] x2 1 = [ 100, 50] x3 1 = ], 99] [ 100, 50] = [ 100, 50] x4 1 = [ 100, 50] + [1, 1] = [ 99, 51] x5 1 = [100, + [ [ 100, 50] =

26 Iteration i + 1 (i < 50) [Kleene] 26 x 0 = x 1 = [ 100, 50] x 2 = x 1 x 4 x 3 = ], 99] x 2 x 4 = x 3 + [1, 1] x 5 = [100, + [ x 2 x i+1 0 = x i+1 1 = [ 100, 50] x i+1 2 = [ 100, 50 + i] x i+1 3 = ], 99] [ 100, 50 + i] = [ 100, 50 + i] x i+1 4 = [ i, 50 + i] + [1, 1] = [ 99, 50 + i + 1] x i+1 5 = [100, + [ [ 99, 50 + i + 1] =

27 Iteration x 0 = x 1 = [ 100, 50] x 2 = x 1 x 4 x 3 = ], 99] x 2 x 4 = x 3 + [1, 1] x 5 = [100, + [ x 2 x0 100 = x1 100 = [ 100, 50] x2 100 = [ 100, 100] x2 100 = ], 99] ([ 100, 100]) = [ 100, 99] x3 100 = [ 100, 99] + [1, 1] = [ 99, 100] x4 100 = [100, + [ ([ 99, 100] = [100, 100]

28 Partial conclusion 28 We were lucky, as there is no reason that we can solve these semantic equations in general in nite time (undecidability results) abstract the semantics! (cf. theory of abstract interpretation, P. et R. Cousot)

29 Notion of approximation of properties 29 Example, the rule of signs: Intuitive notion of a rule of signs (where? means indeterminate sign): add + - mult ? ? How can we formalize this? [does correspond to restricting proof principles to a smaller set of predicates and consequences]

30 Galois connections - informally, for signs 30 + denotes the set of natural numbers denotes the set of opposites of natural numbers We are making approximations: what can we say about the addition of + with? we should be able to return a value I do not know We write this value: (value which will be the maximal element of the lattice of signs) denotes the set of relative integers (Z)

31 Galois connections - informally, for signs 31 In order to have a complete lattice and a Galois connection... The lattice of signs is written: (S, )

32 Galois connections - formally 32 Let C be a complete lattice of concrete properties: (e.g. ( (Z), )) Let A be a complete lattice of abstract properties: (e.g. (S, )) α : C A (abstraction) and γ : A C (concretisation) two monotonic functions (we could forget this hypothesis) such that α(x) A y x C γ(y)

33 Galois connection in the case of signs 33 Here: γ( ) = γ(0) = {0} γ(+) = N; γ( ) = N; γ( ) = Z

34 Galois connection for signs 34 Here: α( ) = α({0}) = 0 α(s) = + if S contains only positive values (and at least a strictly positive one) if S only contains negative values (and at least a strictly negative one) if S contains at least a negative (or zero) value and a distinct positive (or zero) value

35 Galois connections 35 We can check that we have the following properties: (1) α γ(x) A x (2) y C γ α(y) In fact, we even have in the case of signs: α γ(x) = x (Galois insertion). (1) or (2) are equivalent conditions to the fact that (α, γ) is a Galois connection

36 Galois connections 36 α and γ are said to be quasi-inverses: they are almost bijections as for inverses, one determines the other: α(x) = {y x C γ(y)} γ(x) = {y α(y) A x} In other terms: α give the most precise abstract value representing a given concrete property γ gives the semantics of abstract values, in terms of concrete properties

37 Abstract transfer functions 37 The abstract functions corresponding to +, - etc. can in fact be determined by calculus, given a Galois connection (α, γ) between a concrete domain C and an abstract domain A: For all concrete functionals F : C C (for example, the collecting semantics), we dene an abstract functional F : A A by F (y) = α F γ(y) It is the best possible abstraction of F. In practice, α and/or γ are not computable (algorithmically speaking). We use in general an over-approximation F such that F (y) A F (y).

38 Fixed point transfer 38 When we have a Galois connection (α, γ) between a concrete domain C and an abstract domain A, we have the following property, for all concrete functionals F : C C: α(lfp(f )) A lfp(f ) or, equivalently: lfp(f ) C γ lfp(f ) Hence, the abstract calculus gives an over-approximation of the concrete invariants!

39 Example of a computation 39 For the same old program... we then have the following semantic equations: x 0 = x 1 = 0 x 2 = x 1 x 4 x 3 = x 2 x 4 = x 3 + [1, 1] x 5 = + x 2 where we must dene the abstract transfer function corresponding to incr(x 3 ) = x 3 + 1

40 Transfer function of incr : x x We can easily check that: incr

41 Fixed point iteration 41 We begin with x 0 i =. The rst iteration gives: x 0 = x 1 = 0 x 2 = x 1 x 4 x 3 = x 2 x 4 = x 3 + [1, 1] x 5 = + x 2 x0 1 = x1 1 = 0 x2 1 = 0 x3 1 = 0 x4 1 = + x5 1 = +

42 Fixed point iteration 42 The second iteration gives: x 0 = x 1 = 0 x 2 = x 1 x 4 x 3 = x 2 x 4 = x 3 + [1, 1] x0 2 = x1 2 = 0 x2 2 = + x3 2 = + x4 2 = + x5 2 = + x 5 = + x 2 This is the xed point! In fact, we can easily show that in the lattice of signs, we have at most three iterations before getting to the least xed point

43 Use of abstract invariants for validation 43 We have over-approximations of possibles values that variables can take, by solving systems of abstract equations. We can conclude in general for safety properties (i.e., no run time error [RTE]) of programs:

44 Partial conclusion 44 But we are still far from real static analysis based validation! Finer analysis of the ranges of variables to detect at least these RTEs: overows (for given types of variables) division by zero square root of a negative number (not in this course really... but not too far) access of an element in an array, out of bounds Simple analysis: interval analysis

45 Poset of intervals 45 We dene the poset of intervals I with bounds in R as follows: an element of I is [a, b] where a, b R and a b; or is (identied with all [a, b] with a > b); or is, identied with [, ] [a, b] [a, b] [c, d] i a c and b d

46 Lattice of intervals 46 [a, b] [c, d] = [min(a, b), max(c, d)] [a, b] [c, d] = [max(a, b), min(c, d)] Complete! [a i, b i ] = [inf i I a i, sup i I b i ] i I

47 Galois connection with ( (Z), ) 47 Dene: γ([a, b]) = {x Z a x b} γ( ) = γ( ) = Z

48 Necessarily Applying the formula: if S is empty: α(s) = α(s) = [inf S, sup S]

49 Transfer functions for expressions 49 As usual, they are strict i.e. map to, and: [a, b] + [c, d] = [a + c, b + d] [a, b] [c, d] = [a d, b c] [a, b] [c, d] = [min(ac, ad, bc, bd), max(ac, ad, bc, bd)] tests are interpreted by constraint solving within intervals etc. You know them, this is interval arithmetics.

50 Example 50 void main() { int x=[-100,50]; [1] while [2] (x<100) { [3] x=x+1; [4] } [5] } x 0 = x 1 = [ 100, 50] x 2 = x 1 x 4 x 3 = ], 99] x 2 x 4 = x 3 + [1, 1] x 5 = [100, + [ x 2

51 Iteration 1 51 x 0 = x 1 = [ 100, 50] x 2 = x 1 x 4 x 3 = ], 99] x 2 x 4 = x 3 + [1, 1] x 5 = [100, + [ x 2 x0 1 = x1 1 = [ 100, 50] x2 1 = [ 100, 50] x3 1 = ], 99] [ 100, 50] = [ 100, 50] x4 1 = [ 100, 50] + [1, 1] = [ 99, 51] x5 1 = [100, + [ [ 100, 50] =

52 Iteration i + 1 (i < 50) [Kleene] 52 x 0 = x 1 = [ 100, 50] x 2 = x 1 x 4 x 3 = ], 99] x 2 x 4 = x 3 + [1, 1] x 5 = [100, + [ x 2 x i+1 0 = x i+1 1 = [ 100, 50] x i+1 2 = [ 100, 50 + i] x i+1 3 = ], 99] [ 100, 50 + i] = [ 100, 50 + i] x i+1 4 = [ i, 50 + i] + [1, 1] = [ 99, 50 + i + 1] x i+1 5 = [100, + [ [ 99, 50 + i + 1] =

53 Iteration x 0 = x 1 = [ 100, 50] x 2 = x 1 x 4 x 3 = ], 99] x 2 x 4 = x 3 + [1, 1] x 5 = [100, + [ x 2 x0 50 = x1 50 = [ 100, 50] x2 50 = [ 100, 100] x3 50 = ], 99] ([ 100, 100]) = [ 100, 99] x4 50 = [ 100, 99] + [1, 1] = [ 99, 100] x5 50 = [100, + [ ([ 99, 100] = [100, 100]

54 Acceleration of convergence 54 Classically: Use of extrapolation operators (widening) to quickly cut down Kleene iteration sequence, to get to a postxpoint Use of a narrowing operator to get to a xpoint (not necessarily the least one though)

55 Widening 55 We take here (for the widening at point 2): { a if a c [a, b] [c, d] = [e, f ] with e = { otherwise b if d b f = otherwise, and

56 Example 56 x 10 0 = x 10 1 = [ 100, 50] x 10 2 = [ 100, 58] [ 100, 59] = [ 100, [ x 10 3 = ], 99] [ 100, ] = [ 100, 99] x 10 4 = [ 100, 99] + [1, 1] = [ 99, 100] x 10 5 = [100, + [ [ 100, ] = [100, + [

57 Narrowing 57 Here we take: { c if a = [a, b] [c, d] = [e, f ] with e = a { otherwise d if b = f = b otherwise and

58 Example 58 x 11 0 = x 11 1 = [ 100, 50] x 11 2 = [ 100, ] [ 100, 100] = [ 100, 100] x 11 3 = ], 99] [ 100, 100] = [ 100, 99] x 11 4 = [ 100, 99] + [1, 1] = [ 99, 100] x 11 5 = [100, + [ [ 100, 100] = [100, 100] This is the least xed point!

59 To go further 59 Weaker frameworks of abstractions Interprocedural analysis Backward analysis and combinations forward/backward Aliases Relational domains Better solvers for xpoint equations, iteration strategies but also optimisation methods, or policy iteration Underapproximations, test case generation (see Sylvie's talk) Modular analysis, concurrency, objects, functional programs etc.

60 Aliases 60 We have to interpret aliases, even for just nding numerical invariants. Example: int *x, *z, y; y = 1; x = &y; z = &y; y = y+1; (should nd that x and z point to the same location, with value 2) More generally, any variable can point through a series of * (a dereference path), to similar locations. Impossible to nd exactly the set of aliases in general (here, at last control point, this set is {(x, z), ( x, y)}).

61 Relational analyses: interest 61 X = [0,10]; //2 Y = [0,10]; //3 S = X-Y; //4 if (S>=2) //5 Y = Y+2; // 6 In intervals: Y in [0,12] at 6 Using relations: At 5: X-Y>=2 hence Y<=8 So at 6, Y in [0,10]

62 A partial zoo of domains 62

63 A partial zoo of domains Also: our ane arithmetic based domains (FLUCTUAT) (see to play with some of these domains!) 63

64 A last short demo of INTERPROC/APRON 64...