ALGORITHMS AND COMPLEXITY THEORY

Transcription

1 ALGORITHMS AND COMPLEXITY THEORY Chapter 2: Induction and Recursion Jules-R Tapamo Computer Science- Durban - February 2010 Contents 1 The principle of induction 2 2 recursive procedures Principles Some simple examples Time and storage cost of recursion Recursion Trees correctness of algorithms Some definitions proof of correctness of an algorithm computation of a product using a loop induction Proof of Recursive Procedure

2 2 2 RECURSIVE PROCEDURES 1 The principle of induction Induction is critical to many aspects of algorithms and data structures. In particular, virtually all correctness arguments are based on induction. Suppose we are asked to prove the result (2n 1) = n 2 In other words, we have to show that the expression defined recursively on the left is equally defined by the formula on the right, for all positive integers n. We might proceed as follows. The formula is certainly correct when n = 1, since 1 = 1 2. Suppose it is correct for a specific value of n, say n = k, so that (2k 1) = k 2. This fact can be used to simplify the left hand side when n = k + 1, as follows: (2k + 1) = (2k 1) + (2k + 1) = k 2 + (2k + 1) = (k + 1) 2 So if the result is correct when n = k then it is correct when n = k + 1. In practice, we usually present a proof by induction in more descriptive terms as follows: Here n is read for all n P stand for property. P (0) ( n)(p (n) P (n + 1)) ( n)p (n) P(0): property P holds for 0 P (n) P (n + 1): if P holds for n, then it holds for n + 1 ( n)p (n) : P holds for every n Proofs by induction are then done according to the following format: Induction Basis : verify that the property is true for n=0, Induction step : Assume that the property is true for n and deduce the property is true for n + 1 Hence : result by induction. There are several modified forms of the principle of induction. Sometimes it is convenient to take for the induction basis the value n=0; on other hand it may be appropriate to take a value 2 or 3, since the first few cases may be exceptional. Another useful modification is to take as induction hypothesis the assumption that the result is true for all relevant values k n, rather than n = k alone. This formulation is sometimes called the strong induction. 2 recursive procedures 2.1 Principles Recursion is simply a technique of describing something partly in terms of itself? There are many application of recursion. There are four advantages of recursion programming. For many problems the recursive solution is more natural than the alternative non-recursive solution. It is relatively easy to prove the correctness of recursive procedures.

3 2.2 Some simple examples 3 Recursive procedures are relatively easy to analyze and it is also easy to determine their performance. This analysis produces recurrence relations, many which can easily be solved. Recursive procedures are relatively flexible. But there are problems to solve when recursion is used: Recursive procedures may run slowly than equivalent non-recursive ones. There are two causes of this. Firstly, a compiler may be implement recursive calls badly. Secondly, the recursive procedures we write may be simply inefficient. Recursive procedures require more storage space than their non-recursive counterparts. Each recursive call involves the creation of an activation record, and if the depth of the recursion is large this space penalty maybe significant. 2.2 Some simple examples EXAMPLE T he simplest example of recursion is the factorial function, which is defined by: { 1, p=0 p! = p, p > 0 From this definition we have the following function: int n) Fact1(int 1: int f = 1; 2: for i = 1 to n do 3: f = f * i 4: end for 5: return f Which is a non-recursive version of the algorithm. It is proved that { 1, p=0 p! = p (p 1), p > 0 from which we have the following recursive version of the above function. int Fact2(int p) 1: if ( n == 0) then 2: return 1 3: else 4: return n*fact2(n-1) 5: end if

4 4 2 RECURSIVE PROCEDURES EXAMPLE A s a second example of recursion, we consider the highest common factor (HCF) of two positive integers p and q. A description of Euclid s algorithm for finding the HCF usually goes something like this: Divide p by q to give a remainder r. If r = 0 then the HCF is q. Otherwise repeat with q and r taking place of p and q. The non recursive version of Hcf is int Hcf(Int p, int q) 1: int r 2: r = p mod q 3: while r 0 do 4: p = q 5: q = r 6: r = p mod q 7: end while 8: return q Mathematically Hcf can be formulated as follows: { p if p mod q = 0 hcf(p, q) = hcf(q, pmod q) Otherwise From this formula we can derive the following recursive version of Hcf. int Hcf(Int p, int q) 1: int r 2: begin 3: r = p mod q 4: if r 0 then 5: return q 6: else 7: return Hcf(q,r) 8: end if Definition : Given a recursive procedure P: P is a simple recursion if all recursive calls appear in the the body of P. P is a crossed recursion if P calls Q and Q calls P Time and storage cost of recursion From the implementation, the cost in term of storage associated to recursive procedures is clear. If n is the maximum recursive depth, then the store required is n (p + l + 2) where p represents the space required by the parameters and l that required by the local variables. Where the alternative non-recursive solution requires only a small number of local variables for its operation. The time cost of recursive algorithms are mostly based on recurrence relations. It is then convenient to have the notion of size of a problem, so that if T k represents the cost, however defined, of evaluating a procedure of size k then the recurrence relation defines T k in terms of the cost of evaluating the smaller problem(s) into which it is broken down. For linear recursion (recursion where there is only one call) the size is closely related to the recursive depth and T k is defined in terms of T k 1. For the factorial case we have:

5 2.3 Recursion Trees 5 { b + Tk 1 if k > 0 T k = a if k = 0 where a and b are appropriate constants. T n can be determine simple by a process of substitution as follows: T n = b + T n 1 = b + b + T n 2 = 2 b + T n 2. = b n + T 0 = bn + a It means the running time of the recursive version of factorial T n is O(n) ( linear ). 2.3 Recursion Trees Let P be a recursive procedure with only simple recursive calls. There are n recursive calls, as shown in the following algorithm: 1: P(var) 2: if Test(var) then 3: return(val) 4: else 5: Instructions 1 6: P (φ 1 (var)) 7:... 8:... 9: P (φ n (var)) 10: Instructions n+1 11: end if To each call P (v) of P, we can associate a tree A(v) called, execution tree of P for the call P (v), recursively defined as follows : If T est(v) is true, then A(v) is reduced to the root: A(v) = V al If T est(v) is not true, then A(v) is equal to: figure 2.1: Recursion tree This tree is easily extendable to any other form of recursion.

6 6 3 CORRECTNESS OF ALGORITHMS EXAMPLE Sum of elements of an array. To compute the sum of n elements(real) of an array T [1 : n] S(n) = n T [i] It is sufficient to point out that if n = 0, then the sum is equal to 0 and if n 1, S(n) = S(n 1) + T [n] and we have the following recursive algorithm: 1: int SUM( float [] T, int n) 2: if n == 0 then 3: return 0 4: else 5: return(sum(t,n-1)+t[n])) 6: end if Consider for instance the call of SUM(T, 2) The execution tree will be as shown in figure 2.2: i=1 figure 2.2: Recursion tree of SUM(T, 2) 3 correctness of algorithms 3.1 Some definitions It is generally known that the proof of the correctness of a program is not an easy task. We will use informal methods to prove that the recursive procedures discussed in here work as specified. The main proof is done through well-founded induction. It is demonstrated that the program works correctly for all base cases. The induction hypothesis is then all recursive calls of the the procedure work correctly, provided the initial states meet the preconditions of the procedure. Given this hypothesis, the inductive step requires one to show that, under the inductive hypothesis, the program runs correctly, given that the domain is well founded. Before any attempt of proof of program we will give some useful definitions. A block is a section of code with one entry and one exit point. Blocks are main subdivisions of program code and procedure code. A procedure is a block with a name, so it can be called. It usually has parameters, which are input or output. A function is a procedure with some output parameters. Control structure are mechanisms for causing various blocks to be executed. The elementary control structure are: sequence of blocks

7 3.2 proof of correctness of an algorithm 7 alteration ( if condition then true-block, else false-block) procedure call A precondition is an assertion concerning the initial state of a block and postcondition is an assertion regarding the final state of a block. Proving the correctness of a procedure means proving certain logical statements about this procedure. A block is said to be partially correct with respect to the precondition P and the postcondition Q if the final state of the code always satisfies Q, provided that the code starts in P and provided that it terminates. The code is said to be totally correct if for each state satisfying P the code terminates with a final state satisfying Q. EXAMPLE A. For an algorithm to compute a product of nonnegative integers precondition : the input variables m and n are nonnegative integers. postcondition: The output variable p equals m.n. B. For the algorithm to find the quotient and the remainder of the division of one positive integer by another. precondition : the input variables m and n are positive integers. postcondition: The output variables q and r are integers such that a = b.q + r and 0 r < b. 3.2 proof of correctness of an algorithm The proof of correctness of an algorithm consists of showing that if the preconditions for the algorithm is true for collection of values for the input variables and if the statements of the statements of the algorithms are executed, then the post-condition is also true. To prove the correctness of an algorithm we will divide this algorithm into sections with assertions about current state of algorithm variables inserted at adequate chosen points; [Assertion 1: precondition of the algorithm] {algorithms statements} [Assertions 2] {algorithms statements} [Assertions k-1] {algorithms statements} [Assertion 1: post-condition of the algorithm] Successive pairs of assertions are treated as pre- and post-conditions for algorithms between them. Then for i = 1, 2,..., k 1, IF we can prove that if assertion i is true and algorithm statement between assertion i and assertion i+1 are executed, then assertion i+1 is true, THEN the algorithm is correct. Loop invariant is a condition on the variables within a loop, which remains true during the iterations of the loop. The method of loop invariants is used to prove the correctness of a loop with respect to certain pre- and post-conditions. It is based on the principle of Mathematical induction. Suppose an algorithm contains a while loop and that entry to this loop is restricted by a condition G, called guard. And this loop is specified as follows:

8 8 3 CORRECTNESS OF ALGORITHMS [pre-condition of the loop] WHILE(G) [Statements in the body of loop. with no statement branching outside the loop] [END WHILE] Then there theorem assuring that the loop is correct. Theorem:Loop Invariant Theorem: Let a while Loop with a Guard G be given, together with pre- and post -conditions, and a loop invariant I(n). If the following four properties are true, then the loop is correct with respect to its pre- and postconditions: A. Basis property : the precondition for the loop implies that I(0) is true before the first iteration of the loop. B. Inductive property : If the G I(k) =true for k 0 before an iteration of the loop, then I(k + 1) is true. C. Eventual Falsity of Guard : After a finite number iterations of the loop, the guard becomes false. D. Correctness of the post-condition: If N is the least number of the iterations after which G is false and I(N) is true, then the values of the algorithm will be as specified in the post-condition of the loop computation of a product using a loop we want to use a loop to compute the product m x, we have the variables i and product with i = 0 and product = 0. the loop is defined as follows: [pre-condition: m is a nonnegative integer, x is a real, i=0, and product =0] WHILE ( i<>m ) (1) product := product +x (2) i := i+1 ENDWHILE [post-condition: product = m*x] the loop invariant is I(n) : i = n and product = n.x and the guard G of the while loop is G : n <> i. A. Basis property : implies that I(0) is true before the first iteration of the loop. I(0) is i = 0 and product = 0.x which is true before the first iteration. B. Inductive property : If the G I(k) =true for k 0 before an iteration of the loop, then I(k + 1) is true. product old = k.x and i = k since i <> m, the guard is passed and from the statement (1) we have a product new = product old + x = k.x + x = (k + 1)x, after execution of statement (2) we have i new = i old + 1 = k + 1

9 3.3 induction Proof of Recursive Procedure 9 C. Eventual Falsity of Guard : After a finite number iterations of the loop, the guard becomes false. for all n 0, if the loop is iterated n times, then i = n and product =n.x. so after m iteration of the loop, i =m. thus G becomes false after m iterations. D. Correctness of the post-condition: If N is the least number of the iterations after which G is false and I(N) is true, then the values of the algorithm will be as specified in the post-condition. G is false and I(n) is true means product = m.x 3.3 induction Proof of Recursive Procedure Definition : The Domain or Universe of Discourse is the collection of all persons, ideas, symbols, data structures, etc. that affect the logical argument under consideration. The elements of the domain are called individuals. An individual can be, a person, a number, a data structure, or anything we want to reason about. EXAMPLE if the domain consists of persons, individuals can be represented by their names. In the case of natural numbers, individuals are digits representing these numbers( i.e 0,1,2,3,...) In the proposition George and Jack are siblings, George and Jack are individuals in the domain of persons. In the proposition The sum of 4 and 7 is 11, 4, 7 and 11 are individuals in domain of numbers. The fact that the recursion will terminate is crucial, and proofs by induction are unsound unless this fact can be established. Hence it is necessary to introduce the notion of descending sequence, is a sequence of smaller and smaller elements. A Domain in which all descending sequences are finite is said to be well founded. Recursive procedures are partially correct if the base cases are correct and if induction step can be established. Let the following { be the algorithm to find an integer Key in a sorted array Vector[First, Last] of integers, true if Key is found and save its position in Position which returns f alse Otherwise boolean search(int First, int Last, int Key, int [] Vector) 1: if (First > Last ) then 2: return false 3: else 4: Middle = (First+Last)/ 2; 5: if (Key < Vector[Middle]) then 6: return search(first, Middle-1,Key, Vector); 7: else if (Key >Vector[Middle]) then 8: return search(middle+1,last,key,vector); 9: else 10: Position = Middle; 11: return true 12: end if 13: end if

10 10 3 CORRECTNESS OF ALGORITHMS EXAMPLE T o show that the above function is totally correct, it must be shown that it is partially correct and that the domain is well founded. The domain consists of all intervals from a to b, where a and b can assume any values inside the range of the array vector. Induction basis: The base case is the case in which there is no element between First and Last, and the function will return false. Induction hypothesis Choose any value for First and Last. These 2 values provide an interval, and the inductive hypothesis is that the function works for all proper subintervals of this interval. Induction step : Since the array Vector is sorted., Key must be in the interval from First to Middle -1 if Key< Vector[Middle], and it must be in the interval Middle+1 to Last if Key > Vector[Middle]. Both intervals are proper subintervals of the interval with the end points First and Last, and by induction the procedure is correct for these intervals. Then the procedure works correctly. Domain well founded The domain is well founded because all sequence in which the subsequent element is a proper subinterval of preceding element must be finite. Bibliography 1. Baase S. and Gelder A., Computer Algorithms: Introduction to Design and Analysis, Addison Wesley, Knuth D.E., The art of Computer Programming: Fundamental Algorithms, Addison-Wesley, 1997.