CS243, Logic and Computation Recursion and Induction

CS243, Prof. Alvarez 1 DEFINITION BY RECURSION Prof. Sergio A. Alvarez http://www.cs.bc.edu/ alvarez/ Maloney Hall, room 569 alvarez@cs.bc.edu Computer Science Department voice: (617) 552-4333 Boston College fax: (617) 552-6790 Chestnut Hill, MA 02467 USA CS243, Logic and Computation Recursion and Induction You may be familiar with recursion as a programming technique that uses self-reference, as in Algorithm 1. This technique can yield elegant and concise code. Algorithm 1: Input: integer n 1 Output: an integer value determined by the value of n f(n) (1) if n == 1 (2) return 1 (3) else (4) return 2n 1+f(n 1) We will be interested in constructing airtight arguments about such recursive definitions. The fundamental technique for doing so is mathematical induction. Recursion can also be used to define objects more general than functions on integers, and induction will often be the tool of choice for proving statements about such recursively defined objects. 1 Definition by Recursion A recursive definition specifies the elements of a set S in two parts: 1. A basis or base case that defines the simplest objects in S. 2. A recursive step that defines how objects in S can be individually modified, or combined, to produce more complex objects in S. A third part of a recursive definition, closure, is not always stated explicitly: S is taken to consist of all objects that are either defined in the basis, or that can be constructed from basis objects by a finite number of applications of the recursive step. Another way of stating this part is that S is the smallest set that is consistent with the basis and recursive steps. Examples of recursive definitions Recursion on integers. Algorithm 1 provides a first example of definition by recursion. This is a classical example of recursion on the set of positive integers. The basis of the

1 DEFINITION BY RECURSION definition involves the argument value n = 1, and states that F (1) has (returns) the value 1. The recursive case of the definition considers the step from n 1 to n, and states that F (n) is obtained by adding 2n 1 to F (n 1). The reference to a set S in the formal version of definition by recursion may seem inapplicable at first. However, that is just a formality: you can think of the function F from Algorithm 1 as corresponding to the set S of all pairs (n, F (n)), where n is a positive integer. Another example of recursion on integers is exponentiation, that is, raising a base b to an integer power p. This can be done efficiently using recursion in the exponent, by noting that halving the exponent (in integer arithmetic) produces the square root of the desired answer in the case of an even exponent, and misses precisely one copy of the base in the case of an odd exponent: (Basis) b 0 = 1, b 1 = b (Recursive step) if p > 1, then b p = ( b p/2 ) 2 b p mod 2 The Python code below implements the above approach. >>> def power(base, exp):... if exp==0:... return 1... if exp==1:... return base... halfway = power(base, exp/2)... return power(base, exp%2) * halfway**2 Structural recursion. An example of a different type is the following simplified definition of arithmetic expression, which can be used as part of the specification of a programming language such as Python. The definition is simplified in that only addition and multiplication are allowed as arithmetic operators. In contrast with the example of Algorithm 1, this definition of arithmetic expressions involves recursion in the syntactical structure of the expressions, rather than the set of positive integers. Definition 1.1. The set of additive/multiplicative arithmetic expressions is defined as follows. Note that parentheses are a required syntactical element in all but the simplest expressions. 1. A number constant (e.g., 25 or 495.37) is an arithmetic expression. 2. If E and F are arithmetic expressions, then so are (E + F ), and (E F ). The set S referred to in the general description of definition by recursion is again implicit here: S is just the set of all arithmetic expressions of the given type.

1 DEFINITION BY RECURSION Parse trees. You can think of the above definition as being associated with a tree structure for each valid arithmetic expression: the basis constructs the leaves of the tree, and the recursive case constructs a larger tree as a root node corresponding to an arithmetic operator and four child nodes corresponding to the two outer parentheses and two simpler arithmetic expressions that form the operands. For example, the expression (25 (7 + 10)) has the following parse tree: * ( 25 + ) ( 7 10 ) The abbreviated form of the tree, omitting the parentheses, would be the following: * 25 + 7 10 Example 1.1. How to write a Python program that will detect if a string has the form illustrated in the examples below? (*) (*(*)) (*(*(*)))... We do this by capturing the key string pattern as a definition by recursion: (Basis) The empty string has the target form (Recursion) If a string s has the target form, then so does the string (*s) It is now an easy matter to translate this recursive definition into a Python function: >>> def hastargetform(s):... if len(s)==0:... return True... return s[0]== ( and s[1]== * and s[-1]== ) and hastargetform(s[2:-1])

2 Proof by Induction 2 PROOF BY INDUCTION Induction in the present context refers to a technique for proving statements about collections of objects defined by recursion. There may be some property of a newly defined collection of objects that we are interested in proving (like a specific simplified expression for a recursively defined function, or some structural property of a recursively defined tree). Visualize individual objects as dominos standing on edge. Proving the property about a particular object can be visualized as toppling the corresponding domino. Intuitively, a recursive definition lines up the dominos very nicely: the first domino or two are defined by the basis, and the recursive step takes care of connecting consecutive dominos. If the first domino falls over, then the second will follow, then the third, and so on. Assuming that a chain reaction is correctly supported by the setup, all of the dominos will eventually topple. 2.1 Ways of messing up a domino chain How could the above chain reaction possibly not work? Either one of the following would prevent a full chain topple. 1. The first domino doesn t topple. 2. Some pair of consecutive dominos is not correctly set up, so that the earlier of the two can topple without the later one toppling. Addressing both of the above potential concerns will lead to the happy outcome in which all of the dominos topple. 1 This is important enough that we will state it more formally. Using a formal model has the advantages of clarity and specificity about assumptions. A general formal description of proof by induction is given below. 2.2 Principle of proof by induction Theorem 2.1. Principle of proof by induction. Suppose that S is a set of objects defined by recursion, and that P is a property of objects such that: 1. All objects of S defined in the basis of the recursion have property P. 2. Whenever objects in S that are known to have property P are combined as specified in the recursive case of the definition of S, the resulting objects also have property P. Then all objects in S have property P. 2 1 There is actually a third way of messing up the chain, but it is devious enough to postpone discussion. 2 This relies on subtleties in the notion of definition by recursion. See comments at the end of these notes.

2.3 Examples of reasoning using mathematical induction 2 PROOF BY INDUCTION As a matter of terminology, 1 is known as the induction basis (or simply basis), and 2 is known as the inductive step. The antecedent condition in the inductive step, namely that the objects to be combined have the target property, P, is called the induction hypothesis. Thus, the inductive step involves verifying that, if the induction hypothesis holds for a set of objects, it also holds for the result of combining those objects as described in the recursive case of the definition. The term induction refers to the specific-to-general reasoning that is carried out here: the set of objects that satisfy the target property is expanded to contain the additional objects constructed in the recursive step of the definition. 2.3 Examples of reasoning using mathematical induction 2.3.1 Induction on integers As a first example, consider the code in Algorithm 1. Can you find a concise expression for the function computed in this case? What if you want to be sure that things work as you believe? We propose modeling the situation in terms of a recursively defined set to which the principle of proof by induction will apply. Here goes. Notice that we seek an expression that will be valid for all positive integers, n. Thus, it is natural to think of this problem as requiring proving that all positive integers satisfy a certain propoerty. To begin, you must state what you hope to be true for a given n. In the case of Algorithm 1, one sees that F (1) = 1, F (2) = 4, and F (3) = 9. With these examples in hand, a reasonable target property would be the following. The target property of an unspecified positive integer, n. F (n) equals n 2. We now use mathematical induction to verify that this educated guess is actually correct for all positive integers, n. In order to prove that the target property holds for all positive integers, we verify the assumptions of the principle of proof by induction in this particular context. This requires making explicit the choice of the set of positive integers as the recursively defined set, S, in the general principle of proof by induction. We will also explicitly provide a recursive definition for this set, below. Recursive definition of the set of positive integers. (basis) 1 is a positive integer. (recursive step) if n is a positive integer, then n + 1 is a positive integer. (closure) the set of positive integers is the set of all numbers that can be obtained from 1 by a finite number of applications of the recursive step (that is, by adding 1 a finite (zero, one, or more) number of times). This may seem like a silly definition. However, it lays bare the most fundamental assumptions about the nature of the counting numbers: there is a first one, and there is a next one after each one. This establishes the soundness of the domino chain reaction metaphor.

2.3 Examples of reasoning using mathematical induction 2 PROOF BY INDUCTION Proof by induction of the target property for all positive integers. By the principle of proof by induction, any induction proof must be patterned after the recursive definition of the underlying set S, which is the set of positive integers in the current example. The only base case therefore involves the value n = 1, and the recursive case considers the step from n to n + 1. (basis) Claim: F (n) equals n 2 for all n considered in the basis of the recursive definition of F. The only values of n to consider in this case is n = 1. Determining if 1 belongs has the target property requires verifying that the value F (1) is 1 2, that is, 1. That s precisely what it is (see Algorithm 1). Done. Claim: if a positive integer satisfies the target property, then so do all corresponding values considered in the recursive case of the definition of F. Because of the manner in which Algorithm 1 is written, the recursive step involves the transition between argument values n 1 and n, where n is a positive integer other than 1. We assume (induction hypothesis) that the value n 1 satisfies the target property, meaning that F (n 1) equals (n 1) 2 for that particular n. Now we must determine if the value n resulting from the recursive step of the definition also satisfies the target property. We see by the algorithm that F (n) returns the sum 2n 1+F (n 1). However, we know by the induction hypothesis that F (n 1) equals (n 1) 2, and the recursive definition just tacks on the extra 2n 1 term, so we conclude that F (n) equals 2n 1+(n 1) 2. This simplifies to n 2. This shows that n satisfies the target property, so we ve completed the inductive step. Having verified that the two requisite properties hold, we conclude by the principle of proof by induction that all positive integers satisfy the target property. In other words, it is true for all positive integers n that F (n) returns n 2. Case closed. 2.3.2 Structural induction on arithmetic expressions We now give a simple example of proof by induction in the case of a definition by structural recursion, namely Def. 1.1. We consider a quick way of checking for errors in an arithmetic expression. It is intuitively clear that an error-free expression should have the target property below. However, we will need to verify that our formal definition in section 1 actually passes this smell test. The target property of an unspecified expression, E. E has the same number of left and right parentheses. 1. Claim: Any arithmetic expression defined in the base case has the same number of left and right parentheses. This is obvious, as the basis expressions are just numbers without any parentheses at all.

2.3 Examples of reasoning using mathematical induction 2 PROOF BY INDUCTION 2. Claim: if E and F are arithmetic expressions, which has the same number of left and right parentheses (nothing is stated about how the parenthesis counts compare across the two expressions), then each of the expressions E +F and E F has the same number of left and right parentheses (again, only counts within a given expression are considered). We now prove this claim. Each of the expressions (E + F ) and (E F ) have as many left parentheses as E and F combined plus one, and as many right parentheses as E and F combined plus one. Thus, these two expressions defined in the recursive step retain a balance between left and right parentheses, assuming (induction hypothesis) that each of E and F possess that balance. 2.3.3 Structural induction in formal languages Consider the context-free grammar G that has the rules below. The start symbol is S, other variables are A and B, and the three lowercase letters a, b, c are terminal symbols. S AB A aacc ɛ B cbbb ɛ Strings derivable in G include ɛ, acc, cbb, and aacccccbb. We would like to prove assertions about the set of derivable strings. Does induction apply? The answer is yes. In order for induction to be applicable, we must first identify an underlying recursively defined set such that whatever target assertion we wish to prove describes a property that all objects of that set presumably have. The idea is that the rules of G do, in fact, constitute a recursive definition of a set of strings. What set is that? We ll discuss this below. Derivations in the above grammar G. Any derivation according to the rules of G will proceed as follows: Start wth the single variable S. At the first stage of the derivation, there is no choice but to apply the rule S AB, yielding the derived string of variables AB. At each subsequent stage of the derivation, a certain string w of terminals and/or variables will have been derived so far. Application of the rules in the bottom two lines of the description of G will allow the derivation to continue, producing other strings of terminals and/or variables, in the following way. If w matches the pattern on the left-hand side of one of the rules of G in the bottom two lines of the grammar description, then that rule may be applied to continue the derivation. More specifically in the present case: if w has an A in it somewhere, in other words, if w is of the form xay for some prefix string x and some suffix string y, then it is allowable to derive the string xaaccy from w, simply by substituting aacc for the particular occurrence of A that has been singled out for the particular x and y being considered (there could be several distinct occurrences of A within w). The other rules yield additional possibilities. Taken together, all of this produces a definition by recursion of

2.3 Examples of reasoning using mathematical induction 2 PROOF BY INDUCTION the set of strings of terminals and/or variables that can occur during a valid derivation in the grammar G. It is this latter set that we can define by recursion via the grammatical rules of G. Recursive definition of the set of strings derivable in the grammar G. Here s some notation to make the going a little easier. Notice that the strings of terminals and/or variables are just elements of the Kleene closure {S, A, B, a, b, c}. We ll write S w when we mean that w can be derived from the start symbol S by a finite number of applications (zero, one, or more) of the rules of G, starting from S. The set that we will define by recursion is therefore the following: deriv(g) = {w {S, A, B, a, b, c} S w} With suitable notation in place, here s the actual recursive definition of deriv(g): (basis) S deriv(g) and AB deriv(g). This accounts for the start symbol and the result of the first unavoidable rule application in any derivation with at least one step in it. (recursive steps) if xay deriv(g), where x and y are strings of terminals and/or variables, then xaaccy deriv(g) and xy deriv(g). This accounts for the two rules that have A on the left-hand side. if xby deriv(g), where x and y are strings of terminals and/or variables, then xcabby deriv(g) and xy deriv(g). This accounts for the two rules that have B on the left-hand side. (closure property) deriv(g) is the set of all objects that are obtained by a finite number (zero, one, or more) of applications of the recursive steps above, starting from the objects defined in the basis. Example of an induction proof for strings derivable in the grammar G. One may be interested in proving some general assertion about the strings that can be derived in G. Given that the set that is naturally defined recursively by G consists of strings that may include variables in addition to terminals, induction requires extending the target assertion to one that applies to all such more general strings in deriv(g). Here are some examples of possible target assertions of that type: If w deriv(g), then any occurences of a in w precede any occurrences of b in w, and any occurrences of b in w precede any occurrences of c in w. If w deriv(g), then the number of terminal symbols in w is divisible by 3.

3 SUBTLETIES Let s prove the second of the above assertions by induction. As always, the induction proof must be patterned after the recursive definition of the set of the objects to which the target assertion applies. In the present context, that set is deriv(g). Here goes: (basis) Show that the number of terminals in S is divisible by 3 and that the number of terminals in AB is divisible by 3. This is true, because there are 0 terminals in each of these, and 0 is divisible by 3. Done. (recursive steps) Suppose that xay deriv(g) has a number of terminals that is divisible by 3. Show that xaaccy and xy each have a number of terminals that is divisible by 3. This is easy, because the first of these adds precisely 3 terminals, and the second doesn t add any. In both cases, you again end up with a number divisible by 3. Suppose that xby deriv(g), has a number of terminals that is divisible by 3. Show that xcabby and xy each have a number of terminals that is divisible by 3. Again, each of these rules changes the number of terminals by a multiple of 3, so divisibility by 3 of the number of terminals is not affected. Done. That completes the proof. By the principle of proof by induction, we conclude that all strrings in deriv(g) have a number of terminals that is divisible by 3. 3 Subtleties 3.1 Induction proof that all upper dyadic numbers are red Definition of upper dyadic number Consider the universe of all real numbers greater than 1 that differ from the next integer up by an amount of the form 1/2 n for some positive integer, n. Call such numbers upper dyadic. For example, 1.5 qualifies (n = 1), as does 6.75 (n = 2), but 1.25 does not, and neither does 5 1/3. Notice that 1.5 is the smallest upper dyadic number and that the next such number is (x + x )/2, where x is the smallest integer greater than or equal to x. 3 Redness of upper dyadic numbers We define the notion of redness for upper dyadic numbers by recursion, as follows: 1. 1.5 is red. 2. if x is a red upper dyadic number, then so is (x + x )/2. For example, since 1.5 is red, so are 1.75, 1.875, 1.9375, and so on. The above two facts give us a basis and inductive step for a proof by induction. We conclude that all upper dyadic numbers are red. 3 You should not blindly accept such claims prove them.

4 EXERCISES Comments The argument works well in many cases. For example, we can verify the redness of 1.9375 by consdering the chain 1.5, 1.75, 1.875, 1.9375 (basis, plus three applications of the recursive step). However, what about the upper dyadic number 2.5? One realizes that this number (and many others) have no immediate predecessor in the natural ordering associated with the recursive step. That is to say, there is no upper dyadic number x such that (x + x )/2 equals 2.5. It is easy to see that x = 2 would be the only solution, but 2 is not upper dyadic. In fact, it would be impossible to prove that 2.5 is red, since it may well not be and there would be no contradiction of known facts. This is where the easily overlooked closure property in definition by recursion comes in: induction only extends to the closure of the basis elements, defined as the set of all objects that can be constructed from the basis objects in a finite number of applications of the recursive rules. In the present context, the closure will include only the upper dyadic numbers that are less than 2. All such numbers can be proved to be red by induction. Proof of Theorem 2.1. Suppose that S satisfies the assumptions of Theorem 2.1, and let x be any element of S. Since S is assumed to be defined by recursion, this element x must either be one of the objects added to S in the basis of the recursive definition, or else must be an element that can be obtained from known elements of S by a finite number of applications of the recursive step in the definition. If x is a basis element of S, the theorem s assumption 1 that all basis elements have property P shows that x in particular has property P. If, on the other hand, x is not a basis element, then x must be obtainable from basis elements of S by a finite number of recursive step applications. Each such step along the way from these basis elements to x will preserve property P, by assumption 2. Hence, x must have property P also. This proves that x has property P in all cases. Since this argument applies to any element of S, we conclude that all elements of S have property P, as stated. 4 Exercises 1. Give a clear definition by recursion for each of the sets described below. Explain. (a) The set of all positive even integers. (b) The set of all polynomials in a single variable, x. Note: we ll agree that the basis should only define a finite number of different polynomials. 2. For each of the following quantities, discover and state a simple algebraic expression with value equal to the given quantity, and give a proof by induction that the original quantity and your simplified expression are indeed equal for all values of n. (a) 1 2 + 1 2 2 + + 1 2 n (b) The number of parentheses in the expression E n, where E 1 is the single digit 1, and E n for any n > 1 is the parenthesized expression (E n 1 + E n 1 ).

4 EXERCISES (c) 1 2 + 3 + ( 1) n 1 n 3. Prove the following rule for the number of elements in a Cartesian product by induction: S 1 S k = S 1 S k Notice that the symbol denotes Cartesian product on the left and integer product on the right. 4. Define two functions on non-negative integers, n, as follows: { { 1, if n = 0 1, if n = 0 f(n) = g(n) = 2g(n 1), if n > 0 f(n 1) + 3, if n > 0 Find a simple expression for g(2n), valid for all non-negative integers n, and prove it by induction carefully. 5. It is often stated that exponential growth is faster than polynomial growth. In this task, you will prove the following specific version of this statement in detail: There is an integer n 0 such that 2 n is larger than n 2 for all integers n n 0. Prove this statement by induction. Find the smallest value of n 0 that works. 6. Prove the claim made in section 3.1 that the upper dyadic number immediately above x is (x + x )/2.