4.4 The recursion-tree method

Transcription

1 4.4 The recursion-tree method Let us see how a recursion tree would provide a good guess for the recurrence = 3 4 ) Start by nding an upper bound Floors and ceilings usually do not matter when solving recurrences Create a recursion tree for the recurrence = 3 4 having written out the implied constant coefcient > 0 MAT AADS, Fall Sep MAT AADS, Fall Sep

2 MAT AADS, Fall Sep Subproblem sizes decrease by a factor of 4 each time we go down one level => we eventually must reach a boundary condition Subproblem size for a node at depth is /4 It hits =1when /4 = 1or, equivalently, when = log Thus, the tree has log +1levels (at depths 0, 1, 2,, log ) MAT AADS, Fall Sep

3 Each level has 3 the nodes of level above the number of nodes at depth is 3 Subproblem sizes reduce by a factor of 4 for each level we go down, each node at depth, = 0,1,2,,log 1, has cost of 4 Multiplying, we see that the total cost over all nodes at depth is 3 4 ) = (3 16) The bottom level, at depth log, has 3 = nodes, each contributing cost (1), for a total cost of (1), which is ), since (1) is a constant ) MAT AADS, Fall Sep Add up the costs over all levels to determine the cost for the entire tree: 3 = ) 16 (3 16 = ) 1 (3 ) 16)1 By the value of a geometric (or exponential) series = 1 1 MAT AADS, Fall Sep

4 We can use an innite decreasing geometric series as an upper bound = 3 16 ) = (3 16) = ) MAT AADS, Fall Sep We have derived a guess of for our recurrence = 3 4 ) Root of the tree contributes to the total cost a constant fraction of the total cost The cost of the root dominates the total cost If is indeed an upper bound for the recurrence, then it must be a tight bound First recursive call contributes cost of ), must be a lower bound for the recurrence MAT AADS, Fall Sep

5 Use the substitution method to verify that the guess was correct, i.e., is an upper bound for the recurrence We want to show that for some constant >0 = ) = 3 16 as long as (16 13) MAT AADS, Fall Sep A more intricate example is the recurrence ) = /3) + (2/3) + ) represents the constant in the term Adding the values across the levels of the recursion tree yields value for every level The longest simple path from the root to a leaf is (23 (2 3) 1 (2 3) = 1when = log The height of the tree is log MAT AADS, Fall Sep

6 MAT AADS, Fall Sep We expect the solution to the recurrence to be number of levels cost of each level: log ) = lg ) Not every level contributes a cost of If this recursion tree were a complete binary tree of height log, there would be leaves 2 The cost of each leaf is a constant The total cost of all leaves would then be ) which, since log 2>1, is ( lg ) MAT AADS, Fall Sep

7 This recursion tree is not a complete binary tree, however, and has fewer than leaves As we go down from the root, more and more internal nodes are absent Consequently, levels toward the bottom contribute less than to the total cost we can use the substitution method to verify that lgis an upper bound for the solution to the recurrence MAT AADS, Fall Sep The master method Theorem 4.1 (Master theorem) Let and > 1,and ) a function, ) is dened on nonneg. integers by recurrence )+, where we means either or. ) has the following asymptotic bounds: MAT AADS, Fall Sep

8 1. If = ( ) for some constant > 0, then )= 2. If )=, then )= lg 3. If )= for some constant >0, and if ) ) for some constant <1 and all sufciently large, then ) = ) MAT AADS, Fall Sep In each of the three cases, we compare the function ) with the function The larger of the two determines the solution to the recurrence In case 1, the function is the larger, then the solution is )= In case 3, the function ) is the larger, then the solution is ) = In case 2, the two functions are the same size, we multiply by a logarithmic factor, and the solution is )= lg MAT AADS, Fall Sep

9 Be aware of some technicalities: In case 1, not only must ) be smaller than, it must be polynomially smaller ) must be asymptotically smaller than by a factor of for some >0 In case 3, not only must ) be larger than, it also must be polynomially larger and satisfy the regularity condition that () () MAT AADS, Fall Sep Using the master method ) = 93) We have =9,=3,)=, and thus ) Since ) = ), where =1, we can apply case 1 of the master theorem and conclude that the solution is ) MAT AADS, Fall Sep

10 ) = (2 3) +1 Here =1, = 3 2, )=1, and =1 Case 2 applies, since = (1), and thus the solution to the recurrence is (lg ) MAT AADS, Fall Sep () = ( 4) + lg We have =3,=4, lg, and ) Since ) = ), where 0.2, case 3 applies provided that the regularity condition holds for ) For sufciently large, we have that )= 3(4) lg(4) (34) lg = ) for = 34 By case 3, the solution is lg ) MAT AADS, Fall Sep

11 The master method does not apply to ) = 22) lg even if it seems to have the proper form: =2,=2, lg, and One might think that case 3 applies, since is asymptotically larger than It is, however, not polynomially larger: ratio = lg is asymptotically less than for any positive constant Consequently, the recurrence falls into the gap between case 2 and case 3 MAT AADS, Fall Sep The recurrence of the running time of the simple divide-and-conquer algorithm for matrix multiplication ) = 82) ) Now =8,=2, and and so Since is polynomially larger than i.e., for =1 case 1 applies, and () = MAT AADS, Fall Sep

12 The recurrence of the running time of Strassen s algorithm ) = 72) ) Here we have =7,=2, and and so Recalling that 2.80 < lg7 < 2.81, we see that = O( ) for = 0.8 Again, case 1 applies, and we have the solution () = MAT AADS, Fall Sep Probabilistic Analysis and Randomized Algorithms We need to hire a new ofce assistant with the help of an employment agency that sends us a candidate each day We interview that person and then decide either to hire her or not We pay the agency a fee for each interview To actually hire an applicant is more costly We must re the current ofce assistant and pay a substantial hiring fee to the agency MAT AADS, Fall Sep

13 5.1 The hiring problem We are committed to having, at all times, the best possible person for the job After interviewing an applicant, if she is better qualied than the current assistant, we will re the current ofce assistant and hire the new applicant We are willing to pay the price of this strategy, but wish to estimate what that price will be MAT AADS, Fall Sep HIRE-ASSISTANT) 1. best 0 // candidate 0 is a leastqualied dummy candidate 2. for 1to 3. interview candidate 4. if candidate is better than best 5. best 6. hire candidate MAT AADS, Fall Sep

14 Interviewing has a low cost,, whereas hiring is expensive, costing Letting be the number of people hired, the total cost of the algorithm is ) No matter how many people we hire, we always interview candidates and thus always incur the cost Let us concentrate on analyzing, the hiring cost This quantity varies with each run of the algorithm MAT AADS, Fall Sep Worst-case analysis In the worst case, we hire every candidate that we interview candidates come in increasing order of quality we hire times, for a total hiring cost of We have no idea about the order in which the candidates arrive, nor do we have any control over this order It is natural to ask what we expect to happen in a typical or average case MAT AADS, Fall Sep

15 Probabilistic analysis We can assume that the applicants come in a random order Assume that we can compare any two candidates and decide the better qualied there is a total order on the candidates Rank each candidate with a unique number 1,, Use ) to denote the rank of applicant MAT AADS, Fall Sep A higher rank corresponds to a better qualied applicant Ordered list 1, 2,, ) is a permutation of the list 1,2,, The applicants come in a random order this list of ranks is equally likely to be any one of the! permutations of the numbers 1 through Alternatively, the ranks form a uniform random permutation; i.e., each of the possible! permutations appears with equal probability MAT AADS, Fall Sep

16 Randomized algorithms We call an algorithm randomized if its behavior is determined also by values produced by a random-number generator RANDOM A call to RANDOM returns an integer between and, inclusive, with each such integer being equally likely E.g., RANDOM(0,1) produces 0 with probability 1, 2and 1 with probability 12 MAT AADS, Fall Sep A call to RANDOM 3,7 returns either 3,4,5,6,7, each with probability 15 Each integer returned by RANDOM is independent of the integers returned on previous calls In analyzing the running time of a randomized algorithm, we take the expectation of the running time over the distribution of values returned by the RANDOM We refer to the running time of a randomized algorithm as an expected running time MAT AADS, Fall Sep

17 5.2 Indicator random variables Indicator random variables provide a convenient method for converting between probabilities and expectations Suppose we are given a sample space and an event Then the indicator random variable } associated with is dened as 1 ifoccurs 0 ifdoesnotoccur MAT AADS, Fall Sep Let us determine the expected number of heads that we obtain when ipping a fair coin Our sample space is = { }, with Pr = Pr = 12 We dene an indicator random variable, associated with the coin coming up heads () This variable counts the number of heads obtained in this ip = 1 ifhoccurs 0 iftoccurs MAT AADS, Fall Sep

18 Recall that E = Pr } The expected number of heads obtained in one ip of the coin is simply the expected value of our indicator variable E =E = 1 Pr +0Pr = 1 (12) + 0 (12) = 12 Thus the expected number of heads obtained by one ip of a fair coin is 12 MAT AADS, Fall Sep The expected value of an indicator random variable associated with an event is equal to the probability that occurs Lemma 5.1 Given a sample space and an event in, let }. Then E = Pr{} Proof By the denition of an indicator and the denition of expected value, we have = = 1 Pr + 0 Pr = Pr, where denotes, the complement of MAT AADS, Fall Sep

19 Compute the number of heads in coin ips Let be the indicator associated with the event in which the th ip comes up heads: {thethflipresultsintheevent} Let denote the total number of heads in the coin ips, so that = To compute the expected number of heads, take the expectation of both sides to obtain E =E MAT AADS, Fall Sep Linearity of expectation E =E +E makes the use of indicator random variables a powerful analytical technique it applies even when there is dependence among the variables We now can easily compute the expected number of heads: E =E = E 12 = 2 MAT AADS, Fall Sep

20 Analysis of the hiring problem Let be the random variable whose value number of times we hire a new assistant let be the indicator random variable associated with the event in which the th candidate is hired candidateishired = 1 ifcandidateishired 0 ifcandidateisnothired MAT AADS, Fall Sep By Lemma 5.1 E = Pr candidateishired and we must compute the probability that lines 5 6 of HIRE-ASSISTANT are executed Candidate is hired (line 6) exactly when she is better than each of candidates 1,, We assume that the candidates arrive in a random order: the rst candidates have appeared in a random order Any one of these rst candidates is equally likely to be the best-qualied so far MAT AADS, Fall Sep

21 Candidate has a probability of 1/ of being better qualied than candidates 1,,1and thus a probability of 1/ of being hired. By Lemma 5.1, we conclude that E Now we can compute E : E =E = E = = ln (1) by harmonic series = = ln (1) = 1/ MAT AADS, Fall Sep We interview people, but actually hire only approx. ln of them, on average The following lemma summarize this Lemma 5.2 Assuming that candidates come in random order, algorithm HIRE-ASSISTANT has an average-case total hiring cost of ln ) Proof Follows immediately from our denition of the hiring cost and the equation, which shows that the expected number of hires is approximately ln MAT AADS, Fall Sep

22 5.3 Randomized algorithms The algorithm above is deterministic The number of times we hire a new ofce assistant differs for different inputs Given the rank list = 1,2,3,4,5,6,7,8,9,10, a new assistant is always hired 10 times For the list of ranks = 10,9,8,7,6,5,4,3,2,1, a new assistant is hired only once = 5,2,1,8,4,7,10,9,3,6, a new assistant is hired three times MAT AADS, Fall Sep Consider an algorithm that rst permutes the input before determining the best candidate Now we randomize in the algorithm, not in the input distribution For input we cannot say how many times the best is updated it differs with each run Not even an enemy can produce a bad input, random permutation yields input order irrelevant The algorithm performs badly only if the randomnumber generator produces an unlucky permutation MAT AADS, Fall Sep

23 RANDOMIZED-HIRE-ASSISTANT) 1. randomly permute the list of candidates 2. best 0 // candidate 0 is a least-qualied // dummy candidate 3. for 1to 4. interview candidate 5. if candidate is better than candidate best 6. best 7. hire candidate MAT AADS, Fall Sep Randomly permuting arrays We assume that we are given an array which, w.l.o.g., contains the elements 1,2,, Produce a random permutation of the array Assign element ] a random priority ], and sort the elements according to priorities E.g., if our initial array is = 1,2,3,4 and we choose random priorities = 36,3,62,19, we would produce array = 2,4,1,3 We call this procedure PERMUTE-BY-SORTING MAT AADS, Fall Sep

24 PERMUTE-BY-SORTING) 1.. length 2. let [1..]be a new array 3. for 1to 4. RANDOM(1, ) 5. sort, using as sort keys We use a range of 1 to for random numbers to make it likely that all the priorities in are unique MAT AADS, Fall Sep It remains to prove that the procedure produces a uniform random permutation, It is equally likely to produce every permutation of the numbers 1 through Lemma 5.4 PERMUTE-BY-SORTING produces a uniform random permutation of the input, assuming that all priorities are distinct Proof See the book. MAT AADS, Fall Sep

25 It is better to permute the given array in place RANDOMIZE-IN-PLACE does so in ) time In its th iteration, it chooses the element ] randomly from among elements through ] After the th iteration, is never altered RANDOMIZE-IN-PLACE) 1 2 for 1to 3 swap with [RANDOM(, )] MAT AADS, Fall Sep A -permutation on a set of elements is a sequence containing of the elements, with no repetitions There are!! -permutations loop invariant: Just prior to the th iteration of the for loop of lines 2 3, for each possible 1)-permutation of the elements, the subarray [1.. 1] contains this 1)-permutation with probability +1!! MAT AADS, Fall Sep

26 Initialization: loop invariant trivially holds Maintenance: Consider a particular -permutation, and denote the elements in it by,, This permutation consists of an 1)- permutation,, followed by the value that the algorithm places in ] Let denote the event in which the rst 1) iterations have created the particular 1)- permutation,, in [1.. 1] MAT AADS, Fall Sep By the loop invariant, Pr = + 1!! Let be the event that th iteration puts in position ] The -permutation,, appears in 1.. precisely when both and occur Pr Pr Pr{ Pr =1 +1 because in line 3 the algorithm chooses randomly from the +1values in positions.. Pr = ! =!!! MAT AADS, Fall Sep

27 Termination: At termination, +1, and we have that the subarray 1.. is a given -permutation with probability !! = 0!! = 1! Thus, RANDOMIZE-IN-PLACE produces a uniform random permutation MAT AADS, Fall Sep