4.4 The recursion-tree method

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "4.4 The recursion-tree method"

Transcription

1 4.4 The recursion-tree method Let us see how a recursion tree would provide a good guess for the recurrence = 3 4 ) Start by nding an upper bound Floors and ceilings usually do not matter when solving recurrences Create a recursion tree for the recurrence = 3 4 having written out the implied constant coefcient > 0 MAT AADS, Fall Sep MAT AADS, Fall Sep

2 MAT AADS, Fall Sep Subproblem sizes decrease by a factor of 4 each time we go down one level => we eventually must reach a boundary condition Subproblem size for a node at depth is /4 It hits =1when /4 = 1or, equivalently, when = log Thus, the tree has log +1levels (at depths 0, 1, 2,, log ) MAT AADS, Fall Sep

3 Each level has 3 the nodes of level above the number of nodes at depth is 3 Subproblem sizes reduce by a factor of 4 for each level we go down, each node at depth, = 0,1,2,,log 1, has cost of 4 Multiplying, we see that the total cost over all nodes at depth is 3 4 ) = (3 16) The bottom level, at depth log, has 3 = nodes, each contributing cost (1), for a total cost of (1), which is ), since (1) is a constant ) MAT AADS, Fall Sep Add up the costs over all levels to determine the cost for the entire tree: 3 = ) 16 (3 16 = ) 1 (3 ) 16)1 By the value of a geometric (or exponential) series = 1 1 MAT AADS, Fall Sep

4 We can use an innite decreasing geometric series as an upper bound = 3 16 ) = (3 16) = ) MAT AADS, Fall Sep We have derived a guess of for our recurrence = 3 4 ) Root of the tree contributes to the total cost a constant fraction of the total cost The cost of the root dominates the total cost If is indeed an upper bound for the recurrence, then it must be a tight bound First recursive call contributes cost of ), must be a lower bound for the recurrence MAT AADS, Fall Sep

5 Use the substitution method to verify that the guess was correct, i.e., is an upper bound for the recurrence We want to show that for some constant >0 = ) = 3 16 as long as (16 13) MAT AADS, Fall Sep A more intricate example is the recurrence ) = /3) + (2/3) + ) represents the constant in the term Adding the values across the levels of the recursion tree yields value for every level The longest simple path from the root to a leaf is (23 (2 3) 1 (2 3) = 1when = log The height of the tree is log MAT AADS, Fall Sep

6 MAT AADS, Fall Sep We expect the solution to the recurrence to be number of levels cost of each level: log ) = lg ) Not every level contributes a cost of If this recursion tree were a complete binary tree of height log, there would be leaves 2 The cost of each leaf is a constant The total cost of all leaves would then be ) which, since log 2>1, is ( lg ) MAT AADS, Fall Sep

7 This recursion tree is not a complete binary tree, however, and has fewer than leaves As we go down from the root, more and more internal nodes are absent Consequently, levels toward the bottom contribute less than to the total cost we can use the substitution method to verify that lgis an upper bound for the solution to the recurrence MAT AADS, Fall Sep The master method Theorem 4.1 (Master theorem) Let and > 1,and ) a function, ) is dened on nonneg. integers by recurrence )+, where we means either or. ) has the following asymptotic bounds: MAT AADS, Fall Sep

8 1. If = ( ) for some constant > 0, then )= 2. If )=, then )= lg 3. If )= for some constant >0, and if ) ) for some constant <1 and all sufciently large, then ) = ) MAT AADS, Fall Sep In each of the three cases, we compare the function ) with the function The larger of the two determines the solution to the recurrence In case 1, the function is the larger, then the solution is )= In case 3, the function ) is the larger, then the solution is ) = In case 2, the two functions are the same size, we multiply by a logarithmic factor, and the solution is )= lg MAT AADS, Fall Sep

9 Be aware of some technicalities: In case 1, not only must ) be smaller than, it must be polynomially smaller ) must be asymptotically smaller than by a factor of for some >0 In case 3, not only must ) be larger than, it also must be polynomially larger and satisfy the regularity condition that () () MAT AADS, Fall Sep Using the master method ) = 93) We have =9,=3,)=, and thus ) Since ) = ), where =1, we can apply case 1 of the master theorem and conclude that the solution is ) MAT AADS, Fall Sep

10 ) = (2 3) +1 Here =1, = 3 2, )=1, and =1 Case 2 applies, since = (1), and thus the solution to the recurrence is (lg ) MAT AADS, Fall Sep () = ( 4) + lg We have =3,=4, lg, and ) Since ) = ), where 0.2, case 3 applies provided that the regularity condition holds for ) For sufciently large, we have that )= 3(4) lg(4) (34) lg = ) for = 34 By case 3, the solution is lg ) MAT AADS, Fall Sep

11 The master method does not apply to ) = 22) lg even if it seems to have the proper form: =2,=2, lg, and One might think that case 3 applies, since is asymptotically larger than It is, however, not polynomially larger: ratio = lg is asymptotically less than for any positive constant Consequently, the recurrence falls into the gap between case 2 and case 3 MAT AADS, Fall Sep The recurrence of the running time of the simple divide-and-conquer algorithm for matrix multiplication ) = 82) ) Now =8,=2, and and so Since is polynomially larger than i.e., for =1 case 1 applies, and () = MAT AADS, Fall Sep

12 The recurrence of the running time of Strassen s algorithm ) = 72) ) Here we have =7,=2, and and so Recalling that 2.80 < lg7 < 2.81, we see that = O( ) for = 0.8 Again, case 1 applies, and we have the solution () = MAT AADS, Fall Sep Probabilistic Analysis and Randomized Algorithms We need to hire a new ofce assistant with the help of an employment agency that sends us a candidate each day We interview that person and then decide either to hire her or not We pay the agency a fee for each interview To actually hire an applicant is more costly We must re the current ofce assistant and pay a substantial hiring fee to the agency MAT AADS, Fall Sep

13 5.1 The hiring problem We are committed to having, at all times, the best possible person for the job After interviewing an applicant, if she is better qualied than the current assistant, we will re the current ofce assistant and hire the new applicant We are willing to pay the price of this strategy, but wish to estimate what that price will be MAT AADS, Fall Sep HIRE-ASSISTANT) 1. best 0 // candidate 0 is a leastqualied dummy candidate 2. for 1to 3. interview candidate 4. if candidate is better than best 5. best 6. hire candidate MAT AADS, Fall Sep

14 Interviewing has a low cost,, whereas hiring is expensive, costing Letting be the number of people hired, the total cost of the algorithm is ) No matter how many people we hire, we always interview candidates and thus always incur the cost Let us concentrate on analyzing, the hiring cost This quantity varies with each run of the algorithm MAT AADS, Fall Sep Worst-case analysis In the worst case, we hire every candidate that we interview candidates come in increasing order of quality we hire times, for a total hiring cost of We have no idea about the order in which the candidates arrive, nor do we have any control over this order It is natural to ask what we expect to happen in a typical or average case MAT AADS, Fall Sep

15 Probabilistic analysis We can assume that the applicants come in a random order Assume that we can compare any two candidates and decide the better qualied there is a total order on the candidates Rank each candidate with a unique number 1,, Use ) to denote the rank of applicant MAT AADS, Fall Sep A higher rank corresponds to a better qualied applicant Ordered list 1, 2,, ) is a permutation of the list 1,2,, The applicants come in a random order this list of ranks is equally likely to be any one of the! permutations of the numbers 1 through Alternatively, the ranks form a uniform random permutation; i.e., each of the possible! permutations appears with equal probability MAT AADS, Fall Sep

16 Randomized algorithms We call an algorithm randomized if its behavior is determined also by values produced by a random-number generator RANDOM A call to RANDOM returns an integer between and, inclusive, with each such integer being equally likely E.g., RANDOM(0,1) produces 0 with probability 1, 2and 1 with probability 12 MAT AADS, Fall Sep A call to RANDOM 3,7 returns either 3,4,5,6,7, each with probability 15 Each integer returned by RANDOM is independent of the integers returned on previous calls In analyzing the running time of a randomized algorithm, we take the expectation of the running time over the distribution of values returned by the RANDOM We refer to the running time of a randomized algorithm as an expected running time MAT AADS, Fall Sep

17 5.2 Indicator random variables Indicator random variables provide a convenient method for converting between probabilities and expectations Suppose we are given a sample space and an event Then the indicator random variable } associated with is dened as 1 ifoccurs 0 ifdoesnotoccur MAT AADS, Fall Sep Let us determine the expected number of heads that we obtain when ipping a fair coin Our sample space is = { }, with Pr = Pr = 12 We dene an indicator random variable, associated with the coin coming up heads () This variable counts the number of heads obtained in this ip = 1 ifhoccurs 0 iftoccurs MAT AADS, Fall Sep

18 Recall that E = Pr } The expected number of heads obtained in one ip of the coin is simply the expected value of our indicator variable E =E = 1 Pr +0Pr = 1 (12) + 0 (12) = 12 Thus the expected number of heads obtained by one ip of a fair coin is 12 MAT AADS, Fall Sep The expected value of an indicator random variable associated with an event is equal to the probability that occurs Lemma 5.1 Given a sample space and an event in, let }. Then E = Pr{} Proof By the denition of an indicator and the denition of expected value, we have = = 1 Pr + 0 Pr = Pr, where denotes, the complement of MAT AADS, Fall Sep

19 Compute the number of heads in coin ips Let be the indicator associated with the event in which the th ip comes up heads: {thethflipresultsintheevent} Let denote the total number of heads in the coin ips, so that = To compute the expected number of heads, take the expectation of both sides to obtain E =E MAT AADS, Fall Sep Linearity of expectation E =E +E makes the use of indicator random variables a powerful analytical technique it applies even when there is dependence among the variables We now can easily compute the expected number of heads: E =E = E 12 = 2 MAT AADS, Fall Sep

20 Analysis of the hiring problem Let be the random variable whose value number of times we hire a new assistant let be the indicator random variable associated with the event in which the th candidate is hired candidateishired = 1 ifcandidateishired 0 ifcandidateisnothired MAT AADS, Fall Sep By Lemma 5.1 E = Pr candidateishired and we must compute the probability that lines 5 6 of HIRE-ASSISTANT are executed Candidate is hired (line 6) exactly when she is better than each of candidates 1,, We assume that the candidates arrive in a random order: the rst candidates have appeared in a random order Any one of these rst candidates is equally likely to be the best-qualied so far MAT AADS, Fall Sep

21 Candidate has a probability of 1/ of being better qualied than candidates 1,,1and thus a probability of 1/ of being hired. By Lemma 5.1, we conclude that E Now we can compute E : E =E = E = = ln (1) by harmonic series = = ln (1) = 1/ MAT AADS, Fall Sep We interview people, but actually hire only approx. ln of them, on average The following lemma summarize this Lemma 5.2 Assuming that candidates come in random order, algorithm HIRE-ASSISTANT has an average-case total hiring cost of ln ) Proof Follows immediately from our denition of the hiring cost and the equation, which shows that the expected number of hires is approximately ln MAT AADS, Fall Sep

22 5.3 Randomized algorithms The algorithm above is deterministic The number of times we hire a new ofce assistant differs for different inputs Given the rank list = 1,2,3,4,5,6,7,8,9,10, a new assistant is always hired 10 times For the list of ranks = 10,9,8,7,6,5,4,3,2,1, a new assistant is hired only once = 5,2,1,8,4,7,10,9,3,6, a new assistant is hired three times MAT AADS, Fall Sep Consider an algorithm that rst permutes the input before determining the best candidate Now we randomize in the algorithm, not in the input distribution For input we cannot say how many times the best is updated it differs with each run Not even an enemy can produce a bad input, random permutation yields input order irrelevant The algorithm performs badly only if the randomnumber generator produces an unlucky permutation MAT AADS, Fall Sep

23 RANDOMIZED-HIRE-ASSISTANT) 1. randomly permute the list of candidates 2. best 0 // candidate 0 is a least-qualied // dummy candidate 3. for 1to 4. interview candidate 5. if candidate is better than candidate best 6. best 7. hire candidate MAT AADS, Fall Sep Randomly permuting arrays We assume that we are given an array which, w.l.o.g., contains the elements 1,2,, Produce a random permutation of the array Assign element ] a random priority ], and sort the elements according to priorities E.g., if our initial array is = 1,2,3,4 and we choose random priorities = 36,3,62,19, we would produce array = 2,4,1,3 We call this procedure PERMUTE-BY-SORTING MAT AADS, Fall Sep

24 PERMUTE-BY-SORTING) 1.. length 2. let [1..]be a new array 3. for 1to 4. RANDOM(1, ) 5. sort, using as sort keys We use a range of 1 to for random numbers to make it likely that all the priorities in are unique MAT AADS, Fall Sep It remains to prove that the procedure produces a uniform random permutation, It is equally likely to produce every permutation of the numbers 1 through Lemma 5.4 PERMUTE-BY-SORTING produces a uniform random permutation of the input, assuming that all priorities are distinct Proof See the book. MAT AADS, Fall Sep

25 It is better to permute the given array in place RANDOMIZE-IN-PLACE does so in ) time In its th iteration, it chooses the element ] randomly from among elements through ] After the th iteration, is never altered RANDOMIZE-IN-PLACE) 1 2 for 1to 3 swap with [RANDOM(, )] MAT AADS, Fall Sep A -permutation on a set of elements is a sequence containing of the elements, with no repetitions There are!! -permutations loop invariant: Just prior to the th iteration of the for loop of lines 2 3, for each possible 1)-permutation of the elements, the subarray [1.. 1] contains this 1)-permutation with probability +1!! MAT AADS, Fall Sep

26 Initialization: loop invariant trivially holds Maintenance: Consider a particular -permutation, and denote the elements in it by,, This permutation consists of an 1)- permutation,, followed by the value that the algorithm places in ] Let denote the event in which the rst 1) iterations have created the particular 1)- permutation,, in [1.. 1] MAT AADS, Fall Sep By the loop invariant, Pr = + 1!! Let be the event that th iteration puts in position ] The -permutation,, appears in 1.. precisely when both and occur Pr Pr Pr{ Pr =1 +1 because in line 3 the algorithm chooses randomly from the +1values in positions.. Pr = ! =!!! MAT AADS, Fall Sep

27 Termination: At termination, +1, and we have that the subarray 1.. is a given -permutation with probability !! = 0!! = 1! Thus, RANDOMIZE-IN-PLACE produces a uniform random permutation MAT AADS, Fall Sep

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

More information

Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

More information

CMPS 102 Solutions to Homework 1

CMPS 102 Solutions to Homework 1 CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

More information

Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

More information

Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort).

Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort). Chapter 7: Quicksort Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort). The three steps of Quicksort are as follows:

More information

Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4

Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4 Divide d Conquer Textook Reading Chapters 4, 7 & 33.4 Overview Design principle Divide d conquer Proof technique Induction, induction, induction Analysis technique Recurrence relations Prolems Sorting

More information

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1. Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Quiz 1 Quiz 1 Do not open this quiz booklet until you are directed

More information

Binary Search. Search for x in a sorted array A.

Binary Search. Search for x in a sorted array A. Divide and Conquer A general paradigm for algorithm design; inspired by emperors and colonizers. Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3.

More information

CS 102: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4)

CS 102: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4) CS 10: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4) Problem 1. a. Consider the modified binary search algorithm so that it splits the input not into two sets of almost-equal sizes, but into three

More information

HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!

HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)! Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following

More information

Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

More information

2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines 2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

More information

Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1 Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

More information

Converting a Number from Decimal to Binary

Converting a Number from Decimal to Binary Converting a Number from Decimal to Binary Convert nonnegative integer in decimal format (base 10) into equivalent binary number (base 2) Rightmost bit of x Remainder of x after division by two Recursive

More information

Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

More information

14.1 Rent-or-buy problem

14.1 Rent-or-buy problem CS787: Advanced Algorithms Lecture 14: Online algorithms We now shift focus to a different kind of algorithmic problem where we need to perform some optimization without knowing the input in advance. Algorithms

More information

Divide-and-Conquer Algorithms Part Four

Divide-and-Conquer Algorithms Part Four Divide-and-Conquer Algorithms Part Four Announcements Problem Set 2 due right now. Can submit by Monday at 2:15PM using one late period. Problem Set 3 out, due July 22. Play around with divide-and-conquer

More information

Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

More information

, each of which contains a unique key value, say k i , R 2. such that k i equals K (or to determine that no such record exists in the collection).

, each of which contains a unique key value, say k i , R 2. such that k i equals K (or to determine that no such record exists in the collection). The Search Problem 1 Suppose we have a collection of records, say R 1, R 2,, R N, each of which contains a unique key value, say k i. Given a particular key value, K, the search problem is to locate the

More information

Faster deterministic integer factorisation

Faster deterministic integer factorisation David Harvey (joint work with Edgar Costa, NYU) University of New South Wales 25th October 2011 The obvious mathematical breakthrough would be the development of an easy way to factor large prime numbers

More information

Analysis of Algorithms I: Binary Search Trees

Analysis of Algorithms I: Binary Search Trees Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary

More information

A Note on Maximum Independent Sets in Rectangle Intersection Graphs

A Note on Maximum Independent Sets in Rectangle Intersection Graphs A Note on Maximum Independent Sets in Rectangle Intersection Graphs Timothy M. Chan School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1, Canada tmchan@uwaterloo.ca September 12,

More information

GRAPH THEORY LECTURE 4: TREES

GRAPH THEORY LECTURE 4: TREES GRAPH THEORY LECTURE 4: TREES Abstract. 3.1 presents some standard characterizations and properties of trees. 3.2 presents several different types of trees. 3.7 develops a counting method based on a bijection

More information

1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form 1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

More information

Format of Divide-and-Conquer algorithms:

Format of Divide-and-Conquer algorithms: CS 360: Data Structures and Algorithms Divide-and-Conquer (part 1) Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively

More information

Zeros of a Polynomial Function

Zeros of a Polynomial Function Zeros of a Polynomial Function An important consequence of the Factor Theorem is that finding the zeros of a polynomial is really the same thing as factoring it into linear factors. In this section we

More information

Data Structures Fibonacci Heaps, Amortized Analysis

Data Structures Fibonacci Heaps, Amortized Analysis Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure:

More information

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

More information

Approximation Algorithms

Approximation Algorithms Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

More information

MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1. MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

More information

The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)! The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >

More information

Divide And Conquer Algorithms

Divide And Conquer Algorithms CSE341T/CSE549T 09/10/2014 Lecture 5 Divide And Conquer Algorithms Recall in last lecture, we looked at one way of parallelizing matrix multiplication. At the end of the lecture, we saw the reduce SUM

More information

Quiz 1 Solutions. (a) T F The height of any binary search tree with n nodes is O(log n). Explain:

Quiz 1 Solutions. (a) T F The height of any binary search tree with n nodes is O(log n). Explain: Introduction to Algorithms March 9, 2011 Massachusetts Institute of Technology 6.006 Spring 2011 Professors Erik Demaine, Piotr Indyk, and Manolis Kellis Quiz 1 Solutions Problem 1. Quiz 1 Solutions True

More information

Lecture 5 - CPA security, Pseudorandom functions

Lecture 5 - CPA security, Pseudorandom functions Lecture 5 - CPA security, Pseudorandom functions Boaz Barak October 2, 2007 Reading Pages 82 93 and 221 225 of KL (sections 3.5, 3.6.1, 3.6.2 and 6.5). See also Goldreich (Vol I) for proof of PRF construction.

More information

9.2 Summation Notation

9.2 Summation Notation 9. Summation Notation 66 9. Summation Notation In the previous section, we introduced sequences and now we shall present notation and theorems concerning the sum of terms of a sequence. We begin with a

More information

8.1 Makespan Scheduling

8.1 Makespan Scheduling 600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programing: Min-Makespan and Bin Packing Date: 2/19/15 Scribe: Gabriel Kaptchuk 8.1 Makespan Scheduling Consider an instance

More information

Lecture 13 - Basic Number Theory.

Lecture 13 - Basic Number Theory. Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted

More information

Why? A central concept in Computer Science. Algorithms are ubiquitous.

Why? A central concept in Computer Science. Algorithms are ubiquitous. Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

More information

Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2) Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

More information

Analysis of Algorithms I: Optimal Binary Search Trees

Analysis of Algorithms I: Optimal Binary Search Trees Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search

More information

7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization 7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method

More information

Lecture 13: Factoring Integers

Lecture 13: Factoring Integers CS 880: Quantum Information Processing 0/4/0 Lecture 3: Factoring Integers Instructor: Dieter van Melkebeek Scribe: Mark Wellons In this lecture, we review order finding and use this to develop a method

More information

Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1

Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1 Recursion Slides by Christopher M Bourke Instructor: Berthe Y Choueiry Fall 007 Computer Science & Engineering 35 Introduction to Discrete Mathematics Sections 71-7 of Rosen cse35@cseunledu Recursive Algorithms

More information

Homework 5 Solutions

Homework 5 Solutions Homework 5 Solutions 4.2: 2: a. 321 = 256 + 64 + 1 = (01000001) 2 b. 1023 = 512 + 256 + 128 + 64 + 32 + 16 + 8 + 4 + 2 + 1 = (1111111111) 2. Note that this is 1 less than the next power of 2, 1024, which

More information

(x + a) n = x n + a Z n [x]. Proof. If n is prime then the map

(x + a) n = x n + a Z n [x]. Proof. If n is prime then the map 22. A quick primality test Prime numbers are one of the most basic objects in mathematics and one of the most basic questions is to decide which numbers are prime (a clearly related problem is to find

More information

Full and Complete Binary Trees

Full and Complete Binary Trees Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full

More information

Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * *

Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * * Binary Heaps A binary heap is another data structure. It implements a priority queue. Priority Queue has the following operations: isempty add (with priority) remove (highest priority) peek (at highest

More information

The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

More information

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015 CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

More information

Sequential Data Structures

Sequential Data Structures Sequential Data Structures In this lecture we introduce the basic data structures for storing sequences of objects. These data structures are based on arrays and linked lists, which you met in first year

More information

8.1 Min Degree Spanning Tree

8.1 Min Degree Spanning Tree CS880: Approximations Algorithms Scribe: Siddharth Barman Lecturer: Shuchi Chawla Topic: Min Degree Spanning Tree Date: 02/15/07 In this lecture we give a local search based algorithm for the Min Degree

More information

1.2 Solving a System of Linear Equations

1.2 Solving a System of Linear Equations 1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables

More information

Simplified Analyses of Randomized Algorithms for Searching, Sorting, and Selection

Simplified Analyses of Randomized Algorithms for Searching, Sorting, and Selection Simplified Analyses of Randomized Algorithms for Searching, Sorting, and Selection i Michael T. Goodrich Roberto Tamassia Dept. of Info. & Computer Science Dept. of Computer Science University of California

More information

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm. Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

More information

1/1 7/4 2/2 12/7 10/30 12/25

1/1 7/4 2/2 12/7 10/30 12/25 Binary Heaps A binary heap is dened to be a binary tree with a key in each node such that: 1. All leaves are on, at most, two adjacent levels. 2. All leaves on the lowest level occur to the left, and all

More information

Binary Search Trees. Data in each node. Larger than the data in its left child Smaller than the data in its right child

Binary Search Trees. Data in each node. Larger than the data in its left child Smaller than the data in its right child Binary Search Trees Data in each node Larger than the data in its left child Smaller than the data in its right child FIGURE 11-6 Arbitrary binary tree FIGURE 11-7 Binary search tree Data Structures Using

More information

Randomized algorithms

Randomized algorithms Randomized algorithms March 10, 2005 1 What are randomized algorithms? Algorithms which use random numbers to make decisions during the executions of the algorithm. Why would we want to do this?? Deterministic

More information

Math 55: Discrete Mathematics

Math 55: Discrete Mathematics Math 55: Discrete Mathematics UC Berkeley, Spring 2012 Homework # 9, due Wednesday, April 11 8.1.5 How many ways are there to pay a bill of 17 pesos using a currency with coins of values of 1 peso, 2 pesos,

More information

Lecture 2: Universality

Lecture 2: Universality CS 710: Complexity Theory 1/21/2010 Lecture 2: Universality Instructor: Dieter van Melkebeek Scribe: Tyson Williams In this lecture, we introduce the notion of a universal machine, develop efficient universal

More information

Offline sorting buffers on Line

Offline sorting buffers on Line Offline sorting buffers on Line Rohit Khandekar 1 and Vinayaka Pandit 2 1 University of Waterloo, ON, Canada. email: rkhandekar@gmail.com 2 IBM India Research Lab, New Delhi. email: pvinayak@in.ibm.com

More information

Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

More information

How Asymmetry Helps Load Balancing

How Asymmetry Helps Load Balancing How Asymmetry Helps oad Balancing Berthold Vöcking nternational Computer Science nstitute Berkeley, CA 947041198 voecking@icsiberkeleyedu Abstract This paper deals with balls and bins processes related

More information

CS473 - Algorithms I

CS473 - Algorithms I CS473 - Algorithms I Lecture 9 Sorting in Linear Time View in slide-show mode 1 How Fast Can We Sort? The algorithms we have seen so far: Based on comparison of elements We only care about the relative

More information

Lecture 22: November 10

Lecture 22: November 10 CS271 Randomness & Computation Fall 2011 Lecture 22: November 10 Lecturer: Alistair Sinclair Based on scribe notes by Rafael Frongillo Disclaimer: These notes have not been subjected to the usual scrutiny

More information

Lecture 10 Union-Find The union-nd data structure is motivated by Kruskal's minimum spanning tree algorithm (Algorithm 2.6), in which we needed two operations on disjoint sets of vertices: determine whether

More information

COMP 250 Fall 2012 lecture 2 binary representations Sept. 11, 2012

COMP 250 Fall 2012 lecture 2 binary representations Sept. 11, 2012 Binary numbers The reason humans represent numbers using decimal (the ten digits from 0,1,... 9) is that we have ten fingers. There is no other reason than that. There is nothing special otherwise about

More information

1 Introduction to Internet Content Distribution

1 Introduction to Internet Content Distribution OS 521: Advanced Algorithm Design Hashing / ontent Distribution 02/09/12 Scribe: Shilpa Nadimpalli Professor: Sanjeev Arora 1 Introduction to Internet ontent Distribution 1.1 The Hot-Spot Problem 1 ertain

More information

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

More information

by the matrix A results in a vector which is a reflection of the given

by the matrix A results in a vector which is a reflection of the given Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

More information

CS473 - Algorithms I

CS473 - Algorithms I CS473 - Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm View in slide-show mode 1 Reminder: Merge Sort Input array A sort this half sort this half Divide Conquer merge two sorted halves Combine

More information

Lecture 6. Inverse of Matrix

Lecture 6. Inverse of Matrix Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

More information

NODAL ANALYSIS. Circuits Nodal Analysis 1 M H Miller

NODAL ANALYSIS. Circuits Nodal Analysis 1 M H Miller NODAL ANALYSIS A branch of an electric circuit is a connection between two points in the circuit. In general a simple wire connection, i.e., a 'short-circuit', is not considered a branch since it is known

More information

Lecture Notes on Binary Search Trees

Lecture Notes on Binary Search Trees Lecture Notes on Binary Search Trees 15-122: Principles of Imperative Computation Frank Pfenning André Platzer Lecture 17 October 23, 2014 1 Introduction In this lecture, we will continue considering associative

More information

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory

More information

Binary Search Trees CMPSC 122

Binary Search Trees CMPSC 122 Binary Search Trees CMPSC 122 Note: This notes packet has significant overlap with the first set of trees notes I do in CMPSC 360, but goes into much greater depth on turning BSTs into pseudocode than

More information

Elementary Number Theory We begin with a bit of elementary number theory, which is concerned

Elementary Number Theory We begin with a bit of elementary number theory, which is concerned CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,

More information

Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation: CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

More information

Near Optimal Solutions

Near Optimal Solutions Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

More information

Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

More information

Chapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling

Chapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

More information

Lecture 6 Online and streaming algorithms for clustering

Lecture 6 Online and streaming algorithms for clustering CSE 291: Unsupervised learning Spring 2008 Lecture 6 Online and streaming algorithms for clustering 6.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

More information

CS2 Algorithms and Data Structures Note 11. Breadth-First Search and Shortest Paths

CS2 Algorithms and Data Structures Note 11. Breadth-First Search and Shortest Paths CS2 Algorithms and Data Structures Note 11 Breadth-First Search and Shortest Paths In this last lecture of the CS2 Algorithms and Data Structures thread we will consider the problem of computing distances

More information

Polynomials and the Fast Fourier Transform (FFT) Battle Plan

Polynomials and the Fast Fourier Transform (FFT) Battle Plan Polynomials and the Fast Fourier Transform (FFT) Algorithm Design and Analysis (Wee 7) 1 Polynomials Battle Plan Algorithms to add, multiply and evaluate polynomials Coefficient and point-value representation

More information

Chapter 8. Exponential and Logarithmic Functions

Chapter 8. Exponential and Logarithmic Functions Chapter 8 Exponential and Logarithmic Functions This unit defines and investigates exponential and logarithmic functions. We motivate exponential functions by their similarity to monomials as well as their

More information

1.1 Related work For non-preemptive scheduling on uniformly related machines the rst algorithm with a constant competitive ratio was given by Aspnes e

1.1 Related work For non-preemptive scheduling on uniformly related machines the rst algorithm with a constant competitive ratio was given by Aspnes e A Lower Bound for On-Line Scheduling on Uniformly Related Machines Leah Epstein Jir Sgall September 23, 1999 lea@math.tau.ac.il, Department of Computer Science, Tel-Aviv University, Israel; sgall@math.cas.cz,

More information

A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem

A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem John Karlof and Peter Hocking Mathematics and Statistics Department University of North Carolina Wilmington Wilmington,

More information

1 Solving LPs: The Simplex Algorithm of George Dantzig

1 Solving LPs: The Simplex Algorithm of George Dantzig Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

More information

Lecture Notes on Binary Search Trees

Lecture Notes on Binary Search Trees Lecture Notes on Binary Search Trees 15-122: Principles of Imperative Computation Frank Pfenning Lecture 17 March 17, 2010 1 Introduction In the previous two lectures we have seen how to exploit the structure

More information

SOLVING POLYNOMIAL EQUATIONS

SOLVING POLYNOMIAL EQUATIONS C SOLVING POLYNOMIAL EQUATIONS We will assume in this appendix that you know how to divide polynomials using long division and synthetic division. If you need to review those techniques, refer to an algebra

More information

Data Structure [Question Bank]

Data Structure [Question Bank] Unit I (Analysis of Algorithms) 1. What are algorithms and how they are useful? 2. Describe the factor on best algorithms depends on? 3. Differentiate: Correct & Incorrect Algorithms? 4. Write short note:

More information

Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.

Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable. Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

More information

CLASS 3, GIVEN ON 9/27/2010, FOR MATH 25, FALL 2010

CLASS 3, GIVEN ON 9/27/2010, FOR MATH 25, FALL 2010 CLASS 3, GIVEN ON 9/27/2010, FOR MATH 25, FALL 2010 1. Greatest common divisor Suppose a, b are two integers. If another integer d satisfies d a, d b, we call d a common divisor of a, b. Notice that as

More information

CHAPTER 2 Estimating Probabilities

CHAPTER 2 Estimating Probabilities CHAPTER 2 Estimating Probabilities Machine Learning Copyright c 2016. Tom M. Mitchell. All rights reserved. *DRAFT OF January 24, 2016* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This is a

More information

arxiv:1112.0829v1 [math.pr] 5 Dec 2011

arxiv:1112.0829v1 [math.pr] 5 Dec 2011 How Not to Win a Million Dollars: A Counterexample to a Conjecture of L. Breiman Thomas P. Hayes arxiv:1112.0829v1 [math.pr] 5 Dec 2011 Abstract Consider a gambling game in which we are allowed to repeatedly

More information

1 Approximating Set Cover

1 Approximating Set Cover CS 05: Algorithms (Grad) Feb 2-24, 2005 Approximating Set Cover. Definition An Instance (X, F ) of the set-covering problem consists of a finite set X and a family F of subset of X, such that every elemennt

More information

17.3.1 Follow the Perturbed Leader

17.3.1 Follow the Perturbed Leader CS787: Advanced Algorithms Topic: Online Learning Presenters: David He, Chris Hopman 17.3.1 Follow the Perturbed Leader 17.3.1.1 Prediction Problem Recall the prediction problem that we discussed in class.

More information

Guessing Game: NP-Complete?

Guessing Game: NP-Complete? Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple

More information

Introduction to Diophantine Equations

Introduction to Diophantine Equations Introduction to Diophantine Equations Tom Davis tomrdavis@earthlink.net http://www.geometer.org/mathcircles September, 2006 Abstract In this article we will only touch on a few tiny parts of the field

More information

The `Russian Roulette' problem: a probabilistic variant of the Josephus Problem

The `Russian Roulette' problem: a probabilistic variant of the Josephus Problem The `Russian Roulette' problem: a probabilistic variant of the Josephus Problem Joshua Brulé 2013 May 2 1 Introduction The classic Josephus problem describes n people arranged in a circle, such that every

More information

a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2. Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

More information