Computer Algorithms. Merge Sort. Analysis of Merge Sort. Recurrences. Recurrence Relations. Iteration Method. Lecture 6

Similar documents
Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Full and Complete Binary Trees

CS473 - Algorithms I

The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

the recursion-tree method

CSC148 Lecture 8. Algorithm Analysis Binary Search Sorting

CS473 - Algorithms I

Data Structures and Algorithms Written Examination

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Mathematical Induction. Lecture 10-11

Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

Algorithms. Margaret M. Fleck. 18 October 2010

Solutions of Equations in One Variable. Fixed-Point Iteration II

Section IV.1: Recursive Algorithms and Recursion Trees

Appendix: Solving Recurrences [Fa 10] Wil Wheaton: Embrace the dark side! Sheldon: That s not even from your franchise!

Analysis of Algorithms I: Binary Search Trees

3. Mathematical Induction

Why? A central concept in Computer Science. Algorithms are ubiquitous.

Lecture 1: Course overview, circuits, and formulas

HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!

Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called heap-order property

Lecture Notes on Polynomials

Find-The-Number. 1 Find-The-Number With Comps

Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Math 55: Discrete Mathematics

Closest Pair Problem

GRAPH THEORY LECTURE 4: TREES

TAKE-AWAY GAMES. ALLEN J. SCHWENK California Institute of Technology, Pasadena, California INTRODUCTION

Math 55: Discrete Mathematics

Scheduling a sequence of tasks with general completion costs

Lecture 6 Online and streaming algorithms for clustering

Sequential Data Structures

9.2 Summation Notation

Mathematical Induction. Mary Barnes Sue Gordon

Continued Fractions and the Euclidean Algorithm

Catalan Numbers. Thomas A. Dowling, Department of Mathematics, Ohio State Uni- versity.

Binary Search Trees CMPSC 122

Research Tools & Techniques

Approximation Algorithms

Dynamic Programming. Lecture Overview Introduction

Class Overview. CSE 326: Data Structures. Goals. Goals. Data Structures. Goals. Introduction

SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH

Indiana State Core Curriculum Standards updated 2009 Algebra I

Lecture 4 Online and streaming algorithms for clustering

csci 210: Data Structures Recursion

6.042/18.062J Mathematics for Computer Science December 12, 2006 Tom Leighton and Ronitt Rubinfeld. Random Walks

Elementary Number Theory and Methods of Proof. CSE 215, Foundations of Computer Science Stony Brook University

Algorithms Chapter 12 Binary Search Trees

FACTORING LARGE NUMBERS, A GREAT WAY TO SPEND A BIRTHDAY

In mathematics, there are four attainment targets: using and applying mathematics; number and algebra; shape, space and measures, and handling data.

Data Structures. Algorithm Performance and Big O Analysis

Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay

Sample Induction Proofs

From Binomial Trees to the Black-Scholes Option Pricing Formulas

If n is odd, then 3n + 7 is even.

1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D.

Collatz Sequence. Fibbonacci Sequence. n is even; Recurrence Relation: a n+1 = a n + a n 1.

Analysis of Algorithms I: Optimal Binary Search Trees

Discrete Mathematics: Homework 7 solution. Due:

Data Structure [Question Bank]

Converting a Number from Decimal to Binary

Activity 1: Using base ten blocks to model operations on decimals

Section 1.3 P 1 = 1 2. = P n = 1 P 3 = Continuing in this fashion, it should seem reasonable that, for any n = 1, 2, 3,..., =

Union-Find Problem. Using Arrays And Chains

Solutions to Homework 6

From Last Time: Remove (Delete) Operation

5.2 The Master Theorem

The Running Time of Programs

Analysis of Algorithms, I

Data Structures and Algorithms

Introduction. Appendix D Mathematical Induction D1

Answer Key for California State Standards: Algebra I

The following themes form the major topics of this chapter: The terms and concepts related to trees (Section 5.2).

A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem

Algorithm Design and Analysis Homework #1 Due: 5pm, Friday, October 4, 2013 TA === Homework submission instructions ===

Mathematics for Algorithm and System Analysis

Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the school year.

8.1 Min Degree Spanning Tree

Lecture Notes on Linear Search

8 Divisibility and prime numbers

Probability Generating Functions

6.3 Conditional Probability and Independence


Answer: (a) Since we cannot repeat men on the committee, and the order we select them in does not matter, ( )

Continued Fractions. Darren C. Collins

SCORE SETS IN ORIENTED GRAPHS

PUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include

26 Integers: Multiplication, Division, and Order

Chapter 3. Cartesian Products and Relations. 3.1 Cartesian Products

MATHEMATICAL INDUCTION. Mathematical Induction. This is a powerful method to prove properties of positive integers.

1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++

A Note on Maximum Independent Sets in Rectangle Intersection Graphs

Properties of Real Numbers

Data Structures Fibonacci Heaps, Amortized Analysis

IB Maths SL Sequence and Series Practice Problems Mr. W Name

Lectures 5-6: Taylor Series

Complexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar

Transcription:

Computer Algorithms Lecture 6 Merge Sort Sorting Problem: Sort a sequence of n elements into non-decreasing order. Divide: Divide the n-element sequence to be sorted into two subsequences of n/ elements each Conquer: Sort the two subsequences recursively using merge sort. Combine: Merge the two sorted subsequences to produce the sorted answer. Recurrence Ch. 4 (till Master Theorem) Some of these slides are courtesy of D. Plaisted et al, UNC and M. Nicolescu, UNR Analysis of Merge Sort Running time T(n) of Merge Sort: Divide: computing the middle takes () Conquer: solving subproblems takes T(n/) Combine: merging n elements takes (n) Total: T(n) = () if n = T(n) = T(n/) + (n) if n > T(n) = (n lg n) Recurrences Running time of algorithms with recursive calls can be described using recurrences A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs. For divide-and-conquer algorithms: solving_trivial_problem if n Tn ( ) num_pieces Tn ( / subproblem_size_factor) dividing combining if n Example: Merge Sort () if n Tn ( ) Tn ( /) ( n) if n 3 4 Recurrence Relations Equation or an inequality that characterizes a function by its values on smaller inputs. Solution Methods (Chapter 4) Iteration Method Substitution Method. Recursion-tree Method. Master Method. Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Ex: Divide and Conquer. T(n) = () if n c T(n) = a T(n/b) + D(n) + C(n) otherwise Iteration Method T(n) = c + T(n/) T(n) = c + T(n/) T(n/) = c + T(n/4) = c + c + T(n/4) T(n/4) = c + T(n/8) = c + c + c + T(n/8) Assume n = k T(n) = c + c + + c + k times = clgn + = Θ(lgn) 5 6

Iteration Method Example T(n) = n + T(n/) Assume: n = k T(n) = n + T(n/) T(n/) = n/ + T(n/4) T(n) = n + T(n-) Iteration Method Example T(n) = n + T(n-) 7 8 Substitution method Substitution method Two steps Guess the form of the solution Use mathematical induction to find the constants and show the solution works Useful when it is easy to guess the form of the answer Can be used to establish either upper or lower bound on a recurrence Example: determine upper bound for T ( n) T ( n / ) n guess: T(n) = O(n lg n) prove: T(n) lg n for some c Guess a solution T(n) = O(g(n)) Induction goal: apply the definition of the asymptotic notation T(n) d g(n), for some d > 0 and n n 0 Induction hypothesis: T(k) d g(k) for all k < n Prove the induction goal Use the induction hypothesis to find some values of the constants d and n 0 for which the induction goal holds 9 0 Example Exact Function Recurrence: T(n) = if n = T(n) = T(n/) + n if n > Guess: T(n) = n lg n + n. Induction: Basis: n = n lgn + n = = T(n). Hypothesis: n = k/: T(k/) = (k/) lg (k/) + (k/) Inductive Step: n = k: T(k) = T(k /) + k = ((k /)lg(k /) + (k /)) + k = k (lg(k /)) + k = k lg k k + k = k lg k + k Example T(n) = T(n-) + n Guess: T(n) = O(n ) Induction goal: T(n) c n, for some c and n n 0 Induction hypothesis: T(n-) c(n-) Proof of induction goal: T(n) = T(n-) + n c (n-) + n

Substitution Method. Summary Making a good guess Guess the form of the solution, then use mathematical induction to show it correct. Substitute guessed answer for the function when the inductive hypothesis is applied to smaller values hence, the name. Works well when the solution is easy to guess. No general way to guess the correct solution. Need experience and creativity Use recursion trees to generate good guesses If a recurrence is similar to one you are familiar, then guessing a similar solution is reasonable T ( n) T ( n / 7) T(n) = O(n lg n) Why? n 3 4 Recursion-tree Method Example Making a good guess is sometimes difficult with the substitution method. Use recursion trees to devise good guesses. Convert the recurrence into a tree: Each node represents the cost incurred at that level of recursion Sum up the costs of all levels Recursion Trees Show successive expansions of recurrences using trees. Keep track of the time spent on the subproblems of a divide and conquer algorithm. Help organize the algebraic bookkeeping necessary to solve a recurrence. Running time of Merge Sort: T(n) = () if n = T(n) = T(n/) + (n) if n > Rewrite the recurrence as T(n) = c if n = T(n) = T(n/) + if n > c > 0: Running time for the base case and time per array element for the divide and combine steps. 5 6 Recursion Tree for Merge Sort For the original problem, we have a cost of, plus two subproblems each of size (n/) and running time T(n/). Cost of divide and merge. Each of the size n/ problems has a cost of / plus two subproblems, each costing T(n/4). / / Recursion Tree for Merge Sort Continue expanding until the problem size reduces to. / / lg n /4 /4 /4 /4 T(n/) T(n/) Cost of sorting subproblems. T(n/4) T(n/4) T(n/4) T(n/4) 7 c c c c c c Total : lgn+ 8 3

Recursion Tree for Merge Sort Continue expanding until the problem size reduces to. / / Each level has total cost. Each time we go down one level, the number of subproblems doubles, but the cost per subproblem halves cost per level remains the same. T(n) = T(n/) + n T(n/) T(n/) T(n/4) T(n/4) T(n/4) T(n/4) T(n/)=(T/4)+(n/) T(n/4)=(T/8)+(n/4) /4 /4 /4 /4 c c c c c c There are lg n + levels, height is lg n. (Assuming n is a power of.) Can be proved by induction. Total cost = sum of costs at each level = (lg n + ) = lgn + = (n lgn). 9 Subproblem size at level i is: n/ i Subproblem size hits when = n/ i i= lgn Cost of the problem at level i = (n/ i ) No. of nodes at level i = i lgn lgn i i n lgn T ( n) T () n n n O( n) n O( n) n i T(n) = O(n ) 0 T(n) = 3T(n/4) + T(n) = T(n/3) + T(n/3) + n The longest path from the root to a leaf is: n (/3)n (/3) n Subproblem size hits when =(/3) i n i=log 3/ n Subproblem size at level i is: n/4 i Subproblem size hits when = n/4 i i= log 4 n Cost of a node at level i =? Number of nodes at level i =? Cost of the problem at level i =? T(n) =? T(n) =? Other Examples Use the recursion-tree method to determine a guess for the recurrences T(n) = 3T( n/4 ) + (n ). T(n) =T(n/3) + T(n/3) + O(n). Recursion Trees Caution Note Recursion trees only generate guesses. Verify guesses using substitution method. A small amount of sloppiness can be tolerated. Why? If careful when drawing out a recursion tree and summing the costs, can be used as direct proof. 3 4 4

Recursion-Tree Method. Summary Recursion tree Each node represents the cost of a single subproblem somewhere in the set of recursive function invocations Each level denotes iteration Sum the costs within each level of the tree to obtain a set of per-level costs Sum all the per-level costs to determine the total cost of all levels of the recursion Useful when the recurrence describes the running time of a divide-and- conquer algorithm Useful for generating a good guess, which is then verified by the substitution method Use the recursion-tree method to guess the solution (rather than for proving) Example Recurrences T(n) = T(n-) + n Θ(n ) Recursive algorithm that loops through the input to eliminate one item T(n) = T(n/) + c Θ(lgn) Recursive algorithm that halves the input in one step T(n) = T(n/) + n Θ(n) Recursive algorithm that halves the input but must examine every item in the input T(n) = T(n/) + Θ(n) Recursive algorithm that splits the input into halves and does a constant amount of other work 5 6 Three methods for solving recurrences (obtaining asymptotic Θ or O bounds on the solution) Substitution method Recursion-tree method Master method Notes some technical details are neglected Assumption of integer arguments to functions: T(n) Boundary condition for small n T(n) is constant for small n Floors, ceilings Mathematical induction is used to prove our solution for recurrence Showing base case holds Assume the nth step is true Prove the (n+)th step by the result before the nth step 7 5