Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4"

Transcription

1 Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we can use to design algorithms. One is an incremental approach, of the sort of approach where we start with a solution for a single element problem and figure out ways, one increment at a time, to build the solution to the next largest problem from the prior solution. Insertion sort fits this strategy. Question: Why? Because at each step, we have already we insert into its place,, and we get out. Another common strategy is called divide-and-conquer. It is for situations that are naturally recursive and has three components: Divide Conquer Combine This is one of three major algorithm design strategies we ll study in this course. The other two are greedy algorithms (you got a taste of that in 360 with minimum spanning trees) and dynamic programming. II. Merge Sort We ve been discussing the merge sort algorithm informally all along. In this lesson, we ll focus on merge sort and use it as an introduction to the important ideas of analyzing divide-and-conquer algorithms. We looked at merge sort on the first day. The idea is to sort a subarray A[p...r], where we start with p = and r =. However, the algorithm is recursive, and at each level, the values of p and r change. Here s the high-level overview of merge sort: 1. Divide the array into two subarrays and, where q is the midpoint of p and r 2. Conquer by recursively sorting the two subarrays. 3. Combine by the two sorted subarrays A[p..q] and A[q+1..r] to produce a single sorted subarray. Question: All recursive definitions and algorithms need a base case. What s the base case here? Page 1 of 9 Prepared by D. Hogan referencing CLRS - Introduction to Algorithms (3rd ed.) for PSU CMPSC 465

2 Here s the pseudocode for merge sort from CLRS: MERGE-SORT(A, p, r) if p < r // check for base case q = ( p + r) / 2 // divide MERGE-SORT(A, p, q) MERGE-SORT(A, q + 1, r) MERGE(A, p, q, r) Let s trace the algorithm a few times. // conquer // conquer // combine Example: Trace MERGE-SORT for the following initial array: index value Example: Trace merge sort for the following initial array: index value Page 2 of 9 Prepared by D. Hogan referencing CLRS - Introduction to Algorithms (3rd ed.) for PSU CMPSC 465

3 III. Merging in Linear Time The concept of merging is relatively intuitive, but implementing it or expressing it in pseudocode isn t quite as straightforward. The problem s input and output are the following: Input with preconditions: Array A and indices p, q, r s.t. p q < r, subarray A[p..q] is sorted, and subarray A[q+1..r] is sorted Output/postcondition: Subarrays A[p..q] and A[q+1..r] have been merged into a single sorted subarray in A[p..r] Last week, we quickly discussed why this can be done in Θ(n) time, but let s look a little more carefully here. Let s visualize this as two piles of cards, where each pile is sorted and we can see the smallest card from each pile on top. We ll have the input piles face up. We want to merge those cards into a single sorted pile, face down. Let s act this out, but you need to count as we go. I ll leave you space to take notes and keep count So, Each basic step was to and move it to an output pile, exposing a smaller card. We repeatedly perform basic steps until. Then, we finish by. Each basic step takes time. What s the maximum number of basic steps?. What s the running time? The CLRS version does empty pile detection via a sentinel card, whose value is guaranteed to lose in a comparison to any value. So, works for this, and you ll see it in the pseudocode. In this mindset Given that the input array is indexed from p to r, there are exactly non-sentinel cards. So, we can use this to stop the algorithm after basic steps. Thus we can just fill up the output array from index p through index r. Page 3 of 9 Prepared by D. Hogan referencing CLRS - Introduction to Algorithms (3rd ed.) for PSU CMPSC 465

4 Here s the pseudocode for merging from CLRS: Example: Suppose array A is as below and trace MERGE(A, 9, 12, 16): index value Page 4 of 9 Prepared by D. Hogan referencing CLRS - Introduction to Algorithms (3rd ed.) for PSU CMPSC 465

5 IV. The Correctness of the Merge Algorithm Let s study the correctness of the Merge algorithm via proving the correctness of the last loop, which is the heart of the algorithm as far as proving correctness goes: Loop Invariant: Correctness Proof: Initialization: Maintenance: Termination: Page 5 of 9 Prepared by D. Hogan referencing CLRS - Introduction to Algorithms (3rd ed.) for PSU CMPSC 465

6 V. Back to Recurrences The key tool to analyze divide-and-conquer algorithms is the recurrence. We can use a recurrence to describe the running time of a divide-and-conquer algorithm very naturally, because of the recursion in the algorithm. In this world, we use the following conventions: We let T (n) be a function for the running time for a problem of size n. A base case corresponds to a small enough problem size and we use a simple or brute force strategy we say takes Θ(1) time. The recursive case is where we divide a problem into a subproblems, each 1/b the size of the original. We express the time to divide a size n problem as D(n). We express the time to combine the solutions to the subproblems as C(n). So, using these conventions, a general form of a recurrence for the running time of a divide-and-conquer algorithm is As always, our goal with a recurrence is to solve it for a closed form, i.e. a formula that determines the same sequence. Expanding upon the list we have from discrete math, there are several methods for solving recurrences: 1. Iteration: Start with the recurrence and keep applying the recurrence equation until we get a pattern. (The result is a guess at the closed form.) 2. Substitution: Guess the solution; prove it using induction. (The result here is a proven closed form. It's often difficult to come up the guess blindly though, so, in practice, we need another strategy to guess the closed form.) 3. Recursion Tree: Draw a tree that illustrates the decomposition of a problem of size n into subproblems and tally the costs of each level of the tree. Use this to find a closed form. (Like iteration, this is really a guess at the closed form.) 4. Master Theorem: Plugging into a formula that gives an approximate bound on the solution. (The result here is only a bound on the closed form. It is not an exact solution. For many of our purposes, that s good enough.) We ll look at the new techniques in depth in this chapter. Here are some other issues that are specific to the use of recurrences in analyzing algorithms: Floors and ceilings: Expressing boundary conditions: Asymptotic notation: Page 6 of 9 Prepared by D. Hogan referencing CLRS - Introduction to Algorithms (3rd ed.) for PSU CMPSC 465

7 VI. The Merge Sort Recurrence In Epp 11.5, we derived a merge sort recurrence. We ll simplify it a bit here and state it in the same symbols as above. Some notes: The base case occurs when n = 1. For n 2, o The divide step is a computation of the average of p and q and takes Θ(1) o The conquer step is to solve two recursive subproblems, each of size n/2 o The combine step is to merge an n element array, which takes Θ(n) time So the recurrence is: Θ(1) for n = 1 T (n) = 2T (n / 2) + Θ(n) for n > 1 as Θ(n) + Θ(1) = Θ(n). VII. Analysis via Recursion Tree Let s simplify the recurrence as follows, using c as a constant (which, in general is not the same, but works for this case and keeps the analysis clean): T (n) = c for n = 1 2T (n / 2) + cn for n > 1 While we could solve this recurrence with our old iteration strategy, we ll use a new strategy called a recursion tree, which visually represents the recursion and is less prone to errors. At each step: The root of the recursion tree (or subtree) is the cost of dividing and combining at that step. We branch to children, with one child for each subproblem. o o Initially, we label the nodes as the recurrence function value. Then, we apply another round of recursion, repeating the whole process and replacing the roots with their costs as described. We continue this expansion until problem sizes get to the base case, or 1. Let s draw the first step of the recursion tree for merge sort: Let s draw the recursion tree after the first two steps: Page 7 of 9 Prepared by D. Hogan referencing CLRS - Introduction to Algorithms (3rd ed.) for PSU CMPSC 465

8 Now let s draw the complete recursion tree: Now, we must analyze the tree s cost at each level: What is the cost of the top level? What is the cost per subproblem on the second level? How many subproblems? Second level cost? Third level cost? Cost per level? Next, What is the height of the tree? How many levels does the tree have? (we ll prove this momentarily) What is the total cost of the tree? What is the merge sort running time? Page 8 of 9 Prepared by D. Hogan referencing CLRS - Introduction to Algorithms (3rd ed.) for PSU CMPSC 465

9 Finally, let s be clean in our analysis and prove the claim that there are lg n + 1 levels, for problem sizes that are powers of 2, inductively: Homework: CLRS Exercises 2.3-1, Page 9 of 9 Prepared by D. Hogan referencing CLRS - Introduction to Algorithms (3rd ed.) for PSU CMPSC 465

Merge Sort and Recurrences. COSC 581, Algorithms January 14, 2014

Merge Sort and Recurrences. COSC 581, Algorithms January 14, 2014 Merge Sort and Recurrences COSC 581, Algorithms January 14, 2014 Reading Assignments Today s class: Chapter 2, 4.0, 4.4 Reading assignment for next class: Chapter 4.2, 4.5 3 Common Algorithmic Techniques

More information

Quicksort CMPSC 465 Related to CLRS Chapter 7

Quicksort CMPSC 465 Related to CLRS Chapter 7 Quicksort CMPSC 465 Related to CLRS Chapter 7 I. High-Level Overview The heart of the quicksort algorithm is to partition an array into three regions: Then, recursively, we quicksort both of the subarrays

More information

Computer Algorithms. Merge Sort. Analysis of Merge Sort. Recurrences. Recurrence Relations. Iteration Method. Lecture 6

Computer Algorithms. Merge Sort. Analysis of Merge Sort. Recurrences. Recurrence Relations. Iteration Method. Lecture 6 Computer Algorithms Lecture 6 Merge Sort Sorting Problem: Sort a sequence of n elements into non-decreasing order. Divide: Divide the n-element sequence to be sorted into two subsequences of n/ elements

More information

CS420/520 Algorithm Analysis Spring 2010 Lecture 04

CS420/520 Algorithm Analysis Spring 2010 Lecture 04 CS420/520 Algorithm Analysis Spring 2010 Lecture 04 I. Chapter 4 Recurrences A. Introduction 1. Three steps applied at each level of a recursion a) Divide into a number of subproblems that are smaller

More information

1 Recursion and the divide-and-conquer approach

1 Recursion and the divide-and-conquer approach CS 7B - Spring 2014 - Assignment 4. 4/17/14 Write responses on paper and submit programs by email. 1 Recursion and the divide-and-conquer approach Many useful algorithms are recursive in structure: to

More information

Binary Search Trees CMPSC 122

Binary Search Trees CMPSC 122 Binary Search Trees CMPSC 122 Note: This notes packet has significant overlap with the first set of trees notes I do in CMPSC 360, but goes into much greater depth on turning BSTs into pseudocode than

More information

Steven Skiena. skiena

Steven Skiena.  skiena Lecture 3: Recurrence Relations (1997) Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Argue the solution to T(n) =

More information

CS2223 Algorithms B Term 2013 Exam 2 Solutions

CS2223 Algorithms B Term 2013 Exam 2 Solutions CS2223 Algorithms B Term 2013 Exam 2 Solutions Dec. 3, 2013 By Prof. Carolina Ruiz Dept. of Computer Science WPI PROBLEM 1: Solving Recurrences (15 points) Solve the recurrence T (n) = 2T (n/3) + n using

More information

The Selection Problem

The Selection Problem The Selection Problem The selection problem: given an integer k and a list x 1,..., x n of n elements find the k-th smallest element in the list Example: the 3rd smallest element of the following list

More information

Divide and Conquer My T. UF

Divide and Conquer My T. UF Introduction to Algorithms Divide and Conquer @ UF General Approach Divide the problem into a number of subproblems that are smaller instances of the same problem Conquer the subproblems by solving them

More information

Problem Set #1 Solutions

Problem Set #1 Solutions CS161: Design and Analysis of Algorithms Summer 2004 Problem Set #1 Solutions General Notes Regrade Policy: If you believe an error has been made in the grading of your problem set, you may resubmit it

More information

Introduction to Algorithms

Introduction to Algorithms Introduction to Algorithms (2 nd edition) by Cormen, Leiserson, Rivest & Stein Chapter 4: Recurrences (slides enhanced by N. Adlai A. DePano) Overview Define what a recurrence is Discuss three methods

More information

Recurrences (CLRS )

Recurrences (CLRS ) Recurrences (CLRS 4.1-4.2) Last time we discussed divide-and-conquer algorithms Divide and Conquer To Solve P: 1. Divide P into smaller problems P 1,P 2,P 3...P k. 2. Conquer by solving the (smaller) subproblems

More information

Michal Forišek: Early beta verzia skrípt z ADŠ

Michal Forišek: Early beta verzia skrípt z ADŠ Časová zložitosť II Michal Forišek: Early beta verzia skrípt z ADŠ In this part of the article we will focus on estimating the time complexity for recursive programs. In essence, this will lead to finding

More information

The divide and conquer strategy has three basic parts. For a given problem of size n,

The divide and conquer strategy has three basic parts. For a given problem of size n, 1 Divide & Conquer One strategy for designing efficient algorithms is the divide and conquer approach, which is also called, more simply, a recursive approach. The analysis of recursive algorithms often

More information

T(n)=1 for sufficiently small n

T(n)=1 for sufficiently small n Chapter 4. Recurrences Outline Offers three methods for solving recurrences, that is for obtaining asymptotic bounds on the solution In the substitution method,, we guess a bound and then use mathematical

More information

Pseudocode Analysis. COMP3600/6466 Algorithms Lecture 5. Recursive Algorithms. Asymptotic Bounds for Recursions

Pseudocode Analysis. COMP3600/6466 Algorithms Lecture 5. Recursive Algorithms. Asymptotic Bounds for Recursions COMP600/6466 Algorithms Lecture 5 Pseudocode Analysis Iterative Recursive S 0 Dr. Hassan Hijazi Prof. Weifa Liang Recursive Algorithms Asymptotic Bounds for Recursions Total running time = Sum of times

More information

Algorithms complexity of recursive algorithms. Jiří Vyskočil, Marko Genyk-Berezovskyj

Algorithms complexity of recursive algorithms. Jiří Vyskočil, Marko Genyk-Berezovskyj complexity of recursive algorithms Jiří Vyskočil, Marko Genyk-Berezovskyj 2010-2014 Recurrences A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs.

More information

Divide-and-Conquer Algorithms Part One

Divide-and-Conquer Algorithms Part One Divide-and-Conquer Algorithms Part One Announcements Problem Set One completely due right now. Solutions distributed at the end of lecture. Programming section today in Gates B08 from from 3:45PM 5:00PM.

More information

Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

More information

Recurrence Relations

Recurrence Relations Recurrence Relations Introduction Determining the running time of a recursive algorithm often requires one to determine the big-o growth of a function T (n) that is defined in terms of a recurrence relation.

More information

Data Structures and Algorithms Week 3

Data Structures and Algorithms Week 3 Data Structures and Algorithms Week 3 1. Divide and conquer 2. Merge sort, repeated substitutions 3. Tiling 4. Recurrences Recurrences Running times of algorithms with recursive calls can be described

More information

Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort).

Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort). Chapter 7: Quicksort Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort). The three steps of Quicksort are as follows:

More information

Design and Analysis of Algorithms

Design and Analysis of Algorithms Design and Analysis of Algorithms CSE 5311 Lecture 3 Divide-and-Conquer Junzhou Huang, Ph.D. Department of Computer Science and Engineering CSE5311 Design and Analysis of Algorithms 1 Reviewing: Θ-notation

More information

COP 4531 Complexity & Analysis of Data Structures & Algorithms

COP 4531 Complexity & Analysis of Data Structures & Algorithms COP 4531 Complexity & Analysis of Data Structures & Algorithms Lecture 3 Recurrences and Divide and Conquer Thanks to people who contributed to these slides including the text authors and Piyush Kumar,

More information

4.4 The recursion-tree method

4.4 The recursion-tree method 4.4 The recursion-tree method Let us see how a recursion tree would provide a good guess for the recurrence = 3 4 ) Start by nding an upper bound Floors and ceilings usually do not matter when solving

More information

Answers to Homework 1

Answers to Homework 1 Answers to Homework 1 p. 13 1.2 3 What is the smallest value of n such that an algorithm whose running time is 100n 2 runs faster than an algorithm whose running time is 2 n on the same machine? Find the

More information

CSE 5311 Homework 2 Solution

CSE 5311 Homework 2 Solution CSE 5311 Homework 2 Solution Problem 6.2-6 Show that the worst-case running time of MAX-HEAPIFY on a heap of size n is Ω(lg n). (Hint: For a heap with n nodes, give node values that cause MAX- HEAPIFY

More information

CMPS 102 Solutions to Homework 1

CMPS 102 Solutions to Homework 1 CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

More information

Solving recurrences. CS 4407, Algorithms University College Cork, Gregory M. Provan

Solving recurrences. CS 4407, Algorithms University College Cork, Gregory M. Provan Solving recurrences The analysis of divide and conquer algorithms require us to solve a recurrence. Recurrences are a major tool for analysis of algorithms MergeSort A L G O R I T H M S A L G O R I T H

More information

CSC 344 Algorithms and Complexity

CSC 344 Algorithms and Complexity CSC 344 Algorithms and Complexity Lecture #10 Recurrences Recurrence Relations Overview Connection to recursive algorithms Techniques for solving them Methods for generating a guess Induction proofs Master

More information

6.042/18.062J Mathematics for Computer Science October 24, 2006 Tom Leighton and Ronitt Rubinfeld. Recurrences I

6.042/18.062J Mathematics for Computer Science October 24, 2006 Tom Leighton and Ronitt Rubinfeld. Recurrences I 6.042/8.062J Mathematics for Computer Science October 24, 2006 Tom Leighton and Ronitt Rubinfeld Lecture Notes Recurrences I This is the first of two lectures about solving recurrences and recurrent problems.

More information

Chapters 1 and 5.2 of GT

Chapters 1 and 5.2 of GT Subject 2 Spring 2014 Growth of Functions and Recurrence Relations Handouts 3 and 4 contain more examples;the Formulae Collection handout might also help. Chapters 1 and 5.2 of GT Disclaimer: These abbreviated

More information

Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4

Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4 Divide d Conquer Textook Reading Chapters 4, 7 & 33.4 Overview Design principle Divide d conquer Proof technique Induction, induction, induction Analysis technique Recurrence relations Prolems Sorting

More information

How to prove an algorithm is correct?

How to prove an algorithm is correct? How to prove an algorithm is correct? To prove the incorrectness of an algorithm, one counter-example is enough. Proving the correctness of an algorithm is similar to proving a mathematical theorem; fundamentally,

More information

CS 4310 HOMEWORK SET 1

CS 4310 HOMEWORK SET 1 CS 4310 HOMEWORK SET 1 PAUL MILLER Section 2.1 Exercises Exercise 2.1-1. Using Figure 2.2 as a model, illustrate the operation of INSERTION-SORT on the array A = 31, 41, 59, 26, 41, 58. Solution: Assume

More information

Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1

Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1 Recursion Slides by Christopher M Bourke Instructor: Berthe Y Choueiry Fall 007 Computer Science & Engineering 35 Introduction to Discrete Mathematics Sections 71-7 of Rosen cse35@cseunledu Recursive Algorithms

More information

Objective. - mathematical induction, recursive definitions - arithmetic manipulations, series, products

Objective. - mathematical induction, recursive definitions - arithmetic manipulations, series, products Recurrences Objective running time as recursive function solve recurrence for order of growth method: substitution method: iteration/recursion tree method: MASTER method prerequisite: - mathematical induction,

More information

Quick Sort. As the name implies, it is quick, and it is the algorithm generally preferred for sorting. Quick Sort 1

Quick Sort. As the name implies, it is quick, and it is the algorithm generally preferred for sorting. Quick Sort 1 Quick Sort As the name implies, it is quick, and it is the algorithm generally preferred for sorting. Quick Sort 1 Basic Ideas (Another divide-and-conquer algorithm) Pick an element, say P (the pivot)

More information

CS 561, Lecture 3 - Recurrences. Jared Saia University of New Mexico

CS 561, Lecture 3 - Recurrences. Jared Saia University of New Mexico CS 561, Lecture 3 - Recurrences Jared Saia University of New Mexico Recurrence Relations Oh how should I not lust after eternity and after the nuptial ring of rings, the ring of recurrence - Friedrich

More information

6.046J/18.401J LECTURE

6.046J/18.401J LECTURE Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation O-, Ω-, and Θ-notation Recurrences Substitution method Iterating the recurrence Recursion tree Master method Prof. Erik Demaine September

More information

Divide And Conquer Algorithms

Divide And Conquer Algorithms CSE341T/CSE549T 09/10/2014 Lecture 5 Divide And Conquer Algorithms Recall in last lecture, we looked at one way of parallelizing matrix multiplication. At the end of the lecture, we saw the reduce SUM

More information

Algorithms Chapter 4 Recurrences

Algorithms Chapter 4 Recurrences Algorithms Chapter 4 Recurrences Instructor: Ching Chi Lin 林清池助理教授 chingchi.lin@gmail.com Department of Computer Science and Engineering National Taiwan Ocean University Outline The substitution method

More information

If we want to measure the amount of storage that an algorithm uses as a function of the size of the instances, there is a natural unit available Bit.

If we want to measure the amount of storage that an algorithm uses as a function of the size of the instances, there is a natural unit available Bit. (1) Explain why analysis of algorithms is important. When we have a problem to solve, there may be several suitable algorithms available. We would obviously like to choose the best. Analyzing an algorithm

More information

Solving Recurrences. Lecture 13 CS2110 Summer 2009

Solving Recurrences. Lecture 13 CS2110 Summer 2009 Solving Recurrences Lecture 13 CS2110 Summer 2009 In the past Complexity: Big-O definition Non-recursive programs 2 Today Complexity of recursive programs Introduce recurrences Methods to solve recurrences

More information

Everything goes, everything comes back; eternally rolls the wheel of being. (Friedrich Nietzsche)

Everything goes, everything comes back; eternally rolls the wheel of being. (Friedrich Nietzsche) Chapter 6 Linear Recurrences Everything goes, everything comes back; eternally rolls the wheel of being. (Friedrich Nietzsche) This chapter is dedicated to linear recurrences, a special type of equations

More information

Format of Divide-and-Conquer algorithms:

Format of Divide-and-Conquer algorithms: CS 360: Data Structures and Algorithms Divide-and-Conquer (part 1) Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively

More information

7.1 Introduction. CSci 335 Software Design and Analysis III Chapter 7 Sorting. Prof. Stewart Weiss

7.1 Introduction. CSci 335 Software Design and Analysis III Chapter 7 Sorting. Prof. Stewart Weiss Chapter 7 Sorting 7.1 Introduction Insertion sort is the sorting algorithm that splits an array into a sorted and an unsorted region, and repeatedly picks the lowest index element of the unsorted region

More information

Example Correctness Proofs

Example Correctness Proofs Example Correctness Proofs Kevin C. Zatloukal March 8, 2014 In these notes, we give three examples of proofs of correctness. These are intended to demonstrate the sort of proofs we would like to see in

More information

data structures and algorithms exercise class 2: some answers

data structures and algorithms exercise class 2: some answers data structures and algorithms 2016-2017 exercise class 2: some answers 1. Show step by step how the array A = [22, 15, 36, 20, 3, 9, 29] is sorted using selection sort, bubble sort, and merge sort. First

More information

Cpt S 223. School of EECS, WSU

Cpt S 223. School of EECS, WSU Sorting Algorithms 1 Sorting methods Comparison based sorting On 2 ) methods E.g., Insertion, bubble Average time On log n) methods E.g., quick sort On logn) methods E.g., Merge sort, heap sort on-comparison

More information

1. What s wrong with the following proofs by induction?

1. What s wrong with the following proofs by induction? ArsDigita University Month : Discrete Mathematics - Professor Shai Simonson Problem Set 4 Induction and Recurrence Equations Thanks to Jeffrey Radcliffe and Joe Rizzo for many of the solutions. Pasted

More information

Divide-and-Conquer Algorithms Part Four

Divide-and-Conquer Algorithms Part Four Divide-and-Conquer Algorithms Part Four Announcements Problem Set 2 due right now. Can submit by Monday at 2:15PM using one late period. Problem Set 3 out, due July 22. Play around with divide-and-conquer

More information

Comparison Sort. - In Merge Sort, the merge procedure chooses an item from one of two arrays after comparing the top items from both arrays.

Comparison Sort. - In Merge Sort, the merge procedure chooses an item from one of two arrays after comparing the top items from both arrays. Comparison Sort A Comparison Sort is a sorting algorithm where the final order is determined only by comparisons between the input elements. - In Insertion Sort, the proper position to insert the current

More information

CS473 - Algorithms I

CS473 - Algorithms I CS473 - Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm View in slide-show mode 1 Reminder: Merge Sort Input array A sort this half sort this half Divide Conquer merge two sorted halves Combine

More information

CS325: Analysis of Algorithms, Fall Midterm Solutions

CS325: Analysis of Algorithms, Fall Midterm Solutions CS325: Analysis of Algorithms, Fall 2016 Midterm Solutions I don t know policy: you may write I don t know and nothing else to answer a question and receive 25 percent of the total points for that problem

More information

Lecture 4: Recurrences CSCI Algorithms I. Andrew Rosenberg

Lecture 4: Recurrences CSCI Algorithms I. Andrew Rosenberg Lecture 4: Recurrences CSCI 700 - Algorithms I Andrew Rosenberg Last Time Introduction to Recursion Fibonacci Binary Search MergeSort QuickSort Today Mathematical Analysis of Recurrence Relations Mathematical

More information

Lecture 7 Notes Quicksort

Lecture 7 Notes Quicksort Lecture 7 Notes Quicksort 15-122: Principles of Imperative Computation (Spring 2016) Frank Pfenning 1 Introduction In this lecture we consider two related algorithms for sorting that achieve a much better

More information

Sorting algorithms. Mergesort and Quicksort. Merge Sort. Partitioning Choice 1. Partitioning Choice 3. Partitioning Choice 2 2/5/2015

Sorting algorithms. Mergesort and Quicksort. Merge Sort. Partitioning Choice 1. Partitioning Choice 3. Partitioning Choice 2 2/5/2015 Sorting algorithms Mergesort and Quicksort Insertion, selection and bubble sort have quadratic worst case performance The faster comparison based algorithm? O(nlogn) Mergesort and Quicksort Merge Sort

More information

Lecture 2: Analysis of Algorithms (CS )

Lecture 2: Analysis of Algorithms (CS ) Lecture 2: Analysis of Algorithms (CS583-002) Amarda Shehu September 03 & 10, 2014 1 Outline of Today s Class 2 Big-Oh, Big-Omega, Theta Techniques for Finding Asymptotic Relationships Some more Fun with

More information

The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)! The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >

More information

Practice Problems (Midterm)

Practice Problems (Midterm) ECE-250 Algorithms and Data Structures (Winter 2012) Practice Problems (Midterm) Disclaimer: Please do keep in mind that this problem set does not reflect the exact topics or the fractions of each of the

More information

CS473 - Algorithms I

CS473 - Algorithms I CS473 - Algorithms I Lecture 9 Sorting in Linear Time View in slide-show mode 1 How Fast Can We Sort? The algorithms we have seen so far: Based on comparison of elements We only care about the relative

More information

CmSc 250 Intro to Algorithms. Chapter 4. Divide-and-Conquer Algorithms Mergesort, Quicksort, Binary search, Binary trees

CmSc 250 Intro to Algorithms. Chapter 4. Divide-and-Conquer Algorithms Mergesort, Quicksort, Binary search, Binary trees CmSc 250 Intro to Algorithms Chapter 4. Divide-and-Conquer Algorithms Mergesort, Quicksort, Binary search, Binary trees 1. Introduction Divide-and-Conquer is the most-well known algorithm design strategy:

More information

Algorithms Chapter 6 Heapsort

Algorithms Chapter 6 Heapsort Algorithms Chapter 6 Heapsort Assistant Professor: Ching Chi Lin 林清池助理教授 chingchi.lin@gmail.com Department of Computer Science and Engineering National Taiwan Ocean University Outline Heaps Maintaining

More information

Reading 7 : Program Correctness

Reading 7 : Program Correctness CS/Math 240: Introduction to Discrete Mathematics Fall 2015 Instructors: Beck Hasti, Gautam Prakriya Reading 7 : Program Correctness 7.1 Program Correctness Showing that a program is correct means that

More information

COSC242 Lecture 8 More Quicksort

COSC242 Lecture 8 More Quicksort COSC242 Lecture 8 More Quicksort If the pivot divides the array into two pieces of roughly the same size each time, then Quicksort behaves like Mergesort with log n splits of the array. More precisely,

More information

Recurrences. David Kauchak. Recurrence: a function that is defined with respect to itself on smaller inputs.

Recurrences. David Kauchak. Recurrence: a function that is defined with respect to itself on smaller inputs. Recurrences David Kauchak Recurrence: a function that is defined with respect to itself on smaller inputs. Why are we concerned with recurrences? The computational costs of divide and conquer algorithms

More information

1 2-3 Trees: The Basics

1 2-3 Trees: The Basics CS10: Data Structures and Object-Oriented Design (Fall 2013) November 1, 2013: 2-3 Trees: Inserting and Deleting Scribes: CS 10 Teaching Team Lecture Summary In this class, we investigated 2-3 Trees in

More information

CS383, Algorithms Notes on Lossless Data Compression and Huffman Coding

CS383, Algorithms Notes on Lossless Data Compression and Huffman Coding Prof. Sergio A. Alvarez http://www.cs.bc.edu/ alvarez/ Fulton Hall 410 B alvarez@cs.bc.edu Computer Science Department voice: (617) 552-4333 Boston College fax: (617) 552-6790 Chestnut Hill, MA 02467 USA

More information

The Greedy Method. Fundamental Techniques. The Greedy Method. Fractional Knapsack Problem

The Greedy Method. Fundamental Techniques. The Greedy Method. Fractional Knapsack Problem Fundamental Techniques There are some algorithmic tools that are quite specialised. They are good for problems they are intended to solve, but they are not very versatile. There are also more fundamental

More information

COT5405 Analysis of Algorithms Homework 3 Solutions

COT5405 Analysis of Algorithms Homework 3 Solutions COT0 Analysis of Algorithms Homework 3 Solutions. Prove or give a counter example: (a) In the textbook, we have two routines for graph traversal - DFS(G) and BFS(G,s) - where G is a graph and s is any

More information

Mathematical induction

Mathematical induction Mathematical induction If we want to prove that P n holds for for all natural numbers n, we can do the following twostep rocket called mathematical induction: 1. Prove that P 0 holds 2. Prove that if P

More information

Recurrence. Recall that we used induction to prove statements like. n(n + 1) k = n = 2

Recurrence. Recall that we used induction to prove statements like. n(n + 1) k = n = 2 s Recall that we used induction to prove statements like n n(n + 1) k = 0 + 1 + 2 + 3 +... + n = 2 k=0 In problems like this, we used a common pattern: 0 k = 0 k=0 n k = k=0 n 1 k=0 k + n, when n > 0 That

More information

Algorithms Analysis Chapter 7 Quicksort. Quicksort

Algorithms Analysis Chapter 7 Quicksort. Quicksort Algorithms Analysis Chapter 7 Quicksort 1 Quicksort Quicksort pros: Sorts in place Sorts O(n lg n) in the average case Very efficient in practice Quicksort cons: Sorts O(n 2 ) in the worst case not stable

More information

A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points

A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points A. V. Gerbessiotis CS 610-102 Spring 2014 PS 3 Mar 24, 2014 No points Problem 1. Suppose that we insert n keys into a hash table of size m using open addressing and uniform hashing. Let p(n, m) be the

More information

Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1 Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

More information

CS 102: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4)

CS 102: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4) CS 10: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4) Problem 1. a. Consider the modified binary search algorithm so that it splits the input not into two sets of almost-equal sizes, but into three

More information

Loop Invariants and Binary Search

Loop Invariants and Binary Search Loop Invariants and Binary Search Chapter 4.3.3 and 9.3.1-1 - Outline Ø Iterative Algorithms, Assertions and Proofs of Correctness Ø Binary Search: A Case Study - 2 - Outline Ø Iterative Algorithms, Assertions

More information

Divide and Conquer Examples. Top-down recursive mergesort. Gravitational N-body problem.

Divide and Conquer Examples. Top-down recursive mergesort. Gravitational N-body problem. Divide and Conquer Divide the problem into several subproblems of equal size. Recursively solve each subproblem in parallel. Merge the solutions to the various subproblems into a solution for the original

More information

CPSC 221 Basic Algorithms and Data Structures

CPSC 221 Basic Algorithms and Data Structures CPSC 221 Basic Algorithms and Data Structures Sorting Textbook References: Koffman: 10.1-10.4, 10.7-10.10 EPP 3 rd edition: 9.5 EPP 4 th edition: 11.5 Hassan Khosravi January April 2015 CPSC 221 Sorting

More information

Lecture 3: Summations and Analyzing Programs with Loops

Lecture 3: Summations and Analyzing Programs with Loops high school algebra If c is a constant (does not depend on the summation index i) then ca i = c a i and (a i + b i )= a i + b i There are some particularly important summations, which you should probably

More information

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

More information

Algorithms. Theresa Migler-VonDollen CMPS 5P

Algorithms. Theresa Migler-VonDollen CMPS 5P Algorithms Theresa Migler-VonDollen CMPS 5P 1 / 32 Algorithms Write a Python function that accepts a list of numbers and a number, x. If x is in the list, the function returns the position in the list

More information

Converting a Number from Decimal to Binary

Converting a Number from Decimal to Binary Converting a Number from Decimal to Binary Convert nonnegative integer in decimal format (base 10) into equivalent binary number (base 2) Rightmost bit of x Remainder of x after division by two Recursive

More information

2. (a) Explain the strassen s matrix multiplication. (b) Write deletion algorithm, of Binary search tree. [8+8]

2. (a) Explain the strassen s matrix multiplication. (b) Write deletion algorithm, of Binary search tree. [8+8] Code No: R05220502 Set No. 1 1. (a) Describe the performance analysis in detail. (b) Show that f 1 (n)+f 2 (n) = 0(max(g 1 (n), g 2 (n)) where f 1 (n) = 0(g 1 (n)) and f 2 (n) = 0(g 2 (n)). [8+8] 2. (a)

More information

Nearly Complete Binary Trees and Heaps

Nearly Complete Binary Trees and Heaps Nearly Complete Binary Trees and Heaps DEFINITIONS: i) The depth of a node p in a binary tree is the length (number of edges) of the path from the root to p. ii) The height (or depth) of a binary tree

More information

Mergesort and Quicksort

Mergesort and Quicksort Mergesort Mergesort and Quicksort Basic plan: Divide array into two halves. Recursively sort each half. Merge two halves to make sorted whole. mergesort mergesort analysis quicksort quicksort analysis

More information

Lecture Notes on Quicksort

Lecture Notes on Quicksort Lecture Notes on Quicksort 15-122: Principles of Imperative Computation Frank Pfenning Lecture 8 February 3, 2011 1 Introduction In this lecture we revisit the general description of quicksort from last

More information

Induction. Margaret M. Fleck. 10 October These notes cover mathematical induction and recursive definition

Induction. Margaret M. Fleck. 10 October These notes cover mathematical induction and recursive definition Induction Margaret M. Fleck 10 October 011 These notes cover mathematical induction and recursive definition 1 Introduction to induction At the start of the term, we saw the following formula for computing

More information

CLASS 3, GIVEN ON 9/27/2010, FOR MATH 25, FALL 2010

CLASS 3, GIVEN ON 9/27/2010, FOR MATH 25, FALL 2010 CLASS 3, GIVEN ON 9/27/2010, FOR MATH 25, FALL 2010 1. Greatest common divisor Suppose a, b are two integers. If another integer d satisfies d a, d b, we call d a common divisor of a, b. Notice that as

More information

Analysis of Algorithms I: Binary Search Trees

Analysis of Algorithms I: Binary Search Trees Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary

More information

Full and Complete Binary Trees

Full and Complete Binary Trees Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full

More information

Section IV.1: Recursive Algorithms and Recursion Trees

Section IV.1: Recursive Algorithms and Recursion Trees Section IV.1: Recursive Algorithms and Recursion Trees Definition IV.1.1: A recursive algorithm is an algorithm that solves a problem by (1) reducing it to an instance of the same problem with smaller

More information

Parallel and Sequential Data Structures and Algorithms Lecture (Spring 2013) Lecture 19 Quicksort and Sorting Lower Bounds

Parallel and Sequential Data Structures and Algorithms Lecture (Spring 2013) Lecture 19 Quicksort and Sorting Lower Bounds Lecture 19 Quicksort and Sorting Lower Bounds Parallel and Sequential Data Structures and Algorithms, 15-210 (Spring 2013) Lectured by Umut Acar 28 March 2013 1 Quicksort You have surely seen quicksort

More information

Analysis of Algorithms I: Optimal Binary Search Trees

Analysis of Algorithms I: Optimal Binary Search Trees Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search

More information

Homework 5 Solutions

Homework 5 Solutions Homework 5 Solutions 4.2: 2: a. 321 = 256 + 64 + 1 = (01000001) 2 b. 1023 = 512 + 256 + 128 + 64 + 32 + 16 + 8 + 4 + 2 + 1 = (1111111111) 2. Note that this is 1 less than the next power of 2, 1024, which

More information

Vieta s Formulas and the Identity Theorem

Vieta s Formulas and the Identity Theorem Vieta s Formulas and the Identity Theorem This worksheet will work through the material from our class on 3/21/2013 with some examples that should help you with the homework The topic of our discussion

More information

Two General Methods to Reduce Delay and Change of Enumeration Algorithms

Two General Methods to Reduce Delay and Change of Enumeration Algorithms ISSN 1346-5597 NII Technical Report Two General Methods to Reduce Delay and Change of Enumeration Algorithms Takeaki Uno NII-2003-004E Apr.2003 Two General Methods to Reduce Delay and Change of Enumeration

More information