# Binary Search. Search for x in a sorted array A.

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Divide and Conquer A general paradigm for algorithm design; inspired by emperors and colonizers. Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3. Combine these results together. Examples: Binary Search, Merge sort, Quicksort etc. Matrix multiplication, Selection, Convex Hulls.

2 Binary Search Search for x in a sorted array A. Binary-Search (A, p, q, x) 1. if p > q return -1; 2. r = (p + q)/2 3. if x = A[r] return r 4. else if x < A[r] Binary-Search(A, p, r, x) 5. else Binary-Search(A, r + 1, q, x) The initial call is Binary-Search(A, 1, n, x).

3 Binary Search Let T (n) denote the worst-case time to binary search in an array of length n. Recurrence is T (n) = T (n/2) + O(1). T (n) = O(log n).

4 Merge Sort Sort an unordered array of numbers A. Merge-Sort (A, p, q) 1. if p q return A; 2. r = (p + q)/2 3. Merge-Sort (A, p, r) 4. Merge-Sort (A, r + 1, q) 5. MERGE (A, p, q, r) The initial call is Merge-Sort (A, 1, n).

5 Merge Sort Let T (n) denote the worst-case time to merge sort an array of length n. Recurrence is T (n) = 2T (n/2) + O(n). T (n) = O(n log n).

6 Merge Sort: Illustration Divide Merge

7 Multiplying Numbers We want to multiply two n-bit numbers. Cost is number of elementary bit steps. Grade school method has Θ(n 2 ) cost.: xxxxxxxx xxxxxxxx xxxxxxxxx xxxxxxxxx.. xxxxxxxxx xxxxxxxxxxxxxxxx n 2 multiplies, n 2 /2 additions, plus some carries.

8 Why Bother? Doesn t hardware provide multiply? It is fast, optimized, and free. So, why bother? True for numbers that fit in one computer word. But what if numbers are very large. Cryptography (encryption, digital signatures) uses big number keys. Typically 256 to 1024 bits long! n 2 multiplication too slow for such large numbers. Karatsuba s (1962) divide-and-conquer scheme multiplies two n bit numbers in O(n 1.59 ) steps.

9 Karatsuba s Algorithm Let X and Y be two n-bit numbers. Write X = a b Y = c d a, b, c, d are n/2 bit numbers. (Assume n = 2 k.) XY = (a2 n/2 + b)(c2 n/2 + d) = ac2 n + (ad + bc)2 n/2 + bd

10 An Example X = 4729 Y = a = 47; b = 29 c = 13; d = 26. ac = = 611 ad = = 1222 bc = = 377 bd = = 754 XY = XY =

11 Karatsuba s Algorithm This is D&C: Solve 4 problems, each of size n/2; then perform O(n) shifts to multiply the terms by 2 n and 2 n/2. We can write the recurrence as T (n) = 4T (n/2) + O(n) But this solves to T (n) = O(n 2 )!

12 Karatsuba s Algorithm XY = ac2 n + (ad + bc)2 n/2 + bd. Note that (a b)(c d) = (ac + bd) (ad + bc). Solve 3 subproblems: ac, bd, (a b)(c d). We can get all the terms needed for XY by addition and subtraction! The recurrence for this algorithm is T (n) = 3T (n/2) + O(n) = O(n log 2 3 ). The complexity is O(n log 2 3 ) O(n 1.59 ).

13 Recurrence Solving: Review T (n) = 2T (n/2) + cn, with T (1) = 1. By term expansion. T (n) = 2T (n/2) + cn = 2 ( 2T (n/2 2 ) + cn/2 ) + cn = 2 2 T (n/2 2 ) + 2cn = 2 2 ( 2T (n/2 3 ) + cn/2 2) + 2cn = 2 3 T (n/2 3 ) + 3cn. = 2 i T (n/2 i ) + icn Set i = log 2 n. Use T (1) = 1. We get T (n) = n + cn(log n) = O(n log n).

14 The Tree View T (n) = 2T (n/2) + cn, with T (1) = 1. T(n) cn Total Cost cn T(n/2) cn/2 T(n/2) cn/2 2(cn/2) = cn T(n/4) cn/4 T(n/4) cn/4 T(n/4) cn/4 T(n/4) cn/4 4(cn/4) = cn T(n/8) cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 8(cn/8) = cn # leaves = n; # levels = log n. Work per level is O(n), so total is O(n log n).

15 Solving By Induction Recurrence: T (n) = 2T (n/2) + cn. Base case: T (1) = 1. Claim: T (n) = cn log n + cn. T (n) = 2T (n/2) + cn = 2 (c(n/2) log(n/2) + cn/2) + cn = cn (log n 1 + 1) + cn = cn log n + cn

16 More Examples T (n) = 4T (n/2) + cn, T (1) = 1. n Level 0 Work cn n/2 1 4cn/2 = 2cn n/4 2 16cn/4 = 4cn n/8 3 i i 4 cn / 2 i i = 2 cn

17 More Examples n Level 0 Work cn n/2 1 4cn/2 = 2cn n/4 2 16cn/4 = 4cn n/8 3 i i 4 cn / 2 i = 2 i cn Stops when n/2 i = 1, and i = log n. Recurrence solves to T (n) = O(n 2 ).

18 By Term Expansion T (n) = 4T (n/2) + cn = 4 2 T (n/2 2 ) + 2cn + cn = 4 3 T (n/2 3 ) cn + 2cn + cn. = 4 i T (n/2 i ) + cn ( 2 i i ) = 4 i T (n/2 i ) + 2 i cn Terminates when 2 i = n, or i = log n. 4 i = 2 i 2 i = n n = n 2. T (n) = n 2 + cn 2 = O(n 2 ).

19 More Examples T (n) = 2T (n/4) + n, T (1) = 1. T (n) = 2T (n/4) + n ( = 2 2T (n/4 2 ) + ) n/4 + n = 2 2 T (n/4 2 ) + 2 ( n = 2 2 2T (n/4 3 ) + ) n/ n = 2 3 T (n/4 3 ) + 3 n. = 2 i T (n/4 i ) + i n

20 More Examples Terminates when 4 i = n, or when i = log 4 n = log 2 n log 2 4 = 1 2 log n. T (n) = log n + n log 4 n = n(log 4 n + 1) = O( n log n)

21 Master Method ( n ) T (n) = at + f(n) b Total Cost n f(n) n/b a children af(n/b) n/b 2 a a a 2f(n/b 2 ) a a a a a if(n/b i)

22 Master Method Total Cost n f(n) n/b a children af(n/b) n/b 2 a a a 2f(n/b 2 ) a a a a a if(n/b i) # children multiply by factor a at each level. Number of leaves is a log b n = n log b a. Verify by taking logarithm on both sides.

23 Master Method By recursion tree, we get T (n) = Θ(n log b a ) + log b n 1 i=0 a i f Let f(n) = Θ(n p log k n), where p, k 0. ( n b i ) Important: a 1 and b > 1 are constants. Case I: p < log b a. n log b a grows faster than f(n). T (n) = Θ(n log b a )

24 Master Method By recursion tree, we get T (n) = Θ(n log b a ) + log b n 1 i=0 a i f Let f(n) = Θ(n p log k n), where p, k 0. ( n b i ) Case II: p = log b a. Both terms have same growth rates. T (n) = Θ(n log b a log k+1 n)

25 Master Method By recursion tree, we get T (n) = Θ(n log b a ) + log b n 1 i=0 a i f Let f(n) = Θ(n p log k n), where p, k 0. ( n b i ) Case III: p > log b a. n log b a is slower than f(n). T (n) = Θ (f(n))

26 Applying Master Method Merge Sort: T (n) = 2T (n/2) + Θ(n). a = b = 2, p = 1, and k = 0. So log b a = 1, and p = log b a. Case II applies, giving us T (n) = Θ(n log n) Binary Search: T (n) = T (n/2) + Θ(1). a = 1, b = 2, p = 0, and k = 0. So log b a = 0, and p = log b a. Case II applies, giving us T (n) = Θ(log n)

27 Applying Master Method T (n) = 2T (n/2) + Θ(n log n). a = b = 2, p = 1, and k = 1. p = 1 = log b a, and Case II applies. T (n) = 7T (n/2) + Θ(n 2 ). T (n) = Θ(n log 2 n) a = 7, b = 2, p = 2, and log b 2 = log 7 > 2. Case I applied, and we get T (n) = Θ(n log 7 )

28 Applying Master Method T (n) = 4T (n/2) + Θ(n 2 n). a = 4, b = 2, p = 2.5, and k = 0. So log b a = 2, and p > log b a. Case III applies, giving us ( T (n) = 2T (n/2) + Θ T (n) = Θ(n 2 n) n log n ). a = 2, b = 2, p = 1. But k = 1, and so the Master Method does not apply!

29 Matrix Multiplication Multiply two n n matrices: C = A B. Standard method: C ij = n k=1 A ik B kj. This takes O(n) time per element of C, for the total cost of O(n 3 ) to compute C. This method, known since Gauss s time, seems hard to improve. A very surprising discovery by Strassen (1969) broke the n 3 asymptotic barrier. Method is divide and conquer, with a clever choice of submatrices to multiply.

30 Divide and Conquer Let A, B be two n n matrices. We want to compute the n n matrix C = AB. ( ) ( ) a11 a A = 12 b11 b B = 12 a 21 a 22 b 21 b 22 ( ) c11 c C = 12 c 21 c 22 Entries a 11 are n/2 n/2 submatrices.

31 Divide and Conquer The product matrix can be written as: c 11 = a 11 b 11 + a 12 b 21 c 12 = a 11 b 12 + a 12 b 22 c 21 = a 21 b 11 + a 22 b 21 c 22 = a 21 b 12 + a 22 b 22 Recurrence for this D&C algorithm is T (n) = 8T (n/2) + O(n 2 ). But this solves to T (n) = O(n 3 )!

32 Strassen s Algorithm Strassen chose these submatrices to multiply: P 1 = (a 11 + a 22 )(b 11 + b 22 ) P 2 = (a 21 + a 22 )b 11 P 3 = a 11 (b 12 b 22 ) P 4 = a 22 (b 21 b 11 ) P 5 = (a 11 + a 12 )b 22 P 6 = (a 21 a 11 )(b 11 + b 12 ) P 7 = (a 12 a 22 )(b 21 + b 22 )

33 Strassen s Algorithm Then, c 11 = P 1 + P 4 P 5 + P 7 c 12 = P 3 + P 5 c 21 = P 2 + P 4 c 22 = P 1 + P 3 P 2 + P 6 Recurrence for this algorithm is T (n) = 7T (n/2) + O(n 2 ).

34 Strassen s Algorithm The recurrence T (n) = 7T (n/2) + O(n 2 ). solves to T (n) = O(n log 2 7 ) = O(n 2.81 ). Ever since other researchers have tried other products to beat this bound. E.g. Victor Pan discovered a way to multiply two matrices using 143, 640 multiplications. Using more advanced methods, the current best algorithm for multiplying two n n matrices runs in roughly O(n ) time.

35 Quick Sort Algorithm Simple, fast, widely used in practice. Can be done in place; no extra space. General Form: 1. Partition: Divide into two subarrays, L and R; elements in L are all smaller than those in R. 2. Recurse: Sort L and R recursively. 3. Combine: Append R to the end of L. Partition (A, p, q, i) partitions A with pivot A[i].

36 Partition Partition returns the index of the cell containing the pivot in the reorganized array Example: Partition (A, 0, 10, 3). 4, 3, 2, 6, 7, 11, 9, 10, 13, 21, 8

37 Quick Sort Algorithm QuickSort (A, p, q) sorts the subarray A[p q]. Initial call with p = 0 and q = n 1. QuickSort(A, p, q) if p q then return i random(p, q) r Partition(A, p, q, i) Quicksort (A, p, r 1) Quicksort (A, r + 1, q)

38 Analysis of QuickSort Lucky Case: Each Partition splits array in halves. We get T (n) = 2T (n/2) + Θ(n) = Θ(n log n). Unlucky Case: Each partition gives unbalanced split. We get T (n) = T (n 1) + Θ(n) = Θ(n 2 ). In worst case, Quick Sort as bad as BubbleSort. The worst-case occurs when the list is already sorted, and the last element chosen as pivot. But, while BubbleSort always performs poorly on certain inputs, because of random pivot, QuickSort has a chance of doing much better.

39 Analyzing QuickSort T (n): runtime of randomized QuickSort. Assume all elements are distinct. Recurrence for T (n) depends on two subproblem sizes, which depend on random partition element. If pivot is i smallest element, then exactly (i 1) items in L and (n i) in R. Call it an i-split. What s the probability of i-split? Each element equally likely to be chosen as pivot, so the answer is 1 n.

40 Solving the Recurrence T (n) = = 1 n = 2 n = 2 n n i=1 1 (runtime with i-split) + n + 1 n n (T (i 1) + T (n i)) + n + 1 i=1 n T (i 1) + n + 1 i=1 n 1 i=0 T (i) + n + 1

41 Solving the Recurrence Multiply both sides by n. Subtract the same formula for n 1. nt (n) = 2 (n 1)T (n 1) = 2 n 1 i=0 n 2 i=0 T (i) + n 2 + n T (i) + (n 1) 2 + (n 1)

42 Solving the Recurrence nt (n) = (n + 1)T (n 1) + 2n T (n) n + 1 T (n 1) = n T (n 2) = n 1. = T (2) n + 3 i=3 + 2 n n + 2 n i = Θ(1) + 2 ln n Thus, T (n) 2(n + 1) ln n.

43 Median Finding Median of n items is the item with rank n/2. Rank of an item is its position in the list if the items were sorted in ascending order. Rank i item also called ith statistic. Example: {16, 5, 30, 8, 55}. Popular statistics are quantiles: items of rank n/4, n/2, 3n/4. SAT/GRE: which score value forms 95th percentile? Item of rank 0.95n.

44 Median Finding After spending O(n log n) time on sorting, any rank can be found in O(n) time. Can we find a rank without sorting?

45 Min and Max Finding We can find items of rank 1 or n in O(n) time. minimum (A) min A[0] for i = 1 to n 1 do if min > A[i] then min A[i]; return min The algorithm minimum finds the smallest (rank 1) item in O(n) time. A similar algorithm finds maximum item.

46 Both Min and Max Find both min and max using 3n/2 comparisons. MIN-MAX (A) if A = 1, then return min = max = A[0] Divide A into two equal subsets A 1, A 2 (min 1, max 1 ) := MIN-MAX (A 1 ) (min 2, max 2 ) := MIN-MAX (A 2 ) if min 1 min 2 then return min = min 1 else return min = min 2 if max 1 max 2 then return max = max 1 else return max = max 2

47 Both Min and Max The recurrence for this algorithm is T (n) = 2T (n/2) + 2. Verify this solves to T (n) = 3n/2 2.

48 Finding Item of Rank k Direct extension of min/max finding to rank k item will take Θ(kn) time. In particular, finding the median will take Ω(n 2 ) time, which is worse than sorting. Median can be used as a perfect pivot for (deterministic) quick sort. But only if found faster than sorting itself. We present a linear time algorithm for selecting rank k item [BFPRT 1973].

49 Linear Time Selection SELECT (k) 1. Divide items into n/5 groups of 5 each. 2. Find the median of each group (using sorting). 3. Recursively find median of n/5 group medians. 4. Partition using median-of-median as pivot. 5. Let low side have s, and high side have n s items. 6. If k s, call select(k) on low side; otherwise, call select(k s) on high side.

50 Illustration Divide items into n/5 groups of 5 items each. Find the median of each group (using sorting). Use SELECT to recursively find the median of the n/5 group medians. x medians x = median of medians Group 1 Group 2 Group 3 Group 4 Group 5 Group 6

51 Illustration Partition the input by using this median-of-median as pivot. Suppose low side of the partition has s elements, and high side has n s elements. If k s, recursively call SELECT(k) on low side; otherwise, recursively call SELECT(k s) on high side. x Items => x Group 1 Group 6 Group 3 Group 4 Group 5 Group 2

52 Recurrence For runtime analysis, we bound the number of items x, the median of medians. At least half the medians are x. At least half of the n/5 groups contribute at least 3 items to the high side. (Only the last group can contribute fewer. Thus, items x are at least ( n ) n Similarly, items x is also 3n/10 6.

53 Recurrence Recursive call to select is on size 7n/ Let T (n) = worst-case complexity of select. Group medians, and partition take O(n) time. Step 3 has a recursive call T (n/5), and Step 5 has a recursive call T (7n/10 + 6). Thus, we have the recurrence: T (n) T ( n 5 ) + T (7n ) + O(n). Assume T (n) = O(1) for small n 80.

54 Recurrence T (n) T ( n 5 ) + T (7n ) + O(n) Inductively verify that T (n) cn for some constant c. T (n) c(n/5) + c(7n/10 + 6) + O(n) 9cn/10 + 6c + O(n) cn In above, choose c so that c(n/10 6) beats the function O(n) for all n.

55 Convex Hulls 1. Convex hulls are to CG what sorting is to discrete algorithms. 2. First order shape approximation. Invariant under rotation and translation. p 3. Rubber-band analogy.

56 Convex Hulls Many aplications in robotics, shape analysis, line fitting etc. Example: if CH(P 1 ) CH(P 2 ) =, then objects P 1 and P 2 do not intersect. Convex Hull Problem: Given a finite set of points S, compute its convex hull CH(S). (Ordered vertex list.)

57 Divide and Conquer Upper Tangent CH(A) A B CH(B) Sort points by X-coordinates. Divide points into equal halves A and B. Recursively compute CH(A) and CH(B). Merge CH(A) and CH(B) to obtain CH(S).

58 Merging Convex Hulls Lower Tangent a = rightmost point of CH(A). b = leftmost point of CH(B). while ab not lower tangent of CH(A) and CH(B) do 1. while ab not lower tangent to CH(A) set a = a 1 (move a CW); 2. while ab not lower tangent to CH(B) set b = b + 1 (move b CCW); Return ab

59 Tangent Finding b CH(A) a CH(B)

60 Analysis of D&C Upper Tangent CH(A) A B CH(B) Initial sorting takes O(N log N) time. Recurrence T (N) = 2T (N/2) + O(N) O(N) for merging (computing tangents). Recurrence solves to T (N) = O(N log N).

### Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort).

Chapter 7: Quicksort Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort). The three steps of Quicksort are as follows:

### Algorithms Analysis Chapter 7 Quicksort. Quicksort

Algorithms Analysis Chapter 7 Quicksort 1 Quicksort Quicksort pros: Sorts in place Sorts O(n lg n) in the average case Very efficient in practice Quicksort cons: Sorts O(n 2 ) in the worst case not stable

### CS 102: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4)

CS 10: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4) Problem 1. a. Consider the modified binary search algorithm so that it splits the input not into two sets of almost-equal sizes, but into three

### CS2223 Algorithms B Term 2013 Exam 2 Solutions

CS2223 Algorithms B Term 2013 Exam 2 Solutions Dec. 3, 2013 By Prof. Carolina Ruiz Dept. of Computer Science WPI PROBLEM 1: Solving Recurrences (15 points) Solve the recurrence T (n) = 2T (n/3) + n using

### Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

### Data Structures and Algorithms Week 3

Data Structures and Algorithms Week 3 1. Divide and conquer 2. Merge sort, repeated substitutions 3. Tiling 4. Recurrences Recurrences Running times of algorithms with recursive calls can be described

### Format of Divide-and-Conquer algorithms:

CS 360: Data Structures and Algorithms Divide-and-Conquer (part 1) Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively

### CS420/520 Algorithm Analysis Spring 2010 Lecture 04

CS420/520 Algorithm Analysis Spring 2010 Lecture 04 I. Chapter 4 Recurrences A. Introduction 1. Three steps applied at each level of a recursion a) Divide into a number of subproblems that are smaller

### Problem Set #1 Solutions

CS161: Design and Analysis of Algorithms Summer 2004 Problem Set #1 Solutions General Notes Regrade Policy: If you believe an error has been made in the grading of your problem set, you may resubmit it

### Week 7: Divide and Conquer

Agenda: Divide and Conquer technique Multiplication of large integers Exponentiation Matrix multiplication 1 2- Divide and Conquer : To solve a problem we can break it into smaller subproblems, solve each

### CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

### Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4

Divide d Conquer Textook Reading Chapters 4, 7 & 33.4 Overview Design principle Divide d conquer Proof technique Induction, induction, induction Analysis technique Recurrence relations Prolems Sorting

### The divide and conquer strategy has three basic parts. For a given problem of size n,

1 Divide & Conquer One strategy for designing efficient algorithms is the divide and conquer approach, which is also called, more simply, a recursive approach. The analysis of recursive algorithms often

### The Selection Problem

The Selection Problem The selection problem: given an integer k and a list x 1,..., x n of n elements find the k-th smallest element in the list Example: the 3rd smallest element of the following list

### Divide and Conquer My T. UF

Introduction to Algorithms Divide and Conquer @ UF General Approach Divide the problem into a number of subproblems that are smaller instances of the same problem Conquer the subproblems by solving them

### 2 Exercises and Solutions

2 Exercises and Solutions Most of the exercises below have solutions but you should try first to solve them. Each subsection with solutions is after the corresponding subsection with exercises. 2.1 Sorting

### CS473 - Algorithms I

CS473 - Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm View in slide-show mode 1 Reminder: Merge Sort Input array A sort this half sort this half Divide Conquer merge two sorted halves Combine

### CS335 Program2: ConvexHulls A Divide-and-Conquer Algorithm 15 points Due: Tues, October 6

CS335 Program2: ConvexHulls A Divide-and-Conquer Algorithm 15 points Due: Tues, October 6 Convex Hulls There is a complex mathematical definition of a convex hull, but the informal definition is sufficient

### Closest Pair Problem

Closest Pair Problem Given n points in d-dimensions, find two whose mutual distance is smallest. Fundamental problem in many applications as well as a key step in many algorithms. p q A naive algorithm

### Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

### Recurrences (CLRS )

Recurrences (CLRS 4.1-4.2) Last time we discussed divide-and-conquer algorithms Divide and Conquer To Solve P: 1. Divide P into smaller problems P 1,P 2,P 3...P k. 2. Conquer by solving the (smaller) subproblems

### Introduction to Algorithms. Part 1: Divide and Conquer Sorting and Searching

Introduction to Algorithms Part 1: Divide and Conquer Sorting and Searching 1) Convex Hulls: An Example 2) Divide and Conquer 3) Sorting Revisited 4) Parallel Sorting 5) Finding the Median 6) Books c Wayne

### Sorting algorithms. Mergesort and Quicksort. Merge Sort. Partitioning Choice 1. Partitioning Choice 3. Partitioning Choice 2 2/5/2015

Sorting algorithms Mergesort and Quicksort Insertion, selection and bubble sort have quadratic worst case performance The faster comparison based algorithm? O(nlogn) Mergesort and Quicksort Merge Sort

### Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

### Divide-and-Conquer Algorithms Part One

Divide-and-Conquer Algorithms Part One Announcements Problem Set One completely due right now. Solutions distributed at the end of lecture. Programming section today in Gates B08 from from 3:45PM 5:00PM.

### Steven Skiena. skiena

Lecture 3: Recurrence Relations (1997) Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Argue the solution to T(n) =

### Recurrence Relations

Recurrence Relations Introduction Determining the running time of a recursive algorithm often requires one to determine the big-o growth of a function T (n) that is defined in terms of a recurrence relation.

### Quicksort An Example. When lo and hi cross, we are done. The final value of hi is the position in which the partitioning element ends up.

Quicksort An Example We sort the array A = (38 81 22 48 13 69 93 14 45 58 79 72) with quicksort, always choosing the pivot element to be the element in position (left+right)/2. The partitioning during

### Solving Recurrences. Lecture 13 CS2110 Summer 2009

Solving Recurrences Lecture 13 CS2110 Summer 2009 In the past Complexity: Big-O definition Non-recursive programs 2 Today Complexity of recursive programs Introduce recurrences Methods to solve recurrences

### Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Quiz 1 Quiz 1 Do not open this quiz booklet until you are directed

### CSC148 Lecture 8. Algorithm Analysis Binary Search Sorting

CSC148 Lecture 8 Algorithm Analysis Binary Search Sorting Algorithm Analysis Recall definition of Big Oh: We say a function f(n) is O(g(n)) if there exists positive constants c,b such that f(n)

### Merge Sort and Recurrences. COSC 581, Algorithms January 14, 2014

Merge Sort and Recurrences COSC 581, Algorithms January 14, 2014 Reading Assignments Today s class: Chapter 2, 4.0, 4.4 Reading assignment for next class: Chapter 4.2, 4.5 3 Common Algorithmic Techniques

### Divide-and-Conquer Algorithms Part Four

Divide-and-Conquer Algorithms Part Four Announcements Problem Set 2 due right now. Can submit by Monday at 2:15PM using one late period. Problem Set 3 out, due July 22. Play around with divide-and-conquer

### Algorithms. Margaret M. Fleck. 18 October 2010

Algorithms Margaret M. Fleck 18 October 2010 These notes cover how to analyze the running time of algorithms (sections 3.1, 3.3, 4.4, and 7.1 of Rosen). 1 Introduction The main reason for studying big-o

### CSC 143. Two important problems. Searching and Sorting. Review: Linear Search. Example. Review: Binary Search. How Efficient Is Linear Search?

CSC 43 Searching and Sorting [Chapter 9, pp. 402-432] Two important problems Search: finding something in a set of data Sorting: putting a set of data in order Both very common, very useful operations

### Divide And Conquer Algorithms

CSE341T/CSE549T 09/10/2014 Lecture 5 Divide And Conquer Algorithms Recall in last lecture, we looked at one way of parallelizing matrix multiplication. At the end of the lecture, we saw the reduce SUM

### Divide-and-Conquer. Three main steps : 1. divide; 2. conquer; 3. merge.

Divide-and-Conquer Three main steps : 1. divide; 2. conquer; 3. merge. 1 Let I denote the (sub)problem instance and S be its solution. The divide-and-conquer strategy can be described as follows. Procedure

### Linear Time Selection

Chapter 2 Linear Time Selection Definition 1 Given a sequence x 1, x 2,..., x n of numbers, the ith order statistic is the ith smallest element. Maximum or Minimum When i = 1, it is the minimum element

### Michal Forišek: Early beta verzia skrípt z ADŠ

Časová zložitosť II Michal Forišek: Early beta verzia skrípt z ADŠ In this part of the article we will focus on estimating the time complexity for recursive programs. In essence, this will lead to finding

Answers to Homework 1 p. 13 1.2 3 What is the smallest value of n such that an algorithm whose running time is 100n 2 runs faster than an algorithm whose running time is 2 n on the same machine? Find the

### Algorithm Analysis Sorting II. Fall 2013 Carola Wenk

Algorithm Analysis Sorting II Fall 2013 Carola Wenk Is Selection Sort Practical? What is the running time of selection sort for lists that have thousands of items? The theoretical performance is not particularly

### Objective. - mathematical induction, recursive definitions - arithmetic manipulations, series, products

Recurrences Objective running time as recursive function solve recurrence for order of growth method: substitution method: iteration/recursion tree method: MASTER method prerequisite: - mathematical induction,

### Writing programs to solve problem consists of a large. how to represent aspects of the problem for solution

Algorithm Efficiency & Sorting Algorithm efficiency Big-O notation Searching algorithms Sorting algorithms 1 Overview 2 If a given ADT (i.e. stack or queue) is attractive as part of a solution How will

### Design and Analysis of Algorithms

Design and Analysis of Algorithms CSE 5311 Lecture 3 Divide-and-Conquer Junzhou Huang, Ph.D. Department of Computer Science and Engineering CSE5311 Design and Analysis of Algorithms 1 Reviewing: Θ-notation

### 4.4 The recursion-tree method

4.4 The recursion-tree method Let us see how a recursion tree would provide a good guess for the recurrence = 3 4 ) Start by nding an upper bound Floors and ceilings usually do not matter when solving

### Decrease-and-Conquer Algorithms

Decrease-and-Conquer Algorithms Joaquim Madeira Version 0.1 October 2015 U. Aveiro, October 2015 1 Overview Decrease-and-Conquer Linear Search Binary Search Ternary Search Interpolation Search The Selection

### Algorithm Analysis [2]: if-else statements, recursive algorithms. COSC 2011, Winter 2004, Section N Instructor: N. Vlajic

1 Algorithm Analysis []: if-else statements, recursive algorithms COSC 011, Winter 004, Section N Instructor: N. Vlajic Algorithm Analysis for-loop Running Time The running time of a simple loop for (int

### Analysis of Binary Search algorithm and Selection Sort algorithm

Analysis of Binary Search algorithm and Selection Sort algorithm In this section we shall take up two representative problems in computer science, work out the algorithms based on the best strategy to

### Randomized algorithms

Randomized algorithms March 10, 2005 1 What are randomized algorithms? Algorithms which use random numbers to make decisions during the executions of the algorithm. Why would we want to do this?? Deterministic

### Kapitel 1 Multiplication of Long Integers (Faster than Long Multiplication)

Kapitel 1 Multiplication of Long Integers (Faster than Long Multiplication) Arno Eigenwillig und Kurt Mehlhorn An algorithm for multiplication of integers is taught already in primary school: To multiply

### CS 4310 HOMEWORK SET 1

CS 4310 HOMEWORK SET 1 PAUL MILLER Section 2.1 Exercises Exercise 2.1-1. Using Figure 2.2 as a model, illustrate the operation of INSERTION-SORT on the array A = 31, 41, 59, 26, 41, 58. Solution: Assume

### Balanced search trees

Lecture 8 Balanced search trees 8.1 Overview In this lecture we discuss search trees as a method for storing data in a way that supports fast insert, lookup, and delete operations. (Data structures handling

### Recurrences. David Kauchak. Recurrence: a function that is defined with respect to itself on smaller inputs.

Recurrences David Kauchak Recurrence: a function that is defined with respect to itself on smaller inputs. Why are we concerned with recurrences? The computational costs of divide and conquer algorithms

### Solving recurrences. CS 4407, Algorithms University College Cork, Gregory M. Provan

Solving recurrences The analysis of divide and conquer algorithms require us to solve a recurrence. Recurrences are a major tool for analysis of algorithms MergeSort A L G O R I T H M S A L G O R I T H

### Searching & Sorting. Searching & Sorting 1

Searching & Sorting Searching & Sorting 1 The Plan Searching Sorting Java Context Searching & Sorting 2 Search (Retrieval) Looking it up One of most fundamental operations Without computer Indexes Tables

### Mergesort and Quicksort

Mergesort Mergesort and Quicksort Basic plan: Divide array into two halves. Recursively sort each half. Merge two halves to make sorted whole. mergesort mergesort analysis quicksort quicksort analysis

### CS 561, Lecture 3 - Recurrences. Jared Saia University of New Mexico

CS 561, Lecture 3 - Recurrences Jared Saia University of New Mexico Recurrence Relations Oh how should I not lust after eternity and after the nuptial ring of rings, the ring of recurrence - Friedrich

### Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

### Dynamic Programming. Lecture 11. 11.1 Overview. 11.2 Introduction

Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach

### Computational Geometry. Lecture 1: Introduction and Convex Hulls

Lecture 1: Introduction and convex hulls 1 Geometry: points, lines,... Plane (two-dimensional), R 2 Space (three-dimensional), R 3 Space (higher-dimensional), R d A point in the plane, 3-dimensional space,

### The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >

### Biostatistics 615/815

Merge Sort Biostatistics 615/815 Lecture 8 Notes on Problem Set 2 Union Find algorithms Dynamic Programming Results were very ypositive! You should be gradually becoming g y g comfortable compiling, debugging

### Lecture 4: Recurrences CSCI Algorithms I. Andrew Rosenberg

Lecture 4: Recurrences CSCI 700 - Algorithms I Andrew Rosenberg Last Time Introduction to Recursion Fibonacci Binary Search MergeSort QuickSort Today Mathematical Analysis of Recurrence Relations Mathematical

### CSE 421 Algorithms: Divide and Conquer

CSE 421 Algorithms: Divide and Conquer Summer 2011 Larry Ruzzo Thanks to Paul Beame, James Lee, Kevin Wayne for some slides hw2 empirical run times e n Plot Time vs n Fit curve to it (e.g., with Excel)

Divide and Conquer Paradigm Melissa Manley California State University Northridge Author s address: 18111 Nordhoff Street, Northridge, CA, 91330 Permission to make digital/hard copy of all or part of this

### In what follows, we will focus on Voronoi diagrams in Euclidean space. Later, we will generalize to other distance spaces.

Voronoi Diagrams 4 A city builds a set of post offices, and now needs to determine which houses will be served by which office. It would be wasteful for a postman to go out of their way to make a delivery

### Chapters 1 and 5.2 of GT

Subject 2 Spring 2014 Growth of Functions and Recurrence Relations Handouts 3 and 4 contain more examples;the Formulae Collection handout might also help. Chapters 1 and 5.2 of GT Disclaimer: These abbreviated

### COP 4531 Complexity & Analysis of Data Structures & Algorithms

COP 4531 Complexity & Analysis of Data Structures & Algorithms Lecture 3 Recurrences and Divide and Conquer Thanks to people who contributed to these slides including the text authors and Piyush Kumar,

### Divide and Conquer: Counting Inversions

Divide and Conquer: Counting Inversions Rank Analysis Collaborative filtering matches your preference (books, music, movies, restaurants) with that of others finds people with similar tastes recommends

### LECTURE NOTES ON DESIGN AND ANALYSIS OF ALGORITHMS

LECTURE NOTES ON DESIGN AND ANALYSIS OF ALGORITHMS INSTITUTE OF AERONAUTICAL ENGINEERING Dundigal 500 043, Hyderabad CONTENTS UNIT 1 BASIC CONCEPTS Algorithm Performance of Programs Algorithm Design Goals

### Discrete Optimization

Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using

### The Greedy Method. Fundamental Techniques. The Greedy Method. Fractional Knapsack Problem

Fundamental Techniques There are some algorithmic tools that are quite specialised. They are good for problems they are intended to solve, but they are not very versatile. There are also more fundamental

### A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points

A. V. Gerbessiotis CS 610-102 Spring 2014 PS 3 Mar 24, 2014 No points Problem 1. Suppose that we insert n keys into a hash table of size m using open addressing and uniform hashing. Let p(n, m) be the

### 5.4 Closest Pair of Points

5.4 Closest Pair of Points Closest Pair of Points Closest pair. Given n points in the plane, find a pair with smallest Euclidean distance between them. Fundamental geometric primitive. Graphics, computer

### Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

### Divide-and-conquer algorithms

Chapter 2 Divide-and-conquer algorithms The divide-and-conquer strategy solves a problem by: 1 Breaking it into subproblems that are themselves smaller instances of the same type of problem 2 Recursively

### 1 Recursion and the divide-and-conquer approach

CS 7B - Spring 2014 - Assignment 4. 4/17/14 Write responses on paper and submit programs by email. 1 Recursion and the divide-and-conquer approach Many useful algorithms are recursive in structure: to

### Analysis of Algorithms - Recurrences (cont.) -

Analysis of Algoritms - Recurrences (cont.) - Andreas Ermedal MRTC (Mälardalens Real Time Researc Center) andreas.ermedal@md.se Autumn 004 Recurrences: Reersal Recurrences appear frequently in running

### 28 Closest-Point Problems -------------------------------------------------------------------

28 Closest-Point Problems ------------------------------------------------------------------- Geometric problems involving points on the plane usually involve implicit or explicit treatment of distances

### Computer Algorithms. Merge Sort. Analysis of Merge Sort. Recurrences. Recurrence Relations. Iteration Method. Lecture 6

Computer Algorithms Lecture 6 Merge Sort Sorting Problem: Sort a sequence of n elements into non-decreasing order. Divide: Divide the n-element sequence to be sorted into two subsequences of n/ elements

### CSE 548: Analysis of Algorithms. Lecture 2 ( Divide-and-Conquer Algorithms: Integer Multiplication )

CSE 548: Analysis of Algorithms Lecture 2 ( Divide-and-Conquer Algorithms: Integer Multiplication ) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Spring 2015 A right trominois an

### CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.

Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure

### Converting a Number from Decimal to Binary

Converting a Number from Decimal to Binary Convert nonnegative integer in decimal format (base 10) into equivalent binary number (base 2) Rightmost bit of x Remainder of x after division by two Recursive

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### Data Structures. Algorithm Performance and Big O Analysis

Data Structures Algorithm Performance and Big O Analysis What s an Algorithm? a clearly specified set of instructions to be followed to solve a problem. In essence: A computer program. In detail: Defined

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992

CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992 Original Lecture #6: 28 January 1991 Topics: Triangulating Simple Polygons

### Chapter 3. if 2 a i then location: = i. Page 40

Chapter 3 1. Describe an algorithm that takes a list of n integers a 1,a 2,,a n and finds the number of integers each greater than five in the list. Ans: procedure greaterthanfive(a 1,,a n : integers)

### Algorithms and recursion

Read: Chapter 13 from textbook Algorithms and recursion What programs implement is recipes for solving problems. These recipes are called algorithms. We have already seen (in the practice problems from

### Algorithms, Integers

CHAPTER 3 Algorithms, Integers 3.1. Algorithms Consider the following list of instructions to find the maximum of three numbers a, b, c: 1. Assign variable x the value of a. 2. If b > x then assign x the

### recursion, O(n), linked lists 6/14

recursion, O(n), linked lists 6/14 recursion reducing the amount of data to process and processing a smaller amount of data example: process one item in a list, recursively process the rest of the list

### Efficiency of algorithms. Algorithms. Efficiency of algorithms. Binary search and linear search. Best, worst and average case.

Algorithms Efficiency of algorithms Computational resources: time and space Best, worst and average case performance How to compare algorithms: machine-independent measure of efficiency Growth rate Complexity

### A Note on Maximum Independent Sets in Rectangle Intersection Graphs

A Note on Maximum Independent Sets in Rectangle Intersection Graphs Timothy M. Chan School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1, Canada tmchan@uwaterloo.ca September 12,

### CS473 - Algorithms I

CS473 - Algorithms I Lecture 9 Sorting in Linear Time View in slide-show mode 1 How Fast Can We Sort? The algorithms we have seen so far: Based on comparison of elements We only care about the relative

### Searching and Sorting Algorithms

Searching and Sorting Algorithms CS117, Fall 2004 Supplementary Lecture Notes Written by Amy Csizmar Dalal 1 Introduction How do you find someone s phone number in the phone book? How do you find your

### O(n) linear time This means that the algorithm requires a number of steps proportional to the size of the task.

Time complexity : Big O notation f(n) = O(g(n)) means There are positive constants c and k such that: 0

### Chapter. Fundamental Techniques. Contents

Chapter 5 Fundamental Techniques Contents 5.1 The Greedy Method................... 259 5.1.1 The Fractional Knapsack Problem.......... 259 5.1.2 Task Scheduling.................... 261 5.2 Divide-and-Conquer...................

### External Sorting. Chapter 13. Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1

External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing