Decrease-and-Conquer Algorithms

Transcription

1 Decrease-and-Conquer Algorithms Joaquim Madeira Version 0.1 October 2015 U. Aveiro, October

2 Overview Decrease-and-Conquer Linear Search Binary Search Ternary Search Interpolation Search The Selection Problem The Fake-Coin Problem The Russian Multiplication Method U. Aveiro, October

3 Decrease-And-Conquer Exploit the relationship between A solution to a given problem instance A solution to a smaller instance of the same problem General framework (Top-Down) Identify ONE similar and smaller problem instance The smaller instance is solved recursively Solutions for smaller instances are processed to get the solution of the original problem, if needed Compare with Divide-and-Conquer!! U. Aveiro, October

4 Decrease-And-Conquer Where have we seen this before? Computing b n using b n = b n div 2 x b n div 2, if n is even b n = b x b (n-1) div 2 x b (n-1) div 2, if n is odd Base cases? Number of multiplications? Complexity? M(n) = M(n div 2) + 1, if n is even M(n) = M((n-1) div 2) + 2, if n is odd U. Aveiro, October

5 Decrease-And-Conquer How does instance size decrease? Decrease by a constant factor n ; n / 2 ; n / 4; n ; n / 3 ; n / 9; Decrease by a constant n ; n - 1 ; n 2 ; Variable-size decrease Size reduction pattern varies from iteration to iteration U. Aveiro, October

6 Decrease by a Constant Factor Reduce instance size by a constant factor in each iteration Usually, decrease by halving! Complexity? T(1) = c T(n) = T(n / b) + f(n) T(n) in Θ(log n), if f(n) = constant T(n) in Θ(n), if f(n) in Θ(n) Examples? U. Aveiro, October

7 Decrease by a Constant Reduce instance size by a constant in each iteration Usually, decrease by one! Complexity? Examples Insertion sort Others? T(1) = c T(n) = T(n - 1) + f(n) U. Aveiro, October

8 Variable-Size Decrease Size reduction pattern varies from iteration to iteration Examples Euclid s algorithm for the gcd Interpolation search The selection problem U. Aveiro, October

9 Linear Search Given an array of n elements (unsorted!) First occurrence of the smallest element? Idea Compare A[n-1] with the smallest element in A[0..n-2] How many comparisons? Complexity? C(1) = 0 C(n) = C(n-1) + 1 U. Aveiro, October

10 Linear Search Given an array of n elements (unsorted!) Search value / key X : index of first occurrence? Idea Compare A[0] with X If not equal, search in A[1..n-1] Best case? 1 comparison Worst case? C(1) = 1 C(n) = C(n-1) + 1 U. Aveiro, October

11 Binary Search Given a sorted array of n elements : A[left..right] Search value / key X : index? Idea Compare A[middle] with X If equal, return middle If larger, recursively search in A[left..middle - 1] If smaller, recursively search in A[middle + 1..right] Example? Draw the tree of possible searches U. Aveiro, October

12 Binary Search How to compute middle? Be sure to avoid overflow! Shifting! How many comparisons per iteration? Try using just one comparison! When to stop the recursion? How to report a non-existing value / key? Signed vs. unsigned! Iterative vs. recursive implementations? U. Aveiro, October

13 Binary Search Best case? Just 1 iteration Worst case? Always select the largest partition! Odd vs. even number of elements? When do we always have equal-sized partitions? Try to obtain a closed formula for the number of iterations!! Compare with linear search!! U. Aveiro, October

14 Ternary Search Given a sorted array of n elements : A[left..right] Search value / key X : index? Idea Compare A[leftThird] with X If equal, return leftthird If larger, recursively search in A[left..leftThird - 1] Compare A[rightThird] with X If equal, return rightthird If larger, rec. search in A[leftThird + 1..rightThird - 1] If smaller, recursively search in A[rightThird + 1..right] Example? Draw the tree of possible searches U. Aveiro, October

15 Ternary Search How to compute leftthird and rightthird? Ideas? How many comparisons per iteration? Can their number be reduced? When to stop the recursion? How to report a non-existing value / key? Signed vs. unsigned! Iterative vs. recursive implementations? U. Aveiro, October

16 Ternary Search Best case? Just 1 iteration Worst case? Always select the largest partition! When do we always have equal-sized partitions? Try to obtain a closed formula for the number of iterations!! Compare with binary search!! Complexity order Actual number of comparisons U. Aveiro, October

17 Interpolation Search Given a sorted array of n elements : A[left..right] Search value / key X : index? How do we search the Yellow Pages? Idea If possible, estimate a position index, through interpolation If A[index] == X, return index If A[index] > X; recursively search in A[left..index - 1] If A[index] < X; recursively search in A[index + 1..right] Example? U. Aveiro, October

18 Interpolation Search How to compute index? Use the searched value / key and linear interpolation (index left) / (X A[left]) = (right left) / (A[right] A[left]) Check that X A[left] and X A[right]!! Accuracy? Same issues as before! How many comparisons per iteration? When to stop the recursion? How to report a non-existing value / key? Iterative vs. recursive implementations? U. Aveiro, October

19 Interpolation Search Variable-size decrease algorithm!! Best case? Just 1 iteration When does it happen? Worst case? O(n)!! When does it happen? Average case? O(log log n)!! In practice : not faster than binary search!! Calculations U. Aveiro, October

20 The Selection Problem Given an array of n elements (unsorted!) Find the kth smallest element Already met the k = 1 and k = n cases!! Application : find the median : k = (n+1) div 2 Can we do it fast, without having to sort first? E.g., using Mergesort : O(n log n) U. Aveiro, October

21 The Selection Problem Idea Use the partitioning step of Quicksort! How to? Choose a pivot Subdivide into 2 (or 3) subsets Check k against the partition boundaries If needed, proceed recursively BUT, only 1 sub-problem to be solved!! Example? U. Aveiro, October

22 The Selection Problem Variable-size decrease algorithm!! Best case? Just 1 iteration : O(n)!! When does it happen? Worst case? O(n 2 )!! When does it happen? Average case? O(n)!! U. Aveiro, October

23 The Fake-Coin Problem Given n identically looking coins Find the one that is a fake! Use only a balance scale! The fake coin is lighter than a genuine one! Efficient algorithm? U. Aveiro, October

24 The Fake-Coin Problem Idea Divide the coins into two equal piles Leave a coin out, if n is odd Put the two piles into the balance scale Piles have the same weight The left out coin is the fake If not, proceed recursively with the lighter pile Solve just one half-sized sub-problem in each step Base cases? Complexity? Idea : use 3 coin piles, instead of 2!! U. Aveiro, October

25 The Russian Multiplication Method How did Russian peasants multiply two numbers? Given m and n, two positive integers Compute r = m n Measure instance size by the value of n How to apply decrease-and-conquer? U. Aveiro, October

26 The Russian Multiplication Method Trivial case r = m 1 = m n is even r = m n = (2 m) (n / 2) n is odd r = m n = (2 m) ( (n 1) / 2 ) + m Solve an example!! Use a tabular representation Why does it work? / What are we doing? U. Aveiro, October

27 The Russian Multiplication Method Which number, m or n, should be halved? Recursive vs. iterative implementation!! Only simple operations Halving Doubling Adding Complexity? Multiplications / Divisions Additions U. Aveiro, October

28 References A. Levitin, Introduction to the Design and Analysis of Algorithms, 3 rd Ed., Pearson, 2012 Chapter 4 R. Johnsonbaugh and M. Schaefer, Algorithms, Pearson Prentice Hall, 2004 Chapter 4 + Chapter 6 T. H. Cormen et al., Introduction to Algorithms, 3 rd Ed., MIT Press, 2009 Chapter 9 U. Aveiro, October