# CS473 - Algorithms I

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 CS473 - Algorithms I Lecture 9 Sorting in Linear Time View in slide-show mode 1

2 How Fast Can We Sort? The algorithms we have seen so far: Based on comparison of elements We only care about the relative ordering between the elements (not the actual values) The smallest worst-case runtime we have seen so far: O(nlgn) Is O(nlgn) the best we can do? Comparison sorts: Only use comparisons to determine the relative order of elements. 2

3 Decision Trees for Comparison Sorts Represent a sorting algorithm abstractly in terms of a decision tree A binary tree that represents the comparisons between elements in the sorting algorithm Control, data movement, and other aspects are ignored One decision tree corresponds to one sorting algorithm and one value of n (input size) 3

4 Reminder: Insertion Sort (from Lecture 1) Insertion-Sort (A) 1. for j 2 to n do 2. key A[j]; 3. i j - 1; 4. while i > 0 and A[i] > key do 5. A[i+1] A[i]; 6. i i - 1; endwhile 7. A[i+1] key; endfor Iterate over array elts j Loop invariant: The subarray A[1..j-1] is always sorted already sorted j key 4

5 Reminder: Insertion Sort (from Lecture 1) Insertion-Sort (A) 1. for j 2 to n do 2. key A[j]; 3. i j - 1; 4. while i > 0 and A[i] > key do 5. A[i+1] A[i]; 6. i i - 1; endwhile 7. A[i+1] key; endfor Shift right the entries in A[1..j-1] that are > key already sorted < key < key > key > key j j 5

6 Reminder: Insertion Sort (from Lecture 1) Insertion-Sort (A) 1. for j 2 to n do 2. key A[j]; 3. i j - 1; 4. while i > 0 and A[i] > key do 5. A[i+1] A[i]; 6. i i - 1; endwhile 7. A[i+1] key; endfor < key key > key now sorted Insert key to the correct location End of iter j: A[1..j] is sorted j 6

7 Different Outcomes for Insertion Sort and n=3 Input: <a 1, a 2, a 3 > <a 1 a 2 a 3 > <a 1 a 2 a 3 > if a 1 a 2 if a 1 > a 2 <a 2 a 1 a 3 > if a 2 a 3 if a 2 > a 3 if a 1 a 3 if a 1 > a 3 <a 1 a 2 a 3 > <a 1 a 3 a 2 > <a 2 a 1 a 3 > <a 2 a 3 a 1 > if a 1 a 3 if a 1 > a 3 if a 2 a 3 if a 2 > a 3 <a 1 a 3 a 2 > <a 3 a 1 a 2 > <a 2 a 3 a 1 > <a 3 a 2 a 1 > 7

8 Decision Tree for Insertion Sort and n=3 1:2 > 2:3 > 1:3 > <1, 2, 3> 1:3 > <2, 1, 3> 2:3 > <1, 3, 2> <3, 1, 2> <2, 3, 1> <3, 2, 1> 8

9 Decision Tree Model for Comparison Sorts Internal node (i:j): Comparison between elements a i and a j Leaf node: An output of the sorting algorithm Path from root to a leaf: The execution of the sorting algorithm for a given input All possible executions are captured by the decision tree All possible outcomes (permutations) are in the leaf nodes 9

10 Decision Tree for Insertion Sort and n=3 Input: <9, 4, 6> 1:2 > 9 > 4 2:3 > 1:3 > 9 > 6 <1, 2, 3> 1:3 > <2, 1, 3> 2:3 > 4 6 <1, 3, 2> <3, 1, 2> <2, 3, 1> output: <4, 6, 9> <3, 2, 1> 10

11 Decision Tree Model A decision tree can model the execution of any comparison sort: One tree for each input size n View the algorithm as splitting whenever it compares two elements The tree contains the comparisons along all possible instruction traces The running time of the algorithm = the length of the path taken Worst case running time = height of the tree 11

12 Lower Bound for Comparison Sorts Let n be the number of elements in the input array. What is the min number of leaves in the decision tree? n! (because there are n! permutations of the input array, and all possible outputs must be captured in the leaves) What is the max number of leaves in a binary tree of height h? 2 h So, we must have: 2 h n! 12

13 Lower Bound for Decision Tree Sorting Theorem: Any comparison sort algorithm requires Ω(nlgn) comparisons in the worst case. Proof: We ll prove that any decision tree corresponding to a comparison sort algorithm must have height Ω(nlgn) 2 h n! (from previous slide) h lg(n!) lg((n/e) n ) (Stirling s approximation) = nlgn n lge = Ω(nlgn) 13

14 Lower Bound for Decision Tree Sorting Corollary: Heapsort and merge sort are asymptotically optimal comparison sorts. Proof: The O(nlgn) upper bounds on the runtimes for heapsort and merge sort match the Ω(nlgn) worst-case lower bound from the previous theorem. 14

15 Sorting in Linear Time Counting sort: No comparisons between elements Input: A[1.. n], where A[j] {1, 2,, k} Output: B[1.. n], sorted Auxiliary storage: C[1.. k] 15

16 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} A: B: for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] 1 C:

17 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} Step 1: Initialize all counts to 0 A: B: for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] C:

18 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} Step 2: Count the number of occurrences of each value in the input array A: B: j for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] C:

19 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] 1 Step 3: Compute the number of elements less than or equal to each value A: B: i C:

20 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} Step 4: Populate the output array There are C[3] = 3 elts that are 3 j A: for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} B: for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] C:

21 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} Step 4: Populate the output array There are C[4] = 5 elts that are 4 j A: B: 3 for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] C:

22 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} Step 4: Populate the output array There are C[3] = 2 elts that are 3 j A: B: 3 4 for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] C:

23 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} Step 4: Populate the output array There are C[1] = 1 elts that are 1 j A: B: for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] C:

24 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} Step 4: Populate the output array There are C[4] =4 elts that are 4 j A: B: for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] C:

25 Counting Sort for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} After Count Sort: A: B: for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] C:

26 Counting Sort: Runtime Analysis for i 1 to k do C[i] 0 for j 1 to n do C[A[j]] C[A[j]] + 1 // C[i] = {key = i} for i 2 to k do C[i] C[i] + C[i-1] // C[i] = {key i} for j n downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]] 1 Θ(k) Θ(n) Θ(k) Θ(n) Total runtime: Θ(n+k) n: size of the input array k: the range of input values 26

27 Counting Sort: Runtime Runtime is Θ(n+k) If k = O(n), then counting sort takes Θ(n) Question: We proved a lower bound of Θ(nlgn) before! Where is the fallacy? Answer: Θ(nlgn) lower bound is for comparison-based sorting Counting sort is not a comparison sort In fact, not a single comparison between elements occurs! 27

28 Stable Sorting Counting sort is a stable sort: It preserves the input order among equal elements. i.e. The numbers with the same value appear in the output array in the same order as they do in the input array. A: B: Exercise: Which other sorting algorithms have this property? 28

29 Radix Sort Origin: Herman Hollerith s card-sorting machine for the 1890 US Census. Basic idea: Digit-by-digit sorting Two variations: Sort from MSD to LSD (bad idea) Sort from LSD to MSD (good idea) LSD/MSD: Least/most significant digit 29

30 Herman Hollerith ( ) The 1880 U.S. Census took almost 10 years to process. While a lecturer at MIT, Hollerith prototyped punched-card technology. His machines, including a card sorter, allowed the 1890 census total to be reported in 6 weeks. He founded the Tabulating Machine Company in 1911, which merged with other companies in 1924 to form International Business Machines (IBM). 30

31 Hollerith Punched Card 12 rows and 24 columns coded for age, state of residency, gender, etc. Punched card: A piece of stiff paper that contains digital information represented by the presence or absence of holes. 31

32 Hollerith Tabulating Machine and Sorter Mechanically sorts the cards based on the hole locations. Sorting performed for one column at a time Human operator needed to load/retrieve/move cards at each stage 32

33 Hollerith s MSD-First Radix Sort Sort starting from the most significant digit (MSD) Then, sort each of the resulting bins recursively At the end, combine the decks in order sort based on MSD recursive sort recursive sort combine all decks

34 Hollerith s MSD-First Radix Sort To sort a subset of cards recursively: All the other cards need to be removed from the machine, because the machine can handle only one sorting problem at a time. The human operator needs to keep track of the intermediate card piles to sort these two cards recursively, remove all the other cards from the machine intermediate pile 457, 436, 657, 720,

35 Hollerith s MSD-First Radix Sort MSD-first sorting may require: -- very large number of sorting passes -- very large number of intermediate card piles to maintain S(d): # of passes needed to sort d-digit numbers (worst-case) Recurrence: S(d) = 10 S(d-1) + 1 with S(1) = 1 Reminder: Recursive call made to each subset with the same most significant digit (MSD) 35

36 Hollerith s MSD-First Radix Sort Recurrence: S(d) = 10S(d-1) + 1 S(d) = 10 S(d-1) + 1 = 10 (10 S(d-2) + 1) + 1 = 10 (10 (10 S(d-3) + 1) + 1) + 1 = 10 i S(d-i) + 10 i i Iteration terminates when i = d-1 with S(d-(d-1)) = S(1) = 1 36

37 Hollerith s MSD-First Radix Sort P(d): # of intermediate card piles maintained (worst-case) Reminder: Each routing pass generates 9 intermediate piles except the sorting passes on least significant digits (LSDs) There are 10 d-1 sorting calls to LSDs P(d) = 9 (S(d) 10 d-1 ) = 9 ((10 d 1)/9 10 d-1 ) = (10 d d-1 ) = 10 d-1-1 P(d) = 10 d-1-1 Alternative solution: Solve the recurrence: P(d) = 10P(d-1) + 9 P(1) = 0 37

38 Hollerith s MSD-First Radix Sort Example: To sort 3 digit numbers, in the worst case: S(d) = (1/9) (10 3-1) = 111 sorting passes needed P(d) = 10 d-1-1 = 99 intermediate card piles generated MSD-first approach has more recursive calls and intermediate storage requirement Expensive for a tabulating machine to sort punched cards Overhead of recursive calls in a modern computer 38

39 LSD-First Radix Sort Least significant digit (LSD)-first radix sort seems to be a folk invention originated by machine operators. It is the counter-intuitive, but the better algorithm. Basic algorithm: Sort numbers on their LSD first Combine the cards into a single deck in order Continue this sorting process for the other digits from the LSD to MSD Requires only d sorting passes No intermediate card pile generated Stable sorting needed!!! 39

40 LSD-first Radix Sort: Example Step 1: Sort 1 st digit Step 2: Sort 2 nd digit Step 3: Sort 3 rd digit

41 Correctness of Radix Sort (LSD-first) Proof by induction: Base case: d=1 is correct (trivial) Inductive hyp: Assume the first d-1 digits are sorted correctly Prove that all d digits are sorted correctly after sorting digit d sort based on digit d last 2 digits sorted due to ind. hyp Two numbers that differ in digit d are correctly sorted (e.g. 355 and 657) Two numbers equal in digit d are put in the same order as the input correct order 41

42 Radix Sort: Runtime Use counting-sort to sort each digit Reminder: Counting sort complexity: Θ(n+k) n: size of input array k: the range of the values Radix sort runtime: Θ(d(n+k)) d: # of digits How to choose the d and k? 42

43 Radix Sort: Runtime Example 1 We have flexibility in choosing d and k Assume we are trying to sort 32-bit words We can define each digit to be 4 bits Then, the range for each digit k = 2 4 = 16 So, counting sort will take Θ(n+16) The number of digits d = 32/4 = 8 Radix sort runtime: Θ(8n+16) = Θ(n) 4 bits 4 bits 4 bits 4 bits 4 bits 4 bits 4 bits 4 bits 32-bit 43

44 Radix Sort: Runtime Example 2 We have flexibility in choosing d and k Assume we are trying to sort 32-bit words Or, we can define each digit to be 8 bits Then, the range for each digit k = 2 8 = 256 So, counting sort will take Θ(n+256) The number of digits d = 32/8 = 4 Radix sort runtime: Θ(4n+256) = Θ(n) 8 bits 8 bits 8 bits 8 bits 32-bit 44

45 Radix Sort: Runtime Assume we are trying to sort b-bit words Define each digit to be r bits Then, the range for each digit k = 2 r So, counting sort will take Θ(n+2 r ) The number of digits d = b/r Radix sort runtime: b/r digits r bits r bits r bits r bits b-bit 45

46 Radix Sort: Runtime Analysis Minimize T(n, b) by differentiating and setting to 0 Or, intuitively: We want to balance the terms (b/r) and (n + 2 r ) Choose r lgn If we choose r << lgn (n + 2 r ) term doesn t improve If we choose r >> lgn (n + 2 r ) increases exponentially 46

47 Radix Sort: Runtime Analysis Choose r = lgn T(n, b) = Θ(bn/lgn) For numbers in the range from 0 to n d 1, we have: The number of bits b = lg(n d ) = d lgn Radix sort runs in Θ(dn) 47

48 Radix Sort: Conclusions Choose r = lgn T(n, b) = Θ(bn/lgn) Example: Compare radix sort with merge sort/heapsort 1 million (2 20 ) 32-bit numbers (n = 2 20, b = 32) Radix sort: 32/20 = 2 passes Merge sort/heap sort: lgn = 20 passes Downsides: Radix sort has little locality of reference (more cache misses) The version that uses counting sort is not in-place On modern processors, a well-tuned quicksort implementation typically runs faster. 48

### CS473 - Algorithms I

CS473 - Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm View in slide-show mode 1 Reminder: Merge Sort Input array A sort this half sort this half Divide Conquer merge two sorted halves Combine

### Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * *

Binary Heaps A binary heap is another data structure. It implements a priority queue. Priority Queue has the following operations: isempty add (with priority) remove (highest priority) peek (at highest

### Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

### Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Quiz 1 Quiz 1 Do not open this quiz booklet until you are directed

### Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

### How fast can we sort? Sorting. Decision-tree model. Decision-tree for insertion sort Sort a 1, a 2, a 3. CS 3343 -- Spring 2009

CS 4 -- Spring 2009 Sorting Crol Wenk Slides courtesy of Chrles Leiserson with smll chnges by Crol Wenk CS 4 Anlysis of Algorithms 1 How fst cn we sort? All the sorting lgorithms we hve seen so fr re comprison

### Analysis of Algorithms I: Binary Search Trees

Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary

### CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

### A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called heap-order property

CmSc 250 Intro to Algorithms Chapter 6. Transform and Conquer Binary Heaps 1. Definition A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called

### 6 March 2007 1. Array Implementation of Binary Trees

Heaps CSE 0 Winter 00 March 00 1 Array Implementation of Binary Trees Each node v is stored at index i defined as follows: If v is the root, i = 1 The left child of v is in position i The right child of

### Mergesort and Quicksort

Mergesort Mergesort and Quicksort Basic plan: Divide array into two halves. Recursively sort each half. Merge two halves to make sorted whole. mergesort mergesort analysis quicksort quicksort analysis

### Quicksort An Example. When lo and hi cross, we are done. The final value of hi is the position in which the partitioning element ends up.

Quicksort An Example We sort the array A = (38 81 22 48 13 69 93 14 45 58 79 72) with quicksort, always choosing the pivot element to be the element in position (left+right)/2. The partitioning during

### Sorting Algorithms. Nelson Padua-Perez Bill Pugh. Department of Computer Science University of Maryland, College Park

Sorting Algorithms Nelson Padua-Perez Bill Pugh Department of Computer Science University of Maryland, College Park Overview Comparison sort Bubble sort Selection sort Tree sort Heap sort Quick sort Merge

### Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

### CSC148 Lecture 8. Algorithm Analysis Binary Search Sorting

CSC148 Lecture 8 Algorithm Analysis Binary Search Sorting Algorithm Analysis Recall definition of Big Oh: We say a function f(n) is O(g(n)) if there exists positive constants c,b such that f(n)

### Data Structures. Algorithm Performance and Big O Analysis

Data Structures Algorithm Performance and Big O Analysis What s an Algorithm? a clearly specified set of instructions to be followed to solve a problem. In essence: A computer program. In detail: Defined

### Analysis of Binary Search algorithm and Selection Sort algorithm

Analysis of Binary Search algorithm and Selection Sort algorithm In this section we shall take up two representative problems in computer science, work out the algorithms based on the best strategy to

### Divide-and-Conquer Algorithms Part Four

Divide-and-Conquer Algorithms Part Four Announcements Problem Set 2 due right now. Can submit by Monday at 2:15PM using one late period. Problem Set 3 out, due July 22. Play around with divide-and-conquer

### The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

### Binary Heaps. CSE 373 Data Structures

Binary Heaps CSE Data Structures Readings Chapter Section. Binary Heaps BST implementation of a Priority Queue Worst case (degenerate tree) FindMin, DeleteMin and Insert (k) are all O(n) Best case (completely

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### External Sorting. Chapter 13. Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1

External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing

### Binary Search. Search for x in a sorted array A.

Divide and Conquer A general paradigm for algorithm design; inspired by emperors and colonizers. Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3.

### From Last Time: Remove (Delete) Operation

CSE 32 Lecture : More on Search Trees Today s Topics: Lazy Operations Run Time Analysis of Binary Search Tree Operations Balanced Search Trees AVL Trees and Rotations Covered in Chapter of the text From

### CS 102: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4)

CS 10: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4) Problem 1. a. Consider the modified binary search algorithm so that it splits the input not into two sets of almost-equal sizes, but into three

### Randomized algorithms

Randomized algorithms March 10, 2005 1 What are randomized algorithms? Algorithms which use random numbers to make decisions during the executions of the algorithm. Why would we want to do this?? Deterministic

### Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4

Divide d Conquer Textook Reading Chapters 4, 7 & 33.4 Overview Design principle Divide d conquer Proof technique Induction, induction, induction Analysis technique Recurrence relations Prolems Sorting

### Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort).

Chapter 7: Quicksort Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort). The three steps of Quicksort are as follows:

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### Binary Heap Algorithms

CS Data Structures and Algorithms Lecture Slides Wednesday, April 5, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks CHAPPELLG@member.ams.org 2005 2009 Glenn G. Chappell

### Algorithms. Margaret M. Fleck. 18 October 2010

Algorithms Margaret M. Fleck 18 October 2010 These notes cover how to analyze the running time of algorithms (sections 3.1, 3.3, 4.4, and 7.1 of Rosen). 1 Introduction The main reason for studying big-o

### External Sorting. Why Sort? 2-Way Sort: Requires 3 Buffers. Chapter 13

External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing

### CS/COE 1501 http://cs.pitt.edu/~bill/1501/

CS/COE 1501 http://cs.pitt.edu/~bill/1501/ Lecture 01 Course Introduction Meta-notes These notes are intended for use by students in CS1501 at the University of Pittsburgh. They are provided free of charge

### Algorithms and Data Structures

Algorithms and Data Structures CMPSC 465 LECTURES 20-21 Priority Queues and Binary Heaps Adam Smith S. Raskhodnikova and A. Smith. Based on slides by C. Leiserson and E. Demaine. 1 Trees Rooted Tree: collection

### Questions 1 through 25 are worth 2 points each. Choose one best answer for each.

Questions 1 through 25 are worth 2 points each. Choose one best answer for each. 1. For the singly linked list implementation of the queue, where are the enqueues and dequeues performed? c a. Enqueue in

### Binary Search Trees CMPSC 122

Binary Search Trees CMPSC 122 Note: This notes packet has significant overlap with the first set of trees notes I do in CMPSC 360, but goes into much greater depth on turning BSTs into pseudocode than

### 1 2-3 Trees: The Basics

CS10: Data Structures and Object-Oriented Design (Fall 2013) November 1, 2013: 2-3 Trees: Inserting and Deleting Scribes: CS 10 Teaching Team Lecture Summary In this class, we investigated 2-3 Trees in

### Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

### Data Structure [Question Bank]

Unit I (Analysis of Algorithms) 1. What are algorithms and how they are useful? 2. Describe the factor on best algorithms depends on? 3. Differentiate: Correct & Incorrect Algorithms? 4. Write short note:

### Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

### Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new

Reminder: Complexity (1) Parallel Complexity Theory Lecture 6 Number of steps or memory units required to compute some result In terms of input size Using a single processor O(1) says that regardless of

### Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new GAP (2) Graph Accessibility Problem (GAP) (1)

Reminder: Complexity (1) Parallel Complexity Theory Lecture 6 Number of steps or memory units required to compute some result In terms of input size Using a single processor O(1) says that regardless of

### CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

### THIS CHAPTER studies several important methods for sorting lists, both contiguous

Sorting 8 THIS CHAPTER studies several important methods for sorting lists, both contiguous lists and linked lists. At the same time, we shall develop further tools that help with the analysis of algorithms

### Small Maximal Independent Sets and Faster Exact Graph Coloring

Small Maximal Independent Sets and Faster Exact Graph Coloring David Eppstein Univ. of California, Irvine Dept. of Information and Computer Science The Exact Graph Coloring Problem: Given an undirected

### CMSC 451 Design and Analysis of Computer Algorithms 1

CMSC 4 Design and Analysis of Computer Algorithms David M. Mount Department of Computer Science University of Maryland Fall 003 Copyright, David M. Mount, 004, Dept. of Computer Science, University of

### Cpt S 223. School of EECS, WSU

Priority Queues (Heaps) 1 Motivation Queues are a standard mechanism for ordering tasks on a first-come, first-served basis However, some tasks may be more important or timely than others (higher priority)

### IMPROVING PERFORMANCE OF RANDOMIZED SIGNATURE SORT USING HASHING AND BITWISE OPERATORS

Volume 2, No. 3, March 2011 Journal of Global Research in Computer Science RESEARCH PAPER Available Online at www.jgrcs.info IMPROVING PERFORMANCE OF RANDOMIZED SIGNATURE SORT USING HASHING AND BITWISE

### AP Computer Science AB Syllabus 1

AP Computer Science AB Syllabus 1 Course Resources Java Software Solutions for AP Computer Science, J. Lewis, W. Loftus, and C. Cocking, First Edition, 2004, Prentice Hall. Video: Sorting Out Sorting,

### ON THE COMPLEXITY OF THE GAME OF SET. {kamalika,pbg,dratajcz,hoeteck}@cs.berkeley.edu

ON THE COMPLEXITY OF THE GAME OF SET KAMALIKA CHAUDHURI, BRIGHTEN GODFREY, DAVID RATAJCZAK, AND HOETECK WEE {kamalika,pbg,dratajcz,hoeteck}@cs.berkeley.edu ABSTRACT. Set R is a card game played with a

### Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

### GRAPH THEORY LECTURE 4: TREES

GRAPH THEORY LECTURE 4: TREES Abstract. 3.1 presents some standard characterizations and properties of trees. 3.2 presents several different types of trees. 3.7 develops a counting method based on a bijection

### Analysis of Algorithms I: Optimal Binary Search Trees

Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search

### Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

### Today s Outline. Exercise. Binary Search Analysis. Linear Search Analysis. Asymptotic Analysis. Analyzing Code. Announcements. Asymptotic Analysis

Today s Outline Announcements Assignment #1 due Thurs, Oct 7 at 11:45pm Asymptotic Analysis Asymptotic Analysis CSE 7 Data Structures & Algorithms Ruth Anderson Autumn 2010 Exercise Analyzing Code bool

### root node level: internal node edge leaf node CS@VT Data Structures & Algorithms 2000-2009 McQuain

inary Trees 1 A binary tree is either empty, or it consists of a node called the root together with two binary trees called the left subtree and the right subtree of the root, which are disjoint from each

### Computer Science 210: Data Structures. Searching

Computer Science 210: Data Structures Searching Searching Given a sequence of elements, and a target element, find whether the target occurs in the sequence Variations: find first occurence; find all occurences

### Divide And Conquer Algorithms

CSE341T/CSE549T 09/10/2014 Lecture 5 Divide And Conquer Algorithms Recall in last lecture, we looked at one way of parallelizing matrix multiplication. At the end of the lecture, we saw the reduce SUM

### Loop Invariants and Binary Search

Loop Invariants and Binary Search Chapter 4.3.3 and 9.3.1-1 - Outline Ø Iterative Algorithms, Assertions and Proofs of Correctness Ø Binary Search: A Case Study - 2 - Outline Ø Iterative Algorithms, Assertions

### Previous Lectures. B-Trees. External storage. Two types of memory. B-trees. Main principles

B-Trees Algorithms and data structures for external memory as opposed to the main memory B-Trees Previous Lectures Height balanced binary search trees: AVL trees, red-black trees. Multiway search trees:

### Algorithms Chapter 12 Binary Search Trees

Algorithms Chapter 1 Binary Search Trees Outline Assistant Professor: Ching Chi Lin 林 清 池 助 理 教 授 chingchi.lin@gmail.com Department of Computer Science and Engineering National Taiwan Ocean University

### Converting a Number from Decimal to Binary

Converting a Number from Decimal to Binary Convert nonnegative integer in decimal format (base 10) into equivalent binary number (base 2) Rightmost bit of x Remainder of x after division by two Recursive

### DESIGN AND ANALYSIS OF ALGORITHMS

CPS 3 DESIGN AND ANALYSIS OF ALGORITHMS Fall 8 Instructor: Herbert Edelsbrunner Teaching Assistant: Zhiqiang Gu CPS 3 Fall Semester of 8 Table of Contents Introduction 3 I DESIGN TECHNIQUES Divide-and-Conquer

### Section IV.1: Recursive Algorithms and Recursion Trees

Section IV.1: Recursive Algorithms and Recursion Trees Definition IV.1.1: A recursive algorithm is an algorithm that solves a problem by (1) reducing it to an instance of the same problem with smaller

### Persistent Data Structures

6.854 Advanced Algorithms Lecture 2: September 9, 2005 Scribes: Sommer Gentry, Eddie Kohler Lecturer: David Karger Persistent Data Structures 2.1 Introduction and motivation So far, we ve seen only ephemeral

### Solutions for Introduction to algorithms second edition

Solutions for Introduction to algorithms second edition Philip Bille The author of this document takes absolutely no responsibility for the contents. This is merely a vague suggestion to a solution to

### Binary search algorithm

Binary search algorithm Definition Search a sorted array by repeatedly dividing the search interval in half. Begin with an interval covering the whole array. If the value of the search key is less than

### 6.852: Distributed Algorithms Fall, 2009. Class 2

.8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election

### CS711008Z Algorithm Design and Analysis

CS711008Z Algorithm Design and Analysis Lecture 7 Binary heap, binomial heap, and Fibonacci heap 1 Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 The slides were

### Sequential Data Structures

Sequential Data Structures In this lecture we introduce the basic data structures for storing sequences of objects. These data structures are based on arrays and linked lists, which you met in first year

### Algorithm Design and Analysis

Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;

### 5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### Binary Search Trees. A Generic Tree. Binary Trees. Nodes in a binary search tree ( B-S-T) are of the form. P parent. Key. Satellite data L R

Binary Search Trees A Generic Tree Nodes in a binary search tree ( B-S-T) are of the form P parent Key A Satellite data L R B C D E F G H I J The B-S-T has a root node which is the only node whose parent

### Module: Software Instruction Scheduling Part I

Module: Software Instruction Scheduling Part I Sudhakar Yalamanchili, Georgia Institute of Technology Reading for this Module Loop Unrolling and Instruction Scheduling Section 2.2 Dependence Analysis Section

### Algorithms and Data Structures

Algorithm Analysis Page 1 BFH-TI: Softwareschule Schweiz Algorithm Analysis Dr. CAS SD01 Algorithm Analysis Page 2 Outline Course and Textbook Overview Analysis of Algorithm Pseudo-Code and Primitive Operations

### Lecture 7: Approximation via Randomized Rounding

Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

### The Relative Worst Order Ratio for On-Line Algorithms

The Relative Worst Order Ratio for On-Line Algorithms Joan Boyar 1 and Lene M. Favrholdt 2 1 Department of Mathematics and Computer Science, University of Southern Denmark, Odense, Denmark, joan@imada.sdu.dk

### Data Warehousing und Data Mining

Data Warehousing und Data Mining Multidimensionale Indexstrukturen Ulf Leser Wissensmanagement in der Bioinformatik Content of this Lecture Multidimensional Indexing Grid-Files Kd-trees Ulf Leser: Data

### 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008

MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

### Data Structures and Algorithm Analysis (CSC317) Intro/Review of Data Structures Focus on dynamic sets

Data Structures and Algorithm Analysis (CSC317) Intro/Review of Data Structures Focus on dynamic sets We ve been talking a lot about efficiency in computing and run time. But thus far mostly ignoring data

### Exam study sheet for CS2711. List of topics

Exam study sheet for CS2711 Here is the list of topics you need to know for the final exam. For each data structure listed below, make sure you can do the following: 1. Give an example of this data structure

### Scalable Data Analysis in R. Lee E. Edlefsen Chief Scientist UserR! 2011

Scalable Data Analysis in R Lee E. Edlefsen Chief Scientist UserR! 2011 1 Introduction Our ability to collect and store data has rapidly been outpacing our ability to analyze it We need scalable data analysis

### CSE 326: Data Structures B-Trees and B+ Trees

Announcements (4//08) CSE 26: Data Structures B-Trees and B+ Trees Brian Curless Spring 2008 Midterm on Friday Special office hour: 4:-5: Thursday in Jaech Gallery (6 th floor of CSE building) This is

### Balanced Binary Search Tree

AVL Trees / Slide 1 Balanced Binary Search Tree Worst case height of binary search tree: N-1 Insertion, deletion can be O(N) in the worst case We want a tree with small height Height of a binary tree with

### Data Structures Fibonacci Heaps, Amortized Analysis

Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure:

Lecture 10 Union-Find The union-nd data structure is motivated by Kruskal's minimum spanning tree algorithm (Algorithm 2.6), in which we needed two operations on disjoint sets of vertices: determine whether

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### Analysis of Algorithms, I

Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications

### Binary Trees and Huffman Encoding Binary Search Trees

Binary Trees and Huffman Encoding Binary Search Trees Computer Science E119 Harvard Extension School Fall 2012 David G. Sullivan, Ph.D. Motivation: Maintaining a Sorted Collection of Data A data dictionary

### B-Trees. Algorithms and data structures for external memory as opposed to the main memory B-Trees. B -trees

B-Trees Algorithms and data structures for external memory as opposed to the main memory B-Trees Previous Lectures Height balanced binary search trees: AVL trees, red-black trees. Multiway search trees:

### Dynamic Programming. Lecture 11. 11.1 Overview. 11.2 Introduction

Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach

### Scheduling. Open shop, job shop, flow shop scheduling. Related problems. Open shop, job shop, flow shop scheduling

Scheduling Basic scheduling problems: open shop, job shop, flow job The disjunctive graph representation Algorithms for solving the job shop problem Computational complexity of the job shop problem Open

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### Heaps & Priority Queues in the C++ STL 2-3 Trees

Heaps & Priority Queues in the C++ STL 2-3 Trees CS 3 Data Structures and Algorithms Lecture Slides Friday, April 7, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks

### CPSC 211 Data Structures & Implementations (c) Texas A&M University [ 313]

CPSC 211 Data Structures & Implementations (c) Texas A&M University [ 313] File Structures A file is a collection of data stored on mass storage (e.g., disk or tape) Why on mass storage? too big to fit

### The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >