# Cpt S 223. School of EECS, WSU

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Sorting Algorithms 1

2 Sorting methods Comparison based sorting On 2 ) methods E.g., Insertion, bubble Average time On log n) methods E.g., quick sort On logn) methods E.g., Merge sort, heap sort on-comparison based sorting Integer sorting: linear time E.g., Counting sort, bin sort Radix sort, bucket sort Stable vs. non-stable sorting 2

3 Insertion sort: snapshot at a given iteration comparisons Worst-case run-time complexity: n 2 ) Best-case run-time complexity: n) ) When? When? Image courtesy: McQuain WD, VA Tech,

4 The Divide and Conquer Technique Input: A problem of size n Recursive At each level of recursion: Divide) Split the problem of size n into a fixed number of subproblems of smaller sizes, and solve each sub-problem recursively Conquer) Merge the answers to the sub-problems 4

5 Two Divide & Conquer sorts Merge sort Divide is trivial Merge i.e, conquer) does all the work Quick sort Partition i.e, Divide) does all the work Merge i.e, conquer) is trivial 5

6 Merge Sort Main idea: Dividing is trivial Merging M i is non-trivial ti i Input divide) Olg g n) steps How much work at every step? On) sub-problems conquer) Olg n) steps How much work at every step? 6 Image courtesy: McQuain WD, VA Tech, 2004

7 How to merge two sorted arrays? i j A A B[k++] =Populate min{ A1[i], A2[j] } 2. Advance the minimum contributing gpointer B Temporary array to hold the output k n) time Do you always need the temporary array B to store the output, or can you do this inplace? 7

8 Merge Sort : Analysis Merge Sort takes n lg n) time Proof: Let Tn) be the time taken to merge sort n elements Time for each comparison operation=o1) Main observation: To merge two sorted arrays of size n/2, it takes n comparisons at most. Therefore: Tn) = 2 Tn/2) + n Solving the above recurrence: Tn) = 2 [ 2 Tn/2 2 ) + n/2 ] + n = 2 2 Tn/2 2 ) + 2n k times) = 2 k Tn/2 k ) + kn At k = lg n, Tn/2 k ) = T1) = 1 termination case) ==> Tn) = n lg n) 8

9 QuickSort Main idea: Dividing partitioning ) is non-trivial Merging M i is trivial i Divide-and-conquer approach to sorting Like MergeSort, except Don t divide the array in half Partition the array based elements being less than or greater than some element of the array the pivot) i.e., divide phase does all the work; merge phase is trivial. Worst case running time O 2 ) Average case running time O log ) Fastest generic sorting algorithm in practice Even faster if use simple sort e.g., InsertionSort) when array becomes small 9

10 QuickSort Algorithm QuickSort Array: S) 1. If size of S is 0 or 1, return 2. Pivot = Pick an element v in S Q) What s the best way to pick this element? arbitrary? Median? etc) 1. Partition S {v} into two disjoint groups S1 = {x S {v}) x < v} S2 = {x S {v}) x > v} 2. Return QuickSortS1), followed by v, followed by QuickSortS2) 10

11 QuickSort Example 11

12 QuickSort vs. MergeSort Main problem with quicksort: QuickSort may end up dividing the input array into subproblems bl of size 1 and -1 1in the worst case, at every recursive step unlike merge sort which always divides into two halves) When can this happen? Leading to O 2 ) performance =>eed to choose pivot wisely but efficiently) MergeSort is typically implemented using a temporary array for merge step) QuickSort can partition the array in place 12

13 Goal: A good pivot is one that creates two even sized partitions => Median will be best, but finding median could be as tough as sorting itself Picking the Pivot How about choosing the first element? What if array already or nearly sorted? Good for a randomly populated array How about choosing a random element? Good in practice if truly random Still possible to get some bad choices Requires execution of random number generator 13

14 Picking the Pivot Best choice of pivot Median of array But median is expensive to calculate will result in ext strategy: Approximate the median Estimate median as the median of any three elements Median = median {first, middle, last} will result in Has been shown to reduce running time comparisons) by 14% pivot 14

15 How to write the partitioning code? should result in Goal of partitioning: i) Move all elements < pivot to the left of pivot ii) Move all elements > pivot to the right of pivot Partitioning is conceptually straightforward, but easy to do inefficiently i One bad way: Do one pass to figure out how many elements should be on either side of pivot Then create a temp array to copy elements relative to pivot 15

16 should result in Partitioning strategy A good strategy to do partition : do it in place // Swap pivot with last element S[right] i = left j = right 1) While i < j) { // advance i until first element > pivot // decrement j until first element < pivot OK to also swap with S[left] but then the rest of the code should change accordingly // swap A[i] & A[j] only if i<j) } Swap pivot, S[i] ) This is called in place because all operations are done in place of the input array i.e., without Cpt S 223. School of creating EECS, WSUtemp array) 16

17 Partitioning Strategy An in place partitioning algorithm Swap pivot with last element S[right] i = left j = right 1) while i < j) { i++; } until S[i] > pivot { j--; } until S[j] < pivot If i < j), then swap S[i], S[j] ) Swap pivot, S[i] ) eeds a few boundary case handling 17

18 Median of three approach to picking the pivot: => compares the first, last and middle elements and pick the median of those three pivot = min{8,6,0} = 6 Partitioning Example left right Swap pivot with last element S[right] i = left j = right 1) Initial array Swap pivot; initialize i and j i j Move i and j inwards until conditions violated i j Positioned to swap After first swap i j swapped While i < j) { { i++; } until S[i] > pivot { j--; } until S[j] < pivot If i <j) j), then swap S[i], S[j] ) } 18

19 Partitioning Example cont.) After a few steps Before second swap i j After second swap i j i has crossed j j i After final swap with pivot i p Swap pivot, S[i] ) 19

20 Handling Duplicates What happens if all input elements are equal? Special case: Current approach: { i++; } until S[i] > pivot { j--; } until S[j] < pivot What will happen? i will advance all the way to the right end j will advance all the way to the left end => pivot will remain in the right position, creating the left partition to contain -1 elements and empty right partition Worst case O 2 ) performance 20

21 Handling Duplicates A better code Don t skip elements equal to pivot { i++; } until S[i] pivot { j--; } until S[j] pivot Special case: What will happen now? Adds some unnecessary swaps But results in perfect partitioning for array of identical elements Unlikely for input array, but more likely for recursive calls to QuickSort 21

22 Small Arrays When S is small, recursive calls become expensive overheads) General strategy When size < threshold, use a sort more efficient for small arrays e.g., InsertionSort) Good thresholds range from 5 to 20 Also avoids issue with finding median-of-three pivot for array of size 2 or less Has been shown to reduce running time by 15% 22

23 QuickSort Implementation left right 23

24 QuickSort Implementation L C R L C R L C R L C R L C P R 24

25 Assign pivot as median of 3 partition based on pivot Swap should be compiled inline. Recursively sort partitions 25

26 Analysis of QuickSort Let T) = time to quicksort elements Let L = #elements in left partition => #elements in right partition = -L-1 Base: T0) = T1) = O1) T) = TL) + T L 1) + O) Time to Time to Time for partitioning i sort left sort right at current recursive partition partition step 26

27 Analysis of QuickSort Worst-case analysis Pivot is the smallest element L = 0) Pivot is the smallest element L 0) O T O T O T T T ) 1) 1) ) ) 1) 0) ) O O T T O T T ) 1) 2) ) ) 1) ) ) ) ) ) O O i T O O O T T 2 ) ) ) ) 1) 2) 3) ) ) ) ) ) i O O i T 1 ) ) )

28 Analysis of QuickSort Best-case analysis Pivot is the median sorted rank = /2) T ) T / 2) T / 2) O ) T ) 2 T / 2) O ) T ) O log ) Average-case analysis Assuming each partition equally likely T) = O log ) HOW? 28

29 QuickSort: Avg Case Analysis T) = TL) + T-L-1) + O) All partition sizes are equally likely => Avg TL) = Avg T-L-1) = 1/ -1 j=0 Tj) => Avg T) = 2/ [ -1 j=0 Tj) ] + c => T) = 2 [ -1 j=0 Tj) ] + c 2 => 1) Substituting by -1 =>-1) T-1) = 2 [ j=0-2 Tj) ] + c-1) 2 => 2) 1)-2)2) => T) - -1)T-1) = 2 T-1) + c 2-1) 29

30 Avg case analysis T) = +1)T-1) + c 2-1) T)/+1) T-1)/ + c2/+1) Telescope, by substituting with -1, -2, -3,.. 2 T) = O log ) 30

31 Comparison Sorting Sort Worst Case Average Case Best Case Comments InsertionSort Θ 2 ) Θ 2 ) Θ) ) Fast for small MergeSort Θ log ) Θ log ) Θ log ) Requires memory HeapSort Θ log ) Θ log ) Θ log ) Large constants QuickSort Θ 2 ) Θ log ) Θ log ) Small constants 31

32 Comparison Sorting Good sorting applets Sorting benchmark: 32

33 Lower Bound on Sorting What is the best we can do on comparison based sorting? Best worst-case sorting algorithm so far) is O log ) Can we do better? Can we prove a lower bound on the sorting problem, independent of the algorithm? For comparison sorting, no, we can t do better than O log ) Can show lower bound of Ω log ) 33

34 Proving lower bound on sorting using Decision Trees A decision i tree is a binary tree where: Each node lists all left-out open possibilities for deciding) Path of each node represents a decided sorted prefix of elements Each branch represents an outcome of a particular comparison Each leaf represents a particular ordering of the original array elements 34

35 Root = all open possibilities A decision tree to sort three elements {a,b,c} assuming no duplicates) all remaining open possibilities IF a<c: possible IF a<b: possible Height = lg n!) Worst-case evaluation path for any algorithm n! leaves in this tree 35

36 Decision Tree for Sorting The logic of any sorting algorithm that uses comparisons can be represented by a decision tree In the worst case, the number of comparisons used by the algorithm equals the HEIGHT OF THE DECISIO TREE In the average case, the number of comparisons is the average of the depths of all leaves There are! different orderings of elements 36

37 Lower Bound for Comparison Sorting Lemma: A binary tree with L leaves must have depth at least ceillg L) Sorting s decision tree has! leaves Theorem: Any comparison sort may require at least log!) comparisons in the worst case 37

38 Lower Bound for Comparison Sorting Theorem: Any comparison sort requires Ω log ) comparisons Proof uses Stirling s approximation)! 2 / e ) 1 1/ ))! log!) log!) / e) log!) log log ) log ) log e log ) 38

39 Implications of the sorting lower bound theorem Comparison based sorting cannot be achieved in less than n lg n) steps => Merge sort, Heap sort are optimal => Quick sort is not optimal but pretty good as optimal in practice => Insertion sort, bubble sort are clearly sub- optimal, even in practice 39

40 on comparison based sorting Integer sorting e.g., Counting sort Bucket sort Radix sort 40

41 on comparison based Integer Sorting Some input properties allow to eliminate the need for comparison E.g., sorting an employee database by age of employees Counting Sort Given array A[1..], where 1 A[i] M Create array C of size M, where C[i] is the number of i s in A Use C to place elements into new sorted array B Running time Θ+M) = Θ) if M = Θ) 41

42 Counting Sort: Example Input A: =10 M=4 all elements in input between 0 and 3) Count array C: Output sorted array: Time = O + M) If M < ), Time = O) 42

43 Stable vs. nonstable sorting A stable sorting method is one which preserves the original input order among duplicates in the output Input: Output: Useful when each data is a struct of form { key, value } 43

44 How to make counting sort stable? one approach) Input A: =10 M=4 all elements in input between 0 and 3) Count array C: Output sorted array: But this algorithm is OT in-place! Can we make counting sort in place? i.e., without using another array or linked list) 44

45 How to make counting sort in place? void CountingSort_InPlacevector a, int n) { 1. First construct Count array C s.t C[i] stores the last index in the bin corresponding to key i, where the next instance of i should be written to. Then do the following: } i=0; whilei<n) { e=a[i]; if c[e] has gone below range, then continue after i++; ifi==c[e]) i++; tmp = A[c[e]]; [ ote: This code has to keep track A[c[e]--] = e; of the valid range for A[i] = tmp; each key } 45

46 C: End points A:

47 Bucket sort Assume elements of A uniformly distributed over the range [0,1] Create M equal-sized buckets over [0,1], s.t., M Add each element of A into appropriate bucket Sort each bucket internally Can use recursion here, or Can use something like InsertionSort Return concatentation of buckets Average case running time Θ) assuming each bucket will contain Θ1) elements 47

48 Radix Sort Radix sort achieves stable sorting To sort each column, use counting sort On)) => To sort k columns, Onk) time Sort numbers, each with k bits E.g, input {4, 1, 0, 10, 5, 6, 1, 8} lsb msb 48

49 External Sorting What if the number of elements we wish to sort do not fit in memory? Obviously, our existing sort algorithms are inefficient Each comparison potentially requires a disk access Once again, we want to minimize disk accesses 49

50 External MergeSort = number of elements in array A[1..] to be sorted M = number of elements that fit in memory at any given time K= / M CPU RAM k dis M Array: A [ 1.. ] 50

51 External MergeSort Approach OM log M) 1. Read in M amount of A, sort it using local sorting e.g., quicksort), and write it back to disk OKM log M) 2. Repeat above K times until all of A processed 3. Create K input buffers and 1 output buffer, each of size M/K+1) 4. Perform a K-way merge: O log k) 1. Update input buffers one disk-page at a time 2. Write output buffer one disk-page at a time How? K input buffers 1 output buffer 1) 4.2) 51

52 K-way merge In memory: Q) How to merge k sorted arrays of total t size in O lg k) time? For external merge sort: r = M/k+1) and k i=0k L i = M 1 L 1 L 2 L 3 L k sorted Merge & sort??? r output 1 r 52

53 K-way merge a simple algo Input: sorted 1 r L 4 L k-1 Sum of sorted list lengths = = kr) L 1 L 2 L 3 L k + Output merged arrays temp) 2r + Q) What is the problem with this approach? k-1 stages Total time = 2r + 3r + 4r + 5r + + kr = Ok 2 r) = Ok) 3r Even worse, if individual input arrays are of variable sizes +. We can do better than this! 53

54 K-way merge a better algo Input: sorted 1 r L 4 L k-1 Sum of sorted list lengths = = kr) L 1 L 2 L 3 L k Output merged arrays temp) 2r 4r + 2r 4r + 2r Run-time Analysis lg k stages Total time = ) + ) + : lg k times = O lg k) +. 54

55 External MergeSort Computational time T,M): =OK*MlogM)+OlogK) M) O K) = O/M)*M log M)+O log K) = O log M + log K) = O log M) Disk accesses all sequential) P = page size Accesses = O/P) 55

56 Sorting: Summary eed for sorting is ubiquitous in software Optimizing the sort algorithm to the domain is essential Good general-purpose p algorithms available QuickSort Optimizations continue Sort benchmarks

### Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

### CS473 - Algorithms I

CS473 - Algorithms I Lecture 9 Sorting in Linear Time View in slide-show mode 1 How Fast Can We Sort? The algorithms we have seen so far: Based on comparison of elements We only care about the relative

### Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

### Sorting Algorithms. Nelson Padua-Perez Bill Pugh. Department of Computer Science University of Maryland, College Park

Sorting Algorithms Nelson Padua-Perez Bill Pugh Department of Computer Science University of Maryland, College Park Overview Comparison sort Bubble sort Selection sort Tree sort Heap sort Quick sort Merge

### Converting a Number from Decimal to Binary

Converting a Number from Decimal to Binary Convert nonnegative integer in decimal format (base 10) into equivalent binary number (base 2) Rightmost bit of x Remainder of x after division by two Recursive

### Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort).

Chapter 7: Quicksort Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort). The three steps of Quicksort are as follows:

### Mergesort and Quicksort

Mergesort Mergesort and Quicksort Basic plan: Divide array into two halves. Recursively sort each half. Merge two halves to make sorted whole. mergesort mergesort analysis quicksort quicksort analysis

### Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Quiz 1 Quiz 1 Do not open this quiz booklet until you are directed

### CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

### External Sorting. Why Sort? 2-Way Sort: Requires 3 Buffers. Chapter 13

External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing

### External Sorting. Chapter 13. Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1

External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing

### Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

### Cpt S 223. School of EECS, WSU

Priority Queues (Heaps) 1 Motivation Queues are a standard mechanism for ordering tasks on a first-come, first-served basis However, some tasks may be more important or timely than others (higher priority)

### Binary Search. Search for x in a sorted array A.

Divide and Conquer A general paradigm for algorithm design; inspired by emperors and colonizers. Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3.

### 6 March 2007 1. Array Implementation of Binary Trees

Heaps CSE 0 Winter 00 March 00 1 Array Implementation of Binary Trees Each node v is stored at index i defined as follows: If v is the root, i = 1 The left child of v is in position i The right child of

### Divide-and-Conquer Algorithms Part Four

Divide-and-Conquer Algorithms Part Four Announcements Problem Set 2 due right now. Can submit by Monday at 2:15PM using one late period. Problem Set 3 out, due July 22. Play around with divide-and-conquer

### 1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++

Answer the following 1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++ 2) Which data structure is needed to convert infix notations to postfix notations? Stack 3) The

### Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

### The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >

### Algorithms. Margaret M. Fleck. 18 October 2010

Algorithms Margaret M. Fleck 18 October 2010 These notes cover how to analyze the running time of algorithms (sections 3.1, 3.3, 4.4, and 7.1 of Rosen). 1 Introduction The main reason for studying big-o

### Analysis of Algorithms I: Binary Search Trees

Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary

### Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * *

Binary Heaps A binary heap is another data structure. It implements a priority queue. Priority Queue has the following operations: isempty add (with priority) remove (highest priority) peek (at highest

### CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.

Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure

### Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4

Divide d Conquer Textook Reading Chapters 4, 7 & 33.4 Overview Design principle Divide d conquer Proof technique Induction, induction, induction Analysis technique Recurrence relations Prolems Sorting

### A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called heap-order property

CmSc 250 Intro to Algorithms Chapter 6. Transform and Conquer Binary Heaps 1. Definition A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called

### CS 102: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4)

CS 10: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4) Problem 1. a. Consider the modified binary search algorithm so that it splits the input not into two sets of almost-equal sizes, but into three

### Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

### Binary Search Trees. Data in each node. Larger than the data in its left child Smaller than the data in its right child

Binary Search Trees Data in each node Larger than the data in its left child Smaller than the data in its right child FIGURE 11-6 Arbitrary binary tree FIGURE 11-7 Binary search tree Data Structures Using

### Divide And Conquer Algorithms

CSE341T/CSE549T 09/10/2014 Lecture 5 Divide And Conquer Algorithms Recall in last lecture, we looked at one way of parallelizing matrix multiplication. At the end of the lecture, we saw the reduce SUM

### CS473 - Algorithms I

CS473 - Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm View in slide-show mode 1 Reminder: Merge Sort Input array A sort this half sort this half Divide Conquer merge two sorted halves Combine

### Binary Heap Algorithms

CS Data Structures and Algorithms Lecture Slides Wednesday, April 5, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks CHAPPELLG@member.ams.org 2005 2009 Glenn G. Chappell

### Data Structures Fibonacci Heaps, Amortized Analysis

Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure:

### 6. Standard Algorithms

6. Standard Algorithms The algorithms we will examine perform Searching and Sorting. 6.1 Searching Algorithms Two algorithms will be studied. These are: 6.1.1. inear Search The inear Search The Binary

### Sorting algorithm. Sorting algorithms used in computer science are often classified by:

Sorting algorithm In computer science and mathematics, a sorting algorithm is an algorithm that puts elements of a list in a certain order. The most used orders are numerical order and lexicographical

### 1 2-3 Trees: The Basics

CS10: Data Structures and Object-Oriented Design (Fall 2013) November 1, 2013: 2-3 Trees: Inserting and Deleting Scribes: CS 10 Teaching Team Lecture Summary In this class, we investigated 2-3 Trees in

### Binary Search Trees. A Generic Tree. Binary Trees. Nodes in a binary search tree ( B-S-T) are of the form. P parent. Key. Satellite data L R

Binary Search Trees A Generic Tree Nodes in a binary search tree ( B-S-T) are of the form P parent Key A Satellite data L R B C D E F G H I J The B-S-T has a root node which is the only node whose parent

### From Last Time: Remove (Delete) Operation

CSE 32 Lecture : More on Search Trees Today s Topics: Lazy Operations Run Time Analysis of Binary Search Tree Operations Balanced Search Trees AVL Trees and Rotations Covered in Chapter of the text From

### CSC148 Lecture 8. Algorithm Analysis Binary Search Sorting

CSC148 Lecture 8 Algorithm Analysis Binary Search Sorting Algorithm Analysis Recall definition of Big Oh: We say a function f(n) is O(g(n)) if there exists positive constants c,b such that f(n)

### Biostatistics 615/815

Merge Sort Biostatistics 615/815 Lecture 8 Notes on Problem Set 2 Union Find algorithms Dynamic Programming Results were very ypositive! You should be gradually becoming g y g comfortable compiling, debugging

Divide and Conquer Paradigm Melissa Manley California State University Northridge Author s address: 18111 Nordhoff Street, Northridge, CA, 91330 Permission to make digital/hard copy of all or part of this

### Questions 1 through 25 are worth 2 points each. Choose one best answer for each.

Questions 1 through 25 are worth 2 points each. Choose one best answer for each. 1. For the singly linked list implementation of the queue, where are the enqueues and dequeues performed? c a. Enqueue in

### Heap. Binary Search Tree. Heaps VS BSTs. < el el. Difference between a heap and a BST:

Heaps VS BSTs Difference between a heap and a BST: Heap el Binary Search Tree el el el < el el Perfectly balanced at all times Immediate access to maximal element Easy to code Does not provide efficient

### Exam study sheet for CS2711. List of topics

Exam study sheet for CS2711 Here is the list of topics you need to know for the final exam. For each data structure listed below, make sure you can do the following: 1. Give an example of this data structure

### AP Computer Science AB Syllabus 1

AP Computer Science AB Syllabus 1 Course Resources Java Software Solutions for AP Computer Science, J. Lewis, W. Loftus, and C. Cocking, First Edition, 2004, Prentice Hall. Video: Sorting Out Sorting,

### Chapter 13: Query Processing. Basic Steps in Query Processing

Chapter 13: Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions 13.1 Basic Steps in Query Processing 1. Parsing

### THIS CHAPTER studies several important methods for sorting lists, both contiguous

Sorting 8 THIS CHAPTER studies several important methods for sorting lists, both contiguous lists and linked lists. At the same time, we shall develop further tools that help with the analysis of algorithms

### IMPROVING PERFORMANCE OF RANDOMIZED SIGNATURE SORT USING HASHING AND BITWISE OPERATORS

Volume 2, No. 3, March 2011 Journal of Global Research in Computer Science RESEARCH PAPER Available Online at www.jgrcs.info IMPROVING PERFORMANCE OF RANDOMIZED SIGNATURE SORT USING HASHING AND BITWISE

### Heaps & Priority Queues in the C++ STL 2-3 Trees

Heaps & Priority Queues in the C++ STL 2-3 Trees CS 3 Data Structures and Algorithms Lecture Slides Friday, April 7, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks

### Format of Divide-and-Conquer algorithms:

CS 360: Data Structures and Algorithms Divide-and-Conquer (part 1) Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively

### 8. Query Processing. Query Processing & Optimization

ECS-165A WQ 11 136 8. Query Processing Goals: Understand the basic concepts underlying the steps in query processing and optimization and estimating query processing cost; apply query optimization techniques;

### APP INVENTOR. Test Review

APP INVENTOR Test Review Main Concepts App Inventor Lists Creating Random Numbers Variables Searching and Sorting Data Linear Search Binary Search Selection Sort Quick Sort Abstraction Modulus Division

### Dynamic Programming. Lecture 11. 11.1 Overview. 11.2 Introduction

Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach

### Previous Lectures. B-Trees. External storage. Two types of memory. B-trees. Main principles

B-Trees Algorithms and data structures for external memory as opposed to the main memory B-Trees Previous Lectures Height balanced binary search trees: AVL trees, red-black trees. Multiway search trees:

### AS-2261 M.Sc.(First Semester) Examination-2013 Paper -fourth Subject-Data structure with algorithm

AS-2261 M.Sc.(First Semester) Examination-2013 Paper -fourth Subject-Data structure with algorithm Time: Three Hours] [Maximum Marks: 60 Note Attempts all the questions. All carry equal marks Section A

### A binary search tree or BST is a binary tree that is either empty or in which the data element of each node has a key, and:

Binary Search Trees 1 The general binary tree shown in the previous chapter is not terribly useful in practice. The chief use of binary trees is for providing rapid access to data (indexing, if you will)

### The PRAM Model and Algorithms. Advanced Topics Spring 2008 Prof. Robert van Engelen

The PRAM Model and Algorithms Advanced Topics Spring 2008 Prof. Robert van Engelen Overview The PRAM model of parallel computation Simulations between PRAM models Work-time presentation framework of parallel

### root node level: internal node edge leaf node CS@VT Data Structures & Algorithms 2000-2009 McQuain

inary Trees 1 A binary tree is either empty, or it consists of a node called the root together with two binary trees called the left subtree and the right subtree of the root, which are disjoint from each

### Algorithm Analysis [2]: if-else statements, recursive algorithms. COSC 2011, Winter 2004, Section N Instructor: N. Vlajic

1 Algorithm Analysis []: if-else statements, recursive algorithms COSC 011, Winter 004, Section N Instructor: N. Vlajic Algorithm Analysis for-loop Running Time The running time of a simple loop for (int

### Lecture 2 February 12, 2003

6.897: Advanced Data Structures Spring 003 Prof. Erik Demaine Lecture February, 003 Scribe: Jeff Lindy Overview In the last lecture we considered the successor problem for a bounded universe of size u.

### 1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D.

1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D. base address 2. The memory address of fifth element of an array can be calculated

### Randomized algorithms

Randomized algorithms March 10, 2005 1 What are randomized algorithms? Algorithms which use random numbers to make decisions during the executions of the algorithm. Why would we want to do this?? Deterministic

### Rethinking SIMD Vectorization for In-Memory Databases

SIGMOD 215, Melbourne, Victoria, Australia Rethinking SIMD Vectorization for In-Memory Databases Orestis Polychroniou Columbia University Arun Raghavan Oracle Labs Kenneth A. Ross Columbia University Latest

### Tables so far. set() get() delete() BST Average O(lg n) O(lg n) O(lg n) Worst O(n) O(n) O(n) RB Tree Average O(lg n) O(lg n) O(lg n)

Hash Tables Tables so far set() get() delete() BST Average O(lg n) O(lg n) O(lg n) Worst O(n) O(n) O(n) RB Tree Average O(lg n) O(lg n) O(lg n) Worst O(lg n) O(lg n) O(lg n) Table naïve array implementation

### Data Structure with C

Subject: Data Structure with C Topic : Tree Tree A tree is a set of nodes that either:is empty or has a designated node, called the root, from which hierarchically descend zero or more subtrees, which

### B-Trees. Algorithms and data structures for external memory as opposed to the main memory B-Trees. B -trees

B-Trees Algorithms and data structures for external memory as opposed to the main memory B-Trees Previous Lectures Height balanced binary search trees: AVL trees, red-black trees. Multiway search trees:

### Faster deterministic integer factorisation

David Harvey (joint work with Edgar Costa, NYU) University of New South Wales 25th October 2011 The obvious mathematical breakthrough would be the development of an easy way to factor large prime numbers

### A Note on Maximum Independent Sets in Rectangle Intersection Graphs

A Note on Maximum Independent Sets in Rectangle Intersection Graphs Timothy M. Chan School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1, Canada tmchan@uwaterloo.ca September 12,

### Analysis of Binary Search algorithm and Selection Sort algorithm

Analysis of Binary Search algorithm and Selection Sort algorithm In this section we shall take up two representative problems in computer science, work out the algorithms based on the best strategy to

### Datenbanksysteme II: Hashing. Ulf Leser

Datenbanksysteme II: Hashing Ulf Leser Content of this Lecture Hashing Extensible Hashing Linear Hashing Ulf Leser: Implementation of Database Systems, Winter Semester 2012/2013 2 Sorting or Hashing Sorted

### recursion, O(n), linked lists 6/14

recursion, O(n), linked lists 6/14 recursion reducing the amount of data to process and processing a smaller amount of data example: process one item in a list, recursively process the rest of the list

### Zabin Visram Room CS115 CS126 Searching. Binary Search

Zabin Visram Room CS115 CS126 Searching Binary Search Binary Search Sequential search is not efficient for large lists as it searches half the list, on average Another search algorithm Binary search Very

### Analysis of Algorithms I: Optimal Binary Search Trees

Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search

### Closest Pair Problem

Closest Pair Problem Given n points in d-dimensions, find two whose mutual distance is smallest. Fundamental problem in many applications as well as a key step in many algorithms. p q A naive algorithm

### Data Structures and Data Manipulation

Data Structures and Data Manipulation What the Specification Says: Explain how static data structures may be used to implement dynamic data structures; Describe algorithms for the insertion, retrieval

### Data Structures. Algorithm Performance and Big O Analysis

Data Structures Algorithm Performance and Big O Analysis What s an Algorithm? a clearly specified set of instructions to be followed to solve a problem. In essence: A computer program. In detail: Defined

### Divide and Conquer: Counting Inversions

Divide and Conquer: Counting Inversions Rank Analysis Collaborative filtering matches your preference (books, music, movies, restaurants) with that of others finds people with similar tastes recommends

### EFFICIENT EXTERNAL SORTING ON FLASH MEMORY EMBEDDED DEVICES

ABSTRACT EFFICIENT EXTERNAL SORTING ON FLASH MEMORY EMBEDDED DEVICES Tyler Cossentine and Ramon Lawrence Department of Computer Science, University of British Columbia Okanagan Kelowna, BC, Canada tcossentine@gmail.com

### Lecture 6: Binary Search Trees CSCI 700 - Algorithms I. Andrew Rosenberg

Lecture 6: Binary Search Trees CSCI 700 - Algorithms I Andrew Rosenberg Last Time Linear Time Sorting Counting Sort Radix Sort Bucket Sort Today Binary Search Trees Data Structures Data structure is a

### A COMPARATIVE STUDY OF LINKED LIST SORTING ALGORITHMS

A COMPARATIVE STUDY OF LINKED LIST SORTING ALGORITHMS by Ching-Kuang Shene 1 Michigan Technological University Department of Computer Science Houghton, MI 49931-1295 shene@mtu.edu 1 Introduction Carraway

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### Persistent Binary Search Trees

Persistent Binary Search Trees Datastructures, UvA. May 30, 2008 0440949, Andreas van Cranenburgh Abstract A persistent binary tree allows access to all previous versions of the tree. This paper presents

### Parallel Random Access Machine (PRAM) PRAM Algorithms. Shared Memory Access Conflicts. A Basic PRAM Algorithm. Time Optimality.

Parallel Random Access Machine (PRAM) PRAM Algorithms Arvind Krishnamurthy Fall 2 Collection of numbered processors Accessing shared memory cells Each processor could have local memory (registers) Each

### Binary Heaps. CSE 373 Data Structures

Binary Heaps CSE Data Structures Readings Chapter Section. Binary Heaps BST implementation of a Priority Queue Worst case (degenerate tree) FindMin, DeleteMin and Insert (k) are all O(n) Best case (completely

### 8.2 Quicksort. 332 Chapter 8. Sorting

332 Chapter 8. Sorting For randomly ordered data, the operations count goes approximately as N 1.25,at least for N50, however, Quicksort is generally faster. The program follows: void shell(unsigned

### Algorithms and Data Structures

Algorithm Analysis Page 1 BFH-TI: Softwareschule Schweiz Algorithm Analysis Dr. CAS SD01 Algorithm Analysis Page 2 Outline Course and Textbook Overview Analysis of Algorithm Pseudo-Code and Primitive Operations

### CSE 326: Data Structures B-Trees and B+ Trees

Announcements (4//08) CSE 26: Data Structures B-Trees and B+ Trees Brian Curless Spring 2008 Midterm on Friday Special office hour: 4:-5: Thursday in Jaech Gallery (6 th floor of CSE building) This is

### MapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research

MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With

### Class Overview. CSE 326: Data Structures. Goals. Goals. Data Structures. Goals. Introduction

Class Overview CSE 326: Data Structures Introduction Introduction to many of the basic data structures used in computer software Understand the data structures Analyze the algorithms that use them Know

### Searching and Sorting. Searching. Binary Search. Linear Search. k N-1. <= x??? > x. x not in here???

Searching and Sorting It s a fundamental area of CS since the beginning of time. Well understood! ll professional programmers should have a good understanding of it. lso provides good example for discussion

### Krishna Institute of Engineering & Technology, Ghaziabad Department of Computer Application MCA-213 : DATA STRUCTURES USING C

Tutorial#1 Q 1:- Explain the terms data, elementary item, entity, primary key, domain, attribute and information? Also give examples in support of your answer? Q 2:- What is a Data Type? Differentiate

### Analysis of Algorithms: The Non-recursive Case

Analysis of Algorithms: The Non-recursive Case Key topics: * Introduction * Generalizing Running Time * Doing a Timing Analysis * Big-Oh Notation * Big-Oh Operations * Analyzing Some Simple Programs -

### S. Muthusundari. Research Scholar, Dept of CSE, Sathyabama University Chennai, India e-mail: nellailath@yahoo.co.in. Dr. R. M.

A Sorting based Algorithm for the Construction of Balanced Search Tree Automatically for smaller elements and with minimum of one Rotation for Greater Elements from BST S. Muthusundari Research Scholar,

### CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

### Sample Questions Csci 1112 A. Bellaachia

Sample Questions Csci 1112 A. Bellaachia Important Series : o S( N) 1 2 N N i N(1 N) / 2 i 1 o Sum of squares: N 2 N( N 1)(2N 1) N i for large N i 1 6 o Sum of exponents: N k 1 k N i for large N and k

### 8.1 Makespan Scheduling

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programing: Min-Makespan and Bin Packing Date: 2/19/15 Scribe: Gabriel Kaptchuk 8.1 Makespan Scheduling Consider an instance

### Week 7: Divide and Conquer

Agenda: Divide and Conquer technique Multiplication of large integers Exponentiation Matrix multiplication 1 2- Divide and Conquer : To solve a problem we can break it into smaller subproblems, solve each

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### Chapter 6: Episode discovery process

Chapter 6: Episode discovery process Algorithmic Methods of Data Mining, Fall 2005, Chapter 6: Episode discovery process 1 6. Episode discovery process The knowledge discovery process KDD process of analyzing