Cpt S 223. School of EECS, WSU

Transcription

1 Priority Queues (Heaps) 1

2 Motivation Queues are a standard mechanism for ordering tasks on a first-come, first-served basis However, some tasks may be more important or timely than others (higher priority) Priority queues Store tasks using a partial ordering based on priority Ensure highest priority task at head of queue Heaps are the underlying data structure of priority queues 2

3 Priority Queues: Specification Main operations insert (i.e., enqueue) Dynamic insert specification of a priority level (0-high, 1,2.. Low) deletemin (i.e., dequeue) Finds the current minimum element (read: highest priority ) in the queue, deletes it from the queue, and returns it Performance goal is for operations to be fast 3

4 Using priority queues insert() 2 deletemin() Dequeues the next element with the highest priority 4

5 Can we build a data structure better suited to store and retrieve priorities? Simple Implementations Unordered linked list O(1) insert O(n) deletemin Ordered linked list O(n) insert O(1) deletemin Ordered array O(lg n + n) insert O(n) deletemin Balanced BST O(log 2 n) insert and deletemin

6 Binary Heap A priority queue data structure 6

7 Binary Heap A binary heap is a binary tree with two properties Structure property Heap-order property 7

8 Structure Property A binary heap is a complete binary tree Each level (except possibly the bottom most level) is completely filled The bottom most level may be partially filled (from left to right) Height of a complete binary tree with N elements is log 2 N 8

9 Structure property Binary Heap Example N=10 Every level (except last) saturated Array representation: 9

10 Heap-order Property Heap-order property (for a MinHeap ) For every node X, key(parent(x)) key(x) Except root node, which has no parent Thus, minimum key always at root Alternatively, for a MaxHeap, always keep the maximum key at the root Insert and deletemin must maintain heap-order property 10

11 Heap Order Property Minimum element Duplicates are allowed No order implied for elements which do not share ancestor-descendant relationship 11

12 Implementing Complete Binary Trees as Arrays Given element at position i in the array Left child(i) = at position 2i Right child(i) = at position 2i + 1 Parent(i) = at position i / 2 i 2i i/2 2i

13 Just finds the Min without deleting it insert deletemin Stores the heap as a vector Note: a general delete() function is not as important for heaps but could be implemented Fix heap after deletemin 13

14 Heap Insert Insert new element into the heap at the next available slot ( hole ) hole) According to maintaining a complete binary tree Then, percolate the element up the heap while heap-order property not satisfied 14

15 Percolating Up Heap Insert: Example Insert 14: 14 hole 15

16 Percolating Up Heap Insert: Example Insert 14: (1) 14 vs hole 16

17 Percolating Up Heap Insert: Example Insert 14: (1) 14 vs hole (2) 14 vs

18 Percolating Up Heap Insert: Example Insert 14: (1) 14 vs hole (2) 14 vs (3) Heap order prop 14 vs. 13 Structure t prop Path of percolation up 18

19 Heap Insert: Implementation // assume array implementation void insert( const Comparable &x) {? } 19

20 Heap Insert: Implementation O(log N) time 20

21 Heap DeleteMin Minimum element is always at the root Heap decreases by one in size Move last element into hole at root Percolate down while hl heap-order property not satisfied 21

22 Percolating down Heap DeleteMin: Example Make this position empty 22

23 Percolating down Heap DeleteMin: Example Copy 31 temporarily here and move it dow Make this position empty Is 31 > min(14,16)? Yes - swap 31 with min(14,16) 23

24 Percolating down Heap DeleteMin: Example 31 Is 31 > min(19,21)? Yes - swap 31 with min(19,21) 24

25 Percolating down Heap DeleteMin: Example Is 31 > min(19,21)? Yes - swap 31 with min(19,21) Is 31 > min(65,26)? Yes - swap 31 with min(65,26) Percolating down 25

26 Percolating down Heap DeleteMin: Example 31 Percolating down 26

27 Percolating down Heap DeleteMin: Example 31 Heap order prop Structure prop 27

28 Heap DeleteMin: Implementation O(log N) time 28

29 Heap DeleteMin: Implementation Percolate down Left child Right child Pick child to swap with 29

30 Other Heap Operations decreasekey(p,v) Lowers the current value of item p to new priority value v Need to percolate up E.g., promote a job increasekey(p,v) Increases the current value of item p to new priority value v Need to percolate down E.g., demote a job remove(p) Run-times for all three functions? (p) First, decreasekey(p,- ) Then, deletemin E.g., abort/cancel a job O(lg n) 30

31 Improving Heap Insert Time What if all N elements are all available upfront? To build a heap with N elements: Default method takes O(N lg N) time We will now see a new method called buildheap() that will take O(N) time - i.e., optimal 31

32 Building a Heap Construct heap from initial set of N items Solution 1 Perform N inserts O(N log 2 N) worst-case Solution 2 (use buildheap()) Randomly populate initial heap with structure property Perform a percolate-down from each internal node (H[size/2] to H[1]) To take care of heap order property 32

33 BuildHeap Example Input: { 150, 80, 40, 10, 70, 110, 30, 120, 140, 60, 50, 130, 100, 20, 90 } Leaves are all valid heaps (implicitly) Arbitrarily assign elements to heap nodes Structure property satisfied Heap order property p violated Leaves are all valid heaps (implicit) So, let us look at each internal node, from bottom to top, and fix if necessary 33

34 BuildHeap Example Nothing to do Swap with left child Randomly initialized heap Structure property satisfied Heap order property violated Leaves are all valid heaps Cpt (implicit) S 223. School of EECS, WSU 34

35 BuildHeap Example Nothing to do Swap with right child Dotted lines show path of percolating down 35

36 BuildHeap Example Swap with right child & then with 60 Nothing to do Dotted lines show path of percolating down 36

37 BuildHeap Example Swap path Final Heap Dotted lines show path of percolating down 37

38 BuildHeap Implementation Start with lowest, rightmost internal node 38

39 BuildHeap() : Run-time Analysis Run-time =? O(sum of the heights of all the internal nodes) because we may have to percolate all the way down to fix every internal node in the worst-case Theorem 6.1 HOW? For a perfect binary tree of height h, the sum of heights of all nodes is 2 h+1 1 (h + 1) Since h=lg N, N then sum of heights ht is O(N) Will be slightly better in practice Implication: Each insertion costs O(1) amortized time 39

40 40

41 Binary Heap Operations Worst-case Analysis Height of heap is log 2 N insert: O(lg (g N) for each insert In practice, expect less buildheap insert: O(N) for N inserts deletemin: O(lg N) decreasekey: O(lg N) increasekey: O(lg N) remove: O(lg N) 41

42 Applications Operating system scheduling Process jobs by priority Graph algorithms Find shortest path Event simulation Instead of checking for events at each time click, look up next event to happen 42

43 An Application: The Selection Problem Given a list of n elements, find the k th smallest element Algorithm 1: Sort the list => O(n log n) Pick the k th element => O(1) A better algorithm: Use a binary heap (minheap) 43

44 Selection using a MinHeap Input: n elements Algorithm: 1. buildheap(n) ==> >O(n) 2. Perform k deletemin() operations ==> O(k log n) 3. Report the k th deletemin output ==> O(1) Total run-time = O(n + k log n) If k = O(n/log n) then the run-time becomes O(n) 44

45 Other Types of Heaps Binomial Heaps d-heaps Generalization of binary heaps (ie., 2-Heaps) Leftist Heaps Supports merging of two heaps in o(m+n) time (ie., sublinear) Skew Heaps O(log n) amortized run-time Fibonacci Heaps 45

46 Run-time Per Operation Insert DeleteMin Merge (=H 1 +H 2 ) Binary heap O(log n) worst-case O(log n) O(n) O(1) amortized for buildheap Leftist Heap O(log n) O(log n) O(log n) Skew Heap O(log n) O(log n) O(log n) Binomial i O(log n) worst case O(log n) O(log n) Heap O(1) amortized for sequence of n inserts Fibonacci Heap O(1) O(log n) O(1) 46

47 Priority Queues in STL Uses Binary heap Default is MaxHeap Methods Push, top, pop, p, empty, clear #include <priority_queue> int main () { priority_queue<int> Q; Q.push (10); cout << Q.top (); Q.pop (); } Calls DeleteMax() For MinHeap: declare priority_queue as: priority_queue<int, vector<int>, greater<int>> Q; Refer to Book Chapter 6, Fig 6.57 for an example 47

48 Binomial Heaps 48

49 Binomial Heap A binomial heap is a forest of heap-ordered binomial trees, satisfying: i) Structure property, and ii) Heap order property A binomial heap is different from binary heap in that: Its structure property is totally different Its heap-order property (within each binomial tree) is the same as in a binary heap 49

50 Note: A binomial tree need not be a binary tree Definition: A Binomial Tree B k A binomial tree of height k is called B k : It has 2 k nodes k The number of nodes at depth d = ( d ) k ( ) is the form of the co-efficients in binomial theorem d Depth: B 3 : #nodes: d=0 ( 3 0 ) d=1 ( 3 1 ) d=2 ( 3 2 ) d=3 ( 3 3 ) 50

51 What will a Binomial Heap with n=31 nodes look like? We know that: i) A binomial heap should be a forest of binomial trees ii) Each binomial tree has power of 2 elements So how many binomial i trees do we need? B 4 B 3 B 2 B 1 B 0 n = 31 = ( ) 2 51

52 ABi Binomial i lh Heap w/ n=31 nodes B 4 B 3 B 4 B 2 B 1 B 0 B i == B i-1 + B i-1 n = 31 = ( ) 2 trees {B 0, B 1, 1 B 2, B 3, B 4 } st of binomial Fores B 0 B 1 B 3 B 2 52

53 Binomial Heap Property Lemma: There exists a binomial heap for every positive value of n Proof: All values of n can be represented in binary representation Have one binomial tree for each power of two with co-efficient of 1 Eg., n=10 n ==> (1010) 2 ==> forest contains {B 3,B 1 } 53

54 Binomial Heaps: Heap-Order Property Each binomial tree should contain the minimum element at the root of every subtree Just like binary heap, except that the tree here is a binomial i tree structure t (and not a complete binary tree) The order of elements across binomial trees is irrelevant 54

55 Definition: Binomial Heaps A binomial heap of n nodes is: (Structure Property) A forest of binomial trees as dictated by the binary representation of n (Heap-Order Property) Each binomial tree is a min-heap or a max-heap 55

56 Binomial Heaps: Examples Two different heaps: 56

57 Key Properties Could there be multiple trees of the same height in a binomial heap? no What is the upper bound on the number of binomial trees in a binomial heap of n nodes? lg n Given n, can we tell (for sure) if B k exists? B k exists if and only if: the k th least significant bit is 1 in the binary representation of n 57

58 An Implementation of a Binomial Heap Example: n=13 == (1101) 2 B 7 B 6 B 5 B 4 B 3 Maintain a linked list of tree pointers (for the forest) B 2 B 1 B 0 Shown using the left-child, right-sibling pointer method Analogous to a bit-based representation of a binary number n 58

59 Binomial Heap: Operations x <= DeleteMin() Insert(x) Merge(H 1, H 2 ) 59

60 DeleteMin() Goal: Given a binomial heap, H, find the minimum and delete it Observation: The root of each binomial tree in H contains its minimum element Approach: Therefore, return the minimum of all the roots (minimums) Complexity: O(log n) comparisons (because there are only O(log n) trees) 60

61 FindMin() & DeleteMin() Example B 0 B 2 B 3 B 0 B 1 B 2 For DeleteMin(): After delete, how to adjust the heap? New Heap : Merge { B 0, B 2 }&{B 0, B 1, B 2 } 61

62 Insert(x) in Binomial Heap Goal: To insert a new element x into a binomial heap H Observation: Element x can be viewed as a single element binomial heap => Insert(Hx)== (H,x) Merge(H, {x}) So, if we decide how to do merge we will automatically figure out how to implement both insert() and deletemin() 62

63 Merge(H 1,H 2 ) Let n 1 be the number of nodes in H 1 Let n 2 be the number of nodes in H 2 Therefore, the new heap is going to have n 1 +n 2 nodes Assume n = n 1 + n 2 Logic: Merge trees of same height, starting from lowest height trees If only one tree of a given height, then just copy that Otherwise, need to do carryover (just like adding two binary numbers) 63

64 Idea: merge tree of same heights Merge: Example + B 0 B 1 B 2? 13 Cpt S 223. School of EECS,? WSU 64

65 How to Merge Two Binomial Trees of the Same Height? B 2 : + B 2 : B 3 : Simply compare the roots Note: Merge is defined for only binomial Cpt S 223. School trees of with EECS, the WSU same height 65

66 Merge(H 1,H 2 )example carryover ?

67 How to Merge more than two binomial trees of the same height? Merging more than 2 binomial trees of the same height could generate carry- overs ? 26 Merge any two and leave the third as carry-over 67

68 Input: Merge(H 1,H 2 ) : Example + Output: There are two other possible answers Merge cost log(max{n 1,n 2 }) = O(log n) comparisons 68

69 Run-time Complexities unaffected affected Merge takes O(log n) comparisons Corollary: Insert and DeleteMin also take O(log n) It can be further proved that an uninterrupted sequence of m Insert operations takes only O(m) time per operation, implying O(1) amortize time per insert Proof Hint: For each insertion, if i is the least significant bit position with a 0, then number of comparisons required to do the next insert is i+1 If you count the #bit flips for each insert, going from insert of the first element to the insert of the last (n th ) element, then => amortized run-time of O(1) per insert 69

70 Binomial Queue Run-time Summary Insert O(lg n) worst-case O(1) amortized time if insertion is done in an uninterrupted sequence (i.e., without being intervened by deletemins) DeleteMin, FindMin O(lg n) worst-case Merge O(lg n) worst-case 70

71 Run-time Per Operation Insert DeleteMin Merge (=H 1 +H 2 ) Binary heap O(log n) worst-case O(log n) O(n) O(1) amortized for buildheap Leftist Heap O(log n) O(log n) O(log n) Skew Heap O(log n) O(log n) O(log n) Binomial i O(log n) worst case O(log n) O(log n) Heap O(1) amortized for sequence of n inserts Fibonacci Heap O(1) O(log n) O(1) 71

72 Summary Priority queues maintain the minimum or maximum element of a set Support O(log N) operations worst-case insert, deletemin, merge Many applications in support of other algorithms 72