Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn
Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure: A Fibonacci heap consists of a collection of trees satisfying the min heap property. Variables:. : root of the tree containing the (a) minimum key. : circular, doubly linked, unordered list containing the roots of all trees. : number of nodes currently in Algorithm Theory, WS 2012/13 Fabian Kuhn 2
Trees in Fibonacci Heaps Structure of a single node : parent left key degree right child mark. : points to circular, doubly linked and unordered list of the children of.,. : pointers to siblings (in doubly linked list). : will be used later Advantages of circular, doubly linked lists: Deleting an element takes constant time Concatenating two lists takes constant time Algorithm Theory, WS 2012/13 Fabian Kuhn 3
Example Figure: Cormen et al., Introduction to Algorithms Algorithm Theory, WS 2012/13 Fabian Kuhn 4
Simple (Lazy) Operations Initialize Heap :.. Merge heaps and : concatenate root lists update. Insert element into : create new one node tree containing H merge heaps and Get minimum element of : return. Algorithm Theory, WS 2012/13 Fabian Kuhn 5
Operation Delete Min Delete the node with minimum key from and return its element: 1.. ; 2. if. 0 then 3. remove. from. ; 4. add.. (list) to. 5.. ; // Repeatedly merge nodes with equal degree in the root list // until degrees of nodes in the root list are distinct. // Determine the element with minimum key 6. return Algorithm Theory, WS 2012/13 Fabian Kuhn 6
Rank and Maximum Degree Ranks of nodes, trees, heap: Node : : degree of Tree : : rank (degree) of root node of Heap : : maximum degree of any node in Assumption ( : number of nodes in ): for a known function Algorithm Theory, WS 2012/13 Fabian Kuhn 7
Merging Two Trees Given: Heap ordered trees, with Assume: min key of min key of Operation, : Removes tree from root list and adds to child list of 1. Algorithm Theory, WS 2012/13 Fabian Kuhn 8
Consolidation of Root List Array pointing to find roots with the same rank: 0 1 2 Consolidate: Time: 1. for 0to do null;. 2. while. null do 3. delete and return first element of. 4. while nulldo 5. ; 6. ; 7., 8. 9. Create new. and. Algorithm Theory, WS 2012/13 Fabian Kuhn 9
Operation Decrease Key Decrease Key, : (decrease key of node to new value ) 1. if. then return; 2.. ; update. ; 3. if... then return 4. repeat 5.. ; 6.. ; 7. ; 8. until.. ; 9. if. then. ; Algorithm Theory, WS 2012/13 Fabian Kuhn 10
Operation Cut( ) Operation. : Cuts s sub tree from its parent and adds to rootlist 1. if. then 2. // cut the link between and its parent 3... 1; 4. remove from.. (list) 5.. null; 6. add to. 25 1 13 3 2 8 15 25 1 13 19 3 7 2 8 15 31 19 7 31 Algorithm Theory, WS 2012/13 Fabian Kuhn 11
Decrease Key Example Green nodes are marked 5 1 2 15 17 9 14 20 25 13 3 8 12 18 31 19 7 22 Decrease Key, 5 18 9 1 2 15 17 14 20 22 12 25 13 3 8 31 19 7 Algorithm Theory, WS 2012/13 Fabian Kuhn 12
Fibonacci Heap Marks History of a node : is being linked to a node. a child of is cut. a second child of is cut. Hence, the boolean value. indicates whether node has lost a child since the last time was made the child of another node. Algorithm Theory, WS 2012/13 Fabian Kuhn 13
Cost of Delete Min & Decrease Key Delete Min: 1. Delete min. root and add. to. time: 1 2. Consolidate. time: length of. Step 2 can potentially be linear in (size of ) Decrease Key (at node ): 1. If new key parent key, cut sub tree of node time: 1 2. Cascading cuts up the tree as long as nodes are marked time: number of consecutive marked nodes Step 2 can potentially be linear in Exercises: Both operations can take time in the worst case! Algorithm Theory, WS 2012/13 Fabian Kuhn 14
Cost of Delete Min & Decrease Key Cost of delete min and decrease key can be Θ Seems a large price to pay to get insert and merge in 1 time Maybe, the operations are efficient most of the time? It seems to require a lot of operations to get a long rootlist and thus, an expensive consolidate operation In each decrease key operation, at most one node gets marked: We need a lot of decrease key operations to get an expensive decrease key operation Can we show that the average cost per operation is small? We can requires amortized analysis Algorithm Theory, WS 2012/13 Fabian Kuhn 15
Amortization Consider sequence,,, of operations (typically performed on some data structure ) : execution time of operation : total execution time The execution time of a single operation might vary within a large range (e.g., 1, ) The worst case overall execution time might still be small average execution time per operation might be small in the worst case, even if single operations can be expensive Algorithm Theory, WS 2012/13 Fabian Kuhn 16
Analysis of Algorithms Best case Worst case Average case Amortized worst case What it the average cost of an operation in a worst case sequence of operations? Algorithm Theory, WS 2012/13 Fabian Kuhn 17
Example: Binary Counter Incrementing a binary counter: determine the bit flip cost: Operation Counter Value Cost 00000 1 00001 1 2 00010 2 3 00011 1 4 00100 3 5 00101 1 6 00110 2 7 00111 1 8 01000 4 9 01001 1 10 01010 2 11 01011 1 12 01100 3 13 01101 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 18
Accounting Method Observation: Each increment flips exactly one 0 into a 1 00100 1111 00100 0000 Idea: Have a bank account (with initial amount 0) Paying to the bank account costs Take money from account to pay for expensive operations Applied to binary counter: Flip from 0 to 1: pay 1 to bank account (cost: 2) Flip from 1 to 0: take 1 from bank account (cost: 0) Amount on bank account = number of ones We always have enough money to pay! Algorithm Theory, WS 2012/13 Fabian Kuhn 19
Accounting Method Op. Counter Cost To Bank From Bank Net Cost Credit 0 0 0 0 0 1 00 0 0 1 1 2 00 0 1 0 2 3 00 0 1 1 1 4 00 1 0 0 3 5 00 1 0 1 1 6 00 1 1 0 2 7 00 1 1 1 1 8 01 0 0 0 4 9 01 0 0 1 1 10 0 1 0 1 0 2 Algorithm Theory, WS 2012/13 Fabian Kuhn 20
Potential Function Method Most generic and elegant way to do amortized analysis! But, also more abstract than the others State of data structure / system: (state space) Potential function : Operation : : actual cost of operation : state after execution of operation ( : initial state) Φ : potential after exec. of operation : amortized cost of operation : Algorithm Theory, WS 2012/13 Fabian Kuhn 21
Potential Function Method Operation : actual cost: amortized cost: Φ Φ Overall cost: Φ Φ Algorithm Theory, WS 2012/13 Fabian Kuhn 22
Binary Counter: Potential Method Potential function: : Clearly, Φ 0and Φ 0for all 0 Actual cost : 1 flip from 0 to 1 1flips from 1 to 0 Potential difference: Φ Φ 1 1 2 Amortized cost: Φ Φ 2 Algorithm Theory, WS 2012/13 Fabian Kuhn 23
Back to Fibonacci Heaps Worst case cost of a single delete min or decrease key operation is Ω Can we prove a small worst case amortized cost for delete min and decrease key operations? Remark: Data structure that allows operations,, We say that operation has amortized cost if for every execution the total time is where is the number of operations of type Algorithm Theory, WS 2012/13 Fabian Kuhn 24,
Amortized Cost of Fibonacci Heaps Initialize heap, is empty, get min, insert, and merge have worst case cost Delete min has amortized cost Decrease key has amortized cost Starting with an empty heap, any sequence of operations with at most delete min operations has total cost (time). We will now need the marks Cost for Dijkstra: log Algorithm Theory, WS 2012/13 Fabian Kuhn 25
Fibonacci Heaps: Marks Cycle of a node: 1. Node is removed from root list and linked to a node. 2. Child node of is cut and added to root list. 3. Second child of is cut node is cut as well and moved to root list The boolean value. indicates whether node has lost a child since the last time was made the child of another node. 26 Algorithm Theory, WS 2012/13 Fabian Kuhn Winter term 11/12 26
Potential Function System state characterized by two parameters: : number of trees (length of. ) : number of marked nodes that are not in the root list Potential function: Example: 5 18 9 1 2 15 17 14 20 22 12 25 13 3 8 31 19 7 7, 2 Φ 11 Algorithm Theory, WS 2012/13 Fabian Kuhn 27
Actual Time of Operations Operations: initialize heap, is empty, insert, get min, merge actual time: 1 Normalize unit time such that,,,, 1 Operation delete min: Actual time: length of. Normalize unit time such that length of. Operation descrease key: Actual time: length of path to next unmarked ancestor Normalize unit time such that lengthof path to next unmarked ancestor Algorithm Theory, WS 2012/13 Fabian Kuhn 28
Amortized Times Assume operation is of type: initialize heap: actual time: 1, potential: Φ Φ 0 amortized time: Φ Φ 1 is empty, get min: actual time: 1, potential: Φ Φ (heap doesn t change) amortized time: Φ Φ 1 merge: Actual time: 1 combined potential of both heaps: Φ Φ amortized time: Φ Φ 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 29
Amortized Time of Insert Assume that operation is an insert operation: Actual time: 1 Potential function: remains unchanged (no nodes are marked or unmarked, no marked nodes are moved to the root list) grows by 1 (one element is added to the root list), Φ Φ 1 1 Amortized time: Algorithm Theory, WS 2012/13 Fabian Kuhn 30
Amortized Time of Delete Min Assume that operation is a delete min operation: Actual time:. Potential function : : changes from. to at most : (# of marked nodes that are not in the root list) no new marks if node is moved away from root list,. is set to false value of does not change!,. Φ Φ. Amortized Time: Algorithm Theory, WS 2012/13 Fabian Kuhn 31
Amortized Time of Decrease Key Assume that operation is a decrease key operation at node : Actual time: length of path to next unmarked ancestor Potential function : Assume, node and nodes,, are moved to root list,, are marked and moved to root list,. mark is set to true marked nodes go to root list, 1node gets newly marked grows by 1, grows by 1 and is decreased by 1, 1 Φ Φ 1 2 1 Φ 3 Amortized time: Algorithm Theory, WS 2012/13 Fabian Kuhn 32
Complexities Fibonacci Heap Initialize Heap: Is Empty: Insert: Get Min: Delete Min: Decrease Key: amortized Merge (heaps of size and, ): How large can get? Algorithm Theory, WS 2012/13 Fabian Kuhn 33
Rank of Children Lemma: Consider a node of rank and let,, be the children of in the order in which they were linked to. Then, Proof:. Algorithm Theory, WS 2012/13 Fabian Kuhn 34
Size of Trees Fibonacci Numbers: 0, 1, 2: Lemma: In a Fibonacci heap, the size of the sub tree of a node with rank is at least. Proof: : minimum size of the sub tree of a node of rank Algorithm Theory, WS 2012/13 Fabian Kuhn 35
Size of Trees 1, 2, 2: 2 Claim about Fibonacci numbers: 0: 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 36
Size of Trees 1, 2, 2: 2, 1 Claim of lemma: Algorithm Theory, WS 2012/13 Fabian Kuhn 37
Size of Trees Lemma: In a Fibonacci heap, the size of the sub tree of a node with rank is at least. Theorem: The maximum rank of a node in a Fibonacci heap of size is at most. Proof: The Fibonacci numbers grow exponentially: 1 5 1 5 2 For, we need nodes. 1 5 2 Algorithm Theory, WS 2012/13 Fabian Kuhn 38
Summary: Binomial and Fibonacci Heaps Binomial Heap Fibonacci Heap initialize insert get min delete min * decrease key * merge is empty amortized time Algorithm Theory, WS 2012/13 Fabian Kuhn 39