# The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

 To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video
Save this PDF as:

Size: px
Start display at page:

Download "The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,"

## Transcription

1 The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints were connected by the edges we had added to the tree so far. Remember that on a graph with n nodes and e edges, the Kruskal algorithm took O(e log e) steps to sort the edges by cost, O(e) steps to decide whether each edge should be added (O(1) steps for each edge) and O(n 2 ) time to maintain a table telling us which nodes were connected to which. The total time was O(n 2 + e log e), which for sparse graphs is O(n 2 ). Thus keeping track of the components is the bottleneck step in the algorithm in the general case, as it is what is stopping us from running the algorithm in O(e log e) steps. 1

2 The problem of maintaining the connected components of the growing forest in Kruskal s algorithm is a special case of a more general problem called the unionfind problem. In this lecture we ll look at a series of increasingly efficient implementations of data structures that solve the union-find problem. We ll fairly quickly see how to improve the performance to make the Kruskal running time O(e log e), but the later solutions are of interest for other applications, as well as theoretically. Unlike our improved algorithms for integer multiplication and matrix multiplication, these improved data structures are simple and universally practical. The mathematical challenge is analyzing them to see exactly how fast they are. 2

3 A union-find data structure maintains a family of disjoint nonempty sets. Each set in the family has an element called its label. The data structure supports the following three operations: MAKESET(x) creates a new set {x} and adds it to the family. The element x must not be an element of any existing set in the family. UNION(x, y) changes the family by replacing two sets, the one containing x and the one containing y, by a single set that is the union of these two sets. It is a no-op if x and y are already in the same set. FIND(x) returns the label of the set containing x. Note that the changes in the family of sets are monotone. Once an element appears it remains in some set in the family, and once two elements are in the same set they stay in the same set. It is easy to see how to use such a data structure to implement Kruskal s algorithm. We first perform a MAKESET for each vertex of the graph. Then for each edge (u, v) we do a FIND for both u and v, followed by UNION(u, v) if they are not already in the same set. 3

4 Review of Our First Solution Our earlier implementation of Kruskal s algorithm used what was in effect a simple implementation of a unionfind data structure. Let s put it into our new terminology: We maintain a table with each element and the label of its set. A MAKESET is performed by adding a new entry to the table, whose label is itself. (We are not required to check whether the element is already there.) The operation UNION(x, y) operation is performed by sweeping the table, changing the label of x, and of all entries with the same label as x, to the label of y. (Or we could change the y labels to the x labels, as long as we are consistent.) For a FIND(x) operation we simply return the label of the entry x. (We are not charged for any search for x in the table we assume that we are given a pointer to it.) 4

5 A First Improvement This method takes O(n) time for each UNION operation, where n is the number of elements in all the sets, and O(1) for each MAKESET or FIND. In the Kruskal setting we needed n 1 UNION operations and O(e) FIND operations, costing us O(n 2 + e) for the data structure operations or O(n 2 + e log e) when we include the initial sort of the edges. How can we improve this? It seems wasteful to conduct a linear search through the table to update the labels for the UNION. What if we keep each set in a linked list rather than keeping all of them in a table? We make a list node for each element and give the node two pointers one to the label (at the head of the list) and one to the next element of the set. Now a MAKESET is still O(1) (create a new node with a label pointer to itself and a null next pointer) and a FIND is O(1) as well (just return the label pointer of the desired node). How can we implement a UNION? 5

6 To implement UNION(x, y) we must: locate the heads of the two lists and the tail of x s list update the next pointer of x s tail to y s head change the label pointer of each node in y s list to x s head How long does this take? The list heads are given to us by pointers from x and y, and we can keep a single pointer from the head to the tail of each list. The appending is then O(1) time, but we have to change a separate pointer for each node in y s list. This could certainly be O(n) in the worst case. But we noted before that we could either merge x s set into y s or vice versa. Does it make a difference? 6

7 The preponderence of the time taken by our UNION operation is needed to change the labels of the elements of y s set. What if we keep track of the size of each set, and always merge the smaller set into the larger? It seems at though this would save at least some time. This doesn t help us in the worst case, because if our last merge happens to deal with sets of roughly equal size, we will need Θ(n) time to do it. But what we really care about is the total time to perform a series of operations. Determining this total time is our first example of amortized analysis. By breaking up the set of operations in a different way, we can prove a bound on its total size. Let s consider the total time to perform n 1 UNION operations. The bulk of the time spent is proportional to the numer of labels changed. Rather than divide the label changes into phases for each UNION, we group together all the times that the label of a given element is changed. Look at one particular element x. It begins life in a set of size 1. When its label is changed, it must go into a set of size at least 2. The next time it is changed, this set of size at least 2 is merged with another set that is at least as big. By induction, we can see that after i label changes, x is in a set of size at least 2 i. 7

8 Hence each element undergoes at most log n label changes. Every label change is assigned to an element, so there are at most n log n, hence O(n log n), in all. The other operation in the n UNIONs take only O(n) total time. This gives us the result: Theorem: The linked-list implementation, always merging the smaller set into the larger, takes O(n log n + m) time to carry out any sequence of m union-find operations on a set of size n. This is enough to remove the bottleneck in Kruskal s algorithm, so that it takes O(e log e) time. There are faster ways to find an MST. Prim s algorithm (cf. HW#2) takes O(e + n log n) if Fibonacci heaps are used to implement the priority queues needed. [CLRS] has more if you are interested. 8

9 Representing Sets as Trees Can we do even better? One idea is to avoid updating every element of the smaller set each time, by making the structure for each set a tree instead of a linked list. Now the node for each element has just one pointer, to its parent: MAKESET: Create a node whose parent is itself, a root. UNION: Change the parent of the root of the smaller tree to be the root of the larger. FIND(x) : Follow the path from x to the root of its component, and return that root as the label. 9

10 There s a problem here the time for a FIND is no longer O(1) but O(h), where h is the height of the tree. A UNION takes only O(1) once we have found the two roots, but finding them also takes O(h). We ll thus reclassify the UNION as two FIND operations plus the pointer changes to merge the trees, which we ll call LINK. With the smaller-into-larger rule, it is easy to prove by induction that a tree of height h always has at least 2 h nodes, so that a tree with k nodes always has height O(log k). Actually, we get this property from a simpler rule we keep track of the rank (height) of each node and always merge the tree of lower rank into that of higher rank. 10

11 Improvements to the Tree Method When we carry out the operation FIND(x), we compute the label not only of x but of all the nodes on the path from x to its root. If we save this information, we can carry out a future FIND on x, or on one of its ancestors, more quickly. Let s try investing some time after each FIND to change the parent pointers of all the nodes on the path to the newly found root. This is called tree compression and at most doubles the time for a FIND. ([CLRS] gives a nice recursive implementation of this that avoids the need for any new pointers.) What does this buy us? Remember that each UNION includes two FIND operations, and these will tend to flatten the trees for the components they involve. The FIND operations still take O(h) time, where h is the height of the tree, but can we hope that h will be smaller than its former O(log n)? 11

12 Ackermann s Function The analysis of our last (and fastest) implementation of a union-find data structure turns out to involve a function first defined as a mathematical curiosity, Ackermann s function. In CMPSCI 601 we define the primitive recursive functions from N (the natural numbers) to N as the possible results of a particular kind of recursive definition. An equivalent definition is the set of function that can be computer in an imperative language with bounded loops but no unbounded loops. Ackermann devised his function to be computable (by a Turing machine, for example) but faster-growing than any primitive recursive function. 12

13 Definition: We define an infinite sequence of functions A 0, A 1, A 2,... as follows: A 0 (x) = 1 + x for any x N for k > 0, A k (x) is A k 1 applied to x, x times We can compute the first few A i s easily: A 1 (x) is the result of adding 1 to x, x times, or 2x A 2 (x) is the result of doubling x, x times, or x2 x A 3 (x) is greater than the result of raising 2 to the power x, x times. The latter number is called exp (x), a tower of twos x high A 4 (x) is a lot bigger than that... 13

14 Definition: The Ackermann function A is defined by A(k) = A k (2). Thus A(0) = A 0 (2) = = 3, A(1) = A 1 (2) = 2(2) = 4, A(2) = A 2 (2) = = 8, and A(3) = A 3 (2) = A 2 (A 2 (2)) = A 2 (8) = = The next value, A(4), is greater than exp (2048), an inconceivably huge number. Definition: The inverse Ackermann function α(n) is defined to be the smallest number k such that A(k) n. Thus α(n) 4 for any conceivable number. Theorem: The running time needed to perform m unionfind operations on a set of size n is Θ(mα(n)), and the upper bound is achieved by our last algorithm. We ll sketch the proof of the upper bound as time permits. 14

15 A Potential Function The technique used to prove the O(mα(n)) upper bound is an interesting one. In mechanics the total energy of a system consists of both kinetic and potential energy. Here we will define a potential function on states of the data structure, and measure the number of steps taken plus the increase in potential (or minus the decrease in potential) for each operation. We ll prove that each operation costs O(α(n)) in this amortized cost measure. Even though a single operation might take more steps than that, it must reduce the potential at the same time. Since the potential will start at zero and finish non-negative, the O(mα(n)) bound on the total amortized cost implies an O(mα(n)) bound on the actual cost for all m steps. 15

16 The potential of the whole system will be the sum of a non-negative potential for each node. The cost of a root node, or of any node of rank zero, will be α(n) times its rank. Other nodes have a more complicated definition of potential, for which we ll omit the details (see section 21.4 of [CLRS]). There are two parameters of such a node called the level l(x) and iteration number i(x), such that 0 l(x) < α(n) and 1 i(x) r(x). The exact potential of a node x is r(x)[α(n) l(x)] i(x). 16

17 Effect of Moves on the Potential The MAKESET operation creates a new node with zero rank and hence zero potential, so its only cost is O(1) for the actual operations it uses. Recall that we broke the UNION operation into two FIND s and a LINK. The latter changes only one pointer, from the root of one tree to the root of another. If these two roots were at the same rank, the LINK increases the rank of one root by one, and thus increases its potential by α(n). It turns out that no other node s potential is changed, and since there are only O(1) actual operations the total amortized cost is O(α(n)). Finally, a FIND operation involves all the nodes from the node found to its root the number of these nodes, s, may be greater than α. But it turns out that with this definition of potential, a FIND operation with its tree compression decreases the potential of all but possibly α(n) + 2 of these s nodes. With a careful choice of the units with which to measure actual steps, this makes the amortized cost of the FIND operation only O(α(n)). 17

18 So any sequence of m MAKESET, FIND, and LINK operations takes O(mα(n)) time. The same holds, with a different constant, for any sequence of m MAKESET, FIND, and UNION operations. There is a matching lower bound argument that we omit here. It shows that any data structure for union-find, under certain general assumptions, must take Ω(mα(n)) time. Detailed references may be found at the end of Chapter 21 of [CLRS]. 18

Lecture 10 Union-Find The union-nd data structure is motivated by Kruskal's minimum spanning tree algorithm (Algorithm 2.6), in which we needed two operations on disjoint sets of vertices: determine whether

### CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.

Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure

### Analysis of Algorithms, I

Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications

### Minimum Spanning Trees

Minimum Spanning Trees Algorithms and 18.304 Presentation Outline 1 Graph Terminology Minimum Spanning Trees 2 3 Outline Graph Terminology Minimum Spanning Trees 1 Graph Terminology Minimum Spanning Trees

### Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

### Computer Algorithms. Merge Sort. Analysis of Merge Sort. Recurrences. Recurrence Relations. Iteration Method. Lecture 6

Computer Algorithms Lecture 6 Merge Sort Sorting Problem: Sort a sequence of n elements into non-decreasing order. Divide: Divide the n-element sequence to be sorted into two subsequences of n/ elements

### Data Structures Fibonacci Heaps, Amortized Analysis

Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure:

### Lecture 7: Approximation via Randomized Rounding

Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

### CS711008Z Algorithm Design and Analysis

CS711008Z Algorithm Design and Analysis Lecture 7 Binary heap, binomial heap, and Fibonacci heap 1 Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 The slides were

### Complexity of Union-Split-Find Problems. Katherine Jane Lai

Complexity of Union-Split-Find Problems by Katherine Jane Lai S.B., Electrical Engineering and Computer Science, MIT, 2007 S.B., Mathematics, MIT, 2007 Submitted to the Department of Electrical Engineering

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### Union-Find Algorithms. network connectivity quick find quick union improvements applications

Union-Find Algorithms network connectivity quick find quick union improvements applications 1 Subtext of today s lecture (and this course) Steps to developing a usable algorithm. Define the problem. Find

### Steven Skiena. skiena

Lecture 3: Recurrence Relations (1997) Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Argue the solution to T(n) =

### GRAPH THEORY and APPLICATIONS. Trees

GRAPH THEORY and APPLICATIONS Trees Properties Tree: a connected graph with no cycle (acyclic) Forest: a graph with no cycle Paths are trees. Star: A tree consisting of one vertex adjacent to all the others.

### A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points

A. V. Gerbessiotis CS 610-102 Spring 2014 PS 3 Mar 24, 2014 No points Problem 1. Suppose that we insert n keys into a hash table of size m using open addressing and uniform hashing. Let p(n, m) be the

### Euclidean Minimum Spanning Trees Based on Well Separated Pair Decompositions Chaojun Li. Advised by: Dave Mount. May 22, 2014

Euclidean Minimum Spanning Trees Based on Well Separated Pair Decompositions Chaojun Li Advised by: Dave Mount May 22, 2014 1 INTRODUCTION In this report we consider the implementation of an efficient

### Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * *

Binary Heaps A binary heap is another data structure. It implements a priority queue. Priority Queue has the following operations: isempty add (with priority) remove (highest priority) peek (at highest

### Binary Heap Algorithms

CS Data Structures and Algorithms Lecture Slides Wednesday, April 5, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks CHAPPELLG@member.ams.org 2005 2009 Glenn G. Chappell

### Analysis of Algorithms I: Binary Search Trees

Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary

### Mathematical induction & Recursion

CS 441 Discrete Mathematics for CS Lecture 15 Mathematical induction & Recursion Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Proofs Basic proof methods: Direct, Indirect, Contradiction, By Cases,

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Trees and structural induction

Trees and structural induction Margaret M. Fleck 25 October 2010 These notes cover trees, tree induction, and structural induction. (Sections 10.1, 4.3 of Rosen.) 1 Why trees? Computer scientists are obsessed

### 1. What s wrong with the following proofs by induction?

ArsDigita University Month : Discrete Mathematics - Professor Shai Simonson Problem Set 4 Induction and Recurrence Equations Thanks to Jeffrey Radcliffe and Joe Rizzo for many of the solutions. Pasted

### Merge Sort and Recurrences. COSC 581, Algorithms January 14, 2014

Merge Sort and Recurrences COSC 581, Algorithms January 14, 2014 Reading Assignments Today s class: Chapter 2, 4.0, 4.4 Reading assignment for next class: Chapter 4.2, 4.5 3 Common Algorithmic Techniques

### Data Structures. Algorithm Performance and Big O Analysis

Data Structures Algorithm Performance and Big O Analysis What s an Algorithm? a clearly specified set of instructions to be followed to solve a problem. In essence: A computer program. In detail: Defined

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### Counting Spanning Trees

Counting Spanning Trees Bang Ye Wu Kun-Mao Chao Counting Spanning Trees This book provides a comprehensive introduction to the modern study of spanning trees. A spanning tree for a graph G is a subgraph

### CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992

CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992 Original Lecture #6: 28 January 1991 Topics: Triangulating Simple Polygons

### Algorithms complexity of recursive algorithms. Jiří Vyskočil, Marko Genyk-Berezovskyj

complexity of recursive algorithms Jiří Vyskočil, Marko Genyk-Berezovskyj 2010-2014 Recurrences A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs.

### Discrete Mathematics, Chapter 3: Algorithms

Discrete Mathematics, Chapter 3: Algorithms Richard Mayr University of Edinburgh, UK Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 3 1 / 28 Outline 1 Properties of Algorithms

### SIMS 255 Foundations of Software Design. Complexity and NP-completeness

SIMS 255 Foundations of Software Design Complexity and NP-completeness Matt Welsh November 29, 2001 mdw@cs.berkeley.edu 1 Outline Complexity of algorithms Space and time complexity ``Big O'' notation Complexity

### Lecture Notes on Spanning Trees

Lecture Notes on Spanning Trees 15-122: Principles of Imperative Computation Frank Pfenning Lecture 26 April 26, 2011 1 Introduction In this lecture we introduce graphs. Graphs provide a uniform model

### Section IV.1: Recursive Algorithms and Recursion Trees

Section IV.1: Recursive Algorithms and Recursion Trees Definition IV.1.1: A recursive algorithm is an algorithm that solves a problem by (1) reducing it to an instance of the same problem with smaller

### Small Maximal Independent Sets and Faster Exact Graph Coloring

Small Maximal Independent Sets and Faster Exact Graph Coloring David Eppstein Univ. of California, Irvine Dept. of Information and Computer Science The Exact Graph Coloring Problem: Given an undirected

### A tree can be defined in a variety of ways as is shown in the following theorem: 2. There exists a unique path between every two vertices of G.

7 Basic Properties 24 TREES 7 Basic Properties Definition 7.1: A connected graph G is called a tree if the removal of any of its edges makes G disconnected. A tree can be defined in a variety of ways as

### Theorem (The division theorem) Suppose that a and b are integers with b > 0. There exist unique integers q and r so that. a = bq + r and 0 r < b.

Theorem (The division theorem) Suppose that a and b are integers with b > 0. There exist unique integers q and r so that a = bq + r and 0 r < b. We re dividing a by b: q is the quotient and r is the remainder,

### Chapter 4. Trees. 4.1 Basics

Chapter 4 Trees 4.1 Basics A tree is a connected graph with no cycles. A forest is a collection of trees. A vertex of degree one, particularly in a tree, is called a leaf. Trees arise in a variety of applications.

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### Mathematical Induction. Lecture 10-11

Mathematical Induction Lecture 10-11 Menu Mathematical Induction Strong Induction Recursive Definitions Structural Induction Climbing an Infinite Ladder Suppose we have an infinite ladder: 1. We can reach

### Union-Find Problem. Using Arrays And Chains

Union-Find Problem Given a set {,,, n} of n elements. Initially each element is in a different set. ƒ {}, {},, {n} An intermixed sequence of union and find operations is performed. A union operation combines

### Solutions to Exercises 8

Discrete Mathematics Lent 2009 MA210 Solutions to Exercises 8 (1) Suppose that G is a graph in which every vertex has degree at least k, where k 1, and in which every cycle contains at least 4 vertices.

### The Selection Problem

The Selection Problem The selection problem: given an integer k and a list x 1,..., x n of n elements find the k-th smallest element in the list Example: the 3rd smallest element of the following list

### COLORED GRAPHS AND THEIR PROPERTIES

COLORED GRAPHS AND THEIR PROPERTIES BEN STEVENS 1. Introduction This paper is concerned with the upper bound on the chromatic number for graphs of maximum vertex degree under three different sets of coloring

Answers to Homework 1 p. 13 1.2 3 What is the smallest value of n such that an algorithm whose running time is 100n 2 runs faster than an algorithm whose running time is 2 n on the same machine? Find the

### Full and Complete Binary Trees

Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full

### root node level: internal node edge leaf node CS@VT Data Structures & Algorithms 2000-2009 McQuain

inary Trees 1 A binary tree is either empty, or it consists of a node called the root together with two binary trees called the left subtree and the right subtree of the root, which are disjoint from each

### COT5405 Analysis of Algorithms Homework 3 Solutions

COT0 Analysis of Algorithms Homework 3 Solutions. Prove or give a counter example: (a) In the textbook, we have two routines for graph traversal - DFS(G) and BFS(G,s) - where G is a graph and s is any

### Data Structure [Question Bank]

Unit I (Analysis of Algorithms) 1. What are algorithms and how they are useful? 2. Describe the factor on best algorithms depends on? 3. Differentiate: Correct & Incorrect Algorithms? 4. Write short note:

### Discrete Mathematics (2009 Spring) Induction and Recursion (Chapter 4, 3 hours)

Discrete Mathematics (2009 Spring) Induction and Recursion (Chapter 4, 3 hours) Chih-Wei Yi Dept. of Computer Science National Chiao Tung University April 17, 2009 4.1 Mathematical Induction 4.1 Mathematical

### Union-find with deletions

19 Union-find with deletions Haim Kaplan * Nira Shafrir Robert E. Tarjan ~ Abstract In the classical union-find problem we maintain a partition of a universe of n elements into disjoint sets subject to

### Recurrences (CLRS )

Recurrences (CLRS 4.1-4.2) Last time we discussed divide-and-conquer algorithms Divide and Conquer To Solve P: 1. Divide P into smaller problems P 1,P 2,P 3...P k. 2. Conquer by solving the (smaller) subproblems

### Symbol Tables. IE 496 Lecture 13

Symbol Tables IE 496 Lecture 13 Reading for This Lecture Horowitz and Sahni, Chapter 2 Symbol Tables and Dictionaries A symbol table is a data structure for storing a list of items, each with a key and

### Michal Forišek: Early beta verzia skrípt z ADŠ

Časová zložitosť II Michal Forišek: Early beta verzia skrípt z ADŠ In this part of the article we will focus on estimating the time complexity for recursive programs. In essence, this will lead to finding

### From Last Time: Remove (Delete) Operation

CSE 32 Lecture : More on Search Trees Today s Topics: Lazy Operations Run Time Analysis of Binary Search Tree Operations Balanced Search Trees AVL Trees and Rotations Covered in Chapter of the text From

### CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

### 4 Basics of Trees. Petr Hliněný, FI MU Brno 1 FI: MA010: Trees and Forests

4 Basics of Trees Trees, actually acyclic connected simple graphs, are among the simplest graph classes. Despite their simplicity, they still have rich structure and many useful application, such as in

### Discrete Mathematics, Chapter 5: Induction and Recursion

Discrete Mathematics, Chapter 5: Induction and Recursion Richard Mayr University of Edinburgh, UK Richard Mayr (University of Edinburgh, UK) Discrete Mathematics. Chapter 5 1 / 20 Outline 1 Well-founded

### Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

### Balanced search trees

Lecture 8 Balanced search trees 8.1 Overview In this lecture we discuss search trees as a method for storing data in a way that supports fast insert, lookup, and delete operations. (Data structures handling

### Data Structures. Jaehyun Park. CS 97SI Stanford University. June 29, 2015

Data Structures Jaehyun Park CS 97SI Stanford University June 29, 2015 Typical Quarter at Stanford void quarter() { while(true) { // no break :( task x = GetNextTask(tasks); process(x); // new tasks may

### Finding and counting given length cycles

Finding and counting given length cycles Noga Alon Raphael Yuster Uri Zwick Abstract We present an assortment of methods for finding and counting simple cycles of a given length in directed and undirected

### Pseudocode Analysis. COMP3600/6466 Algorithms Lecture 5. Recursive Algorithms. Asymptotic Bounds for Recursions

COMP600/6466 Algorithms Lecture 5 Pseudocode Analysis Iterative Recursive S 0 Dr. Hassan Hijazi Prof. Weifa Liang Recursive Algorithms Asymptotic Bounds for Recursions Total running time = Sum of times

### The following themes form the major topics of this chapter: The terms and concepts related to trees (Section 5.2).

CHAPTER 5 The Tree Data Model There are many situations in which information has a hierarchical or nested structure like that found in family trees or organization charts. The abstraction that models hierarchical

### CS/COE 1501 http://cs.pitt.edu/~bill/1501/

CS/COE 1501 http://cs.pitt.edu/~bill/1501/ Lecture 01 Course Introduction Meta-notes These notes are intended for use by students in CS1501 at the University of Pittsburgh. They are provided free of charge

### 5. A full binary tree with n leaves contains [A] n nodes. [B] log n 2 nodes. [C] 2n 1 nodes. [D] n 2 nodes.

1. The advantage of.. is that they solve the problem if sequential storage representation. But disadvantage in that is they are sequential lists. [A] Lists [B] Linked Lists [A] Trees [A] Queues 2. The

### CS2223 Algorithms B Term 2013 Exam 2 Solutions

CS2223 Algorithms B Term 2013 Exam 2 Solutions Dec. 3, 2013 By Prof. Carolina Ruiz Dept. of Computer Science WPI PROBLEM 1: Solving Recurrences (15 points) Solve the recurrence T (n) = 2T (n/3) + n using

### Learning Outcomes. COMP202 Complexity of Algorithms. Binary Search Trees and Other Search Trees

Learning Outcomes COMP202 Complexity of Algorithms Binary Search Trees and Other Search Trees [See relevant sections in chapters 2 and 3 in Goodrich and Tamassia.] At the conclusion of this set of lecture

### A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called heap-order property

CmSc 250 Intro to Algorithms Chapter 6. Transform and Conquer Binary Heaps 1. Definition A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called

### CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

### Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

### MapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research

MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With

### Divide-and-Conquer Algorithms Part One

Divide-and-Conquer Algorithms Part One Announcements Problem Set One completely due right now. Solutions distributed at the end of lecture. Programming section today in Gates B08 from from 3:45PM 5:00PM.

### Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### Distributed Computing over Communication Networks: Maximal Independent Set

Distributed Computing over Communication Networks: Maximal Independent Set What is a MIS? MIS An independent set (IS) of an undirected graph is a subset U of nodes such that no two nodes in U are adjacent.

### Lecture 4: Recurrences CSCI Algorithms I. Andrew Rosenberg

Lecture 4: Recurrences CSCI 700 - Algorithms I Andrew Rosenberg Last Time Introduction to Recursion Fibonacci Binary Search MergeSort QuickSort Today Mathematical Analysis of Recurrence Relations Mathematical

### Persistent Data Structures

6.854 Advanced Algorithms Lecture 2: September 9, 2005 Scribes: Sommer Gentry, Eddie Kohler Lecturer: David Karger Persistent Data Structures 2.1 Introduction and motivation So far, we ve seen only ephemeral

### CMSC 451: Graph Properties, DFS, BFS, etc.

CMSC 451: Graph Properties, DFS, BFS, etc. Slides By: Carl Kingsford Department of Computer Science University of Maryland, College Park Based on Chapter 3 of Algorithm Design by Kleinberg & Tardos. Graphs

### Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

### Solving Recurrences. Lecture 13 CS2110 Summer 2009

Solving Recurrences Lecture 13 CS2110 Summer 2009 In the past Complexity: Big-O definition Non-recursive programs 2 Today Complexity of recursive programs Introduce recurrences Methods to solve recurrences

### Sample Problems in Discrete Mathematics

Sample Problems in Discrete Mathematics This handout lists some sample problems that you should be able to solve as a pre-requisite to Computer Algorithms Try to solve all of them You should also read

### Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay

Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 17 Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding

### 24: Polygon Partition - Faster Triangulations

24: Polygon Partition - Faster Triangulations CS 473u - Algorithms - Spring 2005 April 15, 2005 1 Previous lecture We described triangulations of simple polygons. See Figure 1 for an example of a polygon

### Dedekind s forgotten axiom and why we should teach it (and why we shouldn t teach mathematical induction in our calculus classes)

Dedekind s forgotten axiom and why we should teach it (and why we shouldn t teach mathematical induction in our calculus classes) by Jim Propp (UMass Lowell) March 14, 2010 1 / 29 Completeness Three common

### A Turán Type Problem Concerning the Powers of the Degrees of a Graph

A Turán Type Problem Concerning the Powers of the Degrees of a Graph Yair Caro and Raphael Yuster Department of Mathematics University of Haifa-ORANIM, Tivon 36006, Israel. AMS Subject Classification:

### , each of which contains a unique key value, say k i , R 2. such that k i equals K (or to determine that no such record exists in the collection).

The Search Problem 1 Suppose we have a collection of records, say R 1, R 2,, R N, each of which contains a unique key value, say k i. Given a particular key value, K, the search problem is to locate the

### Graph Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar

Graph Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text Introduction to Parallel Computing, Addison Wesley, 3. Topic Overview Definitions and Representation Minimum

### Induction. Margaret M. Fleck. 10 October These notes cover mathematical induction and recursive definition

Induction Margaret M. Fleck 10 October 011 These notes cover mathematical induction and recursive definition 1 Introduction to induction At the start of the term, we saw the following formula for computing

### Mathematical Induction

Mathematical Induction Tan Conghui Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong September 15, 2016 Tan Conghui (ENGG 2440B) Tutorial 1 September 15,

### Binary Trees and Huffman Encoding Binary Search Trees

Binary Trees and Huffman Encoding Binary Search Trees Computer Science E119 Harvard Extension School Fall 2012 David G. Sullivan, Ph.D. Motivation: Maintaining a Sorted Collection of Data A data dictionary

### Partitioning edge-coloured complete graphs into monochromatic cycles and paths

arxiv:1205.5492v1 [math.co] 24 May 2012 Partitioning edge-coloured complete graphs into monochromatic cycles and paths Alexey Pokrovskiy Departement of Mathematics, London School of Economics and Political

### CSE373: Data Structures and Algorithms Lecture 2: Proof by Induction. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 2: Proof by Induction Linda Shapiro Winter 2015 Background on Induction Type of mathematical proof Typically used to establish a given statement for all natural

### Lecture 2: Analysis of Algorithms (CS )

Lecture 2: Analysis of Algorithms (CS583-002) Amarda Shehu September 03 & 10, 2014 1 Outline of Today s Class 2 Big-Oh, Big-Omega, Theta Techniques for Finding Asymptotic Relationships Some more Fun with

### Balanced Binary Search Tree

AVL Trees / Slide 1 Balanced Binary Search Tree Worst case height of binary search tree: N-1 Insertion, deletion can be O(N) in the worst case We want a tree with small height Height of a binary tree with

### Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras

Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture No. # 31 Recursive Sets, Recursively Innumerable Sets, Encoding

### 1. R In this and the next section we are going to study the properties of sequences of real numbers.

+a 1. R In this and the next section we are going to study the properties of sequences of real numbers. Definition 1.1. (Sequence) A sequence is a function with domain N. Example 1.2. A sequence of real

### CS420/520 Algorithm Analysis Spring 2010 Lecture 04

CS420/520 Algorithm Analysis Spring 2010 Lecture 04 I. Chapter 4 Recurrences A. Introduction 1. Three steps applied at each level of a recursion a) Divide into a number of subproblems that are smaller

### COMP 250 Fall Mathematical induction Sept. 26, (n 1) + n = n + (n 1)

COMP 50 Fall 016 9 - Mathematical induction Sept 6, 016 You will see many examples in this course and upcoming courses of algorithms for solving various problems It many cases, it will be obvious that