# Algorithms. Algorithms 1.5 UNION-FIND. dynamic connectivity quick find quick union improvements applications ROBERT SEDGEWICK KEVIN WAYNE

Save this PDF as:

Size: px
Start display at page:

Download "Algorithms. Algorithms 1.5 UNION-FIND. dynamic connectivity quick find quick union improvements applications ROBERT SEDGEWICK KEVIN WAYNE"

## Transcription

1 Algorithms ROBERT SEDGEWICK KEVIN WAYNE 1.5 UNION-FIND Algorithms F O U R T H E D I T I O N ROBERT SEDGEWICK KEVIN WAYNE dynamic connectivity quick find quick union improvements applications

2 Subtext of today s lecture (and this course) Steps to developing a usable algorithm. Model the problem. Find an algorithm to solve it. Fast enough? Fits in memory? If not, figure out why not. Find a way to address the problem. Iterate until satisfied. The scientific method. Mathematical analysis. 2

3 1.5 UNION-FIND Algorithms ROBERT SEDGEWICK KEVIN WAYNE dynamic connectivity quick find quick union improvements applications

4 Dynamic connectivity problem Given a set of N objects, support two operation: Connect two objects. Is there a path connecting the two objects? connect 4 and 3 connect 3 and 8 connect 6 and 5 connect 9 and 4 connect 2 and 1 are 0 and 7 connected? are 8 and 9 connected? connect 5 and 0 connect 7 and 2 connect 6 and 1 connect 1 and 0 are 0 and 7 connected?

5 A larger connectivity example Q. Is there a path connecting p and q? p q A. Yes. 5

6 Modeling the objects Applications involve manipulating objects of all types. Pixels in a digital photo. Computers in a network. Friends in a social network. Transistors in a computer chip. Elements in a mathematical set. Variable names in a Fortran program. Metallic sites in a composite system. When programming, convenient to name objects 0 to N 1. Use integers as array index. Suppress details not relevant to union-find. can use symbol table to translate from site names to integers: stay tuned (Chapter 3) 6

7 Modeling the connections We assume "is connected to" is an equivalence relation: Reflexive: p is connected to p. Symmetric: if p is connected to q, then q is connected to p. Transitive: if p is connected to q and q is connected to r, then p is connected to r. Connected component. Maximal set of objects that are mutually connected { 0 } { } { } 3 connected components 7

8 Implementing the operations Find. In which component is object p? Connected. Are objects p and q in the same component? Union. Replace components containing objects p and q with their union union(2, 5) { 0 } { } { } { 0 } { } 3 connected components 2 connected components 8

9 Union-find data type (API) Goal. Design efficient data structure for union-find. Number of objects N can be huge. Number of operations M can be huge. Union and find operations may be intermixed. public class UF UF(int N) initialize union-find data structure with N singleton objects (0 to N 1) void union(int p, int q) add connection between p and q int find(int p) component identifier for p (0 to N 1) boolean connected(int p, int q) are p and q in the same component? public boolean connected(int p, int q) { return find(p) == find(q); } 1-line implementation of connected() 9

10 Dynamic-connectivity client Read in number of objects N from standard input. Repeat: read in pair of integers from standard input if they are not yet connected, connect them and print out pair public static void main(string[] args) { int N = StdIn.readInt(); UF uf = new UF(N); while (!StdIn.isEmpty()) { int p = StdIn.readInt(); int q = StdIn.readInt(); if (!uf.connected(p, q)) { uf.union(p, q); StdOut.println(p + " " + q); } } } % more tinyuf.txt already connected 10

11 1.5 UNION-FIND Algorithms ROBERT SEDGEWICK KEVIN WAYNE dynamic connectivity quick find quick union improvements applications

12 Quick-find [eager approach] Data structure. if and only if Integer array id[] of length N. Interpretation: id[p] is the id of the component containing p. id[] , 5 and 6 are connected 1, 2, and 7 are connected 3, 4, 8, and 9 are connected

13 Quick-find [eager approach] Data structure. Integer array id[] of length N. Interpretation: id[p] is the id of the component containing p id[] Find. What is the id of p? Connected. Do p and q have the same id? id[6] = 0; id[1] = 1 6 and 1 are not connected Union. To merge components containing p and q, change all entries whose id equals id[p] to id[q] id[] after union of 6 and 1 problem: many values can change 13

14 Quick-find demo id[]

15 Quick-find demo id[]

16 Quick-find: Java implementation public class QuickFindUF { private int[] id; public QuickFindUF(int N) { } id = new int[n]; for (int i = 0; i < N; i++) id[i] = i; set id of each object to itself (N array accesses) public int find(int p) { return id[p]; } return the id of p (1 array access) } public void union(int p, int q) { } int pid = id[p]; int qid = id[q]; for (int i = 0; i < id.length; i++) if (id[i] == pid) id[i] = qid; change all entries with id[p] to id[q] (at most 2N + 2 array accesses) 16

17 Quick-find is too slow Cost model. Number of array accesses (for read or write). algorithm initialize union find connected quick-find N N 1 1 order of growth of number of array accesses quadratic Union is too expensive. It takes N 2 array accesses to process a sequence of N union operations on N objects. 17

18 Quadratic algorithms do not scale Rough standard (for now). 109 operations per second. a truism (roughly) since 1950! 109 words of main memory. Touch all words in approximately 1 second. Ex. Huge problem for quick-find. 109 union commands on 10 9 objects. Quick-find takes more than 1018 operations. 30+ years of computer time! Quadratic algorithms don't scale with technology. New computer may be 10x as fast. But, has 10x as much memory want to solve a problem that is 10x as big. With quadratic algorithm, takes 10x as long! time 64T 32T 16T 8T quadratic linearithmic linear size 1K 2K 4K 8K 18

19 1.5 UNION-FIND Algorithms ROBERT SEDGEWICK KEVIN WAYNE dynamic connectivity quick find quick union improvements applications

20 Quick-union [lazy approach] Data structure. Integer array id[] of length N. Interpretation: id[i] is parent of i. Root of i is id[id[id[...id[i]...]]]. keep going until it doesn t change (algorithm ensures no cycles) id[] parent of 3 is 4 root of 3 is 9 20

21 Quick-union [lazy approach] Data structure. Integer array id[] of length N. Interpretation: id[i] is parent of i. Root of i is id[id[id[...id[i]...]]] id[] q p 3 Find. What is the root of p? Connected. Do p and q have the same root? root of 3 is 9 root of 5 is 6 3 and 5 are not connected Union. To merge components containing p and q, set the id of p's root to the id of q's root q id[] only one value changes p 3 21

22 Quick-union demo id[]

23 Quick-union demo id[]

24 Quick-union: Java implementation public class QuickUnionUF { private int[] id; public QuickUnionUF(int N) { id = new int[n]; for (int i = 0; i < N; i++) id[i] = i; } set id of each object to itself (N array accesses) public int find(int i) { while (i!= id[i]) i = id[i]; return i; } chase parent pointers until reach root (depth of i array accesses) } public void union(int p, int q) { int i = find(p); int j = find(q); id[i] = j; } change root of p to point to root of q (depth of p and q array accesses) 24

25 Quick-union is also too slow Cost model. Number of array accesses (for read or write). algorithm initialize union find connected quick-find N N 1 1 quick-union N N N N worst case includes cost of finding roots Quick-find defect. Union too expensive (N array accesses). Trees are flat, but too expensive to keep them flat. Quick-union defect. Trees can get tall. Find/connected too expensive (could be N array accesses). 25

26 1.5 UNION-FIND Algorithms ROBERT SEDGEWICK KEVIN WAYNE dynamic connectivity quick find quick union improvements applications

27 Improvement 1: weighting Weighted quick-union. Modify quick-union to avoid tall trees. Keep track of size of each tree (number of objects). Balance by linking root of smaller tree to root of larger tree. p p q quick-union q reasonable alternatives: union by height or "rank" p smaller tree larger tree might put the larger tree lower smaller tree larger tree weighted p always chooses the better alternative q q larger tree smaller tree smaller tree larger tree 27

28 Weighted quick-union demo id[]

29 Weighted quick-union demo id[]

30 Quick-union and weighted quick-union example quick-union weighted average distance to root: 5.11 weighted average distance to root: 1.52 Quick-union and weighted quick-union (100 sites, 88 union() operations) 30

31 Weighted quick-union: Java implementation Data structure. Same as quick-union, but maintain extra array sz[i] to count number of objects in the tree rooted at i. Find/connected. Identical to quick-union. Union. Modify quick-union to: Link root of smaller tree to root of larger tree. Update the sz[] array. int i = find(p); int j = find(q); if (i == j) return; if (sz[i] < sz[j]) { id[i] = j; sz[j] += sz[i]; } else { id[j] = i; sz[i] += sz[j]; } 31

32 Weighted quick-union analysis Running time. Find: takes time proportional to depth of p. Union: takes constant time, given roots. lg = base-2 logarithm Proposition. Depth of any node x is at most lg N depth 2 x N = 10 depth(x) = 3 lg N 32

33 Weighted quick-union analysis Running time. Find: takes time proportional to depth of p. Union: takes constant time, given roots. lg = base-2 logarithm Proposition. Depth of any node x is at most lg N. Pf. What causes the depth of object x to increase? Increases by 1 when tree T1 containing x is merged into another tree T2. The size of the tree containing x at least doubles since T 2 T 1. Size of tree containing x can double at most lg N times. Why? T2 T1 x 8 16 N lg N 33

34 Weighted quick-union analysis Running time. Find: takes time proportional to depth of p. Union: takes constant time, given roots. Proposition. Depth of any node x is at most lg N. algorithm initialize union find connected quick-find N N 1 1 quick-union N N N N weighted QU N lg N lg N lg N includes cost of finding roots Q. Stop at guaranteed acceptable performance? A. No, easy to improve further. 34

35 Improvement 2: path compression Quick union with path compression. Just after computing the root of p, set the id[] of each examined node to point to that root. 0 root x 9 p

36 Improvement 2: path compression Quick union with path compression. Just after computing the root of p, set the id[] of each examined node to point to that root. 0 root p x

37 Improvement 2: path compression Quick union with path compression. Just after computing the root of p, set the id[] of each examined node to point to that root. 0 root p x

38 Improvement 2: path compression Quick union with path compression. Just after computing the root of p, set the id[] of each examined node to point to that root. 0 root p x

39 Improvement 2: path compression Quick union with path compression. Just after computing the root of p, set the id[] of each examined node to point to that root. x 0 root p Bottom line. Now, find() has the side effect of compressing the tree. 39

40 Path compression: Java implementation Two-pass implementation: add second loop to find() to set the id[] of each examined node to the root. Simpler one-pass variant (path halving): Make every other node in path point to its grandparent. public int find(int i) { } while (i!= id[i]) { id[i] = id[id[i]]; i = id[i]; } return i; only one extra line of code! In practice. No reason not to! Keeps tree almost completely flat. 40

41 Weighted quick-union with path compression: amortized analysis Proposition. [Hopcroft-Ulman, Tarjan] Starting from an empty data structure, any sequence of M union-find ops on N objects makes c ( N + M lg* N ) array accesses. Analysis can be improved to N + M α(m, N). Simple algorithm with fascinating mathematics. N lg* N Linear-time algorithm for M union-find ops on N objects? iterated lg function Cost within constant factor of reading in the data. In theory, WQUPC is not quite linear. In practice, WQUPC is linear. Amazing fact. [Fredman-Saks] No linear-time algorithm exists. in "cell-probe" model of computation 41

42 Summary Key point. Weighted quick union (and/or path compression) makes it possible to solve problems that could not otherwise be addressed. algorithm quick-find quick-union weighted QU QU + path compression weighted QU + path compression worst-case time M N M N N + M log N N + M log N N + M lg* N order of growth for M union-find operations on a set of N objects Ex. [10 9 unions and finds with 10 9 objects] WQUPC reduces time from 30 years to 6 seconds. Supercomputer won't help much; good algorithm enables solution. 42

43 1.5 UNION-FIND Algorithms ROBERT SEDGEWICK KEVIN WAYNE dynamic connectivity quick find quick union improvements applications

44 Union-find applications Percolation. Games (Go, Hex). Dynamic connectivity. Least common ancestor. Equivalence of finite state automata. Hoshen-Kopelman algorithm in physics. Hinley-Milner polymorphic type inference. Kruskal's minimum spanning tree algorithm. Compiling equivalence statements in Fortran. Morphological attribute openings and closings. Matlab's bwlabel() function in image processing. 44

45 Percolation An abstract model for many physical systems: N-by-N grid of sites. Each site is open with probability p (and blocked with probability 1 p). System percolates iff top and bottom are connected by open sites. percolates blocked site does not percolate N = 8 open site open site connected to top no open site connected to top 45

46 Percolation An abstract model for many physical systems: N-by-N grid of sites. Each site is open with probability p (and blocked with probability 1 p). System percolates iff top and bottom are connected by open sites. model system vacant site occupied site percolates electricity material conductor insulated conducts fluid flow material empty blocked porous social interaction population person empty communicates 46

47 Likelihood of percolation Depends on grid size N and site vacancy probability p. p low (0.4) does not percolate p medium (0.6) percolates? p high (0.8) percolates 47

48 Percolation phase transition When N is large, theory guarantees a sharp threshold p*. p > p*: almost certainly percolates. p < p*: almost certainly does not percolate. Q. What is the value of p*? 1 percolation probability N = p* site vacancy probability p 48

49 Monte Carlo simulation Initialize all sites in an N-by-N grid to be blocked. Declare random sites open until top connected to bottom. Vacancy percentage estimates p*. full open site (connected to top) empty open site (not connected to top) blocked site N = 20 49

50 Dynamic connectivity solution to estimate percolation threshold Q. How to check whether an N-by-N system percolates? A. Model as a dynamic connectivity problem and use union-find. N = 5 open site blocked site 50

51 Dynamic connectivity solution to estimate percolation threshold Q. How to check whether an N-by-N system percolates? Create an object for each site and name them 0 to N 2 1. N = open site blocked site 51

52 Dynamic connectivity solution to estimate percolation threshold Q. How to check whether an N-by-N system percolates? Create an object for each site and name them 0 to N 2 1. Sites are in same component iff connected by open sites. N = 5 open site blocked site 52

53 Dynamic connectivity solution to estimate percolation threshold Q. How to check whether an N-by-N system percolates? Create an object for each site and name them 0 to N 2 1. Sites are in same component iff connected by open sites. Percolates iff any site on bottom row is connected to any site on top row. brute-force algorithm: N 2 calls to connected() N = 5 top row bottom row open site blocked site 53

54 Dynamic connectivity solution to estimate percolation threshold Clever trick. Introduce 2 virtual sites (and connections to top and bottom). Percolates iff virtual top site is connected to virtual bottom site. more efficient algorithm: only 1 call to connected() virtual top site N = 5 top row bottom row open site virtual bottom site blocked site 54

55 Dynamic connectivity solution to estimate percolation threshold Q. How to model opening a new site? open this site N = 5 open site blocked site 55

56 Dynamic connectivity solution to estimate percolation threshold Q. How to model opening a new site? A. Mark new site as open; connect it to all of its adjacent open sites. up to 4 calls to union() open this site N = 5 open site blocked site 56

57 Percolation threshold Q. What is percolation threshold p*? A. About for large square lattices. constant known only via simulation 1 percolation probability N = p* site vacancy probability p Fast algorithm enables accurate answer to scientific question. 57

58 Subtext of today s lecture (and this course) Steps to developing a usable algorithm. Model the problem. Find an algorithm to solve it. Fast enough? Fits in memory? If not, figure out why. Find a way to address the problem. Iterate until satisfied. The scientific method. Mathematical analysis. 58

### Union-Find Algorithms. network connectivity quick find quick union improvements applications

Union-Find Algorithms network connectivity quick find quick union improvements applications 1 Subtext of today s lecture (and this course) Steps to developing a usable algorithm. Define the problem. Find

### Union-Find Problem. Using Arrays And Chains

Union-Find Problem Given a set {,,, n} of n elements. Initially each element is in a different set. ƒ {}, {},, {n} An intermixed sequence of union and find operations is performed. A union operation combines

### 2.3 Quicksort. quicksort selection duplicate keys system sorts

2.3 Quicksort quicksort selection duplicate keys system sorts Algorithms in Java, 4 th Edition Robert Sedgewick and Kevin Wayne Copyright 2009 January 22, 2010 4:05:04 PM Two classic sorting algorithms

### 4.2 Sorting and Searching

Sequential Search: Java Implementation 4.2 Sorting and Searching Scan through array, looking for key. search hit: return array index search miss: return -1 public static int search(string key, String[]

### ! Insert a value with specified key. ! Search for value given key. ! Delete value with given key. ! O(log N) time per op.

Symbol Table Review 4.4 Balanced Trees Symbol table: key-value pair abstraction.! a value with specified key.! for value given key.! value with given key. Randomized BST.! O(log ) time per op. unless you

### Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

### The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

### Priority Queues. Client, Implementation, Interface. Priority Queues. Abstract Data Types

Client, Implementation, Interface Priority Queues Priority Queue ADT Binary heaps Heapsort Reference: Chapter 6, Algorithms in Java, 3 rd Edition, Robert Sedgewick. Separate interface and implementation

### 1.4 Arrays Introduction to Programming in Java: An Interdisciplinary Approach Robert Sedgewick and Kevin Wayne Copyright 2002 2010 2/6/11 12:33 PM!

1.4 Arrays Introduction to Programming in Java: An Interdisciplinary Approach Robert Sedgewick and Kevin Wayne Copyright 2002 2010 2/6/11 12:33 PM! A Foundation for Programming any program you might want

### 1.4 Arrays. A Foundation for Programming. Arrays. This lecture. Store and manipulate huge quantities of data.

1.4 Arrays 1 A Foundation for Programming Arrays This lecture. Store and manipulate huge quantities of data. Array. Indexed sequence of values of the same type. any program you might want to write objects

### PRIORITY QUEUES. binary heaps d-ary heaps binomial heaps Fibonacci heaps

PRIORITY QUEUES binary heaps d-ary heaps binomial heaps Fibonacci heaps Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley Copyright 2013 Kevin Wayne http://www.cs.princeton.edu/~wayne/kleinberg-tardos

### 1.2 Built-in Types of Data

1.2 Built-in Types of Data Introduction to Programming in Java: An Interdisciplinary Approach Robert Sedgewick and Kevin Wayne Copyright 2002 2010 1/29/11 6:40 AM! Built-in Data Types Data type. A set

### Motivation Suppose we have a database of people We want to gure out who is related to whom Initially, we only have a list of people, and information a

CSE 220: Handout 29 Disjoint Sets 1 Motivation Suppose we have a database of people We want to gure out who is related to whom Initially, we only have a list of people, and information about relations

### CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.

Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure

### Data Structures. Jaehyun Park. CS 97SI Stanford University. June 29, 2015

Data Structures Jaehyun Park CS 97SI Stanford University June 29, 2015 Typical Quarter at Stanford void quarter() { while(true) { // no break :( task x = GetNextTask(tasks); process(x); // new tasks may

Lecture 10 Union-Find The union-nd data structure is motivated by Kruskal's minimum spanning tree algorithm (Algorithm 2.6), in which we needed two operations on disjoint sets of vertices: determine whether

### Data Structures Fibonacci Heaps, Amortized Analysis

Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure:

### cs2010: algorithms and data structures

cs2010: algorithms and data structures Lecture 11: Symbol Table ADT. Vasileios Koutavas 28 Oct 2015 School of Computer Science and Statistics Trinity College Dublin Algorithms ROBERT SEDGEWICK KEVIN WAYNE

### Mergesort and Quicksort

Mergesort Mergesort and Quicksort Basic plan: Divide array into two halves. Recursively sort each half. Merge two halves to make sorted whole. mergesort mergesort analysis quicksort quicksort analysis

### Binary Heap Algorithms

CS Data Structures and Algorithms Lecture Slides Wednesday, April 5, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks CHAPPELLG@member.ams.org 2005 2009 Glenn G. Chappell

Algorithms ROBERT SEDGEWICK KEVIN WAYNE 3.1 SYMBOL TABLES Algorithms F O U R T H E D I T I O N API elementary implementations ordered operations ROBERT SEDGEWICK KEVIN WAYNE http://algs4.cs.princeton.edu

### 28/9/ Recursion. Overview. Greatest Common Divisor. Greatest Common Divisor. Greatest Common Divisor. Greatest Common Divisor

8/9/01 Overview.3 Recursion What is recursion? When one function calls itself directly or indirectly. Why learn recursion? New mode of thinking. Powerful programming paradigm. Many computations are naturally

### http://algs4.cs.princeton.edu dictionary find definition word definition book index find relevant pages term list of page numbers

ROBERT SEDGEWICK KEVI WAYE F O U R T H E D I T I O ROBERT SEDGEWICK KEVI WAYE ROBERT SEDGEWICK KEVI WAYE Symbol tables Symbol table applications Key-value pair abstraction. Insert a value with specified.

### 1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++

Answer the following 1) The postfix expression for the infix expression A+B*(C+D)/F+D*E is ABCD+*F/DE*++ 2) Which data structure is needed to convert infix notations to postfix notations? Stack 3) The

### Balanced Trees trees red-black trees B-trees. References: Algorithms in Java, Chapter 13.

Balanced Trees 2-3-4 trees red-black trees B-trees References: Algorithms in Java, Chapter 13 http://www.cs.princeton.edu/introalgsds/44balanced 1 Symbol Table Review Symbol table: key-value pair abstraction.

### Symbol Tables. Introduction

Symbol Tables Introduction A compiler needs to collect and use information about the names appearing in the source program. This information is entered into a data structure called a symbol table. The

### Questions 1 through 25 are worth 2 points each. Choose one best answer for each.

Questions 1 through 25 are worth 2 points each. Choose one best answer for each. 1. For the singly linked list implementation of the queue, where are the enqueues and dequeues performed? c a. Enqueue in

### Biostatistics 615/815

Merge Sort Biostatistics 615/815 Lecture 8 Notes on Problem Set 2 Union Find algorithms Dynamic Programming Results were very ypositive! You should be gradually becoming g y g comfortable compiling, debugging

### Symbol Tables. IE 496 Lecture 13

Symbol Tables IE 496 Lecture 13 Reading for This Lecture Horowitz and Sahni, Chapter 2 Symbol Tables and Dictionaries A symbol table is a data structure for storing a list of items, each with a key and

### COS 226 Algorithms and Data Structures Fall Final

COS 22 Algorithms and Data Structures Fall 200 Final This test has 4 questions worth a total of 00 points. You have 80 minutes. The exam is closed book, except that you are allowed to use a one page cheatsheet

### Data Structure. Lecture 3

Data Structure Lecture 3 Data Structure Formally define Data structure as: DS describes not only set of objects but the ways they are related, the set of operations which may be applied to the elements

### Binary Search Trees. basic implementations randomized BSTs deletion in BSTs

Binary Search Trees basic implementations randomized BSTs deletion in BSTs eferences: Algorithms in Java, Chapter 12 Intro to Programming, Section 4.4 http://www.cs.princeton.edu/introalgsds/43bst 1 Elementary

### School of Informatics, University of Edinburgh Computer Science 1 Ah [03 04]

School of Informatics, University of Edinburgh Computer Science 1 Ah [03 04] CS1Ah Lecture Note 25 Binary Search Trees This lecture studies binary trees in more detail, in particular their application

### 1.3 Conditionals and Loops

A Foundation for Programming 1.3 Conditionals and Loops any program you might want to write objects functions and modules graphics, sound, and image I/O arrays conditionals and loops Math primitive data

### Hashing. CS 127 Fall Data structures, so far. New data structure: Hash table. Definitions

CS 127 Fall 2003 Hashing November 12 and 17, 2003 Data structures, so far Linked lists: refer to element starting from the head of the list Arrays: refer to element by its numerical position in the memory

### CS/COE 1501 http://cs.pitt.edu/~bill/1501/

CS/COE 1501 http://cs.pitt.edu/~bill/1501/ Lecture 01 Course Introduction Meta-notes These notes are intended for use by students in CS1501 at the University of Pittsburgh. They are provided free of charge

### Announcements. CSE332: Data Abstractions. Lecture 9: B Trees. Today. Our goal. M-ary Search Tree. M-ary Search Tree. Ruth Anderson Winter 2011

Announcements CSE2: Data Abstractions Project 2 posted! Partner selection due by 11pm Tues 1/25 at the latest. Homework due Friday Jan 28 st at the BEGINNING of lecture Lecture 9: B Trees Ruth Anderson

### 4.3 Stacks and Queues. Pushdown Stacks. Data Structures and Data Types. Collections

Data Structures and Data Types 4.3 Stacks and Queues Data types Set values. Set operations on those values. Some are built in to Java: int, double, char,... Most are not: Complex, Picture, Charge, Stack,

### Algorithm Efficiency and Sorting

Algorithm Efficiency and Sorting How to Compare Different Problems and Solutions Two different problems Which is harder/more complex? Two different solutions to the same problem Which is better? Questions:

### Efficiency of algorithms. Algorithms. Efficiency of algorithms. Binary search and linear search. Best, worst and average case.

Algorithms Efficiency of algorithms Computational resources: time and space Best, worst and average case performance How to compare algorithms: machine-independent measure of efficiency Growth rate Complexity

### 7.1 Introduction. CSci 335 Software Design and Analysis III Chapter 7 Sorting. Prof. Stewart Weiss

Chapter 7 Sorting 7.1 Introduction Insertion sort is the sorting algorithm that splits an array into a sorted and an unsorted region, and repeatedly picks the lowest index element of the unsorted region

### CSC 143. Two important problems. Searching and Sorting. Review: Linear Search. Example. Review: Binary Search. How Efficient Is Linear Search?

CSC 43 Searching and Sorting [Chapter 9, pp. 402-432] Two important problems Search: finding something in a set of data Sorting: putting a set of data in order Both very common, very useful operations

### (Refer Slide Time: 01:35)

Artificial Intelligence Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture - 6 Informed Search -2 Today we start our 6 th lecture. This

### 06 B: Hashing and Priority Queues

06 B: and CS1102S: Data Structures and Algorithms Martin Henz February 26, 2010 Generated on Friday 26 th February, 2010, 09:23 CS1102S: Data Structures and Algorithms 06 B: and 1 1 2 3 CS1102S: Data Structures

### Practice Problems (Midterm)

ECE-250 Algorithms and Data Structures (Winter 2012) Practice Problems (Midterm) Disclaimer: Please do keep in mind that this problem set does not reflect the exact topics or the fractions of each of the

### State Spaces Graph Search Searching. Graph Search. CPSC 322 Search 2. Textbook 3.4. Graph Search CPSC 322 Search 2, Slide 1

Graph Search CPSC 322 Search 2 Textbook 3.4 Graph Search CPSC 322 Search 2, Slide 1 Lecture Overview 1 State Spaces 2 Graph Search 3 Searching Graph Search CPSC 322 Search 2, Slide 2 State Spaces Idea:

### Computer science, analysis of algorithms

Supervisor: Working title: Keywords: Dr. W. Böhm The Myriad Ways of Sorting Computer science, analysis of algorithms Description. In this thesis you should study a topic that arises very frequently in

### Sources: On the Web: Slides will be available on:

C programming Introduction The basics of algorithms Structure of a C code, compilation step Constant, variable type, variable scope Expression and operators: assignment, arithmetic operators, comparison,

### Definition. (American Heritage Dictionary, 2000)

Recursion 1 recursion Definition (American Heritage Dictionary, 2000) NOUN: Mathematics 1. An expression, such as a polynomial, each term of which is determined by application of a formula to preceding

### Binary Trees and Huffman Encoding Binary Search Trees

Binary Trees and Huffman Encoding Binary Search Trees Computer Science E119 Harvard Extension School Fall 2012 David G. Sullivan, Ph.D. Motivation: Maintaining a Sorted Collection of Data A data dictionary

### Ordered Lists and Binary Trees

Data Structures and Algorithms Ordered Lists and Binary Trees Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science University of San Francisco p.1/62 6-0:

### Lecture Notes on Spanning Trees

Lecture Notes on Spanning Trees 15-122: Principles of Imperative Computation Frank Pfenning Lecture 26 April 26, 2011 1 Introduction In this lecture we introduce graphs. Graphs provide a uniform model

### CPSC 221 Basic Algorithms and Data Structures

CPSC 221 Basic Algorithms and Data Structures Sorting Textbook References: Koffman: 10.1-10.4, 10.7-10.10 EPP 3 rd edition: 9.5 EPP 4 th edition: 11.5 Hassan Khosravi January April 2015 CPSC 221 Sorting

### Quicksort algorithm. Quicksort. Divide and Conquer Algorithm. Partition. Partition algorithm. Recursion. In this lecture:

Quicksort In this lecture: quicksort algorithm worst, best and average performance Quicksort algorithm Designed by C. A. R. Hoare in 1962. Recursive algorithm: choose an element in the array ( pivot element

### Data Structures and Algorithms in C++

Data Structures and Algorithms in C++ Second Edition Michael T. Goodrich Department of Computer Science University of California, Irvine Roberto Tamassia Department of Computer Science Brown University

### 6.092: Introduction to Java. Lecture 1: Types, Variables, Methods

6.092: Introduction to Java Lecture 1: Types, Variables, Methods Outline Intro to Java Types and variables Operators Methods The Computer Memory Central Processing Unit (CPU) Input/Output (IO) Devices

### Binary Search Trees. A Generic Tree. Binary Trees. Nodes in a binary search tree ( B-S-T) are of the form. P parent. Key. Satellite data L R

Binary Search Trees A Generic Tree Nodes in a binary search tree ( B-S-T) are of the form P parent Key A Satellite data L R B C D E F G H I J The B-S-T has a root node which is the only node whose parent

### Heidi C. Ellis and Gerard C. Weatherby. Object Structures. Heidi C. Ellis and Gerard C. Weatherby

Sorting Definition Built in Java Sorts * Bubble * Combsort * Selection * Insertion * Quicksort * Radix/Bucket Sort Sorting.1 Sorting: definition Sorting: process by which a collection of items is placed

### Lecture 2 February 12, 2003

6.897: Advanced Data Structures Spring 003 Prof. Erik Demaine Lecture February, 003 Scribe: Jeff Lindy Overview In the last lecture we considered the successor problem for a bounded universe of size u.

### ! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions

Chapter 13: Query Processing Chapter 13: Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions Basic Steps in Query

### CSC 344 Algorithms and Complexity. What is the Brute Force Approach?

CSC 344 Algorithms and Complexity Lecture #3 Internal Sorting What is the Brute Force Approach? A straightforward approach, usually based directly on the problem s statement and definitions of the concepts

### Chapter 6: Programming Languages

Chapter 6: Programming Languages Computer Science: An Overview Eleventh Edition by J. Glenn Brookshear Copyright 2012 Pearson Education, Inc. Chapter 6: Programming Languages 6.1 Historical Perspective

### 12 Abstract Data Types

12 Abstract Data Types 12.1 Source: Foundations of Computer Science Cengage Learning Objectives After studying this chapter, the student should be able to: Define the concept of an abstract data type (ADT).

### A Static Analyzer for Large Safety-Critical Software. Considered Programs and Semantics. Automatic Program Verification by Abstract Interpretation

PLDI 03 A Static Analyzer for Large Safety-Critical Software B. Blanchet, P. Cousot, R. Cousot, J. Feret L. Mauborgne, A. Miné, D. Monniaux,. Rival CNRS École normale supérieure École polytechnique Paris

### Data Structures and Algorithms

Data Structures and Algorithms CS245-2016S-06 Binary Search Trees David Galles Department of Computer Science University of San Francisco 06-0: Ordered List ADT Operations: Insert an element in the list

### You are to simulate the process by making a record of the balls chosen, in the sequence in which they are chosen. Typical output for a run would be:

Lecture 7 Picking Balls From an Urn The problem: An urn has n (n = 10) balls numbered from 0 to 9 A ball is selected at random, its' is number noted, it is set aside, and another ball is selected from

### Binary Search Trees (continued)

Binary Search Trees (continued) Remove: Starting at the root, search for the value to be remove. If we reach an empty tree, the value is not in the tree. Otherwise, at some point the value is the root

### 3.2 Creating Data Types

1 3.2 Creating Data Types Data Types Data type. Set of values and operations on those values. Basic types. Data Type Set of Values Some Operations boolean true, false not, and, or, xor int -2 31 to 2 31-1

### Lecture 2. Uninformed search

Lecture 2 Uninformed search " Reference: Preparing for week 1 Reading: Chapters 1 and 2.1, 2.2, 2.5, 3.1, 3.2, 3.3, 3.4, 3.5 Assignment 1 has now been posted on the course LEARN site Uses MATLAB (a tutorial

### Entrance Examination - Computer Science

Entrance Examination - Computer Science Name and Surname fill in the field Application No. Test Sheet No. 1 Algorithms and data structures 1 Which statement is correct for the data structure known as an

### The while loop is a common repetition structure. ! Check loop-continuation condition. ! Execute a sequence of statements. ! Repeat.

While Loops Lecture 3: Loops The while loop is a common repetition structure.! Check loop-continuation condition.! Execute a sequence of statements.! Repeat. while (boolean expression) statement; while

### In the example above, it is as though the elements are in two-dimensions such a table

2 1 3 2 4 4 5 5 6 6 7 8 9 10 11 12 13 Multiple-Dimensional Arrays Elements of arrays may have more than one index, i.e. In example above, it is as though elements are in two-dimensions such a table with

### Computer Science. 19. Combinational Circuits. Computer Science. Building blocks Boolean algebra Digital circuits Adder circuit. Arithmetic/logic unit

PA R T I I : A L G O R I T H M S, M A C H I N E S, a n d T H E O R Y PA R T I I : A L G O R I T H M S, M A C H I N E S, a n d T H E O R Y Computer Science 9. Combinational Circuits Computer Science 9.

### Searching & Sorting. Searching & Sorting 1

Searching & Sorting Searching & Sorting 1 The Plan Searching Sorting Java Context Searching & Sorting 2 Search (Retrieval) Looking it up One of most fundamental operations Without computer Indexes Tables

### Algorithms. Algorithms GEOMETRIC APPLICATIONS OF BSTS. 1d range search line segment intersection kd trees interval search trees rectangle intersection

Algorithms ROBERT SEDGEWICK KEVIN WAYNE GEOMETRIC APPLICATIONS OF BSTS Algorithms F O U R T H E D I T I O N ROBERT SEDGEWICK KEVIN WAYNE 1d range search line segment intersection kd trees interval search

### Comparison Sort. - In Merge Sort, the merge procedure chooses an item from one of two arrays after comparing the top items from both arrays.

Comparison Sort A Comparison Sort is a sorting algorithm where the final order is determined only by comparisons between the input elements. - In Insertion Sort, the proper position to insert the current

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### Fast Sequential Summation Algorithms Using Augmented Data Structures

Fast Sequential Summation Algorithms Using Augmented Data Structures Vadim Stadnik vadim.stadnik@gmail.com Abstract This paper provides an introduction to the design of augmented data structures that offer

### Cpt S 223. School of EECS, WSU

Sorting Algorithms 1 Sorting methods Comparison based sorting On 2 ) methods E.g., Insertion, bubble Average time On log n) methods E.g., quick sort On logn) methods E.g., Merge sort, heap sort on-comparison

### Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

### Algorithms 2/17/2015. Double Summations. Enough Mathematical Appetizers! Algorithms. Algorithms. Algorithm Examples. Algorithm Examples

Double Summations Table 2 in 4 th Edition: Section 1.7 5 th Edition: Section.2 6 th and 7 th Edition: Section 2.4 contains some very useful formulas for calculating sums. Enough Mathematical Appetizers!

### 1.204 Lecture 10. Greedy algorithms: Job scheduling. Greedy method

1.204 Lecture 10 Greedy algorithms: Knapsack (capital budgeting) Job scheduling Greedy method Local improvement method Does not look at problem globally Takes best immediate step to find a solution Useful

### Faster deterministic integer factorisation

David Harvey (joint work with Edgar Costa, NYU) University of New South Wales 25th October 2011 The obvious mathematical breakthrough would be the development of an easy way to factor large prime numbers

### The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >

### Parallelization: Binary Tree Traversal

By Aaron Weeden and Patrick Royal Shodor Education Foundation, Inc. August 2012 Introduction: According to Moore s law, the number of transistors on a computer chip doubles roughly every two years. First

### An Overview of a Compiler

An Overview of a Compiler Department of Computer Science and Automation Indian Institute of Science Bangalore 560 012 NPTEL Course on Principles of Compiler Design Outline of the Lecture About the course

### Sorting Algorithms. Biostatistics 615/815 Lecture 6

Sorting Algorithms Biostatistics 615/815 Lecture 6 Last Lecture Recursive Functions Natural expression for many algorithms Dynamic Programming Automatic strategy for generating efficient versions of recursive

### Recursion, Search, Selection. Cheng, Wei COMP June 12, Title

Recursion, Search, Selection Cheng, Wei COMP110-001 June 12, 2014 Title Today Introduction to Recursion Introduction to Search & Selection Not a major focus of the final exam But you should be able to

### Guessing Game: NP-Complete?

Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple

### Algorithms. Theresa Migler-VonDollen CMPS 5P

Algorithms Theresa Migler-VonDollen CMPS 5P 1 / 32 Algorithms Write a Python function that accepts a list of numbers and a number, x. If x is in the list, the function returns the position in the list

### 2 Exercises and Solutions

2 Exercises and Solutions Most of the exercises below have solutions but you should try first to solve them. Each subsection with solutions is after the corresponding subsection with exercises. 2.1 Sorting

### Balanced Trees Part One

Balanced Trees Part One Balanced Trees Balanced trees are surprisingly versatile data structures. Many programming languages ship with a balanced tree library. C++: std::map / std::set Java: TreeMap /

### Software Development (cs2500)

Software Development (cs2500) Lecture 53: Tail Recursion and Search M.R.C. van Dongen March 4, 2011 Contents 1 Outline 1 2 Tail Recursion 2 3 Search 4 4 Linear Search 5 5 Binary Search 6 6 Analysis 7 7

### Artificial Intelligence Lecture 2

Artificial Intelligence Lecture 2 Representation From AI admirers to AI programmers. Step 1: Represent the problem so that it is computerfriendly. Step 2: Code the problem in a programming language. Step

### Stacks. Data Structures and Data Types. Collections

Data Structures and Data Types Data types Set values. Set operations on those values. Some are built in to Java: int, double, char,... Most are not: Complex, Picture, Charge, Stack, Queue, Graph,... Data

### COMP 356 Programming Language Structures Notes for Chapter 5 of Concepts of Programming Languages Names, Types and Scopes

Some definitions: COMP 356 Programming Language Structures Notes for Chapter 5 of Concepts of Programming Languages Names, Types and Scopes a name is a string of characters (a word) that represents a program

### Algorithms. Algorithms 2.1 ELEMENTARY SORTS. rules of the game selection sort insertion sort shellsort shuffling ROBERT SEDGEWICK KEVIN WAYNE

Algorithms ROBERT SEDGEWICK KEVIN WAYNE 2.1 ELEMENTARY SORTS Algorithms F O U R T H E D I T I O N ROBERT SEDGEWICK KEVIN WAYNE rules of the game selection sort insertion sort shellsort shuffling http://algs4.cs.princeton.edu

### Efficient representation of integer sets

Efficient representation of integer sets Marco Almeida Rogério Reis Technical Report Series: DCC-2006-06 Version 1.0 Departamento de Ciência de Computadores & Laboratório de Inteligência Artificial e Ciência