CS473 - Algorithms I

Save this PDF as:

Size: px
Start display at page:

Transcription

1 CS473 - Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm View in slide-show mode 1

2 Reminder: Merge Sort Input array A sort this half sort this half Divide Conquer merge two sorted halves Combine 2

3 The Divide-and-Conquer Design Paradigm 1. Divide the problem (instance) into subproblems. 2. Conquer the subproblems by solving them recursively. 3. Combine subproblem solutions. CS473 Lecture 4 3

4 Example: Merge Sort 1. Divide: Trivial. 2. Conquer: Recursively sort 2 subarrays. 3. Combine: Linear- time merge. T(n) = 2 T(n/2) + Θ(n) # subproblems subproblem size work dividing and combining CS473 Lecture 4 4

5 Master Theorem: Reminder T(n) = at(n/b) + f(n) Case 1: T(n) = Case 2: T(n) = Case 3: and a f (n/b) c f (n) for c < 1 T(n) = 5

6 Merge Sort: Solving the Recurrence T(n) = 2 T(n/2) + Θ(n) a = 2, b = 2, f(n) = Θ(n), n log ba = n Case 2: T(n) = holds for k = 0 T(n) = Θ (nlgn) 6

7 Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find CS473 Lecture 4 7

8 Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find CS473 Lecture 4 8

9 Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find CS473 Lecture 4 9

10 Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find CS473 Lecture 4 10

11 Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find CS473 Lecture 4 11

12 Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find CS473 Lecture 4 12

13 Recurrence for Binary Search T(n) = 1 T(n/2) + Ө(1) # subproblems subproblem size work dividing and combining CS473 Lecture 4 13

14 Binary Search: Solving the Recurrence T(n) = T(n/2) + Θ(1) a = 1, b = 2, f(n) = Θ(1), n log ba = n0 = 1 Case 2: T(n) = holds for k = 0 T(n) = Θ (lgn) 14

15 Powering a Number Problem: Compute a n, where n is a natural number Naive-Power (a, n) powerval 1 for i 1 to n powerval powerval. a return powerval What is the complexity? T(n) = Θ (n) 15

16 Powering a Number: Divide & Conquer Basic idea: a n = a n/2. a n/2 even a (n-1)/2. a (n-1)/2. a if n is if n is odd 16

17 Powering a Number: Divide & Conquer POWER (a, n) if n = 0 then return 1 else if n is even then val POWER (a, n/2) return val * val else if n is odd then val POWER (a, (n-1)/2) return val * val * a 17

18 Powering a Number: Solving the Recurrence T(n) = T(n/2) + Θ(1) a = 1, b = 2, f(n) = Θ(1), n log ba = n0 = 1 Case 2: T(n) = holds for k = 0 T(n) = Θ (lgn) 18

19 Matrix Multiplication Input : A = [a ij ], B = [b ij ]. Output: C = [c ij ] = A. B. i, j = 1, 2,..., n. c 11 c c 1n a 11 a a 1n b 11 b b 1n c 21 c c 2n = a 21 a a 2n. b 21 b b 2n c n1 c n2... c nn a n1 a n2... a nn b n1 b n2... b nn c ij = 1 k n a ik.b kj CS473 Lecture 4 19

20 Standard Algorithm for i 1 to n do for j 1 to n do c ij 0 for k 1 to n do c ij c ij + a ik. b kj Running time = Θ(n 3 ) CS473 Lecture 4 20

21 Matrix Multiplication: Divide & Conquer IDEA: Divide the n x n matrix into 2x2 matrix of (n/2)x(n/2) submatrices C A B c 11 =. c 12 a 11 a 12 b 11 b 12 c 21 c 22 a 21 a 22 b 21 b 22 c 11 = a 11 b 11 + a 12 b 21 21

22 Matrix Multiplication: Divide & Conquer IDEA: Divide the n x n matrix into 2x2 matrix of (n/2)x(n/2) submatrices C A B c 11 b 11 =. c 12 a 11 a 12 b 12 c 21 c 22 a 21 a 22 b 21 b 22 c 12 = a 11 b 12 + a 12 b 22 22

23 Matrix Multiplication: Divide & Conquer IDEA: Divide the n x n matrix into 2x2 matrix of (n/2)x(n/2) submatrices C A B c 11 a 11 a 12 =. c 12 b 11 b 12 c 21 c 22 a 21 a 22 b 21 b 22 c 21 = a 21 b 11 + a 22 b 21 23

24 Matrix Multiplication: Divide & Conquer IDEA: Divide the n x n matrix into 2x2 matrix of (n/2)x(n/2) submatrices C A B c 11 c 12 a 11 b 11 =. a 12 b 12 c 21 c 22 a 21 a 22 b 21 b 22 c 22 = a 21 b 12 + a 22 b 22 24

25 Matrix Multiplication: Divide & Conquer C A B c 11 c 12 a 11 a 12 b =. 11 b 12 c 21 c 22 a 21 a 22 b 21 b 22 c 11 = a 11 b 11 + a 12 b 21 c 12 = a 11 b 12 + a 12 b 22 c 21 = a 21 b 11 + a 22 b 21 8 mults of (n/2)x(n/2) submatrices 4 adds of (n/2)x(n/2) submatrices c 22 = a 21 b 12 + a 22 b 22 25

26 Matrix Multiplication: Divide & Conquer MATRIX-MULTIPLY (A, B) // Assuming that both A and B are nxn matrices if n = 1 then return A * B else partition A, B, and C as shown before c 11 = MATRIX-MULTIPLY (a 11, b 11 ) + MATRIX-MULTIPLY (a 12, b 21 ) c 12 = MATRIX-MULTIPLY (a 11, b 12 ) + MATRIX-MULTIPLY (a 12, b 22 ) c 21 = MATRIX-MULTIPLY (a 21, b 11 ) + MATRIX-MULTIPLY (a 22, b 21 ) c 22 = MATRIX-MULTIPLY (a 21, b 12 ) + MATRIX-MULTIPLY (a 22, b 22 ) return C 26

27 Matrix Multiplication: Divide & Conquer Analysis T(n) = 8 T(n/2) + Θ(n 2 ) 8 recursive calls each subproblem has size n/2 submatrix addition 27

28 Matrix Multiplication: Solving the Recurrence T(n) = 8 T(n/2) + Θ(n 2 ) a = 8, b = 2, f(n) = Θ(n 2 ), n log ba = n 3 Case 1: T(n) = T(n) = Θ (n 3 ) No better than the ordinary algorithm! 28

29 Matrix Multiplication: Strassen s Idea C A B c 11 c 12 a 11 a 12 b =. 11 b 12 c 21 c 22 a 21 a 22 b 21 b 22 Compute c 11, c 12, c 21, and c 22 using 7 recursive multiplications 29

30 Matrix Multiplication: Strassen s Idea P 1 = a 11 x (b 12 - b 22 ) P 2 = (a 11 + a 12 ) x b 22 P 3 = (a 21 + a 22 ) x b 11 P 4 = a 22 x (b 21 - b 11 ) P 5 = (a 11 + a 22 ) x (b 11 + b 22 ) P 6 = (a 11 - a 22 ) x (b 21 + b 22 ) P 7 = ( a 11 - a 21 ) x (b 11 + b 12 ) Reminder: Each submatrix is of size (n/2)x(n/2) Each add/sub operation takes Θ(n 2 ) time Compute P 1..P 7 using 7 recursive calls to matrix-multiply How to compute c ij using P 1.. P 7? 30

31 Matrix Multiplication: Strassen s Idea P 1 = a 11 x (b 12 - b 22 ) P 2 = (a 11 + a 12 ) x b 22 P 3 = (a 21 + a 22 ) x b 11 P 4 = a 22 x (b 21 - b 11 ) P 5 = (a 11 + a 22 ) x (b 11 + b 22 ) P 6 = (a 11 - a 22 ) x (b 21 + b 22 ) P 7 = ( a 11 - a 21 ) x (b 11 + b 12 ) c 11 = P 5 + P 4 P 2 + P 6 c 12 = P 1 + P 2 c 21 = P 3 + P 4 c 22 = P 5 + P 1 P 3 P 7 7 recursive multiply calls 18 add/sub operations Does not rely on commutativity of multiplication 31

32 Matrix Multiplication: Strassen s Idea P 1 = a 11 x (b 12 - b 22 ) P 2 = (a 11 + a 12 ) x b 22 P 3 = (a 21 + a 22 ) x b 11 P 4 = a 22 x (b 21 - b 11 ) P 5 = (a 11 + a 22 ) x (b 11 + b 22 ) P 6 = (a 11 - a 22 ) x (b 21 + b 22 ) P 7 = ( a 11 - a 21 ) x (b 11 + b 12 ) e.g. Show that c 12 = P 1 +P 2 c 12 = P 1 + P 2 = a 11 (b 12 b 22 )+(a 11 +a 12 )b 22 = a 11 b 12 -a 11 b 22 +a 11 b 22 +a 12 b 22 = a 11 b 12 +a 12 b 22 32

33 Strassen s Algorithm 1. Divide: Partition A and B into (n/2) x (n/2) submatrices. Form terms to be multiplied using + and. 2. Conquer: Perform 7 multiplications of (n/2) x (n/2) submatrices recursively. 3. Combine: Form C using + and on (n/2) x (n/2) submatrices. Recurrence: T(n) = 7 T(n/2) + Ө(n 2 ) 33

34 Strassen s Algorithm: Solving the Recurrence T(n) = 7 T(n/2) + Θ(n 2 ) a = 7, b = 2, f(n) = Θ(n 2 ), n log ba = n lg7 Case 1: T(n) = T(n) = Θ (n lg7 ) Note: lg

35 Strassen s Algorithm The number 2.81 may not seem much smaller than 3 But, it is significant because the difference is in the exponent. Strassen s algorithm beats the ordinary algorithm on today s machines for n 30 or so. Best to date: Θ(n ) (of theoretical interest only) 35

36 VLSI Layout: Binary Tree Embedding Problem: Embed a complete binary tree with n leaves into a 2D grid with minimum area. Example: 36

37 Binary Tree Embedding Use divide and conquer root LEFT SUBTREE RIGHT SUBTREE 1. Embed the root node 2. Embed the left subtree 3. Embed the right subtree What is the min-area required for n leaves? 37

38 Binary Tree Embedding root H(n/2) EMBED LEFT SUBTREE HERE EMBED RIGHT SUBTREE HERE H(n) = H(n/2) + 1 W(n/2) W(n/2) W(n) = 2W(n/2)

39 Binary Tree Embedding Solve the recurrences: W(n) = 2W(n/2) + 1 H(n) = H(n/2) + 1 W(n) = Ө(n) H(n) = Ө(lgn) Area(n) = Ө(nlgn) 39

40 Binary Tree Embedding Example: W(n) H(n) 40

41 Binary Tree Embedding: H-Tree Use a different divide and conquer method root left subtree 1 subtree 2 subtree 3 right subtree 4 1. Embed root, left, right nodes 2. Embed subtree 1 3. Embed subtree 2 4. Embed subtree 3 5. Embed subtree 4 What is the min-area required for n leaves? 41

42 Binary Tree Embedding: H-Tree H(n/4) H(n/4) SUBTREE 1 SUBTREE 2 SUBTREE 3 SUBTREE 4 W(n) = 2W(n/4) + 1 H(n) = 2H(n/4) + 1 W(n/4) W(n/4) 42

43 Binary Tree Embedding: H-Tree Solve the recurrences: W(n) = 2W(n/4) + 1 H(n) = 2H(n/4) + 1 W(n) = Ө( ) H(n) = Ө( ) Area(n) = Ө(n) 43

44 Binary Tree Embedding: H-Tree Example: W(n) H(n) 44

45 Correctness Proofs Proof by induction commonly used for D&C algorithms Base case: Show that the algorithm is correct when the recursion bottoms out (i.e., for sufficiently small n) Inductive hypothesis: Assume the alg. is correct for any recursive call on any smaller subproblem of size k (k < n) General case: Based on the inductive hypothesis, prove that the alg. is correct for any input of size n 45

46 Example Correctness Proof: Powering a Number POWER (a, n) if n = 0 then return 1 else if n is even then val POWER (a, n/2) return val * val else if n is odd then val POWER (a, (n-1)/2) return val * val * a 46

47 Example Correctness Proof: Powering a Number Base case: POWER (a, 0) is correct, because it returns 1 Ind. hyp: Assume POWER (a, k) is correct for any k < n General case: In POWER (a, n) function: If n is even: val = a n/2 (due to ind. hyp.) it returns val. val = a n If n is odd: val = a (n-1)/2 (due to ind. hyp.) it returns val. val. a = a n The correctness proof is complete 47

48 Maximum Subarray Problem Input: An array of values Output: The contiguous subarray that has the largest sum of elements Input array: the maximum contiguous subarray 48

49 Maximum Subarray Problem: Divide & Conquer Basic idea: Divide the input array into 2 from the middle Pick the best solution among the following: 1. The max subarray of the left half 2. The max subarray of the right half 3. The max subarray crossing the mid-point Crosses the mid-point A Entirely in the left half Entirely in the right half 49

50 Maximum Subarray Problem: Divide & Conquer Divide: Trivial (divide the array from the middle) Conquer: Recursively compute the max subarrays of the left and right halves Combine: Compute the max-subarray crossing the midpoint (can be done in Θ(n) time). Return the max among the following: 1. the max subarray of the left subarray 2. the max subarray of the right subarray 3. the max subarray crossing the mid-point See textbook for the detailed solution. 50

51 Conclusion Divide and conquer is just one of several powerful techniques for algorithm design. Divide-and-conquer algorithms can be analyzed using recurrences and the master method (so practice this math). Can lead to more efficient algorithms CS473 Lecture 4 51

Divide And Conquer Algorithms

CSE341T/CSE549T 09/10/2014 Lecture 5 Divide And Conquer Algorithms Recall in last lecture, we looked at one way of parallelizing matrix multiplication. At the end of the lecture, we saw the reduce SUM

CS473 - Algorithms I

CS473 - Algorithms I Lecture 9 Sorting in Linear Time View in slide-show mode 1 How Fast Can We Sort? The algorithms we have seen so far: Based on comparison of elements We only care about the relative

Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

Binary Search. Search for x in a sorted array A.

Divide and Conquer A general paradigm for algorithm design; inspired by emperors and colonizers. Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3.

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

The PRAM Model and Algorithms. Advanced Topics Spring 2008 Prof. Robert van Engelen

The PRAM Model and Algorithms Advanced Topics Spring 2008 Prof. Robert van Engelen Overview The PRAM model of parallel computation Simulations between PRAM models Work-time presentation framework of parallel

Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

Matrix-Chain Multiplication

Matrix-Chain Multiplication Let A be an n by m matrix, let B be an m by p matrix, then C = AB is an n by p matrix. C = AB can be computed in O(nmp) time, using traditional matrix multiplication. Suppose

Full and Complete Binary Trees

Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full

Format of Divide-and-Conquer algorithms:

CS 360: Data Structures and Algorithms Divide-and-Conquer (part 1) Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

CSC148 Lecture 8. Algorithm Analysis Binary Search Sorting

CSC148 Lecture 8 Algorithm Analysis Binary Search Sorting Algorithm Analysis Recall definition of Big Oh: We say a function f(n) is O(g(n)) if there exists positive constants c,b such that f(n)

FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT MINING SYSTEM

International Journal of Innovative Computing, Information and Control ICIC International c 0 ISSN 34-48 Volume 8, Number 8, August 0 pp. 4 FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT

Divide and Conquer. Textbook Reading Chapters 4, 7 & 33.4

Divide d Conquer Textook Reading Chapters 4, 7 & 33.4 Overview Design principle Divide d conquer Proof technique Induction, induction, induction Analysis technique Recurrence relations Prolems Sorting

Cofactor Expansion: Cramer s Rule

Cofactor Expansion: Cramer s Rule MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Today we will focus on developing: an efficient method for calculating

Mathematical Induction. Lecture 10-11

Mathematical Induction Lecture 10-11 Menu Mathematical Induction Strong Induction Recursive Definitions Structural Induction Climbing an Infinite Ladder Suppose we have an infinite ladder: 1. We can reach

The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >

Parallel Random Access Machine (PRAM) PRAM Algorithms. Shared Memory Access Conflicts. A Basic PRAM Algorithm. Time Optimality.

Parallel Random Access Machine (PRAM) PRAM Algorithms Arvind Krishnamurthy Fall 2 Collection of numbered processors Accessing shared memory cells Each processor could have local memory (registers) Each

OPTIMUM PARTITION PARAMETER OF DIVIDE-AND-CONQUER ALGORITHM FOR SOLVING CLOSEST-PAIR PROBLEM

OPTIMUM PARTITION PARAMETER OF DIVIDE-AND-CONQUER ALGORITHM FOR SOLVING CLOSEST-PAIR PROBLEM Mohammad Zaidul Karim 1 and Nargis Akter 2 1 Department of Computer Science and Engineering, Daffodil International

Mergesort and Quicksort

Mergesort Mergesort and Quicksort Basic plan: Divide array into two halves. Recursively sort each half. Merge two halves to make sorted whole. mergesort mergesort analysis quicksort quicksort analysis

Closest Pair Problem

Closest Pair Problem Given n points in d-dimensions, find two whose mutual distance is smallest. Fundamental problem in many applications as well as a key step in many algorithms. p q A naive algorithm

Analysis of Algorithms I: Optimal Binary Search Trees

Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Quiz 1 Quiz 1 Do not open this quiz booklet until you are directed

2. (a) Explain the strassen s matrix multiplication. (b) Write deletion algorithm, of Binary search tree. [8+8]

Code No: R05220502 Set No. 1 1. (a) Describe the performance analysis in detail. (b) Show that f 1 (n)+f 2 (n) = 0(max(g 1 (n), g 2 (n)) where f 1 (n) = 0(g 1 (n)) and f 2 (n) = 0(g 2 (n)). [8+8] 2. (a)

8.1 Makespan Scheduling

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programing: Min-Makespan and Bin Packing Date: 2/19/15 Scribe: Gabriel Kaptchuk 8.1 Makespan Scheduling Consider an instance

Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort).

Chapter 7: Quicksort Quicksort is a divide-and-conquer sorting algorithm in which division is dynamically carried out (as opposed to static division in Mergesort). The three steps of Quicksort are as follows:

Week 7: Divide and Conquer

Agenda: Divide and Conquer technique Multiplication of large integers Exponentiation Matrix multiplication 1 2- Divide and Conquer : To solve a problem we can break it into smaller subproblems, solve each

Sample Induction Proofs

Math 3 Worksheet: Induction Proofs III, Sample Proofs A.J. Hildebrand Sample Induction Proofs Below are model solutions to some of the practice problems on the induction worksheets. The solutions given

Quiz 1 Solutions. (a) T F The height of any binary search tree with n nodes is O(log n). Explain:

Introduction to Algorithms March 9, 2011 Massachusetts Institute of Technology 6.006 Spring 2011 Professors Erik Demaine, Piotr Indyk, and Manolis Kellis Quiz 1 Solutions Problem 1. Quiz 1 Solutions True

CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

CS 102: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4)

CS 10: SOLUTIONS TO DIVIDE AND CONQUER ALGORITHMS (ASSGN 4) Problem 1. a. Consider the modified binary search algorithm so that it splits the input not into two sets of almost-equal sizes, but into three

1/1 7/4 2/2 12/7 10/30 12/25

Binary Heaps A binary heap is dened to be a binary tree with a key in each node such that: 1. All leaves are on, at most, two adjacent levels. 2. All leaves on the lowest level occur to the left, and all

Divide-and-Conquer Algorithms Part Four

Divide-and-Conquer Algorithms Part Four Announcements Problem Set 2 due right now. Can submit by Monday at 2:15PM using one late period. Problem Set 3 out, due July 22. Play around with divide-and-conquer

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

Zabin Visram Room CS115 CS126 Searching. Binary Search

Zabin Visram Room CS115 CS126 Searching Binary Search Binary Search Sequential search is not efficient for large lists as it searches half the list, on average Another search algorithm Binary search Very

Divide-and-conquer algorithms

Chapter 2 Divide-and-conquer algorithms The divide-and-conquer strategy solves a problem by: 1 Breaking it into subproblems that are themselves smaller instances of the same type of problem 2 Recursively

Divide-and-Conquer. Three main steps : 1. divide; 2. conquer; 3. merge.

Divide-and-Conquer Three main steps : 1. divide; 2. conquer; 3. merge. 1 Let I denote the (sub)problem instance and S be its solution. The divide-and-conquer strategy can be described as follows. Procedure

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1

Recursion Slides by Christopher M Bourke Instructor: Berthe Y Choueiry Fall 007 Computer Science & Engineering 35 Introduction to Discrete Mathematics Sections 71-7 of Rosen cse35@cseunledu Recursive Algorithms

Unit 4: Dynamic Programming

Unit 4: Dynamic Programming Course contents: Assembly-line scheduling Matrix-chain multiplication Longest common subsequence Optimal binary search trees Applications: Rod cutting, optimal polygon triangulation,

Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

Analysis of Algorithms I: Binary Search Trees

Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary

Math 313 Lecture #10 2.2: The Inverse of a Matrix

Math 1 Lecture #10 2.2: The Inverse of a Matrix Matrix algebra provides tools for creating many useful formulas just like real number algebra does. For example, a real number a is invertible if there is

15-750 Graduate Algorithms Spring 2010 1 Cheap Cut (25 pts.) Suppose we are given a directed network G = (V, E) with a root node r and a set of terminal nodes T V. We would like to remove the fewest number

Scheduling a sequence of tasks with general completion costs

Scheduling a sequence of tasks with general completion costs Francis Sourd CNRS-LIP6 4, place Jussieu 75252 Paris Cedex 05, France Francis.Sourd@lip6.fr Abstract Scheduling a sequence of tasks in the acceptation

Binary Heap Algorithms

CS Data Structures and Algorithms Lecture Slides Wednesday, April 5, 2009 Glenn G. Chappell Department of Computer Science University of Alaska Fairbanks CHAPPELLG@member.ams.org 2005 2009 Glenn G. Chappell

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called heap-order property

CmSc 250 Intro to Algorithms Chapter 6. Transform and Conquer Binary Heaps 1. Definition A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called

From Last Time: Remove (Delete) Operation

CSE 32 Lecture : More on Search Trees Today s Topics: Lazy Operations Run Time Analysis of Binary Search Tree Operations Balanced Search Trees AVL Trees and Rotations Covered in Chapter of the text From

Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

Dynamic Programming. Lecture 11. 11.1 Overview. 11.2 Introduction

Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach

csci 210: Data Structures Recursion

csci 210: Data Structures Recursion Summary Topics recursion overview simple examples Sierpinski gasket Hanoi towers Blob check READING: GT textbook chapter 3.5 Recursion In general, a method of defining

5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

Lecture 12: Chain Matrix Multiplication

Lecture 12: Chain Matrix Multiplication CLRS Section 15.2 Outline of this Lecture Recalling matrix multiplication. The chain matrix multiplication problem. A dynamic programming algorithm for chain matrix

Converting a Number from Decimal to Binary

Converting a Number from Decimal to Binary Convert nonnegative integer in decimal format (base 10) into equivalent binary number (base 2) Rightmost bit of x Remainder of x after division by two Recursive

Notes: Chapter 2 Section 2.2: Proof by Induction

Notes: Chapter 2 Section 2.2: Proof by Induction Basic Induction. To prove: n, a W, n a, S n. (1) Prove the base case - S a. (2) Let k a and prove that S k S k+1 Example 1. n N, n i = n(n+1) 2. Example

Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

Research Tools & Techniques

Research Tools & Techniques for Computer Engineering Ron Sass http://www.rcs.uncc.edu/ rsass University of North Carolina at Charlotte Fall 2009 1/ 106 Overview of Research Tools & Techniques Course What

Partitioning and Divide and Conquer Strategies

and Divide and Conquer Strategies Lecture 4 and Strategies Strategies Data partitioning aka domain decomposition Functional decomposition Lecture 4 and Strategies Quiz 4.1 For nuclear reactor simulation,

Solutions to Homework 6

Solutions to Homework 6 Debasish Das EECS Department, Northwestern University ddas@northwestern.edu 1 Problem 5.24 We want to find light spanning trees with certain special properties. Given is one example

Matrix Algebra. Some Basic Matrix Laws. Before reading the text or the following notes glance at the following list of basic matrix algebra laws.

Matrix Algebra A. Doerr Before reading the text or the following notes glance at the following list of basic matrix algebra laws. Some Basic Matrix Laws Assume the orders of the matrices are such that

Binary Search Trees. Data in each node. Larger than the data in its left child Smaller than the data in its right child

Binary Search Trees Data in each node Larger than the data in its left child Smaller than the data in its right child FIGURE 11-6 Arbitrary binary tree FIGURE 11-7 Binary search tree Data Structures Using

Randomized algorithms

Randomized algorithms March 10, 2005 1 What are randomized algorithms? Algorithms which use random numbers to make decisions during the executions of the algorithm. Why would we want to do this?? Deterministic

Data Structures and Algorithms Written Examination

Data Structures and Algorithms Written Examination 22 February 2013 FIRST NAME STUDENT NUMBER LAST NAME SIGNATURE Instructions for students: Write First Name, Last Name, Student Number and Signature where

B such that AB = I and BA = I. (We say B is an inverse of A.) Definition A square matrix A is invertible (or nonsingular) if matrix

Matrix inverses Recall... Definition A square matrix A is invertible (or nonsingular) if matrix B such that AB = and BA =. (We say B is an inverse of A.) Remark Not all square matrices are invertible.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. Inverse matrix Definition. Let A be an n n matrix. The inverse of A is an n n matrix, denoted

1 Determinants. Definition 1

Determinants The determinant of a square matrix is a value in R assigned to the matrix, it characterizes matrices which are invertible (det 0) and is related to the volume of a parallelpiped described

Binary Search Trees CMPSC 122

Binary Search Trees CMPSC 122 Note: This notes packet has significant overlap with the first set of trees notes I do in CMPSC 360, but goes into much greater depth on turning BSTs into pseudocode than

Binary Multiplication

Binary Multiplication Q: How do we multiply two numbers? eg. 12, 345 6, 789 111105 987600 8641500 + 74070000 83, 810, 205 10111 10101 10111 00000 1011100 0000000 + 101110000 111100011 Pad, multiply and

Algebraic and Combinatorial Circuits

C H A P T E R Algebraic and Combinatorial Circuits Algebraic circuits combine operations drawn from an algebraic system. In this chapter we develop algebraic and combinatorial circuits for a variety of

Unit 18 Determinants

Unit 18 Determinants Every square matrix has a number associated with it, called its determinant. In this section, we determine how to calculate this number, and also look at some of the properties of

, each of which contains a unique key value, say k i , R 2. such that k i equals K (or to determine that no such record exists in the collection).

The Search Problem 1 Suppose we have a collection of records, say R 1, R 2,, R N, each of which contains a unique key value, say k i. Given a particular key value, K, the search problem is to locate the

Algorithms Chapter 12 Binary Search Trees

Algorithms Chapter 1 Binary Search Trees Outline Assistant Professor: Ching Chi Lin 林 清 池 助 理 教 授 chingchi.lin@gmail.com Department of Computer Science and Engineering National Taiwan Ocean University

Warshall s Algorithm: Transitive Closure

CS 0 Theory of Algorithms / CS 68 Algorithms in Bioinformaticsi Dynamic Programming Part II. Warshall s Algorithm: Transitive Closure Computes the transitive closure of a relation (Alternatively: all paths

Binary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * *

Binary Heaps A binary heap is another data structure. It implements a priority queue. Priority Queue has the following operations: isempty add (with priority) remove (highest priority) peek (at highest

UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

Lecture 18: Applications of Dynamic Programming Steven Skiena. Department of Computer Science State University of New York Stony Brook, NY 11794 4400

Lecture 18: Applications of Dynamic Programming Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Problem of the Day

Polynomials and the Fast Fourier Transform (FFT) Battle Plan

Polynomials and the Fast Fourier Transform (FFT) Algorithm Design and Analysis (Wee 7) 1 Polynomials Battle Plan Algorithms to add, multiply and evaluate polynomials Coefficient and point-value representation

Analysis of Algorithms, I

Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications

CS335 Program2: ConvexHulls A Divide-and-Conquer Algorithm 15 points Due: Tues, October 6

CS335 Program2: ConvexHulls A Divide-and-Conquer Algorithm 15 points Due: Tues, October 6 Convex Hulls There is a complex mathematical definition of a convex hull, but the informal definition is sufficient

Computing exponents modulo a number: Repeated squaring

Computing exponents modulo a number: Repeated squaring How do you compute (1415) 13 mod 2537 = 2182 using just a calculator? Or how do you check that 2 340 mod 341 = 1? You can do this using the method

Fast Fourier Transform: Theory and Algorithms

Fast Fourier Transform: Theory and Algorithms Lecture Vladimir Stojanović 6.973 Communication System Design Spring 006 Massachusetts Institute of Technology Discrete Fourier Transform A review Definition

A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem

A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem John Karlof and Peter Hocking Mathematics and Statistics Department University of North Carolina Wilmington Wilmington,

10. Graph Matrices Incidence Matrix

10 Graph Matrices Since a graph is completely determined by specifying either its adjacency structure or its incidence structure, these specifications provide far more efficient ways of representing a

Data Structures. Algorithm Performance and Big O Analysis

Data Structures Algorithm Performance and Big O Analysis What s an Algorithm? a clearly specified set of instructions to be followed to solve a problem. In essence: A computer program. In detail: Defined

Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

Mathematical Induction

Mathematical Induction Victor Adamchik Fall of 2005 Lecture 2 (out of three) Plan 1. Strong Induction 2. Faulty Inductions 3. Induction and the Least Element Principal Strong Induction Fibonacci Numbers

UNIT 5C Merge Sort. Course Announcements

UNIT 5C Merge Sort 1 Course Announcements Exam rooms for Lecture 1, 2:30 3:20 Sections A, B, C, D at Rashid Sections E, F, G at Baker A51 (Giant Eagle Auditorium) Exam rooms for Lecture 2, 3:30 4:20 Sections

Algorithm Design and Analysis

Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;

Appendix: Solving Recurrences [Fa 10] Wil Wheaton: Embrace the dark side! Sheldon: That s not even from your franchise!

Change is certain. Peace is followed by disturbances; departure of evil men by their return. Such recurrences should not constitute occasions for sadness but realities for awareness, so that one may be

Lecture 13: The Knapsack Problem

Lecture 13: The Knapsack Problem Outline of this Lecture Introduction of the 0-1 Knapsack Problem. A dynamic programming solution to this problem. 1 0-1 Knapsack Problem Informal Description: We have computed

The Running Time of Programs

CHAPTER 3 The Running Time of Programs In Chapter 2, we saw two radically different algorithms for sorting: selection sort and merge sort. There are, in fact, scores of algorithms for sorting. This situation

Binary search algorithm

Binary search algorithm Definition Search a sorted array by repeatedly dividing the search interval in half. Begin with an interval covering the whole array. If the value of the search key is less than