# Dynamic Programming. Lecture Overview Introduction

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach would take exponential time. In this lecture, we discuss this technique, and present a few key examples. Topics in this lecture include: The basic idea of Dynamic Programming. Example: Longest Common Subsequence. Example: Knapsack. Example: Matrix-chain multiplication Introduction Dynamic Programming is a powerful technique that can be used to solve many problems in time O(n 2 ) or O(n 3 ) for which a naive approach would take exponential time. (Usually to get running time below that if it is possible one would need to add other ideas as well.) Dynamic Programming is a general approach to solving problems, much like divide-and-conquer is a general method, except that unlike divide-and-conquer, the subproblems will typically overlap. This lecture we will present two ways of thinking about Dynamic Programming as well as a few examples. There are several ways of thinking about the basic idea. Basic Idea (version 1): What we want to do is take our problem and somehow break it down into a reasonable number of subproblems (where reasonable might be something like n 2 ) in such a way that we can use optimal solutions to the smaller subproblems to give us optimal solutions to the larger ones. Unlike divide-and-conquer (as in mergesort or quicksort) it is OK if our subproblems overlap, so long as there are not too many of them. 59

2 11.3. EXAMPLE 1: LONGEST COMMON SUBSEQUENCE Example 1: Longest Common Subsequence Definition 11.1 The Longest Common Subsequence (LCS) problem is as follows. We are given two strings: string S of length n, and string T of length m. Our goal is to produce their longest common subsequence: the longest sequence of characters that appear left-to-right (but not necessarily in a contiguous block) in both strings. For example, consider: S = ABAZDC T = BACBAD In this case, the LCS has length 4 and is the string ABAD. Another way to look at it is we are finding a 1-1 matching between some of the letters in S and some of the letters in T such that none of the edges in the matching cross each other. For instance, this type of problem comes up all the time in genomics: given two DNA fragments, the LCS gives information about what they have in common and the best way to line them up. Let s now solve the LCS problem using Dynamic Programming. As subproblems we will look at the LCS of a prefix of S and a prefix of T, running over all pairs of prefixes. For simplicity, let s worry first about finding the length of the LCS and then we can modify the algorithm to produce the actual sequence itself. So, here is the question: say LCS[i,j] is the length of the LCS of S[1..i] with T[1..j]. How can we solve for LCS[i,j] in terms of the LCS s of the smaller problems? Case 1: what if S[i] T[j]? Then, the desired subsequence has to ignore one of S[i] or T[j] so we have: LCS[i,j] = max(lcs[i 1,j],LCS[i,j 1]). Case 2: what if S[i] = T[j]? Then the LCS of S[1..i] and T[1..j] might as well match them up. For instance, if I gave you a common subsequence that matched S[i] to an earlier location in T, for instance, you could always match it to T[j] instead. So, in this case we have: LCS[i,j] = 1 +LCS[i 1,j 1]. So, we can just do two loops (over values of i and j), filling in the LCS using these rules. Here s what it looks like pictorially for the example above, with S along the leftmost column and T along the top row. B A C B A D A B A Z D C We just fill out this matrix row by row, doing constant amount of work per entry, so this takes O(mn) time overall. The final answer (the length of the LCS of S and T) is in the lower-right corner.

3 11.4. MORE ON THE BASIC IDEA, AND EXAMPLE 1 REVISITED 61 How can we now find the sequence? To find the sequence, we just walk backwards through matrix starting the lower-right corner. If either the cell directly above or directly to the right contains a value equal to the value in the current cell, then move to that cell (if both to, then chose either one). If both such cells have values strictly less than the value in the current cell, then move diagonally up-left (this corresponts to applying Case 2), and output the associated character. This will output the characters in the LCS in reverse order. For instance, running on the matrix above, this outputs DABA More on the basic idea, and Example 1 revisited We have been looking at what is called bottom-up Dynamic Programming. Here is another way of thinking about Dynamic Programming, that also leads to basically the same algorithm, but viewed from the other direction. Sometimes this is called top-down Dynamic Programming. Basic Idea (version 2): Suppose you have a recursive algorithm for some problem that gives you a really bad recurrence like T(n) = 2T(n 1)+n. However, suppose that many of the subproblems you reach as you go down the recursion tree are the same. Then you can hope to get a big savings if you store your computations so that you only compute each different subproblem once. You can store these solutions in an array or hash table. This view of Dynamic Programming is often called memoizing. For example, for the LCS problem, using our analysis we had at the beginning we might have produced the following exponential-time recursive program (arrays start at 1): LCS(S,n,T,m) if (n==0 m==0) return 0; if (S[n] == T[m]) result = 1 + LCS(S,n-1,T,m-1); // no harm in matching up else result = max( LCS(S,n-1,T,m), LCS(S,n,T,m-1) ); This algorithm runs in exponential time. In fact, if S and T use completely disjoint sets of characters (so that we never have S[n]==T[m]) then the number of times that LCS(S,1,T,1) is recursively called equals ( n+m 2) m 1. 1 In the memoized version, we store results in a matrix so that any given set of arguments to LCS only produces new work (new recursive calls) once. The memoized version begins by initializing arr[i][j] to unknown for all i,j, and then proceeds as follows: LCS(S,n,T,m) if (n==0 m==0) return 0; if (arr[n][m]!= unknown) return arr[n][m]; // <- added this line (*) if (S[n] == T[m]) result = 1 + LCS(S,n-1,T,m-1); else result = max( LCS(S,n-1,T,m), LCS(S,n,T,m-1) ); arr[n][m] = result; // <- and this line (**) grid. 1 This is the number of different monotone walks between the upper-left and lower-right corners of an n by m

4 11.5. EXAMPLE #2: THE KNAPSACK PROBLEM 62 All we have done is saved our work in line (**) and made sure that we only embark on new recursive calls if we haven t already computed the answer in line (*). In this memoized version, our running time is now just O(mn). One easy way to see this is as follows. First, notice that we reach line (**) at most mn times (at most once for any given value of the parameters). This means we make at most 2mn recursive calls total (at most two calls for each time we reach that line). Any given call of LCS involves only O(1) work (performing some equality checks and taking a max or adding 1), so overall the total running time is O(mn). Comparing bottom-up and top-down dynamic programming, both do almost the same work. The top-down (memoized) version pays a penalty in recursion overhead, but can potentially be faster than the bottom-up version in situations where some of the subproblems never get examined at all. These differences, however, are minor: you should use whichever version is easiest and most intuitive for you for the given problem at hand. More about LCS: Discussion and Extensions. An equivalent problem to LCS is the minimum edit distance problem, where the legal operations are insert and delete. (E.g., the unix diff command, where S and T are files, and the elements of S and T are lines of text). The minimum edit distance to transform S into T is achieved by doing S LCS(S,T) deletes and T LCS(S,T) inserts. In computational biology applications, often one has a more general notion of sequence alignment. Many of these different problems all allow for basically the same kind of Dynamic Programming solution Example #2: The Knapsack Problem Imagine you have a homework assignment with different parts labeled A through G. Each part has a value (in points) and a size (time in hours to complete). For example, say the values and times for our assignment are: A B C D E F G value time Say you have a total of 15 hours: which parts should you do? If there was partial credit that was proportional to the amount of work done (e.g., one hour spent on problem C earns you 2.5 points) then the best approach is to work on problems in order of points/hour (a greedy strategy). But, what if there is no partial credit? In that case, which parts should you do, and what is the best total value possible? 2 The above is an instance of the knapsack problem, formally defined as follows: 2 Answer: In this case, the optimal strategy is to do parts A, B, F, and G for a total of 34 points. Notice that this doesn t include doing part C which has the most points/hour!

5 11.6. EXAMPLE #3: MATRIX PRODUCT PARENTHESIZATION 63 Definition 11.2 In the knapsack problem we are given a set of n items, where each item i is specified by a size s i and a value v i. We are also given a size bound S (the size of our knapsack). The goal is to find the subset of items of maximum total value such that sum of their sizes is at most S (they all fit into the knapsack). We can solve the knapsack problem in exponential time by trying all possible subsets. With Dynamic Programming, we can reduce this to time O(nS). Let s do this top down by starting with a simple recursive solution and then trying to memoize it. Let s start by just computing the best possible total value, and we afterwards can see how to actually extract the items needed. // Recursive algorithm: either we use the last element or we don t. Value(n,S) // S = space left, n = # items still to choose from if (n == 0) return 0; if (s_n > S) result = Value(n-1,S); // can t use nth item else result = maxv_n + Value(n-1, S-s_n), Value(n-1, S); Right now, this takes exponential time. But, notice that there are only O(nS) different pairs of values the arguments can possibly take on, so this is perfect for memoizing. As with the LCS problem, let us initialize a 2-d array arr[i][j] to unknown for all i,j. Value(n,S) if (n == 0) return 0; if (arr[n][s]!= unknown) return arr[n][s]; // <- added this if (s_n > S) result = Value(n-1,S); else result = maxv_n + Value(n-1, S-s_n), Value(n-1, S); arr[n][s] = result; // <- and this Since any given pair of arguments to Value can pass through the array check only once, and in doing so produces at most two recursive calls, we have at most 2n(S + 1) recursive calls total, and the total time is O(nS). So far we have only discussed computing the value of the optimal solution. How can we get the items? As usual for Dynamic Programming, we can do this by just working backwards: if arr[n][s] = arr[n-1][s] then we didn t use the nth item so we just recursively work backwards from arr[n-1][s]. Otherwise, we did use that item, so we just output the nth item and recursively work backwards from arr[n-1][s-s n]. One can also do bottom-up Dynamic Programming Example #3: Matrix product parenthesization Our final example for Dynamic Programming is the matrix product parenthesization problem.

6 11.6. EXAMPLE #3: MATRIX PRODUCT PARENTHESIZATION 64 Say we want to multiply three matrices X, Y, and Z. We could do it like (XY )Z or like X(Y Z). Which way we do the multiplication doesn t affect the final outcome but it can affect the running time to compute it. For example, say X is 100x20, Y is 20x100, and Z is 100x20. So, the end result will be a 100x20 matrix. If we multiply using the usual algorithm, then to multiply an lxm matrix by an mxn matrix takes time O(lmn). So in this case, which is better, doing (XY )Z or X(Y Z)? Answer: X(Y Z) is better because computing Y Z takes 20x100x20 steps, producing a 20x20 matrix, and then multiplying this by X takes another 20x100x20 steps, for a total of 2x20x100x20. But, doing it the other way takes 100x20x100 steps to compute XY, and then multplying this with Z takes another 100x20x100 steps, so overall this way takes 5 times longer. More generally, what if we want to multiply a series of n matrices? Definition 11.3 The Matrix Product Parenthesization problem is as follows. Suppose we need to multiply a series of matrices: A 1 A 2 A 3... A n. Given the dimensions of these matrices, what is the best way to parenthesize them, assuming for simplicity that standard matrix multiplication is to be used (e.g., not Strassen)? There are an exponential number of different possible parenthesizations, in fact ( 2(n 1) ) n 1 /n, so we don t want to search through all of them. Dynamic Programming gives us a better way. As before, let s first think: how might we do this recursively? One way is that for each possible split for the final multiplication, recursively solve for the optimal parenthesization of the left and right sides, and calculate the total cost (the sum of the costs returned by the two recursive calls plus the lmn cost of the final multiplication, where m depends on the location of that split). Then take the overall best top-level split. For Dynamic Programming, the key question is now: in the above procedure, as you go through the recursion, what do the subproblems look like and how many are there? Answer: each subproblem looks like what is the best way to multiply some sub-interval of the matrices A i... A j? So, there are only O(n 2 ) different subproblems. The second question is now: how long does it take to solve a given subproblem assuming you ve already solved all the smaller subproblems (i.e., how much time is spent inside any given recursive call)? Answer: to figure out how to best multiply A i... A j, we just consider all possible middle points k and select the one that minimizes: optimal cost to multiply A i... A k already computed + optimal cost to multiply A k+1...a j already computed + cost to multiply the results. get this from the dimensions This just takes O(1) work for any given k, and there are at most n different values k to consider, so overall we just spend O(n) time per subproblem. So, if we use Dynamic Programming to save our results in a lookup table, then since there are only O(n 2 ) subproblems we will spend only O(n 3 ) time overall. If you want to do this using bottom-up Dynamic Programming, you would first solve for all subproblems with j i = 1, then solve for all with j i = 2, and so on, storing your results in an n by n matrix. The main difference between this problem and the two previous ones we have seen is that any given subproblem takes time O(n) to solve rather than O(1), which is why we get O(n 3 )

7 11.7. HIGH-LEVEL DISCUSSION OF DYNAMIC PROGRAMMING 65 total running time. It turns out that by being very clever you can actually reduce this to O(1) amortized time per subproblem, producing an O(n 2 )-time algorithm, but we won t get into that here High-level discussion of Dynamic Programming What kinds of problems can be solved using Dynamic Programming? One property these problems have is that if the optimal solution involves solving a subproblem, then it uses the optimal solution to that subproblem. For instance, say we want to find the shortest path from A to B in a graph, and say this shortest path goes through C. Then it must be using the shortest path from C to B. Or, in the knapsack example, if the optimal solution does not use item n, then it is the optimal solution for the problem in which item n does not exist. The other key property is that there should be only a polynomial number of different subproblems. These two properties together allow us to build the optimal solution to the final problem from optimal solutions to subproblems. In the top-down view of dynamic programming, the first property above corresponds to being able to write down a recursive procedure for the problem we want to solve. The second property corresponds to making sure that this recursive procedure makes only a polynomial number of different recursive calls. In particular, one can often notice this second property by examining the arguments to the recursive procedure: e.g., if there are only two integer arguments that range between 1 and n, then there can be at most n 2 different recursive calls. Sometimes you need to do a little work on the problem to get the optimal-subproblem-solution property. For instance, suppose we are trying to find paths between locations in a city, and some intersections have no-left-turn rules (this is particulatly bad in San Francisco). Then, just because the fastest way from A to B goes through intersection C, it doesn t necessarily use the fastest way to C because you might need to be coming into C in the correct direction. In fact, the right way to model that problem as a graph is not to have one node per intersection, but rather to have one node per intersection, direction pair. That way you recover the property you need. 3 For details, see Knuth (insert ref).

### Memoization/Dynamic Programming. The String reconstruction problem. CS125 Lecture 5 Fall 2016

CS125 Lecture 5 Fall 2016 Memoization/Dynamic Programming Today s lecture discusses memoization, which is a method for speeding up algorithms based on recursion, by using additional memory to remember

### Dynamic Programming. Applies when the following Principle of Optimality

Dynamic Programming Applies when the following Principle of Optimality holds: In an optimal sequence of decisions or choices, each subsequence must be optimal. Translation: There s a recursive solution.

### Lecture 12: Chain Matrix Multiplication

Lecture 12: Chain Matrix Multiplication CLRS Section 15.2 Outline of this Lecture Recalling matrix multiplication. The chain matrix multiplication problem. A dynamic programming algorithm for chain matrix

### Lecture 21: Dynamic Programming III

Lecture 21: Dynamic Programming III Lecture Overview Subproblems for strings Parenthesization Edit distance (& longest common subseq.) Knapsack Pseudopolynomial Time Review: * 5 easy steps to dynamic programming

### Divide And Conquer Algorithms

CSE341T/CSE549T 09/10/2014 Lecture 5 Divide And Conquer Algorithms Recall in last lecture, we looked at one way of parallelizing matrix multiplication. At the end of the lecture, we saw the reduce SUM

### Steven Skiena. skiena

Lecture 3: Recurrence Relations (1997) Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Argue the solution to T(n) =

### Matrix Product. Matrix-Multiply (A, B) Cost of multiplying these matrices is p q r.

Dynamic Programming A powerful paradigm for algorithm design. Often leads to elegant and efficient algorithms when greedy or diide-and-conquer don t work. DP also breaks a problem into subproblems, but

### CS420/520 Algorithm Analysis Spring 2010 Lecture 04

CS420/520 Algorithm Analysis Spring 2010 Lecture 04 I. Chapter 4 Recurrences A. Introduction 1. Three steps applied at each level of a recursion a) Divide into a number of subproblems that are smaller

### Solutions to Homework 6

Solutions to Homework 6 Debasish Das EECS Department, Northwestern University ddas@northwestern.edu 1 Problem 5.24 We want to find light spanning trees with certain special properties. Given is one example

### Near Optimal Solutions

Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

### Week 7: Divide and Conquer

Agenda: Divide and Conquer technique Multiplication of large integers Exponentiation Matrix multiplication 1 2- Divide and Conquer : To solve a problem we can break it into smaller subproblems, solve each

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Dynamic programming. Chapter 6. 6.1 Shortest paths in dags, revisited S C A B D E

Chapter 6 Dynamic programming In the preceding chapters we have seen some elegant design principles such as divide-andconquer, graph exploration, and greedy choice that yield definitive algorithms for

### The Greedy Method. Fundamental Techniques. The Greedy Method. Fractional Knapsack Problem

Fundamental Techniques There are some algorithmic tools that are quite specialised. They are good for problems they are intended to solve, but they are not very versatile. There are also more fundamental

### Binary Search. Search for x in a sorted array A.

Divide and Conquer A general paradigm for algorithm design; inspired by emperors and colonizers. Three-step process: 1. Divide the problem into smaller problems. 2. Conquer by solving these problems. 3.

### Balanced search trees

Lecture 8 Balanced search trees 8.1 Overview In this lecture we discuss search trees as a method for storing data in a way that supports fast insert, lookup, and delete operations. (Data structures handling

### A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points

A. V. Gerbessiotis CS 610-102 Spring 2014 PS 3 Mar 24, 2014 No points Problem 1. Suppose that we insert n keys into a hash table of size m using open addressing and uniform hashing. Let p(n, m) be the

### Matrix-Chain Multiplication

Matrix-Chain Multiplication Let A be an n by m matrix, let B be an m by p matrix, then C = AB is an n by p matrix. C = AB can be computed in O(nmp) time, using traditional matrix multiplication. Suppose

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### Divide and Conquer My T. UF

Introduction to Algorithms Divide and Conquer @ UF General Approach Divide the problem into a number of subproblems that are smaller instances of the same problem Conquer the subproblems by solving them

### Data Structures and Algorithms Written Examination

Data Structures and Algorithms Written Examination 22 February 2013 FIRST NAME STUDENT NUMBER LAST NAME SIGNATURE Instructions for students: Write First Name, Last Name, Student Number and Signature where

### Recurrences (CLRS )

Recurrences (CLRS 4.1-4.2) Last time we discussed divide-and-conquer algorithms Divide and Conquer To Solve P: 1. Divide P into smaller problems P 1,P 2,P 3...P k. 2. Conquer by solving the (smaller) subproblems

### Algorithms. Theresa Migler-VonDollen CMPS 5P

Algorithms Theresa Migler-VonDollen CMPS 5P 1 / 32 Algorithms Write a Python function that accepts a list of numbers and a number, x. If x is in the list, the function returns the position in the list

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### CSE8393 Introduction to Bioinformatics Lecture 3: More problems, Global Alignment. DNA sequencing

SE8393 Introduction to Bioinformatics Lecture 3: More problems, Global lignment DN sequencing Recall that in biological experiments only relatively short segments of the DN can be investigated. To investigate

### IE 680 Special Topics in Production Systems: Networks, Routing and Logistics*

IE 680 Special Topics in Production Systems: Networks, Routing and Logistics* Rakesh Nagi Department of Industrial Engineering University at Buffalo (SUNY) *Lecture notes from Network Flows by Ahuja, Magnanti

### Algorithms. Margaret M. Fleck. 18 October 2010

Algorithms Margaret M. Fleck 18 October 2010 These notes cover how to analyze the running time of algorithms (sections 3.1, 3.3, 4.4, and 7.1 of Rosen). 1 Introduction The main reason for studying big-o

### Lecture 18: Applications of Dynamic Programming Steven Skiena. Department of Computer Science State University of New York Stony Brook, NY 11794 4400

Lecture 18: Applications of Dynamic Programming Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Problem of the Day

### 1. What s wrong with the following proofs by induction?

ArsDigita University Month : Discrete Mathematics - Professor Shai Simonson Problem Set 4 Induction and Recurrence Equations Thanks to Jeffrey Radcliffe and Joe Rizzo for many of the solutions. Pasted

### Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally

Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which

### CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.

Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure

### Dynamic Programming Problem Set Partial Solution CMPSC 465

Dynamic Programming Problem Set Partial Solution CMPSC 465 I ve annotated this document with partial solutions to problems written more like a test solution. (I remind you again, though, that a formal

### Pigeonhole Principle Solutions

Pigeonhole Principle Solutions 1. Show that if we take n + 1 numbers from the set {1, 2,..., 2n}, then some pair of numbers will have no factors in common. Solution: Note that consecutive numbers (such

### Problem Set #1 Solutions

CS161: Design and Analysis of Algorithms Summer 2004 Problem Set #1 Solutions General Notes Regrade Policy: If you believe an error has been made in the grading of your problem set, you may resubmit it

### 2. (a) Explain the strassen s matrix multiplication. (b) Write deletion algorithm, of Binary search tree. [8+8]

Code No: R05220502 Set No. 1 1. (a) Describe the performance analysis in detail. (b) Show that f 1 (n)+f 2 (n) = 0(max(g 1 (n), g 2 (n)) where f 1 (n) = 0(g 1 (n)) and f 2 (n) = 0(g 2 (n)). [8+8] 2. (a)

### Warshall s Algorithm: Transitive Closure

CS 0 Theory of Algorithms / CS 68 Algorithms in Bioinformaticsi Dynamic Programming Part II. Warshall s Algorithm: Transitive Closure Computes the transitive closure of a relation (Alternatively: all paths

### Notes on Matrix Multiplication and the Transitive Closure

ICS 6D Due: Wednesday, February 25, 2015 Instructor: Sandy Irani Notes on Matrix Multiplication and the Transitive Closure An n m matrix over a set S is an array of elements from S with n rows and m columns.

### 1. Sorting (assuming sorting into ascending order) a) BUBBLE SORT

DECISION 1 Revision Notes 1. Sorting (assuming sorting into ascending order) a) BUBBLE SORT Make sure you show comparisons clearly and label each pass First Pass 8 4 3 6 1 4 8 3 6 1 4 3 8 6 1 4 3 6 8 1

### Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4

Intro. to the Divide-and-Conquer Strategy via Merge Sort CMPSC 465 CLRS Sections 2.3, Intro. to and various parts of Chapter 4 I. Algorithm Design and Divide-and-Conquer There are various strategies we

### Theorem A graph T is a tree if, and only if, every two distinct vertices of T are joined by a unique path.

Chapter 3 Trees Section 3. Fundamental Properties of Trees Suppose your city is planning to construct a rapid rail system. They want to construct the most economical system possible that will meet the

### Closest Pair Problem

Closest Pair Problem Given n points in d-dimensions, find two whose mutual distance is smallest. Fundamental problem in many applications as well as a key step in many algorithms. p q A naive algorithm

### The Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!

The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >

### Michal Forišek: Early beta verzia skrípt z ADŠ

Časová zložitosť II Michal Forišek: Early beta verzia skrípt z ADŠ In this part of the article we will focus on estimating the time complexity for recursive programs. In essence, this will lead to finding

### 1 Review of Least Squares Solutions to Overdetermined Systems

cs4: introduction to numerical analysis /9/0 Lecture 7: Rectangular Systems and Numerical Integration Instructor: Professor Amos Ron Scribes: Mark Cowlishaw, Nathanael Fillmore Review of Least Squares

### Automata and Computability. Solutions to Exercises

Automata and Computability Solutions to Exercises Fall 25 Alexis Maciel Department of Computer Science Clarkson University Copyright c 25 Alexis Maciel ii Contents Preface vii Introduction 2 Finite Automata

### Quicksort An Example. When lo and hi cross, we are done. The final value of hi is the position in which the partitioning element ends up.

Quicksort An Example We sort the array A = (38 81 22 48 13 69 93 14 45 58 79 72) with quicksort, always choosing the pivot element to be the element in position (left+right)/2. The partitioning during

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### CS2 Algorithms and Data Structures Note 11. Breadth-First Search and Shortest Paths

CS2 Algorithms and Data Structures Note 11 Breadth-First Search and Shortest Paths In this last lecture of the CS2 Algorithms and Data Structures thread we will consider the problem of computing distances

### Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology Professors Erik Demaine and Shafi Goldwasser Quiz 1.

Introduction to Algorithms March 10, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Quiz 1 Quiz 1 Do not open this quiz booklet until you are directed

### 1 Review of Newton Polynomials

cs: introduction to numerical analysis 0/0/0 Lecture 8: Polynomial Interpolation: Using Newton Polynomials and Error Analysis Instructor: Professor Amos Ron Scribes: Giordano Fusco, Mark Cowlishaw, Nathanael

### Spring 2007 Math 510 Hints for practice problems

Spring 2007 Math 510 Hints for practice problems Section 1 Imagine a prison consisting of 4 cells arranged like the squares of an -chessboard There are doors between all adjacent cells A prisoner in one

### Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1

Recursion Slides by Christopher M Bourke Instructor: Berthe Y Choueiry Fall 007 Computer Science & Engineering 35 Introduction to Discrete Mathematics Sections 71-7 of Rosen cse35@cseunledu Recursive Algorithms

### 1 Description of The Simpletron

Simulating The Simpletron Computer 50 points 1 Description of The Simpletron In this assignment you will write a program to simulate a fictional computer that we will call the Simpletron. As its name implies

### Chapter 4. Trees. 4.1 Basics

Chapter 4 Trees 4.1 Basics A tree is a connected graph with no cycles. A forest is a collection of trees. A vertex of degree one, particularly in a tree, is called a leaf. Trees arise in a variety of applications.

### Lecture 2 Mathcad basics and Matrix Operations

Lecture 2 Mathcad basics and Matrix Operations Announcements No class or lab Wednesday, 8/29/01 I will be posting a lab worksheet on the web site on Tuesday for you to work through on your own. Operators

### Format of Divide-and-Conquer algorithms:

CS 360: Data Structures and Algorithms Divide-and-Conquer (part 1) Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively

### COMP 250 Fall Mathematical induction Sept. 26, (n 1) + n = n + (n 1)

COMP 50 Fall 016 9 - Mathematical induction Sept 6, 016 You will see many examples in this course and upcoming courses of algorithms for solving various problems It many cases, it will be obvious that

### LabVIEW Day 3: Arrays and Clusters

LabVIEW Day 3: Arrays and Clusters Vern Lindberg By now you should be getting used to LabVIEW. You should know how to Create a Constant, Control, or Indicator. I will assume you know how to create a new

### Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

### Data Structures in Java. Session 15 Instructor: Bert Huang http://www1.cs.columbia.edu/~bert/courses/3134

Data Structures in Java Session 15 Instructor: Bert Huang http://www1.cs.columbia.edu/~bert/courses/3134 Announcements Homework 4 on website No class on Tuesday Midterm grades almost done Review Indexing

Operators Lecture 2 Mathcad Basics + Addition, - Subtraction, * Multiplication, / Division, ^ Power ( ) Specify evaluation order Order of Operations ( ) ^ highest level, first priority * / next priority

Answers to Homework 1 p. 13 1.2 3 What is the smallest value of n such that an algorithm whose running time is 100n 2 runs faster than an algorithm whose running time is 2 n on the same machine? Find the

### Session 6 Number Theory

Key Terms in This Session Session 6 Number Theory Previously Introduced counting numbers factor factor tree prime number New in This Session composite number greatest common factor least common multiple

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### The Selection Problem

The Selection Problem The selection problem: given an integer k and a list x 1,..., x n of n elements find the k-th smallest element in the list Example: the 3rd smallest element of the following list

### Class Overview. CSE 326: Data Structures. Goals. Goals. Data Structures. Goals. Introduction

Class Overview CSE 326: Data Structures Introduction Introduction to many of the basic data structures used in computer software Understand the data structures Analyze the algorithms that use them Know

### 1.2 Solving a System of Linear Equations

1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables

### Computer Algorithms. Merge Sort. Analysis of Merge Sort. Recurrences. Recurrence Relations. Iteration Method. Lecture 6

Computer Algorithms Lecture 6 Merge Sort Sorting Problem: Sort a sequence of n elements into non-decreasing order. Divide: Divide the n-element sequence to be sorted into two subsequences of n/ elements

### Dynamic programming. Doctoral course Optimization on graphs - Lecture 4.1. Giovanni Righini. January 17 th, 2013

Dynamic programming Doctoral course Optimization on graphs - Lecture.1 Giovanni Righini January 1 th, 201 Implicit enumeration Combinatorial optimization problems are in general NP-hard and we usually

### Relations Graphical View

Relations Slides by Christopher M. Bourke Instructor: Berthe Y. Choueiry Introduction Recall that a relation between elements of two sets is a subset of their Cartesian product (of ordered pairs). A binary

### Mining Social Network Graphs

Mining Social Network Graphs Debapriyo Majumdar Data Mining Fall 2014 Indian Statistical Institute Kolkata November 13, 17, 2014 Social Network No introduc+on required Really? We s7ll need to understand

### O(n) linear time This means that the algorithm requires a number of steps proportional to the size of the task.

Time complexity : Big O notation f(n) = O(g(n)) means There are positive constants c and k such that: 0

### Announcements. CSE332: Data Abstractions. Lecture 9: B Trees. Today. Our goal. M-ary Search Tree. M-ary Search Tree. Ruth Anderson Winter 2011

Announcements CSE2: Data Abstractions Project 2 posted! Partner selection due by 11pm Tues 1/25 at the latest. Homework due Friday Jan 28 st at the BEGINNING of lecture Lecture 9: B Trees Ruth Anderson

### Discrete Mathematics Problems

Discrete Mathematics Problems William F. Klostermeyer School of Computing University of North Florida Jacksonville, FL 32224 E-mail: wkloster@unf.edu Contents 0 Preface 3 1 Logic 5 1.1 Basics...............................

### Lecture 2: The Complexity of Some Problems

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 2: The Complexity of Some Problems David Mix Barrington and Alexis Maciel July 18, 2000

### CMPS 102 Solutions to Homework 1

CMPS 0 Solutions to Homework Lindsay Brown, lbrown@soe.ucsc.edu September 9, 005 Problem..- p. 3 For inputs of size n insertion sort runs in 8n steps, while merge sort runs in 64n lg n steps. For which

### Graph Theory for Articulated Bodies

Graph Theory for Articulated Bodies Alba Perez-Gracia Department of Mechanical Engineering, Idaho State University Articulated Bodies A set of rigid bodies (links) joined by joints that allow relative

### CSE 5311 Homework 2 Solution

CSE 5311 Homework 2 Solution Problem 6.2-6 Show that the worst-case running time of MAX-HEAPIFY on a heap of size n is Ω(lg n). (Hint: For a heap with n nodes, give node values that cause MAX- HEAPIFY

### Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

### Computer Algorithms. NP-Complete Problems. CISC 4080 Yanjun Li

Computer Algorithms NP-Complete Problems NP-completeness The quest for efficient algorithms is about finding clever ways to bypass the process of exhaustive search, using clues from the input in order

### Network Flow I. Lecture 16. 16.1 Overview. 16.2 The Network Flow Problem

Lecture 6 Network Flow I 6. Overview In these next two lectures we are going to talk about an important algorithmic problem called the Network Flow Problem. Network flow is important because it can be

### Introduction to Stationary Distributions

13 Introduction to Stationary Distributions We first briefly review the classification of states in a Markov chain with a quick example and then begin the discussion of the important notion of stationary

### Approximating the entropy of a 2-dimensional shift of finite type

Approximating the entropy of a -dimensional shift of finite type Tirasan Khandhawit c 4 July 006 Abstract. In this paper, we extend the method used to compute entropy of -dimensional subshift and the technique

### - Easy to insert & delete in O(1) time - Don t need to estimate total memory needed. - Hard to search in less than O(n) time

Skip Lists CMSC 420 Linked Lists Benefits & Drawbacks Benefits: - Easy to insert & delete in O(1) time - Don t need to estimate total memory needed Drawbacks: - Hard to search in less than O(n) time (binary

### A fairly quick tempo of solutions discussions can be kept during the arithmetic problems.

Distributivity and related number tricks Notes: No calculators are to be used Each group of exercises is preceded by a short discussion of the concepts involved and one or two examples to be worked out

### Graph Theory. Directed and undirected graphs make useful mental models for many situations. These objects are loosely defined as follows:

Graph Theory Directed and undirected graphs make useful mental models for many situations. These objects are loosely defined as follows: Definition An undirected graph is a (finite) set of nodes, some

### Unit 4: Dynamic Programming

Unit 4: Dynamic Programming Course contents: Assembly-line scheduling Matrix-chain multiplication Longest common subsequence Optimal binary search trees Applications: Rod cutting, optimal polygon triangulation,

### Quiz 1 Solutions. (a) T F The height of any binary search tree with n nodes is O(log n). Explain:

Introduction to Algorithms March 9, 2011 Massachusetts Institute of Technology 6.006 Spring 2011 Professors Erik Demaine, Piotr Indyk, and Manolis Kellis Quiz 1 Solutions Problem 1. Quiz 1 Solutions True

### 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008

MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

### Software Development (cs2500)

Software Development (cs2500) Lecture 53: Tail Recursion and Search M.R.C. van Dongen March 4, 2011 Contents 1 Outline 1 2 Tail Recursion 2 3 Search 4 4 Linear Search 5 5 Binary Search 6 6 Analysis 7 7

### 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008

MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

### Solving Single-digit Sudoku Subproblems

Solving Single-digit Sudoku Subproblems David Eppstein Int. Conf. Fun with Algorithms, June 2012 Sudoku An ab ab array of cells, partitioned into a b blocks, partially filled in with numbers from 1 to

### CS188 Spring 2011 Section 3: Game Trees

CS188 Spring 2011 Section 3: Game Trees 1 Warm-Up: Column-Row You have a 3x3 matrix of values like the one below. In a somewhat boring game, player A first selects a row, and then player B selects a column.

CS 252 - Markov Chains Additional Reading 1 and Homework problems 1 Markov Chains The first model we discussed, the DFA, is deterministic it s transition function must be total and it allows for transition

### CS60003 Algorithm Design and Analysis, Autumn

CS60003 Algorithm Design and Analysis, Autumn 2010 11 Class test 1 Maximum marks: 40 Time: 07 08 09 10 Duration: 1 + ǫ hour Roll no: Name: [ Write your answers in the question paper itself. Be brief and

### New algorithms for efficient parallel string comparison

New algorithms for efficient parallel string comparison Peter Krusche Alexander Tiskin Department of Computer Science and DIMAP University of Warwick, Coventry, CV4 7AL, UK SPAA 2010 In this talk......

### GRAPH THEORY and APPLICATIONS. Trees

GRAPH THEORY and APPLICATIONS Trees Properties Tree: a connected graph with no cycle (acyclic) Forest: a graph with no cycle Paths are trees. Star: A tree consisting of one vertex adjacent to all the others.

### The divide and conquer strategy has three basic parts. For a given problem of size n,

1 Divide & Conquer One strategy for designing efficient algorithms is the divide and conquer approach, which is also called, more simply, a recursive approach. The analysis of recursive algorithms often