6. Dynamic Programming
|
|
- Colin Butler
- 7 years ago
- Views:
Transcription
1 lgorithmic Paradigms Dynamic Programming reed Build up a solution incrementally, myopically optimizing some local criterion Divide-and-conquer Break up a problem into two sub-problems, solve each sub-problem independently, and combine solution to sub-problems to form solution to original problem Dynamic programming Break up a problem into a series of overlapping sub-problems, and build up solutions to larger and larger sub-problems lgorithm Design by Éva Tardos and Jon Kleinberg opyright 25 ddison Wesley Slides by Kevin Wayne 2 Dynamic Programming History Dynamic Programming pplications Bellman Pioneered the systematic study of dynamic programming in the 95s Etymology Dynamic programming = planning over time Secretary of Defense was hostile to mathematical research Bellman sought an impressive name to avoid confrontation "it's impossible to use dynamic in a pejorative sense" "something not even a ongressman could object to" Reference: Bellman, R E Eye of the Hurricane, n utobiography reas Bioinformatics ontrol theory Information theory Operations research omputer science: theory, graphics, I, systems, Some famous dynamic programming algorithms nix diff for comparing two files Viterbi for hidden Markov models Smith-Waterman for sequence alignment Bellman-Ford for shortest path routing in networks ocke-kasami-younger for parsing context free grammars 3 4
2 Weighted Interval Scheduling Weighted Interval Scheduling Weighted interval scheduling problem Job j starts at s j, finishes at f j, and has weight or value v j Two jobs compatible if they don't overlap oal: find maximum weight subset of mutually compatible jobs a b c d e f g h Time lgorithm Design by Éva Tardos and Jon Kleinberg opyright 25 ddison Wesley Slides by Kevin Wayne nweighted Interval Scheduling Review Weighted Interval Scheduling Recall reedy algorithm works if all weights are onsider jobs in ascending order of finish time dd job to subset if it is compatible with previously chosen jobs Notation Label jobs by finishing time: f f 2 f n Def p(j) = largest index i < j such that job i is compatible with j Ex: p(8) = 5, p() = 3, p(2) = Observation reedy algorithm can fail spectacularly if arbitrary weights are allowed weight = 999 b weight = a Time Time 8
3 Dynamic Programming: Binary hoice Weighted Interval Scheduling: Brute Force Notation OPT(j) = value of optimal solution to the problem consisting of job requests, 2,, j Brute force algorithm ase : OPT selects job j can't use incompatible jobs { p(j), p(j) 2,, j - must include optimal solution to problem consisting of remaining compatible jobs, 2,, p(j) optimal substructure ase 2: OPT does not select job j must include optimal solution to problem consisting of remaining compatible jobs, 2,, j- # if j = OPT( j) = $ % max v j OPT( p( j)), OPT( j ") { otherwise Input: n, s,,s n, f,,f n, v,,v n Sort jobs by finish times so that f f 2 f n ompute p(), p(2),, p(n) ompute-opt(j) { if (j = ) return else return max(v j ompute-opt(p(j)), ompute-opt(j-)) 9 Weighted Interval Scheduling: Brute Force Weighted Interval Scheduling: Memoization Observation Recursive algorithm fails spectacularly because of redundant sub-problems " exponential algorithms Memoization Store results of each sub-problem in a cache; lookup as needed Ex Number of recursive calls for family of "layered" instances grows like Fibonacci sequence 5 Input: n, s,,s n, f,,f n, v,,v n Sort jobs by finish times so that f f 2 f n ompute p(), p(2),, p(n) p() =, p(j) = j for j = to n M[j] = empty M[j] = global array M-ompute-Opt(j) { if (M[j] is empty) M[j] = max(w j M-ompute-Opt(p(j)), M-ompute-Opt(j-)) return M[j] 2
4 Weighted Interval Scheduling: Running Time utomated Memoization laim Memoized version of algorithm takes O(n log n) time Sort by finish time: O(n log n) omputing p(#) : O(n) after sorting by start time M-ompute-Opt(j): each invocation takes O() time and either utomated memoization Many functional programming languages (eg, Lisp) have built-in support for memoization Q Why not in imperative languages (eg, Java)? (i) returns an existing value M[j] (ii) fills in one new entry M[j] and makes two recursive calls Progress measure $ = # nonempty entries of M[] initially $ =, throughout $ n (ii) increases $ by " at most 2n recursive calls (defun F (n) (if (<= n ) n ( (F (- n )) (F (- n 2))))) Lisp (efficient) static int F(int n) { if (n <= ) return n; else return F(n-) F(n-2); Java (exponential) Overall running time of M-ompute-Opt(n) is O(n) F(39) F(4) F(38) Remark O(n) if jobs are pre-sorted by start and finish times F(3) F(38) F(3) F(3) F(3) F(35) F(3) F(3) F(35) F(35) F(3) F(34) 3 4 Weighted Interval Scheduling: Finding a Solution Weighted Interval Scheduling: Bottom-p Q Dynamic programming algorithms computes optimal value What if we want the solution itself? Do some post-processing Bottom-up dynamic programming nwind recursion Input: n, s,,s n, f,,f n, v,,v n Run M-ompute-Opt(n) Run Find-Solution(n) Find-Solution(j) { if (j = ) output nothing else if (v j M[p(j)] > M[j-]) print j Find-Solution(p(j)) else Find-Solution(j-) Sort jobs by finish times so that f f 2 f n ompute p(), p(2),, p(n) Iterative-ompute-Opt { M[] = for j = to n M[j] = max(v j M[p(j)], M[j-]) # of recursive calls n " O(n) 5
5 Segmented Least Squares 3 Segmented Least Squares Least squares Foundational problem in statistic and numerical analysis iven n points in the plane: (x, y ), (, ),, (x n, y n ) Find a line y = ax b that minimizes the sum of the squared error: n SSE = # ( y i " ax i " b) 2 i= y Solution alculus " min error is achieved when x a = n # x i y i i " (# i x i ) (# i y i ), b = n i " ( x i ) 2 # i # i # i y i " a # i x i n lgorithm Design by Éva Tardos and Jon Kleinberg opyright 25 ddison Wesley Slides by Kevin Wayne 8 Segmented Least Squares Segmented Least Squares Segmented least squares Points lie roughly on a sequence of several line segments iven n points in the plane (x, y ), (, ),, (x n, y n ) with x < < < x n, find a sequence of lines that minimizes f(x) Q What's a reasonable choice for f(x) to balance accuracy and parsimony? goodness of fit Segmented least squares Points lie roughly on a sequence of several line segments iven n points in the plane (x, y ), (, ),, (x n, y n ) with x < < < x n, find a sequence of lines that minimizes: the sum of the sums of the squared errors E in each segment the number of lines L Tradeoff function: E c L, for some constant c > number of lines y y x x 9 2
6 Dynamic Programming: Multiway hoice Segmented Least Squares: lgorithm Notation OPT(j) = minimum cost for points p, p i,, p j e(i, j) = minimum sum of squares for points p i, p i,, p j To compute OPT(j): Last segment uses points p i, p i,, p j for some i ost = e(i, j) c OPT(i-) INPT: n, p,,p N, c Segmented-Least-Squares() { M[] = for j = to n for i = to j compute the least square error e ij for the segment p i,, p j $ & if j = OPT( j) = % min { e(i, j) c OPT(i #) otherwise '& " i " j for j = to n M[j] = min i j (e ij c M[i-]) return M[n] Running time O(n 3 ) can be improved to O(n 2 ) by pre-computing various statistics Bottleneck = computing e(i, j) for O(n 2 ) pairs, O(n) per pair using previous formula 2 22 Knapsack Problem 4 Knapsack Problem Knapsack problem iven n objects and a "knapsack" Item i weighs w i > kilograms and has value v i > Knapsack has capacity of W kilograms oal: fill knapsack so as to maximize total value Ex: { 3, 4 has value 4 Item Value Weight W = reedy: repeatedly add item with maximum ratio v i / w i Ex: { 5, 2, achieves only value = 35 " greedy not optimal lgorithm Design by Éva Tardos and Jon Kleinberg opyright 25 ddison Wesley Slides by Kevin Wayne 24
7 Dynamic Programming: False Start Dynamic Programming: dding a New Variable Def OPT(i) = max profit subset of items,, i ase : OPT does not select item i OPT selects best of {, 2,, i- ase 2: OPT selects item i accepting item i does not immediately imply that we will have to reject other items without knowing what other items were selected before i, we don't even know if we have enough room for i onclusion Need more sub-problems Def OPT(i, w) = max profit subset of items,, i with weight limit w ase : OPT does not select item i OPT selects best of {, 2,, i- using weight limit w ase 2: OPT selects item i new weight limit = w w i OPT selects best of {, 2,, i using this new weight limit # if i = % OPT(i, w) = $ OPT(i ", w) if w i > w % & max{ OPT(i ", w), v i OPT(i ", w " w i ) otherwise 25 2 Knapsack Problem: Bottom-p Knapsack lgorithm Knapsack Fill up an n-by-w array W Input: n, w,,w N, v,,v N % for w = to W M[, w] = for i = to n for w = to W if (w i > w) M[i, w] = M[i-, w] else M[i, w] = max {M[i-, w], v i M[i-, w-w i ] n { {, 2 {, 2, 3 {, 2, 3, 4 {, 2, 3, 4, Item Value Weight return M[n, W] OPT: { 4, 3 value = 22 8 = 4 W =
8 Knapsack Problem: Running Time Running time &(n W) Not polynomial in input size "Pseudo-polynomial" Decision version of Knapsack is NP-complete [hapter 8] 5 RN Secondary Structure Knapsack approximation algorithm There exists a polynomial algorithm that produces a feasible solution that has value within % of optimum [Section 8] 29 lgorithm Design by Éva Tardos and Jon Kleinberg opyright 25 ddison Wesley Slides by Kevin Wayne RN Secondary Structure RN Secondary Structure RN String B = b b 2 b n over alphabet {,,, Secondary structure RN is single-stranded so it tends to loop back and form base pairs with itself This structure is essential for understanding behavior of molecule Ex: Secondary structure set of pairs S = { (b i, b j ) that satisfy: [Watson-rick] S is a matching and each pair in S is a Watson- rick complement: -, -, -, or - [No sharp turns] The ends of each pair are separated by at least 4 intervening bases If (b i, b j ) ' S, then i < j - 4 [Non-crossing] If (b i, b j ) and (b k, b l ) are two pairs in S, then we cannot have i < k < j < l Free energy sual hypothesis is that an RN molecule will form the secondary structure with the optimum total free energy approximate by number of base pairs oal iven an RN molecule B = b b 2 b n, find a secondary structure S that maximizes the number of base pairs complementary base pairs: -,
9 RN Secondary Structure: Examples RN Secondary Structure: Subproblems Examples First attempt OPT(j) = maximum number of base pairs in a secondary structure of the substring b b 2 b j match b t and b n base pair t n Difficulty Results in two sub-problems 4 Finding secondary structure in: b b 2 b t- Finding secondary structure in: b t b t2 b n- OPT(t-) need more sub-problems ok sharp turn crossing Dynamic Programming Over Intervals Bottom p Dynamic Programming Over Intervals Notation OPT(i, j) = maximum number of base pairs in a secondary structure of the substring b i b i b j Q What order to solve the sub-problems? Do shortest intervals first ase If i ( j - 4 OPT(i, j) = by no-sharp turns condition ase 2 Base b j is not involved in a pair OPT(i, j) = OPT(i, j-) RN(b,,b n ) { for k = 5,,, n- for i =, 2,, n-k j = i k ompute M[i, j] i ase 3 Base b j pairs with b t for some i t < j - 4 non-crossing constraint decouples resulting sub-problems OPT(i, j) = max t { OPT(i, t-) OPT(t, j-) return M[, n] using recurrence 8 9 j take max over t such that i t < j-4 and b t and b j are Watson-rick complements Running time O(n 3 ) Remark Same core idea in KY algorithm to parse context-free grammars 35 3
10 Dynamic Programming Summary Recipe haracterize structure of problem Recursively define value of optimal solution ompute value of optimal solution onstruct optimal solution from computed information Sequence lignment Dynamic programming techniques Binary choice: weighted interval scheduling Multi-way choice: segmented least squares dding a new variable: knapsack Dynamic programming over intervals: RN secondary structure Top-down vs bottom-up: different people have different intuitions 3 lgorithm Design by Éva Tardos and Jon Kleinberg opyright 25 ddison Wesley Slides by Kevin Wayne String Similarity Edit Distance How similar are two strings? ocurrance occurrence o c u r r a n c e - o c c u r r e n c e 5 mismatches, gap o c - u r r a n c e pplications Basis for nix diff Speech recognition omputational biology Edit distance [Levenshtein 9, Needleman-Wunsch 9, Smith-Waterman 98] ap penalty ); mismatch penalty * pq ost = sum of gap and mismatch penalties o c c u r r e n c e mismatch, gap T T T - T T T o c - u r r - a n c e T T T T - T T o c c u r r e - n c e * T * T * 2* 2) * mismatches, 3 gaps 39 4
11 Sequence lignment Sequence lignment: Problem Structure oal: iven two strings X = x x m and Y = y y n find alignment of minimum cost Def n alignment M is a set of ordered pairs x i -y j such that each item occurs in at most one pair and no crossings Def The pair x i -y j and x i' -y j' cross if i < i', but j > j' cost( M ) = $ " xi y j $ % $ % (x i, y j ) # M i : x i unmatched j : y j unmatched mismatch Ex: T vs TT Sol: M = -y, x 3 -, x 4 -y 3, x 5 -y 4, x -y gap x x 3 x 4 x 5 x T - - T T Def OPT(i, j) = min cost of aligning strings x x i and y y j ase : OPT matches x i -y j pay mismatch for x i -y j min cost of aligning two strings x x i- and y y j- ase 2a: OPT leaves x i unmatched pay gap for x i and min cost of aligning x x i- and y y j ase 2b: OPT leaves y j unmatched pay gap for y j and min cost of aligning x x i and y y j- " $ $ OPT(i, j) = # $ % $ j& if i = " ' xi y j OPT(i (, j () $ min # & OPT(i (, j) otherwise $ % & OPT(i, j () i& if j = y y 3 y 4 y 5 y 4 42 Sequence lignment: lgorithm Sequence-lignment(m, n, x x m, y y n, ), *) { for i = to m M[, i] = i) for j = to n M[j, ] = j) Sequence lignment in Linear Space for i = to m for j = to n M[i, j] = min(*[x i, y j ] M[i-, j-], ) M[i-, j], ) M[i, j-]) return M[m, n] nalysis &(mn) time and space English words or sentences: m, n omputational biology: m = n =, billions ops OK, but B array? 43 lgorithm Design by Éva Tardos and Jon Kleinberg opyright 25 ddison Wesley Slides by Kevin Wayne
12 Sequence lignment: Linear Space Sequence lignment: Linear Space Q an we avoid using quadratic space? Easy Optimal value in O(m n) space and O(mn) time ompute OPT(i, ) from OPT(i-, ) No longer a simple way to recover alignment itself Edit distance graph Let f(i, j) be shortest path from (,) to (i, j) Observation: f(i, j) = OPT(i, j) Theorem [Hirschberg, 95] Optimal alignment in O(m n) space and O(mn) time - y y 3 y 4 y 5 y lever combination of divide-and-conquer and dynamic programming Inspired by idea of Savitch from complexity theory x " xi y j ) ) i-j x 3 m-n 45 4 Sequence lignment: Linear Space Sequence lignment: Linear Space Edit distance graph Let f(i, j) be shortest path from (,) to (i, j) an compute f (, j) for any j in O(mn) time and O(m n) space Edit distance graph Let g(i, j) be shortest path from (i, j) to (m, n) an compute by reversing the edge orientations and inverting the roles of (, ) and (m, n) j y y 3 y 4 y 5 y y y 3 y 4 y 5 y - - x x i-j ) " xi y j i-j ) x 3 m-n x 3 m-n 4 48
13 Sequence lignment: Linear Space Sequence lignment: Linear Space Edit distance graph Let g(i, j) be shortest path from (i, j) to (m, n) an compute g(, j) for any j in O(mn) time and O(m n) space Observation The cost of the shortest path that uses (i, j) is f(i, j) g(i, j) j y y 3 y 4 y 5 y y y 3 y 4 y 5 y - - x i-j x i-j x 3 m-n x 3 m-n 49 5 Sequence lignment: Linear Space Sequence lignment: Linear Space Observation 2 let q be an index that minimizes f(q, n/2) g(q, n/2) Then, the shortest path from (, ) to (m, n) uses (q, n/2) Divide: find index q that minimizes f(q, n/2) g(q, n/2) using DP lign x q and y n/2 onquer: recursively compute optimal alignment in each piece n / 2 n / 2 y y 3 y 4 y 5 y y y 3 y 4 y 5 y - - x i-j q x i-j q x 3 m-n x 3 m-n 5 52
14 Sequence lignment: Running Time nalysis Warmup Sequence lignment: Running Time nalysis Theorem Let T(m, n) = max running time of algorithm on strings of length at most m and n T(m, n) = O(mn log n) Theorem Let T(m, n) = max running time of algorithm on strings of length m and n T(m, n) = O(mn) T(m, n) " 2T(m, n/2) O(mn) # T(m, n) = O(mn log n) Remark nalysis is not tight because two sub-problems are of size (q, n/2) and (m - q, n/2) In next slide, we save log n factor Pf (by induction on n) O(mn) time to compute f(, n/2) and g (, n/2) and find index q T(q, n/2) T(m - q, n/2) time for two recursive calls hoose constant c so that: T(m, 2) " cm T(2, n) " cn T(m, n) " cmn T(q, n/2) T(m # q, n/2) Base cases: m = 2 or n = 2 Inductive hypothesis: T(m, n) 2cmn T ( m, n) " " = = T ( q, n / 2) T ( m q, n / 2) cmn 2cqn / 2 2c( m q) n / 2 cmn cqn cmn cqn cmn 2cmn 53 54
! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.
Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three
More information! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.
Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of
More informationDynamic Programming. Lecture 11. 11.1 Overview. 11.2 Introduction
Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach
More informationChapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling
Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one
More information11. APPROXIMATION ALGORITHMS
11. APPROXIMATION ALGORITHMS load balancing center selection pricing method: vertex cover LP rounding: vertex cover generalized load balancing knapsack problem Lecture slides by Kevin Wayne Copyright 2005
More informationApproximation Algorithms
Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms
More informationAlgorithm Design and Analysis
Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;
More informationNear Optimal Solutions
Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.
More informationMathematical Induction. Lecture 10-11
Mathematical Induction Lecture 10-11 Menu Mathematical Induction Strong Induction Recursive Definitions Structural Induction Climbing an Infinite Ladder Suppose we have an infinite ladder: 1. We can reach
More information5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1
5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 General Integer Linear Program: (ILP) min c T x Ax b x 0 integer Assumption: A, b integer The integrality condition
More informationSolutions to Homework 6
Solutions to Homework 6 Debasish Das EECS Department, Northwestern University ddas@northwestern.edu 1 Problem 5.24 We want to find light spanning trees with certain special properties. Given is one example
More informationGuessing Game: NP-Complete?
Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple
More informationClassification - Examples
Lecture 2 Scheduling 1 Classification - Examples 1 r j C max given: n jobs with processing times p 1,...,p n and release dates r 1,...,r n jobs have to be scheduled without preemption on one machine taking
More informationApplied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
More information5.1 Bipartite Matching
CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson
More informationThe Goldberg Rao Algorithm for the Maximum Flow Problem
The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }
More informationAnalysis of Algorithms I: Optimal Binary Search Trees
Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search
More informationWhy? A central concept in Computer Science. Algorithms are ubiquitous.
Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online
More informationSIMS 255 Foundations of Software Design. Complexity and NP-completeness
SIMS 255 Foundations of Software Design Complexity and NP-completeness Matt Welsh November 29, 2001 mdw@cs.berkeley.edu 1 Outline Complexity of algorithms Space and time complexity ``Big O'' notation Complexity
More informationRecursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1
Recursion Slides by Christopher M Bourke Instructor: Berthe Y Choueiry Fall 007 Computer Science & Engineering 35 Introduction to Discrete Mathematics Sections 71-7 of Rosen cse35@cseunledu Recursive Algorithms
More informationWelcome to... Problem Analysis and Complexity Theory 716.054, 3 VU
Welcome to... Problem Analysis and Complexity Theory 716.054, 3 VU Birgit Vogtenhuber Institute for Software Technology email: bvogt@ist.tugraz.at office hour: Tuesday 10:30 11:30 slides: http://www.ist.tugraz.at/pact.html
More informationDynamic Programming Problem Set Partial Solution CMPSC 465
Dynamic Programming Problem Set Partial Solution CMPSC 465 I ve annotated this document with partial solutions to problems written more like a test solution. (I remind you again, though, that a formal
More informationOffline sorting buffers on Line
Offline sorting buffers on Line Rohit Khandekar 1 and Vinayaka Pandit 2 1 University of Waterloo, ON, Canada. email: rkhandekar@gmail.com 2 IBM India Research Lab, New Delhi. email: pvinayak@in.ibm.com
More informationCSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015
CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential
More informationLecture 13: The Knapsack Problem
Lecture 13: The Knapsack Problem Outline of this Lecture Introduction of the 0-1 Knapsack Problem. A dynamic programming solution to this problem. 1 0-1 Knapsack Problem Informal Description: We have computed
More information5.4 Closest Pair of Points
5.4 Closest Pair of Points Closest Pair of Points Closest pair. Given n points in the plane, find a pair with smallest Euclidean distance between them. Fundamental geometric primitive. Graphics, computer
More informationProblem Set 7 Solutions
8 8 Introduction to Algorithms May 7, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik Demaine and Shafi Goldwasser Handout 25 Problem Set 7 Solutions This problem set is due in
More informationLecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method
Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming
More information1 Approximating Set Cover
CS 05: Algorithms (Grad) Feb 2-24, 2005 Approximating Set Cover. Definition An Instance (X, F ) of the set-covering problem consists of a finite set X and a family F of subset of X, such that every elemennt
More informationcsci 210: Data Structures Recursion
csci 210: Data Structures Recursion Summary Topics recursion overview simple examples Sierpinski gasket Hanoi towers Blob check READING: GT textbook chapter 3.5 Recursion In general, a method of defining
More informationIntroduction to Parallel Programming and MapReduce
Introduction to Parallel Programming and MapReduce Audience and Pre-Requisites This tutorial covers the basics of parallel programming and the MapReduce programming model. The pre-requisites are significant
More informationDefinition 8.1 Two inequalities are equivalent if they have the same solution set. Add or Subtract the same value on both sides of the inequality.
8 Inequalities Concepts: Equivalent Inequalities Linear and Nonlinear Inequalities Absolute Value Inequalities (Sections 4.6 and 1.1) 8.1 Equivalent Inequalities Definition 8.1 Two inequalities are equivalent
More informationMapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research
MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With
More informationClass Overview. CSE 326: Data Structures. Goals. Goals. Data Structures. Goals. Introduction
Class Overview CSE 326: Data Structures Introduction Introduction to many of the basic data structures used in computer software Understand the data structures Analyze the algorithms that use them Know
More informationBicolored Shortest Paths in Graphs with Applications to Network Overlay Design
Bicolored Shortest Paths in Graphs with Applications to Network Overlay Design Hongsik Choi and Hyeong-Ah Choi Department of Electrical Engineering and Computer Science George Washington University Washington,
More informationCSC2420 Spring 2015: Lecture 3
CSC2420 Spring 2015: Lecture 3 Allan Borodin January 22, 2015 1 / 1 Announcements and todays agenda Assignment 1 due next Thursday. I may add one or two additional questions today or tomorrow. Todays agenda
More informationDiscrete Optimization
Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using
More informationOutline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits
Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique
More informationSoftware Testing. Definition: Testing is a process of executing a program with data, with the sole intention of finding errors in the program.
Software Testing Definition: Testing is a process of executing a program with data, with the sole intention of finding errors in the program. Testing can only reveal the presence of errors and not the
More informationTesting LTL Formula Translation into Büchi Automata
Testing LTL Formula Translation into Büchi Automata Heikki Tauriainen and Keijo Heljanko Helsinki University of Technology, Laboratory for Theoretical Computer Science, P. O. Box 5400, FIN-02015 HUT, Finland
More informationCreating, Solving, and Graphing Systems of Linear Equations and Linear Inequalities
Algebra 1, Quarter 2, Unit 2.1 Creating, Solving, and Graphing Systems of Linear Equations and Linear Inequalities Overview Number of instructional days: 15 (1 day = 45 60 minutes) Content to be learned
More informationFairness in Routing and Load Balancing
Fairness in Routing and Load Balancing Jon Kleinberg Yuval Rabani Éva Tardos Abstract We consider the issue of network routing subject to explicit fairness conditions. The optimization of fairness criteria
More informationClosest Pair Problem
Closest Pair Problem Given n points in d-dimensions, find two whose mutual distance is smallest. Fundamental problem in many applications as well as a key step in many algorithms. p q A naive algorithm
More informationEuclidean Minimum Spanning Trees Based on Well Separated Pair Decompositions Chaojun Li. Advised by: Dave Mount. May 22, 2014
Euclidean Minimum Spanning Trees Based on Well Separated Pair Decompositions Chaojun Li Advised by: Dave Mount May 22, 2014 1 INTRODUCTION In this report we consider the implementation of an efficient
More informationThe Tower of Hanoi. Recursion Solution. Recursive Function. Time Complexity. Recursive Thinking. Why Recursion? n! = n* (n-1)!
The Tower of Hanoi Recursion Solution recursion recursion recursion Recursive Thinking: ignore everything but the bottom disk. 1 2 Recursive Function Time Complexity Hanoi (n, src, dest, temp): If (n >
More informationDefinition 11.1. Given a graph G on n vertices, we define the following quantities:
Lecture 11 The Lovász ϑ Function 11.1 Perfect graphs We begin with some background on perfect graphs. graphs. First, we define some quantities on Definition 11.1. Given a graph G on n vertices, we define
More informationIntroduction to Support Vector Machines. Colin Campbell, Bristol University
Introduction to Support Vector Machines Colin Campbell, Bristol University 1 Outline of talk. Part 1. An Introduction to SVMs 1.1. SVMs for binary classification. 1.2. Soft margins and multi-class classification.
More informationPh.D. Thesis. Judit Nagy-György. Supervisor: Péter Hajnal Associate Professor
Online algorithms for combinatorial problems Ph.D. Thesis by Judit Nagy-György Supervisor: Péter Hajnal Associate Professor Doctoral School in Mathematics and Computer Science University of Szeged Bolyai
More informationLecture 2. Marginal Functions, Average Functions, Elasticity, the Marginal Principle, and Constrained Optimization
Lecture 2. Marginal Functions, Average Functions, Elasticity, the Marginal Principle, and Constrained Optimization 2.1. Introduction Suppose that an economic relationship can be described by a real-valued
More informationEfficient Fault-Tolerant Infrastructure for Cloud Computing
Efficient Fault-Tolerant Infrastructure for Cloud Computing Xueyuan Su Candidate for Ph.D. in Computer Science, Yale University December 2013 Committee Michael J. Fischer (advisor) Dana Angluin James Aspnes
More informationClosest Pair of Points. Kleinberg and Tardos Section 5.4
Closest Pair of Points Kleinberg and Tardos Section 5.4 Closest Pair of Points Closest pair. Given n points in the plane, find a pair with smallest Euclidean distance between them. Fundamental geometric
More informationComputer Algorithms. NP-Complete Problems. CISC 4080 Yanjun Li
Computer Algorithms NP-Complete Problems NP-completeness The quest for efficient algorithms is about finding clever ways to bypass the process of exhaustive search, using clues from the input in order
More information1.204 Lecture 10. Greedy algorithms: Job scheduling. Greedy method
1.204 Lecture 10 Greedy algorithms: Knapsack (capital budgeting) Job scheduling Greedy method Local improvement method Does not look at problem globally Takes best immediate step to find a solution Useful
More informationSeminar. Path planning using Voronoi diagrams and B-Splines. Stefano Martina stefano.martina@stud.unifi.it
Seminar Path planning using Voronoi diagrams and B-Splines Stefano Martina stefano.martina@stud.unifi.it 23 may 2016 This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International
More informationML for the Working Programmer
ML for the Working Programmer 2nd edition Lawrence C. Paulson University of Cambridge CAMBRIDGE UNIVERSITY PRESS CONTENTS Preface to the Second Edition Preface xiii xv 1 Standard ML 1 Functional Programming
More informationE3: PROBABILITY AND STATISTICS lecture notes
E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................
More informationDynamic programming formulation
1.24 Lecture 14 Dynamic programming: Job scheduling Dynamic programming formulation To formulate a problem as a dynamic program: Sort by a criterion that will allow infeasible combinations to be eli minated
More informationSMT 2014 Algebra Test Solutions February 15, 2014
1. Alice and Bob are painting a house. If Alice and Bob do not take any breaks, they will finish painting the house in 20 hours. If, however, Bob stops painting once the house is half-finished, then the
More informationCSCE 310J Data Structures & Algorithms. Dynamic programming 0-1 Knapsack problem. Dynamic programming. Dynamic Programming. Knapsack problem (Review)
CSCE J Data Structures & Algorithms Dynamic programming - Knapsac problem Dr. Steve Goddard goddard@cse.unl.edu CSCE J Data Structures & Algorithms Giving credit where credit is due:» Most of slides for
More informationDynamic programming. Doctoral course Optimization on graphs - Lecture 4.1. Giovanni Righini. January 17 th, 2013
Dynamic programming Doctoral course Optimization on graphs - Lecture.1 Giovanni Righini January 1 th, 201 Implicit enumeration Combinatorial optimization problems are in general NP-hard and we usually
More informationUnit 4: Dynamic Programming
Unit 4: Dynamic Programming Course contents: Assembly-line scheduling Matrix-chain multiplication Longest common subsequence Optimal binary search trees Applications: Rod cutting, optimal polygon triangulation,
More informationDiscuss the size of the instance for the minimum spanning tree problem.
3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can
More informationFixed Point Theorems
Fixed Point Theorems Definition: Let X be a set and let T : X X be a function that maps X into itself. (Such a function is often called an operator, a transformation, or a transform on X, and the notation
More information4.2 Sorting and Searching
Sequential Search: Java Implementation 4.2 Sorting and Searching Scan through array, looking for key. search hit: return array index search miss: return -1 public static int search(string key, String[]
More information1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
More informationRising Rates in Random institute (R&I)
139. Proc. 3rd Car. Conf. Comb. & Comp. pp. 139-143 SPANNING TREES IN RANDOM REGULAR GRAPHS Brendan D. McKay Computer Science Dept., Vanderbilt University, Nashville, Tennessee 37235 Let n~ < n2
More information16. Recursion. COMP 110 Prasun Dewan 1. Developing a Recursive Solution
16. Recursion COMP 110 Prasun Dewan 1 Loops are one mechanism for making a program execute a statement a variable number of times. Recursion offers an alternative mechanism, considered by many to be more
More informationScheduling Shop Scheduling. Tim Nieberg
Scheduling Shop Scheduling Tim Nieberg Shop models: General Introduction Remark: Consider non preemptive problems with regular objectives Notation Shop Problems: m machines, n jobs 1,..., n operations
More information2. (a) Explain the strassen s matrix multiplication. (b) Write deletion algorithm, of Binary search tree. [8+8]
Code No: R05220502 Set No. 1 1. (a) Describe the performance analysis in detail. (b) Show that f 1 (n)+f 2 (n) = 0(max(g 1 (n), g 2 (n)) where f 1 (n) = 0(g 1 (n)) and f 2 (n) = 0(g 2 (n)). [8+8] 2. (a)
More informationScheduling Single Machine Scheduling. Tim Nieberg
Scheduling Single Machine Scheduling Tim Nieberg Single machine models Observation: for non-preemptive problems and regular objectives, a sequence in which the jobs are processed is sufficient to describe
More informationThis unit will lay the groundwork for later units where the students will extend this knowledge to quadratic and exponential functions.
Algebra I Overview View unit yearlong overview here Many of the concepts presented in Algebra I are progressions of concepts that were introduced in grades 6 through 8. The content presented in this course
More information3. Mathematical Induction
3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)
More information1. Prove that the empty set is a subset of every set.
1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since
More informationCatalan Numbers. Thomas A. Dowling, Department of Mathematics, Ohio State Uni- versity.
7 Catalan Numbers Thomas A. Dowling, Department of Mathematics, Ohio State Uni- Author: versity. Prerequisites: The prerequisites for this chapter are recursive definitions, basic counting principles,
More informationFunctions Recursion. C++ functions. Declare/prototype. Define. Call. int myfunction (int ); int myfunction (int x){ int y = x*x; return y; }
Functions Recursion C++ functions Declare/prototype int myfunction (int ); Define int myfunction (int x){ int y = x*x; return y; Call int a; a = myfunction (7); function call flow types type of function
More informationLearning Example. Machine learning and our focus. Another Example. An example: data (loan application) The data and the goal
Learning Example Chapter 18: Learning from Examples 22c:145 An emergency room in a hospital measures 17 variables (e.g., blood pressure, age, etc) of newly admitted patients. A decision is needed: whether
More informationA Practical Scheme for Wireless Network Operation
A Practical Scheme for Wireless Network Operation Radhika Gowaikar, Amir F. Dana, Babak Hassibi, Michelle Effros June 21, 2004 Abstract In many problems in wireline networks, it is known that achieving
More informationStatistical Machine Translation: IBM Models 1 and 2
Statistical Machine Translation: IBM Models 1 and 2 Michael Collins 1 Introduction The next few lectures of the course will be focused on machine translation, and in particular on statistical machine translation
More informationData Structures and Algorithms Written Examination
Data Structures and Algorithms Written Examination 22 February 2013 FIRST NAME STUDENT NUMBER LAST NAME SIGNATURE Instructions for students: Write First Name, Last Name, Student Number and Signature where
More informationLecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs
CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like
More information24. The Branch and Bound Method
24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no
More informationSection IV.1: Recursive Algorithms and Recursion Trees
Section IV.1: Recursive Algorithms and Recursion Trees Definition IV.1.1: A recursive algorithm is an algorithm that solves a problem by (1) reducing it to an instance of the same problem with smaller
More informationGraph Theory Problems and Solutions
raph Theory Problems and Solutions Tom Davis tomrdavis@earthlink.net http://www.geometer.org/mathcircles November, 005 Problems. Prove that the sum of the degrees of the vertices of any finite graph is
More informationComputational Models Lecture 8, Spring 2009
Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown Univ. p. 1 Computational Models Lecture 8, Spring 2009 Encoding of TMs Universal Turing Machines The Halting/Acceptance
More informationOnline Algorithms: Learning & Optimization with No Regret.
Online Algorithms: Learning & Optimization with No Regret. Daniel Golovin 1 The Setup Optimization: Model the problem (objective, constraints) Pick best decision from a feasible set. Learning: Model the
More informationIntroduction to Automata Theory. Reading: Chapter 1
Introduction to Automata Theory Reading: Chapter 1 1 What is Automata Theory? Study of abstract computing devices, or machines Automaton = an abstract computing device Note: A device need not even be a
More informationMany algorithms, particularly divide and conquer algorithms, have time complexities which are naturally
Recurrence Relations Many algorithms, particularly divide and conquer algorithms, have time complexities which are naturally modeled by recurrence relations. A recurrence relation is an equation which
More informationDiagonalization. Ahto Buldas. Lecture 3 of Complexity Theory October 8, 2009. Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach.
Diagonalization Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Background One basic goal in complexity theory is to separate interesting complexity
More informationWarshall s Algorithm: Transitive Closure
CS 0 Theory of Algorithms / CS 68 Algorithms in Bioinformaticsi Dynamic Programming Part II. Warshall s Algorithm: Transitive Closure Computes the transitive closure of a relation (Alternatively: all paths
More informationMathematics Review for MS Finance Students
Mathematics Review for MS Finance Students Anthony M. Marino Department of Finance and Business Economics Marshall School of Business Lecture 1: Introductory Material Sets The Real Number System Functions,
More informationEQUATIONS and INEQUALITIES
EQUATIONS and INEQUALITIES Linear Equations and Slope 1. Slope a. Calculate the slope of a line given two points b. Calculate the slope of a line parallel to a given line. c. Calculate the slope of a line
More informationMath 55: Discrete Mathematics
Math 55: Discrete Mathematics UC Berkeley, Fall 2011 Homework # 5, due Wednesday, February 22 5.1.4 Let P (n) be the statement that 1 3 + 2 3 + + n 3 = (n(n + 1)/2) 2 for the positive integer n. a) What
More informationTransportation Polytopes: a Twenty year Update
Transportation Polytopes: a Twenty year Update Jesús Antonio De Loera University of California, Davis Based on various papers joint with R. Hemmecke, E.Kim, F. Liu, U. Rothblum, F. Santos, S. Onn, R. Yoshida,
More informationA Note on Maximum Independent Sets in Rectangle Intersection Graphs
A Note on Maximum Independent Sets in Rectangle Intersection Graphs Timothy M. Chan School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1, Canada tmchan@uwaterloo.ca September 12,
More informationCartesian Products and Relations
Cartesian Products and Relations Definition (Cartesian product) If A and B are sets, the Cartesian product of A and B is the set A B = {(a, b) :(a A) and (b B)}. The following points are worth special
More information6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, 2010. Class 4 Nancy Lynch
6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, 2010 Class 4 Nancy Lynch Today Two more models of computation: Nondeterministic Finite Automata (NFAs)
More informationWeb Data Extraction: 1 o Semestre 2007/2008
Web Data : Given Slides baseados nos slides oficiais do livro Web Data Mining c Bing Liu, Springer, December, 2006. Departamento de Engenharia Informática Instituto Superior Técnico 1 o Semestre 2007/2008
More informationComputability Theory
CSC 438F/2404F Notes (S. Cook and T. Pitassi) Fall, 2014 Computability Theory This section is partly inspired by the material in A Course in Mathematical Logic by Bell and Machover, Chap 6, sections 1-10.
More informationLecture 2: August 29. Linear Programming (part I)
10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.
More information11 Multivariate Polynomials
CS 487: Intro. to Symbolic Computation Winter 2009: M. Giesbrecht Script 11 Page 1 (These lecture notes were prepared and presented by Dan Roche.) 11 Multivariate Polynomials References: MC: Section 16.6
More information