Linear Progamming and Approximation Algorithms Joshua Wetzel Department of Computer Science Rutgers University Camden wetzeljo@camden.rutgers.edu March 24, 29 Joshua Wetzel / 52
What is Linear Programming? Optimize a linear objective fn subject to linear constraints. Examples: min 7x + x 2 + 3x 3 s.t. x x 2 + 3x 3 5x + 2x 2 x 3 6 x, x 2, x 3 max 6x + 2x 2 4x 3 s.t. 5x + x 2 2x 3 4 2x 2x 2 + 4x 3 2 x, x 2, x 3 Joshua Wetzel 2 / 52
Feasible Solutions: Upper Bounding OPT Linear Program: min 7x + x 2 + 3x 3 s.t. x x 2 + 3x 3 5x + 2x 2 x 3 6 x, x 2, x 3 Feasible Solutions: x = 2, x 2 =, x 3 = 4 7(2) + () + 5(4) = 35 x = 8 5, x 2 = 2, x 3 = 3 7( 8 5 ) + ( 2 ) + 5(3) = 26.7 All constraints are satisfied. OPT 35, OPT 26.7 Joshua Wetzel 3 / 52
Finding a Lower Bound on OPT Linear Program: min 7x + x 2 + 3x 3 s.t. x x 2 + 3x 3 5x + 2x 2 x 3 6 x, x 2, x 3 OPT? Joshua Wetzel 4 / 52
Lower Bounding OPT: Adding the Constraints min 7x + x 2 + 3x 3 s.t. x x 2 + 3x 3 + 5x + 2x 2 x 3 6 6x + x 2 + 2x 3 6 OPT 6 Joshua Wetzel 5 / 52
Lower Bounding OPT min 7x + x 2 + 3x 3 s.t. x x 2 + 3x 3 (y ) + 5x + 2x 2 x 3 6 (y 2 ) x ( y + 5y 2 ) + x 2 ( y + 2y 2 ) + x 3 ( 3y y 2 ) y + 6y 2 y + 6y 2 is a lower bound on OPT if: y + 5y 2 7 y + 2y 2 3y y 2 3 Joshua Wetzel 6 / 52
LP to Lower Bound OPT max y + 6y 2 s.t. y + 5y 2 7 y + 2y 2 3y y 2 3 y, y 2 Joshua Wetzel 7 / 52
Primal LP and Dual LP Primal LP: min 7x + x 2 + 3x 3 s.t. x x 2 + 3x 3 5x + 2x 2 x 3 6 x, x 2, x 3 Dual LP: max y + 6y 2 s.t. y + 5y 2 7 y + 2y 2 3y y 2 3 y, y 2 Every primal LP has a corresponding dual LP. If the primal is a min problem, the dual is a max problem. There is a dual constraint corresponding to each primal variable. Joshua Wetzel 8 / 52
LP Duality Theorems Dual Feasible Dual OPT = Primal OPT Primal Feasible The Primal LP: min 7x + x 2 + 3x 3 s.t. x x 2 + 3x 3 5x + 2x 2 x 3 6 x, x 2, x 3 The Dual LP: max y + 6y 2 s.t. y + 5y 2 7 y + 2y 2 3y y 2 3 y, y 2 Joshua Wetzel 9 / 52
Input: Given G = (V, E) Non-negative weights on vertices Objective: Find a least-weight collection of vertices such that each edge in G in incident on at least one vertex in the collection. Joshua Wetzel / 52
: Example 5 24 5 24 5 6 5 6 3 8 3 8 2 2 COST = 97 COST = 8 Joshua Wetzel / 52
Approximation Algorithms NP-hard problems. No optimal poly-time algorithms are known β-approximation alg., A, for a minimization problem P poly-time algorithm. for every instance I of P, A produces solution of cost at most β OPT(I) OPT(I)? Joshua Wetzel 2 / 52
Approximation Algorithms compute a lower bound on OPT. compare cost of our solution with the lower bound. lower bound OPT upper bound β Joshua Wetzel 3 / 52
Input: Given G = (V, E) Non-negative weights on vertices Objective: Find a least-weight collection of vertices such that each edge in G in incident on at least one vertex in the collection Joshua Wetzel 4 / 52
Unweighted : Algorithm Find a maximal matching in G Include in our cover both vertices incident on each edge of the matching Joshua Wetzel 5 / 52
Unweighted : Example Joshua Wetzel 6 / 52
Unweighted : Example Joshua Wetzel 7 / 52
Unweighted : Example Joshua Wetzel 8 / 52
Unweighted : Example Cost = 6 Joshua Wetzel 9 / 52
Analysis: Feasibility Every black edge shares a vertex with a green edge Joshua Wetzel 2 / 52
Analysis: Approximation Guarantee OPT has to choose at least one endpoint from each green edge. We choose both endpoints for each green edge. Hence: Our Cost 2OPT Joshua Wetzel 2 / 52
Unweighted : Tight Example OPT = 3 COST Alg = 6 Joshua Wetzel 22 / 52
Bad Example 2 2 2 2 OPT = Cost Alg = 2 Joshua Wetzel 23 / 52
: IP Formulation x v if v is in our cover, otherwise min w v x v s.t. v V x a + x b, e = (a, b) x v {, }, v V Joshua Wetzel 24 / 52
LP Relaxation Integer programs have been shown to be NP-hard Relax the integrality constraints x v {, }, v V min s.t. w v x v v V x a + x b, e = (a, b) x v, v V Joshua Wetzel 25 / 52
Why Do This? LP can be solved in polynomial time Every solution to the IP is also a solution to the LP Hence: OPT LP OPT IP We can use OPT LP as a lower bound on OPT IP Joshua Wetzel 26 / 52
LP Solution: Example x a + x b, e = (a, b) x v, v V /2 x a + x b, e = (a, b) x v {, }, v V /2 /2 OPT LP =.5 OPT IP = 2 Joshua Wetzel 27 / 52
Algorithm x optimal LP soln. ˆx v if x v 2, otherwise ˆx v Include v in our cover iff ˆx v = Joshua Wetzel 28 / 52
Analysis: Feasibility min s.t. w v x v v V x a + x b, e = (a, b) x v, v V x a 2 or x b 2 ˆx a = or ˆx b = Joshua Wetzel 29 / 52
Analysis: Approximation Guarantee Our Cost = v C w v = v V w vˆx v v V w v (2x v) = 2 v V w v x v = 2OPT LP 2OPT IP Joshua Wetzel 3 / 52
: Tight Example /2 /2 /2 OPT LP =.5 Our Cost = 3 Joshua Wetzel 3 / 52
Method Primal OPT OPT IP Solving the LP is expensive. Dual Feasible Dual OPT = Primal OPT Primal Feasible Better Alternative: Construct the dual LP Construct an algorithm that manually tightens dual constraints to obtain a maximal dual solution Joshua Wetzel 32 / 52
Constructing the Dual LP Primal LP: min s.t. w v x v v V x a + x b, e = (a, b) (y e ) x v, v V Joshua Wetzel 33 / 52
Primal LP and Dual LP min s.t. Primal LP: w v x max v v V x a + x b, e = (a, b) x v, v V s.t. e E Dual LP: y e y e w v, v V e : e hits v y e, e E Joshua Wetzel 34 / 52
Bar-Yehuda and Even Algorithm Inititally all edges are uncovered. While an uncovered edge in G: Choose an arbitrary edge, e Raise the value of y e for that edge until one of its incident vertices, v, becomes full (i.e y e = w v ) e:e hits v C C {v} Any edge that touches v is considered to be covered Return C as our vertex cover Joshua Wetzel 35 / 52
Bar-Yehuda and Even Algorithm: Example 5 24 5 6 3 8 2 Joshua Wetzel 36 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 24 5 6 3 8 2 Arbitrarily choose e and raise y e until a vertex is full Joshua Wetzel 37 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 24 6 5 6 3 8 2 Joshua Wetzel 38 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 24 6 5 6 3 8 2 Joshua Wetzel 39 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 5 24 6 5 6 3 8 2 Joshua Wetzel 4 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 5 24 6 5 6 3 8 2 Joshua Wetzel 4 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 5 8 24 6 5 6 3 8 2 Joshua Wetzel 42 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 5 8 24 6 5 6 3 8 2 Joshua Wetzel 43 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 5 8 24 6 5 6 3 2 8 2 Joshua Wetzel 44 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 5 8 24 6 5 6 3 2 8 2 Joshua Wetzel 45 / 52
Bar-Yehuda and Even Algorithm: Example y e w v e:e hits v 5 7 5 8 24 6 5 6 3 2 8 2 Joshua Wetzel 46 / 52
Bar-Yehuda and Even Algorithm: Example 5 7 5 8 24 6 5 6 3 2 8 2 Cost = 7 Joshua Wetzel 47 / 52
Bar-Yehuda and Even Algorithm: Analysis 5 7 5 8 24 6 Dual Obj. Fn: max e y e 5 6 3 2 8 2 y e w v e:e hits v Our Cost = wt(red vertices) 2 e hits red 2 e = 2DFS y e 2OPT y e Joshua Wetzel 48 / 52
Bar-Yehuda and Even Algorithm: Tight Example 6 6 COST OPT = 6 COST Bar-Yehuda = 2 Joshua Wetzel 49 / 52
Integrality Gap /2 /2 /2 OPT LP =.5 OPT IP = 2 For a complete graph of n vertices OPT LP = n/2 OPT IP = n OPT lim IP n OPT LP = lim n n (n/2) = 2 Joshua Wetzel 5 / 52
Reference R. Bar-Yehuda and S. Even. A linear time approximation algorithm for the weighted vertex cover problem. J. of Algorithms 2:98-23, 98. Joshua Wetzel 5 / 52
Thank You. Joshua Wetzel 52 / 52