Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one of three desired features. Solve problem to optimality. Solve problem in poly-time. Solve arbitrary instances of the problem. -approximation algorithm. Guaranteed to run in poly-time. Guaranteed to solve arbitrary instance of the problem Guaranteed to find solution within ratio of true optimum. Challenge. Need to prove a solution's value is close to optimum, without even knowing what optimum value is! Load Balancing. Load Balancing Input. m identical machines; n jobs; job j has processing time t j. Job j must run contiguously on one machine. A machine can process at most one job at a time. Def. Let J(i) be the subset of jobs assigned to machine i. The load of machine i is L i = j J(i) t j. Def. The makespan is the maximum load on any machine L = max i L i. Load balancing. Assign each job to a machine to minimize the makespan. Decision Version. Is the makespan bound by a number K? Load Balancing on Machines Load Balancing: Greedy Scheduling Claim. Load balancing is hard even if only machines. Pf. NUMBER-PARTITION P LOAD-BALANCE. NP-complete Greedy-scheduling algorithm. Consider n jobs in some fixed order. Assign job j to machine whose load is smallest so far. e a b f c g d Greedy-Scheduling(m, n, t,t,,t n ) { for i = to m { L i load on machine i J(i) jobs assigned to machine i machine machine length of job f a d Machine f b c Machine e g yes for j = to n { i = argmin k { L k J(i) J(i) {j L i L i + t j return J(),, J(m) machine i has smallest load assign job j to machine i update load of machine i Time L Implementation: O(n log m) using a priority queue. 6
-approximation Load Balancing: Greedy Scheduling Analysis An algorithm for an optimization problem is a -approximation if the solution found by the algorithm is always within a factor of the optimal solution. Minimization Problem: = approximate-solution/optimal-solution Maximization Problem: = optimal-solution/approximate-solution In general,. If =, then the solution is optimal. Theorem. [Graham, 66] Greedy algorithm is a -approximation. First worst-case analysis of an approximation algorithm. Need to compare resulting solution with optimal makespan L*. Lemma. The optimal makespan L* max j t j. Pf. Some machine must process the most time-consuming job. Lemma. The optimal makespan Pf. The total processing time is j t j. L * m j t j. One of m machines must do at least a /m fraction of total work. 7 8 Load Balancing: Greedy Scheduling Analysis Load Balancing: Greedy Scheduling Analysis Theorem. Greedy algorithm is a -approximation. Pf. Consider max load L i of bottleneck machine i. Let j be the last job scheduled on machine i. When job j assigned to machine i, i had smallest load. Its load before assignment is L i -t j L i -t j L k for all k m. Theorem. Greedy algorithm is a -approximation. Pf. Consider max load L i of bottleneck machine i. Let j be the last job scheduled on machine i. When job j assigned to machine i, i had smallest load. Its load before assignment is L i -t j L i -t j L k for all k m. Sum inequalities over all k and divide by m: blue jobs scheduled before j L i t j m k L k m k t k Lemma L * machine i j Now L i (L i t j ) t j L *. L* L* Lemma L i -t j L = L i Load Balancing: Greedy Scheduling Analysis Load Balancing: List Scheduling Analysis Q. Is our analysis tight? A. Essentially yes. Ex: m machines, m(m-) jobs length jobs, one job of length m Greedy solution = m-; Q. Is our analysis tight? A. Essentially yes. Ex: m machines, m(m-) jobs length jobs, one job of length m Greedy solution = m-; Optimal makespan = m; = -/m. m = machine idle machine idle machine idle machine idle machine 6 idle machine 7 idle machine 8 idle machine idle machine idle m = list scheduling makespan = optimal makespan =
Load Balancing: LPT Rule Load Balancing: LPT Rule Longest processing time (LPT). Sort n jobs in descending order of processing time, and then run Greedy scheduling algorithm. Observation. If at most m jobs, then greedy-scheduling is optimal. Pf. Each job put on its own machine. LPT-Greedy-Scheduling(m, n, t,t,,t n ) { Sort jobs so that t t t n for i = to m { L i J(i) for j = to n { i = argmin k L k J(i) J(i) {j L i L i + t j return J(),, J(m) load on machine i jobs assigned to machine i machine i has smallest load assign job j to machine i update load of machine i Lemma. If there are more than m jobs, L* t m+. Pf. Consider first m+ jobs t,, t m+. Since the t i 's are in descending order, each takes at least t m+ time. There are m+ jobs and m machines, so by pigeonhole principle, at least one machine gets two jobs. Theorem. LPT rule is a / approximation algorithm. Pf. Same basic approach as for the first greedy scheduling. L i (L i t j ) t j L *. L* L* Lemma ( by observation, can assume number of jobs > m ) Load Balancing: LPT Rule Q. Is our / analysis tight? A. No.. Center Selection Theorem. [Graham, 6] LPT rule is a /-approximation. Pf. More sophisticated analysis of the same algorithm. Q. Is Graham's / analysis tight? A. Essentially yes. Center Selection Problem Center Selection Problem Input. Set of n sites s,, s n and integer k >. Center selection problem. Select k centers C so that maximum distance from a site to nearest center is minimized. Input. Set of n sites s,, s n and integer k >. Center selection problem. Select k centers C so that maximum distance from a site to nearest center is minimized. r(c) k = Notation. dist(x, y) = distance between x and y. dist(s i, C) = min c C dist(s i, c) = distance from s i to closest center. r(c) = max i dist(s i, C) = smallest covering radius. Goal. Find set of centers C that minimizes r(c), subject to C = k. center site Distance function properties. dist(x, x) = (identity) dist(x, y) = dist(y, x) (symmetry) dist(x, y) dist(x, z) + dist(z, y) (triangle inequality) 7 8
Center Selection Example Greedy Algorithm: A False Start Ex: each site is a point in the plane, a center can be any point in the plane, dist(x, y) = Euclidean distance. Remark: search can be infinite! Greedy algorithm. Put the first center at the best possible location for a single center, and then keep adding centers so as to reduce the covering radius each time by as much as possible. Remark: arbitrarily bad! r(c) greedy center center k = centers site center site Center Selection: Greedy Algorithm Center Selection: Analysis of Greedy Algorithm Greedy algorithm. Repeatedly choose the next center to be the site farthest from any existing center. Greedy-Center-Selection(k, n, s,s,,s n ) { C = { s repeat k- times { Select a site s i with maximum dist(s i, C) Add s i to C site farthest from any center return C Theorem. Let C* = {c i * be an optimal set of centers. Then r(c) r(c*). Pf. (by contradiction) Assume r(c*) < ½ r(c). For each site c i in C, consider ball of radius ½ r(c) around it. Exactly one c i * in each ball; let c i be the site paired with c i *. Consider any site s and its closest center c i * in C*. dist(s, C) = dist(s, c i ) dist(s, c i *) + dist(c i *, c i ) r(c*). Thus r(c) r(c*). ½r(C) r(c*) since c i* is closest center ½r(C) Observation. Upon termination all centers in C are pairwise at least r(c) apart. Pf. By construction of algorithm. C* sites ½r(C) s c i c i * Center Selection Center Selection: Hardness of Approximation Theorem. Let C* be an optimal set of centers. Then r(c) r(c*). Theorem. Greedy algorithm is a -approximation for center selection problem. Remark. Greedy algorithm always places centers at sites, but is still within a factor of of best solution that is allowed to place centers anywhere. e.g., points in the plane Question. Is there hope of a /-approximation? /? Theorem. Unless P = NP, there no -approximation for center-selection problem for any <. Theorem. Unless P = NP, there is no -approximation algorithm for metric k-center problem for any <. Pf. We show how we could use a ( - ) approximation algorithm for k- center to solve DOMINATING-SET in poly-time. Let G = (V, E), k be an instance of DOMINATING-SET. see Exercise 8. Construct instance G' of k-center with sites V and distances d(u, v) = if (u, v) E d(u, v) = if (u, v) E Note that G' satisfies the triangle inequality. Claim: G has dominating set of size k iff there exists k centers C* with r(c*) =. Thus, if G has a dominating set of size k, a ( - )-approximation algorithm on G' must find a solution C* with r(c*) = since it cannot use any edge of distance.
. The Pricing Method: ed Vertex Cover ed Vertex Cover ed vertex cover. Given a graph G with vertex weights, find a vertex cover of minimum weight. weight = + + weight = 6 Pricing Method Pricing Method Pricing method. Each edge must be covered by some vertex. Edge e = (i, j) pays price p e to use vertex i and j. Pricing method. Set prices and find vertex cover simultaneously. Fairness. Edges incident to vertex i should pay w i in total. for each vertex i : p e wi e ( i, j) Lemma. For any vertex cover S and any fair prices p e : e p e w(s). Pf. pe ee pe wi w( S). is e( i, j) is ed-vertex-cover-approx(g, w) { foreach e in E p e = pe wi e ( i, j) while ( edge i-j such that neither i nor j are tight) select such an edge e increase p e as much as possible until i or j tight S set of all tight nodes return S each edge e covered by at least one node in S sum fairness inequalities for each node in S 7 8 Pricing Method Pricing Method: Analysis Theorem. Pricing method is a -approximation. Pf. Algorithm terminates since at least one new node becomes tight after each iteration of while loop. Let S = set of all tight nodes upon termination of algorithm. S is a vertex cover: if some edge i-j is uncovered, then neither i nor j is tight. But then while loop would not terminate. price of edge a-b Let S* be optimal vertex cover. We show w(s) w(s*). vertex weight w(s) w i i S i S p e e(i,j) p e iv e(i, j) p e w(s*). e E Figure.8 all nodes in S are tight S V, prices each edge counted twice fairness lemma
ed Vertex Cover.6 LP Rounding: ed Vertex Cover ed vertex cover. Given an undirected graph G = (V, E) with vertex weights w i, find a minimum weight subset of nodes S such that every edge is incident to at least one vertex in S. A 6F 6 B 7G 6 C H D I 7 E J total weight = ed Vertex Cover: IP Formulation ed Vertex Cover: IP Formulation ed vertex cover. Given an undirected graph G = (V, E) with vertex weights w i, find a minimum weight subset of nodes S such that every edge is incident to at least one vertex in S. Integer programming formulation. Model inclusion of each vertex i using a / variable x i. if vertex i is not in vertex cover x i if vertex i is in vertex cover Vertex covers in - correspondence with / assignments: S = {i V: x i = ed vertex cover. Integer programming formulation. (ILP) min w i x i i V s. t. x i x j (i, j) E x i {, i V Observation. If x* is optimal solution to (ILP), then S = {i V: x* i = is a minimum weight vertex cover. Objective function: minimize i w i x i. For each edge (i, j), must take either i or j: x i + x j. Integer Programming Linear Programming INTEGER-PROGRAMMING. Given integers a ij and b i, find integers x j that satisfy: max c t x s. t. Ax b x integral n a ij x j b i i m j x j j n x j integral j n Observation. Vertex cover formulation proves that integer programming is NP-hard. Linear programming. Max/min linear objective function subject to linear inequalities. Input: integers c j, b i, a ij. Output: real numbers x j. (P) max c t x s. t. Ax b x Linear. No x, xy, arccos(x), x(-x), etc. (P) max n c j x j j n s. t. a ij x j b i i m j x j j n even if all coefficients are / and at most two variables per inequality Simplex algorithm. [Dantzig 7] Can solve LP in practice. Ellipsoid algorithm. [Khachian 7] Can solve LP in poly-time. 6 6
LP Feasible Region ed Vertex Cover: LP Relaxation LP geometry in D. ed vertex cover. Linear programming formulation. x = (LP) min w i x i i V s. t. x i x j (i, j) E x i i V Observation. Optimal value of (LP) is optimal value of (ILP). Pf. LP has fewer constraints. Note. LP is not equivalent to vertex cover. ½ ½ x + x = 6 x = x + x = 6 Q. How can solving LP help us find a small vertex cover? A. Solve LP and round fractional values. ½ 7 8 ed Vertex Cover ed Vertex Cover Theorem. If x* is optimal solution to (LP), then S = {i V : x* i ½ is a vertex cover whose weight is at most twice the min possible weight. Pf. [S is a vertex cover] Consider an edge (i, j) E. Since x* i + x* j, either x* i ½or x* j ½ (i, j) covered. Pf. [S has desired cost] Let S* be optimal vertex cover. Then Theorem. -approximation algorithm for weighted vertex cover. Theorem. [Dinur-Safra ] If P NP, then no -approximation for <.67, even with unit weights. - Open research problem. Close the gap. w i i S* * w i x i w i i S i S LP is a relaxation x* i ½ Polynomial Time Approximation Scheme.8 Knapsack Problem PTAS. ( + )-approximation algorithm for any constant >. Load balancing. [Hochbaum-Shmoys 87] Euclidean TSP. [Arora 6] Consequence. PTAS produces arbitrarily high quality solution, but trades off time for accuracy. This section. PTAS for knapsack problem via rounding and scaling. 7
Knapsack Problem Knapsack is NP-Complete Knapsack problem. Given n objects and a "knapsack." i has value v i > and weighs w i >. we'll assume w i W Knapsack can carry weight up to W. Goal: fill knapsack so as to maximize total value. Ex: {, has value. 6 W = 8 8 6 7 KNAPSACK: Given a finite set X, nonnegative weights w i, nonnegative values v i, a weight limit W, and a target value V, is there a subset S X such that: w i W is v i V is SUBSET-SUM: Given a finite set X, nonnegative values u i, and an integer U, is there a subset S X whose elements sum to exactly U? Claim. SUBSET-SUM P KNAPSACK. Pf. Given instance (u,, u n, U) of SUBSET-SUM, create KNAPSACK instance: v i w i u i V W U u i U is u i U is Knapsack Problem: Dynamic Programming Knapsack Problem: Dynamic Programming II Def. OPT(i, w) = max value subset of items,..., i with weight limit w. Case : OPT does not select item i. OPT selects best of,, i using up to weight limit w Case : OPT selects item i. new weight limit = w w i OPT selects best of,, i using up to weight limit w w i if i OPT(i, w) OPT(i, w) if w i w maxopt(i, w), v i OPT(i, w w i ) otherwise Running time. O(n W). W = weight limit. Not polynomial in input size! Def. OPT(i, v) = min weight subset of items,, i that yields value exactly v. Case : OPT does not select item i. OPT selects best of,, i- that achieves exactly value v Case : OPT selects item i. consumes weight w i, new value needed = v v i OPT selects best of,, i- that achieves exactly value v if v if i, v > OPT(i, v) OPT(i, v) if v i v minopt(i, v), w i OPT(i, v v i ) otherwise 6 if v if i, v > OPT(i, v) OPT(i, v) if v i v minopt(i, v), w i OPT(i, v v i ) otherwise if v if i, v > OPT(i, v) OPT(i, v) if v i v minopt(i, v), w i OPT(i, v v i ) otherwise i = or v = i =, v = 8 x x x x x x x x x x x x x x x 6 W = 8 x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 6 W = 7 8 8
if v if i, v > OPT(i, v) OPT(i, v) if v i v minopt(i, v), w i OPT(i, v v i ) otherwise if v if i, v > OPT(i, v) OPT(i, v) if v i v minopt(i, v), w i OPT(i, v v i ) otherwise i =, v = i =, v = 8 x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 6 W = 8 x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 6 8 x x x x x x x x x x 6 W = if v if i, v > OPT(i, v) OPT(i, v) if v i v minopt(i, v), w i OPT(i, v v i ) otherwise if v if i, v > OPT(i, v) OPT(i, v) if v i v minopt(i, v), w i OPT(i, v v i ) otherwise i =, v = i =, v = 8 x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 6 8 x x x x x x x x x x x x x x x x 6 W = 8 x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 6 8 x x x x x x x x x x x x x x x x 7 8 6 8 6 W = Knapsack Problem: Dynamic Programming II Def. OPT(i, v) = min weight subset of items,, i that yields value exactly v. Case : OPT does not select item i. OPT selects best of,, i- that achieves exactly value v Case : OPT selects item i. Intuition for approximation algorithm. Round all values up to lie in smaller range. Run dynamic programming algorithm on rounded instance. Return optimal items in rounded instance. consumes weight w i, new value needed = v v i OPT selects best of,, i- that achieves exactly value v if v if i, v > OPT(i, v) OPT(i, v) if v i v minopt(i, v), w i OPT(i, v v i ) otherwise,,6, 7,8,,7,8 6 7,, 7 6 8 6 8 7 V* n v max Running time. O(n V*) = O(n v max ). V* = optimal value = maximum v such that OPT(n, v) W. Not polynomial in input size! original instance W = rounded instance W =
Knapsack PTAS. Round up all values: v max = largest value in original instance = precision parameter = scaling factor = v max / n Observation. Optimal solution to problems with or are equivalent. v Intuition. close to v so optimal solution using is nearly optimal; small and integral so dynamic programming algorithm is fast. v ˆ Running time. O(n / ). vi vi vi, vˆ i v ˆ Dynamic program II running time is O(n v ˆ max ), where v ˆ max vmax n v v v Knapsack PTAS. Round up all values: i vi Theorem. If S is solution found by our algorithm and S* is an optimal solution of the original problem, then () v i v i Pf. Let S* be an optimal solution satisfying weight constraint. v i i S* v i i S* v i i S i S (v i ) i S v i n i S () v i i S i S* always round up solve rounded instance optimally never round up by more than S n DP alg can take v max n = v max, v max is v i 6