# CSC Linear Programming and Combinatorial Optimization Lecture 12: Approximation Algorithms using Tools from LP

Save this PDF as:

Size: px
Start display at page:

Download "CSC Linear Programming and Combinatorial Optimization Lecture 12: Approximation Algorithms using Tools from LP"

## Transcription

1 S Linear Programming and ombinatorial Optimization Lecture 12: Approximation Algorithms using Tools from LP Notes taken by Matei David April 12, 2005 Summary: In this lecture, we give three example applications of approximating the solutions to IP by solving the relaxed LP and rounding. We begin with Vertex over, where we get a 2-approximation. Then we look at Set over, for which we give a randomized rounding algorithm which achieves a O(log n)-approximation with high probability. Finally, we again apply randomized rounding to approximate MAXSAT. In the associated tutorial, we revisit MAXSAT. Then, we discuss primaldual algorithms, including one for Set over. LP relaxation It is easy to capture certain combinatorial problems with an IP. We can then relax integrality constraints to get a LP, and solve the latter efficiently. In the process, we might lose something, for the relaxed LP might have a strictly better optimum than the original IP. In last class, we have seen that in certain cases, we can use algebraic properties of the matrix to argue that we do not lose anything in the relaxation, i.e. we get an exact relaxation. Note that, in general, we can add constraints to an IP to the point where we do not lose anything when relaxing it to an LP. However, the size of the inflated IP is usually exponential, and this procedure is not algorithmically doable. 1 Vertex over revisited In the IP/LP formulation of V, we are minimizing x i (or w i x i in the weighted case), subject to constraints of the form x i + x j 1 for all edges i, j. In the IP, x i is 1 when we have to pick set i, and 0 otherwise. In the LP relaxation, we only ask that 0 x i 1. Actually, the upper bound on x i is redundant, as the optimum solution cannot have any x i > 1. Lecture Notes for a course given by Avner Magen, Dept. of omputer Sciecne, University of Toronto. 1

2 Let OP T (G) denote the optimum to the IP (which is what we want), and let OP T f (G) denote the (fractional) optimum solution of the LP (which we can get efficiently). To see that for some G, OP T f (G) < OP T (G), look at K 3. In that case, OP T (K 3 ) = 2, but x 1 = x 2 = x 3 = 1/2 is a feasible LP solution with cost 3/2, so OP T f (K 3 ) < 3/2. In fact, by symmetry we can argue that OP T f (G 3 ) = 3/2. So we can get an optimal fractional solution to the LP with cost OP T f, but what we want is a good integral solution to the IP, with cost close to OP T. If we were able to always find the best integral solution, then we would solve V, so P = NP. Therefore, we ll settle for an integral solution which is not too far from OP T. The idea we are going to use is to round certain fractional x i s to integers. We will then argue about the factor by which the objective function grows during the rounding. Before describing the algorithm for V, note the following Lemma. Lemma Every vertex of the LP relaxation of V is half-integral, meaning that all its coordinates have values 0, 1/2 or 1. Algorithm Let x denote an optimal fractional solution. Given the fact above, the most natural rounding is to take the ceiling, i.e. we set x i = 0 if x i = 0, and x i = 1 if x i {1/2, 1}. In order to argue that we get a 2-approximation, we need to show that (i) x is still feasible, and that (ii) at most a factor of 2 is lost in the rounding. The ratio of the approximation algorithm is a measure of how far the integral optimum OP T is from the integral solution we are building. In general, we do not know OP T, so we use OP T f instead, which is a lower bound on OP T. To argue (i), assume that some constraint x i + x j 1 for some edge (i, j) is now unsatisfied. Then x i = x j = 0. By the rounding, x i = x j = 0. But then x i + x j = 0 < 1, contradicting the assumption that x is feasible. As for (ii), x i 2 x i = 2 OP T f 2 OP T. Note that even without the Lemma, we could round anything below 1/2 to 0, anything equal to or greater than 1/2 to 1, and the argument would still work. Proof of Lemma Let x be a feasible solution which contains other than half-integral values. So, for certain i, x i (0, 1/2) and for certain j, x j (1/2, 1). We claim that x is not a BFS of the LP. To prove this, we show that x is not extreme. The first thing to note is that there can be no edges in between vertices i, j with both x i < 1/2 and x j 1/2, because the constraint x i + x j 1 would not be satisfied. For some small but positive ɛ, consider the solutions x i + ɛ x i (1/2, 1) x i ɛ x i (1/2, 1) x + i = x i ɛ x i (0, 1/2) x i = x i + ɛ x i (0, 1/2) otherwise otherwise x i We can choose ɛ small enough so that values of all vertices do not leave the intervals (0, 1/2) and (1/2, 1), i.e. x + i (1/2, 1) iff x i (1/2, 1) etc. learly, x = (x + +x )/2. We claim both x +, x are still feasible. The only edges (i, j) which x + might violate are those which have at least one vertex, say i, with x i x i 2

3 (0, 1/2). But then x j > 1/2, for otherwise x would not be feasible. If x j = 1, then x + i +x+ j 1. Otherwise, x j (1/2, 1) and x + i +x+ j = (x i ɛ)+(x j +ɛ) = x i +x j 1 by feasibility of x. A similar argument shows that x is feasible. Hence, x is a convex combination of two different feasible solutions, therefore it s not an extreme and not a BFS. Notes There are trivial 2-approximation algorithms for V, the entire machinery of IP/LP solving and rounding wasn t really necessary. However, it does make for a nice application. The algorithm consists of the following steps. relax the IP to an LP; solve the LP; map the fractional optimum to the LP to a feasible integral solution; analyze that not much is lost in the translation. Approximation ratios An algorithm A for a minimization problem is called an α- A(I) approximation if α = sup I OP T (I) 1. Note that α might depend on the size of the input I (e.g. α = log n). For V, we ve seen a constant factor 2-approximation. For a A(I) maximization problem, an algorithm is an α-approximation if α = inf I OP T (I) 1. Integrality Gap The integrality gap for a certain IP/LP formulation of a problem is a bound on the ratio between the true integral optimum and the fractional optimum. The IG of a minimization problem is sup I. and the IG of a maximization OP T (I) OP T f (I) OP T (I) OP T f (I) problem is inf I. Intuitively, the closer the IG is to 1, the better we hope the approximation will be. To analyze the IG for the V formulation we ve seen before, consider K n. For this instance, we have OP T (K n ) = n 1. But x 1 = = x n = 1/2 is a feasible solution to the LP, so OP T f (K n ) n/2. Hence IG 2 2/n. Since on any instance I, the cost of the integral solution we produce is at least OP T (I) and at most 2 OP T f (I), we get IG 2. Therefore, for this formulation of V, taking the supremum we get IG = 2. Tightening an IP Suppose we have an IP for a minimization problem, and we have to relax it to an LP. The LP might achieve smaller optimum cost than the IP by using fractional solutions. onsider some constraint that any integral solution must satisfy. We can safely add any such constraint to the IP/LP formulation, without affecting the integral optimum. However, the new constraint might eliminate certain fractional solutions, so the fractional optimum can only increase, bringing it closer to the integral optimum and lowering the IG. For example, consider any triangle (i, j, k) in a graph instance of V. No (integral) vertex cover of the graph can include less than 2 vertices from that triangle. Hence, we can safely add a constraint of the form x i + x j + x k 2 to the IP/LP formulation. 3

4 We do not change OP T, since any feasible solution of the IP satisfies this constraint. However, we might potentially increase OP T f, thus reducing the IG. What can we hope to get in this way? an we get a better than 2-approximation? What other LP formulation can we consider? For example, if we apply the general tightening technique by Lovasz-Schrijver [LS91], we get in one step constraints about all odd cycles in the graph. Namely, if is an odd cycle containing vertices x 1,..., x l, we can add the constraint l x i l 2. This is strictly stronger than what we can derive from individual edges, which is l l 2. Arora, Bollobos and Lovasz [ABL02] consider all LP formulations which involve a constant fraction of the vertices and that use the natural 0/1 variables, i.e. x i for each vertex i indicating if it is taken in the cover. They also contain formulations that contain all odd cycle constraints among others. They show that despite these relatively strong formulations, the integrality gap is still 2 o(1). 2 Randomized Rounding for Set over In the Set over problem, we have a universe U of n elements, and m subsets S 1,..., S m U. In the unweighted case, we simply want to minimize the number of sets we pick in order to cover U. Notice that V is a special instance of S, where the elements of the universe are the edges, and we have a set for every vertex, containing the edges incident to that vertex. IP/LP formulation For the IP/LP formulation os S, we consider the n m 0/1 matrix A, which has a ij = 1 if i S j and 0 otherwise. The sets correspond to columns in A, and elements of the universe to rows of A. The variable x j is 1 if we have to take S j and 0 otherwise. So, the IP/LP formulation is min x j subject to A x 1 In the IP, x j {0, 1}. In the LP, we only ask x 0. So, given the IP, we relax it to an LP and solve the LP to get optimal fractional solution x. How do we map this to an integral solution? as a probability. So, instead of directly The idea is to think of the quantity x i mapping x to an integral solution, we define a stochastic procedure to do this step. For example, if the optimal fractional solution is x = (0, 1/10, 9/10), we would like to be more likely to select S 3 than S 2, and we will never select S 1. So, in one pass of randomized rounding, we (independently) pick the set S i with probability x i. Note that this rounding has the nice property of leaving integer values unchanged. Suppose we do several (around log n) such passes, and we let our final cover be the union of all the covers in the individual passes. In other words, we take S i in the final cover, if we take S i in any one pass. We would like to say that with high probability we get a cover of U, and that with high probability, the objective function does not increase much. 4

5 We do not cover a certain element a in one pass if none of the sets a belongs to are taken in that pass. So, the probability a is not covered is exactly j:a S j (1 x j ). The constraint corresponding to a in the LP says that j:a S j x j 1. So, Pr[a not covered in one pass] = (1 x j ) e x j = e P x j e 1 = 1 e j:a S j After (log n + 2) iterations, we haven t covered a iff we haven t covered it in any iteration, i.e. with probability at most e (log n+2) 1 4n. By union bound, the probability that there exists an uncovered element after (log n + 2) passes is at most 1/4. On the other hand, E[ x j ] = (log n + 2) x i (log n + 2) OP T f. By Markov s inequality, Pr[ x j > 4 (log n + 2) OP T f ] 1 4 Therefore, the probability that after (log n + 2) iterations we have a cover of cost at most 4 (log n + 2) OP T f is at least 1 1/4 1/4 = 1/2. We can apply the same algorithm several times to amplify the probability of success. This is probably the easiest algorithm using randomized rounding. Note that if OP T is the true integral optimum, OP T f is the fractional optimum and A is the result of rounding OP T f to an integral solution, we have OP T f OP T A. The ratio OP T/OP T f is controlled by the IG, and the ratio A/OP T f is controlled by the rounding process. Ultimately, we are interested in bounding the ratio A/OP T. In particular, if we have a small IG, we also need a small loss in rounding to get a good ratio. For Set over, the IG is Ω(log n). Feige [F??] showed that if P NP, no other technique will do better. For Vertex over, the trivial 2-approximation algorithms are essentially the best known. 3 Randomized Rounding for MAXSAT The input to MAXSAT is a NF formula, which is a collection of clauses. Each clause is a disjunction of literals, and each literal is a boolean variable or the negation of a variable. For example, = x 1 x 3 x 5 x 7 is a clause. The output in the MAXSAT problem is an assignment that maximizes the number of satisfied clauses. IP formulation In the IP formulation, we introduce 0/1 variables x i for every variable in the formula, and 0/1 variables Z for every formula. The semantics is that x i is 1 in the IP iff x i should be set to true in the formula, and Z is 1 iff is satisfied by the assignment to x variables. For every clause, let L + be the set of indices of variables appearing positively in, and let L be the set of indices of variables appearing negatively. For example, if = x 1 x 5 x 7, we have L + = {1, 5} and L = {7}. 5

6 With this notation, the IP/LP is max Z such that x i + (1 x i ) Z i L + i L The IP requires x i, Z {0, 1}, while the LP relaxation requires 0 x i, Z 1. To see that the IP captures the problem, note that there is no point in Z being 0 when is satisfied, for we are maximizing the sum of the Z s. Unlike in the case of V, we do need the 1 constraints. The only constraint we do not actually need is Z 0. Let (x, Z ) denote an optimal fractional solution to the LP relaxation, with cost OP T f. Next, set x i to true in the formula with probability x i. There s no need to set Z, we simply set it to 1 if is satisfied. Note that with MAXSAT, there are no issues about feasibility of the constructed solution, as in the case of V and S. Let us now study the probability a certain clause is not satisfied. Wlog, = x 1 x k. We are assuming all literals are different, for two opposite occurences of the same variable make the clause always true, and two occurences of the same sign are redundant. Furthermore, we are assuming contains only positive literals, for otherwise we can replace x i by y i, and 1 x i by y i. For every clause, we seek to relate Z with the probability we satisfied clause. Pr[ not satisfied] = = k (1 x i ) ( k ) k (1 x i ), geometric mean at most arithmetic mean ( k k x i ) k 1 k ( ) k 1 Z, as k k x i Z The function h(z) = 1 ( ) 1 z k k on the interval z [0, 1] is concave, as d (1 dz h(z) = ( 1) k z ) ( k 1 1 ), k k d 2 (1 dz 2 h(z) = (k 1) z ) ( k 2 1 ) < 0. k k So h(z) = h((1 z) 0 + z 1) (1 z) h(0) + z h(1) = z h(1). Therefore, ( ) ( k ( Pr[ satisfied] 1 1 Z = h(z k ) Z h(1) = Z ) ) k k 6

7 Since Z = OP T f and OP T OP T f, we now have E[number of satisfied clauses] = Pr[ satisfied] ( ( Z ) ) ( OP T f 1 1 ) e ( OP T 1 1 ) e So, if the objective function gains Z, the rounded solution gains (1 1/e) Z in expectation. We thus get an assignment that satisfies at least (1 1/e) OP T f clauses in expectation. As before, the analysis involved bounding individually every Z. Derandomizing the rounding an we derandomize this algorithm, that is, decide how to set the variables deterministically in such a way that we obtain the same number of satisfied clauses, this time no longer in expectation? It turns out that for this problem, we can. We know that the expectation over all choices of x 1 is at least some T. What we can do is to set x 1 to true, simplify the formula, count the number T 1 of clauses we satisfy directly and recompute the number T 2 of clauses we expect to satisfy over the choices over the remaining variables. If T 1 + T 2 T, we leave x 1 set to true and continue. Otherwise, we set x i to false. We know that at least one of these choices gives the number of clauses at least T. E[# satisfied] = Pr[ satisfied] = (Pr[x i = 1] Pr[ satisfied x i = 1] + Pr[x i = 0] Pr[ satisfied x i = 0]) = x i Pr[ satisfied x i = 1] + (1 x i ) Pr[ satisfied x i = 0] = x i E[# satisfied x i = 1] + (1 x i ) E[# satisfied x i = 0] At least one of the two conditional expectations must be at least the unconditional one. 4 Tutorial - Half-Approximation for MAXSAT We are given a NF formula with variables x 1,..., x n, and clause weights w. We want to find a truth assignment that maximizes the sum of the weights of the satisfied clauses. Suppose we randomly and independently assign each x i to either true and false with probabilities 1/2 and 1/2. A clause of size is not satisfied with probability 7

8 2 (we assume each variable occurs exactly once in every clause, either positively or negatively). Let W be the random variable denoting the weight of the satisfied clauses. Than E[W ] = w Pr[ satisfied] = ( w 1 2 ) Since 1, we have E[W ] 1/2 w. Furthermore, the best we can hope for is to satisfy all clauses, so w OP T, and therefore E[W ] 1/2 OP T. This is an easy 2-approximation. We can derandomize this algorithm, so we can get a half-approximation deterministically. onsider a complete binary tree of depth n. We can label every node by the set of literals that are set to true everywhere in the subtree below that node. So label the root node with, label the two children of the root with {x 1 } and {x 1 }, respectively. In all nodes below the child labelled {x 1 }, the variable x 1 is set to true. onversely, below {x 1 }, x 1 is set to false. The labels on level 2 below root are {x 1, x 2 }, {x 1, x 2 }, {x 1, x 2 }, {x 1, x 2 }. Our algorithm can be seen as selecting a certain leaf of this tree, where all vaiables are assigned to either true or false. At the root, we have E[W ]. Set x 1 to true. Some clauses will be satisfied (those that contain x 1 ), say a of them. Other clauses will be simplified (those that contain x 1 ), say b of them. For the simplified clauses, we can get a 1/2-approximation. So if continue along this branch, we can get at least a+b/2 clauses satisfied in total, which is a lower bound on E[W x 1 = 1]. Since E[W ] = 1/2 E[W x 1 = 1]+1/2 E[W x 1 = 0], we know at least one of the conditional expectations is at least E[W ]. We can compute lower bounds on both E[W x 1 = 1] and E[W x 1 = 0], and simply follow the path along which the expectation is larger. 5 Tutorial - omplementary Slackness onsider the primal IP/LP min c j x j subject to a ij x j b i, for all 1 i m x j 0 with its dual m max b i y i subject to m a ij y i c j, for all 1 j n 8

9 y i 0 Let (x, y) be optimum solutions to the primal and the dual. The slackness conditions are: primal : x j 0 dual : y i 0 m a ij y i = c j a ij x j = b i The idea of a primal-dual algorithm for an IP is to use primal-dual slackness to solve the IP. This is not possible without breaking some hardness assumptions, but we can get an approximation algorithm. For example, we can relax dual slackness conditions and only require, for some α > 1, that y i 0 a ij x j α b i Note that by feasibility of x, we get b i n a ijx j. Theorem If we can find feasible primal and dual solutions satisfying primal slackness conditions and α-relaxed dual slackness conditions, then x is an α-approximation. c j x j = j:x j>0 = j:x j >0 = j:x j>0 = i:y i >0 = i:y i >0 i:y i>0 c j x j, since x j 0 ( m x j a ij y i ), by primal slackness m a ij x j y i y i y i j:x j >0 a ij x j, since y i 0 a ij x j, added 0 terms y i α b i, by relaxed dual slackness ( m ) = α b i y i, added 0 terms α OP T, since y feasible. 9

10 The problem, then, is how to chose α, and how to take steps making sure these conditions are satisfied. A general primal-dual algorithm look like the following. start with integral (generally non-feasible) primal solution x = 0 and with feasible dual solution y = 0 while there exists a primal constraint not satisfied for some i increase y i until one or many dual constraints j become tight set x j to 1 for all tight dual constraints To prove a bound on the approximation ratio, we have to chose α well. Unweighted Set over The primal LP is min S x S and the dual LP is subject to x S 1, for all elements i S:i S x 0 max i y i subject to y i 1, for all sets S i S y 0 Relaxed dual slackness can be easily satisfied with α = f. onsider the following primal-dual algorithm, following the general paradigm above: start with x = 0, which is integral but not primal feasible and with y = 0, which is dual feasible while there exists some element i not covered (i.e. S:i S x S < 1) set y i to 1 set x S to 1 for all sets S containing i (i.e. pick all these sets as part of the cover) The primal slackness condition x S 0 i S y i = 1 is satisfied because in the while loop we set y i = 1 for exaclty one element i from each S. The f-relaxed dual slackness condition y i 0 S:i S x S f 1 holds because for every element i, we can select at most all sets containing it, and there are at most f of those. So this algorith is an f-approximation to S. 10

11 References [ABL02] Arora, Bollobas and Lovasz. Proving integrality gaps without knowing the linear program. FOS, [LS91] Lovasz and Schrijver. ones of matrices and set-functions and 0-1 optimization. SIAM Journal on Optimization, [F??] Feige. 11

### Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques. My T. Thai

Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques My T. Thai 1 Overview An overview of LP relaxation and rounding method is as follows: 1. Formulate an optimization

### Lecture 7: Approximation via Randomized Rounding

Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### The multi-integer set cover and the facility terminal cover problem

The multi-integer set cover and the facility teral cover problem Dorit S. Hochbaum Asaf Levin December 5, 2007 Abstract The facility teral cover problem is a generalization of the vertex cover problem.

### Chapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### max cx s.t. Ax c where the matrix A, cost vector c and right hand side b are given and x is a vector of variables. For this example we have x

Linear Programming Linear programming refers to problems stated as maximization or minimization of a linear function subject to constraints that are linear equalities and inequalities. Although the study

### 1 Polyhedra and Linear Programming

CS 598CSC: Combinatorial Optimization Lecture date: January 21, 2009 Instructor: Chandra Chekuri Scribe: Sungjin Im 1 Polyhedra and Linear Programming In this lecture, we will cover some basic material

### Lecture Notes 6: Approximations for MAX-SAT. 2 Notes about Weighted Vertex Cover Approximation

Algorithmic Methods //00 Lecture Notes 6: Approximations for MAX-SAT Professor: Yossi Azar Scribe:Alon Ardenboim Introduction Although solving SAT is nown to be NP-Complete, in this lecture we will cover

### CSL851: Algorithmic Graph Theory Semester I Lecture 4: August 5

CSL851: Algorithmic Graph Theory Semester I 2013-14 Lecture 4: August 5 Lecturer: Naveen Garg Scribes: Utkarsh Ohm Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have not

### Nan Kong, Andrew J. Schaefer. Department of Industrial Engineering, Univeristy of Pittsburgh, PA 15261, USA

A Factor 1 2 Approximation Algorithm for Two-Stage Stochastic Matching Problems Nan Kong, Andrew J. Schaefer Department of Industrial Engineering, Univeristy of Pittsburgh, PA 15261, USA Abstract We introduce

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### Guessing Game: NP-Complete?

Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple

### Discrete Optimization

Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using

### Week 5 Integral Polyhedra

Week 5 Integral Polyhedra We have seen some examples 1 of linear programming formulation that are integral, meaning that every basic feasible solution is an integral vector. This week we develop a theory

### Notes on Matrix Multiplication and the Transitive Closure

ICS 6D Due: Wednesday, February 25, 2015 Instructor: Sandy Irani Notes on Matrix Multiplication and the Transitive Closure An n m matrix over a set S is an array of elements from S with n rows and m columns.

### Algorithm Design and Analysis

Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;

### Good luck, veel succes!

Final exam Advanced Linear Programming, May 7, 13.00-16.00 Switch off your mobile phone, PDA and any other mobile device and put it far away. No books or other reading materials are allowed. This exam

### Answers to some of the exercises.

Answers to some of the exercises. Chapter 2. Ex.2.1 (a) There are several ways to do this. Here is one possibility. The idea is to apply the k-center algorithm first to D and then for each center in D

### Quiz 1 Sample Questions IE406 Introduction to Mathematical Programming Dr. Ralphs

Quiz 1 Sample Questions IE406 Introduction to Mathematical Programming Dr. Ralphs These questions are from previous years and should you give you some idea of what to expect on Quiz 1. 1. Consider the

### CSL851: Algorithmic Graph Theory Semester I Lecture 1: July 24

CSL851: Algorithmic Graph Theory Semester I 2013-2014 Lecture 1: July 24 Lecturer: Naveen Garg Scribes: Suyash Roongta Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have

### THEORY OF SIMPLEX METHOD

Chapter THEORY OF SIMPLEX METHOD Mathematical Programming Problems A mathematical programming problem is an optimization problem of finding the values of the unknown variables x, x,, x n that maximize

### CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992

CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992 Original Lecture #6: 28 January 1991 Topics: Triangulating Simple Polygons

### CS420/520 Algorithm Analysis Spring 2010 Lecture 04

CS420/520 Algorithm Analysis Spring 2010 Lecture 04 I. Chapter 4 Recurrences A. Introduction 1. Three steps applied at each level of a recursion a) Divide into a number of subproblems that are smaller

### 24. The Branch and Bound Method

24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

### Chapter 4. Trees. 4.1 Basics

Chapter 4 Trees 4.1 Basics A tree is a connected graph with no cycles. A forest is a collection of trees. A vertex of degree one, particularly in a tree, is called a leaf. Trees arise in a variety of applications.

### Introduction to Logic in Computer Science: Autumn 2006

Introduction to Logic in Computer Science: Autumn 2006 Ulle Endriss Institute for Logic, Language and Computation University of Amsterdam Ulle Endriss 1 Plan for Today Now that we have a basic understanding

### ARTICLE IN PRESS. European Journal of Operational Research xxx (2004) xxx xxx. Discrete Optimization. Nan Kong, Andrew J.

A factor 1 European Journal of Operational Research xxx (00) xxx xxx Discrete Optimization approximation algorithm for two-stage stochastic matching problems Nan Kong, Andrew J. Schaefer * Department of

### 8.1 Makespan Scheduling

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programing: Min-Makespan and Bin Packing Date: 2/19/15 Scribe: Gabriel Kaptchuk 8.1 Makespan Scheduling Consider an instance

### 9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1

9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1 Seffi Naor Computer Science Dept. Technion Haifa, Israel Introduction

### Discuss the size of the instance for the minimum spanning tree problem.

3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can

### RANDOMIZATION IN APPROXIMATION AND ONLINE ALGORITHMS. Manos Thanos Randomized Algorithms NTUA

RANDOMIZATION IN APPROXIMATION AND ONLINE ALGORITHMS 1 Manos Thanos Randomized Algorithms NTUA RANDOMIZED ROUNDING Linear Programming problems are problems for the optimization of a linear objective function,

### A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem

A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem John Karlof and Peter Hocking Mathematics and Statistics Department University of North Carolina Wilmington Wilmington,

### Definition 11.1. Given a graph G on n vertices, we define the following quantities:

Lecture 11 The Lovász ϑ Function 11.1 Perfect graphs We begin with some background on perfect graphs. graphs. First, we define some quantities on Definition 11.1. Given a graph G on n vertices, we define

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### CSC2420 Fall 2012: Algorithm Design, Analysis and Theory

CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### 2.1 Complexity Classes

15-859(M): Randomized Algorithms Lecturer: Shuchi Chawla Topic: Complexity classes, Identity checking Date: September 15, 2004 Scribe: Andrew Gilpin 2.1 Complexity Classes In this lecture we will look

### Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### Minimum Makespan Scheduling

Minimum Makespan Scheduling Minimum makespan scheduling: Definition and variants Factor 2 algorithm for identical machines PTAS for identical machines Factor 2 algorithm for unrelated machines Martin Zachariasen,

### 4.4 The recursion-tree method

4.4 The recursion-tree method Let us see how a recursion tree would provide a good guess for the recurrence = 3 4 ) Start by nding an upper bound Floors and ceilings usually do not matter when solving

### ONLINE DEGREE-BOUNDED STEINER NETWORK DESIGN. Sina Dehghani Saeed Seddighin Ali Shafahi Fall 2015

ONLINE DEGREE-BOUNDED STEINER NETWORK DESIGN Sina Dehghani Saeed Seddighin Ali Shafahi Fall 2015 ONLINE STEINER FOREST PROBLEM An initially given graph G. s 1 s 2 A sequence of demands (s i, t i ) arriving

### Min-cost flow problems and network simplex algorithm

Min-cost flow problems and network simplex algorithm The particular structure of some LP problems can be sometimes used for the design of solution techniques more efficient than the simplex algorithm.

### Lecture 7 - Linear Programming

COMPSCI 530: Design and Analysis of Algorithms DATE: 09/17/2013 Lecturer: Debmalya Panigrahi Lecture 7 - Linear Programming Scribe: Hieu Bui 1 Overview In this lecture, we cover basics of linear programming,

### Analysis of Approximation Algorithms for k-set Cover using Factor-Revealing Linear Programs

Analysis of Approximation Algorithms for k-set Cover using Factor-Revealing Linear Programs Stavros Athanassopoulos, Ioannis Caragiannis, and Christos Kaklamanis Research Academic Computer Technology Institute

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### The Simplex Method. yyye

Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note #05 1 The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/

### Linear Programming is the branch of applied mathematics that deals with solving

Chapter 2 LINEAR PROGRAMMING PROBLEMS 2.1 Introduction Linear Programming is the branch of applied mathematics that deals with solving optimization problems of a particular functional form. A linear programming

### 1. R In this and the next section we are going to study the properties of sequences of real numbers.

+a 1. R In this and the next section we are going to study the properties of sequences of real numbers. Definition 1.1. (Sequence) A sequence is a function with domain N. Example 1.2. A sequence of real

### Solutions to Exercises 8

Discrete Mathematics Lent 2009 MA210 Solutions to Exercises 8 (1) Suppose that G is a graph in which every vertex has degree at least k, where k 1, and in which every cycle contains at least 4 vertices.

### Homework 15 Solutions

PROBLEM ONE (Trees) Homework 15 Solutions 1. Recall the definition of a tree: a tree is a connected, undirected graph which has no cycles. Which of the following definitions are equivalent to this definition

### CMPSCI611: Approximating MAX-CUT Lecture 20

CMPSCI611: Approximating MAX-CUT Lecture 20 For the next two lectures we ll be seeing examples of approximation algorithms for interesting NP-hard problems. Today we consider MAX-CUT, which we proved to

### Counting Falsifying Assignments of a 2-CF via Recurrence Equations

Counting Falsifying Assignments of a 2-CF via Recurrence Equations Guillermo De Ita Luna, J. Raymundo Marcial Romero, Fernando Zacarias Flores, Meliza Contreras González and Pedro Bello López Abstract

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### Full and Complete Binary Trees

Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### Discrete (and Continuous) Optimization Solutions of Exercises 1 WI4 131

Discrete (and Continuous) Optimization Solutions of Exercises 1 WI4 131 Kees Roos Technische Universiteit Delft Faculteit Informatietechnologie en Systemen Afdeling Informatie, Systemen en Algoritmiek

### GRAPH THEORY and APPLICATIONS. Trees

GRAPH THEORY and APPLICATIONS Trees Properties Tree: a connected graph with no cycle (acyclic) Forest: a graph with no cycle Paths are trees. Star: A tree consisting of one vertex adjacent to all the others.

### 6. Mixed Integer Linear Programming

6. Mixed Integer Linear Programming Javier Larrosa Albert Oliveras Enric Rodríguez-Carbonell Problem Solving and Constraint Programming (RPAR) Session 6 p.1/40 Mixed Integer Linear Programming A mixed

### Generating models of a matched formula with a polynomial delay

Generating models of a matched formula with a polynomial delay Petr Savicky Institute of Computer Science, Academy of Sciences of Czech Republic, Pod Vodárenskou Věží 2, 182 07 Praha 8, Czech Republic

### By W.E. Diewert. July, Linear programming problems are important for a number of reasons:

APPLIED ECONOMICS By W.E. Diewert. July, 3. Chapter : Linear Programming. Introduction The theory of linear programming provides a good introduction to the study of constrained maximization (and minimization)

### 4.6 Linear Programming duality

4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal

### Polytope Examples (PolyComp Fukuda) Matching Polytope 1

Polytope Examples (PolyComp Fukuda) Matching Polytope 1 Matching Polytope Let G = (V,E) be a graph. A matching in G is a subset of edges M E such that every vertex meets at most one member of M. A matching

### 4 Basics of Trees. Petr Hliněný, FI MU Brno 1 FI: MA010: Trees and Forests

4 Basics of Trees Trees, actually acyclic connected simple graphs, are among the simplest graph classes. Despite their simplicity, they still have rich structure and many useful application, such as in

### COLORED GRAPHS AND THEIR PROPERTIES

COLORED GRAPHS AND THEIR PROPERTIES BEN STEVENS 1. Introduction This paper is concerned with the upper bound on the chromatic number for graphs of maximum vertex degree under three different sets of coloring

### 8.1 Min Degree Spanning Tree

CS880: Approximations Algorithms Scribe: Siddharth Barman Lecturer: Shuchi Chawla Topic: Min Degree Spanning Tree Date: 02/15/07 In this lecture we give a local search based algorithm for the Min Degree

### THE SCHEDULING OF MAINTENANCE SERVICE

THE SCHEDULING OF MAINTENANCE SERVICE Shoshana Anily Celia A. Glass Refael Hassin Abstract We study a discrete problem of scheduling activities of several types under the constraint that at most a single

### Math 140a - HW 1 Solutions

Math 140a - HW 1 Solutions Problem 1 (WR Ch 1 #1). If r is rational (r 0) and x is irrational, prove that r + x and rx are irrational. Solution. Given that r is rational, we can write r = a b for some

### 6.852: Distributed Algorithms Fall, 2009. Class 2

.8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### 1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

### Lecture 6 Online and streaming algorithms for clustering

CSE 291: Unsupervised learning Spring 2008 Lecture 6 Online and streaming algorithms for clustering 6.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

### Online Adwords Allocation

Online Adwords Allocation Shoshana Neuburger May 6, 2009 1 Overview Many search engines auction the advertising space alongside search results. When Google interviewed Amin Saberi in 2004, their advertisement

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### CHAPTER 9. Integer Programming

CHAPTER 9 Integer Programming An integer linear program (ILP) is, by definition, a linear program with the additional constraint that all variables take integer values: (9.1) max c T x s t Ax b and x integral

### Recurrence Relations

Recurrence Relations Introduction Determining the running time of a recursive algorithm often requires one to determine the big-o growth of a function T (n) that is defined in terms of a recurrence relation.

### 11. APPROXIMATION ALGORITHMS

11. APPROXIMATION ALGORITHMS load balancing center selection pricing method: vertex cover LP rounding: vertex cover generalized load balancing knapsack problem Lecture slides by Kevin Wayne Copyright 2005

### OHJ-2306 Introduction to Theoretical Computer Science, Fall 2012 8.11.2012

276 The P vs. NP problem is a major unsolved problem in computer science It is one of the seven Millennium Prize Problems selected by the Clay Mathematics Institute to carry a \$ 1,000,000 prize for the

### Dantzig-Wolfe bound and Dantzig-Wolfe cookbook

Dantzig-Wolfe bound and Dantzig-Wolfe cookbook thst@man.dtu.dk DTU-Management Technical University of Denmark 1 Outline LP strength of the Dantzig-Wolfe The exercise from last week... The Dantzig-Wolfe

### Minimize subject to. x S R

Chapter 12 Lagrangian Relaxation This chapter is mostly inspired by Chapter 16 of [1]. In the previous chapters, we have succeeded to find efficient algorithms to solve several important problems such

### Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits

Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique

### An Approximation Algorithm for Bounded Degree Deletion

An Approximation Algorithm for Bounded Degree Deletion Tomáš Ebenlendr Petr Kolman Jiří Sgall Abstract Bounded Degree Deletion is the following generalization of Vertex Cover. Given an undirected graph

### Lecture 9. 1 Introduction. 2 Random Walks in Graphs. 1.1 How To Explore a Graph? CS-621 Theory Gems October 17, 2012

CS-62 Theory Gems October 7, 202 Lecture 9 Lecturer: Aleksander Mądry Scribes: Dorina Thanou, Xiaowen Dong Introduction Over the next couple of lectures, our focus will be on graphs. Graphs are one of

### I. Solving Linear Programs by the Simplex Method

Optimization Methods Draft of August 26, 2005 I. Solving Linear Programs by the Simplex Method Robert Fourer Department of Industrial Engineering and Management Sciences Northwestern University Evanston,

### THE FUNDAMENTAL THEOREM OF ARBITRAGE PRICING

THE FUNDAMENTAL THEOREM OF ARBITRAGE PRICING 1. Introduction The Black-Scholes theory, which is the main subject of this course and its sequel, is based on the Efficient Market Hypothesis, that arbitrages

### A simpler and better derandomization of an approximation algorithm for Single Source Rent-or-Buy

A simpler and better derandomization of an approximation algorithm for Single Source Rent-or-Buy David P. Williamson Anke van Zuylen School of Operations Research and Industrial Engineering, Cornell University,

### (67902) Topics in Theory and Complexity Nov 2, 2006. Lecture 7

(67902) Topics in Theory and Complexity Nov 2, 2006 Lecturer: Irit Dinur Lecture 7 Scribe: Rani Lekach 1 Lecture overview This Lecture consists of two parts In the first part we will refresh the definition

### Solving systems of linear equations over finite fields

Solving systems of linear equations over finite fields June 1, 2007 1 Introduction 2 3 4 Basic problem Introduction x 1 + x 2 + x 3 = 1 x 1 + x 2 = 1 x 1 + x 2 + x 4 = 1 x 2 + x 4 = 0 x 1 + x 3 + x 4 =

### 5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1

5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 General Integer Linear Program: (ILP) min c T x Ax b x 0 integer Assumption: A, b integer The integrality condition

### Approximating Minimum Bounded Degree Spanning Trees to within One of Optimal

Approximating Minimum Bounded Degree Spanning Trees to within One of Optimal ABSTACT Mohit Singh Tepper School of Business Carnegie Mellon University Pittsburgh, PA USA mohits@andrew.cmu.edu In the MINIMUM