# Can linear programs solve NP-hard problems?

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Can linear programs solve NP-hard problems? p. 1/9 Can linear programs solve NP-hard problems? Ronald de Wolf

2 Linear programs Can linear programs solve NP-hard problems? p. 2/9

3 Can linear programs solve NP-hard problems? p. 2/9 Linear programs A factory produces 2 types of goods

4 Can linear programs solve NP-hard problems? p. 2/9 Linear programs A factory produces 2 types of goods - unit of type 1 gives profite1; type 2 gives profite4

5 Can linear programs solve NP-hard problems? p. 2/9 Linear programs A factory produces 2 types of goods - unit of type 1 gives profite1; type 2 gives profite4 It has 4 kinds of resources: 650 of R 1, 100 of R 2,...

6 Can linear programs solve NP-hard problems? p. 2/9 Linear programs A factory produces 2 types of goods - unit of type 1 gives profite1; type 2 gives profite4 It has 4 kinds of resources: 650 of R 1, 100 of R 2,... - type 1 uses 34 units of R 1, type 2 uses 16 units of R 1

7 Can linear programs solve NP-hard problems? p. 2/9 Linear programs A factory produces 2 types of goods - unit of type 1 gives profite1; type 2 gives profite4 It has 4 kinds of resources: 650 of R 1, 100 of R 2,... - type 1 uses 34 units of R 1, type 2 uses 16 units of R 1 Maximizing profit is linear program

8 Can linear programs solve NP-hard problems? p. 2/9 Linear programs A factory produces 2 types of goods - unit of type 1 gives profite1; type 2 gives profite4 It has 4 kinds of resources: 650 of R 1, 100 of R 2,... - type 1 uses 34 units of R 1, type 2 uses 16 units of R 1 Maximizing profit is linear program: max x 1 +4x 2 s.t. 34x 1 +16x x 1,x 2 0

9 Can linear programs solve NP-hard problems? p. 2/9 Linear programs A factory produces 2 types of goods - unit of type 1 gives profite1; type 2 gives profite4 It has 4 kinds of resources: 650 of R 1, 100 of R 2,... - type 1 uses 34 units of R 1, type 2 uses 16 units of R 1 Maximizing profit is linear program: max x 1 +4x 2 Feasible region is a polytope s.t. 34x 1 +16x x 1,x 2 0

10 A bit of history of LPs Can linear programs solve NP-hard problems? p. 3/9

11 Can linear programs solve NP-hard problems? p. 3/9 A bit of history of LPs Simplex algorithm: developed by Kantorovich and Dantzig for use in WWII, published by Dantzig in 1947.

12 Can linear programs solve NP-hard problems? p. 3/9 A bit of history of LPs Simplex algorithm: developed by Kantorovich and Dantzig for use in WWII, published by Dantzig in Walks along vertices of polytope, optimizing objective

13 Can linear programs solve NP-hard problems? p. 3/9 A bit of history of LPs Simplex algorithm: developed by Kantorovich and Dantzig for use in WWII, published by Dantzig in Walks along vertices of polytope, optimizing objective Usually efficient in practice, worst-case exponential time

14 Can linear programs solve NP-hard problems? p. 3/9 A bit of history of LPs Simplex algorithm: developed by Kantorovich and Dantzig for use in WWII, published by Dantzig in Walks along vertices of polytope, optimizing objective Usually efficient in practice, worst-case exponential time Ellipsoid method (Khachiyan 79): takes polynomial time in the worst case, but is not practical

15 Can linear programs solve NP-hard problems? p. 3/9 A bit of history of LPs Simplex algorithm: developed by Kantorovich and Dantzig for use in WWII, published by Dantzig in Walks along vertices of polytope, optimizing objective Usually efficient in practice, worst-case exponential time Ellipsoid method (Khachiyan 79): takes polynomial time in the worst case, but is not practical Interior point method (Karmarkar 84): reasonably efficient in practice

16 Can we solve NP-hard problems by LP? Can linear programs solve NP-hard problems? p. 4/9

17 Can linear programs solve NP-hard problems? p. 4/9 Can we solve NP-hard problems by LP? Famous NP-hard problem: traveling salesman problem Find shortest cycle that goes through each of n cities exactly once

18 Can linear programs solve NP-hard problems? p. 4/9 Can we solve NP-hard problems by LP? Famous NP-hard problem: traveling salesman problem Find shortest cycle that goes through each of n cities exactly once A polynomial-size LP for TSP would show P = NP

19 Can linear programs solve NP-hard problems? p. 4/9 Can we solve NP-hard problems by LP? Famous NP-hard problem: traveling salesman problem Find shortest cycle that goes through each of n cities exactly once A polynomial-size LP for TSP would show P = NP Swart claimed to have found such LPs

20 Can linear programs solve NP-hard problems? p. 4/9 Can we solve NP-hard problems by LP? Famous NP-hard problem: traveling salesman problem Find shortest cycle that goes through each of n cities exactly once A polynomial-size LP for TSP would show P = NP Swart claimed to have found such LPs Yannakakis 88: symmetric LPs for TSP are exponential

21 Can linear programs solve NP-hard problems? p. 4/9 Can we solve NP-hard problems by LP? Famous NP-hard problem: traveling salesman problem Find shortest cycle that goes through each of n cities exactly once A polynomial-size LP for TSP would show P = NP Swart claimed to have found such LPs Yannakakis 88: symmetric LPs for TSP are exponential Swart s LPs were symmetric, so they couldn t work

22 Can linear programs solve NP-hard problems? p. 4/9 Can we solve NP-hard problems by LP? Famous NP-hard problem: traveling salesman problem Find shortest cycle that goes through each of n cities exactly once A polynomial-size LP for TSP would show P = NP Swart claimed to have found such LPs Yannakakis 88: symmetric LPs for TSP are exponential Swart s LPs were symmetric, so they couldn t work 20-year open problem: what about non-symmetric LP?

23 Can linear programs solve NP-hard problems? p. 4/9 Can we solve NP-hard problems by LP? Famous NP-hard problem: traveling salesman problem Find shortest cycle that goes through each of n cities exactly once A polynomial-size LP for TSP would show P = NP Swart claimed to have found such LPs Yannakakis 88: symmetric LPs for TSP are exponential Swart s LPs were symmetric, so they couldn t work 20-year open problem: what about non-symmetric LP? Sometimes non-symmetry helps a lot! (Kaibel et al 10)

24 Can linear programs solve NP-hard problems? p. 4/9 Can we solve NP-hard problems by LP? Famous NP-hard problem: traveling salesman problem Find shortest cycle that goes through each of n cities exactly once A polynomial-size LP for TSP would show P = NP Swart claimed to have found such LPs Yannakakis 88: symmetric LPs for TSP are exponential Swart s LPs were symmetric, so they couldn t work 20-year open problem: what about non-symmetric LP? Sometimes non-symmetry helps a lot! (Kaibel et al 10) Fiorini, Massar, Pokutta, Tiwary, dw (STOC 12): any LP for TSP needs exponential size

25 Polytopes and optimization problems Can linear programs solve NP-hard problems? p. 5/9

26 Can linear programs solve NP-hard problems? p. 5/9 Polytopes and optimization problems Polytope P: convex hull of finite set of points in R d

27 Can linear programs solve NP-hard problems? p. 5/9 Polytopes and optimization problems Polytope P: convex hull of finite set of points in R d bounded intersection of finitely many halfspaces

28 Can linear programs solve NP-hard problems? p. 5/9 Polytopes and optimization problems Polytope P: convex hull of finite set of points in R d bounded intersection of finitely many halfspaces Can be written as system of linear inequalities: P = {x R d Ax b}

29 Can linear programs solve NP-hard problems? p. 5/9 Polytopes and optimization problems Polytope P: convex hull of finite set of points in R d bounded intersection of finitely many halfspaces Can be written as system of linear inequalities: P = {x R d Ax b} Different systems Ax b can define the same P

30 Can linear programs solve NP-hard problems? p. 5/9 Polytopes and optimization problems Polytope P: convex hull of finite set of points in R d bounded intersection of finitely many halfspaces Can be written as system of linear inequalities: P = {x R d Ax b} Different systems Ax b can define the same P The size of P is the minimal number of inequalities

31 Can linear programs solve NP-hard problems? p. 5/9 Polytopes and optimization problems Polytope P: convex hull of finite set of points in R d bounded intersection of finitely many halfspaces Can be written as system of linear inequalities: P = {x R d Ax b} Different systems Ax b can define the same P The size of P is the minimal number of inequalities TSP polytope: each cycle in K n is vector {0,1} (n 2)

32 Can linear programs solve NP-hard problems? p. 5/9 Polytopes and optimization problems Polytope P: convex hull of finite set of points in R d bounded intersection of finitely many halfspaces Can be written as system of linear inequalities: P = {x R d Ax b} Different systems Ax b can define the same P The size of P is the minimal number of inequalities TSP polytope: each cycle in K n is vector {0,1} (n 2). TSP(n) is the convex hull of all those vectors

33 Can linear programs solve NP-hard problems? p. 5/9 Polytopes and optimization problems Polytope P: convex hull of finite set of points in R d bounded intersection of finitely many halfspaces Can be written as system of linear inequalities: P = {x R d Ax b} Different systems Ax b can define the same P The size of P is the minimal number of inequalities TSP polytope: each cycle in K n is vector {0,1} (n 2). TSP(n) is the convex hull of all those vectors Solving TSP w.r.t. weight function w ij : minimize the linear function i,j w ijx ij over x TSP(n)

34 Can linear programs solve NP-hard problems? p. 5/9 Polytopes and optimization problems Polytope P: convex hull of finite set of points in R d bounded intersection of finitely many halfspaces Can be written as system of linear inequalities: P = {x R d Ax b} Different systems Ax b can define the same P The size of P is the minimal number of inequalities TSP polytope: each cycle in K n is vector {0,1} (n 2). TSP(n) is the convex hull of all those vectors Solving TSP w.r.t. weight function w ij : minimize the linear function i,j w ijx ij over x TSP(n) TSP(n) has exponential size

35 Extended formulations of polytopes Can linear programs solve NP-hard problems? p. 6/9

36 Can linear programs solve NP-hard problems? p. 6/9 Extended formulations of polytopes Sometimes extra variables/dimensions can reduce size very much.

37 Can linear programs solve NP-hard problems? p. 6/9 Extended formulations of polytopes Sometimes extra variables/dimensions can reduce size very much. Regular n-gon in R 2 has size n, but is the projection of polytope Q in higher dimension, of size O(logn)

38 Can linear programs solve NP-hard problems? p. 6/9 Extended formulations of polytopes Sometimes extra variables/dimensions can reduce size very much. Regular n-gon in R 2 has size n, but is the projection of polytope Q in higher dimension, of size O(logn) Optimizing over P reduces to optimizing over Q. If Q has small size, this can be done efficiently!

39 Can linear programs solve NP-hard problems? p. 6/9 Extended formulations of polytopes Sometimes extra variables/dimensions can reduce size very much. Regular n-gon in R 2 has size n, but is the projection of polytope Q in higher dimension, of size O(logn) Optimizing over P reduces to optimizing over Q. If Q has small size, this can be done efficiently! How small can size(q) be?

40 Can linear programs solve NP-hard problems? p. 6/9 Extended formulations of polytopes Sometimes extra variables/dimensions can reduce size very much. Regular n-gon in R 2 has size n, but is the projection of polytope Q in higher dimension, of size O(logn) Optimizing over P reduces to optimizing over Q. If Q has small size, this can be done efficiently! How small can size(q) be? Extension complexity: xc(p) = min{size(q) Q projects to P}

41 Can linear programs solve NP-hard problems? p. 6/9 Extended formulations of polytopes Sometimes extra variables/dimensions can reduce size very much. Regular n-gon in R 2 has size n, but is the projection of polytope Q in higher dimension, of size O(logn) Optimizing over P reduces to optimizing over Q. If Q has small size, this can be done efficiently! How small can size(q) be? Extension complexity: xc(p) = min{size(q) Q projects to P} Our goal: strong lower bounds on xc(p) for interesting P

42 How to bound extension complexity? Can linear programs solve NP-hard problems? p. 7/9

43 Can linear programs solve NP-hard problems? p. 7/9 How to bound extension complexity? Slack matrix S of a polytope P = conv(v) with inequalities {A i x b i } and points V = {v j }: S ij = b i A i v j

44 Can linear programs solve NP-hard problems? p. 7/9 How to bound extension complexity? Slack matrix S of a polytope P = conv(v) with inequalities {A i x b i } and points V = {v j }: S ij = b i A i v j NB: every entry is nonnegative; S is not unique

45 Can linear programs solve NP-hard problems? p. 7/9 How to bound extension complexity? Slack matrix S of a polytope P = conv(v) with inequalities {A i x b i } and points V = {v j }: S ij = b i A i v j NB: every entry is nonnegative; S is not unique Yannakakis 88: xc(p) = positive rank of S

46 Can linear programs solve NP-hard problems? p. 7/9 How to bound extension complexity? Slack matrix S of a polytope P = conv(v) with inequalities {A i x b i } and points V = {v j }: S ij = b i A i v j NB: every entry is nonnegative; S is not unique Yannakakis 88: xc(p) = positive rank of S Instead of considering all Q, can focus on slack matrix!

47 Can linear programs solve NP-hard problems? p. 7/9 How to bound extension complexity? Slack matrix S of a polytope P = conv(v) with inequalities {A i x b i } and points V = {v j }: S ij = b i A i v j NB: every entry is nonnegative; S is not unique Yannakakis 88: xc(p) = positive rank of S Instead of considering all Q, can focus on slack matrix! We can use nondeterministic communication complexity to lower bound rank + (S)

48 Can linear programs solve NP-hard problems? p. 7/9 How to bound extension complexity? Slack matrix S of a polytope P = conv(v) with inequalities {A i x b i } and points V = {v j }: S ij = b i A i v j NB: every entry is nonnegative; S is not unique Yannakakis 88: xc(p) = positive rank of S Instead of considering all Q, can focus on slack matrix! We can use nondeterministic communication complexity to lower bound rank + (S) Big problem until now: which polytope to analyze, and how to analyze its slack matrix?

49 Lower bound for correlation polytope Can linear programs solve NP-hard problems? p. 8/9

50 Can linear programs solve NP-hard problems? p. 8/9 Lower bound for correlation polytope Correlation polytope: COR(n) = conv{bb T b {0,1} n }

51 Can linear programs solve NP-hard problems? p. 8/9 Lower bound for correlation polytope Correlation polytope: COR(n) = conv{bb T b {0,1} n } Occurs naturally in NP-hard problems, e.g. MaxClique

52 Can linear programs solve NP-hard problems? p. 8/9 Lower bound for correlation polytope Correlation polytope: COR(n) = conv{bb T b {0,1} n } Occurs naturally in NP-hard problems, e.g. MaxClique We can find 2 n valid constraints (indexed by a {0,1} n ) whose slacks w.r.t. vertex bb T are M ab = (1 a T b) 2

53 Can linear programs solve NP-hard problems? p. 8/9 Lower bound for correlation polytope Correlation polytope: COR(n) = conv{bb T b {0,1} n } Occurs naturally in NP-hard problems, e.g. MaxClique We can find 2 n valid constraints (indexed by a {0,1} n ) whose slacks w.r.t. vertex bb T are M ab = (1 a T b) 2 Nondeterministic communication complexity of M was already analyzed in the quantum computing! (dw 00)

54 Can linear programs solve NP-hard problems? p. 8/9 Lower bound for correlation polytope Correlation polytope: COR(n) = conv{bb T b {0,1} n } Occurs naturally in NP-hard problems, e.g. MaxClique We can find 2 n valid constraints (indexed by a {0,1} n ) whose slacks w.r.t. vertex bb T are M ab = (1 a T b) 2 Nondeterministic communication complexity of M was already analyzed in the quantum computing! (dw 00) Take slack matrix S for COR(n), with 2 n vertices bb T for columns, 2 n a-constraints for first 2 n rows, remaining facets for other rows S =. M ab..

55 Can linear programs solve NP-hard problems? p. 8/9 Lower bound for correlation polytope Correlation polytope: COR(n) = conv{bb T b {0,1} n } Occurs naturally in NP-hard problems, e.g. MaxClique We can find 2 n valid constraints (indexed by a {0,1} n ) whose slacks w.r.t. vertex bb T are M ab = (1 a T b) 2 Nondeterministic communication complexity of M was already analyzed in the quantum computing! (dw 00) Take slack matrix S for COR(n), with 2 n vertices bb T for columns, 2 n a-constraints for first 2 n rows, remaining facets for other rows xc(cor(n)) S =. M ab..

56 Can linear programs solve NP-hard problems? p. 8/9 Lower bound for correlation polytope Correlation polytope: COR(n) = conv{bb T b {0,1} n } Occurs naturally in NP-hard problems, e.g. MaxClique We can find 2 n valid constraints (indexed by a {0,1} n ) whose slacks w.r.t. vertex bb T are M ab = (1 a T b) 2 Nondeterministic communication complexity of M was already analyzed in the quantum computing! (dw 00) Take slack matrix S for COR(n), with 2 n vertices bb T for columns, 2 n a-constraints for first 2 n rows, remaining facets for other rows xc(cor(n)) = rank + (S) S =. M ab..

57 Can linear programs solve NP-hard problems? p. 8/9 Lower bound for correlation polytope Correlation polytope: COR(n) = conv{bb T b {0,1} n } Occurs naturally in NP-hard problems, e.g. MaxClique We can find 2 n valid constraints (indexed by a {0,1} n ) whose slacks w.r.t. vertex bb T are M ab = (1 a T b) 2 Nondeterministic communication complexity of M was already analyzed in the quantum computing! (dw 00) Take slack matrix S for COR(n), with 2 n vertices bb T for columns, 2 n a-constraints for first 2 n rows, remaining facets for other rows xc(cor(n)) = rank + (S) rank + (M) S =. M ab..

58 Can linear programs solve NP-hard problems? p. 8/9 Lower bound for correlation polytope Correlation polytope: COR(n) = conv{bb T b {0,1} n } Occurs naturally in NP-hard problems, e.g. MaxClique We can find 2 n valid constraints (indexed by a {0,1} n ) whose slacks w.r.t. vertex bb T are M ab = (1 a T b) 2 Nondeterministic communication complexity of M was already analyzed in the quantum computing! (dw 00) Take slack matrix S for COR(n), with 2 n vertices bb T for columns, 2 n a-constraints for first 2 n rows, remaining facets for other rows S = xc(cor(n)) = rank + (S) rank + (M) 2 Ω(n). M ab..

59 Consequences for other polytopes Can linear programs solve NP-hard problems? p. 9/9

60 Can linear programs solve NP-hard problems? p. 9/9 Consequences for other polytopes Via reduction from the correlation polytope:

61 Can linear programs solve NP-hard problems? p. 9/9 Consequences for other polytopes Via reduction from the correlation polytope: TSP-polytope has extension complexity 2 n

62 Can linear programs solve NP-hard problems? p. 9/9 Consequences for other polytopes Via reduction from the correlation polytope: TSP-polytope has extension complexity 2 n Recently improved to 2 n by Rothvoss

63 Can linear programs solve NP-hard problems? p. 9/9 Consequences for other polytopes Via reduction from the correlation polytope: TSP-polytope has extension complexity 2 n Recently improved to 2 n by Rothvoss CUT-polytope has extension complexity 2 n

64 Can linear programs solve NP-hard problems? p. 9/9 Consequences for other polytopes Via reduction from the correlation polytope: TSP-polytope has extension complexity 2 n Recently improved to 2 n by Rothvoss CUT-polytope has extension complexity 2 n For specific graphs, the stable-set polytope has extension complexity 2 n

65 Can linear programs solve NP-hard problems? p. 9/9 Consequences for other polytopes Via reduction from the correlation polytope: TSP-polytope has extension complexity 2 n Recently improved to 2 n by Rothvoss CUT-polytope has extension complexity 2 n For specific graphs, the stable-set polytope has extension complexity 2 n So every linear program based on extended formulations needs exponentially many constraints

66 Can linear programs solve NP-hard problems? p. 9/9 Consequences for other polytopes Via reduction from the correlation polytope: TSP-polytope has extension complexity 2 n Recently improved to 2 n by Rothvoss CUT-polytope has extension complexity 2 n For specific graphs, the stable-set polytope has extension complexity 2 n So every linear program based on extended formulations needs exponentially many constraints This rules out many efficient algorithms for NP-hard problems, and refutes all P=NP proofs à la Swart

### Linear Programming. Widget Factory Example. Linear Programming: Standard Form. Widget Factory Example: Continued.

Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.

### Linear Programming I

Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins

### Linear Programming: Introduction

Linear Programming: Introduction Frédéric Giroire F. Giroire LP - Introduction 1/28 Course Schedule Session 1: Introduction to optimization. Modelling and Solving simple problems. Modelling combinatorial

### Chapter 15 Introduction to Linear Programming

Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### Linear Programming is the branch of applied mathematics that deals with solving

Chapter 2 LINEAR PROGRAMMING PROBLEMS 2.1 Introduction Linear Programming is the branch of applied mathematics that deals with solving optimization problems of a particular functional form. A linear programming

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### Quiz 1 Sample Questions IE406 Introduction to Mathematical Programming Dr. Ralphs

Quiz 1 Sample Questions IE406 Introduction to Mathematical Programming Dr. Ralphs These questions are from previous years and should you give you some idea of what to expect on Quiz 1. 1. Consider the

### Minimize subject to. x S R

Chapter 12 Lagrangian Relaxation This chapter is mostly inspired by Chapter 16 of [1]. In the previous chapters, we have succeeded to find efficient algorithms to solve several important problems such

### 5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1

5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 General Integer Linear Program: (ILP) min c T x Ax b x 0 integer Assumption: A, b integer The integrality condition

### Linear Programming, Lagrange Multipliers, and Duality Geoff Gordon

lp.nb 1 Linear Programming, Lagrange Multipliers, and Duality Geoff Gordon lp.nb 2 Overview This is a tutorial about some interesting math and geometry connected with constrained optimization. It is not

### Nonnegative Matrix Factorization: Algorithms, Complexity and Applications

Nonnegative Matrix Factorization: Algorithms, Complexity and Applications Ankur Moitra Massachusetts Institute of Technology July 6th, 2015 Ankur Moitra (MIT) NMF July 6th, 2015 m M n W m M = A inner dimension

### 1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### Using the Simplex Method in Mixed Integer Linear Programming

Integer Using the Simplex Method in Mixed Integer UTFSM Nancy, 17 december 2015 Using the Simplex Method in Mixed Integer Outline Mathematical Programming Integer 1 Mathematical Programming Optimisation

### 1 Polyhedra and Linear Programming

CS 598CSC: Combinatorial Optimization Lecture date: January 21, 2009 Instructor: Chandra Chekuri Scribe: Sungjin Im 1 Polyhedra and Linear Programming In this lecture, we will cover some basic material

### The Simplex Method. yyye

Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note #05 1 The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### 5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

### 3. Evaluate the objective function at each vertex. Put the vertices into a table: Vertex P=3x+2y (0, 0) 0 min (0, 5) 10 (15, 0) 45 (12, 2) 40 Max

SOLUTION OF LINEAR PROGRAMMING PROBLEMS THEOREM 1 If a linear programming problem has a solution, then it must occur at a vertex, or corner point, of the feasible set, S, associated with the problem. Furthermore,

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Geometry of Linear Programming

Chapter 2 Geometry of Linear Programming The intent of this chapter is to provide a geometric interpretation of linear programming problems. To conceive fundamental concepts and validity of different algorithms

### By W.E. Diewert. July, Linear programming problems are important for a number of reasons:

APPLIED ECONOMICS By W.E. Diewert. July, 3. Chapter : Linear Programming. Introduction The theory of linear programming provides a good introduction to the study of constrained maximization (and minimization)

### Transportation Polytopes: a Twenty year Update

Transportation Polytopes: a Twenty year Update Jesús Antonio De Loera University of California, Davis Based on various papers joint with R. Hemmecke, E.Kim, F. Liu, U. Rothblum, F. Santos, S. Onn, R. Yoshida,

### What is Linear Programming?

Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### 26 Linear Programming

The greatest flood has the soonest ebb; the sorest tempest the most sudden calm; the hottest love the coldest end; and from the deepest desire oftentimes ensues the deadliest hate. Th extremes of glory

### 3y 1 + 5y 2. y 1 + y 2 20 y 1 0, y 2 0.

1 Linear Programming A linear programming problem is the problem of maximizing (or minimizing) a linear function subject to linear constraints. The constraints may be equalities or inequalities. 1.1 Example

### Linear Programming for Optimization. Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc.

1. Introduction Linear Programming for Optimization Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc. 1.1 Definition Linear programming is the name of a branch of applied mathematics that

### . P. 4.3 Basic feasible solutions and vertices of polyhedra. x 1. x 2

4. Basic feasible solutions and vertices of polyhedra Due to the fundamental theorem of Linear Programming, to solve any LP it suffices to consider the vertices (finitely many) of the polyhedron P of the

### Lecture 7 - Linear Programming

COMPSCI 530: Design and Analysis of Algorithms DATE: 09/17/2013 Lecturer: Debmalya Panigrahi Lecture 7 - Linear Programming Scribe: Hieu Bui 1 Overview In this lecture, we cover basics of linear programming,

### Good luck, veel succes!

Final exam Advanced Linear Programming, May 7, 13.00-16.00 Switch off your mobile phone, PDA and any other mobile device and put it far away. No books or other reading materials are allowed. This exam

### 6. Mixed Integer Linear Programming

6. Mixed Integer Linear Programming Javier Larrosa Albert Oliveras Enric Rodríguez-Carbonell Problem Solving and Constraint Programming (RPAR) Session 6 p.1/40 Mixed Integer Linear Programming A mixed

### Math 407A: Linear Optimization

Math 407A: Linear Optimization Lecture 4: LP Standard Form 1 1 Author: James Burke, University of Washington LPs in Standard Form Minimization maximization Linear equations to linear inequalities Lower

### Scheduling and (Integer) Linear Programming

Scheduling and (Integer) Linear Programming Christian Artigues LAAS - CNRS & Université de Toulouse, France artigues@laas.fr Master Class CPAIOR 2012 - Nantes Christian Artigues Scheduling and (Integer)

### LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS. 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method

LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method Introduction to dual linear program Given a constraint matrix A, right

### The Simplex Method. What happens when we need more decision variables and more problem constraints?

The Simplex Method The geometric method of solving linear programming problems presented before. The graphical method is useful only for problems involving two decision variables and relatively few problem

### SOLVING LINEAR SYSTEM OF INEQUALITIES WITH APPLICATION TO LINEAR PROGRAMS

SOLVING LINEAR SYSTEM OF INEQUALITIES WITH APPLICATION TO LINEAR PROGRAMS Hossein Arsham, University of Baltimore, (410) 837-5268, harsham@ubalt.edu Veena Adlakha, University of Baltimore, (410) 837-4969,

### 4.6 Linear Programming duality

4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal

### Interior Point Methods and Linear Programming

Interior Point Methods and Linear Programming Robert Robere University of Toronto December 13, 2012 Abstract The linear programming problem is usually solved through the use of one of two algorithms: either

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### 4.1 Learning algorithms for neural networks

4 Perceptron Learning 4.1 Learning algorithms for neural networks In the two preceding chapters we discussed two closely related models, McCulloch Pitts units and perceptrons, but the question of how to

### Computer Algorithms. NP-Complete Problems. CISC 4080 Yanjun Li

Computer Algorithms NP-Complete Problems NP-completeness The quest for efficient algorithms is about finding clever ways to bypass the process of exhaustive search, using clues from the input in order

### Theory of Linear Programming

Theory of Linear Programming Debasis Mishra April 6, 2011 1 Introduction Optimization of a function f over a set S involves finding the maximum (minimum) value of f (objective function) in the set S (feasible

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### I. Solving Linear Programs by the Simplex Method

Optimization Methods Draft of August 26, 2005 I. Solving Linear Programs by the Simplex Method Robert Fourer Department of Industrial Engineering and Management Sciences Northwestern University Evanston,

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### Lecture 2: August 29. Linear Programming (part I)

10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.

### Optimization Modeling for Mining Engineers

Optimization Modeling for Mining Engineers Alexandra M. Newman Division of Economics and Business Slide 1 Colorado School of Mines Seminar Outline Linear Programming Integer Linear Programming Slide 2

### LECTURE: INTRO TO LINEAR PROGRAMMING AND THE SIMPLEX METHOD, KEVIN ROSS MARCH 31, 2005

LECTURE: INTRO TO LINEAR PROGRAMMING AND THE SIMPLEX METHOD, KEVIN ROSS MARCH 31, 2005 DAVID L. BERNICK dbernick@soe.ucsc.edu 1. Overview Typical Linear Programming problems Standard form and converting

### IEOR 4404 Homework #2 Intro OR: Deterministic Models February 14, 2011 Prof. Jay Sethuraman Page 1 of 5. Homework #2

IEOR 4404 Homework # Intro OR: Deterministic Models February 14, 011 Prof. Jay Sethuraman Page 1 of 5 Homework #.1 (a) What is the optimal solution of this problem? Let us consider that x 1, x and x 3

### max cx s.t. Ax c where the matrix A, cost vector c and right hand side b are given and x is a vector of variables. For this example we have x

Linear Programming Linear programming refers to problems stated as maximization or minimization of a linear function subject to constraints that are linear equalities and inequalities. Although the study

### Nonlinear Programming Methods.S2 Quadratic Programming

Nonlinear Programming Methods.S2 Quadratic Programming Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard A linearly constrained optimization problem with a quadratic objective

### Introduction to Linear Programming.

Chapter 1 Introduction to Linear Programming. This chapter introduces notations, terminologies and formulations of linear programming. Examples will be given to show how real-life problems can be modeled

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### Stanford University CS261: Optimization Handout 6 Luca Trevisan January 20, In which we introduce the theory of duality in linear programming.

Stanford University CS261: Optimization Handout 6 Luca Trevisan January 20, 2011 Lecture 6 In which we introduce the theory of duality in linear programming 1 The Dual of Linear Program Suppose that we

### Module 3 Lecture Notes 2. Graphical Method

Optimization Methods: Linear Programming- Graphical Method Module Lecture Notes Graphical Method Graphical method to solve Linear Programming problem (LPP) helps to visualize the procedure explicitly.

### Lecture 1: Linear Programming Models. Readings: Chapter 1; Chapter 2, Sections 1&2

Lecture 1: Linear Programming Models Readings: Chapter 1; Chapter 2, Sections 1&2 1 Optimization Problems Managers, planners, scientists, etc., are repeatedly faced with complex and dynamic systems which

### 1 Linear Programming. 1.1 Introduction. Problem description: motivate by min-cost flow. bit of history. everything is LP. NP and conp. P breakthrough.

1 Linear Programming 1.1 Introduction Problem description: motivate by min-cost flow bit of history everything is LP NP and conp. P breakthrough. general form: variables constraints: linear equalities

### Mathematics Notes for Class 12 chapter 12. Linear Programming

1 P a g e Mathematics Notes for Class 12 chapter 12. Linear Programming Linear Programming It is an important optimization (maximization or minimization) technique used in decision making is business and

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### Tutorial on Convex Optimization for Engineers Part I

Tutorial on Convex Optimization for Engineers Part I M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de

### Chapter 5. Linear Inequalities and Linear Programming. Linear Programming in Two Dimensions: A Geometric Approach

Chapter 5 Linear Programming in Two Dimensions: A Geometric Approach Linear Inequalities and Linear Programming Section 3 Linear Programming gin Two Dimensions: A Geometric Approach In this section, we

### Equilibrium computation: Part 1

Equilibrium computation: Part 1 Nicola Gatti 1 Troels Bjerre Sorensen 2 1 Politecnico di Milano, Italy 2 Duke University, USA Nicola Gatti and Troels Bjerre Sørensen ( Politecnico di Milano, Italy, Equilibrium

### Chapter 1 Linear Programming. Paragraph 2 Developing Ideas for an Algorithm solving Linear Optimization

Chapter 1 Linear Programming Paragraph 2 Developing Ideas for an Algorithm solving Linear Optimization What we did so far How to model an optimization problem Data, Variables, Constraints, Objective We

### Minkowski Sum of Polytopes Defined by Their Vertices

Minkowski Sum of Polytopes Defined by Their Vertices Vincent Delos, Denis Teissandier To cite this version: Vincent Delos, Denis Teissandier. Minkowski Sum of Polytopes Defined by Their Vertices. Journal

### CHAPTER 9. Integer Programming

CHAPTER 9 Integer Programming An integer linear program (ILP) is, by definition, a linear program with the additional constraint that all variables take integer values: (9.1) max c T x s t Ax b and x integral

### A counter-example to the Hirsch conjecture

A counter-example to the Hirsch conjecture Francisco Santos Universidad de Cantabria, Spain http://personales.unican.es/santosf/hirsch Bernoulli Conference on Discrete and Computational Geometry Lausanne,

### Symmetry Handling in Binary Programs via Polyhedral Methods

Symmetry Handling in Binary Programs via Polyhedral Methods Christopher Hojny joint work with Marc Pfetsch Technische Universität Darmstadt Department of Mathematics 26 International Conference on Mathematical

### Permutation Betting Markets: Singleton Betting with Extra Information

Permutation Betting Markets: Singleton Betting with Extra Information Mohammad Ghodsi Sharif University of Technology ghodsi@sharif.edu Hamid Mahini Sharif University of Technology mahini@ce.sharif.edu

### Discrete Optimization

Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using

### This exposition of linear programming

Linear Programming and the Simplex Method David Gale This exposition of linear programming and the simplex method is intended as a companion piece to the article in this issue on the life and work of George

### Inverse Optimization by James Orlin

Inverse Optimization by James Orlin based on research that is joint with Ravi Ahuja Jeopardy 000 -- the Math Programming Edition The category is linear objective functions The answer: When you maximize

### Discuss the size of the instance for the minimum spanning tree problem.

3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can

### Chapter 6. Linear Programming: The Simplex Method. Introduction to the Big M Method. Section 4 Maximization and Minimization with Problem Constraints

Chapter 6 Linear Programming: The Simplex Method Introduction to the Big M Method In this section, we will present a generalized version of the simplex method that t will solve both maximization i and

### 7.4 Linear Programming: The Simplex Method

7.4 Linear Programming: The Simplex Method For linear programming problems with more than two variables, the graphical method is usually impossible, so the simplex method is used. Because the simplex method

### Lecture notes 1: Introduction to linear and (mixed) integer programs

Lecture notes 1: Introduction to linear and (mixed) integer programs Vincent Conitzer 1 An example We will start with a simple example. Suppose we are in the business of selling reproductions of two different

### Solving Curved Linear Programs via the Shadow Simplex Method

Solving Curved Linear rograms via the Shadow Simplex Method Daniel Dadush 1 Nicolai Hähnle 2 1 Centrum Wiskunde & Informatica (CWI) 2 Bonn Universität CWI Scientific Meeting 01/15 Outline 1 Introduction

### Chapter 3 LINEAR PROGRAMMING GRAPHICAL SOLUTION 3.1 SOLUTION METHODS 3.2 TERMINOLOGY

Chapter 3 LINEAR PROGRAMMING GRAPHICAL SOLUTION 3.1 SOLUTION METHODS Once the problem is formulated by setting appropriate objective function and constraints, the next step is to solve it. Solving LPP

### Some representability and duality results for convex mixed-integer programs.

Some representability and duality results for convex mixed-integer programs. Santanu S. Dey Joint work with Diego Morán and Juan Pablo Vielma December 17, 2012. Introduction About Motivation Mixed integer

### Linear Programming Problems

Linear Programming Problems Linear programming problems come up in many applications. In a linear programming problem, we have a function, called the objective function, which depends linearly on a number

### Unit 1. Today I am going to discuss about Transportation problem. First question that comes in our mind is what is a transportation problem?

Unit 1 Lesson 14: Transportation Models Learning Objective : What is a Transportation Problem? How can we convert a transportation problem into a linear programming problem? How to form a Transportation

### Lecture 4 Linear Programming Models: Standard Form. August 31, 2009

Linear Programming Models: Standard Form August 31, 2009 Outline: Lecture 4 Standard form LP Transforming the LP problem to standard form Basic solutions of standard LP problem Operations Research Methods

### Assignment Problems. Guoming Tang

Assignment Problems Guoming Tang A Practical Problem Workers: A, B, C, D Jobs: P, Q, R, S. Cost matrix: Job P Job Q Job R Job S Worker A 1 2 3 4 Worker B 2 4 6 8 Worker C 3 6 9 12 Worker D 4 8 12 16 Given:

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### LINEAR PROGRAMMING MODULE Part 2 - Solution Algorithm

LINEAR PROGRAMMING MODULE Part 2 - Solution Algorithm Name: THE SIMPLEX SOLUTION METHOD It should be obvious to the reader by now that the graphical method for solving linear programs is limited to models

### The multi-integer set cover and the facility terminal cover problem

The multi-integer set cover and the facility teral cover problem Dorit S. Hochbaum Asaf Levin December 5, 2007 Abstract The facility teral cover problem is a generalization of the vertex cover problem.

### An Exact Algorithm for Steiner Tree Problem on Graphs

International Journal of Computers, Communications & Control Vol. I (2006), No. 1, pp. 41-46 An Exact Algorithm for Steiner Tree Problem on Graphs Milan Stanojević, Mirko Vujošević Abstract: The paper

### AM 221: Advanced Optimization Spring Prof. Yaron Singer Lecture 7 February 19th, 2014

AM 22: Advanced Optimization Spring 204 Prof Yaron Singer Lecture 7 February 9th, 204 Overview In our previous lecture we saw the application of the strong duality theorem to game theory, and then saw

### Actually Doing It! 6. Prove that the regular unit cube (say 1cm=unit) of sufficiently high dimension can fit inside it the whole city of New York.

1: 1. Compute a random 4-dimensional polytope P as the convex hull of 10 random points using rand sphere(4,10). Run VISUAL to see a Schlegel diagram. How many 3-dimensional polytopes do you see? How many

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### NP-Completeness. CptS 223 Advanced Data Structures. Larry Holder School of Electrical Engineering and Computer Science Washington State University

NP-Completeness CptS 223 Advanced Data Structures Larry Holder School of Electrical Engineering and Computer Science Washington State University 1 Hard Graph Problems Hard means no known solutions with

### 1 Lagrangian Methods. 1.2 The Dual Problem. 1.1 Lagrangian methods. 1.3 Strong Lagrangian. 1.4 Hyperplanes. The Dual Problem 2.

The Dual Problem Lagrangian Methods. Lagrangian methods Let P(b) denote the optimization problem minimize f(x), subject to h(x) = b, x X. Let x X(b) = {x X : h(x) = b}. We say that x is feasible if x X(b).

### Notes on NP Completeness

Notes on NP Completeness Rich Schwartz November 10, 2013 1 Overview Here are some notes which I wrote to try to understand what NP completeness means. Most of these notes are taken from Appendix B in Douglas

### 18. Linear optimization

18. Linear optimization EE103 (Fall 2011-12) linear program examples geometrical interpretation extreme points simplex method 18-1 Linear program minimize c 1 x 1 +c 2 x 2 + +c n x n subject to a 11 x

### Facility Location: Discrete Models and Local Search Methods

Facility Location: Discrete Models and Local Search Methods Yury KOCHETOV Sobolev Institute of Mathematics, Novosibirsk, Russia Abstract. Discrete location theory is one of the most dynamic areas of operations