# 26 Linear Programming

Save this PDF as:

Size: px
Start display at page:

## Transcription

2 Here, the input consists of a matrix A = (a i j ) n d, a column vector b n, and a row vector c d. Each coordinate of the vector x is called a variable. Each of the linear inequalities is called a constraint. The function x c x is called the objective function. I will always use d to denote the number of variables, also known as the dimension of the problem. The number of constraints is usually denoted n. A linear programming problem is said to be in canonical form¹ if it has the following structure: maximize c j x j subject to a i j x j b i x j 0 for each i = 1.. n for each j = 1.. d We can express this canonical form more compactly as follows. For two vectors x = (x 1, x 2,..., x d ) and y = (y 1, y 2,..., y d ), the expression x y means that x i y i for every index i. max c x s.t. Ax b x 0 Any linear programming problem can be converted into canonical form as follows: For each variable x j, add the equality constraint x j = x + j x j and the inequalities x + j 0 and x j 0. Replace any equality constraint j a i j x j = b i with two inequality constraints j a i j x j b i and j a i j x j b i. Replace any upper bound j a i j x j b i with the equivalent lower bound j a i j x j b i. This conversion potentially double the number of variables and the number of constraints; fortunately, it is rarely necessary in practice. Another useful format for linear programming problems is slack form², in which every inequality is of the form x j 0: max c x s.t. Ax = b x 0 It s fairly easy to convert any linear programming problem into slack form. Slack form is especially useful in executing the simplex algorithm (which we ll see in the next lecture) The Geometry of Linear Programming A point x d is feasible with respect to some linear programming problem if it satisfies all the linear constraints. The set of all feasible points is called the feasible region for that linear program. ¹Confusingly, some authors call this standard form. ²Confusingly, some authors call this standard form. 2

3 The feasible region has a particularly nice geometric structure that lends some useful intuition to the linear programming algorithms we ll see later. Any linear equation in d variables defines a hyperplane in d ; think of a line when d = 2, or a plane when d = 3. This hyperplane divides d into two halfspaces; each halfspace is the set of points that satisfy some linear inequality. Thus, the set of feasible points is the intersection of several hyperplanes (one for each equality constraint) and halfspaces (one for each inequality constraint). The intersection of a finite number of hyperplanes and halfspaces is called a polyhedron. It s not hard to verify that any halfspace, and therefore any polyhedron, is convex if a polyhedron contains two points x and y, then it contains the entire line segment x y. A two-dimensional polyhedron (white) defined by 10 linear inequalities. By rotating d (or choosing a coordinate frame) so that the objective function points downward, we can express any linear programming problem in the following geometric form: Find the lowest point in a given polyhedron. With this geometry in hand, we can easily picture two pathological cases where a given linear programming problem has no solution. The first possibility is that there are no feasible points; in this case the problem is called infeasible. For example, the following LP problem is infeasible: maximize x y subject to 2x + y 1 x + y 2 x, y 0 An infeasible linear programming problem; arrows indicate the constraints. The second possibility is that there are feasible points at which the objective function is arbitrarily large; in this case, we call the problem unbounded. The same polyhedron could be unbounded for some objective functions but not others, or it could be unbounded for every objective function. 3

4 A two-dimensional polyhedron (white) that is unbounded downward but bounded upward Example 1: Shortest Paths We can compute the length of the shortest path from s to t in a weighted directed graph by solving the following very simple linear programming problem. maximize d t subject to d s = 0 d v d u l u v Here, l u v is the length of the edge u v. Each variable d v represents a tentative shortest-path distance from s to v. The constraints mirror the requirement that every edge in the graph must be relaxed. These relaxation constraints imply that in any feasible solution, d v is at most the shortest path distance from s to v. Thus, somewhat counterintuitively, we are correctly maximizing the objective function to compute the shortest path! In the optimal solution, the objective function d t is the actual shortest-path distance from s to t, but for any vertex v that is not on the shortest path from s to t, d v may be an underestimate of the true distance from s to v. However, we can obtain the true distances from s to every other vertex by modifying the objective function: maximize v d v subject to d s = 0 d v d u l u v There is another formulation of shortest paths as an LP minimization problem using an indicator variable x u v for each edge u v. minimize l u v x u v u v subject to x u s x s w = 1 u w x u t x t w = 1 u x u v x v w = 0 u w w x u v 0 for every vertex v s, t Intuitively, x u v = 1 means u v lies on the shortest path from s to t, and x u v = 0 means u v does not lie on this shortest path. The constraints merely state that the path should start at s, end at t, and either pass through or avoid every other vertex v. Any path from s to t in particular, the shortest path clearly implies a feasible point for this linear program. 4

5 However, there are other feasible solutions, possibly even optimal solutions, with non-integral values that do not represent paths. Nevertheless, there is always an optimal solution in which every x e is either 0 or 1 and the edges e with x e = 1 comprise the shortest path. (This fact is by no means obvious, but a proof is beyond the scope of these notes.) Moreover, in any optimal solution, even if not every x e is an integer, the objective function gives the shortest path distance! 26.3 Example 2: Maximum Flows and Minimum Cuts Recall that the input to the maximum (s, t)-flow problem consists of a weighted directed graph G = (V, E), two special vertices s and t, and a function assigning a non-negative capacity c e to each edge e. Our task is to choose the flow f e across each edge e, as follows: maximize f s w subject to w u f u s f v w f u v = 0 w u f u v c u v f u v 0 for every vertex v s, t Similarly, the minimum cut problem can be formulated using indicator variables similarly to the shortest path problem. We have a variable S v for each vertex v, indicating whether v S or v T, and a variable X u v for each edge u v, indicating whether u S and v T, where (S, T) is some (s, t)-cut.³ minimize c u v X u v u v subject to X u v + S v S u 0 X u v 0 S s = 1 S t = 0 Like the minimization LP for shortest paths, there can be optimal solutions that assign fractional values to the variables. Nevertheless, the minimum value for the objective function is the cost of the minimum cut, and there is an optimal solution for which every variable is either 0 or 1, representing an actual minimum cut. No, this is not obvious; in particular, my claim is not a proof! 26.4 Linear Programming Duality Each of these pairs of linear programming problems is related by a transformation called duality. For any linear programming problem, there is a corresponding dual linear program that can be obtained by a mechanical translation, essentially by swapping the constraints and the variables. ³These two linear programs are not quite syntactic duals; I ve added two redundant variables S s and S t to the min-cut program to increase readability. 5

6 The translation is simplest when the LP is in canonical form: Primal (Π) max c x s.t. Ax b x 0 Dual ( ) min y b s.t. ya c y 0 We can also write the dual linear program in exactly the same canonical form as the primal, by swapping the coefficient vector c and the objective vector b, negating both vectors, and replacing the constraint matrix A with its negative transpose.⁴ Primal (Π) max c x s.t. Ax b x 0 Dual ( ) max b y s.t. A y c y 0 Written in this form, it should be immediately clear that duality is an involution: The dual of the dual linear program is identical to the primal linear program Π. The choice of which LP to call the primal and which to call the dual is totally arbitrary.⁵ The Fundamental Theorem of Linear Programming. A linear program Π has an optimal solution x if and only if the dual linear program has an optimal solution y such that c x = y Ax = y b. The weak form of this theorem is trivial to prove. Weak Duality Theorem. If x is a feasible solution for a canonical linear program Π and y is a feasible solution for its dual, then c x yax y b. Proof: Because x is feasible for Π, we have Ax b. Since y is positive, we can multiply both sides of the inequality to obtain yax y b. Conversely, y is feasible for and x is positive, so yax c x. It immediately follows that if c x = y b, then x and y are optimal solutions to their respective linear programs. This is in fact a fairly common way to prove that we have the optimal value for a linear program. ⁴For the notational purists: In these formulations, x and b are column vectors, and y and c are row vectors. This is a somewhat nonstandard choice. Yes, that means the dot in c x is redundant. Sue me. ⁵For historical reasons, maximization LPs tend to be called primal and minimization LPs tend to be called dual. This is a pointless religious tradition, nothing more. Duality is a relationship between LP problems, not a type of LP problem. 6

7 26.5 Duality Example Before I prove the stronger duality theorem, let me first provide some intuition about where this duality thing comes from in the first place.⁶ Consider the following linear programming problem: maximize 4x 1 + x 2 + 3x 3 subject to x 1 + 4x 2 2 3x 1 x 2 + x 3 4 x 1, x 2, x 3 0 Let σ denote the optimum objective value for this LP. The feasible solution x = (1, 0, 0) gives us a lower bound σ 4. A different feasible solution x = (0, 0, 3) gives us a better lower bound σ 9. We could play this game all day, finding different feasible solutions and getting ever larger lower bounds. How do we know when we re done? Is there a way to prove an upper bound on σ? In fact, there is. Let s multiply each of the constraints in our LP by a new non-negative scalar value y i : maximize 4x 1 + x 2 + 3x 3 subject to y 1 ( x 1 + 4x 2 ) 2 y 1 y 2 (3x 1 x 2 + x 3 ) 4 y 2 x 1, x 2, x 3 0 Because each y i is non-negative, we do not reverse any of the inequalities. Any feasible solution (x 1, x 2, x 3 ) must satisfy both of these inequalities, so it must also satisfy their sum: (y 1 + 3y 2 )x 1 + (4 y 1 y 2 )x 2 + y 2 x 3 2 y y 2. Now suppose that each y i is larger than the ith coefficient of the objective function: y y 2 4, 4y 1 y 2 1, y 2 3. This assumption lets us derive an upper bound on the objective value of any feasible solution: 4x 1 + x 2 + 3x 3 (y y 2 )x 1 + (4 y 1 y 2 )x 2 + y 2 x 3 2 y y 2. ( ) In particular, by plugging in the optimal solution (x 1, x 2, x 3 ) for the original LP, we obtain the following upper bound on σ : σ = 4x 1 + x 2 + 3x 3 2 y y 2. Now it s natural to ask how tight we can make this upper bound. How small can we make the expression 2 y y 2 without violating any of the inequalities we used to prove the upper bound? This is just another linear programming problem. minimize 2y y 2 subject to y y y 1 y 2 1 y 2 3 y 1, y 2 0 ⁶This example is taken from Robert Vanderbei s excellent textbook Linear Programming: Foundations and Extensions [Springer, 2001], but the idea appears earlier in Jens Clausen s 1997 paper Teaching Duality in Linear Programming: The Multiplier Approach. 7

8 In fact, this is precisely the dual of our original linear program! Moreover, inequality ( ) is just an instantiation of the Weak Duality Theorem Strong Duality The Fundamental Theorem can be rephrased in the following form: Strong Duality Theorem. If x is an optimal solution for a canonical linear program Π, then there is an optimal solution y for its dual, such that c x = y Ax = y b. Proof (sketch): I ll prove the theorem only for non-degenerate linear programs, in which (a) the optimal solution (if one exists) is a unique vertex of the feasible region, and (b) at most d constraint hyperplanes pass through any point. These non-degeneracy assumptions are relatively easy to enforce in practice and can be removed from the proof at the expense of some technical detail. I will also prove the theorem only for the case n d; the argument for under-constrained LPs is similar (if not simpler). To develop some intuition, let s first consider the very special case where x = (0, 0,..., 0). Let e i denote the ith standard basis vector, whose ith coordinate is 1 and all other coordinates are 0. Because x i = 0 for all i, our non-degeneracy assumption implies the strict inequality a i x < b i for all i. Thus, any sufficiently small ball around the origin does not intersect any other constraint hyperplane a i x = b i. Thus, for all i, and for any sufficiently small δ > 0, the vector δe i is feasible. Because x is the unique optimum, we must have δc i = c (δe i ) < c x = 0. We conclude that c i < 0 for all i. Now let y = (0, 0,..., 0) as well. We immediately observe that ya c and y 0; in other words, y is a feasible solution for the dual linear program. But y b = 0 = c x, so the weak duality theorem implies that y is an optimal solution to, and the proof is complete for this very special case. Now let us consider the more general case. Let x be the optimal solution for the linear program Π; our non-degeneracy assumption implies that this solution is unique, and that exactly d of the n linear constraints are satisfied with equality. Without loss of generality (by permuting the constraints and possibly changing coordinates), we can assume that these are the first d constraints. Thus, we have a i x = b i for all i d, a i x < b i for all i d + 1, where a i denotes the ith row of A. Let A denote the d d matrix containing the first d rows of A. Our non-degeneracy assumption implies that A has full rank, and thus has a well-defined inverse V = A 1. Now define a vector y n by setting y j := c v j for all j d, y j := 0 for all j d + 1, where v j denotes the jth column of V = A 1. Note that a i v j = 0 if i j, and a i v j = 1 if i = j. To simplify notation, let y = (y 1, y 2,..., y d ) and let b = (b 1, b 2,..., b d ) = A x. Because y i = 0 for all i d + 1, we immediately have y b = y b = cv b = ca 1 b = c x 8

9 and ya = y A = cva = ca 1 A = c. The point x lies on exactly d constraint hyperplanes; moreover, any sufficiently small ball around x intersects only those d constraint hyperplanes. Consider the point x = x ɛv j, for some index 1 j d and some sufficiently small ɛ > 0. We have a i x = a i x ɛ(a i v j ) = b i for all i j, and a j x = a j x ɛ(a j v j ) = b j ɛ < b j. Thus, x is a feasible point for Π. Because x is the unique optimum for Π, we must have c x = c x ɛ(c v j ) < c x. We conclude that y j = c v j > 0 for all j. We have shown that ya c and y 0, so y is a feasible solution for the dual linear program. We have also shown that y b = c x, so by the Weak Duality Theorem, y is also an optimal solution for, and the proof is complete! We can also give a useful geometric interpretation to the vector y d. Each linear equation a i x = b i defines a hyperplane in d with normal vector a i. The normal vectors a 1,..., a d are linearly independent (by non-degeneracy) and therefore describe a coordinate frame for the vector space d. The definition of y implies that c = y A = d i=1 y ia i. In other words, y lists the coefficients of the objective vector c in the coordinate frame a 1,..., a d Complementary Slackness Complementary Slackness Theorem. Let x be an optimal solution to a canonical linear program Π, and let y be an optimal solution to its dual. Then for every index i, we have y i > 0 if and only if a i x = b i. Symmetrically, for every index j, we have x j > 0 if and only if y a j = c j. To be written Exercises 1. (a) Describe how to transform any linear program written in general form into an equivalent linear program written in slack form. maximize subject to c j x j a i j x j b i a i j x j = b i a i j x j b i for each i = 1.. p for each i = p p + q for each i = p + q n = max c x s.t. Ax = b x 0 (b) Describe precisely how to dualize a linear program written in slack form. (c) Describe precisely how to dualize a linear program written in general form: In all cases, keep the number of variables in the resulting linear program as small as possible. 9

10 2. A matrix A = (a i j ) is skew-symmetric if and only if a ji = a i j for all indices i j; in particular, every skew-symmetric matrix is square. A canonical linear program max{c x Ax b; x 0} is self-dual if the matrix A is skew-symmetric and the objective vector c is equal to the constraint vector b. (a) Prove that any self-dual linear program Π is syntactically equivalent to its dual program. (b) Show that any linear program Π with d variables and n constraints can be transformed into a self-dual linear program with n + d variables and n + d constraints. The optimal solution to the self-dual program should include both the optimal solution for Π (in d of the variables) and the optimal solution for the dual program (in the other n variables). 3. (a) Give a linear-programming formulation of the maximum-cardinality bipartite matching problem. The input is a bipartite graph G = (U V ; E), where E U V ; the output is the largest matching in G. Your linear program should have one variable for each edge. (b) Now dualize the linear program from part (a). What do the dual variables represent? What does the objective function represent? What problem is this!? 4. Give a linear-programming formulation of the minimum-cost feasible circulation problem. Here you are given a flow network whose edges have both capacities and costs, and your goal is to find a feasible circulation (flow with value 0) whose cost is as small as possible. 5. An integer program is a linear program with the additional constraint that the variables must take only integer values. (a) Prove that deciding whether an integer program has a feasible solution is NP-complete. (b) Prove that finding the optimal feasible solution to an integer program is NP-hard. [Hint: Almost any NP-hard decision problem can be formulated as an integer program. Pick your favorite.] 6. Helly s theorem states that for any collection of convex bodies in d, if every d + 1 of them intersect, then there is a point lying in the intersection of all of them. Prove Helly s theorem for the special case where the convex bodies are halfspaces. Equivalently, show that if a system of linear inequalities Ax b does not have a solution, then we can select d + 1 of the inequalities such that the resulting subsystem also does not have a solution. [Hint: Construct a dual LP from the system by choosing a 0 cost vector.] 7. Given points (x 1, y 1 ), (x 2, y 2 ),..., (x n, y n ) in the plane, the linear regression problem asks for real numbers a and b such that the line y = ax + b fits the points as closely as possible, according to some criterion. The most common fit criterion is minimizing the L 2 error, 10

11 defined as follows:⁷ ɛ 2 (a, b) = n (y i ax i b) 2. i=1 But there are several other fit criteria, some of which can be optimized via linear programming. (a) The L 1 error (or total absolute deviation) of the line y = ax + b is defined as follows: ɛ 1 (a, b) = n y i ax i b. i=1 Describe a linear program whose solution (a, b) describes the line with minimum L 1 error. (b) The L error (or maximum absolute deviation) of the line y = ax + b is defined as follows: ɛ (a, b) = max n y i ax i b. i=1 Describe a linear program whose solution (a, b) describes the line with minimum L error. ⁷This measure is also known as sum of squared residuals, and the algorithm to compute the best fit is normally called (ordinary/linear) least squares fitting. Copyright 2014 Jeff Erickson. This work is licensed under a Creative Commons License ( Free distribution is strongly encouraged; commercial distribution is expressly forbidden. See for the most recent revision. 11

### max cx s.t. Ax c where the matrix A, cost vector c and right hand side b are given and x is a vector of variables. For this example we have x

Linear Programming Linear programming refers to problems stated as maximization or minimization of a linear function subject to constraints that are linear equalities and inequalities. Although the study

### Lecture 7 - Linear Programming

COMPSCI 530: Design and Analysis of Algorithms DATE: 09/17/2013 Lecturer: Debmalya Panigrahi Lecture 7 - Linear Programming Scribe: Hieu Bui 1 Overview In this lecture, we cover basics of linear programming,

### 1 Polyhedra and Linear Programming

CS 598CSC: Combinatorial Optimization Lecture date: January 21, 2009 Instructor: Chandra Chekuri Scribe: Sungjin Im 1 Polyhedra and Linear Programming In this lecture, we will cover some basic material

### Chapter 15 Introduction to Linear Programming

Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of

### Linear Programming I

Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins

### I. Solving Linear Programs by the Simplex Method

Optimization Methods Draft of August 26, 2005 I. Solving Linear Programs by the Simplex Method Robert Fourer Department of Industrial Engineering and Management Sciences Northwestern University Evanston,

### Geometry of Linear Programming

Chapter 2 Geometry of Linear Programming The intent of this chapter is to provide a geometric interpretation of linear programming problems. To conceive fundamental concepts and validity of different algorithms

### By W.E. Diewert. July, Linear programming problems are important for a number of reasons:

APPLIED ECONOMICS By W.E. Diewert. July, 3. Chapter : Linear Programming. Introduction The theory of linear programming provides a good introduction to the study of constrained maximization (and minimization)

### LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS. 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method

LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method Introduction to dual linear program Given a constraint matrix A, right

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### 4.6 Linear Programming duality

4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal

### Quiz 1 Sample Questions IE406 Introduction to Mathematical Programming Dr. Ralphs

Quiz 1 Sample Questions IE406 Introduction to Mathematical Programming Dr. Ralphs These questions are from previous years and should you give you some idea of what to expect on Quiz 1. 1. Consider the

### THEORY OF SIMPLEX METHOD

Chapter THEORY OF SIMPLEX METHOD Mathematical Programming Problems A mathematical programming problem is an optimization problem of finding the values of the unknown variables x, x,, x n that maximize

### Linear Programming. Non-Lecture J: Linear Programming

The greatest flood has the soonest ebb; the sorest tempest the most sdden calm; the hottest love the coldest end; and from the deepest desire oftentimes enses the deadliest hate. Socrates Th extremes of

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### Week 5 Integral Polyhedra

Week 5 Integral Polyhedra We have seen some examples 1 of linear programming formulation that are integral, meaning that every basic feasible solution is an integral vector. This week we develop a theory

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### Min-cost flow problems and network simplex algorithm

Min-cost flow problems and network simplex algorithm The particular structure of some LP problems can be sometimes used for the design of solution techniques more efficient than the simplex algorithm.

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### 1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

### The Simplex Method. yyye

Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note #05 1 The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/

### What is Linear Programming?

Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to

### Minimize subject to. x S R

Chapter 12 Lagrangian Relaxation This chapter is mostly inspired by Chapter 16 of [1]. In the previous chapters, we have succeeded to find efficient algorithms to solve several important problems such

### Linear Programming for Optimization. Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc.

1. Introduction Linear Programming for Optimization Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc. 1.1 Definition Linear programming is the name of a branch of applied mathematics that

### Good luck, veel succes!

Final exam Advanced Linear Programming, May 7, 13.00-16.00 Switch off your mobile phone, PDA and any other mobile device and put it far away. No books or other reading materials are allowed. This exam

### Stanford University CS261: Optimization Handout 6 Luca Trevisan January 20, In which we introduce the theory of duality in linear programming.

Stanford University CS261: Optimization Handout 6 Luca Trevisan January 20, 2011 Lecture 6 In which we introduce the theory of duality in linear programming 1 The Dual of Linear Program Suppose that we

### AM 221: Advanced Optimization Spring Prof. Yaron Singer Lecture 7 February 19th, 2014

AM 22: Advanced Optimization Spring 204 Prof Yaron Singer Lecture 7 February 9th, 204 Overview In our previous lecture we saw the application of the strong duality theorem to game theory, and then saw

### Linear Programming, Lagrange Multipliers, and Duality Geoff Gordon

lp.nb 1 Linear Programming, Lagrange Multipliers, and Duality Geoff Gordon lp.nb 2 Overview This is a tutorial about some interesting math and geometry connected with constrained optimization. It is not

### . P. 4.3 Basic feasible solutions and vertices of polyhedra. x 1. x 2

4. Basic feasible solutions and vertices of polyhedra Due to the fundamental theorem of Linear Programming, to solve any LP it suffices to consider the vertices (finitely many) of the polyhedron P of the

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### Duality in Linear Programming

Duality in Linear Programming 4 In the preceding chapter on sensitivity analysis, we saw that the shadow-price interpretation of the optimal simplex multipliers is a very useful concept. First, these shadow

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### Arrangements And Duality

Arrangements And Duality 3.1 Introduction 3 Point configurations are tbe most basic structure we study in computational geometry. But what about configurations of more complicated shapes? For example,

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Theory of Linear Programming

Theory of Linear Programming Debasis Mishra April 6, 2011 1 Introduction Optimization of a function f over a set S involves finding the maximum (minimum) value of f (objective function) in the set S (feasible

### Chapter 3 LINEAR PROGRAMMING GRAPHICAL SOLUTION 3.1 SOLUTION METHODS 3.2 TERMINOLOGY

Chapter 3 LINEAR PROGRAMMING GRAPHICAL SOLUTION 3.1 SOLUTION METHODS Once the problem is formulated by setting appropriate objective function and constraints, the next step is to solve it. Solving LPP

### Linear Programming Problems

Linear Programming Problems Linear programming problems come up in many applications. In a linear programming problem, we have a function, called the objective function, which depends linearly on a number

### Linear Programming: Chapter 5 Duality

Linear Programming: Chapter 5 Duality Robert J. Vanderbei October 17, 2007 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/ rvdb Resource

### We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.

Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to

### Linear Programming. Widget Factory Example. Linear Programming: Standard Form. Widget Factory Example: Continued.

Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.

### Lecture 1: Linear Programming Models. Readings: Chapter 1; Chapter 2, Sections 1&2

Lecture 1: Linear Programming Models Readings: Chapter 1; Chapter 2, Sections 1&2 1 Optimization Problems Managers, planners, scientists, etc., are repeatedly faced with complex and dynamic systems which

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### LINEAR PROGRAMMING P V Ram B. Sc., ACA, ACMA Hyderabad

LINEAR PROGRAMMING P V Ram B. Sc., ACA, ACMA 98481 85073 Hyderabad Page 1 of 19 Question: Explain LPP. Answer: Linear programming is a mathematical technique for determining the optimal allocation of resources

### Introduction to Linear Programming.

Chapter 1 Introduction to Linear Programming. This chapter introduces notations, terminologies and formulations of linear programming. Examples will be given to show how real-life problems can be modeled

### Lattice Point Geometry: Pick s Theorem and Minkowski s Theorem. Senior Exercise in Mathematics. Jennifer Garbett Kenyon College

Lattice Point Geometry: Pick s Theorem and Minkowski s Theorem Senior Exercise in Mathematics Jennifer Garbett Kenyon College November 18, 010 Contents 1 Introduction 1 Primitive Lattice Triangles 5.1

### Linear Inequalities and Linear Programming. Systems of Linear Inequalities in Two Variables

Linear Inequalities and Linear Programming 5.1 Systems of Linear Inequalities 5.2 Linear Programming Geometric Approach 5.3 Geometric Introduction to Simplex Method 5.4 Maximization with constraints 5.5

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### Using the Simplex Method in Mixed Integer Linear Programming

Integer Using the Simplex Method in Mixed Integer UTFSM Nancy, 17 december 2015 Using the Simplex Method in Mixed Integer Outline Mathematical Programming Integer 1 Mathematical Programming Optimisation

### Section Notes 3. The Simplex Algorithm. Applied Math 121. Week of February 14, 2011

Section Notes 3 The Simplex Algorithm Applied Math 121 Week of February 14, 2011 Goals for the week understand how to get from an LP to a simplex tableau. be familiar with reduced costs, optimal solutions,

### The Simplex Method. What happens when we need more decision variables and more problem constraints?

The Simplex Method The geometric method of solving linear programming problems presented before. The graphical method is useful only for problems involving two decision variables and relatively few problem

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Sensitivity Analysis 3.1 AN EXAMPLE FOR ANALYSIS

Sensitivity Analysis 3 We have already been introduced to sensitivity analysis in Chapter via the geometry of a simple example. We saw that the values of the decision variables and those of the slack and

### Chap 4 The Simplex Method

The Essence of the Simplex Method Recall the Wyndor problem Max Z = 3x 1 + 5x 2 S.T. x 1 4 2x 2 12 3x 1 + 2x 2 18 x 1, x 2 0 Chap 4 The Simplex Method 8 corner point solutions. 5 out of them are CPF solutions.

### Linear Programming: Theory and Applications

Linear Programming: Theory and Applications Catherine Lewis May 11, 2008 1 Contents 1 Introduction to Linear Programming 3 1.1 What is a linear program?...................... 3 1.2 Assumptions.............................

### 3 Does the Simplex Algorithm Work?

Does the Simplex Algorithm Work? In this section we carefully examine the simplex algorithm introduced in the previous chapter. Our goal is to either prove that it works, or to determine those circumstances

### Lecture 2: August 29. Linear Programming (part I)

10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### Math 018 Review Sheet v.3

Math 018 Review Sheet v.3 Tyrone Crisp Spring 007 1.1 - Slopes and Equations of Lines Slopes: Find slopes of lines using the slope formula m y y 1 x x 1. Positive slope the line slopes up to the right.

### CHAPTER 17. Linear Programming: Simplex Method

CHAPTER 17 Linear Programming: Simplex Method CONTENTS 17.1 AN ALGEBRAIC OVERVIEW OF THE SIMPLEX METHOD Algebraic Properties of the Simplex Method Determining a Basic Solution Basic Feasible Solution 17.2

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### The Graphical Simplex Method: An Example

The Graphical Simplex Method: An Example Consider the following linear program: Max 4x 1 +3x Subject to: x 1 +3x 6 (1) 3x 1 +x 3 () x 5 (3) x 1 +x 4 (4) x 1, x 0. Goal: produce a pair of x 1 and x that

### Solving Linear Programs

Solving Linear Programs 2 In this chapter, we present a systematic procedure for solving linear programs. This procedure, called the simplex method, proceeds by moving from one feasible solution to another,

### OPRE 6201 : 2. Simplex Method

OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2

### The Canonical and Dictionary Forms of Linear Programs

The Canonical and Dictionary Forms of Linear Programs Recall the general optimization problem: Minimize f(x) subect to x D (1) The real valued function f is called the obective function, and D the constraint

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Actually Doing It! 6. Prove that the regular unit cube (say 1cm=unit) of sufficiently high dimension can fit inside it the whole city of New York.

1: 1. Compute a random 4-dimensional polytope P as the convex hull of 10 random points using rand sphere(4,10). Run VISUAL to see a Schlegel diagram. How many 3-dimensional polytopes do you see? How many

### 24. The Branch and Bound Method

24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

### Linear Programming. Solving LP Models Using MS Excel, 18

SUPPLEMENT TO CHAPTER SIX Linear Programming SUPPLEMENT OUTLINE Introduction, 2 Linear Programming Models, 2 Model Formulation, 4 Graphical Linear Programming, 5 Outline of Graphical Procedure, 5 Plotting

### Linear Programming Notes V Problem Transformations

Linear Programming Notes V Problem Transformations 1 Introduction Any linear programming problem can be rewritten in either of two standard forms. In the first form, the objective is to maximize, the material

### Linear programming and reductions

Chapter 7 Linear programming and reductions Many of the problems for which we want algorithms are optimization tasks: the shortest path, the cheapest spanning tree, the longest increasing subsequence,

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### SUPPLEMENT TO CHAPTER

SUPPLEMENT TO CHAPTER 6 Linear Programming SUPPLEMENT OUTLINE Introduction and Linear Programming Model, 2 Graphical Solution Method, 5 Computer Solutions, 14 Sensitivity Analysis, 17 Key Terms, 22 Solved

### 1 Portfolio mean and variance

Copyright c 2005 by Karl Sigman Portfolio mean and variance Here we study the performance of a one-period investment X 0 > 0 (dollars) shared among several different assets. Our criterion for measuring

### 5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

### IEOR 4404 Homework #2 Intro OR: Deterministic Models February 14, 2011 Prof. Jay Sethuraman Page 1 of 5. Homework #2

IEOR 4404 Homework # Intro OR: Deterministic Models February 14, 011 Prof. Jay Sethuraman Page 1 of 5 Homework #.1 (a) What is the optimal solution of this problem? Let us consider that x 1, x and x 3

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### SOLVING LINEAR SYSTEM OF INEQUALITIES WITH APPLICATION TO LINEAR PROGRAMS

SOLVING LINEAR SYSTEM OF INEQUALITIES WITH APPLICATION TO LINEAR PROGRAMS Hossein Arsham, University of Baltimore, (410) 837-5268, harsham@ubalt.edu Veena Adlakha, University of Baltimore, (410) 837-4969,

### Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible:

Cramer s Rule and the Adjugate Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible: Theorem [Cramer s Rule] If A is an invertible

### Mathematics for Economics (Part I) Note 5: Convex Sets and Concave Functions

Natalia Lazzati Mathematics for Economics (Part I) Note 5: Convex Sets and Concave Functions Note 5 is based on Madden (1986, Ch. 1, 2, 4 and 7) and Simon and Blume (1994, Ch. 13 and 21). Concave functions

### Lecture notes 1: Introduction to linear and (mixed) integer programs

Lecture notes 1: Introduction to linear and (mixed) integer programs Vincent Conitzer 1 An example We will start with a simple example. Suppose we are in the business of selling reproductions of two different

### The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23

(copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, high-dimensional

### Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain

Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain 1. Orthogonal matrices and orthonormal sets An n n real-valued matrix A is said to be an orthogonal

### Section 1.1. Introduction to R n

The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to

### Using the Simplex Method to Solve Linear Programming Maximization Problems J. Reeb and S. Leavengood

PERFORMANCE EXCELLENCE IN THE WOOD PRODUCTS INDUSTRY EM 8720-E October 1998 \$3.00 Using the Simplex Method to Solve Linear Programming Maximization Problems J. Reeb and S. Leavengood A key problem faced

### Graphical method. plane. (for max) and down (for min) until it touches the set of feasible solutions. Graphical method

The graphical method of solving linear programming problems can be applied to models with two decision variables. This method consists of two steps (see also the first lecture): 1 Draw the set of feasible

### Chapter 4. Duality. 4.1 A Graphical Example

Chapter 4 Duality Given any linear program, there is another related linear program called the dual. In this chapter, we will develop an understanding of the dual linear program. This understanding translates

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### This exposition of linear programming

Linear Programming and the Simplex Method David Gale This exposition of linear programming and the simplex method is intended as a companion piece to the article in this issue on the life and work of George

### EXPLICIT ABS SOLUTION OF A CLASS OF LINEAR INEQUALITY SYSTEMS AND LP PROBLEMS. Communicated by Mohammad Asadzadeh. 1. Introduction

Bulletin of the Iranian Mathematical Society Vol. 30 No. 2 (2004), pp 21-38. EXPLICIT ABS SOLUTION OF A CLASS OF LINEAR INEQUALITY SYSTEMS AND LP PROBLEMS H. ESMAEILI, N. MAHDAVI-AMIRI AND E. SPEDICATO

### Linear Programming with Post-Optimality Analyses

Linear Programming with Post-Optimality Analyses Wilson Problem: Wilson Manufacturing produces both baseballs and softballs, which it wholesales to vendors around the country. Its facilities permit the

### 5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1

5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 General Integer Linear Program: (ILP) min c T x Ax b x 0 integer Assumption: A, b integer The integrality condition

### Vector Spaces II: Finite Dimensional Linear Algebra 1

John Nachbar September 2, 2014 Vector Spaces II: Finite Dimensional Linear Algebra 1 1 Definitions and Basic Theorems. For basic properties and notation for R N, see the notes Vector Spaces I. Definition

### Basic solutions in standard form polyhedra

Recap, and outline of Lecture Previously min s.t. c x Ax = b x Converting any LP into standard form () Interpretation and visualization () Full row rank assumption on A () Basic solutions and bases in

### A Hybrid Algorithm for Solving the Absolute Value Equation

A Hybrid Algorithm for Solving the Absolute Value Equation Olvi L. Mangasarian Abstract We propose a hybrid algorithm for solving the NP-hard absolute value equation (AVE): Ax x = b, where A is an n n

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three