# Stanford University CS261: Optimization Handout 5 Luca Trevisan January 18, 2011

Save this PDF as:

Size: px
Start display at page:

Download "Stanford University CS261: Optimization Handout 5 Luca Trevisan January 18, 2011"

## Transcription

1 Stanford University CS261: Optimization Handout 5 Luca Trevisan January 18, 2011 Lecture 5 In which we introduce linear programming. 1 Linear Programming A linear program is an optimization problem in which we have a collection of variables, which can take real values, and we want to find an assignment of values to the variables that satisfies a given collection of linear inequalities and that maximizes or minimizes a given linear function. (The term programming in linear programming, is not used as in computer programming, but as in, e.g., tv programming, to mean planning.) For example, the following is a linear program. maximize x 1 + x 2 subject to x 1 + 2x 2 1 2x 1 + x 2 1 x 1 0 x 2 0 (1) The linear function that we want to optimize (x 1 + x 2 in the above example) is called the objective function. A feasible solution is an assignment of values to the variables that satisfies the inequalities. The value that the objective function gives to an assignment is called the cost of the assignment. For example, x 1 := 1 and 3 x 2 := 1 is a feasible solution, of cost 2. Note that if x 3 3 1, x 2 are values that satisfy the inequalities, then, by summing the first two inequalities, we see that that is, 3x 1 + 3x 2 2 1

2 x 1 + x and so no feasible solution has cost higher than 2 3, so the solution x 1 := 1 3, x 2 := 1 3 is optimal. As we will see in the next lecture, this trick of summing inequalities to verify the optimality of a solution is part of the very general theory of duality of linear programming. Linear programming is a rather different optimization problem from the ones we have studied so far. Optimization problems such as Vertex Cover, Set Cover, Steiner Tree and TSP are such that, for a given input, there is only a finite number of possible solutions, so it is always trivial to solve the problem in finite time. The number of solutions, however, is typically exponentially big in the size of the input and so, in order to be able to solve the problem on reasonably large inputs, we look for polynomial-time algorithms. In linear programming, however, each variable can take an infinite number of possible values, so it is not even clear that the problem is solvable in finite time. As we will see, it is indeed possible to solve linear programming problems in finite time, and there are in fact, polynomial time algorithms and efficient algorithms that solve linear programs optimally. There are at least two reasons why we are going to study linear programming in a course devoted to combinatorial optimization: Efficient linear programming solvers are often used as part of the toolkit to design exact or approximate algorithms for combinatorial problems. The powerful theory of duality of linear programming, that we will describe in the next lecture, is a very useful mathematical theory to reason about algorithms, including purely combinatorial algorithms for combinatorial problems that seemingly have no connection with continuous optimization. 2 A Geometric Interpretation 2.1 A 2-Dimensional Example Consider again the linear program (1). Since it has two variables, we can think of any possible assignment of values to the variables as a point (x 1, x 2 ) in the plane. With this interpretation, every inequality, for example x 1 + 2x 2 1, divides the plane into two regions: the set of points (x 1, x 2 ) such that x 1 + 2x 2 > 1, which are definitely not feasible solutions, and the points such that x+1+2x 2 1, which satisfy the inequality 2

3 and which could be feasible provided that they also satisfy the other inequalities. The line with equation x 1 + 2x 2 = 1 is the boundary between the two regions. The set of feasible solutions to (1) is the set of points which satisfy all four inequalities, shown in blue below: The feasible region is a polygon with four edges, one for each inequality. This is not entirely a coincidence: for each inequality, for example x 1 + 2x 2 1, we can look at the line which is the boundary between the region of points that satisfy the inequality and the region of points that do not, that is, the line x 1 +2x 1 = 1 in this example. The points on the line that satisfy the other constraints form a segment (in the example, the segment of the line x 1 + 2x 2 = 1 such that 0 x 1 1/3), and that segment is one of the edges of the polygon of feasible solutions. Although it does not happen in our example, it could also be that if we take one of the inequalities, consider the line which is the boundary of the set of points that satisfy the inequality, and look at which points on the line are feasible for the linear program, we end up with the empty set (for example, suppose that in the above example we also had the inequality x 1 + x 2 1); in this case the inequality does not give rise to an edge of the polygon of feasible solutions. Another possibility is that the line intersects the feasible region only at one point (for example suppose we also had the inequality x 1 + x 2 0). Yet another possibility is that our polygon is unbounded, in which case one of its edges is not a segment but a half-line (for example, suppose we did not have the inequality x 1 0, then the half-line of points such that x 2 = 0 and x 1 1 would have been an edge). To look for the best feasible solution, we can start from an arbitrary point, for example the vertex (0, 0). We can then divide the plane into two regions: the set of points whose cost is greater than or equal to the cost of (0, 0), that is the set of points such that x 1 + x 2 0, and the set of points of cost lower than the cost of (0, 0), that is, 3

4 the set of points such that x 1 + x 2 < 0. Clearly we only need to continue our search in the first region, although we see that actually the entire set of feasible points is in the first region, so the point (0, 0) is actually the worst solution. So we continue our search by trying another vertex, for example (1/2, 0). Again we can divide the plane into the set of points of cost greater than or equal to the cost of (1/2, 0), that is the points such that x 1 + x 2 1/2, and the set of points of cost lower than the cost of (1, 0). We again want to restrict ourselves to the first region, and we see that we have now narrowed down our search quite a bit: the feasible points in the first region are shown in red: So we try another vertex in the red region, for example ( 1, ) 1 3 3, which has cost 2. If 3 we consider the region of points of cost greater than or equal to the cost of the point (1/3, 1/3), that is, the region x 1 + x 2 2/3, we see that the point (1/3, 1/3) is the only feasible point in the region, and so there is no other feasible point of higher cost, and we have found our optimal solution. 2.2 A 3-Dimensional Example Consider now a linear program with three variables, for example maximize x 1 + 2x 2 x 3 subject to x 1 + x 2 1 x 2 + x 3 1 x 1 0 x 2 0 x 3 0 (2) 4

5 In this case we can think of every assignment to the variables as a point (x 1, x 2, x 3 ) in three-dimensional space. Each constraint divides the space into two regions as before; for example the constraint x 1 + x 2 1 divides the space into the region of points (x 1, x 2, x 3 ) such that x 1 + x 2 1, which satisfy the equation, and points such that x 1 + x 2 > 1, which do not. The boundary between the regions is the plane of points such that x 1 + x 2 = 1. The region of points that satisfy an inequality is called a half-space. The set of feasible points is a polyhedron (plural: polyhedra). A polyhedron is bounded by faces, which are themselves polygons. For example, a cube has six faces, and each face is a square. In general, if we take an inequality, for example x 3 0, and consider the plane x 3 = 0 which is the boundary of the half-space of points that satisfy the inequality, and we consider the set of feasible points in that plane, the resulting polygon (if it s not the empty set) is one of the faces of our polyhedron. For example, the set of feasible points in the place x 3 = 0 is the triangle given by the inequalities x 1 0, x 2 0, x 1 + x 2 1, with vertices (0, 0, 0), (0, 1, 0) and (1, 0, 0). So we see that a 2-dimensional face is obtained by taking our inequalities and changing one of them to equality, provided that the resulting feasible region is two-dimensional; a 1-dimensional edge is obtained by changing two inequalities to equality, again provided that the resulting constraints define a 1-dimensional region; and a vertex is obtained by changing three inequalities to equality, provided that the resulting point is feasible for the other inequalities. As before, we can start from a feasible point, for example the vertex (0, 0, 0), of cost zero, obtained by changing the last three inequalities to equality. We need to check if there are feasible points, other than (0, 0, 0), such that x 1 + 2x 2 x 3 0. That is, we are interested in the set of points such that x 1 + x 2 1 x 2 + x 3 1 x 1 0 x 2 0 x 3 0 x 1 + 2x 2 x 3 0 which is again a polyhedron, of which (0, 0, 0) is a vertex. To find another vertex, if any, we try to start from the three inequalities that we changed to equality to find (0, 0, 0), and remove one to see if we get an edge of non-zero length or just the point (0, 0, 0) again. For example, if we keep x 1 = 0, x 2 = 0, we see that only feasible value of x 3 is zero, so there is no edge; if we keep x 1 = 0, x 3 = 0, we have that the region 0 x 2 1 is feasible, and so it is an edge of the above polyhedron. The other vertex of that edge is (0, 1, 0), which is the next solution that we shall consider. It is a solution of cost 5

6 2, so, in order to look for a better solution, we want to consider the polyhedron x 1 + x 2 1 x 2 + x 3 1 x 1 0 x 2 0 x 3 0 x 1 + 2x 2 x 3 2 In order to see if this polyhedron has any edge of non-zero length, we again keep only two of the three equations that defined our vertex (0, 1, 0), that is only two of the equations x 1 = 0, x 3 = 0, x 1 + x 2 = 1. If we keep the first two, (0, 1, 0) is the only feasible point. If we keep x 3 = 0 and x 1 + x 2 = 1, then (0, 1, 0) is again the only feasible point. If we keep x 1 = 0 and x x 2 = 1, that is x 1 = 0 and x 2 = 1, we see again that the only feasible point is (0, 1, 0). These are the only three edges that could have had (0, 1, 0) as an endpoint, and since (0, 1, 0) is a vertex of the above polytope, we have to conclude that the polytope has no edge, and so it is made of the single point (0, 1, 0). This means that (0, 1, 0) is the optimal solution of (2) 2.3 The General Case In general, if we have linear program with n variables x 1,..., x n, we can think of every assignment to the variables as an n-dimensional point (x 1,..., x n ) in R n. Every inequality a 1 x a n x n b divides the space R n into the region that satisfies the inequality and the region that does not satisfy the inequality, with the hyperplane a 1 x 1 + a n x n being the boundary between the two regions. The two regions are called half-spaces. The set of feasible points is an intersection of half-spaces and is called a polytope. Generalizing the approach that we have used in the previous two examples, the following is the outline of an algorithm to find an optimal solution to a given maximization linear program: 1. Start from a vertex (a 1,..., a n ) in the feasible region, by changing n of the inequalities to equality in such a way that: (i) the resulting n equations are linearly independent, and (ii) the unique solution is feasible for the remaining inequalities; If there is no such vertex, output linear program is infeasible. 6

7 2. Consider the n possible edges of the polytope of feasible solutions that have (a 1,..., a n ) as an endpoint. That is, for each of the n equations that identified (a 1,..., a n ), set back that equation to an inequality, and consider the set of solutions that are feasible for the other n 1 equations and for the inequalities (this set of points can be a line, a half-line, a segment, or just the point (a 1,..., a n )). If there is an edge that contains points of arbitrarily large cost, then output optimum is unbounded Else, if there are edges that contain points of cost larger than (a 1,..., a n ), then let (b 1,..., b n ) be the second endpoint of one of such edges (a 1,..., a n ) := (b 1,..., b n ); go to 2 Else, output (a 1,..., a n ) is an optimal solution This is the outline of an algorithm called the Simplex algorithm. It is not a complete description because: We haven t discussed how to find the initial vertex. This is done by constructing a new polytope such that finding an optimal solution in the new polytope either gives us a feasible vertex for the original linear program, or a certificate that the original problem is infeasible. We then apply the Simplex algorithm to the new polytope. Now, this looks like the beginning of an infinite loop, but the new polytope is constructed in such a way that it is easy to find an initial feasible vertex. We haven t discussed certain special cases; for example it is possible for a polytope to not have any vertex, for example if the number of inequalities is less than the number of variables. (In such a case we can either add inequalities or eliminate variables in a way that do not change the optimum but that creates vertices.) Also there can be cases in which a vertex of a polytope in R n is the endpoint of more than n edges (consider a pyramid with a square base in R 3 : the top vertex has is an endpoint of four edges), while the algorithm, as described above, considers at most n edges for every vertex. The simplex algorithm shows that a linear program can always be solved in finite time, and in fact in time that is at most exponential in the number of variables. This is because each iteration takes polynomial time and moves to a new vertex, and if there are m inequalities and n variables there can be at most ( m n) m n vertices. Unfortunately, for all the known variants of the simplex method (which differ in the way they choose the vertex to move to, when there is more than possible choice) there 7

8 are examples of linear programs on which the algorithm takes exponential time. In practice, however, the simplex algorithm is usually very fast, even on linear programs with tens or hundreds of thousands of variables and constraints. 2.4 Polynomial Time Algorithms for LInear Programming Two (families of) polynomial time algorithms for linear programming are known. One, called the ellipsoid algorithm, starts by finding an ellipsoid (the high-dimensional analog of an ellipse, a squashed disk ) that contains the optimal solution, and then, at every step, it constructs another ellipsoid whose volume is a smaller than the previous one, while still being guaranteed to contain the optimal solution. After several iterations, the algorithm identifies a tiny region that contains the optimal solution. It is known that if a linear program has a finite optimum, the values of the x i in the optimal solution are rational numbers in which both the denominator and numerator have a polynomial number of digits in the size of the input (assuming all coefficients in the objective function and in the inequalities are also rational numbers and that we count the number of digits in their fractional representation when we compute the size of the input), and so if the final ellipsoid is small enough there is only one point with such rational coordinates in the ellipsoid. The other algorithm, which is actually a family of algorithms, uses the interior point method, in which the algorithm computes a sequence of points in the interior of the polytope (in contrast to the simplex algorithm, which finds a sequence of vertices on the exterior of the polytope), where each point is obtained from the previous one by optimizing a properly chosen function that favors points of higher cost for the objective function, and disfavors points that are too close to the boundary of the polytope. Eventually, the algorithm finds a point that is arbitrary close to the optimal vertex, and the actual optimal vertex can be found, like in the ellipsoid method, once it is the unique point with bounded rational coordinates that is close to the current point. 2.5 Summary Here are the important points to remember about what we discussed so far: In a linear program we are given a set of variables x 1,..., x n and we want to find an assignment to the variables that satisfies a given set of linear equalities, and that maximizes or minimizes a given linear objective function of the variables. An assignment that satisfies the inequalities is called a feasible solution; If we think of every possible assignment to the variables as a point in R n, then the set of feasible solutions forms a polytope; 8

9 It is possible that there is no feasible solution to the given inequalities, in which case we call the linear program infeasible. If the linear program is feasible, it is possible that it is of maximization type and there are solutions of arbitrarily large cost, or that it is of minimization type and there are solutions of arbitrarily small cost. In this case we say that the linear program is unbounded. Sometimes we will say that the optimum of an unbounded maximization linear program is +, and that the optimum of an unbounded minimization linear program is, even though this is not entirely correct because there is no feasible solution of cost + or, but rather a sequence of solutions such the limit of their cost is + or. If the linear program is feasible and not unbounded then it has a finite optimum, and we are interested in finding a feasible solution of optimum cost. The simplex algorithm, the ellipsoid algorithm, and the interior point algorithms are able, given a linear program, to determine if it is feasible or not, if feasible they can determine if it is bounded or unbounded, and if feasible and bounded they can find a solution of optimum cost. All three run in finite time; in the worst case, the simplex algorithm runs in exponential time, while the other algorithms run in time polynomial in the size of the input. When we refer to the size of the input we assume that all coefficients are rational numbers, and the size of the input is the total number of bits necessary to represent the coefficients of the objective function and of the inequalities as ratios of integers. For example, the rational number a/b requires log 2 a+log 2 b+ O(1) bits to be represented, if a and b have no common factor. 3 Standard Form for Linear Programs We say that a maximization linear program with n variables is in standard form if for every variable x i we have the inequality x i 0 and all other m inequalities are of type. A linear program in standard form can be written as Let us unpack the above notation. maximize subject to c T x Ax b x 0 (3) 9

10 The vector c = function, x = c 1. x 1. x n c n R n is the column vector of coefficients of the objective is the column vector of variables, c T = (c 1,..., c n ) is the transpose of c, a row vector, and c T x is the matrix product of the 1 n matrix c T times the n 1 matrix x, which is the value that is, the objective function. c 1 x 1 + c n x n The matrix A is the n m matrix of coefficients of the left-hand sides of the inequalities, and b is the m-dimensional vector of right-hand sides of the inequalities. When we write a b, for two vectors a = a 1. a m and b = b 1. b m we mean the m inequalities a 1 b 1,..., a m b m, so the inequality Ax b means the collection of inequalities a 1,1 x a 1,n x n b 1 a m,1 x a m,n x n b m Putting a linear program in standard form is a useful first step for linear programming algorithms, and it is also useful to develop the theory of duality as we will do in the next lecture. It is easy to see that given an arbitrary linear program we can find an equivalent linear program in standard form. If we have a linear program in which a variable x is not required to be 0, we can introduce two new variables x and x, apply the substitution x x x in every inequality in which x appears, and add the inequalities x 0, x 0. Any feasible solution for the new linear program is feasible for the original one, after assigning x := x x, with the same cost; also, any solution that was feasible for the original program can be converted into a solution for the new linear program, with the same cost, by assigning x := x, x := 0 if x > 0 and x := 0, x = x if x 0. 10

11 If we have an inequality of type, other than the inequalities x i 0, we can change sign to the left-hand side and the right-hand side, and change the direction of the inequality. Some definitions of linear programming allow equations as constraints. If we have an equation a T x = b, we rewrite it as the two inequalities a T x b and a T x b. The standard form for a minimization problem is minimize subject to c T x Ax b x 0 (4) As we did for maximization problems, every minimization problem can be put into normal form by changing the sign of inequalities and doing the substitution x x x for every variable x that does not have a non-negativity constraint x 0. 11

### Stanford University CS261: Optimization Handout 6 Luca Trevisan January 20, In which we introduce the theory of duality in linear programming.

Stanford University CS261: Optimization Handout 6 Luca Trevisan January 20, 2011 Lecture 6 In which we introduce the theory of duality in linear programming 1 The Dual of Linear Program Suppose that we

### Chapter 15 Introduction to Linear Programming

Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of

### What is Linear Programming?

Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### Linear Programming. Widget Factory Example. Linear Programming: Standard Form. Widget Factory Example: Continued.

Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.

### Using the Simplex Method in Mixed Integer Linear Programming

Integer Using the Simplex Method in Mixed Integer UTFSM Nancy, 17 december 2015 Using the Simplex Method in Mixed Integer Outline Mathematical Programming Integer 1 Mathematical Programming Optimisation

### . P. 4.3 Basic feasible solutions and vertices of polyhedra. x 1. x 2

4. Basic feasible solutions and vertices of polyhedra Due to the fundamental theorem of Linear Programming, to solve any LP it suffices to consider the vertices (finitely many) of the polyhedron P of the

### I. Solving Linear Programs by the Simplex Method

Optimization Methods Draft of August 26, 2005 I. Solving Linear Programs by the Simplex Method Robert Fourer Department of Industrial Engineering and Management Sciences Northwestern University Evanston,

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### max cx s.t. Ax c where the matrix A, cost vector c and right hand side b are given and x is a vector of variables. For this example we have x

Linear Programming Linear programming refers to problems stated as maximization or minimization of a linear function subject to constraints that are linear equalities and inequalities. Although the study

### Linear Programming I

Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins

### Linear Programming for Optimization. Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc.

1. Introduction Linear Programming for Optimization Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc. 1.1 Definition Linear programming is the name of a branch of applied mathematics that

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### 1 Polyhedra and Linear Programming

CS 598CSC: Combinatorial Optimization Lecture date: January 21, 2009 Instructor: Chandra Chekuri Scribe: Sungjin Im 1 Polyhedra and Linear Programming In this lecture, we will cover some basic material

### Graphical method. plane. (for max) and down (for min) until it touches the set of feasible solutions. Graphical method

The graphical method of solving linear programming problems can be applied to models with two decision variables. This method consists of two steps (see also the first lecture): 1 Draw the set of feasible

### Linear Programming, Lagrange Multipliers, and Duality Geoff Gordon

lp.nb 1 Linear Programming, Lagrange Multipliers, and Duality Geoff Gordon lp.nb 2 Overview This is a tutorial about some interesting math and geometry connected with constrained optimization. It is not

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### Geometry of Linear Programming

Chapter 2 Geometry of Linear Programming The intent of this chapter is to provide a geometric interpretation of linear programming problems. To conceive fundamental concepts and validity of different algorithms

### The Simplex Method. yyye

Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note #05 1 The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/

### Lecture 2: August 29. Linear Programming (part I)

10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.

### By W.E. Diewert. July, Linear programming problems are important for a number of reasons:

APPLIED ECONOMICS By W.E. Diewert. July, 3. Chapter : Linear Programming. Introduction The theory of linear programming provides a good introduction to the study of constrained maximization (and minimization)

### Chap 4 The Simplex Method

The Essence of the Simplex Method Recall the Wyndor problem Max Z = 3x 1 + 5x 2 S.T. x 1 4 2x 2 12 3x 1 + 2x 2 18 x 1, x 2 0 Chap 4 The Simplex Method 8 corner point solutions. 5 out of them are CPF solutions.

### Mathematics Notes for Class 12 chapter 12. Linear Programming

1 P a g e Mathematics Notes for Class 12 chapter 12. Linear Programming Linear Programming It is an important optimization (maximization or minimization) technique used in decision making is business and

### 1.1 Solving a Linear Equation ax + b = 0

1.1 Solving a Linear Equation ax + b = 0 To solve an equation ax + b = 0 : (i) move b to the other side (subtract b from both sides) (ii) divide both sides by a Example: Solve x = 0 (i) x- = 0 x = (ii)

### Good luck, veel succes!

Final exam Advanced Linear Programming, May 7, 13.00-16.00 Switch off your mobile phone, PDA and any other mobile device and put it far away. No books or other reading materials are allowed. This exam

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### Section Notes 3. The Simplex Algorithm. Applied Math 121. Week of February 14, 2011

Section Notes 3 The Simplex Algorithm Applied Math 121 Week of February 14, 2011 Goals for the week understand how to get from an LP to a simplex tableau. be familiar with reduced costs, optimal solutions,

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### Linear Programming: Introduction

Linear Programming: Introduction Frédéric Giroire F. Giroire LP - Introduction 1/28 Course Schedule Session 1: Introduction to optimization. Modelling and Solving simple problems. Modelling combinatorial

### The Canonical and Dictionary Forms of Linear Programs

The Canonical and Dictionary Forms of Linear Programs Recall the general optimization problem: Minimize f(x) subect to x D (1) The real valued function f is called the obective function, and D the constraint

### The Graphical Method: An Example

The Graphical Method: An Example Consider the following linear program: Maximize 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2 0, where, for ease of reference,

### Notes on polyhedra and 3-dimensional geometry

Notes on polyhedra and 3-dimensional geometry Judith Roitman / Jeremy Martin April 23, 2013 1 Polyhedra Three-dimensional geometry is a very rich field; this is just a little taste of it. Our main protagonist

### Outline. Linear Programming (LP): Simplex Search. Simplex: An Extreme-Point Search Algorithm. Basic Solutions

Outline Linear Programming (LP): Simplex Search Benoît Chachuat McMaster University Department of Chemical Engineering ChE 4G03: Optimization in Chemical Engineering 1 Basic Solutions

### Special Situations in the Simplex Algorithm

Special Situations in the Simplex Algorithm Degeneracy Consider the linear program: Maximize 2x 1 +x 2 Subject to: 4x 1 +3x 2 12 (1) 4x 1 +x 2 8 (2) 4x 1 +2x 2 8 (3) x 1, x 2 0. We will first apply the

### SOLVING LINEAR SYSTEM OF INEQUALITIES WITH APPLICATION TO LINEAR PROGRAMS

SOLVING LINEAR SYSTEM OF INEQUALITIES WITH APPLICATION TO LINEAR PROGRAMS Hossein Arsham, University of Baltimore, (410) 837-5268, harsham@ubalt.edu Veena Adlakha, University of Baltimore, (410) 837-4969,

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### Algebraic Simplex Method - Introduction Previous

Algebraic Simplex Method - Introduction To demonstrate the simplex method, consider the following linear programming model: This is the model for Leo Coco's problem presented in the demo, Graphical Method.

### Advanced Algebra 2. I. Equations and Inequalities

Advanced Algebra 2 I. Equations and Inequalities A. Real Numbers and Number Operations 6.A.5, 6.B.5, 7.C.5 1) Graph numbers on a number line 2) Order real numbers 3) Identify properties of real numbers

### LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS. 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method

LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method Introduction to dual linear program Given a constraint matrix A, right

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

### Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.

This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### LINEAR PROGRAMMING P V Ram B. Sc., ACA, ACMA Hyderabad

LINEAR PROGRAMMING P V Ram B. Sc., ACA, ACMA 98481 85073 Hyderabad Page 1 of 19 Question: Explain LPP. Answer: Linear programming is a mathematical technique for determining the optimal allocation of resources

### 5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### Theory of Linear Programming

Theory of Linear Programming Debasis Mishra April 6, 2011 1 Introduction Optimization of a function f over a set S involves finding the maximum (minimum) value of f (objective function) in the set S (feasible

### 4.6 Linear Programming duality

4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal

### Simplex method summary

Simplex method summary Problem: optimize a linear objective, subject to linear constraints 1. Step 1: Convert to standard form: variables on right-hand side, positive constant on left slack variables for

### Portable Assisted Study Sequence ALGEBRA IIA

SCOPE This course is divided into two semesters of study (A & B) comprised of five units each. Each unit teaches concepts and strategies recommended for intermediate algebra students. The first half of

### is redundant. If you satisfy the other four constraints, then you automatically satisfy x 1 0. (In fact, if the first two constraints hold, then x 1 m

Linear Programming Notes II: Graphical Solutions 1 Graphing Linear Inequalities in the Plane You can solve linear programming problems involving just two variables by drawing a picture. The method works

### 18. Linear optimization

18. Linear optimization EE103 (Fall 2011-12) linear program examples geometrical interpretation extreme points simplex method 18-1 Linear program minimize c 1 x 1 +c 2 x 2 + +c n x n subject to a 11 x

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### Fundamentals of Operations Research. Prof. G. Srinivasan. Department of Management Studies. Indian Institute of Technology, Madras. Lecture No.

Fundamentals of Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Lecture No. # 22 Inventory Models - Discount Models, Constrained Inventory

### MINI LESSON. Lesson 5b Solving Quadratic Equations

MINI LESSON Lesson 5b Solving Quadratic Equations Lesson Objectives By the end of this lesson, you should be able to: 1. Determine the number and type of solutions to a QUADRATIC EQUATION by graphing 2.

### 4.1 Learning algorithms for neural networks

4 Perceptron Learning 4.1 Learning algorithms for neural networks In the two preceding chapters we discussed two closely related models, McCulloch Pitts units and perceptrons, but the question of how to

### Algebra 2 Chapter 1 Vocabulary. identity - A statement that equates two equivalent expressions.

Chapter 1 Vocabulary identity - A statement that equates two equivalent expressions. verbal model- A word equation that represents a real-life problem. algebraic expression - An expression with variables.

### Overview of Math Standards

Algebra 2 Welcome to math curriculum design maps for Manhattan- Ogden USD 383, striving to produce learners who are: Effective Communicators who clearly express ideas and effectively communicate with diverse

### Linear Programming Problems

Linear Programming Problems Linear programming problems come up in many applications. In a linear programming problem, we have a function, called the objective function, which depends linearly on a number

### Domain Essential Question Common Core Standards Resources

Middle School Math 2016-2017 Domain Essential Question Common Core Standards First Ratios and Proportional Relationships How can you use mathematics to describe change and model real world solutions? How

### This exposition of linear programming

Linear Programming and the Simplex Method David Gale This exposition of linear programming and the simplex method is intended as a companion piece to the article in this issue on the life and work of George

### CHAPTER 17. Linear Programming: Simplex Method

CHAPTER 17 Linear Programming: Simplex Method CONTENTS 17.1 AN ALGEBRAIC OVERVIEW OF THE SIMPLEX METHOD Algebraic Properties of the Simplex Method Determining a Basic Solution Basic Feasible Solution 17.2

### Module 3 Lecture Notes 2. Graphical Method

Optimization Methods: Linear Programming- Graphical Method Module Lecture Notes Graphical Method Graphical method to solve Linear Programming problem (LPP) helps to visualize the procedure explicitly.

### Module1. x 1000. y 800.

Module1 1 Welcome to the first module of the course. It is indeed an exciting event to share with you the subject that has lot to offer both from theoretical side and practical aspects. To begin with,

### Week 5 Integral Polyhedra

Week 5 Integral Polyhedra We have seen some examples 1 of linear programming formulation that are integral, meaning that every basic feasible solution is an integral vector. This week we develop a theory

### Induction Problems. Tom Davis November 7, 2005

Induction Problems Tom Davis tomrdavis@earthlin.net http://www.geometer.org/mathcircles November 7, 2005 All of the following problems should be proved by mathematical induction. The problems are not necessarily

### Chapter 3 LINEAR PROGRAMMING GRAPHICAL SOLUTION 3.1 SOLUTION METHODS 3.2 TERMINOLOGY

Chapter 3 LINEAR PROGRAMMING GRAPHICAL SOLUTION 3.1 SOLUTION METHODS Once the problem is formulated by setting appropriate objective function and constraints, the next step is to solve it. Solving LPP

### Linear Programming Notes V Problem Transformations

Linear Programming Notes V Problem Transformations 1 Introduction Any linear programming problem can be rewritten in either of two standard forms. In the first form, the objective is to maximize, the material

### Linear Programming is the branch of applied mathematics that deals with solving

Chapter 2 LINEAR PROGRAMMING PROBLEMS 2.1 Introduction Linear Programming is the branch of applied mathematics that deals with solving optimization problems of a particular functional form. A linear programming

### Mathematical Induction

Chapter 2 Mathematical Induction 2.1 First Examples Suppose we want to find a simple formula for the sum of the first n odd numbers: 1 + 3 + 5 +... + (2n 1) = n (2k 1). How might we proceed? The most natural

### 3. Evaluate the objective function at each vertex. Put the vertices into a table: Vertex P=3x+2y (0, 0) 0 min (0, 5) 10 (15, 0) 45 (12, 2) 40 Max

SOLUTION OF LINEAR PROGRAMMING PROBLEMS THEOREM 1 If a linear programming problem has a solution, then it must occur at a vertex, or corner point, of the feasible set, S, associated with the problem. Furthermore,

### Lecture 3. Mathematical Induction

Lecture 3 Mathematical Induction Induction is a fundamental reasoning process in which general conclusion is based on particular cases It contrasts with deduction, the reasoning process in which conclusion

### LINEAR PROGRAMMING PROBLEM: A GEOMETRIC APPROACH

59 LINEAR PRGRAMMING PRBLEM: A GEMETRIC APPRACH 59.1 INTRDUCTIN Let us consider a simple problem in two variables x and y. Find x and y which satisfy the following equations x + y = 4 3x + 4y = 14 Solving

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### Fundamentals of Operations Research Prof.G. Srinivasan Department of Management Studies Lecture No. # 03 Indian Institute of Technology, Madras

Fundamentals of Operations Research Prof.G. Srinivasan Department of Management Studies Lecture No. # 03 Indian Institute of Technology, Madras Linear Programming solutions - Graphical and Algebraic Methods

### CAMI Education linked to CAPS: Mathematics

- 1 - TOPIC 1.1 Whole numbers _CAPS Curriculum TERM 1 CONTENT Properties of numbers Describe the real number system by recognizing, defining and distinguishing properties of: Natural numbers Whole numbers

### The Graphical Simplex Method: An Example

The Graphical Simplex Method: An Example Consider the following linear program: Max 4x 1 +3x Subject to: x 1 +3x 6 (1) 3x 1 +x 3 () x 5 (3) x 1 +x 4 (4) x 1, x 0. Goal: produce a pair of x 1 and x that

### (Refer Slide Time: 00:00:56 min)

Numerical Methods and Computation Prof. S.R.K. Iyengar Department of Mathematics Indian Institute of Technology, Delhi Lecture No # 3 Solution of Nonlinear Algebraic Equations (Continued) (Refer Slide

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### Solving Mixed Integer Linear Programs Using Branch and Cut Algorithm

1 Solving Mixed Integer Linear Programs Using Branch and Cut Algorithm by Shon Albert A Project Submitted to the Graduate Faculty of North Carolina State University in Partial Fulfillment of the Requirements

### The Simplex Method. What happens when we need more decision variables and more problem constraints?

The Simplex Method The geometric method of solving linear programming problems presented before. The graphical method is useful only for problems involving two decision variables and relatively few problem

### Actually Doing It! 6. Prove that the regular unit cube (say 1cm=unit) of sufficiently high dimension can fit inside it the whole city of New York.

1: 1. Compute a random 4-dimensional polytope P as the convex hull of 10 random points using rand sphere(4,10). Run VISUAL to see a Schlegel diagram. How many 3-dimensional polytopes do you see? How many

### North Carolina Math 1

Standards for Mathematical Practice 1. Make sense of problems and persevere in solving them. 2. Reason abstractly and quantitatively 3. Construct viable arguments and critique the reasoning of others 4.

### 3 Does the Simplex Algorithm Work?

Does the Simplex Algorithm Work? In this section we carefully examine the simplex algorithm introduced in the previous chapter. Our goal is to either prove that it works, or to determine those circumstances

MA 134 Lecture Notes August 20, 2012 Introduction The purpose of this lecture is to... Introduction The purpose of this lecture is to... Learn about different types of equations Introduction The purpose

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### Chapter 1 Linear Programming. Paragraph 2 Developing Ideas for an Algorithm solving Linear Optimization

Chapter 1 Linear Programming Paragraph 2 Developing Ideas for an Algorithm solving Linear Optimization What we did so far How to model an optimization problem Data, Variables, Constraints, Objective We

### Lecture notes 1: Introduction to linear and (mixed) integer programs

Lecture notes 1: Introduction to linear and (mixed) integer programs Vincent Conitzer 1 An example We will start with a simple example. Suppose we are in the business of selling reproductions of two different

### COGNITIVE TUTOR ALGEBRA

COGNITIVE TUTOR ALGEBRA Numbers and Operations Standard: Understands and applies concepts of numbers and operations Power 1: Understands numbers, ways of representing numbers, relationships among numbers,

### High School Algebra 1 Common Core Standards & Learning Targets

High School Algebra 1 Common Core Standards & Learning Targets Unit 1: Relationships between Quantities and Reasoning with Equations CCS Standards: Quantities N-Q.1. Use units as a way to understand problems

### Can linear programs solve NP-hard problems?

Can linear programs solve NP-hard problems? p. 1/9 Can linear programs solve NP-hard problems? Ronald de Wolf Linear programs Can linear programs solve NP-hard problems? p. 2/9 Can linear programs solve

### Identify examples of field properties: commutative, associative, identity, inverse, and distributive.

Topic: Expressions and Operations ALGEBRA II - STANDARD AII.1 The student will identify field properties, axioms of equality and inequality, and properties of order that are valid for the set of real numbers

### Lecture 1: Linear Programming Models. Readings: Chapter 1; Chapter 2, Sections 1&2

Lecture 1: Linear Programming Models Readings: Chapter 1; Chapter 2, Sections 1&2 1 Optimization Problems Managers, planners, scientists, etc., are repeatedly faced with complex and dynamic systems which

### Chapter 4: The Mechanics of the Simplex Method

Chapter 4: The Mechanics of the Simplex Method The simplex method is a remarkably simple and elegant algorithmic engine for solving linear programs. In this chapter we will examine the internal mechanics

### OPRE 6201 : 2. Simplex Method

OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2

### Linear Inequalities and Linear Programming. Systems of Linear Inequalities in Two Variables

Linear Inequalities and Linear Programming 5.1 Systems of Linear Inequalities 5.2 Linear Programming Geometric Approach 5.3 Geometric Introduction to Simplex Method 5.4 Maximization with constraints 5.5

### Math 181 Handout 16. Rich Schwartz. March 9, 2010

Math 8 Handout 6 Rich Schwartz March 9, 200 The purpose of this handout is to describe continued fractions and their connection to hyperbolic geometry. The Gauss Map Given any x (0, ) we define γ(x) =