# 1 Solving LPs: The Simplex Algorithm of George Dantzig

Size: px
Start display at page:

Transcription

1 Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example. Consider the LP (.) max 5x + 4x + 3x 3 s.t. x + 3x + x 3 5 4x + x + x 3 3x + 4x + x 3 8 x, x, x 3 In devising our solution procedure we take a standard mathematical approach; reduce the problem to one that we already know how to solve. Since the structure of this problem is essentially linear, we will try to reduce it to a problem of solving a system of linear equations, or perhaps a series of such systems. By encoding the problem as a system of linear equations we bring into play our knowledge and experience with such systems in the new context of linear programming. In order to encode the LP (.) as a system of linear equations we must first transform linear inequalities into linear equation. This is done by introducing a new non-negative variable, called a slack variable, for each inequality: x 4 = 5 [x + 3x + x 3, x 5 = [4x + x + x 3, x 6 = 8 [3x + 4x + x 3. To handle the objective, we introduce a new variable z: z = 5x + 4x + 3x 3. Then all of the information associated with the LP (.) can be coded as follows: (.) x + 3x + x 3 + x 4 = 5 4x + x + x 3 + x 5 = 3x + 4x + x 3 + x 6 = 8 z + 5x + 4x + 3x 3 = x, x, x 3, x 4, x 5, x 6. The new variables x 4, x 5, and x 6 are called slack variables since they take up the slack in the linear inequalities. This system can also be written using block structured matrix notation as [ [ [ A I z b c T =, x

2 where A = , I =, b = The augmented matrix associated with the system (.) is [ A I (.3) b c and is referred to as the initial simplex tableau for the LP (.). Again consider the system 5 8, and c = (.4) x 4 = 5 x 3x x 3 x 5 = 4x x x 3 x 6 = 8 3x 4x x 3 z = 5x + 4x + 3x 3. This system defines the variables x 4, x 5, x 6 and z as linear combinations of the variables x, x, and x 3. We call this system a dictionary for the LP (.). More specifically, it is the initial dictionary for the the LP (.). This initial dictionary defines the objective value z and the slack variables as a linear combination of the initial decision variables. The variables that are defined in this way are called the basic variables, while the remaining variables are called nonbasic. The set of all basic variables is called the basis. A particular solution to this system is easily obtained by setting the non-basic variables equal to zero. In this case, we get x 4 = 5 x 5 = x 6 = 8 z =. Note that the solution (.5) x x x 3 x 4 x 5 x 6 = 5 8 is feasible for the extended system (.) since all components are non-negative. For this reason, we call the dictionary (.4) a feasible dictionary for the LP (.), and we say that this LP has feasible origin.

3 In general, a dictionary for the LP (.) is any system of 4 linear equations that defines three of the variables x,..., x 6 and z in terms of the remaining 3 variables and has the same solution set as the initial dictionary. The variables other than z that are being defined in the dictionary are called the basis for the dictionary, and the remaining variables are said to be non-basic in the dictionary. Every dictionary identifies a particular solution to the linear system obtained by setting the non-basic variables equal to zero. Such a solution is said to be a basic feasible solution (BFS) for the LP (.) if it componentwise non-negative, that is, all of the numbers in the vector are non-negative so that the point lies in the feasible region for the LP. The grand strategy of the simplex algorithm is to move from one feasible dictionary representation of the system (.) to another (and hence from one BFS to another) while simultaneously increasing the value of the objective variable z. In the current setting, beginning with the dictionary (.4), what strategy might one employ in order to determine a new dictionary whose associated BFS gives a greater value for the objective variable z? Each feasible dictionary is associated with one and only one feasible point. This is the associated BFS obtained by setting all of the non-basic variables equal to zero. This is how we obtain (.5). To change the feasible point identified in this way, we need to increase the value of one of the non-basic variables from its current value of zero. Note that we cannot decrease the value of a non-basic variable since we wish to remain feasible, that is, we wish to keep all variables non-negative. Note that the coefficient of each of the non-basic variables in the representation of the objective value z in (.4) is positive. Hence, if we pick any one of these variables and increase its value from zero while leaving remaining two at zero, we automatically increase the value of the objective variable z. Since the coefficient on x in the representation of z is the greatest, we can increase z the fastest by increasing x. By how much can we increase x and still remain feasible? For example, if we increase x to 3 then (.4) says that x 4 =, x 5 =, x 6 = which is not feasible. Let us consider this question by examining the equations in (.4) one by one. Note that the first equation in the dictionary (.4), x 4 = 5 x 3x x 3, shows that x 4 remains non-negative as long as we do not increase the value of x beyond 5/ (remember, x and x 3 remain at the value zero). Similarly, using the second equation in the dictionary (.4), x 5 = 4x x x 3, x 5 remains non-negative if x /4. Finally, the third equation in (.4), x 6 = 8 3x 4x x 3, implies that x 6 remains non-negative if x 8/3. Therefore, we remain feasible, i.e. keep all variables non-negative, if our increase to the variable x remains less than { 5 5 = min, 4, 8 }. 3 3

4 If we now increase the value of x to 5, then the value of x 4 is driven to zero. One way to think of this is that x enters the basis while x 4 leaves the basis. Mechanically, we obtain the new dictionary having x basic and x 4 non-basic by using the defining equation for x 4 in the current dictionary: x 4 = 5 x 3x x 3. By moving x to the left hand side of this equation and x 4 to the right, we get the new equation x = 5 x 4 3x x 3 or equivalently x = 5 x 4 3 x x 3. The variable x can now be eliminated from the remaining two equations in the dictionary by substituting in this equation for x where it appears in these equations: x = 5 x 4 3 x x 3 ( 5 x 5 = 4 x 4 3 x ) x 3 x x 3 = + x 4 + 5x ( 5 x 6 = 8 3 x 4 3 x ) x 3 4x x 3 = + 3 x 4 + x x 3 ( 5 z = 5 x 4 3 x ) x 3 + 4x + 3x 3 = 5 5 x 4 7 x + x 3. When this substitution is complete, we have the new dictionary and the new BFS: (.6) x = 5 x 4 3 x x 3 and the associated BFS is (.7) x x x 3 x 4 x 5 x 6 x 5 = + x 4 + 5x x 6 = + 3 x 4 + x x 3 z = 5 5 x 4 7 x + x 3, = 5/ / 4 with z = 5.

5 This process might seem very familiar to you. It is simply Gaussian elimination. As we know from our knowledge of linear systems of equations, Gaussian elimination can be performed in a matrix context with the aid of the augmented matrix (or, simplex tableau) (.3). We return to this observation later to obtain a more efficient computational technique. We now have a new dictionary (.6) which identifies the basic feasible solution (BFS) (.7) with associated objective value z = 5. Can we improve on this BFS and obtain a higher objective value? Let s try the same trick again, and repeat the process we followed in going from the initial dictionary (.4) to the new dictionary (.6). Note that the coefficient of x 3 in the representation of z in the new dictionary (.6) is positive. Hence if we increase the value of x 3 from zero, we will increase the value of z. By how much can we increase the value of x 3 and yet keep all the remaining variables non-negative? As before, we see that the first equation in the dictionary (.6) combined with the need to keep x non-negative implies that we cannot increase x 3 by more than (5/)/(/) = 5. However, the second equation in (.6) places no restriction on increasing x 3 since x 3 does not appear in this equation. Finally, the third equation in (.6) combined with the need to keep x 6 non-negative implies that we cannot increase x 3 by more than (/)/(/) =. Therefore, in order to preserve the non-negativity of all variables, we can increase x 3 by at most = min{5, }. When we do this x 6 is driven to zero, so x 3 enters the basis and x 6 leaves. More precisely, first move x 3 to the left hand side of the defining equation for x 6 in (.6), or, equivalently, x 3 = + 3 x 4 + x x 6, x 3 = + 3x 4 + x x 6, then substitute this expression for x 3 into the remaining equations, yielding the dictionary x = 5 x 4 3 x [ + 3x 4 + x x 6 = x 4 x + x 6 x 5 = + x 4 + 5x z = 5 5 x 4 7 x + [ + 3x 4 + x x 6 = 3 x 4 3x x 6, x 3 = + 3x 4 + x x 6 x = x 4 + x + x 6 x 5 = + x 4 + x z = 3 x 4 3x x 6 5

6 which identifies the feasible solution x x x 3 x 4 x 5 x 6 = having objective value z = 3. Can we do better? NO! This solution is optimal! The coefficient on the variables in the cost row of the dictionary, z = 3 x 4 3x x 6, are all non-positive, so increasing any one of their values will not increase the value of the objective. (Why does this prove optimality?) The process of moving from one feasible dictionary to the next is called a simplex pivot. The overall process of stringing a sequence of simplex pivots together in order to locate an optimal solution is called the Simplex Algorithm. The simplex algorithm is consistently ranked as one of the ten most important algorithmic discoveries of the th century ( The algorithm is generally attributed to George Dantzig (94-5)who is known as the father of linear programming. In 984 Narendra Karmarkar published a paper describing a new approach to solving linear programs that was both numerically efficient and had polynomial complexity. This new class of methods are called interior point methods. These new methods have revolutionized the optimization field over the last 3 years, and they have led to efficient numerical methods for a wide variety of optimization problems well beyond the confines of linear programming. However, the simplex algorithm continues as an important numerical method for solving LPs, and for many specially structured LPs it is still the most efficient algorithm.. Simplex Pivoting: Tableau Format (Augmented Matrix Format) We now review the implementation of the simplex algorithm by applying Gaussian elimination to the augmented matrix (.3), also known as the simplex tableau. For this problem, the initial simplex tableau is given by (.8) [ A I c b = Each simplex pivot on a dictionary corresponds to one step of Gaussian elimination on the augmented matrix associated with the dictionary. For example, in the first simplex pivot, x

7 enters the basis and x 4 leaves the basis. That is, we use the first equation of the dictionary to rewrite x as a function of the remaining variables, and then use this representation to eliminate x form the remaining equations. In terms of the augmented matrix (.8), this corresponds to first making the coefficient for x in the first equation the number by dividing this first equation through by. Then use this entry to eliminate the column under x, that is, make all other entries in this column zero (Gaussian elimination): Pivot column 3 5 5/ 4 / / / / / 5/ 5 / / 3/ / 7/ / 5/ 5/ ratios Pivot row In this illustration, we have placed a line above the cost row to delineate its special roll in the pivoting process. In addition, we have also added a column on the right hand side which contains the ratios that we computed in order to determine the pivot row. Recall that we must use the smallest ratio in order to keep all variables in the associated BFS non-negative. Note that we performed the exact same arithmetic operations but in the more efficient matrix format. The new augmented matrix, (.9) 3/ / / 5 / / 3/ 7/ / 5/ 5/ / 5/ is the augmented matrix for the dictionary (.6). The initial augmented matrix (.8) has basis x 4, x 5, and x 6. The columns associated with these variables in the initial tableau (.8) are distinct columns of the identity matrix. Correspondingly, the basis for the second tableau is x, x 5, and x 6, and again this implies that the columns for these variables in the tableau (.9) are the corresponding distinct columns of the identity matrix. In tableau format, this will always be true of the basic variables, i.e., their associated columns are distinct columns of the identity matrix. To recover the BFS (basic feasible solution) associated with this tableau we first set the non-basic variables equal to zero (i.e. the variables not associated with columns of the identity matrix (except in very unusual circumstances)): x =, x 3 =, and x 4 =. To find the value of the basic variables go to the column associated with that variable (for example, x is in the second 7,

8 column), in that column find the row with the number in it, then in that row go to the number to the right of the vertical bar (for x this is the first row with the number to the right of the bar being 5/). Then set this basic variable equal to that number (x = 5/). Repeating this for x 5 and x 6 we get x 5 = and x 6 = /. To get the corresponding value for z, look at the z row and observe that the corresponding linear equation is z 7 x + x 3 5 x 4 = 5, but x, x 3, and x 4 are non-basic and so take the value zero giving z = 5/, or z = 5/. Of course this is all exactly the same information we obtained from the dictionary approach. The simplex, or augmented matrix approach is simply a more efficient computational procedure. We will use the simplex procedure in class to solve LPs. In addition, on quizzes and exams you will be required to understand how to go back and forth between these two representations, i.e the dictionary representation and its corresponding simplex tableau (or, augmented matrix). Let us now continue with the second simplex pivot. In every tableau we always reserve the bottom row for encoding the linear relationship between the objective variable z and the currently non-basic variables. For this reason we call this row the cost row, and to distiguish its special role, we place a line above it in the tableau (this is reminiscent of the way we place a vertical bar in an augmented matrix to distinguish the right hand side of a linear equation). In the cost row of the tableau (.9), [,, 7/, /, 5/,,, 5/, we see a positive coefficient, /, in the 4th column. Hence the cost row coefficient for the non-basic variable x 3 in this tableau is /. This indicates that if we increase the the value of x 3, we also increase the value of the objective z. This is not true for any of the other currently non-basic variables since their cost row coefficients are all non-positive. Thus, the only way we can increase the value of z is to bring x 3 into the basis, or equivalently, pivot on the x 3 column which is the 4th column of the tableau. For this reason, we call the x 3 column the pivot column. Now if x 3 is to enter the basis, then which variable leaves? Just as with the dictionary representation, the variable that leaves the basis is that currently basic variable whose non-negativity places the greatest restriction on increasing the value of x 3. This restriction is computed as the smallest ratio of the right hand sides and the positive coefficients in the x 3 column: = min{(5/)/(/), (/)/(/)}. The ratios are only computed with the positive coefficients since a non-positive coefficient means that by increasing this variable we do not decrease the valued of the corresponding basic variable and so it is not a restricting equation. Since the minimum ratio in this instance is and it comes from the third row, we find that the pivot row is the third row. Looking across the third row, we see that this row identifies x 6 as a basic variable since the x 6 column is a column of the identity with a in the third row. Hence x 6 is the variable leaving the 8

9 basis when x 3 enters. The intersection of the pivot column and the pivot row is called the pivot. In this instance it is the number / which is the (3, 4) entry of the simplex tableau. Pivoting on this entry requires us to first make it by multiplying this row through by, and then to apply Gaussian elimination to force all other entries in this column to zero: Pivot column r 3/ / / 5/ 5 5 / / 7/ / 5/ 5/ 3/ / pivot row This simplex tableau is said to be optimal since it is feasible (the associated BFS is nonnegative) and the cost row coefficients for the variables are all non-positive. A BFS that is optimal is called and optimal basic feasible solution. The optimal BFS is obtained by setting the non-basic variables equal to zero and setting the basic variables equal to the value on the right hand side corresponding to the one in its column: x =, x =, x 3 =, x 4 =, x 5 =, x 6 =. The optimal objective value is obtained by taking the negative of the number in the lower right hand corner of the optimal tableau: z = 3. We now recap the complete sequence of pivots in order to make a final observation that will help streamline the pivoting process: pivots are circled, / / / 5/ 5 / / 3/ / 7/ / 5/ 5/

10 Observe from this sequence of pivots that the z column is never touched, that is, it remains the same in all tableaus. Essentially, it just serves as a place holder reminding us that in the linear equation for the cost row the coefficient of z is. Therefore, for the sake of expediency we will drop this column from our simplex computations in most settings, and simply re-insert it whenever instructive or convenient. However, it is very important to always remember that it is there! Indeed, we will make explicit and essential use of this column in order to arrive at a full understanding of the duality theory for linear programming. After removing this column, the above pivots take the following form: / / / 5/ 5 / / 3/ / 7/ / 5/ 5/ We close this section with a final example of simplex pivoting on a tableau giving only the essential details. The LP Simplex Iterations maximize 3x + y 4z subject to x + 4y 5 x + 4y z 6 x + y z x, y, z

11 Optimal Solution ratios / / x y z = 4 optimal value = 8 A final word of advise, when doing simplex pivoting by hand, it is helpful to keep the tableaus vertically aligned in order to keep track of the arithmetic operations. Lined paper helps to keep the rows straight. But the columns need to be straight as well. Many students find that it is easy to keep both the rows and columns straight if they do pivoting on graph paper having large boxes for the numbers..3 Dictionaries: The General Case for LPs in Standard Form Recall the following standard form for LPs: P : maximize c T x subject to Ax b x, where c R n, b R m, A R m n and the inequalities Ax b and x are to be interpreted componentwise. We now provide a formal definition for a dictionary associated with an LP in standard form. Let n (D I ) x n+i = b i a ij x j z = j= n c j x j j=

12 be the defining system for the slack variables x n+i, i =,, n and the objective variable z. A dictionary for P is any system of the form (D B ) x i = b i j N â ij x j i B z = ẑ + j N ĉ j x j where B and N are index sets contained in the set of integers {,..., n + m} satisfying () B contains m elements, () B N = (3) B N = {,,..., n + m}, and such that the systems (D I ) and (D B ) have identical solution sets. The set {x j : j B} is said to be the basis associated with the dictionary (D B ) (sometimes we will refer to the index set B as the basis for the sake of simplicity), and the variables x i, i N are said to be the non-basic variables associated with this dictionary. The point identified by this dictionary is (.) x i = b i i B x j = j N. The dictionary is said to be feasible if b i for i N. If the dictionary D B is feasible, then the point identified by the dictionary (.) is said to be a basic feasible solution (BFS) for the LP. A feasible dictionary and its associated BFS are said to be optimal if ĉ j j N. Simplex Pivoting by Matrix Multiplication As we have seen simplex pivoting can either be performed on dictionaries or on the augmented matrices that encode the linear equations of a dictionary in matrix form. In matrix form, simplex pivoting reduces to our old friend, Gaussian elimination. In this section, we show that Gaussian elimination can be represented as a consequence of left multiplication by a specially designed matrix called a Gaussian pivot matrix. Consider the vector v R m block decomposed as a v = α b where a R s, α R, and b R t with m = s + + t. Assume that α. We wish to determine a matrix G such that Gv = e s+

13 where for j =,..., n, e j is the unit coordinate vector having a one in the jth position and zeros elsewhere. We claim that the matrix I s s α a G = α α b I t t does the trick. Indeed, I s s α a Gv = α α b I t t a α b = a a α α b + b = = e s+. The matrix G is called a Gaussian Pivot Matrix. Note that G is invertible since I a G = α, b I x and that for any vector of the form w = where x R s y R t, we have y Gw = w. The Gaussian pivot matrices perform precisely the operations required in order to execute a simplex pivot. That is, each simplex pivot can be realized as left multiplication of the simplex tableau by the appropriate Gaussian pivot matrix. For example, consider the following initial feasible tableau: where the (, ) element is chosen as the pivot element. In this case, [ s =, t =, a = 4, α =, and b = 5, and so the corresponding Gaussian pivot matrix is I α a G = α = α. b I 5 3

14 Multiplying the simplex on the left by G gives = Repeating this process with the new pivot element in the (3, 3) position yields the Gaussian pivot matrix G =, and left multiplication by G gives = yielding the optimal tableau. If (.4) [ A I b c T is the initial tableau, then G G [ A I b c T = That is, we would be able to go directly from the initial tableau to the optimal tableau if we knew the matrix G = G G = = 5 4

15 beforehand. Moreover, the matrix G is invertible since both G and G are invertible: 4 4 G = G G = = (you should check that GG = I by doing the multiplication by hand). In general, every sequence of simplex pivots has a representation as left multiplication by some invertible matrix since pivoting corresponds to left multiplication of the tableau by a Gaussian pivot matrix, and Gaussian pivot matrices are always invertible. We now examine the consequence of this observation more closely in the general case. In this discussion, it is essential that we include the column associated with the objective variable z which we have largely ignored up to this point. Recall the initial simplex tableau, or augmented matrix associated with the system (D I ): [ T = A I b c T In this discussion, it is essential that we include the first column, i.e. the column associated with the objective variable z in the augmented matrix. Let the matrix [ Â R T k = b ĉ T y T ẑ be another simplex tableau obtained from the initial tableau after a series of k simplex pivots. Note that the first column remains unchanged. Indeed, the fact that simplex pivots do not alter the first column is the reason why we drop it in our hand computations. But in the discussion that follows its presence and the fact that it remains unchanged by simplex pivoting is very important. Since T k is another simplex tableau the m (n + m) matrix [Â R must posses among its columns the m columns of the m m identity matrix. These columns of the identity matrix correspond precisely to the basic variables associated with this tableau. Our prior discussion on Gaussian pivot matrices tells us that T k can be obtained from T by multiplying T on the left by some nonsingular (m+) (m+) matrix G where G is the product of a sequence of Gaussian pivot matrices. In order to better understand the action of G on T we need to decompose G into a block structure that is conformal with that of T : [ M u G = v T, β. 5

16 where M IR m m, u, v IR m, and β IR. Then [ Â R b ĉ T y T = T k ẑ = GT [ [ M u A I b = v T β c T [ u MA + uc T M Mb = β v T A + βc T v T v T b. By equating the blocks in the matrices on the far left and far right hand sides of this equation, we find from the first column that u = and β =. Here we see the key role played by our knowledge of the structure of the objective variable column (the first column). From the (, 3) and the (, 3) terms on the far left and right hand sides of (.5), we also find that M = R, and v = y. Putting all of this together gives the following representation of the k th tableau T k : [ [ [ R A I b RA R Rb (.5) T k = y T c T = c T y T A y T y T, b where the matrix R is necessarily invertible since the matrix [ R G = y T is invertible (prove this!): [ R G = y T R. (check by multiplying out GG ) The matrix R is called the record matrix for the tableau as it keeps track of all of the transformations required to obtain the new tableau. Again, the variables associated with the columns of the identity correspond to the basic variables. The tableau T k is said to be primal feasible, or just feasible, if b = Rb. We will return to this beautiful algebraic structure for the simplex algorithm later in the notes as it is the key ingredient in our understanding of the duality theory of linear programming. But first we address the question of whether or not the simplex algorithm works. That is, is the simplex algorithm a reliable methods for solving LPs? 6

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### Chapter 6. Linear Programming: The Simplex Method. Introduction to the Big M Method. Section 4 Maximization and Minimization with Problem Constraints

Chapter 6 Linear Programming: The Simplex Method Introduction to the Big M Method In this section, we will present a generalized version of the simplex method that t will solve both maximization i and

### Linear Programming. April 12, 2005

Linear Programming April 1, 005 Parts of this were adapted from Chapter 9 of i Introduction to Algorithms (Second Edition) /i by Cormen, Leiserson, Rivest and Stein. 1 What is linear programming? The first

### Linear Programming Notes V Problem Transformations

Linear Programming Notes V Problem Transformations 1 Introduction Any linear programming problem can be rewritten in either of two standard forms. In the first form, the objective is to maximize, the material

### Special Situations in the Simplex Algorithm

Special Situations in the Simplex Algorithm Degeneracy Consider the linear program: Maximize 2x 1 +x 2 Subject to: 4x 1 +3x 2 12 (1) 4x 1 +x 2 8 (2) 4x 1 +2x 2 8 (3) x 1, x 2 0. We will first apply the

### Linear Programming Problems

Linear Programming Problems Linear programming problems come up in many applications. In a linear programming problem, we have a function, called the objective function, which depends linearly on a number

### IEOR 4404 Homework #2 Intro OR: Deterministic Models February 14, 2011 Prof. Jay Sethuraman Page 1 of 5. Homework #2

IEOR 4404 Homework # Intro OR: Deterministic Models February 14, 011 Prof. Jay Sethuraman Page 1 of 5 Homework #.1 (a) What is the optimal solution of this problem? Let us consider that x 1, x and x 3

### OPRE 6201 : 2. Simplex Method

OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Sensitivity Analysis 3.1 AN EXAMPLE FOR ANALYSIS

Sensitivity Analysis 3 We have already been introduced to sensitivity analysis in Chapter via the geometry of a simple example. We saw that the values of the decision variables and those of the slack and

### Standard Form of a Linear Programming Problem

494 CHAPTER 9 LINEAR PROGRAMMING 9. THE SIMPLEX METHOD: MAXIMIZATION For linear programming problems involving two variables, the graphical solution method introduced in Section 9. is convenient. However,

### What is Linear Programming?

Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to

### Using the Simplex Method to Solve Linear Programming Maximization Problems J. Reeb and S. Leavengood

PERFORMANCE EXCELLENCE IN THE WOOD PRODUCTS INDUSTRY EM 8720-E October 1998 \$3.00 Using the Simplex Method to Solve Linear Programming Maximization Problems J. Reeb and S. Leavengood A key problem faced

### Linear Programming I

Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins

### Duality in Linear Programming

Duality in Linear Programming 4 In the preceding chapter on sensitivity analysis, we saw that the shadow-price interpretation of the optimal simplex multipliers is a very useful concept. First, these shadow

### Linear Programming for Optimization. Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc.

1. Introduction Linear Programming for Optimization Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc. 1.1 Definition Linear programming is the name of a branch of applied mathematics that

### Linear Programming in Matrix Form

Linear Programming in Matrix Form Appendix B We first introduce matrix concepts in linear programming by developing a variation of the simplex method called the revised simplex method. This algorithm,

### 1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### 3. Evaluate the objective function at each vertex. Put the vertices into a table: Vertex P=3x+2y (0, 0) 0 min (0, 5) 10 (15, 0) 45 (12, 2) 40 Max

SOLUTION OF LINEAR PROGRAMMING PROBLEMS THEOREM 1 If a linear programming problem has a solution, then it must occur at a vertex, or corner point, of the feasible set, S, associated with the problem. Furthermore,

### Linear Programming Notes VII Sensitivity Analysis

Linear Programming Notes VII Sensitivity Analysis 1 Introduction When you use a mathematical model to describe reality you must make approximations. The world is more complicated than the kinds of optimization

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### The Graphical Method: An Example

The Graphical Method: An Example Consider the following linear program: Maximize 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2 0, where, for ease of reference,

### 4.6 Linear Programming duality

4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal

### Chapter 2 Solving Linear Programs

Chapter 2 Solving Linear Programs Companion slides of Applied Mathematical Programming by Bradley, Hax, and Magnanti (Addison-Wesley, 1977) prepared by José Fernando Oliveira Maria Antónia Carravilla A

### 1.2 Solving a System of Linear Equations

1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### 7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### Solving Linear Programs

Solving Linear Programs 2 In this chapter, we present a systematic procedure for solving linear programs. This procedure, called the simplex method, proceeds by moving from one feasible solution to another,

### Solutions to Math 51 First Exam January 29, 2015

Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not

### LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS. 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method

LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method Introduction to dual linear program Given a constraint matrix A, right

### Review of Fundamental Mathematics

Review of Fundamental Mathematics As explained in the Preface and in Chapter 1 of your textbook, managerial economics applies microeconomic theory to business decision making. The decision-making tools

### Linear Programming. Solving LP Models Using MS Excel, 18

SUPPLEMENT TO CHAPTER SIX Linear Programming SUPPLEMENT OUTLINE Introduction, 2 Linear Programming Models, 2 Model Formulation, 4 Graphical Linear Programming, 5 Outline of Graphical Procedure, 5 Plotting

### Row Echelon Form and Reduced Row Echelon Form

These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Zeros of a Polynomial Function

Zeros of a Polynomial Function An important consequence of the Factor Theorem is that finding the zeros of a polynomial is really the same thing as factoring it into linear factors. In this section we

### EdExcel Decision Mathematics 1

EdExcel Decision Mathematics 1 Linear Programming Section 1: Formulating and solving graphically Notes and Examples These notes contain subsections on: Formulating LP problems Solving LP problems Minimisation

### Solving Systems of Linear Equations Using Matrices

Solving Systems of Linear Equations Using Matrices What is a Matrix? A matrix is a compact grid or array of numbers. It can be created from a system of equations and used to solve the system of equations.

PYTHAGOREAN TRIPLES KEITH CONRAD 1. Introduction A Pythagorean triple is a triple of positive integers (a, b, c) where a + b = c. Examples include (3, 4, 5), (5, 1, 13), and (8, 15, 17). Below is an ancient

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### 5.5. Solving linear systems by the elimination method

55 Solving linear systems by the elimination method Equivalent systems The major technique of solving systems of equations is changing the original problem into another one which is of an easier to solve

### 1 Review of Least Squares Solutions to Overdetermined Systems

cs4: introduction to numerical analysis /9/0 Lecture 7: Rectangular Systems and Numerical Integration Instructor: Professor Amos Ron Scribes: Mark Cowlishaw, Nathanael Fillmore Review of Least Squares

### SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89. by Joseph Collison

SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89 by Joseph Collison Copyright 2000 by Joseph Collison All rights reserved Reproduction or translation of any part of this work beyond that permitted by Sections

### 3.1 Solving Systems Using Tables and Graphs

Algebra 2 Chapter 3 3.1 Solve Systems Using Tables & Graphs 3.1 Solving Systems Using Tables and Graphs A solution to a system of linear equations is an that makes all of the equations. To solve a system

### Operation Research. Module 1. Module 2. Unit 1. Unit 2. Unit 3. Unit 1

Operation Research Module 1 Unit 1 1.1 Origin of Operations Research 1.2 Concept and Definition of OR 1.3 Characteristics of OR 1.4 Applications of OR 1.5 Phases of OR Unit 2 2.1 Introduction to Linear

### This exposition of linear programming

Linear Programming and the Simplex Method David Gale This exposition of linear programming and the simplex method is intended as a companion piece to the article in this issue on the life and work of George

### International Doctoral School Algorithmic Decision Theory: MCDA and MOO

International Doctoral School Algorithmic Decision Theory: MCDA and MOO Lecture 2: Multiobjective Linear Programming Department of Engineering Science, The University of Auckland, New Zealand Laboratoire

### Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.

Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry

### Lecture 1: Systems of Linear Equations

MTH Elementary Matrix Algebra Professor Chao Huang Department of Mathematics and Statistics Wright State University Lecture 1 Systems of Linear Equations ² Systems of two linear equations with two variables

### Methods for Finding Bases

Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### Lecture 2: August 29. Linear Programming (part I)

10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.

### 5.1 Radical Notation and Rational Exponents

Section 5.1 Radical Notation and Rational Exponents 1 5.1 Radical Notation and Rational Exponents We now review how exponents can be used to describe not only powers (such as 5 2 and 2 3 ), but also roots

### Linear Programming. Widget Factory Example. Linear Programming: Standard Form. Widget Factory Example: Continued.

Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.

### NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

### Chapter 4 Online Appendix: The Mathematics of Utility Functions

Chapter 4 Online Appendix: The Mathematics of Utility Functions We saw in the text that utility functions and indifference curves are different ways to represent a consumer s preferences. Calculus can

### A linear combination is a sum of scalars times quantities. Such expressions arise quite frequently and have the form

Section 1.3 Matrix Products A linear combination is a sum of scalars times quantities. Such expressions arise quite frequently and have the form (scalar #1)(quantity #1) + (scalar #2)(quantity #2) +...

### LINEAR INEQUALITIES. Mathematics is the art of saying many things in many different ways. MAXWELL

Chapter 6 LINEAR INEQUALITIES 6.1 Introduction Mathematics is the art of saying many things in many different ways. MAXWELL In earlier classes, we have studied equations in one variable and two variables

### Solution of Linear Systems

Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start

### MATH2210 Notebook 1 Fall Semester 2016/2017. 1 MATH2210 Notebook 1 3. 1.1 Solving Systems of Linear Equations... 3

MATH0 Notebook Fall Semester 06/07 prepared by Professor Jenny Baglivo c Copyright 009 07 by Jenny A. Baglivo. All Rights Reserved. Contents MATH0 Notebook 3. Solving Systems of Linear Equations........................

### Simplex method summary

Simplex method summary Problem: optimize a linear objective, subject to linear constraints 1. Step 1: Convert to standard form: variables on right-hand side, positive constant on left slack variables for

### Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix multiplication).

MAT 2 (Badger, Spring 202) LU Factorization Selected Notes September 2, 202 Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### Part 1 Expressions, Equations, and Inequalities: Simplifying and Solving

Section 7 Algebraic Manipulations and Solving Part 1 Expressions, Equations, and Inequalities: Simplifying and Solving Before launching into the mathematics, let s take a moment to talk about the words

### Linear Programming: Theory and Applications

Linear Programming: Theory and Applications Catherine Lewis May 11, 2008 1 Contents 1 Introduction to Linear Programming 3 1.1 What is a linear program?...................... 3 1.2 Assumptions.............................

### The last three chapters introduced three major proof techniques: direct,

CHAPTER 7 Proving Non-Conditional Statements The last three chapters introduced three major proof techniques: direct, contrapositive and contradiction. These three techniques are used to prove statements

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Solving Linear Systems, Continued and The Inverse of a Matrix

, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

### Equations, Inequalities & Partial Fractions

Contents Equations, Inequalities & Partial Fractions.1 Solving Linear Equations 2.2 Solving Quadratic Equations 1. Solving Polynomial Equations 1.4 Solving Simultaneous Linear Equations 42.5 Solving Inequalities

### 56:171 Operations Research Midterm Exam Solutions Fall 2001

56:171 Operations Research Midterm Exam Solutions Fall 2001 True/False: Indicate by "+" or "o" whether each statement is "true" or "false", respectively: o_ 1. If a primal LP constraint is slack at the

### Nonlinear Programming Methods.S2 Quadratic Programming

Nonlinear Programming Methods.S2 Quadratic Programming Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard A linearly constrained optimization problem with a quadratic objective

### MATH 551 - APPLIED MATRIX THEORY

MATH 55 - APPLIED MATRIX THEORY FINAL TEST: SAMPLE with SOLUTIONS (25 points NAME: PROBLEM (3 points A web of 5 pages is described by a directed graph whose matrix is given by A Do the following ( points

### Matrices 2. Solving Square Systems of Linear Equations; Inverse Matrices

Matrices 2. Solving Square Systems of Linear Equations; Inverse Matrices Solving square systems of linear equations; inverse matrices. Linear algebra is essentially about solving systems of linear equations,

### 1 Determinants and the Solvability of Linear Systems

1 Determinants and the Solvability of Linear Systems In the last section we learned how to use Gaussian elimination to solve linear systems of n equations in n unknowns The section completely side-stepped

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### LU Factorization Method to Solve Linear Programming Problem

Website: wwwijetaecom (ISSN 2250-2459 ISO 9001:2008 Certified Journal Volume 4 Issue 4 April 2014) LU Factorization Method to Solve Linear Programming Problem S M Chinchole 1 A P Bhadane 2 12 Assistant

### Lecture Notes 2: Matrices as Systems of Linear Equations

2: Matrices as Systems of Linear Equations 33A Linear Algebra, Puck Rombach Last updated: April 13, 2016 Systems of Linear Equations Systems of linear equations can represent many things You have probably

### 10.2 ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS. The Jacobi Method

578 CHAPTER 1 NUMERICAL METHODS 1. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS As a numerical technique, Gaussian elimination is rather unusual because it is direct. That is, a solution is obtained after

### 8.1. Cramer s Rule for Solving Simultaneous Linear Equations. Introduction. Prerequisites. Learning Outcomes. Learning Style

Cramer s Rule for Solving Simultaneous Linear Equations 8.1 Introduction The need to solve systems of linear equations arises frequently in engineering. The analysis of electric circuits and the control

### Linear Algebra and TI 89

Linear Algebra and TI 89 Abdul Hassen and Jay Schiffman This short manual is a quick guide to the use of TI89 for Linear Algebra. We do this in two sections. In the first section, we will go over the editing

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### EQUATIONS and INEQUALITIES

EQUATIONS and INEQUALITIES Linear Equations and Slope 1. Slope a. Calculate the slope of a line given two points b. Calculate the slope of a line parallel to a given line. c. Calculate the slope of a line

### EXCEL SOLVER TUTORIAL

ENGR62/MS&E111 Autumn 2003 2004 Prof. Ben Van Roy October 1, 2003 EXCEL SOLVER TUTORIAL This tutorial will introduce you to some essential features of Excel and its plug-in, Solver, that we will be using

### Typical Linear Equation Set and Corresponding Matrices

EWE: Engineering With Excel Larsen Page 1 4. Matrix Operations in Excel. Matrix Manipulations: Vectors, Matrices, and Arrays. How Excel Handles Matrix Math. Basic Matrix Operations. Solving Systems of

### LINEAR EQUATIONS IN TWO VARIABLES

66 MATHEMATICS CHAPTER 4 LINEAR EQUATIONS IN TWO VARIABLES The principal use of the Analytic Art is to bring Mathematical Problems to Equations and to exhibit those Equations in the most simple terms that

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

### General Framework for an Iterative Solution of Ax b. Jacobi s Method

2.6 Iterative Solutions of Linear Systems 143 2.6 Iterative Solutions of Linear Systems Consistent linear systems in real life are solved in one of two ways: by direct calculation (using a matrix factorization,

### Session 7 Bivariate Data and Analysis

Session 7 Bivariate Data and Analysis Key Terms for This Session Previously Introduced mean standard deviation New in This Session association bivariate analysis contingency table co-variation least squares

### Notes on Factoring. MA 206 Kurt Bryan

The General Approach Notes on Factoring MA 26 Kurt Bryan Suppose I hand you n, a 2 digit integer and tell you that n is composite, with smallest prime factor around 5 digits. Finding a nontrivial factor

### 0.1 Linear Programming

0.1 Linear Programming 0.1.1 Objectives By the end of this unit you will be able to: formulate simple linear programming problems in terms of an objective function to be maximized or minimized subject

### Solving simultaneous equations using the inverse matrix

Solving simultaneous equations using the inverse matrix 8.2 Introduction The power of matrix algebra is seen in the representation of a system of simultaneous linear equations as a matrix equation. Matrix

### 3. Mathematical Induction

3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)