# Chapter 15 Introduction to Linear Programming

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1

2 Brief History of Linear Programming The goal of linear programming is to determine the values of decision variables that maximize or minimize a linear objective function, where the decision variables are subject to linear constraints. A linear programming problem is a special case of a general constrained optimization problem. The objective function is linear, and the set of feasible points is determined by a set of linear equations and/or inequalities. 2

3 Brief History of Linear Programming The solution to a linear programming problem can be found by searching through a particular finite number of feasible points, known as basic feasible solutions. We can simply compare the basic feasible solutions and find one that minimizes or maximizes the objective function brute-force approach. An alternative approach is to use experienced planners to optimize this problem. Such an approach relies on heuristics. Heuristics come close, but give suboptimal answers. 3

4 Brief History of Linear Programming Efficient methods became available in the late 1930s. In 1939, Kantorovich presented a number of solutions to some problems related to production and transportation planning. During World War II, Koopmans contributed significantly to the solution of transportation problems. They were awarded a Nobel Prize in Economics in 1975 for their work on the theory of optimal allocation of resources. In 1947, Dantzig developed the simplex method. 4

5 Brief History of Linear Programming The simplex method has the undesirable property that in the worst case, the number of steps required to find a solution grows exponentially with the number of variables. Thus, the simplex method is said to have exponential worst-case complexity. Khachiyan, in 1979, devise a polynomial complexity algorithm. In 1984, Karmarkar proposed a new linear programming algorithm that has polynomial complexity and appears to solve some complicated real-world problems of scheduling, routing, and planning more efficiently than the simplex method. His work led to the development of many interior-point methods. This approach is currently still an active research area. 5

6 Simple Examples of Linear Programs Formally, a linear program is an optimization problem of the form where. The vector inequality means that each component of is nonnegative. Several variations of this problem are possible. For example, we can maximize, or the constraints may be in the form of inequalities, such as or. In fact, these variations can all be rewritten into the standard form shown above. 6

7 A manufacturer produces four different products: there are three inputs to this production process: labor in person-weeks, kilograms of raw material A, and boxes of raw material B. Each product has different input requirements. In determining each week s production schedule, the manufacturer cannot use more than the available amounts of labor and the two raw materials. The relevant information is presented in this table. Every production decision must satisfy the restrictions on the availability of inputs. These constraints can be written using the data in this table. 7

8 Because negative production levels are not meaningful, we must impose the following nonnegativity constraints on the production levels: Now, suppose that one unit of product sells for \$6, and sell for \$4, \$7, \$5, respectively. Then, the total revenue for any production decision is The problem is then to maximize subject to the given constraints (the three inequalities and four nonnegativity constraints). 8

9 Using vector notation with, the problem can be written in the compact form where 9

10 Diet Problem. Assume that different food types are available. The th food sells at a price per unit. In addition, there are basic nutrients. To achieve a balanced diet, you must receive at least units of the th nutrient per day. Assume that each unit of food contains units of the th nutrient. Denote by the number of units of food in the diet. The objective is to select the to minimize the total cost of the diet: subject to the nutritional constraints, and the nonnegativity constraints 10

11 In the more compact vector notation, this problem becomes where is an -dimensional column vector, is an -dimensional row vector, is an matrix, and is an -dimensional column vector. 11

12 A manufacturer produces two different products,, using three machines:. Each machine can be used for only a limited amount of time. Production times of each product on each machine are given in this table. The objective is to maximize the combined time of utilization of all three machines. Every production decision must satisfy the constraints on the available time. These restrictions can be written as 12

13 The combined production time of all three machines is Thus, writing, the problem in compact notation has the form where 13

14 A manufacturing companying has plants in cities A, B, and C. The company produces and distributes its product to dealers in various cities. On a particular day, the company has 30 units of its product in A, 40 in B, and 30 in C. The company plans to ship 20 units to D, 20 to E, 25 to F, and 35 to G, following orders received from dealers. The transportation costs per unit of each product between the cities are given in this table. In the table, the quantities supplied and demand appear at the right and along the bottom of the table. The quantities to be transported from the plants to different destinations are represented by the decision variables. 14

15 The problem can be stated in the form 15

16 In this problem one of the constraint equations is redundant because it can be derived from the rest of the constraint equations. The mathematical formulation of the transportation problem is then in a linear programming form with twelve decision variables and six linearly independent constraint equations. Obviously, we also require nonnegativity of the decision variables, since a negative shipment is impossible and does not have a valid interpretation. 16

17 An electric circuit is designed to use a 30-V source to charge 10-V, 6-V, and 20-V batteries connected in parallel. Physical constraints limit the currents to a maximum of 4A, 3A, 3A, 2A, and 2A, respectively. In addition, the batteries must not be discharged; that is, the currents must not be negative. We wish to find the values of the currents such that the total power transferred to the batteries is maximized. 17

18 The total power transferred to the batteries is the sum of the powers transferred to each battery and is given by W. From the circuit, we observe that the currents satisfy the constraints and. Therefore, the problem can be posed as the following linear program: 18

19 Consider the wireless communication system. There are mobile users. For each, user transmits a signal to the base station with power and an attenuation factor of (i.e., the actual signal power received at the base station from user is ). When the base station is receiving from user the total power received from all other users is considered interference (i.e., ). For the communication with user to be reliable, the signal-to-interference ratio must exceed a threshold, where the signal is the power received from user. 19

20 We are interested in minimizing the total power transmitted by all users subject to having reliable communication for all users. We can formulate the problem as a linear programming problem of the form The total power transmitted is. The signal-tointerference ratio for user 20

21 Hence, the problem can be written as We can write the above as the linear programming problem In matrix form, 21

22 Two-Dimensional Linear Programs Consider the following linear program where and First, we note that the set of equations specifies a family of straight lines in. Each member of this family can be obtained by setting equal to some real number. Thus, for example,,, and are three parallel lines belonging to the family. 22

23 Two-Dimensional Linear Programs Now, suppose that we try to choose several values for and and observe how large we can make while still satisfying the constraints on and. We first try and. This point satisfies the constraints. For this point,. If we now select and, then and this point yields a larger value for than does. There are infinitely many points satisfying the constraints. Therefore, we need a better method than trial and error to solve the problem. 23

24 Two-Dimensional Linear Programs For the example above we can easily solve the problem using geometric arguments. First let us sketch the constraints in. The region of feasible points (the set of points satisfying the constraints, ) is depicted by the shaded region in this figure. Geometrically, maximizing subject to the constraints can be thought of as finding the straight line that intersects the shaded region and has the largest. The point is the solution. 24

25 Suppose that you are given two different types of concrete. The first type contains 30% cement, 40% gravel, and 30% sand (all percentages of weight). The second type contains 10% cement, 20% gravel, and 70% sand. The first type of concrete costs \$5 per pound and the second type costs \$1 per pound. How many pounds of each type of concrete should you buy and mix together so that your cost is minimized but you get a concrete mixture that has at least a total of 5 pounds of cement, 3 pounds of gravel, and 4 pounds of sand? 25

26 The problem can be represented as Using the graphical method described above, we get a solution of, which means that we would purchase 50 pounds of the second type of concrete. In some case, there may be more than one point of intersection, and therefore any one of them is a solution. 26

27 Convex Polyhedra and Linear Programming We discuss linear programs from a geometric point of view. The set of points satisfying these constraints can be represented as the intersection of a finite number of closed half-spaces. Thus, the constraints define a convex polytope. We assume, for simplicity, that this polytope is nonempty and bounded. In other words, the equations of constraints define a polyhedron in. Let be a hyperplane of support of this polyhedron. If the dimension of is less than, then the set of all points common to the hyperplane and the polyhedron coincides with. 27

28 Convex Polyhedra and Linear Programming If the dimension of is equal to, then the set of all points common to the hyperplane and the polyhedron is a face of the polyhedron. If this face is -dimensional, then there exists only one hyperplane of support, namely, the carrier of this face. If the dimension of the face is less than, then there exist an infinite number of hyperplanes of support whose intersection with this polyhedron yields this face. 28

29 Convex Polyhedra and Linear Programming The goal of our linear programming problem is to maximize a linear objective function on the convex polyhedron. Next, let be the hyperplane defined by the equation. Draw a hyperplane of support to the polyhedron, which is parallel to and positioned such that the vector points in the direction of the half-space that does not contain. 29

30 Convex Polyhedra and Linear Programming The equation of the hyperplane has the form, and for all, we have. Denote by the convex polyhedron that is the intersection of the hyperplane of support with the polyhedron. We now show that is constant on and that is the set of all points in for which attains its maximum value. To this end, let and be two arbitrary points in. This implies that both and belong to. Hence, which means that is constant on. 30

31 Convex Polyhedra and Linear Programming Let be a point of, and let be a point of ; that is, is a point of that does not belong to. Then, which implies that Thus, the values of at the points of that do not belong to are smaller than the values at points of. Hence, achieves its maximum on at points in. 31

32 Convex Polyhedra and Linear Programming It may happen that contains only a single point, in which case achieves its maximum at a unique point. This occurs when the hyperplane of support passes through an extreme point of. 32

33 Standard Form Linear Programs We refer to a linear program of the form as a linear program in standard form. Here matrix composed of real entries,, is an Without loss of generality, we assume that. If a component of is negative, say the th component, we multiply the th constraint by -1 to obtain a positive right-hand side. 33

34 Standard Form Linear Programs Theorems and solution techniques for linear programs are usually stated for problems in standard form. Other forms of linear programs can be converted to the standard form. If a linear program is in the form then by introducing surplus variables original problem into the standard form, we can convert the 34

35 Standard Form Linear Programs In more compact notation, the formulation above can be represented as where is the identity matrix. 35

36 Standard Form Linear Programs If, on the other hand, the constraints have the form then we introduce slack variables into the form to convert the constraints where is the vector of slack variables. Note that neither surplus nor slack variables contribute to the objective function 36

37 Standard Form Linear Programs At first glace, it may appear that the two problems are different, in that the first problem refers to the intersection of half-spaces in the -dimensional space, whereas the second problem refers to an intersection of half-spaces and hyperplanes in the -dimensional space. It turns out that both formulations are algebraically equivalent in the sense that a solution to one of the problems implies a solution to the other. 37

38 Suppose that we are given the inequality constraint. We convert this to an equality constraint by introducing a slack variable to obtain Consider the sets and Are the sets and equal? It is clear that indeed they are; we give a geometric interpretation for their equality. Consider a third set. From this figure we can see that the set consists of all points on the line to the left and above the point of intersection of the line with the -axis. 38

39 This set, being a subset of, is of course not the same set as the set (a subset of ). However, we can project the set onto the -axis. We can associate with each point a point on the orthogonal projection of onto the -axis, and vice versa. Note that 39

40 Consider the inequality constraints where are positive numbers. Again, we introduce a slack variable to get Define the sets We again see that is not the same as. However, the orthogonal projection of onto the -plane allows us to associate the resulting set with the set. 40

41 We associate the points resulting from the orthogonal projection of onto the -plane with the points in. Note that 41

42 Suppose that we wish to maximize subject to the constraints where, for simplicity, we assume that each and The set of feasible points is depicted in this figure. Let be the set of points satisfying the constraints. 42

43 Introducing a slack variable, we convert the constraints into standard form: Let be the set of points satisfying the constraints. As illustrated in this figure, this set is a line segment (in ). We now project onto the -plane. The projected set consists of the points, with for some. In this figure this set is marked by a heavy line in the -plane. We can associate the points on the projection with the corresponding points in the set 43

44 Consider the following optimization problem To convert the problem into a standard form linear programming problem, we perform the following steps: 1. Change the objective function to: 2. Substitute 3. Write as and 4. Introduce slack variables, and convert the inequalities above to and 5. Write 44

45 Hence, we obtain 45

46 Basic Solutions In the following discussion we only consider linear programming problems in standard form. Consider the system of equalities, where In dealing with this system of equations, we frequently need to consider a subset of columns of the matrix. For convenience, we often reorder the columns of so that the columns we are interested in appear first. Specifically, let be a square matrix whose columns are linearly independent columns of. 46

47 Basic Solutions If necessary, we reorder the columns of so that the columns in appear first: has the form, where is an matrix whose columns are the remaining columns of. The matrix is nonsingular, and thus we can solve the equation for the -vector. The solution is. Let be the -vector whose first components are equal to and the remaining components are equal to zero; that is,. Then, is a solution to 47

48 Basic Solutions Definition 15.1: We call a basic solution to with respect to the basis. We refer to the components of the vector as basic variables and the columns of as basic columns. If some of the basic variables of a basic solution are zero, then the basic solution is said to be a degenerate basic solution. A vector satisfying,, is said to be a feasible solution. A feasible solution that is also basic is called a basic feasible solution. If the basic feasible solution is a degenerate basic solution, then it is called a degenerate basic feasible solution. Note that in any basic feasible solution, 48

49 Consider the equation where denotes the th column of the matrix Then, is a basic feasible solution with respect to the basis, is a degenerate basic feasible solution with respect to the basis (as well as and ), is a feasible solution that is not basic, and is a basic solution with respect to the basis, but is not feasible. 49

50 Consider the system of linear equations, where We now find all solutions of this system. Note that every solution of has the form, where is a particular solution of and is a solution to We form the augmented matrix of the system Using elementary row operations, we transform this matrix into the form given by 50

51 The corresponding system of equations is given by Solving for the leading unknowns and, we obtain where and are arbitrary real numbers. If is a solution, then we have 51 where we have substituted and for and, respectively.

52 Using vector notation, we may write the system of equations above as Note that we have infinitely many solutions, parameterized by. For the choice we obtain a particular solution to, given by 52

53 Any other solution has the form, where The total number of possible basic solution is at most to find basic solutions that are feasible, we check each of the basic solutions for feasibility. 53

54 Our first candidate for a basic feasible solution is obtained by setting, which corresponds to the basis Solving, we obtain, and hence is a basic solution that is not feasible. For our second candidate basic feasible solution, we set. We have the basis. Solving yields. Hence, is a basic feasible solution. A third candidate basic feasible solution is obtained by setting. However, the matrix is singular. Therefore, cannot be a basis, and we do not have a basic solution corresponding to 54

55 We get our fourth candidate for a basic solution by setting. We have a basis, resulting in, which is a basic feasible solution. Our fifth candidate for a basic feasible solution corresponds to setting, with the basis This results in, which is a basic solution that is not feasible. Finally, the sixth candidate for a basic feasible solution is obtained by setting. This results in the basis, and, which is a basic solution but is not feasible. 55

56 Properties of Basic Solutions Definition 15.2: Any vector that yields the minimum value of the objective function over the set of vectors satisfying the constraints, is said to be an optimal feasible solution. An optimal feasible solution that is basic is said to be an optimal basic feasible solution. Theorem 15.1: Fundamental Theorem of LP. Consider a linear program in standard form 1. If there exists a feasible solution, then there exists a basic feasible solution 2. If there exists an optimal feasible solution, then there exists an optimal basic feasible solution. 56

57 Proof of Theorem 15.1 Suppose that is a feasible solution and it has positive components. Without loss of generality, we can assume that the first components are positive, whereas the remaining components are zero. Then, in terms of the columns of, this solution satisfies There are now two cases to consider. Case 1: If are linearly independent, then. If, then the solution is basic and the proof is done. If then, since, we can find columns of from the remaining columns so that the resulting set of columns forms a basis. Hence, the solution is a (degenerate) basic feasible solution corresponding to the basis above. 57

58 Proof of Theorem 15.1 Case 2: Assume that are linearly dependent. Then, there exist numbers, not all zero, such that We can assume that there exists at least one that is positive, for if all the are nonpositive, we can multiply the equation above by -1. Multiply the equation by a scalar and subtract the resulting equation from to obtain Let. Then, for any we can write 58

59 Proof of Theorem 15.1 Let. Then, the first components of are nonnegative, and at least one of these components is zero. We then have a feasible solution with at most positive components. We can repeat this process until we get linearly independent columns of, after which we are back to case 1. Therefore, part 1 is proved. 59

60 Proof of Theorem 15.1 We now prove part 2. Suppose that is an optimal feasible solution and only the first variables are nonzero. Then, we have two cases to consider. The first case is exactly the same as part 1. The second case follows the same arguments as in part 1, but in addition we must show that is optimal for any. We do this by showing that. To this end, assume that Note that for of sufficiently small magnitude, the vector is feasible. We can choose such that. This contradicts the optimality of. We can now use the procedure from part 1 to obtain an optimal basic feasible solution from a given optimal feasible solution. 60

61 Consider the system of equations Find a nonbasic feasible solution to this system and use the method in the proof of the fundamental theorem of LP to find a basic feasible solution. Recall that solutions for the system have the form where. Note that if, then 61

62 There are constants, such that For example, let. Note that where If, then is a basic feasible solution. 62

63 Properties of Basic Solutions Observe that the fundamental theorem of LP reduces the task of solving a linear programming problem to that of searching over a finite number of basic feasible solution. That is, we need only check basic feasible solutions for optimality. As mentioned before, the total number of basic solutions is at most Although this number is finite, it may be quite large. For example: Therefore, a more efficient method of solving linear programs is needed. To this end, we next analyze a geometric interpretation of the fundamental theorem of LP. 63

64 Geometric View of Linear Programs Recall that a set is said to be convex if, for every and every real number, the point In other words, a set is convex if given two points in the set, every point on the linear segment joining these two points is also a member of the set. Note that the set of points satisfying the constraints is convex. To see this, let satisfy the constraints. Then, for all, Also, for, we have 64

65 Geometric View of Linear Programs Recall that a point in a convex set is said to be an extreme point of if there are no two distinct points and in such that for some. In other words, an extreme point is a point that does not lie strictly within the line segment connecting two other points of the set. Therefore, if is an extreme point, and for some and, then. In the following theorem we show that extreme points of the constraint set are equivalent to basic feasible solutions. 65

66 Geometric View of Linear Programs Theorem 15.2: Let be the convex set consisting of all feasible solutions, that is, all -vector satisfying where,. Then, is an extreme point of if and only if is a basic feasible solution to Form this theorem, it follows that the set of extreme points of the constraint set is equal to the set of basic feasible solutions. Combining this observation with the fundamental theorem of LP (Theorem 15.1), we can see that in solving linear programming problems we need only examine the extreme points of the constraint set. 66

67 Consider the following LP problem We introduce slack variables to convert this LP problem into standard form 67

68 In the remainder of the example we consider only the problem in standard form. We can represent the constraints above as that is,. Note that is a feasible solution. But for this, the value of the objective function is zero. We already know that the minimum of the objective function (if it exists) is achieved at an extreme point of the constraint set defined by the constraints. The point is an extreme point of the set of feasible solutions, but it turns out that it does not minimize the objective function. 68

69 Therefore, we need to seek the solution among the other extreme points. To do this we move from one extreme point to an adjacent extreme point such that the value of the objective function decreases. Here, we define two extreme points to be adjacent if the corresponding basic columns differ by only one vector. We begin with. We have. To select an adjacent extreme point, let us choose to include as a basic column in the new basis. We need to remove either,, or from the old basis. 69

70 We first express as a linear combination of the old basic columns:. Multiplying both sides of this equation by, we get We now add this equation to the equation. Collecting terms yields We want to choose in such a way that each of the coefficients above is nonnegative and at the same time, one of the coefficients,, or becomes zero. Clearly, does the job. The result is. The corresponding basic feasible solution (extreme point) is. For this solution, the objective function value is -30, which is an improvement relative to the objective function value at the old extreme point. 70

71 We now apply the same procedure as above to move to another adjacent extreme point, which hopefully further decreases the value of the objective function. This time, we choose to enter the new basis. We have and Substituting, we obtain The solution is and the corresponding value of the objective function is -44, which is smaller than the value at the previous extreme point. 71

72 To complete the example we repeat the procedure once more. This time, we select and express it as a combination of the vectors in the previous basis,,, and : and hence The largest permissible value for is 3. The corresponding basic feasible solution is, with an objective function of -50. The solution turns out to be an optimal solution to our problem in standard form. Hence, the solution to the original problem is, which we can easily obtain graphically. 72

73 The technique used in this example for moving from one extreme point to an adjacent extreme point is also used in the simplex method for solving LP problems. The simplex method is essentially a refined method of performing these manipulations. 73

### Linear Programming I

Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### What is Linear Programming?

Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to

### Special Situations in the Simplex Algorithm

Special Situations in the Simplex Algorithm Degeneracy Consider the linear program: Maximize 2x 1 +x 2 Subject to: 4x 1 +3x 2 12 (1) 4x 1 +x 2 8 (2) 4x 1 +2x 2 8 (3) x 1, x 2 0. We will first apply the

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### Linear Programming. Solving LP Models Using MS Excel, 18

SUPPLEMENT TO CHAPTER SIX Linear Programming SUPPLEMENT OUTLINE Introduction, 2 Linear Programming Models, 2 Model Formulation, 4 Graphical Linear Programming, 5 Outline of Graphical Procedure, 5 Plotting

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### The Graphical Method: An Example

The Graphical Method: An Example Consider the following linear program: Maximize 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2 0, where, for ease of reference,

### Linear Programming Notes V Problem Transformations

Linear Programming Notes V Problem Transformations 1 Introduction Any linear programming problem can be rewritten in either of two standard forms. In the first form, the objective is to maximize, the material

### 1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

### OPRE 6201 : 2. Simplex Method

OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2

### Linear Programming. Widget Factory Example. Linear Programming: Standard Form. Widget Factory Example: Continued.

Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### Lecture 2: August 29. Linear Programming (part I)

10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

### Solving Linear Programs

Solving Linear Programs 2 In this chapter, we present a systematic procedure for solving linear programs. This procedure, called the simplex method, proceeds by moving from one feasible solution to another,

### Linear Programming for Optimization. Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc.

1. Introduction Linear Programming for Optimization Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc. 1.1 Definition Linear programming is the name of a branch of applied mathematics that

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### Chapter 6. Linear Programming: The Simplex Method. Introduction to the Big M Method. Section 4 Maximization and Minimization with Problem Constraints

Chapter 6 Linear Programming: The Simplex Method Introduction to the Big M Method In this section, we will present a generalized version of the simplex method that t will solve both maximization i and

### Standard Form of a Linear Programming Problem

494 CHAPTER 9 LINEAR PROGRAMMING 9. THE SIMPLEX METHOD: MAXIMIZATION For linear programming problems involving two variables, the graphical solution method introduced in Section 9. is convenient. However,

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### 3 Does the Simplex Algorithm Work?

Does the Simplex Algorithm Work? In this section we carefully examine the simplex algorithm introduced in the previous chapter. Our goal is to either prove that it works, or to determine those circumstances

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Linear Programming: Theory and Applications

Linear Programming: Theory and Applications Catherine Lewis May 11, 2008 1 Contents 1 Introduction to Linear Programming 3 1.1 What is a linear program?...................... 3 1.2 Assumptions.............................

### This exposition of linear programming

Linear Programming and the Simplex Method David Gale This exposition of linear programming and the simplex method is intended as a companion piece to the article in this issue on the life and work of George

### Simplex method summary

Simplex method summary Problem: optimize a linear objective, subject to linear constraints 1. Step 1: Convert to standard form: variables on right-hand side, positive constant on left slack variables for

### Duality in Linear Programming

Duality in Linear Programming 4 In the preceding chapter on sensitivity analysis, we saw that the shadow-price interpretation of the optimal simplex multipliers is a very useful concept. First, these shadow

### 4.6 Linear Programming duality

4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal

### 3. Evaluate the objective function at each vertex. Put the vertices into a table: Vertex P=3x+2y (0, 0) 0 min (0, 5) 10 (15, 0) 45 (12, 2) 40 Max

SOLUTION OF LINEAR PROGRAMMING PROBLEMS THEOREM 1 If a linear programming problem has a solution, then it must occur at a vertex, or corner point, of the feasible set, S, associated with the problem. Furthermore,

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### Chapter 2 Solving Linear Programs

Chapter 2 Solving Linear Programs Companion slides of Applied Mathematical Programming by Bradley, Hax, and Magnanti (Addison-Wesley, 1977) prepared by José Fernando Oliveira Maria Antónia Carravilla A

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### 7.4 Linear Programming: The Simplex Method

7.4 Linear Programming: The Simplex Method For linear programming problems with more than two variables, the graphical method is usually impossible, so the simplex method is used. Because the simplex method

### SECTION 8-1 Systems of Linear Equations and Augmented Matrices

86 8 Systems of Equations and Inequalities In this chapter we move from the standard methods of solving two linear equations with two variables to a method that can be used to solve linear systems with

### Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.

Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

### Using the Simplex Method to Solve Linear Programming Maximization Problems J. Reeb and S. Leavengood

PERFORMANCE EXCELLENCE IN THE WOOD PRODUCTS INDUSTRY EM 8720-E October 1998 \$3.00 Using the Simplex Method to Solve Linear Programming Maximization Problems J. Reeb and S. Leavengood A key problem faced

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### . P. 4.3 Basic feasible solutions and vertices of polyhedra. x 1. x 2

4. Basic feasible solutions and vertices of polyhedra Due to the fundamental theorem of Linear Programming, to solve any LP it suffices to consider the vertices (finitely many) of the polyhedron P of the

### Linear Programming. April 12, 2005

Linear Programming April 1, 005 Parts of this were adapted from Chapter 9 of i Introduction to Algorithms (Second Edition) /i by Cormen, Leiserson, Rivest and Stein. 1 What is linear programming? The first

### Solving Simultaneous Equations and Matrices

Solving Simultaneous Equations and Matrices The following represents a systematic investigation for the steps used to solve two simultaneous linear equations in two unknowns. The motivation for considering

### Vector Spaces II: Finite Dimensional Linear Algebra 1

John Nachbar September 2, 2014 Vector Spaces II: Finite Dimensional Linear Algebra 1 1 Definitions and Basic Theorems. For basic properties and notation for R N, see the notes Vector Spaces I. Definition

### 9.4 THE SIMPLEX METHOD: MINIMIZATION

SECTION 9 THE SIMPLEX METHOD: MINIMIZATION 59 The accounting firm in Exercise raises its charge for an audit to \$5 What number of audits and tax returns will bring in a maximum revenue? In the simplex

### Recall that two vectors in are perpendicular or orthogonal provided that their dot

Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### Linear Programming Problems

Linear Programming Problems Linear programming problems come up in many applications. In a linear programming problem, we have a function, called the objective function, which depends linearly on a number

### We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.

Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to

### Sensitivity Analysis 3.1 AN EXAMPLE FOR ANALYSIS

Sensitivity Analysis 3 We have already been introduced to sensitivity analysis in Chapter via the geometry of a simple example. We saw that the values of the decision variables and those of the slack and

### Math 215 HW #1 Solutions

Math 25 HW # Solutions. Problem.2.3. Describe the intersection of the three planes u+v+w+z = 6 and u+w+z = 4 and u + w = 2 (all in four-dimensional space). Is it a line or a point or an empty set? What

### Linear Algebra Notes

Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note

### 4.5 Linear Dependence and Linear Independence

4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then

### Operation Research. Module 1. Module 2. Unit 1. Unit 2. Unit 3. Unit 1

Operation Research Module 1 Unit 1 1.1 Origin of Operations Research 1.2 Concept and Definition of OR 1.3 Characteristics of OR 1.4 Applications of OR 1.5 Phases of OR Unit 2 2.1 Introduction to Linear

### Module1. x 1000. y 800.

Module1 1 Welcome to the first module of the course. It is indeed an exciting event to share with you the subject that has lot to offer both from theoretical side and practical aspects. To begin with,

### These axioms must hold for all vectors ū, v, and w in V and all scalars c and d.

DEFINITION: A vector space is a nonempty set V of objects, called vectors, on which are defined two operations, called addition and multiplication by scalars (real numbers), subject to the following axioms

### Duality of linear conic problems

Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### Converting a Linear Program to Standard Form

Converting a Linear Program to Standard Form Hi, welcome to a tutorial on converting an LP to Standard Form. We hope that you enjoy it and find it useful. Amit, an MIT Beaver Mita, an MIT Beaver 2 Linear

### Basic Terminology for Systems of Equations in a Nutshell. E. L. Lady. 3x 1 7x 2 +4x 3 =0 5x 1 +8x 2 12x 3 =0.

Basic Terminology for Systems of Equations in a Nutshell E L Lady A system of linear equations is something like the following: x 7x +4x =0 5x +8x x = Note that the number of equations is not required

### Section 1.1. Introduction to R n

The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to

### MATH2210 Notebook 1 Fall Semester 2016/2017. 1 MATH2210 Notebook 1 3. 1.1 Solving Systems of Linear Equations... 3

MATH0 Notebook Fall Semester 06/07 prepared by Professor Jenny Baglivo c Copyright 009 07 by Jenny A. Baglivo. All Rights Reserved. Contents MATH0 Notebook 3. Solving Systems of Linear Equations........................

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### 7.1 Modelling the transportation problem

Chapter Transportation Problems.1 Modelling the transportation problem The transportation problem is concerned with finding the minimum cost of transporting a single commodity from a given number of sources

### Nonlinear Programming Methods.S2 Quadratic Programming

Nonlinear Programming Methods.S2 Quadratic Programming Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard A linearly constrained optimization problem with a quadratic objective

### Can linear programs solve NP-hard problems?

Can linear programs solve NP-hard problems? p. 1/9 Can linear programs solve NP-hard problems? Ronald de Wolf Linear programs Can linear programs solve NP-hard problems? p. 2/9 Can linear programs solve

### Chapter 6. Cuboids. and. vol(conv(p ))

Chapter 6 Cuboids We have already seen that we can efficiently find the bounding box Q(P ) and an arbitrarily good approximation to the smallest enclosing ball B(P ) of a set P R d. Unfortunately, both

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### Linear Programming Supplement E

Linear Programming Supplement E Linear Programming Linear programming: A technique that is useful for allocating scarce resources among competing demands. Objective function: An expression in linear programming

### MATHEMATICS (CLASSES XI XII)

MATHEMATICS (CLASSES XI XII) General Guidelines (i) All concepts/identities must be illustrated by situational examples. (ii) The language of word problems must be clear, simple and unambiguous. (iii)

### Diagonalisation. Chapter 3. Introduction. Eigenvalues and eigenvectors. Reading. Definitions

Chapter 3 Diagonalisation Eigenvalues and eigenvectors, diagonalisation of a matrix, orthogonal diagonalisation fo symmetric matrices Reading As in the previous chapter, there is no specific essential

### An Introduction to Linear Programming

An Introduction to Linear Programming Steven J. Miller March 31, 2007 Mathematics Department Brown University 151 Thayer Street Providence, RI 02912 Abstract We describe Linear Programming, an important

### Figure 1.1 Vector A and Vector F

CHAPTER I VECTOR QUANTITIES Quantities are anything which can be measured, and stated with number. Quantities in physics are divided into two types; scalar and vector quantities. Scalar quantities have

### 9 Multiplication of Vectors: The Scalar or Dot Product

Arkansas Tech University MATH 934: Calculus III Dr. Marcel B Finan 9 Multiplication of Vectors: The Scalar or Dot Product Up to this point we have defined what vectors are and discussed basic notation

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### Linear Programming in Matrix Form

Linear Programming in Matrix Form Appendix B We first introduce matrix concepts in linear programming by developing a variation of the simplex method called the revised simplex method. This algorithm,

### POWER SETS AND RELATIONS

POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty

### MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column

### Unified Lecture # 4 Vectors

Fall 2005 Unified Lecture # 4 Vectors These notes were written by J. Peraire as a review of vectors for Dynamics 16.07. They have been adapted for Unified Engineering by R. Radovitzky. References [1] Feynmann,

### Row Echelon Form and Reduced Row Echelon Form

These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation

### 26 Linear Programming

The greatest flood has the soonest ebb; the sorest tempest the most sudden calm; the hottest love the coldest end; and from the deepest desire oftentimes ensues the deadliest hate. Th extremes of glory

### Solving Linear Systems, Continued and The Inverse of a Matrix

, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

### 4 UNIT FOUR: Transportation and Assignment problems

4 UNIT FOUR: Transportation and Assignment problems 4.1 Objectives By the end of this unit you will be able to: formulate special linear programming problems using the transportation model. define a balanced

### Data Envelopment Analysis: A Primer for Novice Users and Students at all Levels

Data Envelopment Analysis: A Primer for Novice Users and Students at all Levels R. Samuel Sale Lamar University Martha Lair Sale Florida Institute of Technology In the three decades since the publication

### LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS. 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method

LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method Introduction to dual linear program Given a constraint matrix A, right

### Definition: A square matrix A is block diagonal if A has the form A 1 O O O A 2 O A =

The question we want to answer now is the following: If A is not similar to a diagonal matrix, then what is the simplest matrix that A is similar to? Before we can provide the answer, we will have to introduce

### Chapter 4. Duality. 4.1 A Graphical Example

Chapter 4 Duality Given any linear program, there is another related linear program called the dual. In this chapter, we will develop an understanding of the dual linear program. This understanding translates

### Lecture 2. Marginal Functions, Average Functions, Elasticity, the Marginal Principle, and Constrained Optimization

Lecture 2. Marginal Functions, Average Functions, Elasticity, the Marginal Principle, and Constrained Optimization 2.1. Introduction Suppose that an economic relationship can be described by a real-valued

### Solutions to Math 51 First Exam January 29, 2015

Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not

### 4: SINGLE-PERIOD MARKET MODELS

4: SINGLE-PERIOD MARKET MODELS Ben Goldys and Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2015 B. Goldys and M. Rutkowski (USydney) Slides 4: Single-Period Market

### THE FUNDAMENTAL THEOREM OF ARBITRAGE PRICING

THE FUNDAMENTAL THEOREM OF ARBITRAGE PRICING 1. Introduction The Black-Scholes theory, which is the main subject of this course and its sequel, is based on the Efficient Market Hypothesis, that arbitrages

### Lecture L3 - Vectors, Matrices and Coordinate Transformations

S. Widnall 16.07 Dynamics Fall 2009 Lecture notes based on J. Peraire Version 2.0 Lecture L3 - Vectors, Matrices and Coordinate Transformations By using vectors and defining appropriate operations between

### Lecture 7: Finding Lyapunov Functions 1

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.243j (Fall 2003): DYNAMICS OF NONLINEAR SYSTEMS by A. Megretski Lecture 7: Finding Lyapunov Functions 1

### UNIT 1 LINEAR PROGRAMMING

OUTLINE Session : Session 2: Session 3: Session 4: Session 5: Session 6: Session 7: Session 8: Session 9: Session 0: Session : Session 2: UNIT LINEAR PROGRAMMING Introduction What is Linear Programming

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

### 5.3 The Cross Product in R 3

53 The Cross Product in R 3 Definition 531 Let u = [u 1, u 2, u 3 ] and v = [v 1, v 2, v 3 ] Then the vector given by [u 2 v 3 u 3 v 2, u 3 v 1 u 1 v 3, u 1 v 2 u 2 v 1 ] is called the cross product (or

### Solving Linear Diophantine Matrix Equations Using the Smith Normal Form (More or Less)

Solving Linear Diophantine Matrix Equations Using the Smith Normal Form (More or Less) Raymond N. Greenwell 1 and Stanley Kertzner 2 1 Department of Mathematics, Hofstra University, Hempstead, NY 11549