1 LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method
2 Introduction to dual linear program Given a constraint matrix A, right hand side vector b, and cost vector c, we have a corresponding linear programming problem: Questions: 1. Can we use the same data set of (A, b, c) to construct another linear programming problem? 2. If so, how is this new linear program related to the original primal problem?
3 Guessing A is an m by n matrix, b is an m-vector, c is an n-vector. Can the roles of b and c can be switched? - If so, we ll have m variables and n constraints. Correspondingly, (1) the transpose of matrix A should be considered to accommodate rows for columns, and vise versa? (2) nonnegative variables for free variables? (3) equality constraints for inequality constraints? (4) minimizing objective for maximizing objective?
4 Dual linear program Consider the following linear program Remember the original linear program
6 Dual of LP in other form Symmetric pair
7 Karmarkar s form LP
8 How are they related? We denote the original LP by (P) as the primal problem and the new LP by (D) as dual problem. (P) and (D) are defined by the same data set (A, b, c). Problem (D) is a linear program with m variables and n constraints. The right-hand-side vector and the cost vector change roles in (P) and (D). What else?
9 Many interesting questions Feasibility: - Can problems (P) and (D) be both feasible? - One is feasible, while the other is infeasible? - Both are infeasible? Basic solutions: - Is there any relation between the basic solutions of (P) and that of (D)? bfs? optimal solutions? Optimality: -Can problems (P) and (D) both have a unique optimal solution? - Both have infinitely many? - One unique, the other infinitely many?
10 Examples Both (P) and (D) are infeasible: Both (P) and (D) have infinitely many optimal solutions:
11 Dual relationship If we take (D) as the primal problem, what ll happen? Convert (D) into its standard form by 1. taking w = u v 2. adding slacks s 3. minimizing objective New dual
12 Dual of the dual = primal Rearrange the new dual We have
14 Weak duality theorem If x is a primal feasible solution to (P) and w is a dual feasible solution to (D), then (Weak Duality Theorem): If x is a primal feasible solution to (P) and w is a dual feasible solution to (D), then.
16 Corollaries 1. If x is primal feasible, w is dual feasible, and then x is primal optimal, and w is dual optimal. 2. If the primal is unbounded below, then the dual is infeasible. (Is the converse statement true? -- watch for infeasibility) 3. If the dual is unbounded above, then the primal is infeasible. (Is the converse statement true?)
17 Strong duality theorem Questions: - Can the results of weak duality be stronger? - Is there any gap between the primal optimal value and the dual optimal value? Strong Duality Theorem:
18 Proof of strong duality theorem Proof: Note that the dual of the dual is the primal and the fact that If x is primal feasible, w is dual feasible and then x is primal optimal and w is dual optimal. We only need to show that if the primal has a finite optimal bfs x, then there exists a dual feasible solution w such that
19 Proof - continue Applying the simplex method at the optimal bsf x with basis B, we define. Then Thus w is dual feasible and
21 Corresponding basic solutions At a primal basic solution x with basis B, we defined a dual basic solution.
22 Further implications of strong duality theorem Theorem of Alternatives Existence of solutions of systems of equalities and inequalities Famous Farkas Lemma (another form)
23 Proof of Farkas Lemma Since w = 0 is dual feasible, we know (P) is infeasible if and only if (D) is unbounded above. Note that (P) is infeasible if and only if (I) has no solution. (D) is unbounded above if and only if (II) has a solution. Hence, (I) has no solution if and only if (II) has a solution.
24 Complementary slackness Consider the symmetric pair LP
30 Implication Solving a linear programming problem is equivalent to solving a system of linear inequalities.
31 Economic interpretation of duality Is there any special meaning of the dual variables? What is a dual problem trying to do? What s the role of the complementary slackness in decision making?
32 Dual variables Consider a nondegenerate linear program:
33 Dual variable for shadow price Moreover, we have Note: dual variable indicates the minimum unit price that one has to charge for additional demand i. It is also called shadow price or equilibrium price.
34 Dual LP problem Consider the following production scenario: n products to be produced m resources in hand market selling price for each product is known technology matrix is given by
35 A manufacturer s view
36 Complementary slackness condition Observation: 1. is the maximum marginal price the manufacturer is willing to pay the supplier for resource i. 2. When resource i is not fully utilized, i.e., the complementary slackness condition implies This means the manufacturer is not willing to pay a penny for buying any additional amount! 3. When the supplier asks too much, i.e., then This means the manufacturer is not going to produce any product j!
37 Sensitivity analysis Sensitivity is a post-optimality analysis of a linear program in which, some components of (A, b, c) may change after obtaining an optimal solution with an optimal basis and an optimal objective value. Questions of interests:
38 Fundamental concepts No matter how the data (A, b, c) change, we need to make sure that 1. Feasibility (c is not involved): bfs x is feasible if and only if 2. Optimality (b is not involved):
39 Change in the cost vector Scenario: Consider Question: Within which range of the current optimal solution remains to be optimal? Note: when c = (0,,0,1,0, 0), we have the regular sensitivity analysis on each cost coefficient.
40 Geometric view Fact:
41 Analysis Compare (1) (2) i.e.,
42 Analysis - continue
43 Analysis - continue
44 Change in the r-h-s vector Scenario: Fact: Question:
47 Changes in the constraint matrix Since both feasibility and optimality are involved, a general analysis is difficult. We only consider simple cases such as adding a new variable, removing a variable, adding a new constraint.
48 Adding a new variable Why? A new product, service or activity is introduced. Analysis:
49 Removing a variable Why? An activity is no longer available.
50 Adding a new constraint Why? A new restriction is enforces.
53 Dual simplex method What s the dual simplex method? - It is a simplex based algorithm that works on the dual problem directly. In other words, it hops from one vertex to another vertex along some edge directions in the dual space. It keeps dual feasibility and complementary slackness, but seeks primal feasibility.
54 Motivation Applying the (revised) simplex method to solve the dual problem:
55 Corresponding basic solutions At a primal basic solution x with basis B, we defined a dual basic solution.
56 Basic ideas of dual simplex method Starting with a dual basic feasible solution
57 Basic ideas of dual simplex method Checking optimality
58 Basic ideas of dual simplex method Pivoting move
59 Related issues Question 1: Analysis:
60 Related issues Question 2: Where is the dual feasibility information? Guess: must be from the fundamental matrix and vector c.
61 Related issues Question 3: When Answer: Sherman-Morrison-Woodbury formula. Sherman, J.; Morrison, W. J. (1949). "Adjustment of an Inverse Matrix Corresponding to Changes in the Elements of a Given Column or a Given Row of the Original Matrix (abstract)". Annals of Mathematical Statistics 20: 621. doi: /aoms/
62 Sherman-Morrison-Woodbury Formula Lemma:
63 Proof from wikipedia.org/wiki
68 Dual simplex method
69 Dual simplex method
71 Example - continue
72 How to start the dual simplex method?
74 Remarks (1) Solving a standard form LP by the dual simplex method is mathematically equivalent to solving its dual LP by the revised (primal) simplex method. (2) The work of solving an LP by the dual simplex method is about the same as of by the revised (primal) simplex method. (3) The dual simplex method is useful for the sensitivity analysis.
75 Complexity of the simplex method
76 Worst case performance of the simplex method Klee-Minty Example: Victor Klee, George J. Minty, How good is the simplex algorithm? in (O. Shisha edited) Inequalities, Vol. III (1972), pp
4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal
Linear Programming: Chapter 5 Duality Robert J. Vanderbei October 17, 2007 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/ rvdb Resource
Stanford University CS261: Optimization Handout 6 Luca Trevisan January 20, 2011 Lecture 6 In which we introduce the theory of duality in linear programming 1 The Dual of Linear Program Suppose that we
LECTURE 6: INTERIOR POINT METHOD 1. Motivation 2. Basic concepts 3. Primal affine scaling algorithm 4. Dual affine scaling algorithm Motivation Simplex method works well in general, but suffers from exponential-time
Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1
Linear Programming Linear programming refers to problems stated as maximization or minimization of a linear function subject to constraints that are linear equalities and inequalities. Although the study
COMPSCI 530: Design and Analysis of Algorithms DATE: 09/17/2013 Lecturer: Debmalya Panigrahi Lecture 7 - Linear Programming Scribe: Hieu Bui 1 Overview In this lecture, we cover basics of linear programming,
Chapter 2 LINEAR PROGRAMMING PROBLEMS 2.1 Introduction Linear Programming is the branch of applied mathematics that deals with solving optimization problems of a particular functional form. A linear programming
Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.
Special Situations in the Simplex Algorithm Degeneracy Consider the linear program: Maximize 2x 1 +x 2 Subject to: 4x 1 +3x 2 12 (1) 4x 1 +x 2 8 (2) 4x 1 +2x 2 8 (3) x 1, x 2 0. We will first apply the
Linear Programming Notes VII Sensitivity Analysis 1 Introduction When you use a mathematical model to describe reality you must make approximations. The world is more complicated than the kinds of optimization
CS 598CSC: Combinatorial Optimization Lecture date: January 21, 2009 Instructor: Chandra Chekuri Scribe: Sungjin Im 1 Polyhedra and Linear Programming In this lecture, we will cover some basic material
The Simplex Method The geometric method of solving linear programming problems presented before. The graphical method is useful only for problems involving two decision variables and relatively few problem
Linear Programming and the Simplex Method David Gale This exposition of linear programming and the simplex method is intended as a companion piece to the article in this issue on the life and work of George
Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S
Duality in Linear Programming 4 In the preceding chapter on sensitivity analysis, we saw that the shadow-price interpretation of the optimal simplex multipliers is a very useful concept. First, these shadow
The Essence of the Simplex Method Recall the Wyndor problem Max Z = 3x 1 + 5x 2 S.T. x 1 4 2x 2 12 3x 1 + 2x 2 18 x 1, x 2 0 Chap 4 The Simplex Method 8 corner point solutions. 5 out of them are CPF solutions.
APPLIED ECONOMICS By W.E. Diewert. July, 3. Chapter : Linear Programming. Introduction The theory of linear programming provides a good introduction to the study of constrained maximization (and minimization)
Does the Simplex Algorithm Work? In this section we carefully examine the simplex algorithm introduced in the previous chapter. Our goal is to either prove that it works, or to determine those circumstances
Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the
Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins
LINEAR PROGRAMMING P V Ram B. Sc., ACA, ACMA 98481 85073 Hyderabad Page 1 of 19 Question: Explain LPP. Answer: Linear programming is a mathematical technique for determining the optimal allocation of resources
Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.
Linear Inequalities and Linear Programming 5.1 Systems of Linear Inequalities 5.2 Linear Programming Geometric Approach 5.3 Geometric Introduction to Simplex Method 5.4 Maximization with constraints 5.5
Converting a Linear Program to Standard Form Hi, welcome to a tutorial on converting an LP to Standard Form. We hope that you enjoy it and find it useful. Amit, an MIT Beaver Mita, an MIT Beaver 2 Linear
The greatest flood has the soonest ebb; the sorest tempest the most sudden calm; the hottest love the coldest end; and from the deepest desire oftentimes ensues the deadliest hate. Th extremes of glory
494 CHAPTER 9 LINEAR PROGRAMMING 9. THE SIMPLEX METHOD: MAXIMIZATION For linear programming problems involving two variables, the graphical solution method introduced in Section 9. is convenient. However,
Linear Programming Problems Linear programming problems come up in many applications. In a linear programming problem, we have a function, called the objective function, which depends linearly on a number
lp.nb 1 Linear Programming, Lagrange Multipliers, and Duality Geoff Gordon lp.nb 2 Overview This is a tutorial about some interesting math and geometry connected with constrained optimization. It is not
Chapter 4 Duality Given any linear program, there is another related linear program called the dual. In this chapter, we will develop an understanding of the dual linear program. This understanding translates
Theory of Linear Programming Debasis Mishra April 6, 2011 1 Introduction Optimization of a function f over a set S involves finding the maximum (minimum) value of f (objective function) in the set S (feasible
Econ 172A, W2001: Final Examination, Possible Answers There were 480 possible points. The first question was worth 80 points (40,30,10); the second question was worth 60 points (10 points for the first
Algebraic Simplex Method - Introduction To demonstrate the simplex method, consider the following linear programming model: This is the model for Leo Coco's problem presented in the demo, Graphical Method.
Solving Linear Programming Problem with Fuzzy Right Hand Sides: A Penalty Method S.H. Nasseri and Z. Alizadeh Department of Mathematics, University of Mazandaran, Babolsar, Iran Abstract Linear programming
Optimization Methods Draft of August 26, 2005 I. Solving Linear Programs by the Simplex Method Robert Fourer Department of Industrial Engineering and Management Sciences Northwestern University Evanston,
Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may
The Graphical Simplex Method: An Example Consider the following linear program: Max 4x 1 +3x Subject to: x 1 +3x 6 (1) 3x 1 +x 3 () x 5 (3) x 1 +x 4 (4) x 1, x 0. Goal: produce a pair of x 1 and x that
The graphical method of solving linear programming problems can be applied to models with two decision variables. This method consists of two steps (see also the first lecture): 1 Draw the set of feasible
The Canonical and Dictionary Forms of Linear Programs Recall the general optimization problem: Minimize f(x) subect to x D (1) The real valued function f is called the obective function, and D the constraint
Contents 1. Index 2. Introduction 3. Syntax 4. Explanation of Simplex Algorithm Index Objective function the function to be maximized or minimized is called objective function. E.g. max z = 2x + 3y Constraints
Linear Programming with Post-Optimality Analyses Wilson Problem: Wilson Manufacturing produces both baseballs and softballs, which it wholesales to vendors around the country. Its facilities permit the
Chapter 2 Geometry of Linear Programming The intent of this chapter is to provide a geometric interpretation of linear programming problems. To conceive fundamental concepts and validity of different algorithms
42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions
Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to
Section Notes 3 The Simplex Algorithm Applied Math 121 Week of February 14, 2011 Goals for the week understand how to get from an LP to a simplex tableau. be familiar with reduced costs, optimal solutions,
L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:
Simplex method summary Problem: optimize a linear objective, subject to linear constraints 1. Step 1: Convert to standard form: variables on right-hand side, positive constant on left slack variables for
Integer Using the Simplex Method in Mixed Integer UTFSM Nancy, 17 december 2015 Using the Simplex Method in Mixed Integer Outline Mathematical Programming Integer 1 Mathematical Programming Optimisation
Linear Programming Notes V Problem Transformations 1 Introduction Any linear programming problem can be rewritten in either of two standard forms. In the first form, the objective is to maximize, the material
Chapter 6: Sensitivity Analysis Suppose that you have just completed a linear programming solution which will have a major impact on your company, such as determining how much to increase the overall production
Introduction: In the previous chapter, we discussed about the graphical method for solving linear programming problems. Although the graphical method is an invaluable aid to understand the properties of
The multi-integer set cover and the facility teral cover problem Dorit S. Hochbaum Asaf Levin December 5, 2007 Abstract The facility teral cover problem is a generalization of the vertex cover problem.
Operation Research Module 1 Unit 1 1.1 Origin of Operations Research 1.2 Concept and Definition of OR 1.3 Characteristics of OR 1.4 Applications of OR 1.5 Phases of OR Unit 2 2.1 Introduction to Linear
7.4 Linear Programming: The Simplex Method For linear programming problems with more than two variables, the graphical method is usually impossible, so the simplex method is used. Because the simplex method
Bulletin of the Iranian Mathematical Society Vol. 30 No. 2 (2004), pp 21-38. EXPLICIT ABS SOLUTION OF A CLASS OF LINEAR INEQUALITY SYSTEMS AND LP PROBLEMS H. ESMAEILI, N. MAHDAVI-AMIRI AND E. SPEDICATO
Chapter 1 Introduction to Linear Programming. This chapter introduces notations, terminologies and formulations of linear programming. Examples will be given to show how real-life problems can be modeled
Sensitivity analysis for production planning model of an oil company Thesis Supervised by By Yu Da Assoc. Professor Tibor Illés Eötvös Loránd University Faculty of Natural Sciences 2004 ACKNOWLEDGEMENT
Lecture notes 1: Introduction to linear and (mixed) integer programs Vincent Conitzer 1 An example We will start with a simple example. Suppose we are in the business of selling reproductions of two different
Linear Programming April 1, 005 Parts of this were adapted from Chapter 9 of i Introduction to Algorithms (Second Edition) /i by Cormen, Leiserson, Rivest and Stein. 1 What is linear programming? The first
LINEAR PROGRAMMING A Concise Introduction Thomas S Ferguson Contents 1 Introduction 3 The Standard Maximum and Minimum Problems 4 The Diet Problem 5 The Transportation Problem 6 The Activity Analysis Problem
CHAPTER 17 Linear Programming: Simplex Method CONTENTS 17.1 AN ALGEBRAIC OVERVIEW OF THE SIMPLEX METHOD Algebraic Properties of the Simplex Method Determining a Basic Solution Basic Feasible Solution 17.2
Linear Programming Exercises Week 1 Exercise 1 Consider the case of the Betta Machine Products Company described in the lecture notes. (a) Use a graphical method to obtain the new optimal solution when
24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no
CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like
Chapter 2 Solving Linear Programs Companion slides of Applied Mathematical Programming by Bradley, Hax, and Magnanti (Addison-Wesley, 1977) prepared by José Fernando Oliveira Maria Antónia Carravilla A
International Doctoral School Algorithmic Decision Theory: MCDA and MOO Lecture 2: Multiobjective Linear Programming Department of Engineering Science, The University of Auckland, New Zealand Laboratoire
A counter-example to the Hirsch conjecture Francisco Santos Universidad de Cantabria, Spain http://personales.unican.es/santosf/hirsch Bernoulli Conference on Discrete and Computational Geometry Lausanne,
A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem John Karlof and Peter Hocking Mathematics and Statistics Department University of North Carolina Wilmington Wilmington,
Linear Programming Models: Standard Form August 31, 2009 Outline: Lecture 4 Standard form LP Transforming the LP problem to standard form Basic solutions of standard LP problem Operations Research Methods
Optimization in R n Introduction Rudi Pendavingh Eindhoven Technical University Optimization in R n, lecture Rudi Pendavingh (TUE) Optimization in R n Introduction ORN / 4 Some optimization problems designing
Solving Linear Programs 2 In this chapter, we present a systematic procedure for solving linear programs. This procedure, called the simplex method, proceeds by moving from one feasible solution to another,
Interior Point Methods and Linear Programming Robert Robere University of Toronto December 13, 2012 Abstract The linear programming problem is usually solved through the use of one of two algorithms: either
SOLUTION OF LINEAR PROGRAMMING PROBLEMS THEOREM 1 If a linear programming problem has a solution, then it must occur at a vertex, or corner point, of the feasible set, S, associated with the problem. Furthermore,
Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least
, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing