4.6 Linear Programming duality

Transcription

1 4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal objective function value. Example: The value of a maximum feasible flow is equal to the capacity of a cut (separating the source s and the sink t) of minimum capacity. E. Amaldi Foundations of Operations Research Politecnico di Milano 1

2 Motivation: estimate of the optimal value Given max z = 4x 1 + x 2 + 5x 3 + 3x 4 x 1 x 2 x 3 + 3x 4 1 (I) 5x 1 + x 2 + 3x 3 + 8x 4 55 (II) -x 1 + 2x 2 + 3x 3 5x 4 3 (III) x i 0 i =1,, 4 find an estimate of the optimal value z *. Any feasible solution is a lower bound. Lower bounds: (0,0,1,0) z * 5 (2,1,1,1/3) z * 15 (3,0,2,0) z * 22 Even if we are lucky, we are not sure it is the optimal solution! E. Amaldi Foundations of Operations Research Politecnico di Milano 2

3 Upper bounds: By multiplying 5x 1 + x 2 + 3x 3 + 8x 4 55 (II) by 5/3, we obtain an inequality that dominates the objective function: 4x 1 + x 2 + 5x 3 + 3x 4 25/3x 1 + 5/3x 2 + 5x /3x 4 275/3 feasible solution z * 275/3. By adding constraints (II) and (III), we obtain: 4x 1 + 3x 2 + 6x 3 + 3x 4 58 z * 58 better upper bound. Linear combinations with nonnegative multipliers of inequality constraints yields valid inequalities E. Amaldi Foundations of Operations Research Politecnico di Milano 3

4 General strategy: Linearly combine the constraints with nonnegative multiplicative factors ( i-th constraint multiplied by y i 0 ). first case: y 1 =0, y 2 =5/3, y 3 =0 second case: y 1 =0, y 2 =1, y 3 =1 In general, such linear combination of (I), (II), (III) reads y 1 (x 1 x 2 x 3 +3x 4 ) + y 2 (5x 1 + x 2 + 3x 3 + 8x 4 ) + y 3 (-x 1 + 2x 2 + 3x 3 5x 4 ) y y 2 + 3y 3 E. Amaldi Foundations of Operations Research Politecnico di Milano 4

5 which is equivalent to: (y 1 + 5y 2 y 3 ) x 1 + (-y 1 + y 2 + 2y 3 ) x 2 + (-y 1 + 3y 2 + 3y 3 ) x 3 + (3y 1 + 8y 2 5y 3 ) x 4 y y 2 + 3y 3 (*) Observation: y i 0 so that the inequality direction is unchanged. In order to use the left hand side of (*) as upper bound on z = 4x 1 + x 2 + 5x 3 + 3x 4 E. Amaldi Foundations of Operations Research Politecnico di Milano 5

6 z = 4x 1 + x 2 + 5x 3 + 3x 4 we must have y 1 + 5y 2 y 3 4 -y 1 + y 2 + 2y 3 1 -y 1 + 3y 2 + 3y 3 5 3y 1 + 8y 2 5y 3 3 y i 0, i =1, 2, 3. In such a case, any feasible solution x satisfies 4x 1 + x 2 + 5x 3 + 3x 4 y y 2 + 3y 3 In particular: z * y y 2 + 3y 3 E. Amaldi Foundations of Operations Research Politecnico di Milano 6

7 Since we look for the best possible upper bound on z * : (D) min y y 2 + 3y 3 Original problem: max z = 4x 1 + x 2 + 5x 3 + 3x 4 y 1 + 5y 2 y 3 4 -y 1 + y 2 + 2y 3 1 -y 1 + 3y 2 + 3y 3 5 3y 1 + 8y 2 5y 3 3 y i 0 i =1, 2, 3 x 1 x 2 x 3 + 3x 4 1 (I) 5x 1 + x 2 + 3x 3 + 8x 4 55 (II) -x 1 + 2x 2 + 3x 3 5x 4 3 (III) x i 0 i =1,, 4 Definition: The problem (D) is the dual problem, while the original problem is the primal problem. E. Amaldi Foundations of Operations Research Politecnico di Milano 7

8 In matrix form: max z = c T x Primal (P) Ax b x 0 min w=b T y Dual (D) A T y c y 0 or y T A c T E. Amaldi Foundations of Operations Research Politecnico di Milano 8

9 Dual problem max z = c T x min w=b T y (P) Ax b x 0 (D) A T y c y 0 Dual of an LP in standard form? min z=c T x Ax = b x 0 E. Amaldi Foundations of Operations Research Politecnico di Milano 9

10 Standard form: (P) min z = c T x Ax = b x 0 with A an mn matrix -max -c T x A b -A x -b x 0 dual A x b A T - min (b T -b T ) y 1 y 2 y 1 y 2 (A T -A T ) -c y 1 0, y 2 0 y = y 1 y 2 E. Amaldi Foundations of Operations Research Politecnico di Milano 10

11 - min (b T -b T ) y 1 y 2 y 1 y 2 (A T -A T ) -c y 1 0, y 2 0 (D) max w=b T y A T y c y R m y y 2 y 1 unrestricted in sign! -min -b T (y 2 -y 1 ) -A T (y 2 -y 1 ) -c y 1 0, y 2 0 E. Amaldi Foundations of Operations Research Politecnico di Milano 11

12 Property: The dual of the dual problem coincides with the primal problem. max (P) z = c T x Ax b x 0 min (D) w=b T y A T y c y 0 Observation: it doesn't matter which one is a maximum or minimum problem. E. Amaldi Foundations of Operations Research Politecnico di Milano 12

13 General transformation rules Primal (min) Dual (max) m constraints n variables coefficients obj. fct right hand side A equality constraints unrestriced variables inequality constraints ( ) variables 0 ( 0) m variables n constraints right hand side coefficients obj. fct E. Amaldi Foundations of Operations Research Politecnico di Milano 13 A T unrestriced variables equality constraints variables 0 ( 0) inequality constraints ( )

14 Example: (P) max x 1 + x 2 x 1 x 2 2 3x 1 + 2x 2 12 x 1, x 2 0 max x 1 + x 2 x 1 - x 2 2-3x 1-2x 2-12 x 1, x 2 0 dual min 2y 1-12y 2 y 1-3y 2 1 -y 1-2y 2 1 y 1, y 2 0 E. Amaldi Foundations of Operations Research Politecnico di Milano 14

15 Example: using the above rules (P) max x 1 + x 2 x 1 x x 1 + 2x 2 12 x 1, x 2 0 dual min 2y y 2 y 1 + 3y 2 1 -y 1 + 2y 2 1 y 1 0, y 2 0 y~ 1 -y 1 min 2y 1-12y ~ 2 y 1-3y ~ 2 1 -y 1-2y ~ 2 1 y 1 0, y~ 2 0 E. Amaldi Foundations of Operations Research Politecnico di Milano 15

16 Exercize: (P) min 10x x x 3 2x 1 x 2 1 x 2 + x 3 2 x 1 x 3 = 3 x 1 0, x 2 0, x 3 unrestricted Dual? E. Amaldi Foundations of Operations Research Politecnico di Milano 16

17 Weak duality theorem min (P) z = c T x Ax b x 0 max (D) w=b T y A T y c y 0 X {x : Ax b, x 0} ø and Y {y : A T y c, y 0} ø, For each feasible solution x X of (P) and each feasible solution y Y of (D) we have b T y c T x. E. Amaldi Foundations of Operations Research Politecnico di Milano 17

18 Proof Given x X and y Y, we have Ax b, x 0 and A T y c, y 0 which imply that b T y x T A T y x T c = c T x x T A T c E. Amaldi Foundations of Operations Research Politecnico di Milano 18

19 Consequence: If x * is a feasible solution for the primal problem (P) ( x * X ), y * is a feasible solution (D) ( y * Y ), and the values of the respective objective functions coincide then c T x * = b T y *, x * is optimal for (P) and y * is optimal for (D). E. Amaldi Foundations of Operations Research Politecnico di Milano 19

20 Strong duality theorem If X = {x : Ax b, x 0} ø and min{c T x : x X}isfinite, there exist x * X and y * Y such that c T x * = b T y *. min{ c T x : x X } = max{ b T y : y Y } optimum z = c T x x X feasible for (P) z * = w * w = b T y y Y feasible for (D) E. Amaldi Foundations of Operations Research Politecnico di Milano 20

21 Proof Derive an optimal solution of (D) from one of (P) Given min c T x max y T b (P) Ax = b x 0 (D) y T A c T y R m and x* is an optimal feasible solution of (P) x x* = * B with x * N x * B = B -1 b x * N = 0 provided (after a finite of iterations) by the Simplex algorithm with Bland's rule. E. Amaldi Foundations of Operations Research Politecnico di Milano 21

22 Let us consider y T c T B B -1 y is a feasible solution of (D): c T N = c T N (c T B B -1 )N = c T N y T N 0 T reduced costs of the nonbasic variables optimality of x * y T N c T N E. Amaldi Foundations of Operations Research Politecnico di Milano 22

23 c T B = c T B (c T B B -1 )B = c T B y T B = 0 T y T B c T B reduced costs of the basic variables I y is an optimal solution of (D): y T b= (c T B B -1 )b= c T B (B -1 b) = c T B x * B = c T x * Hence y = y * E. Amaldi Foundations of Operations Research Politecnico di Milano 23

24 Corollary For any pair of primal-dual problems (P) and (D), only four cases can arise: P D optimal solution unbounded LP Infeasible LP optimal solution 1) unbounded LP infeasible LP 2) 3) 4) E. Amaldi Foundations of Operations Research Politecnico di Milano 24

25 Strong duality theorem 1) Weak duality theorem 2) and 3) 4) can arise: (P) min -4x 1 2x 2 -x 1 + x 2 2 x 1 x 2 1 x 1, x 2 0 (D) max 2y 1 + y 2 -y 1 + y 2-4 y 1 y 2-2 y 1, y 2 0 empty feasible regions E. Amaldi Foundations of Operations Research Politecnico di Milano 25

26 Economic interpretation The primal and dual problems correspond to two complementary point of views on the same market. Diet problem: n aliments j=1,, n m nutrients i=1,, m (vitamines, ) a ij b i c j quantity of i-th nutrient in one unit of j-th aliment requirement of i-th nutrient cost of one unit of j-th aliment E. Amaldi Foundations of Operations Research Politecnico di Milano 26

27 min n j= 1 c j x j (P) n j= 1 a ij x j b i i = 1,..., m x j 0 j = 1,...,n max m i= 1 b i y i (D) m i= 1 a ij y y i i c 0 j j = 1,..., n i = 1,...,m E. Amaldi Foundations of Operations Research Politecnico di Milano 27

28 Interpretation of the dual problem: A company that produces pills of the m nutrients needs to decide the nutrient unit prices y i so as to maximize income. If the costumer buys nutrient pills he will buy b i units for each i, 1 i m. The price of the nutrient pills must be competitive: m i= 1 aij yi c j j = 1,...,n cost of the pills that are equivalent to 1 unit of j-th aliment E. Amaldi Foundations of Operations Research Politecnico di Milano 28

29 If both linear programs (P) and (D) admit a feasible solution, the strong duality theorem implies that z * = w * An equilibrium exists (two alternatives with the same cost). Observation: Strong connection with Game theory (zero-sum games). E. Amaldi Foundations of Operations Research Politecnico di Milano 29

30 Optimality conditions Given min z = c T x (P) X Ax b x 0 (D) max w=b T y Y y T A c T y 0 two feasible solutions x * X and y * Y are optimal y * T b= c T x * If x j and y i are unknown, it is a single equation in n+m unknowns! E. Amaldi Foundations of Operations Research Politecnico di Milano 30

31 Since y *T b y *T Ax * c T x *, we have Ax * c T y *T b= y *T Ax * and y *T Ax * = c T x * therefore y *T (Ax * -b) = 0 and (c T -y *T A) x * = 0 0 T 0 0 T 0 m+n equations in n+m unknowns necessary and sufficient optimality conditions! E. Amaldi Foundations of Operations Research Politecnico di Milano 31

32 Complementary slackness conditions x * X and y * Y are optimal solutions of, respectively, (P) and (D) if and only if i-th row of A slack s i of i-th constraint of (P) y * i (a T i x * -b i ) = 0 i = 1,, m (c T j y *T A j ) x * j = 0 j = 1,, n slack of j-th constraint of (D) j-th column of A At optimality, the product of each variable with the corresponding slack variable of the relative dual is = 0. E. Amaldi Foundations of Operations Research Politecnico di Milano 32

33 Economic interpretation for the diet problem n a j1 ij x * j b i y * i 0 If the optimal diet includes an excess of i-th nutrient, the costumer is not willing to pay y * i > 0. y * i 0 n a j1 ij x * j b i If the company selects a price y * i > 0, the costumer must not have an excess of i-th nutrient. E. Amaldi Foundations of Operations Research Politecnico di Milano 33

34 m i1 y * i a ij c j x * j 0 it is not convenient for the costumer to buy aliment j price of the pills equivalent to the nutrients contained in one unit of j-th aliment is lower than the price of the aliment. x * j 0 m i1 y * i a ij c If costumer includes the j-th aliment in optimal diet, the company must have selected competitive prices y * i (the price of the nutrients in pills contained in a unit of j-th aliment is not lower than c j ). j E. Amaldi Foundations of Operations Research Politecnico di Milano 34

35 Example: (P) min 13x x 2 + 6x 3 s.t. 5x 1 + x 2 + 3x 3 = 8 3x 1 + x 2 = 3 x 1, x 2, x 3 0 (D) max 8y 1 + 3y 2 s.t. 5y 1 + 3y 2 13 y 1 + y y 1 6 Verify that the feasible x * = (1, 0, 1) is an optimal, non degenerate solution of (P). Suppose it is true and derives, via the complementary slackness conditions, the corresponding optimal solution of (D). E. Amaldi Foundations of Operations Research Politecnico di Milano 35

36 Since (P) is in standard form, the conditions y i * (a T i x * -b i ) = 0 are automatically satisfied i, 1 i 2. The condition (c T j y *T A j ) x * j = 0 is satisfied for j = 2 because x * 2= 0. Since x * 1 > 0 and x * 3 > 0, we obtain the conditions: 5y 1 + 3y 2 = 13 3y 1 = 6 and hence the optimal solution y * 1 = 2 and y * 2 = 1 of (D) with b T y * =19 = c T x *. E. Amaldi Foundations of Operations Research Politecnico di Milano 36