Lecture 3: Duality. 1. Duality for LP in canonical form. Duality, complementarity, and interpretations.

Transcription

1 Lecture 3: Duality 1. Duality for LP in canonical form. Duality, complementarity, and interpretations. 2. Duality for general LP problems. 3. Duality for LP in standard form Föreläsning 5 1 Dualitet

2 Reading instructions for the slides: The geometrical interpretation of the complementarity conditions will be considered in more detail later in the course for non-linear problems. It is probably easier to understand then, so do not spend to much time on it here. The economical interpretation of primal and dual problems can be skimmed through Föreläsning 5 2 Dualitet

3 Duality for LP in canonic form Primal Dual minimixe c T x maximize b T y (P) s.t. Ax b x 0 (D) s.t. A T y c y 0 Primal feasible region Dual feasible region F P = {x R n : Ax b; x 0} F D = {y R m : A T y c; y 0} ˆx F P is the optimal solution if ŷ F D is the optimal solution if c Tˆx c T x, x F P b T ŷ b T y, y F D Föreläsning 5 3 Dualitet

4 How are the primal and dual related? Consider the following pair of primal and dual optimization problems Primal Dual minimize x 1 + x 2 maximize 200y y 2 (P) s.t x x x x (D) s.t y y y y x 1,x 2 0 y 1,y 2 0 What is the relation between the optimal values? the objective function values (for an arbitrary feasible point)? the optimal solutions and the constraints at the optimum? Föreläsning 5 4 Dualitet

5 Illustration of the feasible region, iso-cost lines, and the optimal solution x 2 c ˆx y 2 ŷ b F P F D x y 1 The Optimal values are the same c Tˆx = b T ŷ = For each x F P and y F D it holds that c T x c Tˆx = b T ŷ b T y Föreläsning 5 5 Dualitet

6 Let ŝ = s 1 = Aˆx b = s (S) ˆr = r 1 = c A T ŷ = r (R) The optimal solution satisfies 1. ŝ = Aˆx b 0, ˆx 0 (primal feasibility) 2. ˆr = c A T ŷ 0, ŷ 0 (dual feasibility) 3. x j r j = 0, j = 1, 2 and y i s i = 0, i = 1, 2 (complementarity) Föreläsning 5 6 Dualitet

7 The complementarity conditions can be given a geometrical interpretation. From (R) and (S) (R) 1 = (S) 200 = it follows that the gradients of the objective functions (c and b respectively) can be given as linear combinations of the active constraints. This is illustrated in the figure. (Note: incorrect scale) x 2 c y 2 b F P F D x 1 y 1 Föreläsning 5 7 Dualitet

8 Economical interpretation We consider the production of two products P 1 and P 2 Machines M 1 and M 2 are used for the production of P 1 and P 2 The Production capacities for the machines are [day/unit]: M M P1 P2 product price P 1 b 1 = 200 P 2 b 2 = 400 The price for the products [SEK/unit]: In the dual, production volume is determined for profitmaximization. Föreläsning 5 8 Dualitet

9 Assume that you have the option of renting out the machines M 1 and M 2. What is the lowest possible price x k SEK/day for renting out M k, k = 1, 2. The lowest price is determined from the following optimization problem minimize x 1 + x 2 s.t x x x x x 1,x 2 0 where the constraints ensure that you earn at least as much by renting out the machines as the profit you would have made producing products P 1 and P 2. Föreläsning 5 9 Dualitet

10 The primal and dual can be seen as alternatives to each others. If your business idea is to rent out M 1 and M 2, then the dual provides information about the lowest production volume necessary for instead using the machines for producing P 1 and P 2. if your business idea is to produce P 1 and P 2, then the primal provides information of the lowest renting price that makes it profitable to rent out the machines instead. (break-even price) What is called primal and dual is actually arbitrary, since it can be shown that the dual of the dual is the primal. (at least for linear programs) Föreläsning 5 10 Dualitet

11 Theoretical results The following result from the book (Griva, Nash & Sofer) confirms the relations in the example: Theorem 6.4[Weak duality] For every x F P and every y F D it holds that c T x b T y. A direct implication from this is that, if ˆx F P and ŷ F D and c Tˆx = b T ŷ, then ˆx and ŷ are optimal to (P) and (D), respectively. Föreläsning 5 11 Dualitet

12 Theorem 6.9[Strong duality - stronger than the book ] There are four alternative for the optimization problems (P) and (D) (a) F P, F d. Then there is (at least) one optimal solution ˆx to (P) and (at least) one optimal solution to (D). They satisfy c Tˆx = b T ŷ. (b) F P but F D =. Then the primal is unbounded (from below). (c) F D but F P =. Then the dual is unbounded (from above). (d) F P = and F D =. The proof of (a) is non-trivial and is usually based on Farkas lemma. Case (b) can be interpreted as a consequence of the lack of a lower limit for the primal due to the lack of solutions to the dual. Similar interpretation holds for (c). Föreläsning 5 12 Dualitet

13 Notation A = a a 1n. [ ] = â 1... â n = ā T 1. b = a m1 b a mn, y = y 1., s = s 1., ā T m b m y m s m c = c 1., x = x 1., r = r 1. c n x n r n Föreläsning 5 13 Dualitet

14 Theorem 6.12 [Complementary slackness] x R n is optimal for (P) and y R m is optimal for (D) if and only if x j 0, r j 0, x j r j = 0, y i 0, s i 0, y i s i = 0, j = 1,...,n, i = 1,...,m, where s = Ax b och r = c A T y. Terminology: Non-negativity constraints for the primal: x j 0, j = 1,...,n. General constraints for the primal: s i = ā T i x b i 0, i = 1,...,m. Non-negativity constraints for the dual: y i 0, i = 1,...,m. General constraints for the dual: r j = c j â T j y 0, j = 1,...,n. Föreläsning 5 14 Dualitet

15 A consequence of the complementarity Theorem is that if r j = c j â T j y > 0 then x j = 0, i.e., the j:th non-negativity constraint of the primal is active. if y i > 0 then s i = ā T i x b i = 0, i.e., the i:th general constraint of the primal is active. if s i = ā T i x b i > 0 then y i = 0, i.e., the i:th non-negativity constraint of the dual is active. if x j > 0 then r j = c j â T j y = 0, i.e., the j:th general constraint of the dual is active. Föreläsning 5 15 Dualitet

16 Geometric interpretation of the complementarity constraints [ T Let e j = ] R n, where the 1 is in position j. Then c = r A T y = e j r j ā i y i j:r j >0 i:y i >0 i.e., the gradient of the primal objective function is a linear combination (with coefficients > 0) of the normal vectors of the active constraints. [ T Let e i = ] R m, where the 1 is in position i. Then b = Ax s = â j x j e i s i j:x j >0 i:s i >0 i.e., the gradient of the dual objective function is a linear combination of the normal vectors of the active constraints. Föreläsning 5 16 Dualitet

17 Graphic illustration of the geometrical interpretation â 1 b â 2 F P c ā2 ā 1 F D c can be written as a linear combination of ā 1 and ā 2 b can be written as a linear combination of â 1 and â 2 Föreläsning 5 17 Dualitet

18 Duality for general LP-problems Primal minimize c T 1x 1 + c T 2x 2 s.t. A 11 x 1 + A 12 x 2 b 1 A 21 x 1 + A 22 x 2 = b 2 x 1 0,x 2 free Dual maximize b T 1y 1 + b T 2y 2 s.t. A T 11y 1 + A T 21y 2 c 1 A T 12y 1 + A T 22y 2 = c 2 y 1 0, y 2 free Two methods for deriving the dual Reformulate the problem in canonical form, for which the dual is know. By using Lagrange-relaxation (more about this later in the course) Föreläsning 5 18 Dualitet

19 Duality for LP in standard form Primal Dual (P) minimize c T x s.t. Ax = b x 0 (D) maximize b T x s.t. A T y c F P = {x R n : Ax = b; x 0} F D = {y R m : A T y c} Weak duality: x F P and y F D c T x b T y = c T x y T Ax = (c A T y) T x 0. Complementarity: x F P and y F D is optimal for (P) and (D) respectively, if and only if (c A T y) T x = 0, i.e. if (c i â T i y)x i = 0 Föreläsning 5 19 Dualitet

20 A consequence of complementarity Assume that we with the simplex method determined an optimal solution of the primal (P). This means that we have basic- and non-basic variables with indecies β = (β 1,...,β m ) and η = (η 1,...,η l ) so that x B = B 1 b x N = 0 B T y = c B ĉ N = c N N T y 0 B T y = c B N T y c N c T x = c T B x B = b T y A T y c c T x = b T y The last implication means that y is optimal for (D). Föreläsning 5 20 Dualitet

21 Conversely, assume that we have found an optimal solution y to the dual (D). We can construct an optimal solution x to the primal by using the complementarity constraints: (c i â T i y) x i = 0, i = 1,...,n If c i â T i y > 0 it holds that x i = 0, i.e. i η (non-basic variable). If there are n m such non-active dual constraints, then we have found a set of basic variable indecies β for which the optimal x is given by x B = B 1 b and x N = 0 If it is easy to solve the dual, this is a good alternative. This is usually the case if n >> m. Föreläsning 5 21 Dualitet

22 Example minimize x 1 + 2x 2 + 3x 3 + 4x 4 + 5x 5 s.t. 5x 1 + 4x 2 + 3x 3 + 2x 2 + x 1 = 1 x k 0,k = 1,...,5 The dual is maximize y s.t. 5y 1, 4y 2, 3y 3, 2y 4, y 5 which has the optimal solution ŷ = 1. From the complementarity 5 constraint it is clear that ˆx 2 = ˆx 3 = ˆx 4 = ˆx 5 = 0 and for the primal constraint to be satisfied ˆx 1 = 1. This is the optimal solution to the 5 primal. Föreläsning 5 22 Dualitet

23 Reading instructions Chapter in the book. Föreläsning 5 23 Dualitet