Solution to homework problem # and 7.1

Transcription

1 Let us first review some basic concepts. Solution to homewk problem # and Derive the dual problem from the Lagrangian duality. It wks f convex problems, including all linear programming problems. Given a convex optimization problem fx gi x 0, f i = 1,...,m 1 h i x = 0, f i = 1,...,m 2 let us first write down its Lagrangian function and v i are free variables. Then we have m 1 m 2 Lx,u,v = fx + u i g i x + v i h i x, i=1 i=1 where u i 0, f i = 1,...,m 1, primal problem min x dual problem max u 0,v max Lx,u,v u 0,v min Lx,u,v x To derive the dual problem, we need to compute min x Lx,u,v, which can be calculated by using x Lx,u,v = 0. Indeed, x L = 0, together with, gives the constraints of the dual problem. Suppose gu,v = min x Lx,u,v, then the dual problem is Do not fget, and DO NOT add v 0. gu, v x Lx,u,v = 0 2. F linear programming problems, we have some easy-to-use fmulas. However, one need to be careful on choosing the crect fmula to use. In the following table, the first primal problem is written in its canonical fm, and the second is in its standard fm. primal problem dual problem f = c t x g = bt y Ax b A y + u = c x 0 y 0, f = c t x g = b t y Ax = b A y + u = c x 0 g = b t y A t y c y 0 g = b t y A t y c The fmula given in textbook uses v instead of y. This v is different from the v in Lagrangian duality fms. 3. The Lagrangian duality in item 1 and the fmulas in item 2 should give exactly the same dual problem. Sometime they may have differently-looking fms. However, by eliminating slack variables using a change of variable, these fms can be shown to be equivalent. 1

2 Solution f # The primal problem is f = x 3y x + y = 6 x + y 4 The Lagrangian function can be written as Lx,y,u,v = x 3y + u x + y 4 + vx + y 6, where. Notice that v should be a free variable. Then the dual problem is equivalent to the following saddle point problem: max min Lx,y,u,v. u 0, v x,y To compute min x,y Lx,y,u,v, let us take x 0 = y 0 and when u, v satisfy these two equations, we have { 1 u + v = u + v = 0 min Lx,y,u,v = 1 u + vx u + vy 4u 6v = 0x + 0y 4u 6v. x,y Substitute this into max u 0, v min x,y Lx,y,u,v, the dual problem can be written as, g = 4u 6v 1 u + v = u + v = 0 Notice that is required by the Lagrangian function, and v should be a free variable. It is not hard to find that the primal problem has min f = 16 at x,y = 1,5, and the dual problem has max g = 16 at u,v = 1,2. The details are skipped. 2

3 Alternative solution f # You can also solve the problem as follows. The primal problem is equivalent to f = x 3y x + y 6 x y 6 x + y 4 Then the Lagrangian function becomes Lx,y,u 1,u 2,u 3 = x 3y + u 1 x + y 6 + u 2 x y u 3 x + y 4 where u 1, u 2, u 3 0. Then min x,y Lx,y,u 1,u 2,u 3 is taken at the point where { x u 1 u 2 u 3 = 0 = y u 1 u 2 + u 3 = 0, and its values is Hence the dual problem can also be written as min x,y Lx,y,u 1,u 2,u 3 = 6u 1 + 6u 2 4u 3. g = 6u 1 + 6u 2 4u u 1 u 2 u 3 = u 1 u 2 + u 3 = 0 u 1, u 2, u 3 0 Notice that by setting u = u 3, which should satisfy, and v = u 1 u 2, which should be a free variable, we get exactly the same fm as in the previous solution. 3

4 Solution f # 7.1. The primal problem is This problem can be rewritten into a canonical fm: z = x 1 + 3x 2 x 1 + 4x 2 10 x 1 + 2x 2 10 x 1, x 2 0 f = x 1 3x x 1 2 x 1, x Hence by the fmula f the canonical fm on page 1 of this document, we have the dual problem: g = 10y 1 10y 2 y 1 y 2 + u 1 = 1 4y 1 2y 2 + u 2 = 3 y 0, g = 10y 1 10y 2 y 1 y 2 1 4y 1 2y 2 3 y 0 If you prefer to first write the primal problem into its standard fm, that s fine. The standard fm f the primal problem is f = x 1 3x 2 + 0x 3 + 0x x = x 1, x 2, x 3, x Use the fmula f the standard fm on page 1 of this document, we have the dual problem: g = 10y y 2 y 1 + y 2 + u 1 = 1 4y 1 + 2y 2 + u 2 = 3 y 1 + u 3 = 0 y 2 + u 4 = 0 g = 10y y 2 y 1 + y 2 1 4y 1 + 2y 2 3 y 1 0 y 2 0 It is not hard to see that all above four different fms in 4 boxes f the dual problem are equivalent. F example, take the two fms on the right. By changing the sign of variables y 1 and y 2, we can see that they are exactly the same. Finally, one can also use Lagrangian duality to find the dual f this problem. There are two possibilities. 1. Use the Lagrangian function f the canonical fm, which should be Lx,u = x 1 3x 2 + u 1 x 1 + 4x u 2 x 1 + 2x u 3 x 1 + u 4 x 2 where u 1, u 2, u 3, u 4 0. It is imptant not to fget constraints x 1 0 and x 2 0 in the Lagrangian function. Then using the fmula f the Lagrangian duality on page 1 of this document, we have the dual problem g = 10u 1 10u u 1 + u 2 u 3 = u 1 + 2u 2 u 4 = 0 u 1, u 2, u 3, u 4 0 4

5 2. Use the Lagrangian function f the standard fm, which should be Lx,u,v = x 1 3x 2 + v 1 x 1 + 4x 2 + x v 2 x 1 + 2x 2 + x u 1 x 1 + u 2 x 2 + u 3 x 3 + u 4 x 4 where u 1, u 2, u 3, u 4 0. Again, it is imptant not to fget constraints x i 0, f i = 1,2,3,4, in the Lagrangian function. Also, notice that v 1, v 2 are free variables, since they cresponds to equation type constraints of the standard fm. Then using the fmula f the Lagrangian duality, the dual problem is g = 10v 1 10v v 1 + v 2 u 1 = v 1 + 2v 2 u 2 = 0 v 1 u 3 = 0 v 2 u 4 = 0 u 1, u 2, u 3, u 4 0 Altogether, f problem # 7.1, we already have 6 different fms f the dual problem. All of them are equivalent. this is left as an exercise f you. It can also be shown that the minimum of the primal problem is 10, and the maximum of the dual problem no matter in which fm is 10. 5