Chap 8. Linear Programming and Game Theory

Transcription

1 Chap 8. Linear Programming and Game Theory KAIST wit Lab 유인철

2 I. Linear Inequalities

3 Introduction Linear Programming (from wikipedia) Mathematical method for determining a way to achieve the best outcome (such as maximum profit or lowest cost) in a given mathematical model for some list of requirements A technique for the optimization of a linear objective function, subject to some constraints Objective of this chapter To see the geometric meaning of linear inequalities To find the optimal soloution Hyperplane and Halfspace Hyperplane : Halfspace : 3

4 The Feasible Set and the Cost Function Feasible set the intersection of halfspaces composition of the solutions to a family of linear inequalities like Example) 24,, Fundamental constraint to linear programming

5 Problem in linear programming Find the point that lies in the feasible set The point is feasible minimizes or maximize the cost The point is optimal Cost increases Example : in the previous figure Cost : 2 3 Cost decreases Minimum cost : 2 3 6, 2 Vector (,2) is feasible and optimal The optimal vector occurs at a corner of the feasible set Possible categories. The feasible set is empty 2. The cost function is unbounded on the feasible set 3. The cost reaches its minimum (or maximum) on the feasible set 5

6 Slack Variables Slack Variable To change the inequality to an equation Result : equation + nonnegativity constraints Primal problem : Constraints : Example : in the previous example Inequality : 24 Slack variable : 24 Nonnegativity constraint : 6

7 The Diet Problem and Its Dual Diet problem in linear programming 2 sources of protein : A pound of peanut butter : a unit of protein A steak : 2 unit of protein Diet : at least 4 units are required Contains Cost : x pounds of peanut butter y steaks A pound of peanut butter : $2 A steak : $3 Constraints : 24,, the cost of whole diet : 2 3 optimal diet for minimum cost = 2 steaks (, 2) 7

8 Duality theorem The minimum in the given primal problem equals the maximum in its dual. Every linear program has dual. Example : Dual problem of previous one 2,23 optimal price : $.5 maximum revenue : 4 $6 8

9 Typical Applications. Production Planning : General motors Chevrolet Buick Cadillac profit $2 $3 $5 miles per gallon Assemble duration minute 2 minutes 3 minutes Also, the average car must get 8 miles per gallon What is the maximum profit in 8 hours (48 minutes)? 9

10 2. Portfolio Selection federal bonds municipals junk bonds interest 5% 6% 9% We can buy amounts,, not exceeding a total of $, No more than $2, can be invested in junk bonds The portfolio s average quality must be no lower than municipals What is the maximum interest?

11 Problem Set 8. Cost :. 4 2 Cost : 3 2,2 2 4, Cost : 2,2 4 4,

12 II. The Simplex Method

13 Introduction Linear Programming with n unknowns m constraints Minimum problem feasible vectors meet conditions Optimal occurs at the point where the planes first touch the feasible set Compute : Find all the corners of the feasible set Simplex method A systematic way to solve linear programs 3

14 The Geometry : Movement Along Edges Corner in linear algebra The meeting point of different planes Subspace of Possible intersection point of linear programming Review the feasible set : the intersection space of given by equation equations (by slack var.) + fundamental constraints equations (,, ) + fundamental constraints equations, but only equations are needed 4

15 How equations are chosen for intersection point? constraints : 26, 26,, 26 26,,, P : intersection of & 2 6 equations,, can be chosen out of equations The number of possible intersection points : C!!! 5

16 The intersection point is a genuine corner if It also satisfies the remaining inequality constraints. Otherwise, it is a complete fake. Ex) in the previous figure P : & 6, 6 J : & 3, 3 J 6

17 Key idea of simplex methods Go from corner to corner along the edges of the feasible set Edge of the feasible set If one of the intersecting planes is removed, remaining equations form an edge that comes out of the corner P : & 6, 6 : : 7

18 Newly defined linear programming by slack variable Slack variable : Equality constraints and nonnegativity Minimize, subject to, Newly defined Minimize, subject to, 8

19 Example. 9

20 The Simplex Algorithm Free variables vs. Basic variables in constraints Free variables : n components Basic variables (Pivot variables) : m components : equations pivots Basic feasible solutions of A solution is basic when of its components are zero, and it is feasible when it satisfies Setting the free variables to zero, basic solution is a genuine corner if its nonzero components are positive Phase of simplex method Phase I : find one basic feasible solution Phase II : move step by step to the optimal 2

21 Which corner do we go to next? basic variables remain basic Only basic variables will become free ( become zero), while other basic variables will stay positive Terminology Entering variable : the free variable which will become basic Leaving variable : the basic variable which will become free Ex) in the previous example If P moves to Q, Entering variable, : Leaving variable 2

22 Example 2 ),, 8, 9: genuine 7 3 There is possibility that cost can be minimized 2) As is increased, : entering variable : leaving variable 3) At this point,

23 Observation of Example 2 Quick Way, The right sides divided by the coefficients of the entering variable Ratio : 4, 3 hits zero first 23

24 The next step of Example 2 The current step ends at the new corner 2,,,,3 Free variable :,, Leaving variable Basic variable :, Entering variable Pivot by substituting 9 Repeat Phase II,,,,

25 Tableau (or large matrix) The Tableau One way to organize the simplex step into matrix Applying row operation to tableau Renumbering if necessary,, Tableau at corner Cost : Constraints :, :, : :, : 25

26 Ex) For Example 2 cost : 7 3 constraints :

27 Reduced tableau The basic variables will stand alone when elimination multiplies by Ex)

28 Fully reduced : R=rref(T) Subtract times the top block row from bottom row 2 2 Ex)

29 Meaning of matrix R Cost : Constraints : At corner : & Stopping test (optimality condition) r : coefficients of 29

30 Example 3. Figure 8.3 Minimize Subject to 26,26 <solve problem using tableau> Slack variable : , 3

31 ) Find initial feasible point Let basis variables(free variable) be zero 26 26, 6, 6: fake Find other feasible corner as initial point, 6, 6: genuine We choose this point as initial one 3

32 2) Renumbering Take basic variable(=non zeros) firstly This means that problem is changed as below , , 32

33 3) Make tableau and take row operation ,, at point 6 6, 33

34 4) Stopping test There is possibility that cost can be minimized Let be variable which becomes basic (=non zero) 3 34

35 5) Find the variable which becomes free(=zero) Let : a column of : the index of smallest ratio,, Then, becomes free variable (=zero) Ratio : : becomes zero 35

36 6) Find next solution Tableau is reconstructed by changing column and ,, /3 2/3 2/3 / /3 2/3 /3 2/3 /3 /

37 7) Stopping test again /3 2/3 /3 2/3 /3 / Since, this point is optimal. 4 at the point 2 2, 37

38 The Organization of a Simplex Step Reduced simplex method For computational purposes, only,, are used ) 2) 38

39 reduce the computation since has the form as below 39

40 Karmarkar s Method Wonho Kang 4

41 Definition Karmarkar s Method An iterative algorithm that, given an initial point and parameter, generates a sequence,,, which are the solutions of linear program (LP) Assumption. LP has a strictly feasible point, and the set of optimal point is bounded 2. LP has a special canonical form: minimize z subject to,, 3. The value of the objective at the optimum is known and is equal to zero, 4

42 Minimize Subject to Karmarkar s Method Minimize,,, Subject to, z Minimize,,, Subject to y,,,, 42

43 Minimize Subject to y Karmarkar s Method 43

44 Karmarkar s Method Canonical form minimize z subject to,, Denote nullspace of Ω Define simplex,,,, Denote center of the simplex,,, 44

45 Rewritten canonical form minimize z subject to Ω Karmarkar s Method Note that the constraint set (or feasible set) Ω can be represented as Ω,, Ω, Ω, 45

46 Karmarkar s Method Procedure. Initialize: Set ; /; 2. Update: Set Ψ where Ψ is an update map 3. Check stopping criterion: If the condition 2 is satisfied, then stop where is a given termination parameter; 4. Iterate: Set, go to 2 46

47 Karmarkar s Method First update step in the algorithm. Compute the orthogonal projector onto the nullspace of 2. Compute the normalized orthogonal projection of onto the nullspace of 3. Compute the steepest descent direction vector, / 4. Compute using where is the prespecified step size,, Karmarkar recommands a value of ¼ in his original paper 47

48 General update step. Compute the matrices Karmarkar s Method,, and is, in general, not at the center of the simplex, so whose diagonal entries are the components of the vector is used to transform this point to the center 2. Compute the orthogonal projector onto the nullspace of 48

49 General update step Karmarkar s Method 3. Compute the normalized orthogonal projection of onto the nullspace of 4. Compute the steepest descent direction vector 5. Compute using 6. Compute by applying the inverse transformation 49

50 Karmarkar s Method Example Minimize 3 3 subject to 2 3,, 3 3 Ω 2 3 Δ, 3 5

51 Solution. Initialize: 2. Set ; Karmarkar s Method 5. / /3 /3 /3 6.,, 7. 8.,, / / / / 5

52 Karmarkar s Method 2. Update: Set Ψ where Ψ is an update map ) Compute the orthogonal projector onto the nullspace of

53 Karmarkar s Method 2. Update: Set Ψ where Ψ is an update map 2) Compute the normalized orthogonal projection of onto the nullspace of

54 Karmarkar s Method 2. Update: Set Ψ where Ψ is an update map 3) Compute the steepest descent direction vector / / 6 7 is the radius of the largest sphere 8) inscribed in the set 9, 54

55 Karmarkar s Method 2. Update: Set Ψ where Ψ is an update map 3) Compute the steepest descent direction vector /

56 Karmarkar s Method 2. Update: Set Ψ where Ψ is an update map 4) Compute 5 /3 /3 / ) Note: makes be guaranteed 5) to lie in the constraint set 6,, 7 8) (a strictly interior point of the set), 56

57 3. Check stopping criterion: Karmarkar s Method 4. If the condition 2 is satisfied, then stop where is a given termination parameter /3 /3 / cf) /4 3/ if 2.4, then stop, or 4. Iterate: Set, go to 2 57

58 Karmarkar s Method c c d x () x ()

59 Karmarkar s Method /3 /3 /3 Minimize Minimize Subject to Subject to 59

60 Karmarkar s Method

61 Karmarkar s Method c c d x () x ()

62 Karmarkar s Method c c d x' () x' (2).8 x () x (2)

63 III. The Dual Problem

64 The Dual Problem Minimize Subject to, n feasible set Dual Maximize Subject to, y feasible set Ex) In Section 8., Minimize 2 Subject to Dual Maximize 4 Subject to

65 Duality theorem When both problems have feasible vectors, the minimum cost c the maximum income Weak duality If x and y are feasible in the primal and dual problems, then (proof) x and y are feasible and y 65

66 Reverse of duality theorem If the vectors x and y are feasible and, then x and y are optimal Ex) Minimize 4 subject to 2 6, , 6 7, 4 Dual Maximize 6 7 subject to 2 5, 3 4 : /2 66

67 Complementary slackness condition Relationship between the primal and dual Assume (P) means primal problem and have a optimal solution and (D) means dual problem and have a optimal solution, then. If in (P), then 2. If in (D), then (proof) c c should be always satisfied. However, from this equation, if, is 67

68 The proof of Duality From simplex method,, by reordering, In dual problem, if y is feasible If, optimal 68