Linear Programming Lecture II

Transcription

1 Linear Programming II Linear Programming Lecture II! LP duality! Strong duality theorem! Bonus proof of LP duality! Applications Kevin Wayne Computer Science Department Princeton University COS 523 Fall LP Duality LP Duality Goal. Find a lower bound on optimal value. Easy. Any feasible solution provides one. Goal. Find an upper bound on optimal value. Ex 1. Multiply 2 nd inequality by 6: 24 A + 24 B # 960. Ex 1. (A, B) = (34, 0)! z* " 442 Ex 2. (A, B) = (0, 32)! z* " 736 Ex 3. (A, B) = (7.5, 29.5)! z* " 776 Ex 4. (A, B) = (12, 28)! z* " 800! z* = 13 A + 23 B # 24 A + 24 B # 960. objective function 3 4

2 LP Duality LP Duality Goal. Find an upper bound on optimal value. Ex 2. Add 2 times 1 st inequality to 2 nd inequality: #! z* = 13 A + 23 B # 14 A + 34 B # Goal. Find an upper bound on optimal value. Ex 2. Add 1 times 1 st inequality to 2 times 2 nd inequality: #! z* = 13 A + 23 B # 13 A + 23 B # 800. Recall lower bound. (A, B) = (34, 0)! z* " 442 Combine upper and lower bounds: z* = LP Duality LP Duality: Economic Interpretation Idea. Add nonnegative combination (C, H, M) of the constraints s.t. Brewer: find optimal mix of beer and ale to maximize profits. 13A + 23B " (5C + 4H + 35M ) A + (15C + 4H + 20M ) B " 480C +160H +1190M Dual problem. Find best such upper bound. Entrepreneur: buy individual resources from brewer at min cost.! C, H, M = unit price for corn, hops, malt.! Brewer won't agree to sell resources if 5C + 4H + 35M < 13. min 480C + 160H M s. t. 5C + 4H + 35M " 13 15C + 4H + 20M " 23 C, H, M " 0 min 480C + 160H M s. t. 5C + 4H + 35M " 13 15C + 4H + 20M " 23 C, H, M " 0 7 8

3 LP Duals Double Dual Canonical form. Canonical form. max c x min y b s. t. A y " c max c x min y b s. t. A y " c Property. he dual of the dual is the primal. Pf. Rewrite as a maximization problem in canonical form; take dual. (D') max " y b s. t. "A y # "c y $ 0 (DD) min " c z s. t. "(A ) z # "b z # LP dual recipe. aking Duals Linear Programming II Primal maximize minimize Dual constraints variables a x = b i a x # b a x " b i x j " 0 x j # 0 unrestricted y i unrestricted y i " 0 y i # 0 a y " c j a y # c j a y = c j variables constraints! LP duality! Strong duality theorem! Bonus proof of LP duality! Applications Pf. Rewrite LP in standard form and take dual. 11

4 LP Strong Duality LP Weak Duality heorem. [Gale-Kuhn-ucker 1951, Dantzig-von Neumann 1947] For A $ % m&n, b $ % m, c $ % n, if and are nonempty, then max = min. heorem. For A $ % m&n, b $ % m, c $ % n, if and are nonempty, then max # min. max c x min y b s. t. A y " c max c x min y b s. t. A y " c Generalizes:! Dilworth's theorem.! König-Egervary theorem.! Max-flow min-cut theorem.! von Neumann's minimax theorem.! Pf. Suppose x $ % m is feasible for and y $ % n is feasible for.!, A x # b! y A x # y b! x " 0, A y " c! y A x " c x! Combine: c x # y A x # y b. Pf. [ahead] Projection Lemma Projection Lemma Weierstrass' theorem. Let X be a compact set, and let f(x) be a continuous function on X. hen min { f(x) : x $ X } exists. Projection lemma. Let X ' % m be a nonempty closed convex set, and let y ( X. hen there exists x * $ X with minimum distance from y. Moreover, for all x $ X we have (y x*) (x x*) # 0. Weierstrass' theorem. Let X be a compact set, and let f(x) be a continuous function on X. hen min { f(x) : x $ X } exists. Projection lemma. Let X ' % m be a nonempty closed convex set, and let y ( X. hen there exists x * $ X with minimum distance from y. Moreover, for all x $ X we have (y x*) (x x*) # 0. obtuse angle y x 2 y x* x X Pf.! Define f(x) = y x.! Want to apply Weierstrass, but X not necessarily bounded.! X ) *! there exists x' $ X.! Define X' = { x $ X : y x # y x' } so that X' is closed, bounded, and min { f(x) : x $ X } = min { f(x) : x $ X' }.! By Weierstrass, min exists. y x* X' x x' X 15 16

5 Projection Lemma Separating Hyperplane heorem Weierstrass' theorem. Let X be a compact set, and let f(x) be a continuous function on X. hen min { f(x) : x $ X } exists. Projection lemma. Let X ' % m be a nonempty closed convex set, and let y ( X. hen there exists x * $ X with minimum distance from y. Moreover, for all x $ X we have (y x*) (x x*) # 0. Pf.! x* min distance! y x * 2 # y x 2 for all x $ X.! By convexity: if x $ X, then x* + + (x x*) $ X for all 0 < + < 1.! y x* 2 # y x* +(x x*) 2 = y x* (x x*) (y x*) (x - x*)! hus, (y x*) (x - x*) # ½ + (x x*) 2.! Letting +, 0 +, we obtain the desired result. heorem. Let X ' % m be a nonempty closed convex set, and let y ( X. hen there exists a hyperplane H = { x $ % m : a x = - } where a $ % m, - $ % that separates y from X. Pf. a x " - for all x $ X a y < -! Let x* be closest point in X to y.! By projection lemma, (y x*) (x x*) # 0 for all x $ X! Choose a = x * y ) 0 and - = a x*.! If x $ X, then a (x x*) " 0; thus! a x " a x* = -.! Also, a y = a (x* a) = - a 2 < - y x* x H = { x $ % m : a x = -} X Farkas' Lemma Another heorem of the Alternative heorem. For A $ % m&n, b $ % m exactly one of the following two systems holds: (I) " x # $ n (II) " y # $ m s. t. Ax = b s. t. A y % 0 x % 0 y b < 0 Corollary. For A $ % m&n, b $ % m exactly one of the following two systems holds: (II) " y # $ m (I) " x # $ n s. t. A y % 0 s. t. Ax % b y b < 0 x & 0 y % 0 Pf. [not both] Suppose x satisfies (I) and y satisfies (II). hen 0 > y b = y Ax " 0, a contradiction. Pf. [at least one] Suppose (I) infeasible. We will show (II) feasible.! Consider S = { A x : x " 0 } so that S closed, convex, b ( S.! Let y $ % m, - $ % be a hyperplane that separates b from S: y b < -, y s " - for all s $ S.! 0 $ S! - # 0! y b < 0! y Ax " - for all x " 0! y A " 0 since x can be arbitrarily large. Pf. Apply Farkas' lemma to: (I' ) " x # $ n, s # $ m s. t. A x + I s = b x, s % 0 (II' ) " y # $ m s. t. A y % 0 I y % 0 y b <

6 LP Strong Duality LP Strong Duality heorem. [strong duality] For A $ % m&n, b $ % m, c $ % n, if and are nonempty then max = min. Pf. [max # min] Weak LP duality. Pf. [min # max] Suppose max < -. We show min < -. (I) " x # $ n max c x s. t. Ax % b &c x % &' x ( 0 (II) " y # $ m, z # $ s. t. A y % c z & 0 y b % ' z < 0 y, z & 0! By definition of -, (I) infeasible! (II) feasible by Farkas' Corollary. min y b s. t. A y " c (II) " y # $ m, z # $ s. t. A y % cz & 0 y b %' z < 0 y, z & 0 Let y, z be a solution to (II). Case 1. [z = 0]! hen, { y $ % m : A, y b < 0, } is feasible.! Farkas Corollary! { x $ % n : Ax # b, x " 0 } is infeasible.! Contradiction since by assumption is nonempty. Case 2. [z > 0]! Scale y, z so that y satisfies (II) and z = 1.! Resulting y feasible to and y b < Linear Programming II Simplex Algorithm: Dual Solution Observation. Final simplex tableaux reveals dual solution!! LP duality! Strong duality theorem! Bonus proof of LP duality! Applications max Z subject to " 1 S C " 2 S H " Z = "800 B C S H = 28 A " 1 10 C H = 12 " 25 6 C " 85 8 H + S M = 110 A, B, S C, S H, S M # 0 Z = 800 A* = 12, B* = 28 C* = 1, H* = 2, M* = 0 24

7 Review: Simplex ableaux Simplex ableaux: Dual Solution subtract c B A B times constraints subtract c B A B times constraints c B + c N = Z A B + A N = b " 0 (c N " c B A B "1 A N ) = Z " c B A B "1 b I + A "1 B A N = A "1 B b # 0 c B + c N = Z A B + A N = b " 0 (c N " c B A B "1 A N ) = Z " c B A B "1 b I + A "1 B A N = A "1 B b # 0 initial tableaux tableaux corresponding to basis B multiply by A B initial tableaux tableaux corresponding to basis B multiply by A B Primal solution. = A B b " 0 = 0 Optimal basis. c N c B A B A N # 0 Primal solution. = A B b " 0 = 0 Optimal basis. c N c B A B A N # 0 Dual solution. y = c B A B y b = c B A B "1 b = c B + c B = c x max = min [ ] [ ] [ c B A B A N ] [ c N ] y A = y A B y A N = c B A B A B c B A B A N = c B " c B = c dual feasible Simplex Algorithm: LP Duality subtract c B A B times constraints Linear Programming II c B + c N = Z A B + A N = b " 0 (c N " c B A B "1 A N ) = Z " c B A B "1 b I + A "1 B A N = A "1 B b # 0 initial tableaux tableaux corresponding to basis B! LP duality multiply by A B! Strong duality theorem! Alternate proof of LP duality Primal solution. = A B b " 0 = 0! Applications Optimal basis. c N c B A B A N # 0 Dual solution. y = c B A B Simplex algorithm yields constructive proof of LP duality. 27

8 LP Duality: Economic Interpretation LP Duality: Sensitivity Analysis Brewer: find optimal mix of beer and ale to maximize profits. Entrepreneur: buy individual resources from brewer at min cost. min 480C + 160H M s. t. 5C + 4H + 35M " 13 15C + 4H + 20M " 23 C, H, M " 0 A* = 12 B* = 28 OP = 800 C* = 1 H* = 2 M* = 0 OP = 800 Q. How much should brewer be willing to pay (marginal price) for additional supplies of scarce resources? A. corn $1, hops $2, malt $0. Q. Suppose a new product "light beer" is proposed. It requires 2 corn, 5 hops, 24 malt. How much profit must be obtained from light beer to justify diverting resources from production of beer and ale? A. At least 2 ($1) + 5 ($2) + 24 ($0) = $12 / barrel. LP duality. Market clears LP is in NP. co-np LP. For A $ % m&n, b $ % m, c $ % n,! $ %, does there exist x $ % n such that: Ax = b, x " 0, c x " -? heorem. LP is in NP. co-np. Pf.! Already showed LP is in NP.! If LP is infeasible, then apply Farkas' Lemma to get certificate of infeasibility: (II) " y # $ m, z # $ s. t. A y % 0 y b & ' z < 0 z % 0 or equivalently, y b - z = 31