18. Linear optimization

Transcription

1 18. Linear optimization EE103 (Fall ) linear program examples geometrical interpretation extreme points simplex method 18-1

2 Linear program minimize c 1 x 1 +c 2 x 2 + +c n x n subject to a 11 x 1 +a 12 x 2 + +a 1n x n b 1 a 21 x 1 +a 22 x 2 + +a 2n x n b 2 a m1 x 1 +a m2 x 2 + +a mn x n b m n optimization (decision) variables x 1,..., x n m linear inequality constraints matrix notation minimize c T x subject to Ax b the inequality between vectors Ax b is interpreted elementwise Linear optimization 18-2

3 Production planning a company makes three products, in quantities x 1, x 2, x 3 per month profit per unit is 1.0 (for product 1), 1.4 (product 2), 1.6 (product 3) products use different amounts of resources (labor, material,... ) fraction of total available resource needed per unit of each product optimal production plan product 1 product 2 product 3 resource 1 1/1000 1/800 1/500 resource 2 1/1200 1/700 1/600 maximize x x x 3 subject to (1/1000)x 1 +(1/800)x 2 +(1/500)x 3 1 (1/1200)x 1 +(1/700)x 2 +(1/600)x 3 1 x 1 0, x 2 0, x 3 0 solution: x 1 = 462, x 2 = 431, x 3 = 0 Linear optimization 18-3

4 Network flow optimization t 1 x 1 x x 3 x 5 x 4 x x 8 x 7 x 9 6 t capacity constraints 0 x x x x x x x x x 9 1 maximize t subject to flow conservation at nodes capacity constraints on the arcs Linear optimization 18-4

5 linear programming formulation maximize t subject to t = x 1 +x 2, x 1 +x 3 = x 4, et cetera 0 x 1 3, 0 x 2 2, et cetera (t = x 1 +x 2 is equivalent to inequalities t x 1 +x 2, t x 1 +x 2,... ) solution Linear optimization 18-5

6 Data fitting fit a straight line to m points (u i,v i ) by minimizing sum of absolute errors minimize m α+βu i v i i= v dashed: least-squares solution; solid: minimizes sum of absolute errors u Linear optimization 18-6

7 linear programming formulation m minimize i=1 y i subjec to y i α+βu i v i y i, i = 1,...,m variables α, β, y 1,..., y m inequalities are equivalent to y i α+βu i v i the optimal y i satisfies y i = α+βu i y i Linear optimization 18-7

8 Terminology minimize c T x subject to Ax b problem data: n-vector c, m n-matrix A, m-vector b x is feasible if Ax b feasible set is set of all feasible points x is optimal if it is feasible and c T x c T x for all feasible x optimal value: p = c T x unbounded problem: c T x is unbounded below on feasible set infeasible probem: feasible set is empty Linear optimization 18-8

9 Example x 2 minimize x 1 x 2 subject to 2x 1 +x 2 3 x 1 +4x 2 5 x 1 0, x 2 0 x 1 + 4x 2 = 5 2x 1 + x 2 = 3 x 1 feasible set is shaded Linear optimization 18-9

10 solution x 1 x 2 = 2 x 2 minimize x 1 x 2 subject to 2x 1 +x 2 3 x 1 +4x 2 5 x 1 0, x 2 0 x 1 x 2 = 1 x 1 x 2 = 5 x 1 x 2 = 0 x 1 x 2 = 4 x 1 optimal solution is x = (1,1), optimal value is p = 2 x 1 x 2 = 3 Linear optimization 18-10

11 Hyperplanes and halfspaces hyperplane solution set of one linear equation with nonzero coefficient vector a a T x = b halfspace solution set of one linear inequality with nonzero coefficient vector a a T x b Linear optimization 18-11

12 Geometrical interpretation G = {x a T x = b} H = {x a T x b} a a u = (b/ a 2 )a x x u G 0 x u x u H u = (b/ a 2 )a satisfies a T u = b x is in G if a T (x u) = 0, i.e., x u is orthogonal to a x is in H if a T (x u) 0, i.e., angle (x u,a) π/2 Linear optimization 18-12

13 Example a T x = 0 x 2 a T x = 10 x 2 a = (2,1) a = (2,1) x 1 x 1 a T x = 5 a T x = 5 a T x 3 Linear optimization 18-13

14 Polyhedron definition: the solution set of a finite number of linear inequalities a T 1x b 1, a T 2x b 2,..., a T mx b m matrix notation: Ax b where A = a T 1 a T 2. a T m, b = b 1 b 2. b m geometrical interpretation: intersection of m halfspaces Linear optimization 18-14

15 example (n = 2) x 1 +x 2 1, 2x 1 +x 2 2, x 1 0, x 2 0 x 2 2x 1 +x 2 = 2 x 1 +x 2 = 1 x 1 Linear optimization 18-15

16 example (n = 3) 0 x 1 2, 0 x 2 2, 0 x 3 2, x 1 +x 2 +x 3 5 x 3 (0,0,2) (0,2,2) (1,2,2) (2,0,2) (2,1,2) (2,2,1) (0,2,0) x 2 (2,0,0) (2,2,0) x 1 Linear optimization 18-16

17 Extreme points let ˆx P, where P is a polyhedron defined by inequalities a T k x b k if a T kˆx = b k, we say the kth inequality is active at ˆx if a T kˆx < b k, we say the kth inequality is inactive at ˆx ˆx is called an extreme point of P if A I(ˆx) = a T k 1 a T k 2. a T k p has a zero nullspace where I(ˆx) = {k 1,...,k p } are the indices of the active constraints at ˆx an extreme point ˆx is nondegenerate if p = n, degenerate if p > n Linear optimization 18-17

18 example x 1 x 2 1, 2x 1 +x 2 1, x 1 0, x 2 0 ˆx = (1,0) is an extreme point x 2 A I(ˆx) = [ ] 2x 1 + x 2 = 1 x 1 x 2 = 1 x 1 ˆx = (0,1) is an extreme point A I(ˆx) = ˆx = (1/2,1/2) is not an extreme point A I(ˆx) = [ 1 1 ] Linear optimization 18-18

19 Simplex algorithm minimize x 1 +x 2 x 3 subject to 0 x 1 2, 0 x 2 2, 0 x 3 2 x 1 +x 2 +x 3 5 x 3 (0,0,2) x 2 x 1 (2,2,0) move from one extreme point to another extreme point with lower cost Linear optimization 18-19

20 One iteration of the simplex method suppose ˆx is a nondegenerate extreme point renumber constraints so that active constraints are 1,..., n active constraints at ˆx: a T 1 ˆx = b 1,..., a T nˆx = b n inactive constraints at ˆx: a T n+1ˆx < b n+1,..., a T mˆx < b m matrix of active constraints: define I = I(ˆx) = {1,...,n} a T 1 A I = a T 2. (an n n nonsingular matrix) a T n Linear optimization 18-20

21 step 1 (test for optimality) solve A T Iz = c i.e., find z that satisfies n z k a k = c k=1 if z k 0 for all k, then ˆx is optimal and we return ˆx proof. consider any feasible x: n c T x = z k a T kx k=1 n z k b k = k=1 n z k a T kˆx = c Tˆx k=1 (the inequality follows from z k 0, a T k x b k) Linear optimization 18-21

22 step 2 (compute step) choose a j with z j > 0 and solve i.e., find v that satisfies A I v = e j a T j v = 1, a T kv = 0 for k = 1,...,n and k j for small t > 0, ˆx+tv is feasible and has a smaller cost than ˆx proof: a T j (ˆx+tv) = b j t a T k(ˆx+tv) = b k for k = 1,...,n and k j and for sufficiently small positive t a T k(ˆx+tv) b k for k = n+1,...,m finally, c T (ˆx+tv) = c Tˆx+tz j < c Tˆx Linear optimization 18-22

23 step 3 (compute step size) maximum t such that ˆx+tv is feasible, i.e., a T kˆx+ta T kv b k, k = 1,...,m the maximum step size is b k a T t max = min kˆx k:a T k v>0 a T k v (if a T kv 0 for all k, then the problem is unbounded below) step 4 (update ˆx) ˆx := ˆx+t max v Linear optimization 18-23

24 Example minimize x 1 +x 2 x 3 subject to 0 x 1 2, 0 x 2 2, 0 x 3 2 x 1 +x 2 +x 3 5 at ˆx = (2,2,0) x 3 A I = z = A T I c = (1,1,1) 2. for j = 3, v = A 1 I (0,0, 1) = (0,0,1) 3. ˆx+tv is feasible for t 1 x 1 (2,2,1) (2,2,0) x 2 Linear optimization 18-24

25 Example minimize x 1 +x 2 x 3 subject to 0 x 1 2, 0 x 2 2, 0 x 3 2 x 1 +x 2 +x 3 5 at ˆx = (2,2,1) x 3 A I = (2,1,2) 1. z = A T I c = (2,2, 1) 2. for j = 2, v = A 1 I (0, 1,0) = (0, 1,1) (2,2,1) x 2 3. ˆx+tv is feasible for t 1 x 1 Linear optimization 18-25

26 Example minimize x 1 +x 2 x 3 subject to 0 x 1 2, 0 x 2 2, 0 x 3 2 x 1 +x 2 +x 3 5 at ˆx = (2,1,2) x 3 A I = (2,0,2) (2,1,2) 1. z = A T I c = (0, 2,1) x 2 2. for j = 3, v = A 1 I ( 1,0,0) = (0, 1,0) 3. ˆx+tv is feasible for t 1 x 1 Linear optimization 18-26

27 Example minimize x 1 +x 2 x 3 subject to 0 x 1 2, 0 x 2 2, 0 x 3 2 x 1 +x 2 +x 3 5 at ˆx = (2,0,2) x 3 A I = (2,0,2) (0,0,2) 1. z = A T I c = (1, 1, 1) x 2 2. for j = 1, v = A 1 I ( 1,0,0) = ( 1,0,0) 3. ˆx+tv is feasible for t 2 x 1 Linear optimization 18-27

28 Example minimize x 1 +x 2 x 3 subject to 0 x 1 2, 0 x 2 2, 0 x 3 2 x 1 +x 2 +x 3 5 x 3 at ˆx = (0,0,2) (0,0,2) A I = z = A T I c = ( 1, 1, 1) therefore, ˆx is optimal x 2 x 1 Linear optimization 18-28

29 Practical aspects at each iteration, we solve two sets of linear equations A T Iz = c, A I v = e j using an LU factorization or sparse LU factorization of A I implementation requires a phase 1 to find first extreme point simple modifications handle problems with degenerate extreme points very large LPs (several 100,000 variables and constraints) are routinely solved in practice Linear optimization 18-29