Analytic center cutting-plane method

Size: px

Start display at page:

Download "Analytic center cutting-plane method"

Brittany Summers
7 years ago
Views:

1 L. Vandenberghe EE236C (Spring ) Analytic center cutting-plane method analytic center cutting-plane method computing the analytic center pruning constraints lower bound and stopping criterion 1

2 Analytic center and ACCPM analytic center of a set of inequalities Ax b x ac = argmin z m log(b i a T i z) i=1 analytic center cutting-plane method (ACCPM) localization method that represents P k by set of inequalities A (k), b (k) selects analytic center of A (k) x b (k) as query point x (k+1) Analytic center cutting-plane method 2

3 ACCPM algorithm outline given an initial polyhedron P 0 = {x A (0) x b (0) } known to contain C repeat for k = 1,2, compute x (k), the analytic center of A (k 1) x b (k 1) 2. query cutting-plane oracle at x (k) 3. if x (k) C, quit; otherwise, add returned cutting plane a T z b: [ ] [ A (k) A (k 1) =, b (k) b (k 1) = b a T if P k = {x A (k) x b (k) } =, quit ] Analytic center cutting-plane method 3

4 Constructing cutting planes cutting planes for optimal set C of convex problem minimize f 0 (x) subject to f i (x) 0, i = 1,...,m if x (k) is not feasible, say f j (x (k) ) > 0, we have (deep) feasibility cut f j (x (k) )+g T j (z x (k) ) 0 where g j f j (x (k) ) if x (k) is feasible, we have (deep) objective cut g T 0(z x (k) )+f 0 (x (k) ) f (k) best 0 where g 0 f 0 (x (k) ) and f (k) best = min{f 0(x (i) ) i k, x (i) feasible} Analytic center cutting-plane method 4

5 Computing the analytic center minimize φ(x) = m log(b i a T i x) i=1 domφ = {x a T i x < b i, i = 1,...,m} challenge: we are not given a point in domφ some options use phase I to find x domφ, followed by standard Newton method standard Newton method applied to dual problem infeasible start Newton method (EE236B lecture 11, BV.3) Analytic center cutting-plane method 5

6 Dual Newton method dual analytic centering problem m maximize g(z) = logz i b T z +m subject to A T z = 0 i=1 optimality conditions x, z are primal and dual optimal if b i a T i x = 1/z i, A T z = 0, z 0, Ax b Analytic center cutting-plane method 6

7 Initialization of dual Newton method dual method is interesting when a strictly feasible z is easy to find, e.g., dual feasibility requires A = I I B A T z = z 1 z 2 +B T z 3 = 0, z = (z 1,z 2,z 3 ) 0 (for example, can pick any z 3 0 and find corresponding z 1, z 2 ) this corresponds to variable bounds in (primal) centering problem, e.g., P 0 = {x l x u} Analytic center cutting-plane method 7

8 Dual Newton equation analytic centering problem m minimize logz i +b T z i=1 subject to A T z = 0 Newton equation [ diag(z) 2 A A T 0 ][ z w ] = [ b diag(z) ] can be solved by elimination of z: solve ( A T diag(z) 2 A ) w = A T (diag(z) 2 b z) and take z = z diag(z) 2 (b Aw) Analytic center cutting-plane method 8

9 Stopping criterion for dual Newton method Newton decrement at z is λ(z) = ( z T g(z) ) 1/2 = diag(z) 1 z 2 λ(z) = 0 implies w is the analytic center: b Aw = diag(z) 1 1 λ(z) < 1 implies x = w is primal feasible b Aw = diag(z) 1 (1 diag(z) 1 z) 0 terminating with small λ(z) gives strictly feasible, approximate center Analytic center cutting-plane method 9

10 Infeasible start Newton method reformulated analytic centering problem (variables x and y) minimize m logy i, subject to y = b Ax i=1 optimality conditions y 0, z 0, r(x,y,z) = y +Ax b A T z z diag(y) 1 1 = 0 initialization: can start from any x, z, and any y 0 example: take previous analytic center as x, and choose y as y i = b i a T i x if b i a T i x > 0, y i = 1 otherwise Analytic center cutting-plane method

11 Newton equation for infeasible Newton method A I A T 0 diag(y) 2 I x y z = y +Ax b A T z z diag(y) 1 1 can be solved by block elimination of y, z: solve and take (A T diag(y) 2 A) x = A T diag(y) 2 (b Ax 2y) y = b y Ax A x, z = diag(y) 2 (y y) z Analytic center cutting-plane method 11

12 Pruning constraints enclosing ellipsoid at analytic center if x ac is the analytic center of a T i x b i, i = 1,...,m, then the ellipsoid E = {z (z x ac ) T 2 φ(x ac )(z x ac ) m 2 } contains P = {z a T i x b i, i = 1,...,m} proof in BV page 420 from expression for Hessian, { E = z m ( ) a T 2 } i (z x ac ) m 2 i=1 b i a T i x ac Analytic center cutting-plane method 12

13 (ir)relevance measure for constraint a T i x b i η i = b i a T i x ac (a T i 2 φ(x ac ) 1 a i ) 1/2 if η i m, then constraint a T i x b i is redundant proof: the optimal value of (with H = 2 φ(x ac )) is maximize a T i z subject to (z x ac ) T H(z x ac ) m 2 m a T i H 1 a i +a T i x ac the constraint is redundant if this is less than b i Analytic center cutting-plane method 13

14 common ACCPM constraint dropping schemes drop all constraints with η i m (guaranteed to not change P) drop constraints in order of irrelevance, keeping constant number, usually 3n 5n Analytic center cutting-plane method 14

15 Lower bound in ACCPM suppose we apply ACCPM to a convex problem minimize f 0 (x) subject to f i (x) 0, i = 1,...,m Gx h (1) the inequalities A (k 1) x b (k 1) at iteration k can be divided in two sets A f x b f includes the constraints Gx h plus the feasibility cuts A o x b o +c o includes the objective cuts f 0 (x (i) )+g (i)t 0 (x x (i) ) f (i) best, with g (i)t 0 x (i) f 0 (x (i) ) stored in the vector b o and f (i) best in c o Analytic center cutting-plane method 15

16 piecewise-linear relaxation: the problem minimize max(a o x b o ) subject to A f x b f is a relaxation of the problem (1) (max(y) for vector y means max i y i ) f 0 (x) max(a o x b o ) for all x (by convexity) optimal set is contained in the polyhedron A f x b f (by construction) dual of PWL relaxation maximize b T ou b T f v subject to A T ou+a T f v = 0 1 T u = 1 u 0, v 0 dual feasible points give lower bounds on optimal value of (1) Analytic center cutting-plane method 16

17 dual feasible point from analytic centering x (k) is the analytic center of A o x b o +c o, A f x b f ; hence A T oz o +A T f z f = 0, where z o = diag(b o +c o A o x (k) ) 1 1, z f = diag(b f A f x (k) ) 1 1 normalizing gives a dual feasible point for the PWL relaxation: u = 1 1 T z o z o, v = 1 1 T z o z f l (k) = b T ou b T f v is a lower bound on optimal value of (1) from x (k) we get a readily computed lower bound Analytic center cutting-plane method 17

18 Stopping criterion keep track of best point found, and best lower bound best function value so far f (k) best = min f 0 (x (i) ) i=1,...,k x (i) feasible best lower bound so far l (k) best = max i=1,...,k l(i) can stop when f (k) best l(k) best ǫ to guarantee ǫ-suboptimality Analytic center cutting-plane method 18

19 Example: piecewise linear minimization minimize max i=1,...,m (a T i x+b i) subject to 1 x 1 n = 0 variables, m = 200 terms, f f(x (k) ) f -1-2 f (k) best f k k Analytic center cutting-plane method 19

20 computed lower bound on optimal value f (k) best l(k) (dashed line) and f (k) best f (solid line) 4 f (k) best f and f (k) best l(k) k Analytic center cutting-plane method 20

21 ACCPM with constraint dropping same problem; convergence with and without pruning (to 3n constraints) 1 no pruning 1 no pruning pruning pruning 0 0 f(x (k) ) f -1-2 f (k) best f k k Analytic center cutting-plane method 21

22 References Y. Ye, Interior-Point Algorithms. Theory and Analysis (1997) gives a convergence proof of ACCPM (the bound on the number of iterations is n 2 times a function of R/r) S. Boyd, course notes for EE364b, Convex Optimization II Analytic center cutting-plane method 22

Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725

Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725 Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T