Semidefinite Programming in Polynomial Optimization

Transcription

1 Semidefinite Programming in Polynomial Optimization Monique LAURENT CWI, Amsterdam SIAM Conference on Optimization Stockholm 2005 Semidefinite Programmingin Polynomial Optimization p. 1

2 Polynomial Optimization Problem (P) p min := inf x K p(x) where K := {x R n g 1 (x) 0...g m (x) 0} and p,g 1,...,g m are polynomials in n variables Semidefinite Programmingin Polynomial Optimization p. 2

3 Unconstrained Polynomial Minimization p min := inf x R n p(x) Example: An n n matrix M is copositive if p min 0 for p(x) := n i,j=1 x2 i x2 j M ij, i.e., p(x) is nonnegative on R n Example: A sequence a 1,...,a n N can be partitioned if p min = 0 for p(x) = ( n i=1 a ix i ) 2 + n i=1 (x2 i 1)2 Semidefinite Programmingin Polynomial Optimization p. 3

4 0/1 Linear Programming min c T x s.t. Ax b, x 2 i = x i i Example: The stability number α(g) of G = (V, E) can be computed via any of the programs: α(g) = max i V x i s.t. x i + x j 1 (ij E), x 2 i = x i (i V ) 1 α(g) = min xt (I + A G )x s.t. i V x i = 1, x i 0 (i V ) Hence: (P) is an NP-hard problem Semidefinite Programmingin Polynomial Optimization p. 4

5 Strategy Approximate (P) by a hierarchy of convex (semidefinite) relaxations Shor (1987), Nesterov, Lasserre, Parrilo (2000-) Such relaxations can be constructed using representations of nonnegative polynomials as sums of squares of polynomials and the dual theory of moments Semidefinite Programmingin Polynomial Optimization p. 5

6 Underlying paradigm: Testing whether a polynomial p is nonnegative is hard but one can test whether p is a sum of squares of polynomials efficiently via semidefinite programming Semidefinite Programmingin Polynomial Optimization p. 6

7 Some notation about polynomials Multivariate polynomial: p(x) = α Z n + p α x α 1 1 xα n n }{{} x α = α Z n + p α x α deg(x α ) = α := n i=1 α i deg(p) = max deg(x α ) s.t. p α 0 We identify a polynomial p(x) with its sequence of coefficients p = (p α ) α d, if deg(p) = d Semidefinite Programmingin Polynomial Optimization p. 7

8 p(x) is a sum of squares of polynomials (SOS) if p(x) = m [u j (x)] 2 for some polynomials u 1,...,u m j=1 Then, deg(p) is even and deg(u i ) deg(p)/2 Example: x 2 + y 2 + 2xy + z 6 = (x + y) 2 + (z 3 ) 2 is SOS Semidefinite Programmingin Polynomial Optimization p. 8

9 Recognizing SOS of pol s via SDP p(x) = p α x α α 2d i.e., p(x) = j is a sum of squares of polynomials [u j (x)] 2, where deg(u j ) d p(x) = z T ( j u j u T j )z, setting z := (x β ) β d }{{} X 0 The following semidefinite program is feasible: X 0 X β,γ = p α ( α 2d) β, γ d β+γ=α Semidefinite Programmingin Polynomial Optimization p. 9

10 system in matrix variable X of order ( n+d) ( d n+d ) d, with ) equations ( n+2d 2d polynomial size fixing either n or d Semidefinite Programmingin Polynomial Optimization p. 10

11 When does a polynomial have a decomposition as a sum of squares? Hilbert [1888] classified the pairs (n,d) for which every nonnegative polynomial of degree d in n variables is a sum of squares of polynomials: n = 1 d = 2 n = 2, d = 4 E.g., the Motzkin polynomial: x 4 y 2 + x 2 y 4 3x 2 y is nonnegative but not a SOS Semidefinite Programmingin Polynomial Optimization p. 11

12 Artin [1927] solved Hilbert s 17th problem [1900] ( pi ) 2, where p i,q i R[x 1,...,x n ] p 0 on R n = p = i q i That is, p q 2 is SOS for some q R[x 1,...,x n ] Sometimes, the common denominator is known: Pólya [1928] + Reznick [1995]: For p homogeneous p > 0 on R n \ {0} = p ( n i=1 x 2 i) r SOS for some r N Semidefinite Programmingin Polynomial Optimization p. 12

13 (Dual) SOS approximations for (P) following [Lasserre 2001] Recall: K = {x R n g l (x) 0 (l = 1,...,m)} p min := inf x K p(x) = sup λ s.t. p(x) λ 0 x K Replacing the nonnegativity condition by a stronger SOS condition: formulate (SOSt) σ t := sup λ s.t. p(x) λ = s 0 (x) + m l=1 s l(x)g l (x) s 0, s l are SOS deg(s 0 ), deg(s l g l ) 2t for any t deg(p)/2, deg(g l )/2 Semidefinite Programmingin Polynomial Optimization p. 13

14 σ t σ t+1 p min Example: K = R n (unconstrained case), deg(p) = 2d σ t = σ d p min, with equality iff p(x) p min is SOS Equality in the univariate case (Shor [1987], Nesterov [2000]) Semidefinite Programmingin Polynomial Optimization p. 14

15 (Primal) Moment Approximations p min = inf x K p(x) = inf µ P K p(x)µ(dx) = inf y M K p T y p(x)µ(dx) = α p α x α µ(dx) µ P K if µ is a probability measure supported by K y M K if y α = x α µ(dx) }{{} moment of order α of the measure µ α, for some µ P K Semidefinite Programmingin Polynomial Optimization p. 15

16 Lemma: If y is the sequence of moments of µ P K then y 0 = 1 M t (y) := (y α+β ) α, β t 0 for t 0 moment matrix of order t M t (g l y) := ( γ (g l) γ y α+β+γ ) α, β t 0 for t 0 localizing matrix Moreover: supp(µ) Zeros(p) p KerM t (y) Semidefinite Programmingin Polynomial Optimization p. 16

17 Proof: For p = (p α ) α t, p T M t (y)p = p(x) 2 µ(dx) 0 p T M t (g l y)p = g l (x)p(x) 2 µ(dx) 0 Semidefinite Programmingin Polynomial Optimization p. 17

18 If we relax the moment condition y M K " by requiring positive semidefiniteness of its moment matrix, we obtain a hierarchy of tractable SDP relaxations: (MOMt) p t := inf p T y s.t. y 0 = 1, M t (y) 0 M t dl (g l y) 0 (l = 1,...,m) for any t deg(p)/2, d l := deg(g l )/2 p t p t+1 p min Semidefinite Programmingin Polynomial Optimization p. 18

19 Lemma: The semidefinite programs (SOSt) and (MOMt) are dual of each other. Weak duality: σ t p t No duality gap, e.g., if K has a nonempty interior Semidefinite Programmingin Polynomial Optimization p. 19

20 Properties of the bounds σ t p t p min For fixed t, one can compute σ t, p t in polynomial time (to any fixed precision) Asymptotic (finite) convergence to p min via some representation theorems for positive polynomials from real algebraic geometry Optimality certificate via some theorems of Curto and Fialkow about moment matrices Extracting global minimizers via the eigenvalue method for solving systems of polynomial equations Semidefinite Programmingin Polynomial Optimization p. 20

21 Asymptotic convergence Representation theorem: [Putinar 1993] (more elementary proof by [Schweighofer 2003]) If u l SOS : {x l u l (x)g l (x) 0} is compact ( ) then p > 0 on K = p = s 0 + l s lg l with s 0,s l SOS Convergence theorem: [Lasserre 2001] If ( ) then lim σ t = lim p t = p min t t Semidefinite Programmingin Polynomial Optimization p. 21

22 Note: ( ) holds, e.g., if {x g l (x) 0} is compact for some l if the equations x 2 i = x i are present in the description of K (0/1 case) if the radius R of a ball containing K is known, then add the (redundant) constraint R 2 i x2 i 0 to the description of K Note: A representation result valid for p 0 on K" would give a finite convergence result Semidefinite Programmingin Polynomial Optimization p. 22

23 Finite Convergence: Finite Variety Assume the equations: h 1 (x) = 0...h k (x) = 0 are present in the description of K and V := {x C n h 1 (x) = 0...h k (x) = 0} is finite Then: p t = p min for t large enough [Laurent 2002] Moreover: σ t = p t = p min for t large enough if h 1,...,h k form a Groebner basis [Laurent 2002] or if they generate a radical ideal [Parrilo 2002] Representation result in radical case: [Parrilo 2002] p 0 on K = p = s 0 + l s lg l (s 0,s l SOS) Semidefinite Programmingin Polynomial Optimization p. 23

24 In the 0/1 case p t = p min for t n + d where d := max l deg(g l )/2 [Lasserre 2001] The hierarchy of bounds p t refines other combinatorial hierarchies, e.g., by Lovász-Schrijver [1991] (cf. [Laurent 2003]) Note: One can use the equations h 1 (x) = 0...h k (x) = 0 to reduce the number of variables in the moment relaxations (MOMt) Semidefinite Programmingin Polynomial Optimization p. 24

25 Optimality Certificate Theorem: [Henrion-Lasserre 2005] If y is an optimum solution to the program (MOMt) satisfying: (RC) rankm t (y) = rankm t d (y), where d = max l deg(g l )/2 then: p t = p min Semidefinite Programmingin Polynomial Optimization p. 25

26 Proof By a theorem of [Curto-Fialkow 1996] (cf. [Laurent 2004]) y is the sequence of moments of an r-atomic nonnegative measure µ µ = r i=1 λ i δ xi, where λ i > 0, i λ i = 1, x i K = y = r i=1 λ i (x α i ) α 2t = p min p t = p T y = r i=1 λ i p(x i ) p min Moreover: x 1,...,x r are global minimizers over K Semidefinite Programmingin Polynomial Optimization p. 26

27 Extracting global minimizers Then, supp(µ) = {x 1,...,x r } {global minimizers} Moreover, supp(µ) = Zeros(p) p KerM t (y) Therefore: One can find the global minimizers x 1,...,x r by computing the common zeros to a system of polynomial equations Use the eigenvalue method Based on computing the eigenvalues of the r r multiplication matrices in the polynomial ring R[x 1,...,x n ]/I, where I is the ideal generated by the kernel of M t (y) Semidefinite Programmingin Polynomial Optimization p. 27

28 Implementations of the SOS/moment relaxation method GloptiPoly by Henrion, Lasserre SOSTOOLS by Prajna, Papachristodoulou, Parrilo Semidefinite Programmingin Polynomial Optimization p. 28

29 Example 1 min p = 25(x 1 2) 2 (x 2 2) 2 (x 3 1) 2 (x 4 4) 2 (x 5 1) 2 (x 6 4) 2 s.t. (x 3 3) 2 + x 4 4, (x 5 3) 2 + x 6 4 x 1 3x 2 2, x 1 + x 2 2, x 1 + x 2 6, x 1 + x 2 2, 1 x 3 5, 0 x 4 6, 1 x 5 5, 0 x 6 10, x 1,x 2 0 order t bound p t solution extracted 1 unbounded none (5,1,5,0,5,10) The global minimum is found at the relaxation of order t = 2 Semidefinite Programmingin Polynomial Optimization p. 29

30 Example 2 min p = x 1 x 2 s.t. x 2 2x 4 1 8x x x 2 4x x x2 1 96x x 1 3, 0 x 2 4 order t bound p t solution extracted 2-7 none none (2.3295,3.1785) The global minimum is found at the relaxation of order t = 4 Semidefinite Programmingin Polynomial Optimization p. 30

31 Unconstrained polynomial minimization As K = R n, the relaxation scheme just gives one bound: σ d = p d p min, with equality iff p(x) p min is SOS Idea: Get better bounds by transforming the unconstrained problem into a constrained problem If p has a minimum: p min = p grad := inf x V R grad p(x) where V R grad := {x Rn p x i = 0 (i = 1,...,n)} Semidefinite Programmingin Polynomial Optimization p. 31

32 If, moreover, a bound R is known on the norm of a global minimizer: p min = p ball := inf R 2 i x2 i 0p(x) Semidefinite Programmingin Polynomial Optimization p. 32

33 When p attains its minimum: The ball approach : p ball can be approximated via Lasserre s relaxation scheme (asymptotic convergence, as Putinar s assumption holds!) seems to work well only if the radius R of the ball is not too large... The gradient variety approach: Representation result: [Demmel, Nie, Sturmfels 2004] p > 0 on V R grad = p = s 0 + n i=1 u i p x i where s 0 SOS, u i R[x 1,...,x n ] Semidefinite Programmingin Polynomial Optimization p. 33

34 Convergence result: [Demmel, Nie, Sturmfels 2004] There is asymptotic convergence of the SOS and moment bounds to p grad Moreover: When p x 1,..., p x n generate a radical ideal, the representation result holds for p 0, implying the finite convergence Semidefinite Programmingin Polynomial Optimization p. 34

35 What if p is not known to have a minimum? Strategy: Perturb the polynomial p [Hanzon-Jibetean 2003] [Jibetean-Laurent 2004] n p ǫ (x) := p(x) + ǫ( x 2d+2 i ) for small ǫ > 0 i=1 p ǫ has a minimum and lim ǫ 0 (p ǫ ) min = p min the gradient variety of p ǫ is finite one can compute (p ǫ ) min via the gradient variety approach finite convergence of the moment bounds to (p ǫ ) min (in 2nd steps) Semidefinite Programmingin Polynomial Optimization p. 35

36 one can use the gradient equations: p ǫ = (2d + 2)x 2d+1 x i i + p x i = 0 i to reduce # variables ( only variables y α with α i 2d i) and to express (p ǫ ) min via a SDP with a single LMI the limits of global minimizers of p ǫ (as ǫ 0) are global minimizers of p (if any) Semidefinite Programmingin Polynomial Optimization p. 36

37 Examples Motzkin polynomial: p = x 2 y 2 (x 2 + y 2 3) + 1 Then, p min = 0, attained at (±1, ±1), and p is not SOS ǫ order t (r t,r t 1,r t 2 ) (p ǫ ) t extracted solutions 3 (10,6,3) none (4,4,4) (±0.9453, ±0.9453) (4,4,4) (±0.9935, ±0.9935) (4,4,4) (±0.9993, ±0.9993) (4,4,4) (±0.9999, ±0.9999) Semidefinite Programmingin Polynomial Optimization p. 37

38 Example: p = (xy 1) 2 + y 2 Then, p min = 0 is not attained ǫ order t (r t,r t 1,r t 2 ) (p ǫ ) t extracted solutions 2 (4,2,1) none (2,2,2) ±(1.3981, ) (2,2,2) ±(1.9499, ) (2,2,2) ±(2.6674, ) (2,2,2) ±(3.6085, ) (2,2,2) ±(4.8511, ) (3,2,2) ±(6.4986, ) (3,2,2) ±(8.6882, ) (8,4,2) none (7,4,2) none Semidefinite Programmingin Polynomial Optimization p. 38

39 Testing copositivity of a matrix M M copositive iff p min = 0 for p(x) := n i,j=1 x2 i x2 j M ij Example 1: M = M is copositive, e.g., since ( i x2 i )p is SOS [Parrilo 2000] Semidefinite Programmingin Polynomial Optimization p. 39

40 ǫ order t (r t,r t 1,r t 2 ) (p ǫ ) t extracted solutions 2 (5,15) none (18,6,3) none (4,4,4) (±0.7058, 0, 0, ±0.7058) (9,7,5) none Semidefinite Programmingin Polynomial Optimization p. 40

41 Example 2: M = ǫ order t (r t, r t 1, r t 2 ) (p ǫ ) t extracted solutions 1 3 (8,7,4) none 1 4 (8,8,7) ±(0.8165, , , 0, 0) (8,7,4) none (8,8,7) ±(2.5820, ,2.5820, 0,0) (8,7,4) none certificate that M is not copositive Semidefinite Programmingin Polynomial Optimization p. 41

42 Approximating the stability number via matrix copositivity and Pólya s representation result [de Klerk-Pasechnik 2002] [Parrilo 2000] n M strictly copositive, i.e., p M := x 2 i x2 j M ij > 0 on R n \ {0} i,j=1 [Pólya] = ( i x 2 i )r p M is SOS for some r N = M copositive [Motzkin-Straus 1965] α(g) = min t s.t. t(i + A G ) J is copositive Semidefinite Programmingin Polynomial Optimization p. 42

43 Define: ϑ (r) (G) := min t s.t. t(i + A G ) J K (r) where K (r) := {M ( i x2 i )r p M is SOS} Theorem: [dkp] ϑ (r) (G) = α(g) if r α(g) 2 Conjecture: [dkp] ϑ (r) (G) = α(g) if r α(g) 1 Motivation: this rate of convergence holds for other hierarchies, e.g., by Lovász-Schrijver, Lasserre Theorem: [Gvozdenović-L 2004] True for α(g) 8 Lasserre s hierarchy refines the hierarchy ϑ (r) (G) Other results by [Peña-Vera-Zuluaga 2005] Semidefinite Programmingin Polynomial Optimization p. 43

44 Yet another representation result Theorem: [Lasserre 2004] p 0 on R n = ǫ > 0 r N for which ( r ) n x 2k i p r,ǫ := p + ǫ k! k=0 i=1 is SOS Note: p r,ǫ p 1 0 as ǫ ց 0 Although there are much more nonnegative polynomials than SOS polynomials (at fixed degree, when n ) [Blekherman 2003] Semidefinite Programmingin Polynomial Optimization p. 44

45 new relaxation scheme for p min = inf x R n p(x) inf y T p r,ǫ s.t. M r (y) 0, y 0 = 1 extension to the constrained case Semidefinite Programmingin Polynomial Optimization p. 45

46 Many other interesting results unfortunately not covered here... SDP versus LP relaxations [Sherali-Adams] [Lasserre] Exploiting the symmetry to reduce the size of the SDP [Parrilo-Gaterman] [Schrijver] [Dukanovic-Rendl] [de Klerk-Pasechnik-Schrijver] [Laurent]... invariant theory, block-diagonalization of C -algebras Exploiting the sparsity of the polynomials to reduce the size of the SDP [Reznick 1978] [Kim-Kojima-Muramatsu-Waki]... Extension to matrix polynomial optimization [Kojima] [Hol-Scherer] [Henrion-Lasserre]... Semidefinite Programmingin Polynomial Optimization p. 46