Sum of Squares. 1 Positive Semidefinite Matrices and Nonnegative Polynomials

Transcription

1 Sum of Squares If a real polynomial can be written as a sum of squares, then this sum provides a certificate that the polynomial has no real roots. In this lecture, mainly following [10] we will see how with semidefinite programming we may decide whether a polynomial can be written as a sum of squares. A recent survey is in [4]. 1 Positive Semidefinite Matrices and Nonnegative Polynomials Consider a real polynomial p R[x]. If p can be written as a sum of squares, i.e.: if there exist r polynomials q i R[x], i = 1,2,...,r, such that p = q 2 1 +q q 2 r, then p(x) 0, for all x R. Moreover, for any ǫ > 0, p+ǫ > 0 is positive and has no real roots. Not every nonnegative polynomial is a sum of squares. We will make the connection between positive semidefinite matrices and sum of squares. A matrix A is positive semidefinite if for all vectors x: x T Ax 0. A symmetric positive semidefinite matrix A admits a decomposition as A = R T R, where R is an upper triangular matrix. This is known as the Cholesky decomposition (the command chol in Octave) of A. Matrices arise in the representation of polynomials as follows. Take as p for example a fourth degree polynomial, which we may represent as p(x) = [x 2 x 1] a 11 a 12 a 13 a 12 a 22 a 23 x2 x = x T Ax. (1) a 13 a 23 a 33 1 If A is positive semidefinite, then its Cholesky decomposition A = R T R gives p(x) = x T Ax = x T R T Rx = (Rx) T (Rx) = y 2 1 +y 2 2 +y 3 3, y = Rx. (2) We are interested in the positive definite cone (or PSD cone) of all real symmetric positive semidefinite n-by-n matrices PSD(R n ) = { A R n n A T = A and A is positive semidefinite }. (3) That PSD(R n ) is a cone follows from the inequalities x T Ax 0. Semidefinite programming (or SDP for short) offers efficient algorithms for the following problems: SDP decision: givenr linearfunctions l 1,l 2,...,l r, doesthereexistsana PSD(R n )suchthatl i (A) = 0, for i = 1,2,...,r? SDP optimization: for r +1 linear functions l 0,l 1,...,l r, minimize l 0 (A) subject to A PSD(R n ) and l i (A) = 0, for i = 1,2,...,r. While this clearly generalizes the classical linear programming models, interior-point methods provide efficient algorithms to solve these problems [11]. For the polynomial p(x) = x 4 x 2 2x+2 (example taken from [10]), we consider x T Ax = a 11 x 4 +2a 12 x 3 +(2a 13 +a 22 )x 2 +a 23 x+a 33. (4) Now we want to find those A PSD(R n ) satisfying the linear equations a 11 = 1, a 12 = 0, 2a 13 +a 22 = 1, a 23 = 1, and a 33 = 2. It turns out that this SDP decision problem has a unique solution so that p can be written as a sum of squares. Consider for example the set defined by 1 x 2 +y 2 2. This is clearly not a convex set. Can we use convex optimization to compute with this set? The idea is to view the set defined by 1 x 2 +y 2 2 as the projection of the convex set { (x,y,z) R 3 x 2 + y 2 z,1 z 4 }. This lifting is a useful relaxation technique. Jan Verschelde UIC, Dept of Math, Stat & CS Lecture 32, page 1

2 2 Control of Nonlinear Systems via Lyapunov Functions Following [6] (see also [8]), consider a nonlinear system x(t) t = f(x(t)), (5) where f(x) is a real polynomial system in x. To prove asymptotic stability of x = 0 as a fixed point of this nonlinear system, we must find a Lyapunov function V(x) satisfying for all x 0 : V(x) > 0 and V(x(t)) t = ( ) T V f(x) < 0, (6) x for all nonzero x in a neighborhood of the origin. Expressing V(x) as a sum of squares gives an explicit way of showing its positivity. To automate the search for a Lyapunov function, parameters λ are introduced, so V = V(x,λ). For z a monomial vector, we then represent V as a quadratic form in z: V(x,λ) = z T Q(λ)z. The parameters λ are then determined so that Q(λ) is positive semidefinite. 3 Global Optimization Via semidefinite programming we may find the global minimum of a polynomial function f on R n. Without semidefinite programming, we could compute all critical points of the system defined by all partial derivatives f x i (x) = 0, for i = 1,2,...,n. The number of solutions of this system however grows exponentially and solving this system of partial derivatives is feasible only for a very modest number of variables and for low degree polynomials. Instead, we consider a relaxation of the problem: SOS Relaxation: Find the largest λ R such that f(x 1,x 2,...,x n ) λ is a sum of squares. The dimension of the vector space( N for which ) we consider PSD(R N ) is proportional to the number n+d of monomials in f, i.e. it grows like, where d = deg(f). Although also this dimension grows d exponentially, the growth is more moderate compared to the system of partial derivatives. 4 Sum of Squares and Radical Ideals The definition of radical of an ideal I is I = { f R[x] f k I for some integer k 1 }. (7) Hilbert s Nullstellensatz states I = I(V) for a variety V. In words, the radical of an ideal is the vanishing ideal of the solutions. Lemma 4.1 gives a criterion for an ideal to be radical, limiting the powers to squares. Lemma 4.1 Let I be an ideal in R[x]. I is radical if and only if for all f R[x] : f 2 I f I. (8) Proof. First, consider the in the if and only if, assuming I is radical. If I is radical, then for all integer k 1: f k I implies f I. Take k = 2 and (8) follows. Jan Verschelde UIC, Dept of Math, Stat & CS Lecture 32, page 2

3 Second, to show the in the if and only if, assuming (8) holds, we show that f I, if f k I, for some integer k 1. For example, let k = 4: if f 4 I, then (f 2 ) 2 I implies f 2 I, using (8). Applying (8) once more results in f I. For k = 2 l, applying (8) l times yields f I. If k is not a power of 2, we multiply f k sufficiently many times with f till we have f 2l I. The real radical of an ideal I is R I = f R[x] f2k + p 2 j I for some k 1,p j R[x]. (9) Similar to Lemma 4.1, we have a criterion for an ideal to be real radical in Lemma 4.2. Lemma 4.2 Let I be an ideal in R[x]. I is real radical if and only if for all p j R[x] : p 2 j I p j I. (10) Proof. First, consider the in the if and only if, assuming I is real radical. If I is real radical, i.e.: I = R I, then for all integer k 1 and p j R[x]: f 2k + p 2 j I implies f I. Take k = 2 and (10) follows. Second, toshowthe intheif and only if, assuming(10)holds. Letf,p j R[x]suchthatf 2k + p 2 j I holds. By (10), we have f k,p j I. As (10) implies (8), we apply Lemma 4.2 to deduce f I. For a polynomial system f(x) = 0, with f = (f 1,f 2,...,f N ), f i C[x], for i = 1,2,...,N. Hilbert s Nullstellensatz implies f 1 (0) = f = 1. (11) So if the system f(x) = 0 has no solutions, then there are polynomials g i C[x], i = 1,2,...,N, such that 1 = g 1 f 1 +g 2 f 2 + +g N f N. The polynomials g i provide a certificate that f(x) = 0 has no solutions. See [2] for more on Hilbert s Nullstellensatz. Either a system of real polynomial equations and inequalities has a real solution, or there is a certificate that no solution exists. Such certificate consists in a polynomial identity listed in the theorem [10] below. Theorem 4.1 (Real Nullstellensatz) The system of real polynomial equations and inequalities f 1 (x) = 0,f 2 (x) = 0,...,f r (x) = 0 g 1 (x) 0,g 2 (x) 0,...,g s (x) 0 h 1 (x) > 0,h 2 (x) > 0,...,h t (x) > 0 (12) has a solution for x R n, or there exists a polynomial identity r α i f i + b 2 jν g ν1 1 gν2 2 gνs s + c 2 jν h ν1 1 hν2 2 hνs t + i=1 j j k ν {0,1} s ν {0,1} t d 2 k + t l=1 h u l l = 0, (13) where u j N and a i, b jν, c jν, and d k are polynomials. To get a better understanding of this theorem it is instructive to consider certain special cases, for example s = 0 = t. In real algebraic geometry the focus is on real roots and one studies semi-algebraic sets [1]. In [6], it is stated that a Real Nullstellensatz certificate of bounded degree can be computed efficiently by semidefinite programming. Jan Verschelde UIC, Dept of Math, Stat & CS Lecture 32, page 3

4 5 Moment Matrices In this section, we follow [5] and [9]. We can represent a polynomial with support A as f(x) = a Ac a x a R[x] by its coefficient vector c A with respect to some fixed monomial basis (x a ) a A. The corresponding dual basis uses differentials ( a ) a A. Given a sequence of numbers y = (y a ) a A R m, for m = #A, we consider a linear functional Λ (R[x]) as Λ = a Ay a a. Then y is the coordinate sequence y = (Λ(x a )) a A. Given the linear form Λ, we define the moment matrix as M(y) = (Λ(x a+b ) a,b = (y a+b ) a,b, (14) with rows and columns indexed by a,b A. For a polynomial f as denoted above, we have the inner product f,f Λ = c T A M(y)c A = Λ(f 2 ). (15) By construction, the positive semidefiniteness of M(y) is equivalent to Λ(f 2 ) 0. An interpretation of the moment matrix is the matrix representing the quasi-hankel operator: We have h Λ(f) = Λ(hf) for all f R[x]. The kernel of the moment matrix is H Λ : R[x] (R[x]) : h h Λ. (16) kerm(y) = { f R[x] M(y)c A = 0 }. (17) The kernel kerm(y) is an ideal in R[x]. If M(y) is positive semidefinite, then kerm(y) is real radical. In computations, we consider polynomials of degree at most d, denoted by R[x] d. Given a linear form Λ = a Ay a a, the matrix is the truncated moment matrix. M d/2 (y) = Λ(x a+b ) a,b = (y a+b ) a,b (18) The method is to find y for which the rank of the moment matrix is maximum. This rank equals the number of real solutions. Denote V C (I) the solutions in C n and by V R (I) = V C (I) R n the real solutions. We end with an algorithm of [9]. Algorithm 5.1 (Moment-Matrix algorithm) Input: I = f 1,f 2,...,f N, with #V R (I) <. Output: a border or Gröbner basis for J := kerm d (y), with V C (J) = V C (I). 1. d := D := N max i=1 deg(f i); 2. Find a generic y; 3. if (rankm s (y) = rankm s 1 (y), for some D s d/2 or rankm s (y) = rankm s t (y) for some t s d/2 ) 4. then return a basis for the column space of M d 1 (y); 5. else d := d+1; go to step 2; 6. end if. Jan Verschelde UIC, Dept of Math, Stat & CS Lecture 32, page 4

5 6 Exercises 1. Verify that we may write every fourth degree polynomial as (1), although not necessarily in a unique way. Can you generalize this representation for polynomials of any degree? 2. Suppose we want to compute the global minimum of a cubic polynomial f in 10 variables. Use Bézout s theorem to bound the number of solutions of the system of all partial derivatives. Compare this number with the number of monomials in f. 3. Install SOSTOOLS on your computer and use it for example through its Macaulay 2 interface [7]. Report running times for the examples in [7]. 4. Consider the family of homogeneous polynomials M jk of [3]: where j and k are positive integers. M jk (x 1,x 2,x 3 ) = jx 6 3 +x2 1 x2 2 (jx2 1 +jx2 2 kx2 3 ), (19) Give examples of numerical evaluations of M jk for some instances of j and k at well chosen values for x 1, x 2, and x 3 to illustrate the importance of positivity on the accuracy of the evaluation. References [1] S. Basu, R. Pollack, and M.-F. Roy. Algorithms in Real Algebraic Geometry, volume 10 of Algorithms and Computation in Mathematics. Springer-Verlag, [2] D. Cox, J. Little, and D. O Shea. Ideals, Varieties and Algorithms. An Introduction to Computational Algebraic Geometry and Commutative Algebra. Undergraduate Texts in Mathematics. Springer Verlag, second edition, [3] J. Demmel, I. Dumitriu, and O. Holtz. Toward accurate polynomial evaluation in rounded arithmetic. In L.M. Pardo, A. Pinkus, E. Süli, and M.J. Todd, editors, Foundations of Computational Mathematics, Santander 2005, volume 331 of London Mathematical Society Lecture Note Series, pages Cambridge University Press, arxiv:math/ v2 [math.na] 18 Jan [4] J.-B. Lasserre. A sum of squares approximation of nonnegative polynomials. SIAM Review, 49(4): , [5] J.-B. Lasserre, M. Laurent, and P. Rostalski. Semidefinite characterization and computation of zerodimensional real radical ideals. Foundations of Computational Mathematics, 8(5): , [6] P. Parrilo. Structured Semidefinite Programs and Semialgebraic Geometry Methods in Robustness and Optimization. PhD thesis, California Institute of Technology, [7] H. Peyrl and P.A. Parrilo. A Macaulay 2 package for computing sum of squares decompositions of polynomials with rational coefficients. In J. Verschelde and S.M. Watt, editors, SNC 07. Proceedings of the 2007 International Workshop on Symbolic-Numeric Computation, pages ACM, [8] S. Prajna, P.A. Parrilo, and A. Rantzer. Nonlinear control synthesis by convex optimization. IEEE Transactions on Automatic Control, 42(2): , [9] P. Rostalski. Algebraic Moments. Real Root Finding and Related Topics. PhD thesis, ETH Zürich, [10] B. Sturmfels. Solving Systems of Polynomial Equations. Number 97 in CBMS Regional Conference Series in Mathematics. AMS, [11] L. Vandenberghe and S. Boyd. Semidefinite programming. SIAM Review, 38(1):49 95, Jan Verschelde UIC, Dept of Math, Stat & CS Lecture 32, page 5