# AN EXTENSION OF THE ELIMINATION METHOD FOR A SPARSE SOS POLYNOMIAL

Save this PDF as:

Size: px
Start display at page:

Download "AN EXTENSION OF THE ELIMINATION METHOD FOR A SPARSE SOS POLYNOMIAL"

## Transcription

1 AN EXTENSION OF THE ELIMINATION METHOD FOR A SPARSE SOS POLYNOMIAL Hayato Waki Masakazu Muramatsu The University of Electro-Communications (September 20, 2011) Abstract We propose a method to reduce the sizes of SDP relaxation problems for a given polynomial optimization problem (POP). This method is an extension of the elimination method for a sparse SOS polynomial in [8] and exploits sparsity of polynomials involved in a given POP. In addition, we show that this method is a partial application of a facial reduction algorithm, which generates a smaller SDP problem with an interior feasible solution. In general, SDP relaxation problems for POPs often become highly degenerate because of a lack of interior feasible solutions. As a result, the resulting SDP relaxation problems obtained by this method may have an interior feasible solution, and one may be able to solve the SDP relaxation problems effectively. Numerical results in this paper show that the resulting SDP relaxation problems obtained by this method can be solved fast and accurately. Keywords: Semidefinite programming, semidefinite programming relaxation, polynomial optimization, facial reduction algorithm 1. Introduction The problem of detecting whether a given polynomial is globally nonnegative or not, appears in various fields in science and engineering. For such problems, Parrilo [11] proposed an approach via semidefinite programming (SDP) and sums of squares (SOS) of polynomials. Indeed, he replaced the problem by another problem of detecting whether a given polynomial can be represented as an SOS polynomial or not. If the answer is yes, then the global nonnegativity of the polynomial is guaranteed. It is known in Powers and Wörmann [13] that the latter problem can be converted as an SDP problem equivalently. Therefore, one can solve the latter problem by using existing SDP solvers, such as SeDuMi [18], SDPA [5], SDPT3 [19] and CSDP [1]. However, in the case where the given polynomial is large-scale, i.e., the polynomial has a lot of variables and/or higher degree, the resulting SDP problem becomes too large-scale to be handled by the state of the arts computing technology, practically. To recover this difficulty, for a sparse polynomial, i.e., a polynomial which has few monomials, Kojima et al. in [8] proposed an effective method for reducing the size of the resulting SDP problem by exploiting the sparsity of the given polynomial. Following [23], we call the method the elimination method for a sparse SOS polynomial (EMSSOSP). In this paper, we deal with the problem to detect whether a given polynomial is nonnegative over a given semialgebraic set or not. More precisely, for given polynomials f, f 1,..., f m, the problem is to detect whether f is nonnegative over the set D := {x R n f 1 (x) 0,..., f m (x) 0} or not. For this problem, we apply a similar argument in 1

2 Parrilo [11]. Then we obtain the following problem: Find σ j for j = 1,..., m, m subject to f(x) = f j (x)σ j (x) ( x R n ), σ j is an SOS polynomial for j = 1,..., m. (1.1) If we find all SOS polynomials σ j in (1.1), then the given polynomial f is nonnegative over the set D. However, (1.1) is still intractable because the degree of each σ j is unknown in advance. Therefore, we put a bound of the degree of σ j. Then, we can convert (1.1) to an SDP problem by applying Lemma 2.1 in [8] or Theorem 1 in [13] to the problem. In this case, we have the same computational difficulty as the problem of finding an SOS representation of a given polynomial, namely the resulting SDP problem becomes large-scale if f, f 1,..., f m has a lot of variables and/or higher degree. The contribution of this paper is to propose a method for reducing the size of the resulting SDP problem if f, f 1,..., f m are sparse. To this end, we extend EMSSOSP, so that the proposed method removes unnecessary monomials in any representation of f with f 1,..., f m from σ j (j = 1,..., m). If f, f 1,..., f m are sparse, then we can expect that the resulting SDP problem obtained by our proposed method becomes smaller than the SDP problem obtained from (1.1). In this paper, we call our proposed method EEM, which stands for the Extension of Elimination Method. Another contribution of this paper is to show that EEM is a partial application of a facial reduction algorithm (FRA). FRA was proposed by Borwein and Wolkowicz [2, 3]. Ramana et al. [17] showed that FRA for SDP with nonempty feasible region generates an equivalent SDP with an interior feasible solution. In addition, Pataki [12] simplified FRA of Borwein and Wolkowicz. Waki and Muramatsu [22] proposed FRA for conic optimization problems. It is pointed out in [23] that EMSSOSP is a partial application of FRA. In general, SDP relaxation problems for polynomial optimization problems (POPs) become highly degenerate because of a lack of interior feasible solutions. As a consequence, the resulting SDP problems obtained by EEM may have an interior feasible solution, and thus we can expect an improvement on the computational efficiency of primal-dual interior-point methods. The organization of this paper is as follows: We discuss our problem and usage of the sparsity of given polynomials f, f 1,..., f m in Section 2 and propose EEM in Section 3. SDP relaxations [9, 20] for POPs are applications of EEM. We apply EEM to some test problems in GLOBAL Library [6] and randomly generated problems, and solve the resulting SDP relaxation problems by SeDuMi [18]. In Section 4, we present the numerical results. When we execute EEM to remove unnecessary monomials in SOS representations, there is a flexibility in EEM. We focus on the flexibility and show facts on SDP relaxation for POPs and SOS representations of a given polynomial in Section 5. A relationship between our method and FRA is provided in Section 6. Section 7 is devoted to concluding remarks. We give some discussion and proofs on EEM in Appendix Notation and symbols S n and S n + denote the sets of n n symmetric matrices and n n symmetric positive semidefinite matrices, respectively. For X, Y S n, we define X Y = n i, X ijy ij. For 2

3 every finite set A, #(A) denotes the number of elements in A. We define A + B = {a + b a A, b B} for A, B R n. We remark that A+B = when either A or B is empty. For A R n and α R, αa denotes the set {αa a A}. Let N n be the set of n-dimensional nonnegative integer vectors. We define N n r := {α N n n i=1 α i r}. Let R[x] be the set of polynomials with n-dimensional real vector x. For every α N n, x α denotes the monomial x α 1 1 x α n n. For f R[x], let F be the set of exponents α of monomials x α whose coefficients f α are nonzero. Then we can write f(x) = α F f αx α. We call F the support of f. The degree deg(f) of f is the maximum value of α := n i=1 α i over the support F. For G N n, R G [x] denotes the set of polynomials whose supports are contained in G. In particular, R r [x] is the set of polynomials with the degree up to r. 2. On our problem and exploiting the sparsity of given polynomials f, f 1,..., f m 2.1. Our problem In this subsection, we discuss how to convert (1.1) into an SDP problem. For a finite set G N n, let Σ G be the set of SOS polynomials whose supports are contained in G. In particular, we denote Σ r if G = N n r. Let d = deg(f)/2, d j = deg(f j )/2 for all j = 1,..., m and r = max{d, d 1,..., d m }. We fix a positive integer r r and define r j = r d j for all j = 1,..., m. Then we obtain the following SOS problem from (1.1): Find σ j Σ rj for all j = 1,..., m, m subject to f(x) = f j (x)σ j (x) ( x R n ) (2.1) We say that f has an SOS representation with f 1,..., f m if (2.2) has a solution (σ 1,..., σ m ). In this case, (1.1) also has a solution, and thus f is nonnegative over the set D. We assume that σ j Σ Gj in any SOS representations of f with f 1,..., f m. Then we can obtain the following SOS problem from (2.1): Find σ j Σ Gj for all j = 1,..., m, m subject to f(x) = f j (x)σ j (x) ( x R n ). (2.2) Note that SOS problem (2.2) is equivalent to SOS problem (2.1) by the assumption. Let F and F j be the support of f and f j, respectively. Without loss of generality, we assume F m (F j + G j + G j ). In fact, if F m (F j + G j + G j ), then (2.2) does not have any solutions. To reformulate (2.2) into an SDP problem, we use the following lemma: Lemma 2.1 (Lemma 2.1 in [8]) Let G be a finite subset of N n and u(x, G) = (x α : α G). Then, f is in Σ G if and only if there exists a positive semidefinite matrix V S #(G) + such that f(x) = u(x, G) T V u(x, G) for all x R n. We apply Lemma 2.1 to σ j in (2.2). Then we can reformulate (2.2) into the following problem: { Find V j S #(G j) + for all j = 1,..., m, subject to f(x) = m u(x, G j) T V j u(x, G j )f j (x) ( x R n ), 3 (2.3)

4 where u(x, G j ) be the column vector consisting of all monomials x α (α G j ). We define the matrices E j,α S #(Gj) for all α m (F j + G j + G j ) and for all j = 1,..., m as follows: { (fj ) (E j,α ) β,γ = δ if α = β + γ + δ and δ F j 0 o.w. (for all β, γ G j ). Comparing the coefficients in both sides of the identity in (2.3), we obtain the following SDP: { Find V j S #(G j) + for all j = 1,..., m, subject to f α = m E j,α V j, (α m (F (2.4) j + G j + G j )). We observe form (2.4) that the resulting SDP (2.4) may become small enough to handle it if G j is much smaller than N n r j for all j = 1,..., m. Remark 2.2 Some sets G j may be empty. For instance, if G 1 =, then the problem (2.1) is equivalent to the problem of finding an SOS representation of f with f 2,..., f m under the condition σ j Σ rj Exploiting the sparsity of given polynomials f, f 1,..., f m We present a lemma which plays an essential role in our proposed method EEM. EEM applies this lemma to (2.1) repeatedly, so that we obtain (2.2). This lemma is an extension of Corollary 3.2 in [8] and Proposition 3.7 in [4]. Lemma 2.3 Let G j N n be a finite set for all j = 1,..., m. f and f j denote polynomials with support F and F j, respectively. For δ m (F j + G j + G j ), we define J(δ) {1,..., m}, B j (δ) G j and T j F j + G j + G j as follows: J(δ) := {j {1,..., m} δ F j + 2G j }, B j (δ) := {α G j δ 2α F j }, T j := {γ + α + β γ F j, α, β G j, α β}. Assume that f has an SOS representation with f 1,..., f m and G 1,..., G m as follows: k m j f(x) = f j (x) α Gj(g 2 j,i ) α x α, (2.5) i=1 where (g j,i ) α is the coefficient of the polynomial g j,i. In addition, we assume that for a fixed δ m (F j + G j + G j ) \ (F m T j), J(δ) and (B 1 (δ),..., B m (δ)) satisfy { (fj ) δ 2α > 0 for all j J(δ) and α B j (δ) or (2.6) (f j ) δ 2α < 0 for all j J(δ) and α B j (δ). Then, f has an SOS representation with f 1,..., f m and G 1 \ B 1 (δ),..., G m \ B m (δ), i.e., (g j,i ) α = 0 for all j J(δ), α B j (δ) and i = 1,..., k j and k m j f(x) = f j (x) 2 (g j,i ) α x α. (2.7) i=1 α G j \B j (δ) 4

5 We postpone a proof of Lemma 2.3 until Appendix A. Remark 2.4 We have the following remarks on Lemma If f is not sparse, i.e., f has a lot of monomials with nonzero coefficient, then the set m (F j + G j + G j ) \ (F m T j) may be empty. In this case, we do not have any candidates δ for (2.6). In addition, if f 1,..., f m are sparse, then coefficients to be checked in (2.6) may be few in number, and thus we can expect that there exists δ such that J(δ) and B j (δ) satisfy (2.6). 2. Lemma 2.3 is an extension to Corollary 3.2 in Kojima et al. [8] and (2) of Proposition 3.7 in Choi et al. [4]. In fact, the authors deal with the case where m = 1 and f 1 = 1 in these papers. In that case, we have F 1 = {0} and the coefficient (f 1 ) 0 of a constant term in f is 1. We give an example of notation J(δ), B j (δ) and T j. Example 2.5 Let f = x, f 1 = 1, f 2 = x and f 3 = x 2 1. Then we have F = {1}, F 1 = {0}, F 2 = {1} and F 3 = {0, 2}. We consider the case where we have G 1 = {0, 1, 2}, G 2 = G 3 = {0, 1}. Then we obtain ( 3 ) (F j + G j + G j ) \ F = {0, 2, 3, 4}, T 1 = {1, 2, 3}, T 2 = {2} and T 3 = {1, 3}. In this case, we can choose δ {0, 4}. If we choose δ = 4, then J(δ) = {1, 3} and we obtain B 1 (δ) = {2}, B 2 (δ) = and B 3 (δ) = {1}. Moreover, J(δ) and (B 1 (δ), B 2 (δ), B 3 (δ)) satisfy (2.6). Lemma 2.3 implies that if f has an SOS representation with f j and G j, then f also has an SOS representation with f j, G 1 \ B 1 (δ) = {0, 1}, G 2 \ B 2 (δ) = {0, 1} and G 3 \ B 3 (δ) = {0}. 3. An extension of EMSSOSP For given polynomials f, f 1,..., f m and a positive integer r r, we set r j = r deg(f j )/2 for all j. We assume that f can be represented as follows: f(x) = m f j (x)σ j (x) (3.1) for some σ j Σ rj. We remark that the support of σ j is contained in N n 2r j. By applying Lemma 2.3 repeatedly, our method may remove unnecessary monomials of σ j in (3.1) for all SOS representations of f with f 1,..., f m and G 1,..., G m before deciding all coefficients of σ j. We give the detail of our method in Algorithm 3.1. Algorithm 3.1 (The elimination method for a sparse SOS representation with f 1,..., f m ) Input polynomials f, f 1,..., f m, Output G := (G 1,..., G m). Step 1 Set i = 0 and G 0 := (N n r 1,..., N n r m ). Step 2 For G i = (G i 1,..., G i m), if there does not exist any δ m (F j + G i j + G i j) \ (F m T j i ) such that B(δ) = (B 1 (δ),..., B m (δ)) and J(δ) satisfy (2.6) and B j (δ) for some j = 1,..., m, then stop and return G i. 5

6 Step 3 Otherwise set G i+1 j = G i j \ B j (δ) for all j = 1,..., m, and i = i + 1, and go back to Step 2. We call Algorithm 3.1 EEM in this paper. In this section, we show that EEM always returns the smallest set (G 1,..., G m) in a set family. To this end, we give some notation and definitions, and use some results in Appendix B. For G = (G 1,..., G m ), H = (H 1,..., H m ) X := N n r 1 N n r m, we define G H := (G 1 H 1,..., G m H m ), G \ H := (G 1 \ H 1,..., G m \ H m ) and G H if G j H j for all j = 1,..., m. Definition 3.2 We define a function P : 2 X 2 X {true, false} as follows: true if all B j are empty or there exists δ m (F j + G j + G j ) \ (F m T j) P (G, B) = such that B = B(δ) and, B and J(δ) satisfy (2.6), false otherwise for all G = (G 1,..., G m ), B := (B 1,..., B m ) X. Moreover, let G 0 := (N n r 1,..., N n r m ). We define a set family Γ(G 0, P ) 2 X as follows: G Γ(G 0, P ) if and only if G = G 0 or there exists G Γ(G 0, P ) such that G G and P (G, G \ G) = true. The following theorem guarantees that EEM always returns the smallest set (G 1,..., G m) Γ(G 0, P ). Theorem 3.3 Let P and G(G 0, P ) be as in Definition 3.2. Assume that f has an SOS representation with f 1,..., f m and G 0. Then, Γ(G 0, P ) has the smallest set G in the sense that G G for all G Γ(G 0, P ). In addition, EEM described in Algorithm 3.1 always returns the smallest set G in the set family Γ(G 0, P ). For this proof, we use some results in Appendix B. We postpone the proof till Appendix C. We give an example to see a behavior of EEM. Example 3.4 (Continuation of Example 2.5) Let f = x, f 1 = 1, f 2 = x and f 3 = x 2 1. Clearly, f is nonnegative over the set D = {x R f 1, f 2, f 3 0} = [1, + ). Let r = 2. Then we have r 1 = 2, r 2 = r 3 = 1. We consider the following SOS problem: { Find σ j Σ rj for j = 1, 2, 3, (3.2) subject to f(x) = σ 1 (x)f 1 (x) + σ 2 (x)f 2 (x) + σ 3 (x)f 3 (x) ( x R). The initial G 0 = (N 2, N 1, N 1 ). From Example 2.5, we have already known δ 0 = 4, G 1 1 = G 0 1 \ B1(δ 0 0 ) = {0, 1} and G 1 = ({0, 1}, {0, 1}, {0}). Table 1 shows δ, J(δ), B j (δ) and T j in Example 3.4 in the ith iteration of EEM for the identity of (3.2). For G 1 Γ(G 0, P ), we choose δ 1 = 3 3 (F j + G 1 j + G 1 j) \ (F 3 T j 1 ) = {0, 3}. Then we obtain J(δ 1 ) and Bj 1 (δ 1 ) in the third row of Table 1 and G 2 = ({0, 1}, {0}, {0}). For G 2 Γ(G 0, P ), we choose δ 2 = 2 3 (F j + G 2 j + G 2 j) \ (F 3 T j 2 ) = {0, 2}. Then we obtain J(δ 2 ) and Bj 2 (δ 2 ) in the forth row of Table 1 and G 3 = ({0}, {0}, ). This implies that f 3 is redundant for all SOS representations of f with f 1, f 2, f 3 and G 3. For G 3 Γ(G 0, P ), we choose δ 3 = 0 3 (F j + G 3 j + G 3 j) \ (F 3 T j 3 ) = {0}. Then we obtain J(δ 3 ) and Bj 3 (δ 3 ) in the fifth row of Table 1 and G 4 = (, {0}, ). This implies that f 1 is redundant for all SOS representations of f with f 1, f 2, f 3 and G 4. 6

7 For G 4 Γ(G 0, P ), because the set 3 (F j +G 4 j +G 4 j)\(f 3 T j 4 ) is empty, EEM stops and returns G = (, {0}, ). Then by using G, from SOS problem (3.2), we obtain the following Linear Programming (LP): { Find λ 2 0 subjecto to λ 2 = 1. Because this LP has the solution λ 2 = 1, SOS problem (3.2) has a solution (σ 1, σ 2, σ 3 ) = (0, 1, 0). This implies that f = 0 f f f 3. Table 1: δ, J(δ), B j (δ) and T j in Example 3.4 δ i J(δ i ) B1(δ i i ) B2(δ i i ) B3(δ i i ) T1 i T2 i T3 i i = 0 4 {1, 3} {2} {1} {1, 2, 3} {2} {1, 3} i = 1 3 {2} {1} {1} {2} i = 2 2 {1, 3} {1} {0} {1} i = 3 0 {1} {0} i = 4 4. Numerical results for some POPs In this section, we present the numerical performance of EEM for Lasserre s and sparse SDP relaxations for Polynomial Optimization Problems (POPs) in [6] and randomly generated POPs with a sparse structure. To this end, we explain how to apply EEM to SDP relaxations for POPs. For given polynomials f, f 1,..., f m, POP is formulated as follows: inf x R n {f(x) f j(x) 0 (j = 1,..., m)} (4.1) We can reformulate POP (4.1) into the following problem: sup {ρ f(x) ρ 0 ( x D)}. (4.2) ρ R Here D is the feasible region of POP (4.1). We choose an integer r with r r. For (4.2) and r, we consider the following SOS problem: { } ρ m r := sup ρ ρ R,σ j Σ rj f(x) ρ = σ 0 + σ j (x)f j (x) ( x R n ) (4.3) where we define r j = r deg(f j )/2 for all j = 1,..., m and r 0 = r. If we find a feasible solution (ρ, σ 0, σ 1,..., σ m ) of (4.3), then ρ is a lower bound of optimal value of (4.1), clearly. It follows that ρ r+1 ρ r for all r r because we have Σ k Σ k+1 for all k N. We apply EEM to the identity in (4.3). Then, we can regard f(x) ρ in (4.3) as a polynomial with variable x. It should be noted that the support F of f(x) ρ always contains 0 because ρ is regarded as a constant term in the identity in (4.3). Moreover, let 7

8 f 0 (x) = 1 for all x R n. We replace σ 0 (x) by σ 0 (x)f 0 (x) in the identity in (4.3). Then we can apply EEM directly, so that we obtain finite sets G 0, G 1,..., G m. We can construct an SDP problem by using G 0, G 1,..., G m and applying a similar argument described in subsection 2.1. It was proposed in [20] to construct a smaller SDP relaxation problem than Lasserre s SDP relaxation when a given POP has a special sparse structure called correlative sparsity. We call their method the WKKM sparse SDP relaxation for POPs in this paper. In the numerical experiments, we have tested EEM applied to both Lasserre s and the WKKM sparse SDP relaxations. We use a computer with Intel (R) Xeon (R) 2.40 GHz cpus and 24GB memory, and Matlab R2009b and SeDuMi 1.21 with the default parameters to solve the resulting SDP relaxation problems. In particular, the default tolerance for stopping criterion of SeDuMi is 1.0e-9. We use SparsePOP [21] to make SDP relaxation problems. To see the quality of the approximate solution obtained by SeDuMi, we check DIMACS errors. If the six errors are sufficiently small, then the solution is regarded as an optimal solution. See [10] for the definitions. To check whether the optimal value of an SDP relaxation problem is the exact optimal value of a given POP or not, we use the following two criteria ɛ obj and ɛ feas : Let ˆx be a candidate of an optimal solution of the POP obtained by Lasserre s or the WKKM sparse SDP relaxation. See [20] for the way to obtain ˆx. We define: ɛ obj := the optimal value of the SDP relaxation f(ˆx), max{1, f(ˆx)} ɛ feas := min{f k (ˆx) (k = 1,..., m)}. If ɛ feas = 0, then ˆx is feasible for the POP. In addition, if ɛ obj = 0, then ˆx is an optimal solution of the POP and f(ˆx) is the optimal value of the POP. Some POPs in [6] are so badly scaled that the resulting SDP relaxation problems suffer severe numerical difficulty. We may obtain inaccurate values and solutions for such POPs. To avoid this difficulty, we apply a linear transformation to the variables in POP with finite lower and upper bounds on variables x i (i = 1,..., n). See [24] for the effect of such transformations. Although EMSSOSP is designed for an unconstrained POP, we can apply it to POP (4.1) in such a way that it removes unnecessary monomials in σ 0 in (4.3). It is presented in subsection 6.3 of [20] that such application of EMSSOSP is effective for a large-scale POP. In this section, we also compare EEM with EMSSOSP. Table 2 shows notation used in the description of the numerical results in subsection 4.1 and Numerical results for GLOBAL Library In this numerical experiment, we solve some POPs in GLOBAL Library [6]. This library contains POPs which have polynomial equalities h(x) = 0. In this case, we divide h(x) = 0 into two polynomial inequalities h(x) 0 and h(x) 0 and replace them by their polynomial inequalities in the original POP. We remark that in our tables, a POP whose initial is B is obtained by adding some lower and upper bounds. In addition, we do not 8

9 Method sizea nnza #LP #SDP a.sdp m.sdp SDPobj POPobj etime n.e. p.v. iter. Table 2: Notation a method for reducing the size of the SDP relaxation problems. Orig. means that we do not apply EMSSOSP and EEM to POPs. EMSSOSP and EEM mean that we apply EMSSOSP and EEM to POPs, respectively. the size of coefficient matrix A in the SeDuMi input format the number of nonzero elements in coefficient matrix A in the SeDuMi input format the number of linear constraints in the SeDuMi input format the number of positive semidefinite constraints in the SeDuMi input format the average of the sizes of positive semidefinite constraints in the Se- DuMi input format the maximum of the sizes of positive semidefinite constraints in the SeDuMi input format the objective value obtained by SeDuMi for the resulting SDP relaxation problem the value of f at a solution ˆx retrieved by SparesPOP cpu time consumed by SeDuMi or SDPA-GMP in seconds n.e. = 1 if SeDuMi cannot find an accurate solution due to numerical difficulty. Otherwise, n.e. = 0 phase value returned by SDPA-GMP. If it is pdopt, then SDPA- GMP terminates normally. the number of iterations in SeDuMi 9

10 apply the WKKM sparse SDP relaxation to POPs Bex3 1 4, st e01, st e09 and st e34 because their POPs do not have the sparse structure. Table 3: Numerical results of Lasserre s SDP relaxation problems. Problem r Method SDPobj POPobj ɛ obj ɛ feas ɛ feas etime Bex Orig e e e e e Bex EMSSOSP e e e e e Bex EEM e e e e e Bex Orig e e e e e Bex EMSSOSP e e e e e Bex EEM e e e e e Bex5 2 2 case1 2 Orig e e e e e Bex5 2 2 case1 2 EMSSOSP e e e e e Bex5 2 2 case1 2 EEM e e e e e Bex5 2 2 case2 2 Orig e e e e e Bex5 2 2 case2 2 EMSSOSP e e e e e Bex5 2 2 case2 2 EEM e e e e e Bex5 2 2 case3 2 Orig e e e e e Bex5 2 2 case3 2 EMSSOSP e e e e e Bex5 2 2 case3 2 EEM e e e e e Bst e07 2 Orig e e e e e Bst e07 2 EMSSOSP e e e e e Bst e07 2 EEM e e e e e Bst e33 2 Orig e e e e e Bst e33 2 EMSSOSP e e e e e Bst e33 2 EEM e e e e e st e01 3 Orig e e e e e st e01 3 EMSSOSP e e e e e st e01 3 EEM e e e e e st e09 3 Orig e e e e e st e09 3 EMSSOSP e e e e e st e09 3 EEM e e e e e st e34 2 Orig e e e e e st e34 2 EMSSOSP e e e e e st e34 2 EEM e e e e e

11 Table 4: Numerical errors, iterations, DIMACS errors and sizes of Lasserre s SDP relaxation problems. We omit err2 and err3 from this table because they are zero for all POPs and methods. Problem r Method n.e. iter. err1 err4 err5 err6 sizea nnza #LP #SDP a.sdp m.sdp Bex Orig e e e e Bex EMSSOSP e e e e Bex EEM e e e e Bex Orig e e e e Bex EMSSOSP e e e e Bex EEM e e e e Bex5 2 2 case1 2 Orig e e e e Bex5 2 2 case1 2 EMSSOSP e e e e Bex5 2 2 case1 2 EEM e e e e Bex5 2 2 case2 2 Orig e e e e Bex5 2 2 case2 2 EMSSOSP e e e e Bex5 2 2 case2 2 EEM e e e e Bex5 2 2 case3 2 Orig e e e e Bex5 2 2 case3 2 EMSSOSP e e e e Bex5 2 2 case3 2 EEM e e e e Bst e07 2 Orig e e e e Bst e07 2 EMSSOSP e e e e Bst e07 2 EEM e e e e Bst e33 2 Orig e e e e Bst e33 2 EMSSOSP e e e e Bst e33 2 EEM e e e e st e01 3 Orig e e e e st e01 3 EMSSOSP e e e e st e01 3 EEM e e e e st e09 3 Orig e e e e st e09 3 EMSSOSP e e e e st e09 3 EEM e e e e st e34 2 Orig e e e e st e34 2 EMSSOSP e e e e st e34 2 EEM e e e e

12 Table 5: Numerical results of the WKKM sparse SDP relaxation problems. Problem r Method SDPobj POPobj ɛ obj ɛ feas ɛ feas etime Bex Orig e e e e e Bex EMSSOSP e e e e e Bex EEM e e e e e Bex5 2 2 case1 2 Orig e e e e e Bex5 2 2 case1 2 EMSSOSP e e e e e Bex5 2 2 case1 2 EEM e e e e e Bex5 2 2 case1 3 Orig e e e e e Bex5 2 2 case1 3 EMSSOSP e e e e e Bex5 2 2 case1 3 EEM e e e e e Bex5 2 2 case2 2 Orig e e e e e Bex5 2 2 case2 2 EMSSOSP e e e e e Bex5 2 2 case2 2 EEM e e e e e Bex5 2 2 case2 3 Orig e e e e e Bex5 2 2 case2 3 EMSSOSP e e e e e Bex5 2 2 case2 3 EEM e e e e e Bex5 2 2 case3 2 Orig e e e e e Bex5 2 2 case3 2 EMSSOSP e e e e e Bex5 2 2 case3 2 EEM e e e e e Bex5 2 2 case3 3 Orig e e e e e Bex5 2 2 case3 3 EMSSOSP e e e e e Bex5 2 2 case3 3 EEM e e e e e Bst e07 2 Orig e e e e e Bst e07 2 EMSSOSP e e e e e Bst e07 2 EEM e e e e e Bst e33 2 Orig e e e e e Bst e33 2 EMSSOSP e e e e e Bst e33 2 EEM e e e e e

13 Table 6: Numerical errors, iterations, DIMACS errors and sizes of the WKKM sparse SDP relaxation problems. We omit err2 and err3 from this table because they are zero for all POPs and methods. Problem r Method n.e. iter. err1 err4 err5 err6 sizea nnza #LP #SDP a.sdp m.sdp Bex Orig e e e e Bex EMSSOSP e e e e Bex EEM e e e e Bex5 2 2 case1 2 Orig e e e e Bex5 2 2 case1 2 EMSSOSP e e e e Bex5 2 2 case1 2 EEM e e e e Bex5 2 2 case1 3 Orig e e e e Bex5 2 2 case1 3 EMSSOSP e e e e Bex5 2 2 case1 3 EEM e e e e Bex5 2 2 case2 2 Orig e e e e Bex5 2 2 case2 2 EMSSOSP e e e e Bex5 2 2 case2 2 EEM e e e e Bex5 2 2 case2 3 Orig e e e e Bex5 2 2 case2 3 EMSSOSP e e e e Bex5 2 2 case2 3 EEM e e e e Bex5 2 2 case3 2 Orig e e e e Bex5 2 2 case3 2 EMSSOSP e e e e Bex5 2 2 case3 2 EEM e e e e Bex5 2 2 case3 3 Orig e e e e Bex5 2 2 case3 3 EMSSOSP e e e e Bex5 2 2 case3 3 EEM e e e e Bst e07 2 Orig e e e e Bst e07 2 EMSSOSP e e e e Bst e07 2 EEM e e e e Bst e33 2 Orig e e e e Bst e33 2 EMSSOSP e e e e Bst e33 2 EEM e e e e

14 Tables 3 and 4 show the numerical results for Lasserre s SDP relaxation [9] for some POPs in [6]. Tables 5 and 6 show the numerical results for the WKKM sparse SDP relaxation [20] for some POPs in [6]. We observe the following. From Tables 4 and 6, the sizes of SDP relaxation problems obtained by EEM are the smallest of the three for all POPs. As a result, EEM spends the least cpu time to solve the resulting SDP problems except for Bst 07 on Table 3 and Bex case2 on Table 5. Table 4 tells us that EEM needs one more iteration than that by EMSSOSP for Bst 07. Looking at the behavior of SeDuMi in this case carefully, we noticed that SeDuMi consumes much more CPU time in the last iteration for computing the search direction than in the other iterations. Also in the case of Bex case2, EEM needs three more iterations than that by EMSSOSP. Exact reasons of them should be investigated in further research. For all POPs except for Bex5 2 2 case1, 2, 3, the optimal values of the SDP relaxation problems are the same. For the WKKM sparse SDP relaxation of Bex5 2 2 case1, 2, 3, all three methods cannot obtain accurate solutions; their DIMACS errors are not sufficiently small. Consequently, these computed optimal values are considered to be inaccurate. From Tables 4 and 6, for almost all POPs, DIMACS errors for SDP relaxation problems obtained by EEM are smaller than the other methods. This means that SeDuMi returns more accurate solutions for the resulting SDP relaxation problems by EEM. We cannot obtain optimal values and solutions for some POPs, e.g., Bex5 2 2 case1, 2, 3. In contrast, the optimal values and solutions for them are obtained in [20]. At a glance, it seems that the numerical result for some POPs may be worse than [20]. The reason is as follows: In [20], the authors add some valid polynomial inequalities to POPs and apply Lasserre s or the WKKM sparse SDP relaxation, so that they obtain tighter lower bounds or the exact values. See Section 5.5 in [20] for the details. In the experiments of this section, however, we do not add such valid polynomial inequalities in order to observe the efficiency of EMSSOSP and EEM Numerical results for randomly generated POP with a special structure Let C i = {i, i + 1, i + 2} for all i = 1,..., n 2. x Ci and y Ci denote the subvectors (x i, x i+1, x i+2 ) and (y i, y i+1, y i+2 ) of x, y R n, respectively. We consider the following POP: n 2 ( ) inf x,y R n (x Ci, y Ci ) T xci P i y Ci i=1 (4.4) subject to x T C i Q i y Ci + c T i x Ci + d T i y Ci + γ i 0 (i = 1,..., n 2), 0 x i, y i 1 (i = 1,..., n), where c i, d i R 3, Q i R 3 3, and P i S 6 6 is a symmetric positive semidefinite matrix. This POP has 2n variables and 5n 2 polynomial inequalities. In this subsection, we generate POP (4.4) randomly and apply the WKKM sparse SDP relaxation with relaxation order r = 2, 3. The SDP relaxation problems obtained by Lasserre s SDP relaxation are too large-scale to be handled for these problems. 14

15 Tables 7 and 8 show the numerical results of the WKKM sparse SDP relaxation with relaxation order r = 2. To obtain more accurate values and solutions, we use SDPA-GMP [5]. Tables 11 and 12 show the numerical result by SDPA-GMP with tolerance ɛ = 1.0e-15 and precision 256. With this precision, SDPA-GMP calculate floating point numbers with approximately 77 significant digits. Tables 9 and 10 show the numerical results of the WKKM sparse SDP relaxation with relaxation order r = 3. Tables 13 and 14 show the numerical result by SDPA-GMP with the same tolerance and precision as above. In this case, we solve only 2n = 24 and 44 because otherwise the resulting SDP problems become too large-scale to be solved by SDPA-GMP. 15

16 Table 7: Numerical results of the WKKM sparse SDP relaxation problems with relaxation order r = 2 for POP (4.4). etime 2n Method SDPobj POPobj ɛ obj ɛ feas ɛ feas 24 Orig e e e e e EMSSOSP e e e e e EEM e e e e e Orig e e e e e EMSSOSP e e e e e EEM e e e e e Orig e e e e e EMSSOSP e e e e e EEM e e e e e Orig e e e e e EMSSOSP e e e e e EEM e e e e e Orig e e e e e EMSSOSP e e e e e EEM e e e e e Table 8: Numerical errors, iterations, DIMACS errors and sizes of the WKKM sparse SDP relaxation problems with relaxation order r = 2 for POP (4.4). We omit err2 and err3 from this table because they are zero for all POPs and methods. 2n Method n.e. iter. err1 err4 err5 err6 sizea nnza #LP #SDP a.sdp m.sdp 24 Orig e e e e EMSSOSP e e e e EEM e e e e Orig e e e e EMSSOSP e e e e EEM e e e e Orig e e e e EMSSOSP e e e e EEM e e e e Orig e e e e EMSSOSP e e e e EEM e e e e Orig e e e e EMSSOSP e e e e EEM e e e e

17 Table 9: Numerical results of the WKKM sparse SDP relaxation problems with relaxation order r = 3 for POP (4.4). etime 2n Method SDPobj POPobj ɛ obj ɛ feas ɛ feas 24 Orig e e e e e EMSSOSP e e e e e EEM e e e e e Orig e e e e e EMSSOSP e e e e e EEM e e e e e Orig e e e e e EMSSOSP e e e e e EEM e e e e e Orig e e e e e EMSSOSP e e e e e EEM e e e e e Orig e e e e e EMSSOSP e e e e e EEM e e e e e Table 10: Numerical errors, iterations, DIMACS errors and sizes of the sparse SDP relaxation problems with relaxation order r = 3 for POP (4.4). We omit err2 and err3 from this table because they are zero for all POPs and methods. 2n Method n.e. iter. err1 err4 err5 err6 sizea nnza #LP #SDP a.sdp m.sdp 24 Orig e e e e EMSSOSP e e e e EEM e e e e Orig e e e e EMSSOSP e e e e EEM e e e e Orig e e e e EMSSOSP e e e e EEM e e e e Orig e e e e EMSSOSP e e e e EEM e e e e Orig e e e e EMSSOSP e e e e EEM e e e e

18 Table 11: Numerical Results by SDPA-GMP with ɛ=1.0e-15 for the WKKM sparse SDP relaxation problems with relaxation order r = 2 in POP (4.4). 2n Method p.v. iter. etime SDPobj by SDPA-GMP SDPobj by SeDuMi 24 Orig. pdopt e e EMSSOSP pdopt e e EEM pdopt e e Orig. pdopt e e EMSSOSP pdopt e e EEM pdopt e e Orig. pdopt e e EMSSOSP pdopt e e EEM pdopt e e Orig. pdopt e e EMSSOSP pdopt e e EEM pdopt e e Orig. pdopt e e EMSSOSP pdopt e e EEM pdopt e e-01 Table 12: DIMACS errors by SDPA-GMP with ɛ=1.0e-15 for the sparse SDP relaxation problems with relaxation order r = 2 in POP (4.4). We omit err2 and err4 from this table because they are zero for all POPs and methods. 2n Method err1 err3 err5 err6 24 Orig e e e e EMSSOSP 2.013e e e e EEM 1.682e e e e Orig e e e e EMSSOSP 1.549e e e e EEM 2.012e e e e Orig e e e e EMSSOSP 7.600e e e e EEM 2.402e e e e Orig e e e e EMSSOSP 1.859e e e e EEM 9.632e e e e Orig e e e e EMSSOSP 8.722e e e e EEM 1.798e e e e-16 We observe the following. The sizes of the resulting SDP relaxation problems by EEM is again the smallest in the three methods. In particular, when we apply EEM and the WKKM sparse SDP relaxation with relaxation order r = 2, positive semidefinite constraints corresponding to the quadratic constraints in (4.2) are replaced by linear constraints in SDP relaxation problems. EEM removes all monomials except for the constant term in σ j Σ 1 because those monomials are redundant for all SOS representations of f. Then σ j Σ 1 for all j = 1,..., n can be replaced by σ j Σ 0 for all j = 1,..., n. This is equivalent to σ j 0 for all j = 1,..., n. Therefore, we obtain n linear constraints in the resulting SDP relaxation problems. 18

19 Table 13: Numerical Results by SDPA-GMP with ɛ=1.0e-15 for the WKKM sparse SDP relaxation problems with relaxation order r = 3 in POP (4.4). 2n Method p.v. iter. etime SDPobj by SDPA-GMP SDPobj by SeDuMi 24 Orig. pdopt e e EMSSOSP pdopt e e EEM pdopt e e Orig. pdopt e e EMSSOSP pdopt e e EEM pdopt e e-01 Table 14: DIMACS errors by SDPA-GMP with ɛ=1.0e-15 for the sparse SDP relaxation problems with relaxation order r = 3 in POP (4.4). We omit err2 and err4 from this table because they are zero for all POPs and methods. 2n Method err1 err3 err5 err6 24 Orig e e e e EMSSOSP 3.433e e e e EEM 1.229e e e e Orig e e e e EMSSOSP 5.010e e e e EEM 4.523e e e e-16 From Tables 11 and 12, SDPA-GMP with precision 256 can solve all SDP relaxation problems accurately. In particular, SDPA-GMP solves SDP relaxation problems obtained by EEM more than 8 and 50 times faster than EMSSOSP and Orig., respvectively. From Tables 7, 8 and 11, SeDuMi returns the optimal values of SDP relaxation problems obtained by EEM almost exactly as accurately as SDPA-GMP and more than 15 times faster than SDPA-GMP, while SeDuMi terminates before we obtain accurate solutions of SDP relaxation problems obtained by the other methods. From Table 10, in SDP relaxation problems with relaxation order r = 3, DIMACS errors for SDP relaxation problems by EEM are the smallest in all methods. Moreover, from Tables 13 and 14, the optimal values of SDP relaxation problems by EEM coincide the optimal values found by SDPA-GMP for 2n = 24 and 44. However, SeDuMi cannot obtain accurate solutions because these values are larger than the tolerance 1.0e-9 of SeDuMi. 5. An application of EEM to specific POPs As we have seen in Section 3, we have a flexibility in choosing δ although EEM always returns the smallest set G Γ(G 0, P ). We focus on this flexibility and we prove the following two facts in this section: (i) if POP (4.1) satisfies a specific assumption, each optimal value of the SDP relaxation problem with relaxation order r > r is equal to that of the relaxation order r. To prove this fact, we choose δ to be the largest element in m j=0 (F j + G j + G j ) \ (F m j=0 T j) with the graded lexicographic order 1. (ii) We give an 1 We define the graded lexicographic order α β for α, β N n as follows: α > β or, α = β and α i > β i for the smallest index i with α i β i. 19

20 extension of Proposition 4 in [7], where we choose δ to be the smallest element. First of all, we state (i) exactly. Let γ j be the largest element with the graded lexicographic order in F j. For γ N n, we define I(γ) = {k {0, 1,..., m} γ k γ (mod 2)}. We impose the following assumption on polynomials in POP (4.1). Assumption 5.1 For any fixed j {0, 1,..., m}, for each k I(γ j ), the largest monomial x γ k in fk has the same sign as (f j ) γj The following theorem guarantees that we do not need to increase a relaxation order for POP which satisfies Assumption 5.1 in order to obtain a tighter lower bound. Theorem 5.2 Assume r > r. Then under Assumption 5.1, the optimal value of the SDP relaxation problem with relaxation order r is the same as that of relaxation order r. We postpone a proof of Theorem 5.2 till Appendix D. We give two examples for Theorem 5.2. Example 5.3 Let f = x, f 1 = 1, f 2 = x, f 3 = x 2 1. We consider the following POP: inf{x x 0, x 2 1 0}. (5.1) Then clearly, we have γ 1 = 0, γ 2 = 1, γ 3 = 2 and I(γ 1 ) = I(γ 3 ) = {1, 3}, I(γ 2 ) = {2}, and this POP satisfies Assumption 5.1. Therefore, it follows from Theorem 5.2 that the optimal value of each SDP relaxation problem with relaxation order r 1 is equal to the optimal value of the SDP relaxation problem with relaxation order 1. We give the SOS problem with relaxation order 1: sup {ρ x ρ = σ 1 (x) + xσ 2 + (x 2 1)σ 3 ( x R)}. ρ R,σ 1 Σ 1,σ 2,σ 3 0 Furthermore, we can apply EEM to the identity to reduce the size of the SOS problem above. Then the obtained SOS problem is equivalent to LP as follows: sup {ρ x ρ = σ 1 + xσ 2 ( x R)} = sup {ρ σ 1 = ρ, σ 2 = 1}. ρ R,σ 1,σ 2 0 ρ R,σ 1,σ 2 0 Clearly, the optimal value of this LP is 0, and thus the optimal value of the SDP relaxation problem with arbitrary relaxation order is 0. This POP is dealt with in [25] and it is shown by using positive semidefiniteness in SDP relaxation problems that the optimal values of all SDP relaxation problems are 0. In [22], it is shown that the approach is FRA and this fact is a motivation to show a relationship between EEM and FRA. We give the details in Section 6. Example 5.4 Let f = x, f 1 = 1, f 2 = 2 x, f 3 = x 2 1. Then clearly, we have the same γ j and I(γ j ) as in Example 5.3. This POP also satisfies Assumption 5.1. We solve SDP relaxation problem with relaxation order r = 1. Then we obtain the following SOS problem with relaxation order r = 1. sup {ρ x ρ = σ 1 (x) + (2 x)σ 2 + (x 2 1)σ 3 ( x R)}. ρ R,σ 1 Σ 1,σ 2,σ 3 0 Applying EEM to the identity, we obtain the following LP problem: sup {ρ x ρ = σ 1 + (2 x)σ 2 ( x R)} = sup {ρ ρ = σ 1 + 2σ 2, 1 = σ 2 } = 2. ρ,σ 1,σ 2 0 ρ,σ 1,σ 2 0 From this result, the optimal value of the SDP relaxation problem with arbitrary relaxation order is 2, which is equal to the optimal value of the POP. 20

### Solving polynomial least squares problems via semidefinite programming relaxations

Solving polynomial least squares problems via semidefinite programming relaxations Sunyoung Kim and Masakazu Kojima August 2007, revised in November, 2007 Abstract. A polynomial optimization problem whose

### An Introduction on SemiDefinite Program

An Introduction on SemiDefinite Program from the viewpoint of computation Hayato Waki Institute of Mathematics for Industry, Kyushu University 2015-10-08 Combinatorial Optimization at Work, Berlin, 2015

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### Vector Spaces II: Finite Dimensional Linear Algebra 1

John Nachbar September 2, 2014 Vector Spaces II: Finite Dimensional Linear Algebra 1 1 Definitions and Basic Theorems. For basic properties and notation for R N, see the notes Vector Spaces I. Definition

### Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.

Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

### An Overview Of Software For Convex Optimization. Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.

An Overview Of Software For Convex Optimization Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu In fact, the great watershed in optimization isn t between linearity

### LINEAR SYSTEMS. Consider the following example of a linear system:

LINEAR SYSTEMS Consider the following example of a linear system: Its unique solution is x +2x 2 +3x 3 = 5 x + x 3 = 3 3x + x 2 +3x 3 = 3 x =, x 2 =0, x 3 = 2 In general we want to solve n equations in

### 3 Does the Simplex Algorithm Work?

Does the Simplex Algorithm Work? In this section we carefully examine the simplex algorithm introduced in the previous chapter. Our goal is to either prove that it works, or to determine those circumstances

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### A Lagrangian-DNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems

A Lagrangian-DNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems Sunyoung Kim, Masakazu Kojima and Kim-Chuan Toh October 2013 Abstract. We propose

### FE670 Algorithmic Trading Strategies. Stevens Institute of Technology

FE670 Algorithmic Trading Strategies Lecture 6. Portfolio Optimization: Basic Theory and Practice Steve Yang Stevens Institute of Technology 10/03/2013 Outline 1 Mean-Variance Analysis: Overview 2 Classical

### Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for

### Conic optimization: examples and software

Conic optimization: examples and software Etienne de Klerk Tilburg University, The Netherlands Etienne de Klerk (Tilburg University) Conic optimization: examples and software 1 / 16 Outline Conic optimization

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### Duality of linear conic problems

Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least

### MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column

### 4.5 Linear Dependence and Linear Independence

4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then

### I. GROUPS: BASIC DEFINITIONS AND EXAMPLES

I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### (x + a) n = x n + a Z n [x]. Proof. If n is prime then the map

22. A quick primality test Prime numbers are one of the most basic objects in mathematics and one of the most basic questions is to decide which numbers are prime (a clearly related problem is to find

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### Quotient Rings and Field Extensions

Chapter 5 Quotient Rings and Field Extensions In this chapter we describe a method for producing field extension of a given field. If F is a field, then a field extension is a field K that contains F.

### PART I. THE REAL NUMBERS

PART I. THE REAL NUMBERS This material assumes that you are already familiar with the real number system and the representation of the real numbers as points on the real line. I.1. THE NATURAL NUMBERS

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### Ri and. i=1. S i N. and. R R i

The subset R of R n is a closed rectangle if there are n non-empty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an

### Linear Algebra I. Ronald van Luijk, 2012

Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.

### 4.6 Linear Programming duality

4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set.

MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. Vector space A vector space is a set V equipped with two operations, addition V V (x,y) x + y V and scalar

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### MATH 110 Spring 2015 Homework 6 Solutions

MATH 110 Spring 2015 Homework 6 Solutions Section 2.6 2.6.4 Let α denote the standard basis for V = R 3. Let α = {e 1, e 2, e 3 } denote the dual basis of α for V. We would first like to show that β =

### it is easy to see that α = a

21. Polynomial rings Let us now turn out attention to determining the prime elements of a polynomial ring, where the coefficient ring is a field. We already know that such a polynomial ring is a UF. Therefore

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### The Dirichlet Unit Theorem

Chapter 6 The Dirichlet Unit Theorem As usual, we will be working in the ring B of algebraic integers of a number field L. Two factorizations of an element of B are regarded as essentially the same if

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### 11 Multivariate Polynomials

CS 487: Intro. to Symbolic Computation Winter 2009: M. Giesbrecht Script 11 Page 1 (These lecture notes were prepared and presented by Dan Roche.) 11 Multivariate Polynomials References: MC: Section 16.6

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### (Quasi-)Newton methods

(Quasi-)Newton methods 1 Introduction 1.1 Newton method Newton method is a method to find the zeros of a differentiable non-linear function g, x such that g(x) = 0, where g : R n R n. Given a starting

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### These axioms must hold for all vectors ū, v, and w in V and all scalars c and d.

DEFINITION: A vector space is a nonempty set V of objects, called vectors, on which are defined two operations, called addition and multiplication by scalars (real numbers), subject to the following axioms

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### 24. The Branch and Bound Method

24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### Appendix A. Appendix. A.1 Algebra. Fields and Rings

Appendix A Appendix A.1 Algebra Algebra is the foundation of algebraic geometry; here we collect some of the basic algebra on which we rely. We develop some algebraic background that is needed in the text.

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### Convex analysis and profit/cost/support functions

CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m

### Discrete Optimization

Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using

### THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### Integer Factorization using the Quadratic Sieve

Integer Factorization using the Quadratic Sieve Chad Seibert* Division of Science and Mathematics University of Minnesota, Morris Morris, MN 56567 seib0060@morris.umn.edu March 16, 2011 Abstract We give

### CHAPTER 9. Integer Programming

CHAPTER 9 Integer Programming An integer linear program (ILP) is, by definition, a linear program with the additional constraint that all variables take integer values: (9.1) max c T x s t Ax b and x integral

### 1. Prove that the empty set is a subset of every set.

1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since

### Linear Programming Notes V Problem Transformations

Linear Programming Notes V Problem Transformations 1 Introduction Any linear programming problem can be rewritten in either of two standard forms. In the first form, the objective is to maximize, the material

### Theta Functions. Lukas Lewark. Seminar on Modular Forms, 31. Januar 2007

Theta Functions Lukas Lewark Seminar on Modular Forms, 31. Januar 007 Abstract Theta functions are introduced, associated to lattices or quadratic forms. Their transformation property is proven and the

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### What is Linear Programming?

Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to

### SYSTEMS OF EQUATIONS

SYSTEMS OF EQUATIONS 1. Examples of systems of equations Here are some examples of systems of equations. Each system has a number of equations and a number (not necessarily the same) of variables for which

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

The Hadamard Product Elizabeth Million April 12, 2007 1 Introduction and Basic Results As inexperienced mathematicians we may have once thought that the natural definition for matrix multiplication would

### Sec 4.1 Vector Spaces and Subspaces

Sec 4. Vector Spaces and Subspaces Motivation Let S be the set of all solutions to the differential equation y + y =. Let T be the set of all 2 3 matrices with real entries. These two sets share many common

Quadratic Functions, Optimization, and Quadratic Forms Robert M. Freund February, 2004 2004 Massachusetts Institute of echnology. 1 2 1 Quadratic Optimization A quadratic optimization problem is an optimization

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### Lecture 5: Conic Optimization: Overview

EE 227A: Conve Optimization and Applications January 31, 2012 Lecture 5: Conic Optimization: Overview Lecturer: Laurent El Ghaoui Reading assignment: Chapter 4 of BV. Sections 3.1-3.6 of WTB. 5.1 Linear

### Polynomial Invariants

Polynomial Invariants Dylan Wilson October 9, 2014 (1) Today we will be interested in the following Question 1.1. What are all the possible polynomials in two variables f(x, y) such that f(x, y) = f(y,

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### Vectors. Philippe B. Laval. Spring 2012 KSU. Philippe B. Laval (KSU) Vectors Spring /

Vectors Philippe B Laval KSU Spring 2012 Philippe B Laval (KSU) Vectors Spring 2012 1 / 18 Introduction - Definition Many quantities we use in the sciences such as mass, volume, distance, can be expressed

### NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### Linear Codes. Chapter 3. 3.1 Basics

Chapter 3 Linear Codes In order to define codes that we can encode and decode efficiently, we add more structure to the codespace. We shall be mainly interested in linear codes. A linear code of length

### The determinant of a skew-symmetric matrix is a square. This can be seen in small cases by direct calculation: 0 a. 12 a. a 13 a 24 a 14 a 23 a 14

4 Symplectic groups In this and the next two sections, we begin the study of the groups preserving reflexive sesquilinear forms or quadratic forms. We begin with the symplectic groups, associated with

### MODULAR ARITHMETIC. a smallest member. It is equivalent to the Principle of Mathematical Induction.

MODULAR ARITHMETIC 1 Working With Integers The usual arithmetic operations of addition, subtraction and multiplication can be performed on integers, and the result is always another integer Division, on

### Row Echelon Form and Reduced Row Echelon Form

These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation

### Methods for Finding Bases

Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,

### Real Roots of Univariate Polynomials with Real Coefficients

Real Roots of Univariate Polynomials with Real Coefficients mostly written by Christina Hewitt March 22, 2012 1 Introduction Polynomial equations are used throughout mathematics. When solving polynomials

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### Math 312 Homework 1 Solutions

Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please

### SOLVING LINEAR SYSTEMS

SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### An Accelerated First-Order Method for Solving SOS Relaxations of Unconstrained Polynomial Optimization Problems

An Accelerated First-Order Method for Solving SOS Relaxations of Unconstrained Polynomial Optimization Problems Dimitris Bertsimas, Robert M. Freund, and Xu Andy Sun December 2011 Abstract Our interest

### FACTORING SPARSE POLYNOMIALS

FACTORING SPARSE POLYNOMIALS Theorem 1 (Schinzel): Let r be a positive integer, and fix non-zero integers a 0,..., a r. Let F (x 1,..., x r ) = a r x r + + a 1 x 1 + a 0. Then there exist finite sets S

### Introduction to Finite Fields (cont.)

Chapter 6 Introduction to Finite Fields (cont.) 6.1 Recall Theorem. Z m is a field m is a prime number. Theorem (Subfield Isomorphic to Z p ). Every finite field has the order of a power of a prime number

### Regression Using Support Vector Machines: Basic Foundations

Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Section 6.1 - Inner Products and Norms

Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,

### Algebra 2 Chapter 1 Vocabulary. identity - A statement that equates two equivalent expressions.

Chapter 1 Vocabulary identity - A statement that equates two equivalent expressions. verbal model- A word equation that represents a real-life problem. algebraic expression - An expression with variables.

### Convex Programming Tools for Disjunctive Programs

Convex Programming Tools for Disjunctive Programs João Soares, Departamento de Matemática, Universidade de Coimbra, Portugal Abstract A Disjunctive Program (DP) is a mathematical program whose feasible

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Introduction to Matrix Algebra

Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

### THE DIMENSION OF A VECTOR SPACE

THE DIMENSION OF A VECTOR SPACE KEITH CONRAD This handout is a supplementary discussion leading up to the definition of dimension and some of its basic properties. Let V be a vector space over a field

### Lecture 7: Finding Lyapunov Functions 1

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.243j (Fall 2003): DYNAMICS OF NONLINEAR SYSTEMS by A. Megretski Lecture 7: Finding Lyapunov Functions 1