THE POLYNOMIAL EIGENVALUE PROBLEM


 Giles Mason
 1 years ago
 Views:
Transcription
1 THE POLYNOMIAL EIGENVALUE PROBLEM A thesis submitted to the University of Manchester for the degree of Doctor of Philosophy in the Faculty of Engineering and Physical Sciences 2005 Michael Berhanu School of Mathematics
2 Contents Abstract 11 Declaration 12 Copyright 13 Statement 14 Acknowledgements 15 1 Introduction Applications of PEPs Notations General Notations Matrix Notation and Special Matrices Mathematical Background Linear Algebra Normed Linear Vector Spaces Scalar Product and Scalar Product Spaces Matrices, Vectors and their Norms Differential Calculus Special Matrix Subsets
3 1.5 (J, J)Orthogonal and (J, J)Unitary Matrices Matrix Operators Properties Condition Number and Backward Error The Polynomial Eigenvalue Problem Homogeneous PEPs Condition Numbers for Eigenvalues and Eigenvectors Introduction A Differential Calculus Approach Preliminaries Projective Spaces Condition Numbers Perturbation Analysis Link to the NonHomogeneous Form Particular Case: the GEP Hermitian Structured Condition Numbers Conclusion Backward Errors Introduction Normwise Backward Error Normwise Structured Backward Error for the Symmetric PEP Normwise Structured Backward Error for the Symmetric GEP Real Eigenpair Complex Eigenvalues Matrix Factorizations and their Sensitivity Introduction
4 4.2 Zeroing with (J 1, J 2 )Orthogonal Matrices Unified Rotations Householder Reflectors Error Analysis Zeroing Strategies Introduction to Matrix Factorization A General Method for Computing the Condition Number The HR Factorization Perturbation of the HR Factorization Numerical Experiments The Indefinite Polar Factorization Perturbation of the IPF The Polar Factorization Numerical Experiments The Hyperbolic Singular Value Decomposition Perturbation of the HSVD Numerical Experiments Sensitivity of Hyperbolic Eigendecompositions Perturbation Analysis of the Diagonalization by Hyperbolic Matrices Condition Number Theorems Numerical Solutions of PEPs Introduction QEPs with a Rank one Damping Matrix Preliminaries Real Eigenvalues with M > 0, K
5 5.2.3 General Case Solving PEPs Through Linearization Different Linearisations Companion Linearization Symmetric Linearization Influence of the Linearization Pseudocode Numerical Examples with condpolyeig Lack of Numerical Tools condpolyeig Numerical Examples An Overview of Algorithms for Symmetric GEPs The ErhlichAberth Method LR Algorithm HR Algorithm The HZ Algorithm Introduction Symmetric Diagonal Reduction Tridiagonal Diagonal Reduction HR or HZ Iterations Preliminaries Practical Implementation of One HZ Step Implementing the Bulge Chasing Pseudocodes Shifting Strategies Flops Count and Storage
6 6.8 Eigenvectors Iterative Refinement Newton s Method Implementation Numerical Experiments with HZ and Comparisons The HZ Algorithm Standard Numerical Experiment Symmetric GEPs and Iterative Refinement HZ on TridiagonalDiagonal Pairs Bessel Matrices Lui Matrices Clement Matrices Symmetric QEPs Wave Equation Simply Supported Beam Conclusion Summary Future Projects and Improvements Bibliography 211 6
7 List of Tables 4.1 Relative errors for c and s Perturbation bounds of the HR factorization Values of dg R (A) 2 A ɛ F and 2κ 2 (A ɛ ) A ɛ F as ɛ Perturbation bounds of the indefinite polar factorization Perturbation bounds of the IPF using bounds for the condition numbers c H and c S Perturbation bounds for the singular values from HSVD Perturbation bounds for the orthogonal and hyperbolic factors List of eigentools Eigenvalues of P (A θ, α, b) Condition number and backward error for λ = Condition number and backward error for λ = 1 + θ Average number of iterations for each shifting strategy Average number of iterations per eigenvalue for each shifting strategy Comparison of the number of floating point operations in the HZ and QZ algorithms Numerical results for randomly generated tridiagonaldiagonal pairs Numerical results with randomly generated symmetric pairs
8 7.3 Largest eigenvalue condition number for test matrices 1 10 with n = 100 and n = Largest relative error of the computed eigenvalues for test matrices 1 10 with n = Largest relative error of the computed eigenvalues for test matrices 1 10 with n = Number of HZ iterations and ErhlichAberth iterations, n = Normwise backward errors for test matrices 110 with n = Largest relative error of the computed eigenvalues of the modified Clement matrices with n = 50 and n = Largest normwise QEP backward error
9 List of Figures 1.1 A 2 degree of freedom massspring damped system Condition number and perturbation bounds of the IPF of Hilbert matrices with log 10 ( dg S (A) 2 ) ( ), log 10 ( dg H (A) 2 ) ( ), log 10 (c S ) ( ) and log 10 (c H ) (+) Comparison between the condition number and its bounds with log 10 ( dg Q (A) 2 ) ( ), log 10 ( dg H (A) 2 ) ( ), log 10 (c Q,1 ) (+), log 10 (c H,1 ) ( ), log 10 (c Q,2 ) ( ) and log 10 (c H,2 ) ( ) Spectrum computed with the companion linearization Spectrum computed with the symmetric linearization Normwise unstructured backward errors before ( ) and after (+) iterative refinement The eigenvalues of tests 1 to 4 in the complex plan for n = The eigenvalues of tests 5 to 8 in the complex plan for n = The eigenvalues of tests 9 and 10 in the complex plan for n = Relative errors of the eigenvalues of the Bessel matrix with n = 18, a = 8.5 computed with HZ ( ), EA ( ) and with QR (+) Eigenvalues of Bessel matrices computed in extended precision ( ) and with HZ ( ), EA ( ) and with QR (+)
10 7.7 The eigenvalues of Liu s matrix 5 computed with HZ ( ), EA ( ) and QR (+) The eigenvalues of Liu s matrices 14 and 28 computed with HZ ( ) using shifting strategy mix 1, EA ( ) and QR (+) The eigenvalues of Liu s matrices 14 and 28 computed with HZ ( ) using shifting strategy mix 2 and random shifts, EA ( ) and QR (+) Eigenvalue condition numbers for the Clement matrix for n = 50 and Eigenvalues of the Clement matrix with n = 200 and n = 300 computed with MATLAB s function eig The eigenvalues of the modified Clement matrices for n = The eigenvalues of the modified Clement matrices for n = Eigenvalues of the wave equation for n = Backward errors of the approximate eigenpairs (with λ = α/β) of the wave problem computed with HZ ( ) and QZ (+) with n = Eigenvalues of the beam problem with n=200 computed with HZ ( ) and QZ (+) Backward errors of the approximate eigenpairs (with λ = α/β) of the beam problem computed with HZ ( ) and QZ (+) with n=
11 Abstract In this thesis, we consider polynomial eigenvalue problems. We extend results on eigenvalue and eigenvector condition numbers of matrix polynomials to condition numbers with perturbations measured with a weighted Frobenius norm. We derive an explicit expression for the backward error of an approximate eigenpair of a matrix polynomial written in homogeneous form. We consider structured eigenvalue condition numbers for which perturbations have a certain structure such as symmetry, Hermitian or sparsity. We also obtain explicit and/or computable expressions for the structured backward error of an eigenpair. We present a robust implementation of the HZ (or HR) algorithm for symmetric generalized eigenvalue problems. This algorithm has the advantage of preserving pseudosymmetric tridiagonal forms. It has been criticized for its numerical instability. We propose an implementation of the HZ algorithm that allows stability in most cases and comparable results with other classical algorithms for ill conditioned problems. The HZ algorithm is based on the HR factorization, an extension of the QR factorization in which the H factor is hyperbolic. This yields us to the sensitivity analysis of hyperbolic factorizations. 11
12 Declaration No portion of the work referred to in this thesis has been submitted in support of an application for another degree or qualification of this or any other university or other institution of learning. 12
13 Copyright Copyright in text of this thesis rests with the Author. Copies (by any process) either in full, or of extracts, may be made only in accordance with instructions given by the Author and lodged in the John Rylands University Library of Manchester. Details may be obtained from the Librarian. This page must form part of any such copies made. Further copies (by any process) of copies made in accordance with such instructions may not be made without the permission (in writing) of the Author. The ownership of any intellectual property rights which may be described in this thesis is vested in the University of Manchester, subject to any prior agreement to the contrary, and may not be made available for use by third parties without the written permission of the University, which will prescribe the terms and conditions of any such agreement. Further information on the conditions under which disclosures and exploitation may take place is available from the Head of the Department of Mathematics. 13
14 Statement The material in Chapter 4 is based on the technical report Perturbation Bounds for Hyperbolic Matrix Factorizations, Numerical Analysis Report 469, Manchester Centre for Computational Mathematics, June This work has been submitted for publication in SIAM J. Matrix Anal. Appl. The material in Chapter 6 is based on the technical report A Robust Implementation of the HZ Algorithm (with Françoise Tisseur), Numerical Analysis Report, Manchester Centre for Computational Mathematics. In Preparation. 14
15 Acknowledgements I am extremely grateful to my supervisor Françoise Tisseur for her help, guidance and for sharing with me her expertise. I would like to express my gratitude to Nick Higham for his many helpful suggestions and constructive remarks. Many thanks to my fellow students and friends Matthew Smith, Harikrishna Patel, Craig Lucas, Gareth Hargreaves, Anna Mills and Philip Davis for the enjoyable 3... years in Manchester. ɛυχαριστ ω πoλυ Maria Pampaka, Maria Mastorikou, Panagiotis Kallinikos ( Dr, elare ), mucha gracias to the spanish crew, Big Hands,... Mariella Tsopela thank you for everything, φiλακια. Thanks to my father Berhanu H/W who gave me in my childhood the thirst of knowledge. I am extremely grateful to my sisters Bethlam (Koki), Deborah (Lili), Myriam (Poly). Thanks Lili for your patience and help. Finally, a lot of thanks goes to my mother, Fiorenza Vitali, for her encouragement and unconditional love. I dedicate this thesis to her. Merci beaucoup. 15
16 Chapter 1 Introduction We consider the matrix polynomial (or λmatrix) of degree m P (A, λ) = λ m A m + λ m 1 A m A 0, (1.1) where A k C n n, k = 0: m. The polynomial eigenvalue problem (PEP) is to find an eigenvalue λ and corresponding nonzero eigenvector x satisfying P (A, λ)x = 0. The case m = 1 corresponds to the generalized eigenvalue problem (GEP) Ax = λbx and if A 0 = I we have the standard eigenvalue problem (SEP) Ax = λx. (1.2) Another important case is the quadratic eigenvalue problem (QEP) with m = 2. The importance of PEPs lies in the diverse roles they play in the solution of problems in science and engineering. We briefly outline some examples. 16
17 d 4 k 4 d 5 k Applications of PEPs QEPs and more generally PEPs appear in a variety of problems in a wide range of applications. There are numerous examples where PEPs arise naturally. Some physical phenomena are modeled by a second order ordinary differential equation (ODE) with matrix coefficients M z + Dż + Kz = f(t), (1.3) z(0) = a, (1.4) ż(0) = b. (1.5) The solutions of the homogeneous equation are of the form e λt u, with u a constant vector. This leads to the QEP (λ 2 M + λd + K)u = 0. (1.6) PSfrag replacements k 1 k 2 m 1 m 2 k 3 d 1 k 6 d 2 k 7 d 6 d 7 d 3 Figure 1.1: A 2 degree of freedom massspring damped system. A well known example is the damped massspring system. In Figure 1.1, we consider the 2 degree of freedom massspring damped system. The dynamics of this system, under some assumptions, are governed by an ODE of the form (1.3) (1.5). In this case z = (x 1, y 1, x 2, y 2 ) denotes the coordinates of the masses m 1 and 17
18 m 2, M = diag(m 1, m 1, m 2, m 2 ) is the mass matrix, D = diag(d 1 +d 2, d 4 +d 6, d 2 + d 3, d 5 + d 7 ) is the damping matrix and K = diag(k 1 + k 2, k 4 + k 6, k 2 + k 3, k 5 + k 7 ) is the stiffness matrix with d i > 0, k i > 0 for 1 i 7. QEPs arise in structural mechanics, control theory, fluid mechanics and we refer to Tisseur and Meerbergen s survey [73] for more specific applications. Interesting practical examples of higher order PEPs are given in [52]. 1.2 Notations General Notations K denotes the field R or C. The colon notation: i = 1: n means the same as i = 1, 2,..., n. ᾱ denotes the conjugate of the complex number α. K m n denotes the set of m n matrices with coefficients in K. M n (K) m denotes the set of mtuples of n n matrices with coefficients in K. For x K n, x = (x k ) 1 k n = (x k ), x k denotes the kth component of x. e k denotes the vector with the kth component equal to 1 and all the other entries are zero. For A K m n, A = (α ij ) 1 i m, 1 j n = (α ij ), α ij denotes the (i, j) element of A. We often use the tilde notation to denote a perturbed quantity and the hat notation to denote a computed quantity. 18
19 1.2.2 Matrix Notation and Special Matrices Let A K m n, A = (α ij ). A is a square matrix if m = n. A T K n m is the transpose of A and it is defined by A T = (α ji ). A is symmetric if A T = A. A is Jsymmetric if JA is symmetric for some J R n m. A is skewsymmetric if A T = A. A K n m is the conjugate transpose of A and it is defined by A = (ᾱ ji ). A is Hermitian if A = A. A is skewhermitian if A = A. A is diagonal if α ij = 0 for i j. The identity matrix of order n, I n or simply I, is the diagonal matrix that has all its diagonal entries equal to 1. A permutation matrix is a matrix obtained from the identity matrix by row or column permutation. A K m n with m n is upper trapezoidal if α ij = 0 for i > j. A square matrix A is upper triangular if α ij = 0 for i > j and lower triangular if i < j. If all the diagonal elements of A are equal to 1 then A is called a unit upper or lower triangular. A is an upper Hessenberg matrix if α ij = 0 for i > j
20 A is a tridiagonal matrix if A and A T are upper Hessenberg matrices. For a square matrix A, A 1 denotes its inverse. It is the unique matrix such that A 1 A = A 1 = I. A is also said to be nonsingular when A 1 exits. Otherwise A is singular. For B = (b ij ) K m n the Schur product is defined by A B = (a ij b ij ). For B = (b ij ) K p q the Kronecker product is defined by A B = (a ij B). 1.3 Mathematical Background We recall in this Section some mathematical properties of norms, linear spaces and differentiable functions. A particular attention is given to the linear vector spaces K n and K m n. In the rest of this chapter, E denotes a linear vector space over K, K n or K m n Linear Algebra Let V = {v 1,..., v n } where v k E for 1 k n. The linear subspace generated by V is defined by { n } spanv = α k v k, α k K. k=1 A linear combination is a vector of the type n α k v k, k=1 where (α 1,..., α 2 ) K n. The vectors in V are said to be linearly independent if n α k v k = 0 α k = 0 for k = 1: n. k=1 20
21 The number of linearly independent vectors is the dimension of spanv in K and it is denoted by dim(v ) = dim K (V ). Let V 1 and V 2 be two linear subspaces of E. If V 1 V 2 = {0} and E = V 1 + V 2 then E is said to be the direct sum of V 1 and V 2 and the direct sum decomposition is denoted by E = V 1 V 2. Let A : E 1 E 2 be a linear map or a matrix. The range of A is the linear subspace defined by range(a) = {y E 2 : y = Ax, x E 1 } = A(E 1 ). The null space of A is the linear subspace defined by null(a) = {x E 1 : Ax = 0}. The rank of A is the dimension of range(a), rank(a) = dim(range(a)). With these notations, it follows that dim(e 1 ) = rank(a) + dim(null(a)). A K m n is of full rank if rank(a) = min(m, n). If rank(a) < min(m, n) then A is rank deficient Normed Linear Vector Spaces Definition 1.1 Let E be a linear vector space. A norm is a map : E R satisfying the following properties: 21
22 1. x 0 with equality if and only if x = 0, 2. (λ, x) K E, λx = λ x, 3. (x, y) E 2, x + y x + y. For x E, V x denotes an open neighborhood of x. The open ball of radius ɛ 0 centered at x is defined by B(x, ɛ) = {y E, y x ɛ}. In this thesis, only E = K n and E = K m n are the spaces considered. Thus, all the norms are equivalent meaning that for any norms α and β on E, there exists µ 1 > 0, µ 2 > 0 such that µ 1 α β µ 2 β Scalar Product and Scalar Product Spaces In this thesis,, denotes a bilinear form (respectively a sesquilinear form) over E E if K = R (respectively K = C). Let M K n n be nonsingular. The form, M is defined by x, y M = x, My = y M x for all x, y K n. In what follows, we assume that the form, M is symmetric if K = R, that is or Hermitian if K = C x, y M = y, x M y, x M = x, y M. Definition 1.2 In this thesis, we say that the symmetric or Hermitian form, M is a scalar product if, M is positive definite, that is, x E \ {0}, x, x M > 0. (1.7) Otherwise, we refer to, M as an indefinite scalar product. 22
23 In the rest of this paragraph, we only consider definite positive scalar products. The CauchySchwartz inequality (x, y) E 2, x, y x, x y, y, (1.8) applies to any definite positive scalar product. Then, following Definition 1.1 and using (1.8), x x, x defines a norm over E. This norm is known a the 2norm and it is usually denoted by 2. Definition 1.3 For a given scalar product, matrices that preserve the scalar product are called orthogonal if K = R or unitary if K = C. O n (respectively U n ) denotes the set of n n orthogonal matrices (respectively the set of m m unitary matrices). It follows immediately that Q T Q = I n, Q O n, Q Q = I n, Q U n. For F E, F denotes the orthogonal complement of F and it is defined by F = {x E : x, y = 0, y F}. If F is a linear subspace of E then we have the direct sum decomposition E = F F Matrices, Vectors and their Norms (x, y) x, y = y x is the usual scalar product over K n. The induced vector 2norm is denoted by 2 and it is defined by x 2 = ( n k=1 x k 2 ) 1 2 = x x. 23
24 Other useful norms over K n are given by x 1 = n x k, k=1 x = max 1 k n x k. Let A = (a ij ) K m n. The subordinated matrix norm of A is defined by Ax α A α,β = sup, x 0 x β where α is a norm over K m and β is a norm over K n. It follows that A 1 = max 1 j n m a ij, i=1 A 2 = ρ(a A), n A = max 1 i m a ij, j=1 where for X K n n, the spectral radius ρ(x) is ρ(x) = max{ λ, det(x λi) = 0}. The matrix subordinated 2norm is invariant under orthogonal or unitary transformations, Q 1 XQ 2 2 = X 2, for all X K m n and orthogonal or unitary Q 1, Q 2. The trace of a square matrix is the sum of its diagonal elements and for X K n n, X = (x ij ) it is denoted by trace(x) = n x kk. (X, Y ) trace(y X) is the usual scalar product over K m n. The induced matrix k=1 norm is known as the Frobenius norm and it is defined by ( m ) 1 2 n A F = a ij 2. i=1 24 j=1
25 The Frobenius norm is invariant under orthogonal or unitary transformations, for all X K m n, U U m and V U n. UXV F = X F, Definition 1.4 Let µ = ( 1 µ k ) 0 k m, with µ k > 0. The µweighted Frobenius norm is induced by the innerproduct over M n (C) m+1, ( m ) 1 A, B = trace B µ ka k k and it is denoted by A F,µ = A, A. k=0 The µweighted 2norm is defined by, ( m A 2,µ = Differential Calculus k=0 A k µ k 2 2 ) 1 2. Let f : E F, where E, F are two normed vector spaces. f is differentiable or Fréchet differentiable at x V x E, where V x is an open neighborhood of x if there exists a linear map df(x) : E F, such that lim h 0 1 (f(x + h) f(x) df(x)h) = 0. h In this thesis, we only consider the case where E has a finite dimension. Thus, if f is linear, then f is differentiable and df = f. All the vector spaces are vector spaces on R and thus all the functions are considered as functions of real variables and the differentiation is real. The following theorem is the wellknown implicit function theorem [4], [63] that we are going to use several times in this thesis. Theorem 1.1 Let f : E F G (x, y) f(x, y) 25
26 be differentiable, where E, F and G are normed vector spaces. Assume that f(x, y) = 0 and that f (x, y) is nonsingular for some (x, y) E F. Then, y there exist a neighborhood of x, V x, a neighborhood of y, V y and a differentiable function ϕ : V x V y such that y = ϕ(x) and for all x V x, f( x, ϕ( x)) = 0. Moreover, dϕ(x) = ( f ) 1 f (x, y) y (x, y). x Definition 1.5 Let f : R n R p. Assume that rank(df(x)) = p whenever f(x) = 0. Then, f 1 ({0}) is a (n p)dimensional manifold in R n. We now give a fundamental result from optimization, the Lagrange multipliers theorem [4]. Theorem 1.2 Let g : E R be differentiable, where E is a normed vector spaces of finite dimension n. Let S E be a differentiable manifold of dimension d defined by S = {y E, f k (y) = 0, k = 1: n d}. Assume that x S is an extremum of g on S. Then, there exist n d scalars c k, k = 1: n d, such that n d dg(x) = c k df k (x). k=1 We refer to [4] and [63] for a more detailed presentation of differential calculus and manifolds. 1.4 Special Matrix Subsets (K) denotes the set of upper triangular matrices in K n n with a real diagonal. Sym(K) and Skew(K) are the linear subspaces of symmetric matrices and skewsymmetric matrices, respectively, with coefficients in K. Herm and SkewH 26
27 are the linear subspaces of Hermitian matrices and skewhermitian matrices, respectively. dim denotes the dimension of a linear space in R. We recall that dim (R) = dim Sym(R) = n2 + n, (1.9) 2 dim (C) = dim Herm = dim SkewH = n 2, (1.10) dim Skew(R) = n2 n, (1.11) 2 dim Sym(C) = n 2 + n, dim Skew(C) = n 2 n. (1.12) Note that SkewH = iherm. For x K n, diag(x) denotes the n n diagonal matrix with diagonal x. For X K n n, we denote Π d (X) the diagonal part, Π u (X) the strictly upper triangular part and Π l (X), the strictly lower triangular part of X. 1.5 (J, J)Orthogonal and (J, J)Unitary Matrices We denote by diag k n(±1) the set of all n n diagonal matrices with k diagonal elements equal to 1 and n k equal to 1. A matrix J diag k n (±1) for some k is called a signature matrix. A matrix H R n n is said to be (J, J)orthogonal if H T JH = J, where J, J diag n k(±1). We denote by O n (J, J) the set of n n (J, J)orthogonal matrices. If J = J then we say that H is Jorthogonal or pseudoorthogonal and the set of Jorthogonal matrices is denoted by O n (J). We say that a matrix is hyperbolic if it is (J, J)orthogonal or pseudoorthogonal with J ±I. We recall that if J = ±I, then O n (±I) = O n is the set of orthogonal matrices. We extend the definition of (J, J)orthogonal matrices to rectangular matrices in R m n, with m n. H R m n is (J, J)orthogonal if H T JH = J with 27
28 J diag k m (±1) and J diagq n (±1). We denote by O mn(j, J) the set of (J, J) orthogonal in R m n. The definition of signature matrices can be extended and generalized to complex signature matrices. Let U = {z C : z = 1} denote the unit circle in C. We define the set of complex signature matrices as diagonal matrices such that each diagonal entry is in U and we denote the set of n n complex signature matrices by diag n (U). (J, J)unitary matrices are the complex counterpart of (J, J)orthogonal matrices and we say that a matrix H K n n is (J, J)unitary matrix if H JH = J where J and J are complex signature matrices. We denote by U n (J, J) the set of n n (J, J)unitary matrices. A similar set is the set of complex (J, J)orthogonal matrices that we denote by O n (J, J, C). We say that a matrix H K n n is complex (J, J)orthogonal if H T JH = J, where J, J diagn (U). Similarly, we denote by U mn (J, J) we denote the set of m n (J, J)unitary matrices and by O mn (J, J, C) the set of m n complex (J, J)orthogonal matrices. We show that O mn (J, J), U mn (J, J) and O mn (J, J, C) can respectively be identified to R d, R n2 and R 2d, with d = n2 n 2. We show that each of these sets are manifolds and we compute their dimension. Then, the introduction of local coordinate systems enable us to make the identification mentioned above. Lemma 1.3 O n (J, J), U n (J, J) and O n (J, J, C) are manifolds with respective dimension d, n 2 and 2d with d = n2 n 2. Proof. Let q 1 : R n n R n n and q 2, q 3 : C n n C n n be defined by q 1 (X) = X T JX J, q 2 (X) = X JX J and q 3 (X) = X T JX J. We recall that O n (J, J) = q 1 1 ({0}), U n (J, J) = q 1 2 ({0}), and O n (J, J, C) = q 1 3 ({0}). For 1 k 3, q k is clearly differentiable. We have that dq 1 (H 1 ) H 1 = H T 1 J H 1 + H T 1 JH 1, 28
29 dq 2 (H 2 ) H 2 = H 2 J H 2 + H 2 JH 2, dq 3 (H 3 ) H 3 = H T 3 J H 3 + H T 3 JH 3. To compute the dimension of the three manifolds, we need to determine their tangent spaces that is the null space of each dq k (H k ), k = 1: 3, with H k being in one of these manifolds. We have that null(dq 1 (H)) = JH T Skew(R), null(dq 2 (H)) = JH SkewH, null(dq 3 (H)) = JH T Skew(C). Thus, following the dimensions given by (1.9)(1.12), O n (J, J) is a n2 n 2 dimensional manifold, U n (J, J) is a n 2 dimensional manifold and O n (J, J, C) is n 2 n a dimensional manifold. Let X O mn (J, J), Y U mn (J, J) and Z O mn (J, J, C). There exists differentiable onetoone functions φ k, 1 k 3, open sets V 1 R d, V 2 R n2, V 3 R 2d, V X R m n, V Y C m n and V Z C m n such that φ 1 (V 1 ) = V X O n (J, J), (1.13) φ 2 (V 2 ) = V Y U n (J, J), (1.14) φ 3 (V 3 ) = V Z O n (J, J, C). (1.15) Moreover, the differential of these maps φ k have full rank over the entire space where they are defined. 1.6 Matrix Operators Properties For an operator or a linear map T defined on K n n, the 2norm is defined by T 2 = sup T (X) F. X F =1 29
30 Some authors denote this norm by F,F. The choice of this norm is justified by its differentiability properties and its computational simplicity. We now present some notations and we give some results that are needed throughout this thesis. Theorem 1.4 Let A, B, X K n n. We define the operators T 2 X = X A and T 1 X = AXB. Then, If A and B are nonsingular then T 2 2 = max a ij, (1.16) ij T 1 2 = (A B) 2 = A 2 B 2, (1.17) min T 1 (X) F = A B (1.18) X F =1 Proof. It is straightforward to show that the right hand side of (1.16) is an upper bound for T 2 2. Let a pq = max ij a ij. Then, the bound is attained by e p e T q. Let A = Q 1 S 1 Z T 1 and B = Q 2S 2 Z T 2 be the singular value decompositions of A and B. Then (A B) = (Q 1 Q 2 )(S 1 S 2 )(Z T 1 Z T 2 ) so that (A B) 2 = (S 1 S 2 ) 2 = A 2 B 2 proving the second part of (1.17). We have T 1 (X) F = (A B)vec(X) 2, T 1 2 = (A B) 2 = A 2 B 2. Similarly, for (1.18), we have min T 1(X) F = min (S 1 S 2 )vec(z 2 XZ1 T ) F, X F =1 X F =1 = A B
A Modern Course on Curves and Surfaces. Richard S. Palais
A Modern Course on Curves and Surfaces Richard S. Palais Contents Lecture 1. Introduction 1 Lecture 2. What is Geometry 4 Lecture 3. Geometry of InnerProduct Spaces 7 Lecture 4. Linear Maps and the Euclidean
More informationOrthogonal Bases and the QR Algorithm
Orthogonal Bases and the QR Algorithm Orthogonal Bases by Peter J Olver University of Minnesota Throughout, we work in the Euclidean vector space V = R n, the space of column vectors with n real entries
More informationControllability and Observability of Partial Differential Equations: Some results and open problems
Controllability and Observability of Partial Differential Equations: Some results and open problems Enrique ZUAZUA Departamento de Matemáticas Universidad Autónoma 2849 Madrid. Spain. enrique.zuazua@uam.es
More information1 Sets and Set Notation.
LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most
More informationChapter 5. Banach Spaces
9 Chapter 5 Banach Spaces Many linear equations may be formulated in terms of a suitable linear operator acting on a Banach space. In this chapter, we study Banach spaces and linear operators acting on
More informationTHE PROBLEM OF finding localized energy solutions
600 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 45, NO. 3, MARCH 1997 Sparse Signal Reconstruction from Limited Data Using FOCUSS: A Reweighted Minimum Norm Algorithm Irina F. Gorodnitsky, Member, IEEE,
More informationHow Bad is Forming Your Own Opinion?
How Bad is Forming Your Own Opinion? David Bindel Jon Kleinberg Sigal Oren August, 0 Abstract A longstanding line of work in economic theory has studied models by which a group of people in a social network,
More informationA Tutorial on Support Vector Machines for Pattern Recognition
c,, 1 43 () Kluwer Academic Publishers, Boston. Manufactured in The Netherlands. A Tutorial on Support Vector Machines for Pattern Recognition CHRISTOPHER J.C. BURGES Bell Laboratories, Lucent Technologies
More informationSemiSimple Lie Algebras and Their Representations
i SemiSimple Lie Algebras and Their Representations Robert N. Cahn Lawrence Berkeley Laboratory University of California Berkeley, California 1984 THE BENJAMIN/CUMMINGS PUBLISHING COMPANY Advanced Book
More informationSubspace Pursuit for Compressive Sensing: Closing the Gap Between Performance and Complexity
Subspace Pursuit for Compressive Sensing: Closing the Gap Between Performance and Complexity Wei Dai and Olgica Milenkovic Department of Electrical and Computer Engineering University of Illinois at UrbanaChampaign
More informationCOSAMP: ITERATIVE SIGNAL RECOVERY FROM INCOMPLETE AND INACCURATE SAMPLES
COSAMP: ITERATIVE SIGNAL RECOVERY FROM INCOMPLETE AND INACCURATE SAMPLES D NEEDELL AND J A TROPP Abstract Compressive sampling offers a new paradigm for acquiring signals that are compressible with respect
More informationIterative Methods for Optimization
Iterative Methods for Optimization C.T. Kelley North Carolina State University Raleigh, North Carolina Society for Industrial and Applied Mathematics Philadelphia Contents Preface xiii How to Get the Software
More informationTHESE. Christof VÖMEL CERFACS
THESE pour obtenir LE TITRE DE DOCTEUR DE L INSTITUT NATIONAL POLYTECHNIQUE DE TOULOUSE Spécialité: Informatique et Télécommunications par Christof VÖMEL CERFACS Contributions à la recherche en calcul
More informationAcknowledgements. I would like to thank my supervisor, Prof. Nicholas Young, for the patient guidance, encouragement
Acknowledgements I would like to thank my supervisor, Prof. Nicholas Young, for the patient guidance, encouragement and advice he has provided throughout my time as his student. I have been extremely lucky
More informationA Tutorial on Spectral Clustering
A Tutorial on Spectral Clustering Ulrike von Luxburg Max Planck Institute for Biological Cybernetics Spemannstr. 38, 7276 Tübingen, Germany ulrike.luxburg@tuebingen.mpg.de This article appears in Statistics
More informationSome Applications of Laplace Eigenvalues of Graphs
Some Applications of Laplace Eigenvalues of Graphs Bojan MOHAR Department of Mathematics University of Ljubljana Jadranska 19 1111 Ljubljana, Slovenia Notes taken by Martin Juvan Abstract In the last decade
More informationDecoding by Linear Programming
Decoding by Linear Programming Emmanuel Candes and Terence Tao Applied and Computational Mathematics, Caltech, Pasadena, CA 91125 Department of Mathematics, University of California, Los Angeles, CA 90095
More informationFrom Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
SIAM REVIEW Vol. 51,No. 1,pp. 34 81 c 2009 Society for Industrial and Applied Mathematics From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images Alfred M. Bruckstein David
More informationIEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, 2013. ACCEPTED FOR PUBLICATION 1
IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, 2013. ACCEPTED FOR PUBLICATION 1 ActiveSet Newton Algorithm for Overcomplete NonNegative Representations of Audio Tuomas Virtanen, Member,
More informationOptimization with SparsityInducing Penalties. Contents
Foundations and Trends R in Machine Learning Vol. 4, No. 1 (2011) 1 106 c 2012 F. Bach, R. Jenatton, J. Mairal and G. Obozinski DOI: 10.1561/2200000015 Optimization with SparsityInducing Penalties By
More informationA Course on Number Theory. Peter J. Cameron
A Course on Number Theory Peter J. Cameron ii Preface These are the notes of the course MTH6128, Number Theory, which I taught at Queen Mary, University of London, in the spring semester of 2009. There
More informationHighOrder Contrasts for Independent Component Analysis
158 JeanFrançois Cardoso LETTER Communicated by Anthony Bell HighOrder Contrasts for Independent Component Analysis JeanFrançois Cardoso Ecole Nationale Supérieure des Télécommunications, 75634 Paris
More informationLevel sets and extrema of random processes and fields. JeanMarc Azaïs Mario Wschebor
Level sets and extrema of random processes and fields. JeanMarc Azaïs Mario Wschebor Université de Toulouse UPS CNRS : Institut de Mathématiques Laboratoire de Statistiques et Probabilités 118 Route de
More informationSampling 50 Years After Shannon
Sampling 50 Years After Shannon MICHAEL UNSER, FELLOW, IEEE This paper presents an account of the current state of sampling, 50 years after Shannon s formulation of the sampling theorem. The emphasis is
More informationRECENTLY, there has been a great deal of interest in
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 47, NO. 1, JANUARY 1999 187 An Affine Scaling Methodology for Best Basis Selection Bhaskar D. Rao, Senior Member, IEEE, Kenneth KreutzDelgado, Senior Member,
More informationFoundations of Data Science 1
Foundations of Data Science John Hopcroft Ravindran Kannan Version /4/204 These notes are a first draft of a book being written by Hopcroft and Kannan and in many places are incomplete. However, the notes
More informationFoundations of Data Science 1
Foundations of Data Science John Hopcroft Ravindran Kannan Version 2/8/204 These notes are a first draft of a book being written by Hopcroft and Kannan and in many places are incomplete. However, the notes
More informationChoosing Multiple Parameters for Support Vector Machines
Machine Learning, 46, 131 159, 2002 c 2002 Kluwer Academic Publishers. Manufactured in The Netherlands. Choosing Multiple Parameters for Support Vector Machines OLIVIER CHAPELLE LIP6, Paris, France olivier.chapelle@lip6.fr
More informationScattered Node Compact Finite DifferenceType Formulas Generated from Radial Basis Functions
Scattered Node Compact Finite DifferenceType Formulas Generated from Radial Basis Functions Grady B. Wright a,,, Bengt Fornberg b,2 a Department of Mathematics, University of Utah, Salt Lake City, UT
More informationSketching as a Tool for Numerical Linear Algebra
Foundations and Trends R in Theoretical Computer Science Vol. 10, No. 12 (2014) 1 157 c 2014 D. P. Woodruff DOI: 10.1561/0400000060 Sketching as a Tool for Numerical Linear Algebra David P. Woodruff IBM
More information