sup As we have seen, this sup is attained, so we could use max instead.

Transcription

1 Real Analysis 2 (G ) Real Analysis for Matrices Professor Mel Hausner Definition of norm. If A is an m n real matrix, it is possible to define a norm A m satisfying the basic inequality Ax A m x. In particular, sup Ax / x <. (1) x 0 Here, x is taken as a vector in R n, or equivalently, an n 1 column vector, using the usual norm in R n, namely x = x 2 i. We can prove inequality (1) as follows: Consider the function f(u) = Au defined on the unit sphere S = {u: u =1}. f(u) is a continuous, real valued function on the compact set S. 1 As such it attains a maximum value M. Thus Au M if u = 1. Now if x 0,x/ x is a unit vector, so A(x/ x ) M. Thus Ax M x for all x. 2 We now define A m = sup x 0 Ax / x = sup Au (2) u S As we have seen, this sup is attained, so we could use max instead. In what follows, we use the following results, all based on this definition. (i) Ax A m x for all x. (ii) If Ax M x for all x, then A m M. Note that (i) and (ii) show that A m = 0 if and only if A =O. The following theorem relates this definition of norm to inner products. Theorem: If A M m n then A m = sup Au, v (u R n and v R m.) (3) u = v =1 Proof: We may assume A O. Using the Cauchy-Schwarz inequality, Au, v Au v A m when u and v are unit vectors. Thus Au, v A m. sup u = v =1 Now choose a unit vector u 0 such that Au 0 = A m. If we take v 0 = Au 0 / Au 0,wefind that Au 0,v 0 = Au 0,Au 0 / Au 0 = Au 0 = A m. Thus, sup Au, v A m. This prove the result. u = v =1 1 In this context, a closed, bounded set. 2 This is clearly true for x =0. 1

2 Any easy corollary is: A m = A t m (4) Compatability with R n norm. If A is a n 1 column vector a, A acts on x R 1 = R, so ax / x = x a / x = a. So the definition is consistent with the usual norm in R n. Similarly, if A is a 1 n row vector a, we can use Equation (4) to give this result for row vectors. Other Norms. Norms of m n matrices are not so easily computable. Here are some alternative commonly defined norms on the space M m n of all m n real matrices. (i) A 2 = a 2 ij. i,j (ii) A 1 = a ij. i,j (iii) A = max a ij. i,j (iv) A m = sup Au = sup u =1 x 0 Ax x. Equivalence of Norms. Two norms a and b on M m n are said to be equivalent if for some numbers M,N > 0, we have for all A M m n, A a M A b and A b N A a The idea is that for equivalent norms, anything small with respect to one of the norms is also small with respect to the other. We now show that all of these norms are equivalent. We do this in four steps. A) A m A 2. We compute the maximum of Au over all unit vectors u. Suppose the rows of A are r 1,...,r m. The components of Au are r 1,u,..., r m,u. Thus m Au 2 m = r m,u 2 i=1 This proves the result, using the definition of A m. B) A 2 A 1. To prove this note that for positive x i, m m r m 2 u 2 = r m 2 =( A 2 ) 2 i=1 i=1 ( x i ) 2 = x 2 i +2 i<j x i x j x 2 i It follows that A 2 = (aij ) 2 a ij = A 1 2

3 C) A 1 mn A. Let M = A = max i,j a ij. Then for each i, j, we have a ij M. There are mn inequalities here. Summing over all i, j, we get A 1 = a ij mnm = mn A. D) A A m. Let e 1,...,e n be the standard basis vectors in R n. Using Equation (3), Ae j, ±e i A m. This gives a ij A m. Therefore A = max i,j a ij A m. In what follows, we shall usually use A m as the norm. To simplify typography, we shall use A instead of A m. This should not cause confusion, since x will be well defined according as x is a vector or a matrix. in the case of n 1or1 n vectors, we have already seen that the vector norm is the same as the matrix norm. The equivalence of the norms is useful when discussing limits and continuity. For example, if A n is a sequence of matrices, and A n A a 0 with respect to any of the norms we defined, then this limit will be true for all of the norms. Some Properties of the Norm. The norm of the identity. It follows immediately from the definition that if I is the identity n n matrix, then I =1. 3 The norm is positive. If A O, then A > 0. This follows from the definition, by observing that there exists x such that Ax 0. Of course, we have O =0. The Triangle Inequality, A + B A + B. Here A and B are n n matrices. The proof: If x R n, then (A + B)x = Ax + Bx Ax + Bx A x + B x =( A + B ) x Using Definition (2), we have the result. This result is also true for all of the norms previously defined. Scalar Multiples. It is easily verified that if a is any real number then aa = a A. The triangle inequality and the scalar multiplication property, along with the positivity of the norm comes up frequently in analysis. A vector space with such a norm defined on it is called a normed linear space. All of the above norms make M m n into a normed linear space. The Multiplicative Property, AB A B. Here A and B are respectively m n and n p matrices. If x R p, we have (AB)x = A(Bx) A Bx A B x 3 Note that I 2 = n, I 1 = n, I =1. 3

4 Hence AB A B. As a corollary, if A is invertible, A 1 1/ A. For we have I = AA 1 = I. Taking norms, we get 1 = AA 1 A A 1. Some topology. If V is a normed linear space, we define the distance d(a, B) between two elements using the definition d(a, B) = B A. The function d satisfies the following distance axioms for a metric space: 1. d(a, B) 0 with equality if and only if A = B. 2. d(a, B) =d(b, A). 3. d(a, B) d(a, C)+d(C, B). Using the distance function, limits can be defined in a natural manner: A n A if and only if for any ɛ>0 there exists an integer N 0 such that if n>n 0 we have A n A <ɛ. This can also be written d(a n,a) <ɛ. Equivalently, we have A n A if and only if A n A 0. We can do the same for continuous variables. If A(t) is defined in a neighborhood of t = t 0,we define A = lim A(t) to mean A(t) A 0ast t 0. Equivalently, the classical definition t t0 is: For any ɛ>0 there exists a δ>0 such that if 0 < t t 0 <δthen A(t) A < 0. In particular a function A(t) is continuous at t = t 0 if it is defined a neighborhood of t 0 and lim A(t) =A(t 0 ). t t 0 Completeness. Let V be a normed linear space. A Cauchy sequence a n in V is a sequence with the property that lim A n A m =0. Using the ɛ, N 0 version, this means for any m,n ɛ>0there exists an integer N 0 such that if n, m > N 0 then A n A m <ɛ.anormed linear space V is said to be complete if any Cauchy sequence converges to a limit. We take for granted that the reals R is a complete space. It follows the space M m n is complete. We can do this component by component as follows. If A n is a Cauchy sequence, then so is each component of A n. In fact we have (a ij ) n (a ij ) m < A n A m. Therefore, (a ij ) n is a Cauchy sequence, and it approaches a limit a ij. It is now an easy matter to show that A n A. Given any ɛ>0, each entry of A n A has absolute value less than ɛ for n sufficiently large. Thus A n A 0asn. Thus by the remarks above, A n A 0asn or A n A as n. Real Analysis. We review some results from calculus. Many of these results have the same proof as in calculus, except that absolute values would have to be replaced by norms. In many cases, these results can be proved by using components, and item (1) below. We first consider functions from the reals (or some interval thereof) to M m n. Continuity Results. 1. A(t) is continuous at t = t 0 if and only if each component a ij (t) ofa(t) is continuous at t = t If A(t) and B(t) are continuous at t = t 0 then so is A(t)+B(t) 4

5 3. If A(t) is continuous at t = t 0, and f(t) is a real valued function continuous at t = t 0, then f(t)a(t) is continuous at t = t If A(t) and B(t) are continuous at t = t 0 then so is A(t)B(t). Here, as in future such results, it is naturally assumed that the product is well defined. 5. If A(t) is a square matrix and is continuous at t = t 0, then det(a(t)) and trace(a(t)) are continuous at t = t If A(t) is continuous at t = t 0, and A(t) is invertible, then so is A(t) 1. Calculus Results. Here we have to be careful because commutativity of multiplication is no longer present. As usual, we define 1. If da dt exists, so does da dt exists for every i, j, then so does da dt. da dt = lim A(t + h) A(t) h 0 h for every i, j and then da = da ij ij dt ij dt. Conversely, if da ij dt 2. If A(t) is differentiable at t = t 0, then it is continuous there. For a proof, note that ( ) A(t + h) A(t) A(t + h) A(t) =h h 3. Now let h 0. d da(t) (A(t)+B(t)) = + db(t) dt dt dt We write this simply as (A + B) = A + B. 4. (AB) = AB + A B. 5. If f is a differentiable scalar, (fa) = fa + f A. 6. In particular, if c is constant, (ca) = ca. 7. (A 1 ) = A 1 A A 1 We can show this as follows: Start with A(t + h) 1 A(t) 1 = A(t + h) 1 (A(t + h) A(t))A(t) 1 Now divide by h and let h 0. 5

6 Series. If A n is an infinite series of matrices, convergence is defined in the usual way. n Letting S n = A k, S = A n, means that S n S as n. In this case we say the series converges to S. Because M m n is complete, we have the following Cauchy criterion: n A n converges if and only if lim n,m k=m A k =0 As a consequence, we have the following very useful theorem on absolute convergence. If A n <, then absolutely. A n converges. In this case we say that the series converges Power Series of a Matrix Variable. In this section, we work with square, n n matrices. We consider a series f(x) = a n x n. Here x is a real or complex variable, and a n is a sequence of real or complex numbers. We assume that the series has a radius of convergence R, where 0 <R. This implies that the series converges absolutely if x<r. Now we claim that if A <R, then the series a n A n converges absolutely. In fact, a n A n = a n A n a n A n converges. It is reasonable to write f(a) = a n A n. This is a generalization of the procedures we used before when we substituted a matrix for a variable in a polynomial. For example, we write e A A n n+1 An = and log(i + A) = ( 1) n! n=1 n As with polynomials, formal identities in x remain correct for matrix values A. For example, we have (e x ) 2 = e 2x. Namely It follows that ( 2 ( x /k!) k 2 = 2 k x /k!) k (5) ( 2 ( A /k!) k 2 = 2 k A /k!) k (6) 6

7 for any square matrix. The reason is that Equation (5) is equivalent to a system of identities n involving the coefficients. In this case, the equations are (1/k!)(1/(n k)!) = 2 n /n!. And this system is enough to prove (6). This argument generalizes to the following principle. Preservation of Identities. If f(x), g(x), h(x) are power series in x, convergent for x <R, and if f(x)g(x) =h(x) for x <R, then f(a)g(a) =h(a) for all matrices A with A <R. If f(x) and g(x) are power series in x and g(0) = 0, and both converge for x <R, then f(g(x)) is a power series in x convergent for a sufficiently small radius of convergence. The computation is as follows: f(x) = a n x n and g(x) = n=a b n x n.so f(g(x)) = a 0 +a 1 x(b 1 +b 2 x+b 3 x )+a 2 x 2 (b 1 +b 2 x+b 3 x ) 2 +a 3 x 3 (b 1 +b 2 x+b 3 x ) This gives f(g(x)) = a 0 + a 1 b 1 x +(a 1 b 2 + a 2 b 2 1 )x2 +(a 1 b 3 +2a 2 b 1 b 2 + a 3 b 3 1 )x It is clear that the coefficients of the composite function f(g) are finite algebraic functions of the coefficients of f and g. and it can be proved that the resulting series converges for sufficiently small values of x. We have the similar preservation of identities theorem in this case. If f(x), g(x), h(x) are power series in x, convergent for x <Rwith g(0) = 0, and if f(g(x)) = h(x) for x <R, then f(g(a)) = h(a) for all matrices A with A <R Here is an illustrative example. We have e log(1+x) =1+x for x < 1. Thus, e log(1+a) =1+A or all (square) matrices with A < 1. Here e A is defined as above, and log(i + A) = A A 2 /2+A 3 /3...for A < 1. Note: did you ever check that e log(1+x) =1+x formally? It states that 1+x(1 x/2+x 2 /3...)+x 2 (1 x/2+x 2 /3...) 2 /2!+x 3 (1 x/2+x 2 /3...) 3 /3!+...=1+x Anyway it s true! Check out a few terms. Power Series with Matrix Coefficients. In the previous section, we considered a series of the form F (A) = a n A n where a n is a sequence of complex coefficients and A is a square matrix. We now consider series of the form F (t) = A n t n where t is a real (or complex) variable and A n. Before, we had a matrix valued power series function of a matrix A, whereas now we now have a matrix valued power series function of a real (or complex) variable t. 7 r=0

8 The theory of such functions very closely follows the classical theory. We summarize the results. The proofs follow the classical proofs on these results. 1) Any series A n t n has a radius of convergence R, where 0 R. For R = 0, the series converges only for t = 0. For R =, the series converges for all t. If 0 <R<, the series converges absolutely for t <R, and diverges for t >R. If S<R, the series converges uniformly for t S. 2) If F (t) = A n t n, then F (t) = na n t n 1 = (n +1)A n+1 t n for t within the radius of convergence. The derived series has the same radius of convergence. Similarly, A n (b n+1 a n+1 )/(n + 1) when a and b are inside the radius of convergence. b a F (t) = 8