CHAPTER III  MARKOV CHAINS


 Bryce Bruce
 1 years ago
 Views:
Transcription
1 CHAPTER III  MARKOV CHAINS JOSEPH G. CONLON 1. General Theory of Markov Chains We have already discussed the standard random walk on the integers Z. A Markov Chain can be viewed as a generalization of this. We shall only consider in this Chapter Markov chains on a countable (finite or infinite) state space F. In the finite case we can identify F as the set of positive integers F = {1, 2,..., m} for some m = F, and in the infinite case as all positive integers F = {n Z : n 1}. The chain is determined by a set of transition probabilities p t (i, j), i, j F, t = 0, 1, 2,..., where t denotes discrete time and p t (i, j) is the probability of moving from state i to state j at time t. Thus (1.1) p t (i, j) 0, i, j, F ; p t (i, j) = 1, i F. To define a probability space associated with these transition probabilities we need to set a distribution function π : F R for the initial state of the chain. Thus π( ) satisfies (1.2) π(j) 0, j F ; π(j) = 1. The probability space Ω is given by Ω = F, which is the infinite product of the state space F, and the σ field F is the Borel field generated by finite dimensional rectangles as usual. Random variables X n : Ω F, n = 0, 1, 2..., are defined by X n (ω) = ω n, n = 0, 1,..., where ω = (ω 0, ω 1,..) Ω. The probability measure P depends on π( ), so we shall denote it by P π. The measure P π is determined once we know the p.d.f. for every set of variables (X 0,..., X m ), m 0. This is given by the formula m 1 (1.3) P (X 0 = j 0, X 1 = j 1,..., X m = j m ) = π(j 0 ) p t (j t, j t+1 ). Note the p.d.f for X 0 is π( ), and for any subset A F, (1.4) P (X t+1 A Xt, X t 1,.., X 0 ) = P (X t+1 A Xt ) with probability 1. The so called Markov property or no memory property (1.4) characterizes a Markov chain, and can be an alternative starting point for the theory of Markov chains. Note the similarity of (1.4) to the definition of a Martingale. One big advantage of studying Markov chains is a technique is available to compute many expectation values. This is the so called backward Kolmogorov equation. Thus let f : F R be a function and suppose we want to evaluate the expectation j F j F (1.5) E [ f(x T ) X0 = i ], i F. 1 t=0
2 2 JOSEPH G. CONLON Then (1.5) is given by u(i, 0) where u(i, t), i F, t = 0,.., T, is a solution to the terminal value problem, (1.6) u(i, t) = j F p t (i, j)u(j, t + 1), i F, t < T ; u(i, T ) = f(i), i F. Note the equation (1.6) is to be solved backwards in time from time T to time 0. We turn now to Markov chains where the transition probabilities are independent of time, whence p t (i, j) = p(i, j), t = 0, 1,.., i, j F. An important property of these Markov chains is the strong Markov property. Observe from (1.4) (1.7) P (X t+1 A Xt, X t 1,.., X 0 ) = P (X t+1 A Xt ) = P (X 1 A X0 ) with probability 1. The strong Markov property is a random version of (1.7). Thus let τ : Ω Z be a nonnegative measurable function which has the property (1.8) {ω Ω : τ(ω) = T } F(X 0, X 1,.., X T ), T = 0, 1,... Such a function is called a stopping time. Proposition 1.1 (Strong Markov Property). Consider the time independent Markov chain (X 0, X 1,..) with initial distribution π( ) and let τ( ) be a stopping time for this chain. Then the sequence of variables (X τ, X τ+1, X τ+2,...) has the same distribution as the sequence (X 0, X 1,..) with initial distribution π τ ( ), where π τ ( ) is the pdf of X τ under π( ). Proof. We illustrate the proof with the simplest case, which is then easy to generalize. Thus (1.9) P (X τ = j 0, X τ+1 = j 1, X τ+2 = j 2 ) = P (τ = m, X m = j 0, X m+1 = j 1, X m+2 = j 2 ). m=0 Now from (1.8) it follows (1.10) P (τ = m, X m = j 0, X m+1 = j 1, X m+2 = j 2 ) = P (τ = m, X m = j 0 )p(j 0, j 1 )p(j 1, j 2 ). Since we have (1.11) we conclude P (τ = m, X m = j 0 ) = π τ (j 0 ), m=0 (1.12) P (X τ = j 0, X τ+1 = j 1, X τ+2 = j 2 ) = π τ (j 0 )p(j 0, j 1 )p(j 1, j 2 ), whence the result holds. Definition 1. An initial distribution π( ) for the time independent Markov chain is said to be stationary if it satisfies the equation (1.13) π(i)p(i, j) = π(j), j F. i F
3 MATH Evidently (1.13) implies if X 0 has distribution π( ) then X 1 also has distribution π( ). One can easily see further the entire sequence (X 0, X 1,..) is stationary in the sense of Definition 4 of Chapter II page 18. An obvious question to ask then is does a Markov chain have a stationary distribution, and if so is it unique? We can give a satisfactory answer to the uniqueness question by introducing the notion of indecomposability of the chain. We say the chain is decomposable if there exists disjoint subsets A 0, A 1 of F such (1.14) p(i, j) = 1, i A k, k = 0, 1. j A k Thus (1.14) states the chain allows no communication between the subsets A 0 and A 1 of F. Hence we may reduce the original chain to independent chains on the reduced state spaces A 0, A 1. A chain which is not decomposable is called indecomposable. Proposition 1.2. Suppose the Markov chain is indecomposable and a stationary distribution π( ) for it exists. Then π( ) is unique and the associated stationary sequence (X 0, X 1,...) is ergodic. Proof. We first show for stationary π( ) the corresponding stationary sequence (X 0, X 1,...) is ergodic. Thus let T be the shift operator on Ω, P = P π be the probability measure on Ω, and C F be an invariant set. We define a function φ : F R by φ(j) = P (C X 0 = j), j F. Evidently φ(j) is only uniquely defined if π(j) > 0. Otherwise we can take φ(j) to be anything we want. Note now (1.15) φ(x 0 ) = P (C X 0 ) with probability 1, and (1.16) P (C X 0 ) = E[ P (C X 1, X 0 ) X 0 ] with probability 1. By the invariance of C we have C F(X 1, X 2,...), whence it follows from the Markov property (1.4) (1.17) P (C X 1, X 0 ) = P (C X 1 ) = φ(x 1 ) with probability 1. Evidently (1.16), (1.17) imply (1.18) p(i, j)φ(j) = φ(i), i F with π(i) > 0. j F Note from (1.13) if π(i) > 0 and π(j) = 0 then p(i, j) = 0, whence the values of φ(j) for π(j) = 0 do not enter on the LHS of (1.18). Observe next for any set C F(X 0, X 1,...) one has (1.19) lim n P (C X n, X n 1,..., X 0 ) = χ C with probability 1. This intuitively obvious result follows from the Martingale convergence theorem. In fact define a sequence of random variables Z n, n = 0, 1, 2,.., by Z n = P (C X n, X n 1,..., X 0 ). Then from the definition of conditional expectation one has (1.20) E[ Z n Xn 1,..., X 0 ] = Z n 1, n = 1, 2,..., with probability 1, which implies (1.21) E[ Z n Z n 1,..., Z 0 ] = Z n 1, n = 1, 2,..., with probability 1.
4 4 JOSEPH G. CONLON Hence lim n Z n exists with probability 1, and it is easy to see the limit must be χ C. Observe now for the invariant set C, one can see as in (1.17) P (C X n, X n 1,..., X 0 ) = φ(x n ), whence we have (1.22) lim n φ(x n) = χ C with probability 1. Since we may assume 0 φ(j) 1, j F, and we have from (1.22) (1.23) E[ φ(x 0 ){1 φ(x 0 )} ] = lim n E[ φ(x n){1 φ(x n )} ] = 0, we conclude φ(j) can only take the values 0 or 1 for j F with π(j) > 0. Now let us define sets A 1, A 0 by (1.24) A 1 = {i F : π(i) > 0, φ(i) = 1}, A 0 = {i F : π(i) > 0, φ(i) = 0}. It follows from (1.18) (1.25) j A 1 p(i, j) = 1, i A 1. Note in (1.25) we are again using the fact if π(i) > 0 and π(j) = 0 then p(i, j) = 0. Replacing the function φ( ) by 1 φ( ) in (1.18), we see also from (1.25) (1.26) p(i, j) = 1, i A 0. j A 0 Evidently (1.25), (1.26) and the indecomposability assumption imply either A 0 or A 1 is empty, so let us assume A 0 is empty. It follows then from (1.15) (1.27) P (C) = E[ φ(x 0 ) ] = 1. We similarly see P (C) = 0 if A 1 is empty. We have proved the ergodicity of the stationary sequence. We can prove the uniqueness of π( ) by using Proposition 4.3 of Chapter II. Thus let π 1 ( ) and π 2 ( ) be two invariant distributions with corresponding probability measures P 1, P 2. From Proposition 4.3 of Chapter II it follows P 1 and P 2 are mutually singular, so there exists E F such P 1 (E) = 1, P 2 (E) = 0. Now let C F be the set (1.28) C = n=0t n E, whence it follows T 1 C C. Evidently P 1 (C) = 1, and P 2 (C) = 0 by the measure preserving property of P 2. We consider now the distribution π( ) defined by π( ) = [π 1 ( ) + π 2 ( )]/2, which evidently also satisfies (1.13). The corresponding probability measure is P = [P 1 + P 2 ]/2. Hence T is measure preserving and ergodic under P. Since T 1 C C, it follows from the measure preserving property of T (1.29) P ([C T 1 C] [T 1 C C]) = 0. Lemma 3.2 of Chapter II implies then P (C) = 0 or P (C) = 1. However we easily see P (C) = [P 1 (C) + P 2 (C)]/2 = 1/2, which is a contradiction.
5 MATH Definition 2. A time independent Markov chain is said to be irreducible if all states communicate. That is for any i, j F, (1.30) P (X n = j X 0 = i) > 0 for some n 1 which can depend on (i, j). Note irreducibility implies indecomposability. Lemma 1.1. Suppose a Markov chain is time independent irreducible and a stationary distribution π( ) for it exists. Then π(j) > 0 for all j F. Proof. We use equation (1.13) and choose i F such π(i) > 0. Evidently if (1.30) holds for n = 1 then π(j) > 0. Observe now (1.13) implies (1.31) π(i 1 )p(i 1, i 2 )p(i 2, j) = π(j), j F. Since i 1,i 2 F (1.32) P (X 2 = j X 0 = i) = p(i, i 2 )p(i 2, j), we conclude if (1.30) holds for n = 2 then π(j) > 0. Evidently we can generalize this argument to any n 1. We can apply Proposition 5.2 of Chapter II to obtain a relation between recurrence for the Markov chain and the invariant measure. Proposition 1.3. Suppose a time independent Markov chain is irreducible and has stationary distribution π( ). Then for all i F one has P (X n = i infinitely often X 0 = i) = 1. Furthermore if τ i is the first recurrence time to i one has E[ τ i X 0 = i] = 1/π(i). Proof. The formula for the recurrence time is immediate from Proposition 5.2 since the stationary sequence is ergodic. This result evidently also implies i 2 F (1.33) P (X n = i for some finite n X 0 = i) = 1. Recurrence i.e. return to i infinitely often follows immediately from (1.33). 2. Markov Chains with Finite State Space We consider the case when the Markov chain is time independent and the state space is finite, F = m, in which case the transition probabilities p(i, j), i, j F, form an m m transition matrix T with nonnegative entries which satisfy (2.1) T1 = 1, 1 = [1, 1,..., ]. Equation (1.13) can be written in matrix form as (2.2) πt = π or alternatively T π = π, where T is the adjoint of T. Now (2.1) implies T has eigenvalue 1, and hence T has eigenvalue 1. Thus a nontrivial solution to (2.2) exists. Proposition 2.1. There exists a nontrivial solution π to (2.2) with all nonnegative entries. If the chain is indecomposable see (1.14) it is unique.
6 6 JOSEPH G. CONLON Proof. Evidently (1.13) implies (2.3) π(i) p(i, j) π(j), j F. i F If strict inequality holds in (2.3) for some j F, then on summing (2.3) over j F we conclude (2.4) π(i) > π(j), i F j F which is an impossibility. We have shown therefore the vector with entries π(i), i F, is an eigenvector of T with eigenvalue 1. Assume now the chain is indecomposable and π 1 ( ) and π 2 ( ) are two distinct invariant measures. Setting π( ) = π 1 ( ) π 2 ( ) and defining sets A 0, A 1 by (2.5) A 0 = {i F : π(i) > 0}, A 1 = {i F : π(i) < 0}, we see from the fact i F π(i) = 0, both A 0 and A 1 are nonempty. Using the fact π( ) is a solution to (2.2), we conclude equality holds in (2.3) for all j F. It follows if j / A 0 A 1, then p(i, j) = 0 for all i A 0 A 1. Further if j A 1 then p(i, j) = 0 for i A 0. We conclude (2.6) p(i, j) = 1, i A 0. j A 0 Since we get a similar result for A 1 the chain is decomposable, a contradiction. We have already seen in Proposition 1.2 the stationary sequence (X 0, X 1,...) associated with the Markov chain is ergodic if the chain is indecomposable. Next we wish to show under certain conditions on the transition matrix T the sequence is strong mixing. In order to do this we first prove a classical theorem in the theory of positive matrices. Theorem 2.1 (PerronFrobenius Theorem). Suppose T = (t i,j ) is an n n matrix with entries t i,j satisfying t i,j 0 and such for some positive integer k the entries of T k are all strictly positive. Then there exists an eigenvalue r of T such : (a) r is real and strictly positive. (b) r is a simple root of the characteristic polynomial of T. (c) If λ C is a root of the characteristic polynomial of T different from r then λ < r. (d) The eigenvector of T for r can be chosen so all its entries are strictly positive. Proof. We first construct an eigenvector for T with all positive entries. To do this let S be the set (2.7) S = {x = (x 1,.., x n ) R n : x i > 0, i = 1,.., n, x = 1}, so S is the part of the unit sphere which is strictly in the positive quadrant (if n = 2). Define now a function r : S R by (Tx) i (2.8) r(x) = min. 1 i n x i
7 MATH It is clear r( ) is a nonnegative continuous function on S and is bounded above as n (2.9) r(x) max t i,j. It follows from (2.9) 1 i n j=1 (2.10) r = sup r(x) <, x S Hence for all x S, (2.11) (Tx) i rx i for some i = i(x), 1 i n, depending on x S. Let x N S, N = 1, 2,..., be a sequence such lim N r(x N ) = r. Then some subsequence of x N converges to a point x S, the closure of S in R n so x = 1. Furthermore (2.8) implies (2.12) z = [T r]x is a vector with all nonnegative entries. Since the matrix T k has all strictly positive entries, it follows the vector w k = T k z has all strictly positive entries if z 0. In case (2.12) implies (2.13) r(t k x / T k x ) > r, a contradiction to the definition of r, from which we conclude z = 0. This also implies T k x is an eigenvector of T with eigenvalue r which has all strictly positive entries. We have proven (a) and (d). Next we turn to (b). Let λ C be an eigenvalue of T in which case there exists a nonzero vector x C n such Tx = λx. Letting x R n be the vector x = ( x 1,.., x n ), we see (2.14) (T x) i λ x i, i = 1,.., n, whence it follows from (2.11) λ r. Supposing now λ = r, we can argue in the same way we established x is an eigenvector of T with eigenvalue r, x is also an eigenvector of T with eigenvalue r. Thus we have (2.15) T k x = λ k x, T k x = r k x. This implies (2.16) (T k x) i = (T k x) i, 1 i n. We conclude from (2.16) using the fact all entries of T k are strictly positive (2.17) x = e iθ x, for some θ [0, 2π). Thus λ = r implies λ = r, which completes the proof of (c). Note the argument of the previous paragraph rules out the possibility of having 2 independent eigenvectors with eigenvalue r. In fact if two such existed we could construct an eigenvector x = (x 1,..., x n ) all of whose entries do not have identical phase e iθ in (2.17). This yields a contradiction. To complete the proof of (b) we need to exclude the possibility T has any generalized eigenvectors with eigenvalue r. Let us suppose a generalized eigenvector exists, whence there exists y C d such (2.18) [T r]y = x,
8 8 JOSEPH G. CONLON where x is the unique eigenvector of T with eigenvalue r and all positive entries. From what we have already proved, there also exists a unique eigenvector x of T with eigenvalue r and all positive entries. Applying x to the left side of the equation (2.18) we obtain (2.19) x [T r]y = x x > 0. Since x [T r] = 0 we have a contradiction. Corollary 2.1. Suppose a time independent Markov chain on a finite state space F is irreducible. Then it has a unique invariant distribution π( ) and π(j) > 0 for all j F. Proof. For N = 1, 2,..., let K N be the matrix (2.20) K N = 1 N N 1 n=0 T n. Irreducibility implies for large enough N the matrix K N has all strictly positive entries. Since πk N = π it follows from Theorem 2.1 π( ) is unique and all entries of π( ) are strictly positive. Definition 3. An n n matrix T has a spectral gap if there is exactly one simple root λ C of the characteristic polynomial for T which satisfies λ = σ(t) = max{ λ C : λ eigenvalue of T}. Note the PerronFrobenius theorem implies a matrix T with nonnegative entries such T k has strictly positive entries for some k 1 has a spectral gap. Proposition 2.2. Suppose the transition matrix T for a finite state Markov chain has a spectral gap. Then the associated stationary sequence (X 0, X 1,...) is strong mixing. Proof. From (3.26) of Chapter II we need to show for any bounded Borel measurable functions f, g : R m+1 R one has (2.21) lim n E[ f(x 0,..., X m )g(x n, X n+1,..., X n+m ) ] = E[ f(x 0, X 1,..., X m ) ]E[ g(x 0,..., X m ) ]. Observe now for n m the LHS of (2.21) can be written as (2.22) E[ f(x 0,..., X m ) z F T n m X m,z E[ g(x 0, X 1,..., X m ) X 0 = z ] ], where T n m y,z, y, z F, denotes the entries of the matrix T n m. Hence (2.21) will follow from (2.22) if we can show (2.23) lim k Tk y,z = π(z), y, z F. Using the notation of (2.1), (2.2), consider the matrix A = T 1π, where we are taking π( ) to be a row vector and 1 a column vector. By the spectral gap condition all eigenvalues of A have absolute value strictly less than 1, whence lim k A k = 0. Since A k = T k 1π the limit (2.23) holds.
9 MATH We have used the PerronFrobenius theorem to establish some properties of time independent Markov chains, assuming certain conditions on the transfer matrix. Another condition on the transfer matrix which can help us understand the corresponding Markov chain is so called reversibility. This condition is in effect stating the transfer matrix is selfadjoint with respect to some positive definite diagonal quadratic form. In case much more is known about the transfer matrix than in the general case, in particular all its eigenvalues are real and the matrix is diagonalizable. To see what this states about the entries (t i,j ) of the n n matrix T, suppose the quadratic form is given by n (2.24) [ξ, ζ] µ = µ(i)ξ i ζ i, ξ, ζ R n, i=1 where µ R n has all positive entries. Then the selfadjointness condition (2.25) [ξ, Tζ] µ = [Tξ, ζ] µ, is equivalent to (2.26) µ(i)t i,j = µ(j)t j,i, 1 i, j n. From (2.26) we see if we normalize the vector µ( ) so the sum of its elements is 1, then µ is the invariant measure for the Markov chain. Note for a reversible indecomposable Markov chain, if T does not have eigenvalue 1 there is a spectral gap and hence the chain is strong mixing. 3. Examples of Markov Chains We first note a trivial case of a time independent Markov chain, which is a sequence of i.i.d. random variables (X 0, X 1,..). Let p(i), i F, be the probability density function of X 0. Then the transition probabilities are p(i, j) = p(j), i, j F. If F is finite then all the rows of the matrix T are identical, so T has rank 1. In particular the pdf of X 0 is the invariant measure π and the matrix A = T 1π is the 0 matrix. Hence the sequence is strong mixing by Proposition 2.2. Another example of a time independent Markov chain is the sum S n = X X n, n = 0, 1, 2,.. of i.i.d. variables. In case p(i, j) = p(j i), where p( ) is the pdf of X 1. Thus sums of i.i.d. variables give rise to chains which are spatially translation invariant. It is therefore often useful to use the Fourier transform to analyse them. We essentially did this in our proof of the CLT in Chapter I. In the case of the Bernoulli variables X 0 = ±1 with probability 1/2, for which S n is the standard random walk on Z, we have p(i, j) = p(j, i) = 1/2 if and only if i j = 1. This chain is therefore reversible, and it is also easy to see it is irreducible. We have in Chapter 1 proved it is recurrent, but no invariant measure exists (compare Proposition 1.3). To see this note if π( ) is an invariant measure then trivially τ π( ) is also an invariant measure where τπ(i) = π(i + 1), i Z. The uniqueness theorem Proposition 1.2 implies then τπ( ) = π( ), in which case π : Z R is a constant function, contradicting the normalization condition for π( ). University of Michigan, Department of Mathematics, Ann Arbor, MI address:
Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo
Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for
More information1 Eigenvalues and Eigenvectors
Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x
More informationSection 6.1  Inner Products and Norms
Section 6.1  Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,
More informationCHAPTER IV  BROWNIAN MOTION
CHAPTER IV  BROWNIAN MOTION JOSEPH G. CONLON 1. Construction of Brownian Motion There are two ways in which the idea of a Markov chain on a discrete state space can be generalized: (1) The discrete time
More informationTheta Functions. Lukas Lewark. Seminar on Modular Forms, 31. Januar 2007
Theta Functions Lukas Lewark Seminar on Modular Forms, 31. Januar 007 Abstract Theta functions are introduced, associated to lattices or quadratic forms. Their transformation property is proven and the
More informationPresentation 3: Eigenvalues and Eigenvectors of a Matrix
Colleen Kirksey, Beth Van Schoyck, Dennis Bowers MATH 280: Problem Solving November 18, 2011 Presentation 3: Eigenvalues and Eigenvectors of a Matrix Order of Presentation: 1. Definitions of Eigenvalues
More informationBANACH AND HILBERT SPACE REVIEW
BANACH AND HILBET SPACE EVIEW CHISTOPHE HEIL These notes will briefly review some basic concepts related to the theory of Banach and Hilbert spaces. We are not trying to give a complete development, but
More informationTHE PRIME NUMBER THEOREM
THE PRIME NUMBER THEOREM NIKOLAOS PATTAKOS. introduction In number theory, this Theorem describes the asymptotic distribution of the prime numbers. The Prime Number Theorem gives a general description
More informationContinuity of the Perron Root
Linear and Multilinear Algebra http://dx.doi.org/10.1080/03081087.2014.934233 ArXiv: 1407.7564 (http://arxiv.org/abs/1407.7564) Continuity of the Perron Root Carl D. Meyer Department of Mathematics, North
More informationMATH10212 Linear Algebra. Systems of Linear Equations. Definition. An ndimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0534405967. Systems of Linear Equations Definition. An ndimensional vector is a row or a column
More informationNotes on Determinant
ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 918/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without
More informationNo: 10 04. Bilkent University. Monotonic Extension. Farhad Husseinov. Discussion Papers. Department of Economics
No: 10 04 Bilkent University Monotonic Extension Farhad Husseinov Discussion Papers Department of Economics The Discussion Papers of the Department of Economics are intended to make the initial results
More informationChapter 6. Orthogonality
6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be
More informationHOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!
Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following
More informationDuality of linear conic problems
Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least
More informationInner Product Spaces
Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and
More informationMath 550 Notes. Chapter 7. Jesse Crawford. Department of Mathematics Tarleton State University. Fall 2010
Math 550 Notes Chapter 7 Jesse Crawford Department of Mathematics Tarleton State University Fall 2010 (Tarleton State University) Math 550 Chapter 7 Fall 2010 1 / 34 Outline 1 SelfAdjoint and Normal Operators
More informationLECTURE 4. Last time: Lecture outline
LECTURE 4 Last time: Types of convergence Weak Law of Large Numbers Strong Law of Large Numbers Asymptotic Equipartition Property Lecture outline Stochastic processes Markov chains Entropy rate Random
More informationContinued Fractions and the Euclidean Algorithm
Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction
More informationLinear Algebra I. Ronald van Luijk, 2012
Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.
More information1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
More informationI. GROUPS: BASIC DEFINITIONS AND EXAMPLES
I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called
More informationsome algebra prelim solutions
some algebra prelim solutions David Morawski August 19, 2012 Problem (Spring 2008, #5). Show that f(x) = x p x + a is irreducible over F p whenever a F p is not zero. Proof. First, note that f(x) has no
More informationGROUPS SUBGROUPS. Definition 1: An operation on a set G is a function : G G G.
Definition 1: GROUPS An operation on a set G is a function : G G G. Definition 2: A group is a set G which is equipped with an operation and a special element e G, called the identity, such that (i) the
More informationSimilarity and Diagonalization. Similar Matrices
MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that
More informationQuadratic Functions, Optimization, and Quadratic Forms
Quadratic Functions, Optimization, and Quadratic Forms Robert M. Freund February, 2004 2004 Massachusetts Institute of echnology. 1 2 1 Quadratic Optimization A quadratic optimization problem is an optimization
More informationOrthogonal Diagonalization of Symmetric Matrices
MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding
More informationDiscrete Mathematics: Solutions to Homework (12%) For each of the following sets, determine whether {2} is an element of that set.
Discrete Mathematics: Solutions to Homework 2 1. (12%) For each of the following sets, determine whether {2} is an element of that set. (a) {x R x is an integer greater than 1} (b) {x R x is the square
More informationMath212a1010 Lebesgue measure.
Math212a1010 Lebesgue measure. October 19, 2010 Today s lecture will be devoted to Lebesgue measure, a creation of Henri Lebesgue, in his thesis, one of the most famous theses in the history of mathematics.
More informationT ( a i x i ) = a i T (x i ).
Chapter 2 Defn 1. (p. 65) Let V and W be vector spaces (over F ). We call a function T : V W a linear transformation form V to W if, for all x, y V and c F, we have (a) T (x + y) = T (x) + T (y) and (b)
More informationTHE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS
THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear
More informationWHICH LINEARFRACTIONAL TRANSFORMATIONS INDUCE ROTATIONS OF THE SPHERE?
WHICH LINEARFRACTIONAL TRANSFORMATIONS INDUCE ROTATIONS OF THE SPHERE? JOEL H. SHAPIRO Abstract. These notes supplement the discussion of linear fractional mappings presented in a beginning graduate course
More informationx if x 0, x if x < 0.
Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the
More informationMathematics Course 111: Algebra I Part IV: Vector Spaces
Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 19967 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are
More informationMATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.
MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α
More informationAPPLICATIONS OF THE ORDER FUNCTION
APPLICATIONS OF THE ORDER FUNCTION LECTURE NOTES: MATH 432, CSUSM, SPRING 2009. PROF. WAYNE AITKEN In this lecture we will explore several applications of order functions including formulas for GCDs and
More informationMetric Spaces. Chapter 7. 7.1. Metrics
Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some
More information1 Sets and Set Notation.
LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most
More informationLecture12 The PerronFrobenius theorem.
Lecture12 The PerronFrobenius theorem. 1 Statement of the theorem. 2 Proof of the Perron Frobenius theorem. 3 Graphology. 3 Asymptotic behavior. The nonprimitive case. 4 The Leslie model of population
More information4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION
4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION STEVEN HEILMAN Contents 1. Review 1 2. Diagonal Matrices 1 3. Eigenvectors and Eigenvalues 2 4. Characteristic Polynomial 4 5. Diagonalizability 6 6. Appendix:
More informationLecture 1: Univariate Time Series B41910: Autumn Quarter, 2008, by Mr. Ruey S. Tsay
Lecture 1: Univariate Time Series B41910: Autumn Quarter, 2008, by Mr. Ruey S. Tsay 1 Some Basic Concepts 1. Time Series: A sequence of random variables measuring certain quantity of interest over time.
More informationPerron Frobenius theory and some extensions
and some extensions Department of Mathematics University of Ioannina GREECE Como, Italy, May 2008 History Theorem (1) The dominant eigenvalue of a matrix with positive entries is positive and the corresponding
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete
More informationPOWER SETS AND RELATIONS
POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty
More informationINDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS
INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem
More informationMINIMAL GENERATOR SETS FOR FINITELY GENERATED SHIFTINVARIANT SUBSPACES OF L 2 (R n )
MINIMAL GENERATOR SETS FOR FINITELY GENERATED SHIFTINVARIANT SUBSPACES OF L 2 (R n ) MARCIN BOWNIK AND NORBERT KAIBLINGER Abstract. Let S be a shiftinvariant subspace of L 2 (R n ) defined by N generators
More informationMath 504, Fall 2013 HW 3
Math 504, Fall 013 HW 3 1. Let F = F (x) be the field of rational functions over the field of order. Show that the extension K = F(x 1/6 ) of F is equal to F( x, x 1/3 ). Show that F(x 1/3 ) is separable
More informationLecture 13: Martingales
Lecture 13: Martingales 1. Definition of a Martingale 1.1 Filtrations 1.2 Definition of a martingale and its basic properties 1.3 Sums of independent random variables and related models 1.4 Products of
More informationIRREDUCIBLE OPERATOR SEMIGROUPS SUCH THAT AB AND BA ARE PROPORTIONAL. 1. Introduction
IRREDUCIBLE OPERATOR SEMIGROUPS SUCH THAT AB AND BA ARE PROPORTIONAL R. DRNOVŠEK, T. KOŠIR Dedicated to Prof. Heydar Radjavi on the occasion of his seventieth birthday. Abstract. Let S be an irreducible
More informationMATH 110 Spring 2015 Homework 6 Solutions
MATH 110 Spring 2015 Homework 6 Solutions Section 2.6 2.6.4 Let α denote the standard basis for V = R 3. Let α = {e 1, e 2, e 3 } denote the dual basis of α for V. We would first like to show that β =
More informationSPECTRAL POLYNOMIAL ALGORITHMS FOR COMPUTING BIDIAGONAL REPRESENTATIONS FOR PHASE TYPE DISTRIBUTIONS AND MATRIXEXPONENTIAL DISTRIBUTIONS
Stochastic Models, 22:289 317, 2006 Copyright Taylor & Francis Group, LLC ISSN: 15326349 print/15324214 online DOI: 10.1080/15326340600649045 SPECTRAL POLYNOMIAL ALGORITHMS FOR COMPUTING BIDIAGONAL
More informationInner Product Spaces and Orthogonality
Inner Product Spaces and Orthogonality week 34 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,
More informationMaster s Theory Exam Spring 2006
Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem
More informationLinear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.
Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a subvector space of V[n,q]. If the subspace of V[n,q]
More information1. (First passage/hitting times/gambler s ruin problem:) Suppose that X has a discrete state space and let i be a fixed state. Let
Copyright c 2009 by Karl Sigman 1 Stopping Times 1.1 Stopping Times: Definition Given a stochastic process X = {X n : n 0}, a random time τ is a discrete random variable on the same probability space as
More informationMath 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.
Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(
More informationVector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a nonempty
More informationThe Ideal Class Group
Chapter 5 The Ideal Class Group We will use Minkowski theory, which belongs to the general area of geometry of numbers, to gain insight into the ideal class group of a number field. We have already mentioned
More informationProbability and Statistics
CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2b  0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute  Systems and Modeling GIGA  Bioinformatics ULg kristel.vansteen@ulg.ac.be
More informationREMARKS ON ABSOLUTE CONTINUITY IN THE CONTEXT OF FREE PROBABILITY AND RANDOM MATRICES arxiv: v2 [math.pr] 8 Feb 2015
REMARKS ON ABSOLUTE CONTINUITY IN THE CONTEXT OF FREE PROBABILITY AND RANDOM MATRICES arxiv:409.227v2 [math.pr] 8 Feb 205 ARIJIT CHAKRABARTY AND RAJAT SUBHRA HAZRA Abstract. In this note, we show that
More informationRi and. i=1. S i N. and. R R i
The subset R of R n is a closed rectangle if there are n nonempty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an
More information2.3 Convex Constrained Optimization Problems
42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions
More information(January 14, 2009) End k (V ) End k (V/W )
(January 14, 29) [16.1] Let p be the smallest prime dividing the order of a finite group G. Show that a subgroup H of G of index p is necessarily normal. Let G act on cosets gh of H by left multiplication.
More informationDiagonalisation. Chapter 3. Introduction. Eigenvalues and eigenvectors. Reading. Definitions
Chapter 3 Diagonalisation Eigenvalues and eigenvectors, diagonalisation of a matrix, orthogonal diagonalisation fo symmetric matrices Reading As in the previous chapter, there is no specific essential
More informationFactorization Theorems
Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization
More informationThe determinant of a skewsymmetric matrix is a square. This can be seen in small cases by direct calculation: 0 a. 12 a. a 13 a 24 a 14 a 23 a 14
4 Symplectic groups In this and the next two sections, we begin the study of the groups preserving reflexive sesquilinear forms or quadratic forms. We begin with the symplectic groups, associated with
More informationIntroduction to MonteCarlo Methods
Introduction to MonteCarlo Methods Bernard Lapeyre Halmstad January 2007 MonteCarlo methods are extensively used in financial institutions to compute European options prices to evaluate sensitivities
More informationInner products on R n, and more
Inner products on R n, and more Peyam Ryan Tabrizian Friday, April 12th, 2013 1 Introduction You might be wondering: Are there inner products on R n that are not the usual dot product x y = x 1 y 1 + +
More informationExercises with solutions (1)
Exercises with solutions (). Investigate the relationship between independence and correlation. (a) Two random variables X and Y are said to be correlated if and only if their covariance C XY is not equal
More informationMATH PROBLEMS, WITH SOLUTIONS
MATH PROBLEMS, WITH SOLUTIONS OVIDIU MUNTEANU These are free online notes that I wrote to assist students that wish to test their math skills with some problems that go beyond the usual curriculum. These
More information1. Prove that the empty set is a subset of every set.
1. Prove that the empty set is a subset of every set. Basic Topology Written by MenGen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since
More informationIdeal Class Group and Units
Chapter 4 Ideal Class Group and Units We are now interested in understanding two aspects of ring of integers of number fields: how principal they are (that is, what is the proportion of principal ideals
More informationDate: April 12, 2001. Contents
2 Lagrange Multipliers Date: April 12, 2001 Contents 2.1. Introduction to Lagrange Multipliers......... p. 2 2.2. Enhanced Fritz John Optimality Conditions...... p. 12 2.3. Informative Lagrange Multipliers...........
More informationTHE DIMENSION OF A VECTOR SPACE
THE DIMENSION OF A VECTOR SPACE KEITH CONRAD This handout is a supplementary discussion leading up to the definition of dimension and some of its basic properties. Let V be a vector space over a field
More information1. Let P be the space of all polynomials (of one real variable and with real coefficients) with the norm
Uppsala Universitet Matematiska Institutionen Andreas Strömbergsson Prov i matematik Funktionalanalys Kurs: F3B, F4Sy, NVP 0050615 Skrivtid: 9 14 Tillåtna hjälpmedel: Manuella skrivdon, Kreyszigs bok
More informationMetric Spaces Joseph Muscat 2003 (Last revised May 2009)
1 Distance J Muscat 1 Metric Spaces Joseph Muscat 2003 (Last revised May 2009) (A revised and expanded version of these notes are now published by Springer.) 1 Distance A metric space can be thought of
More informationChapter 3. Cartesian Products and Relations. 3.1 Cartesian Products
Chapter 3 Cartesian Products and Relations The material in this chapter is the first real encounter with abstraction. Relations are very general thing they are a special type of subset. After introducing
More informationExamination paper for Solutions to Matematikk 4M and 4N
Department of Mathematical Sciences Examination paper for Solutions to Matematikk 4M and 4N Academic contact during examination: Trygve K. Karper Phone: 99 63 9 5 Examination date:. mai 04 Examination
More informationRecursion Theory in Set Theory
Contemporary Mathematics Recursion Theory in Set Theory Theodore A. Slaman 1. Introduction Our goal is to convince the reader that recursion theoretic knowledge and experience can be successfully applied
More informationCONTINUED FRACTIONS AND PELL S EQUATION. Contents 1. Continued Fractions 1 2. Solution to Pell s Equation 9 References 12
CONTINUED FRACTIONS AND PELL S EQUATION SEUNG HYUN YANG Abstract. In this REU paper, I will use some important characteristics of continued fractions to give the complete set of solutions to Pell s equation.
More information1 Norms and Vector Spaces
008.10.07.01 1 Norms and Vector Spaces Suppose we have a complex vector space V. A norm is a function f : V R which satisfies (i) f(x) 0 for all x V (ii) f(x + y) f(x) + f(y) for all x,y V (iii) f(λx)
More informationClassification of Cartan matrices
Chapter 7 Classification of Cartan matrices In this chapter we describe a classification of generalised Cartan matrices This classification can be compared as the rough classification of varieties in terms
More informationMath 4310 Handout  Quotient Vector Spaces
Math 4310 Handout  Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable
More informationn k=1 k=0 1/k! = e. Example 6.4. The series 1/k 2 converges in R. Indeed, if s n = n then k=1 1/k, then s 2n s n = 1 n + 1 +...
6 Series We call a normed space (X, ) a Banach space provided that every Cauchy sequence (x n ) in X converges. For example, R with the norm = is an example of Banach space. Now let (x n ) be a sequence
More information6.4 The Infinitely Many Alleles Model
6.4. THE INFINITELY MANY ALLELES MODEL 93 NE µ [F (X N 1 ) F (µ)] = + 1 i
More informationLecture 3: Linear Programming Relaxations and Rounding
Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can
More informationMathematical Methods of Engineering Analysis
Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................
More informationChapter 1. Metric Spaces. Metric Spaces. Examples. Normed linear spaces
Chapter 1. Metric Spaces Metric Spaces MA222 David Preiss d.preiss@warwick.ac.uk Warwick University, Spring 2008/2009 Definitions. A metric on a set M is a function d : M M R such that for all x, y, z
More informationMA651 Topology. Lecture 6. Separation Axioms.
MA651 Topology. Lecture 6. Separation Axioms. This text is based on the following books: Fundamental concepts of topology by Peter O Neil Elements of Mathematics: General Topology by Nicolas Bourbaki Counterexamples
More informationPART I. THE REAL NUMBERS
PART I. THE REAL NUMBERS This material assumes that you are already familiar with the real number system and the representation of the real numbers as points on the real line. I.1. THE NATURAL NUMBERS
More informationMA106 Linear Algebra lecture notes
MA106 Linear Algebra lecture notes Lecturers: Martin Bright and Daan Krammer Warwick, January 2011 Contents 1 Number systems and fields 3 1.1 Axioms for number systems......................... 3 2 Vector
More informationIntroduction to Topology
Introduction to Topology Tomoo Matsumura November 30, 2010 Contents 1 Topological spaces 3 1.1 Basis of a Topology......................................... 3 1.2 Comparing Topologies.......................................
More informationby the matrix A results in a vector which is a reflection of the given
Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the yaxis We observe that
More informationMathematical Background
Appendix A Mathematical Background A.1 Joint, Marginal and Conditional Probability Let the n (discrete or continuous) random variables y 1,..., y n have a joint joint probability probability p(y 1,...,
More information3. INNER PRODUCT SPACES
. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.
More informationAdaptive Search with Stochastic Acceptance Probabilities for Global Optimization
Adaptive Search with Stochastic Acceptance Probabilities for Global Optimization Archis Ghate a and Robert L. Smith b a Industrial Engineering, University of Washington, Box 352650, Seattle, Washington,
More informationQuotient Rings and Field Extensions
Chapter 5 Quotient Rings and Field Extensions In this chapter we describe a method for producing field extension of a given field. If F is a field, then a field extension is a field K that contains F.
More information1 Review of complex numbers
1 Review of complex numbers 1.1 Complex numbers: algebra The set C of complex numbers is formed by adding a square root i of 1 to the set of real numbers: i = 1. Every complex number can be written uniquely
More information1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each)
Math 33 AH : Solution to the Final Exam Honors Linear Algebra and Applications 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each) (1) If A is an invertible
More informationSufficient Statistics and Exponential Family. 1 Statistics and Sufficient Statistics. Math 541: Statistical Theory II. Lecturer: Songfeng Zheng
Math 541: Statistical Theory II Lecturer: Songfeng Zheng Sufficient Statistics and Exponential Family 1 Statistics and Sufficient Statistics Suppose we have a random sample X 1,, X n taken from a distribution
More information