# Markov Chains, Stochastic Processes, and Advanced Matrix Decomposition

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Markov Chains, Stochastic Processes, and Advanced Matrix Decomposition Jack Gilbert Copyright (c) 2014 Jack Gilbert. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled GNU Free Documentation License.

2 2 Introduction This paper seeks to explore, in some detail, the basics of stochastic processes and their extensive relationship with Markov chains. This is primarily a study of the field of statistics. However, by broadening our scope to consider techniques outside the realm of statistics, we can utilize advanced techniques from linear algebra (and some basic ones, too!), thus broadening and enhancing our understanding of these topics. Probability Spaces We proceed with our study of Markov chains by defining some of the basics of probability spaces. Understanding these basic definitions is crucial to understanding the more complicated ideas presented later in the paper, especially in our observations of matrix decompositions. Lets begin. Definition 1. An event is a subset of a sample space. An event A is said to occur if and only if the observed outcome ω A. Definition 2. Given a sample space Ω and an event A, the events complement A c is the event which occurs if and only if A does not occur. A c = {ω Ω ω / A}. Definition 3. If Ω is a sample space and if P is a function which associates a number for each event in Ω, then P is called the probability measure provided that a. for any event A, 0 P (A) 1 b. P (Ω) = 1 c. For any sequence A 1, A 2,... of disjoint events, P ( i=1 A i) = P (A i ). i=1 Random Variables and Expected Outcomes We will now shift our focus to the study of random variables, a key component of the study of stochastic processes. Definition 4. A random variable X with values in the set E is a function which assigns a value X(ω) E to each outcome ω Ω. When E is finite, X is said to be a discrete random variable. Definition 5. The discrete random variables X 1,..., X n are said to be independent if P {X 1 = a 1,..., X n = a n } = P {X 1 = a 1 } P {X n = a n } for every a 1,..., a n E. Expected values are the potential outcomes for each random variable. value for a random variable X taking values in the set E R + is The expected

3 3 E[X] = a E ap {X = a}. With these definitions in place, we can proceed to define Markov chains, which are the primary focus of this paper. Markov Chains We now begin our study of Markov chains. Definition 6. The stochastic process X = {X n ; n N} is called a Markov chain if P {X n+1 = j X 0,..., X n } = P {X n+1 = j X n } for every j E, n N. So a Markov chain is a sequence of random variables such that for any n, X n+1 is conditionally independent of X 0,..., X n 1 given X n. We use P {X n+1 = j X n = i} = P (i, j) where i, j E is independent of n. The probabilities P (i, j) are called the transition probabilities for the Markov chain X. The Markov Chain is said to be time homogenous. We can arrange the P (i, j) into a square matrix P called the transition matrix of the Markov chain X. For example, for the set E = {1, 2,... } the transition matrix is P (0, 0) P (0, 1) P (0, 2) P = P (1, 0) P (1, 1) P (1, 2) (1)... The basic notation we will use with regard to these matrices is as follows: M(i, j) refers to the entry in row i, column j of matrix M. Column vectors are represented by letters, i.e. f(i) is column fs i-th entry. Row vectors are greek letters, i.e. π(j) is row πs j-th entry. Definition 7. Let P be a square matrix with entries P (i, j) where i, j E. P is called a Markov matrix over E if a. For every i, j E, P (i, j) 0 b. For every i E, j E P (i, j) = 1. We can now present our first theorem.

4 4 Theorem 1. For every n, m N with m 1 and i 0,..., i m E, P {X n+i = i,..., X n+m = i m X n = i 0 } = P (i 0, i 1 )P (i 1, i 2 ) P (i m 1, i m ). The following corollary provides more insight into this theorem. Corollary 1. Let π be a probability distribution on E. Suppose P {X 0 = i} = π(i) for every i E. Then for every m N and i 0,..., i m E, P {X 0 = i 0, X 1 = i 1,..., X m = i m } = π(i 0 )P (i 0, i 1 ) P (i m 1, i m ). Theorem 1 and Corollary 1 show that the joint distribution of X 0,..., X m is completely specified for every m once the initial distribution π and transition matrix P are known. We can get the joint distribution of X n1,..., X nk for any integer k 1 and n 1,..., n k N. This leads us to the following proposition. For any m N, P {X n+m = j X n = i} = P m (i, j) for every i, j E and n N. In other words, the probability that the chain moves from state i to that j in m steps is the (i, j) entry of the n-th power of the transition matrix P. Thus for any m, n N, which in turn becomes P m+n = P m P n P m+n (i, j) = k E P m (i, k)p n (k, j); i, j E. This is called the Chapman-Kolmogorov equation. This states that when starting at state i, in order for process X to be in state j after m + n steps, it must be in some intermediate state k after the m-th step and then move from state k into state j in the remaining n steps. This is illustrated in the following example. Example 1. Let X = {X n ; n N} be a Markov chain with state space E = {a, b, c} and transition matrix Then P = (2) P {X 1 = b, X 2 = c, X 3 = a, X 4 = c, X 5 = a, X 6 = c, X 7 = b X 0 = c} (3) = P (c, b)p (b, c)p (c, a)p (a, c)p (c, a)p (a, c)p (c, b) (4) = = (5) (6)

5 The two-step transition probabilities are given by P = where in this case P {X n+2 = c X n = b} = P 2 (b, c) = 1 6. (7) 5 State Spaces With an understanding of the Chapman-Kolmogorov equation as the basis of our study of Markov chains and Markov matrices we can move on to our classification of the various states we will encounter throughout this paper. We will define these states now. Definition 8. Given a Markov chain X, a state space E, a transition matrix P, let T be the time of the first visit to state j and let N j be the total visits to state j. Then a. j is recurrent if P j {T < } = 1. Otherwise, j is transient if P j {T = + } > 0 b. A recurrent state j is null if E j [T = ; otherwise j is non-null c. A recurrent state j is periodic with period δ if δ 2 is the largest integer for P j {T = nδ for some n 1} = 1. So if j is recurrent, F (j, j) = 1 so the probability of returning to state j is 1. If j is transient, F (j, j) < 1 and there is a positive probability of never returning to state j. This leads us to our next theorem. Theorem 2. If j is transient or recurrent null, then for every i E, lim n P n (i, j) = 0. If j is recurrent non-null aperiodic, then π(j) = lim n P n (j, j) > 0 and for every i E, lim n P n (i, j) = F (i, j)π(j). So if j is periodic with period δ then returning to j is only possible on steps δ, 2δ, 3δ,... and the same is true of each successive return to j. Hence P n (j, j) = P j {X n = j} > 0only ifn {0, δ, 2δ,... }. In order to tell whether P n (j, j) > 0 or not, we do not need to calculate P n (j, j). Instead we say j can be reached from i, i j, if there exists an integer n 0 such that P n (i, j) > 0. Thus for i j, i j if and only if F (i, j) > 0. In order for i j there must be a sequence i 1, i 2,..., i n of states such that P (i, i 1 ) > 0, P (i 1, i 2 ) > 0,..., P (i n, j) > 0. Otherwise P n (i, j) = 0 for every n and thus it is impossible for i j. Definition 9. a. A set of states is closed if no state outside the set can be reached from within the set

6 6 b. A state forming a closed set by itself is called an absorbing state c. A closed set is irreducible if no proper subset of it is closed d. A Markov chain is irreducible if its only closed set is the set of all states. Example 2. Consider the Markov chain with state space E = {a, b, c, d, e} and transition matrix P = (8) The closed sets are {a, b, c, d, e} and {a, c, e}. Since there exist two closed sets, the chain is not irreducible. By deleting the second and fourth rows and column we end up with the matrix Q = (9) which is the Markov matrix corresponding to the restriction of X to the closed set {a, c, e}. We can rearrange P for easier analysis as such: P* = Recurrence and Irreducible Markov Chains We have a lemma which shows that if j is recurrent and j k, then k j and F (k, j) = 1. This should be intuitive based on our definition of a recurrent state: we know since j is recurrent, F (j, j) = 1, so if j k then it follows that k i must at some point return to j, i j. This notion of recurrence gives us the basis of the following theorem. Theorem 3. From a recurrent state, only recurrent states can be reached. From this theorem we know that no transient state can be reached from any recurrent state and thus the set of all recurrent states is closed. This leads us to one of the main results of Markov matrices. First, a lemma, and then our main theorem: Lemma 1. For each recurrent state j there exists an irreducible closed set C which includes j.

7 Proof. Let j be a recurrent state and let C be the set of all states which can be reached from j. Then C is a closed set. We show if i, k C then i k: If i C then j i. Since j is recurrent, our previous lemma implies that i j. There must be some state k such that j k, and thus i k. We will use this lemma in formulating the following theorem. Theorem 4. In a Markov chain, the recurrent states can be divided uniquely into irreducible closed sets C 1, C 2,... The proof follows directly from the lemma preceding it. In addition to C = C 1 C 2... of recurrent states, the chain includes transient states as well (since recurrent states can be reached via transient states). Using this theorem we can arrange our transition matrix in the following form: P P P = 0 0 P 3 0 (10)... vdots. Q 1 Q 2 Q 3 Q where P 1, P 2,... are the Markov matrices corresponding to sets C 1, C 2,... of states. Each one of these sets is an irreducible Markov chain. The following theorem ties this all together. Theorem 5. Let X be an irreducible Markov chain. Then either all states are transient, or all are recurrent null, or all are recurrent non-null. Either all states are aperiodic, or else all are periodic with the same period δ. The following corollaries conclude this section of the paper. Corollary 2. Let C be an irreducible closed set with finitely many states. Then no state in C is recurrent null. Corollary 3. If C is an irreducible closed set with finitely many states, then C has no transient states. In the case with finitely many states we now have all the tools necessary to classify the states! We first identify the irreducible closed sets, then use our theorem and corollaries to conclude that all states belonging to an irreducible closed set are recurrent non-null; the remaining states, if any, are transient; and periodicity is determined by again applying our theorem. 7 Markov Chains and Linear Algebra How does this study of Markov chains and transition matrices relate to linear algebra? First, a theorem to give you some of the flavor:

8 8 Theorem 6. Let X be an irreducible Markov chain. Consider the system of linear equations #» v (j) = #» v (i)p (i, j), j E. i E Then all states are recurrent non-null if and only if there exists a solution #» v with #» v (j) = 1. j E If there is a solution #» v then #» v (j) > 0 for every j E, and #» v is unique. We now fully engage with linear applications of Markov chains and stochastic matrices. The Perron-Frobenius Theorem We can use the general case of a theorem proved by a mathematician named Perron in 1907 and extended by Frobenius in 1912 to obtain a corollary with some interesting properties pertaining to our understanding of Markov chains. The proof of this theorem has been omitted, as it favors an analysis approach which would not be suited for this paper; however, its corollary will be crucial to our discussion. Theorem 7. Perron-Frobenius part 1: Let A be a square matrix of size n with non-negative entries. Then 1. A has a positive eigenvalue λ 0 with left eigenvector x #» 0 such that x #» 0 is non-negative and non-zero 2. If λ is any other eigenvalue of A, λ λ 0 3. If λ is an eigenvalue of A and λ = λ 0, then µ = λ λ 0 is a root of unity and µ k λ 0 is an eigenvalue of A for k = 0, 1, 2,... Theorem 8. Perron-Frobenius part 2: Let A be a square matrix of size n with non-negative entries such that A m has all positive entries for some m. Then 1. A has a positive eigenvalue λ 0 with a corresponding left eigenvector x #» 0 where the entries of x #» 0 are positive 2. If λ is any other eigenvalue of A, λ < λ 0 3. λ 0 has multiplicity 1. We will now give the corollary related to the Perron-Frobenius theorems. Corollary 4. Let P be an irreducible Markov matrix. Then 1 is a simple eigenvalue of P. For any other eigenvalue λ of P we have λ 1. If P is aperiodic then λ < 1 for all other eigenvalues of P. If P is periodic with period δ then there are δ eigenvalues with an absolute value equal to 1. These are all distinct and are λ 1 = 1, λ 2 = c,..., λ δ = c δ 1 ; c = e 2πi/δ.

9 Here well use an example which will reveal some of the more interesting results of these theorems and their corollary. 9 Example P = P is irreducible and periodic with period δ = 2. Then P 2 = (11) (12) The eigenvalues of the 2 by 2 matrix in the top-left corner of P 2 are 1 and This is sufficient enough to compute all eigenvalues of P by using the Perron-Frobenius theorems, which imply that the set of eigenvalues of a periodic irreducible matrix P, when regarded as a system of points in the complex plane, goes over into itself under a rotation of the plane by the angle 2π/δ. Wow! Since 1, 0.22 are eigenvalues of P, their square roots will be eigenvalues for P : 1, 1, i 0.22, i The final eigenvalue must go into itself by a rotation of 180 degrees and thus must be 0. It is worth taking the time to pause and consider an idea that has been implicit in several of these theorems. We will now take the time to make this explicit: Theorem 9. All finite stochastic matrices P have 1 as an eigenvalue and there exist nonnegative eigenvectors corresponding to λ = 1. We will use this theorem throughout the remainder of the paper, so it is worth proving. Proof. Since each row of P sums to 1, #» y is a right eigenvector. We try to find a left eigenvector with non-negative components that sum to 1: Since all finite chains have at least one positive persistent state, we know there exists a closed irreducible subset S and the Markov chain associated with S is irreducible positive persistent. We know for S there exists an invariant probability vector. Assume P is a square matrix of size n and rewrite P in block form with [ ] P1 0 P = R Q where P 1 is the probability transition matrix corresponding to S. Let #» π = (π 1, π 2,..., π k ) be the invariant probability vector for P 1. Define #» γ = (π 1, π 2,..., π k, 0, 0,..., 0) and note #» γ P = #» γ. Hence #» γ is a left eigenvector for P corresponding to λ = 1. Additionally, (13)

10 10 n γ i = 1. i=1 This theorem shows that λ = 1 is the largest possible eigenvalue for a finite stochastic matrix P. This fact is related to the irreducibility of the Markov chain determined by P, which we will show in the next theorem. Theorem 10. If P is the transition matrix for a finite Markov chain, then the multiplicity of the eigenvalue 1 is equal to the number of irreducible subsets of the chain. Proof. The first half of this proof goes as follows. Arrange P in block form based on the irreducible subsets of the chain C 1, C 2,... as before. P P P = P m. R 1 R 2 R m Q Each P k corresponds to a subset C k for an irreducible positive persistent chain. The #» x i s for each C i are linearly independent; thus the multiplicity for λ = 1 is at least equal to the number of subsets C. (14) Advanced Matrix Decomposition and Markov Chains The last topic we will examine in this paper deals with matrix decompositions and Markov chains. We will start by looking at infinite stochastic matrices. We can apply techniques of linear decomposition to infinite stochastic matrices to characterize matrices with some interesting stochastic properties. We will begin with a theorem. Theorem 11. Let {X t ; t = 0, 1, 2,... } be a Markov chain with state {0, 1, 2,... } and a transition matrix P. Let P be irreducible and consist of persistent positive recurrent states. Then I P = (A I)(B S) j; where A is strictly upper triangular with a ij = E i [number of times X = j before X reaches j i ], i < B is strictly lower triangular with b ij = P i {X1 i = j}, i > j; i 1 and S is diagonal where s j = b ij (and s 0 = 0). Moreover, a ij < and i < j. j=0

11 This primary result can then be used to find the stationary distribution, the matrix of mean first-passage times, and the fundamental matrix for discrete-time Markov chains. How nice! The final result we will examine takes the previous theorem and extends it to for nullrecurrent and transient cases. Theorem 12. Define E D to be the diagonal matrix with entries e i on the diagonal where e i = P i (X n / {0, 1, 2,... }) for n 1. Then for every infinite irreducible stochastic matrix P with A, B and entries a ij, b ij respectively, I P = (A I)(B S) + E D 2. P is recurrent if and only if I P = (A I)(B S), or in other words, if E D = 0. Spectral Representations Suppose we have a diagonalizable matrix A. Define B k to be the matrix obtained by multiplying the column vector f k with the row vector π k where f k, π k are from A. Explicitly, B k = f k π k. Then we can represent A in the following manner: A = λ 1 B 1 + λ 2 B λ n B n. This is the spectral representation of A, and it holds some key properties relevant to our discussion of Markov chains. For example, for the k-th power of A we have A k = λ k 1B λ k nb n. This provides the means for computing the k-step transition matrix of A. This may then be used to obtain the limits of A k as k, along with estimates of the rate of convergence and bounds for error involved in certain calculations. An example will highlight some of these properties: Example 4. Let P = [ 0.8 ] We get from P 1 = 1 that λ 1 = 1 and f 1 = 1 is its eigenvector. Since the trace of P is 1.5 and the sum of the eigenvalues of P equal the trace of P, λ 2 must be 0.5. We can now calculate B 1 : [ ] B 1 = f 1 π 1 = (16) and since P 0 = I = B 1 + B 2 for k = 0, (15)

12 12 B 2 = [ 0.4 ] Thus the spectral representation for P k is [ ] [ ] P k = + (0.5) k, k = 0, 1,... (18) The limit as k has (0.5) k approach zero, so [ ] P = lim P k = k Using spectral representations and given certain conditions we can relatively easily calculate powers of P. This is done via a process quite similar to diagonalization. We begin by first applying a theorem which holds for general finite matrices, and then we apply our result to stochastic matrices. Theorem 13. Let A be a square matrix of size n with n linearly independent eigenvectors x #» 1, x #» 2,..., x #» n and associated eigenvalues λ 1, λ 2,..., λ n. Define the matrices L, D by #» x 1 x #» 2 L =. #» and λ λ D =.... (20) λ n Then A = L 1 DL. This can be applied for size n stochastic matrices. If P contains a set n of linearly independent left eigenvectors then P k can be determined from L 1 DL. Conclusion Markov chains and their affiliated transition matrices have a wide variety of application both within the field of math and in real-world application. By understanding how linear algebra interacts with Markov chains, one gains a deeper intuition as to how stochastic matrices can be manipulated while still retaining their statistical properties in such a way that proves useful. x n (17) (19)

13 Bibliography [1] Robert Beezer. A Second Course In Linear Algebra. Open-License, [2] Erhan Cinlar Introduction to Stochastic Processes. Prentice-Hall, Inc., first edition, [3] Daniel Heyman A Decomposition Theorem for Infinite Stochastic Matrices. J. Appl. Prob. 32, , [4] Yiqiang Zhao, Wei Li, W. John Braun On a Decomposition for Infinite Transition Matrices [5] Dean L. Isaacson, Richard W. Madsen Markov Chains: Theory and Applications. John Wiley Sons, first edition,

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### 1 Eigenvalues and Eigenvectors

Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Presentation 3: Eigenvalues and Eigenvectors of a Matrix

Colleen Kirksey, Beth Van Schoyck, Dennis Bowers MATH 280: Problem Solving November 18, 2011 Presentation 3: Eigenvalues and Eigenvectors of a Matrix Order of Presentation: 1. Definitions of Eigenvalues

### Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### Continuity of the Perron Root

Linear and Multilinear Algebra http://dx.doi.org/10.1080/03081087.2014.934233 ArXiv: 1407.7564 (http://arxiv.org/abs/1407.7564) Continuity of the Perron Root Carl D. Meyer Department of Mathematics, North

### Notes on Orthogonal and Symmetric Matrices MENU, Winter 2013

Notes on Orthogonal and Symmetric Matrices MENU, Winter 201 These notes summarize the main properties and uses of orthogonal and symmetric matrices. We covered quite a bit of material regarding these topics,

### Chapter 6. Orthogonality

6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

### Vector Spaces II: Finite Dimensional Linear Algebra 1

John Nachbar September 2, 2014 Vector Spaces II: Finite Dimensional Linear Algebra 1 1 Definitions and Basic Theorems. For basic properties and notation for R N, see the notes Vector Spaces I. Definition

### Definition: A square matrix A is block diagonal if A has the form A 1 O O O A 2 O A =

The question we want to answer now is the following: If A is not similar to a diagonal matrix, then what is the simplest matrix that A is similar to? Before we can provide the answer, we will have to introduce

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

### 1. (First passage/hitting times/gambler s ruin problem:) Suppose that X has a discrete state space and let i be a fixed state. Let

Copyright c 2009 by Karl Sigman 1 Stopping Times 1.1 Stopping Times: Definition Given a stochastic process X = {X n : n 0}, a random time τ is a discrete random variable on the same probability space as

### The Moore-Penrose Inverse and Least Squares

University of Puget Sound MATH 420: Advanced Topics in Linear Algebra The Moore-Penrose Inverse and Least Squares April 16, 2014 Creative Commons License c 2014 Permission is granted to others to copy,

### Orthogonal Diagonalization of Symmetric Matrices

MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

The Hadamard Product Elizabeth Million April 12, 2007 1 Introduction and Basic Results As inexperienced mathematicians we may have once thought that the natural definition for matrix multiplication would

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.

Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry

### Diagonalisation. Chapter 3. Introduction. Eigenvalues and eigenvectors. Reading. Definitions

Chapter 3 Diagonalisation Eigenvalues and eigenvectors, diagonalisation of a matrix, orthogonal diagonalisation fo symmetric matrices Reading As in the previous chapter, there is no specific essential

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### 7 Communication Classes

this version: 26 February 2009 7 Communication Classes Perhaps surprisingly, we can learn much about the long-run behavior of a Markov chain merely from the zero pattern of its transition matrix. In the

### HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!

Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following

### 9.2 Summation Notation

9. Summation Notation 66 9. Summation Notation In the previous section, we introduced sequences and now we shall present notation and theorems concerning the sum of terms of a sequence. We begin with a

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Lecture 1: Schur s Unitary Triangularization Theorem

Lecture 1: Schur s Unitary Triangularization Theorem This lecture introduces the notion of unitary equivalence and presents Schur s theorem and some of its consequences It roughly corresponds to Sections

### Lecture12 The Perron-Frobenius theorem.

Lecture12 The Perron-Frobenius theorem. 1 Statement of the theorem. 2 Proof of the Perron Frobenius theorem. 3 Graphology. 3 Asymptotic behavior. The non-primitive case. 4 The Leslie model of population

### LS.6 Solution Matrices

LS.6 Solution Matrices In the literature, solutions to linear systems often are expressed using square matrices rather than vectors. You need to get used to the terminology. As before, we state the definitions

### SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH

31 Kragujevac J. Math. 25 (2003) 31 49. SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH Kinkar Ch. Das Department of Mathematics, Indian Institute of Technology, Kharagpur 721302, W.B.,

### 4.5 Linear Dependence and Linear Independence

4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### Similar matrices and Jordan form

Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive

### We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.

Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to

### SYSTEMS OF EQUATIONS

SYSTEMS OF EQUATIONS 1. Examples of systems of equations Here are some examples of systems of equations. Each system has a number of equations and a number (not necessarily the same) of variables for which

### 2.5 Elementary Row Operations and the Determinant

2.5 Elementary Row Operations and the Determinant Recall: Let A be a 2 2 matrtix : A = a b. The determinant of A, denoted by det(a) c d or A, is the number ad bc. So for example if A = 2 4, det(a) = 2(5)

### Section 6.1 - Inner Products and Norms

Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,

### The Second Eigenvalue of the Google Matrix

0 2 The Second Eigenvalue of the Google Matrix Taher H Haveliwala and Sepandar D Kamvar Stanford University taherh,sdkamvar @csstanfordedu Abstract We determine analytically the modulus of the second eigenvalue

### Inner Product Spaces

Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

### MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set.

MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. Vector space A vector space is a set V equipped with two operations, addition V V (x,y) x + y V and scalar

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.

Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### 4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION

4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION STEVEN HEILMAN Contents 1. Review 1 2. Diagonal Matrices 1 3. Eigenvectors and Eigenvalues 2 4. Characteristic Polynomial 4 5. Diagonalizability 6 6. Appendix:

### IRREDUCIBLE OPERATOR SEMIGROUPS SUCH THAT AB AND BA ARE PROPORTIONAL. 1. Introduction

IRREDUCIBLE OPERATOR SEMIGROUPS SUCH THAT AB AND BA ARE PROPORTIONAL R. DRNOVŠEK, T. KOŠIR Dedicated to Prof. Heydar Radjavi on the occasion of his seventieth birthday. Abstract. Let S be an irreducible

### Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### 1 Sets and Set Notation.

LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

### BASIC THEORY AND APPLICATIONS OF THE JORDAN CANONICAL FORM

BASIC THEORY AND APPLICATIONS OF THE JORDAN CANONICAL FORM JORDAN BELL Abstract. This paper gives a basic introduction to the Jordan canonical form and its applications. It looks at the Jordan canonical

### MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

### Introduction to Matrix Algebra

Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### WHICH LINEAR-FRACTIONAL TRANSFORMATIONS INDUCE ROTATIONS OF THE SPHERE?

WHICH LINEAR-FRACTIONAL TRANSFORMATIONS INDUCE ROTATIONS OF THE SPHERE? JOEL H. SHAPIRO Abstract. These notes supplement the discussion of linear fractional mappings presented in a beginning graduate course

### Bonus-malus systems and Markov chains

Bonus-malus systems and Markov chains Dutch car insurance bonus-malus system class % increase new class after # claims 0 1 2 >3 14 30 14 9 5 1 13 32.5 14 8 4 1 12 35 13 8 4 1 11 37.5 12 7 3 1 10 40 11

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Solving Linear Systems, Continued and The Inverse of a Matrix

, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

### Math 4310 Handout - Quotient Vector Spaces

Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### LECTURE 4. Last time: Lecture outline

LECTURE 4 Last time: Types of convergence Weak Law of Large Numbers Strong Law of Large Numbers Asymptotic Equipartition Property Lecture outline Stochastic processes Markov chains Entropy rate Random

### Solution to Homework 2

Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### Perron Frobenius theory and some extensions

and some extensions Department of Mathematics University of Ioannina GREECE Como, Italy, May 2008 History Theorem (1) The dominant eigenvalue of a matrix with positive entries is positive and the corresponding

### The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23

(copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, high-dimensional

### NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

### Review Jeopardy. Blue vs. Orange. Review Jeopardy

Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round \$200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components

Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they

### Solving Systems of Linear Equations Using Matrices

Solving Systems of Linear Equations Using Matrices What is a Matrix? A matrix is a compact grid or array of numbers. It can be created from a system of equations and used to solve the system of equations.

### Helpsheet. Giblin Eunson Library MATRIX ALGEBRA. library.unimelb.edu.au/libraries/bee. Use this sheet to help you:

Helpsheet Giblin Eunson Library ATRIX ALGEBRA Use this sheet to help you: Understand the basic concepts and definitions of matrix algebra Express a set of linear equations in matrix notation Evaluate determinants

### Section 2.1. Section 2.2. Exercise 6: We have to compute the product AB in two ways, where , B =. 2 1 3 5 A =

Section 2.1 Exercise 6: We have to compute the product AB in two ways, where 4 2 A = 3 0 1 3, B =. 2 1 3 5 Solution 1. Let b 1 = (1, 2) and b 2 = (3, 1) be the columns of B. Then Ab 1 = (0, 3, 13) and

### Linear Algebra Notes

Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note

### Matrix Algebra and Applications

Matrix Algebra and Applications Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Matrix Algebra and Applications 1 / 49 EC2040 Topic 2 - Matrices and Matrix Algebra Reading 1 Chapters

### State of Stress at Point

State of Stress at Point Einstein Notation The basic idea of Einstein notation is that a covector and a vector can form a scalar: This is typically written as an explicit sum: According to this convention,

### We seek a factorization of a square matrix A into the product of two matrices which yields an

LU Decompositions We seek a factorization of a square matrix A into the product of two matrices which yields an efficient method for solving the system where A is the coefficient matrix, x is our variable

### 3. INNER PRODUCT SPACES

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

### Matrices and Polynomials

APPENDIX 9 Matrices and Polynomials he Multiplication of Polynomials Let α(z) =α 0 +α 1 z+α 2 z 2 + α p z p and y(z) =y 0 +y 1 z+y 2 z 2 + y n z n be two polynomials of degrees p and n respectively. hen,

### Some Notes on Taylor Polynomials and Taylor Series

Some Notes on Taylor Polynomials and Taylor Series Mark MacLean October 3, 27 UBC s courses MATH /8 and MATH introduce students to the ideas of Taylor polynomials and Taylor series in a fairly limited

### 7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method

### Solving Linear Diophantine Matrix Equations Using the Smith Normal Form (More or Less)

Solving Linear Diophantine Matrix Equations Using the Smith Normal Form (More or Less) Raymond N. Greenwell 1 and Stanley Kertzner 2 1 Department of Mathematics, Hofstra University, Hempstead, NY 11549

### UNCOUPLING THE PERRON EIGENVECTOR PROBLEM

UNCOUPLING THE PERRON EIGENVECTOR PROBLEM Carl D Meyer INTRODUCTION Foranonnegative irreducible matrix m m with spectral radius ρ,afundamental problem concerns the determination of the unique normalized

### APPLICATIONS OF THE ORDER FUNCTION

APPLICATIONS OF THE ORDER FUNCTION LECTURE NOTES: MATH 432, CSUSM, SPRING 2009. PROF. WAYNE AITKEN In this lecture we will explore several applications of order functions including formulas for GCDs and

### SPECTRAL POLYNOMIAL ALGORITHMS FOR COMPUTING BI-DIAGONAL REPRESENTATIONS FOR PHASE TYPE DISTRIBUTIONS AND MATRIX-EXPONENTIAL DISTRIBUTIONS

Stochastic Models, 22:289 317, 2006 Copyright Taylor & Francis Group, LLC ISSN: 1532-6349 print/1532-4214 online DOI: 10.1080/15326340600649045 SPECTRAL POLYNOMIAL ALGORITHMS FOR COMPUTING BI-DIAGONAL

### (January 14, 2009) End k (V ) End k (V/W )

(January 14, 29) [16.1] Let p be the smallest prime dividing the order of a finite group G. Show that a subgroup H of G of index p is necessarily normal. Let G act on cosets gh of H by left multiplication.

### MATH 551 - APPLIED MATRIX THEORY

MATH 55 - APPLIED MATRIX THEORY FINAL TEST: SAMPLE with SOLUTIONS (25 points NAME: PROBLEM (3 points A web of 5 pages is described by a directed graph whose matrix is given by A Do the following ( points

### Iterative Methods for Solving Linear Systems

Chapter 5 Iterative Methods for Solving Linear Systems 5.1 Convergence of Sequences of Vectors and Matrices In Chapter 2 we have discussed some of the main methods for solving systems of linear equations.

### Suk-Geun Hwang and Jin-Woo Park

Bull. Korean Math. Soc. 43 (2006), No. 3, pp. 471 478 A NOTE ON PARTIAL SIGN-SOLVABILITY Suk-Geun Hwang and Jin-Woo Park Abstract. In this paper we prove that if Ax = b is a partial signsolvable linear

### EXAMENSARBETEN I MATEMATIK

EXAMENSARBETEN I MATEMATIK MATEMATISKA INSTITUTIONEN, STOCKHOLMS UNIVERSITET Some observations on Fatou sets of rational functions av Per Alexanderson 2007 - No 13 MATEMATISKA INSTITUTIONEN, STOCKHOLMS

### Factorization Theorems

Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

### Matrix Representations of Linear Transformations and Changes of Coordinates

Matrix Representations of Linear Transformations and Changes of Coordinates 01 Subspaces and Bases 011 Definitions A subspace V of R n is a subset of R n that contains the zero element and is closed under

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### CONTROLLABILITY. Chapter 2. 2.1 Reachable Set and Controllability. Suppose we have a linear system described by the state equation

Chapter 2 CONTROLLABILITY 2 Reachable Set and Controllability Suppose we have a linear system described by the state equation ẋ Ax + Bu (2) x() x Consider the following problem For a given vector x in

### Spectral Measure of Large Random Toeplitz Matrices

Spectral Measure of Large Random Toeplitz Matrices Yongwhan Lim June 5, 2012 Definition (Toepliz Matrix) The symmetric Toeplitz matrix is defined to be [X i j ] where 1 i, j n; that is, X 0 X 1 X 2 X n

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables