Markov Chains and Applications
|
|
- Tyrone Simon
- 7 years ago
- Views:
Transcription
1 Markov Chains and Applications Alexander Volfovsky August 7, 2007 Abstract In this paper I provide a quick overview of Stochastic processes and then quickly delve into a discussion of Markov Chains. There is some assumed knowledge of basic calculus, probability, and matrix theory. I build up Markov Chain theory towards a limit theorem. I prove the Fundamental Theorem of Markov Chains relating the stationary distribution to the limiting distribution. I then employ this limiting theorem in a Markov Chain Monte Carlo example. Contents Introduction 2 2 Markov Chains 2 2. Theorems and lemmas Applications Markov Chain Monte Carlo 7 3. Statement Solution Further Discussion Appendix I would like to thank Prof. Lawler for all of his incites into stochastic processes during his course on random walks. Also, I would like to thank Sam Raskin for inspirational conversations when the paper seemed to be taking unfortunate turns for the worse. The book that is used in order to nd statements of some of the theorems is the reference guide by Larry Wasserman, All of Statistics: A Concise Course in Statistical Inference. Published by Springer Books in 2004.
2 Introduction In a deterministic world, it is good to know that sometimes randomness can still occur. A stochastic process is the exact opposite of a deterministic one, and is a random process that can have multiple outcomes as time progresses. This means that if we know an initial condition for the process and the function by which it is dened, we can speak of likely outcomes of the process. One of the most commonly discussed stochastic processes is the Markov chain. Section 2 denes Markov chains and goes through their main properties as well as some interesting examples of the actions that can be performed with Markov chains. The conclusion of this section is the proof of a fundamental central limit theorem for Markov chains. We conclude the dicussion in this paper by drawing on an important aspect of Markov chains: the Markov chain Monte Carlo (MCMC) methods of integration. While we provide an overview of several commonly used algorithms that fall under the title of MCMC, Section 3 employs importance sampling in order to demonstrate the power of MCMC. 2 Markov Chains Markov chains are stochastic processes that have the Markov Property, named after Russian mathematician Andrey Markov. Denition of Markov Property Informally it is the condition that given a state, the past and future states are independent of it. Formally we can dene it as follows: P (X n = x X 0,..., X n ) = P (X n = x X n ) n x. We now present some important theorems and lemmas regarding Markov Chains. 2. Theorems and lemmas First some notation: P always represents a transition matrix, while p ij represents an element of it. X j is always a random variable. Denition State i is recurrent if Otherwise it is transient. P (X n = i for some n X 0 = i) =. Denition A chain is irreducible if every state can be reached from any other one. That is p ij () > 0 i, j 2
3 We state without proof the following results: A state is recurrent if and only if n p ii (n) = and it is transient if and only if n p ii (n) <. 2 We delve right in with a lemma that connects the notions of reccurence and irreducibility. Lemma 2. In an irreducible chain, all the states are either transient or recurrent. Proof We take the shortest path from state i to state j (let it have n steps), and the shortest path from j to i (let it have m steps). Thus we have p ij (n) = a > 0 and p ji (m) = b > 0 and so we have p ii (l + n + m) p ij (n) p jj (l) p ji (m) = abp jj (l) p jj (l + n + m) p ji (m) p ii (l) p ij (n) = abp ii (l). So it is obvious that either n p ii (n) and n p jj (n) are both nite or are both innite. Thus from the above results we note that all the states of an irreducible chain are either transient or recurrent, as desired. Lemma 2.2 Facts about reccurrence.. If state i is recurrent and i j, then j is reccurent. 2. If state i is transient and i j, then j is transient. 3. The states of a nite, irreducible Markov chain are all recurrent. Proof. We employ the denition of reccurrence. Thus n, m 0 st p ij (n) > 0 and p ji (m) > 0. Thus we have p jj (l) l= p jj (l) l=n+m+ p ji (m) p ii (k) p ij (n) k= ( ) = p ji (m) p ii (k) p ij (n) k= = where the last part follows from the recurrence of i. 2 I hope that at this point the reader realizes a fundamental truth about transient states (as it becomes relevant soon). We have that given a nonzero probability p of returning to transient state i after starting from it, the distribution of the number of times that we return to that state is geometric. This requires only a little justication in one's mind. 3
4 2. We apply similar logic as above. Since i j n > 0 st p ij (n) > 0 so for m > n we have p ii (m) p ij (n) p ji (m n) and thus we have: p ii (k) k= p ii (k) k=n+ p ij (n) ( k=n+ ( ) = p ij (n) p ji (l) l= ) p ji (k n) which implies that l= p ji (l) p ij(n) ( l= p ji (l)) < as desired. 3. We know from Lemma 2. that since the chain is irreducible, all the states are either recurrent or transient. First assume that all states are transient. Now, x a state i and consider the number of times that we pass state j after starting at i. Since we only have nitely many states, the expectation for the number of times that we pass state j for some state j would be innite. So this implies that the expected number of returns to state j after starting at state j would also be innite. But that contradicts the geometric distribution of the number of returns, which has the expectation be at one over the probability of returning. Denition Mean recurrence time for a recurrent state i is m i = n nf ii (n) where f ii (n) is the probability of getting from i to i in exactly n steps. A state is null recurrent if m i = and non-null otherwise. We should note that we used exactly this value in the proof of (3) in lemma 2. so the above result can easily be extended to the fact that a nite state Markov chain has all its recurrent states be non-null. Denition The period of state i, d (i) = d if p ii (n) = 0 for d n and d = gcd {n p ii (n) > 0}. Thus a state is periodic if d (i) > and aperiodic otherwise. Denition For π i = lim n p ij (n d (i)), if greater than zero then non-null recurrent otherwise, null recurrent. Lemma 2.3 If i j then d (i) = d (j) Proof We consider m, n st p ij (n) > 0 and p ji (m) > 0 thus we can note that from the Kolmogorov Chapman equations we have 3 p ii (m + n) = p ik (n) p ki (m) p ij (n) p ji (m). k= 3 The proof of the Kolmogorov Chapman Equations is provided in the Appendix 4
5 Now by denition p ii (n + m) > 0 and d (i) (n + m). Now we can consider p ii (m + l + n) and apply the same reasoning as above to arrive at: p ii (m + l + n) = p ir (n) p rk (l) p ki (m) k= t= p ij (n) p jj (l) p ji (m) So if we have that p jj (l) > 0 then d (j) l implying as desired that p ii (m + l + n) > 0 and so d (i) (n + m + l) but combining this with d (i) (m + n) we get that d (i) l and so since d (j) = gcd {l p jj (l) > 0} we get that d (j) d (i). We can apply the same logic going from j to i and so we arrive at the conclusion that d (i) = d (j) as desired. Denition A chain is ergodic if all of its states are non-null recurent and aperiodic. Denition Let π be a probability mass function, then we say that π is a stationary probability distribution if π = πp (that is, if πis an eighenvector of the transition probability matrix P ) Denition A limiting distribution exists if P n π. π for some π Theorem 2.4 (Fundamental theorem for Markov Chains) An irreducible, ergodic Markov chain has a unique stationary distribution π. The limiting distribution exists and is equal to π. Proof Since the chain is ergodic, it is non-null recurrent which implies from above that π j = lim n p ij (n) > 0 i and π j =. Now we consident for any M M i=0 p ij (n) i=0 p ij (n) =. Now, letting n we get that M i=0 π i M which implies from above that the same is true for the ininte case i=0 π i. Now we consider the probability of moving from i to j in n + steps so p ij (n + ) = k=0 p ik (n) p kj M k=0 p ik (n) p kj M. Now we again can let n which implies that π i M k=0 π kp kj M which implies that π i k=0 π kp kj. Now we assume the inequality is strict for some i which leads to the following contradiction: i=0 π i = = π k p ki i=0 k=0 π k p ki k=0 π k. k=0 5 i=0
6 Thus we come to the conclusion that π i = k=0 π kp ki i. Now we can consider π i = π i / k=0 π k to be a stationary distribution. So we have shown existence. Now to show uniqueness we consider the following: π i = P (X n = i) = P (X n = i X 0 = j) P (X 0 = j) = p ji (n) π j. j=0 So we have that π i M j=0 p ji (n) π j and taking M, n we get that π i j=0 π jπ i = π i but we know that from the transition matrix that p ji (n) and so π i M j=0 p ji (n) π j + j=m+ π j M and so taking n we get π i M j=0 π i π j + j=m+ π j M. Now we know that πis a stationary distribution so it sums up to, and so we let M and we get π i j=0 π i π j = π i which implies that the stationary distribution is unique. The above process thus shows the existence of a limiting distribution and so we now know that an ergodic chain converges to its stationary distribution. This proof allows us to take any bounded function g and say with probability that lim g (X n ) E π (g) g (j) π j N N n= j which is a very strong result that we use constantly in Markov chain Monte Carlo. We conclude this section of lemmas and theorems with a useful denition and a small lemma. Denition π satises detailed balance if π i p ij = p ji π j Lemma 2.5 If π satises detailed balance then it is a stationary distribution. Proof We consider the j th element of πp which is i π ip ij = i π jp ji = π j i p ji = π j as desired. 2.2 Applications In this section I provide a few basic examples of Markov chains to illustrate the points made above. All the examples come from Chapter 23 of Larry Wasserman's book (provided there as exercises rather than as solved examples). Example Consider a two-state Markov chain with state χ = {, 2} and transition matrix [ ] a a P = b b where 0 < a, b <. Prove that lim P n = n [ b a+b b a+b a a+b a a+b ]. j=0 6
7 Solution To verify this we need to rst show that the chain is irreducible and ergodic. Irreducible is easy since all states communicate with nonzero probabilities. To show that it is ergodic we need to show that all states are reccurent, non-null and aperiodic. Since p ii () > 0 we have that this chain is aperiodic. Now at time n we must be at some location so j S p ij (n) = for any i. Since this is true for every n we can take the limit as n to get lim n j S p ij (n) = but in our case S is nite so we can move the limit under the sum. Now, if every state in our MC is transient or null reccurent we would have that lim n p ij (n) = 0 thus contradicting the above statement, so at least one state must be positive reccurent. Now, since we are in a nite state MC we have that all states are positive reccurent (due to question (7) below) since all the states communicate. Thus it is clear that our state is irreducible and ergodic and so it has a unique stationary distribution. Now we can solve: [ ] [ a a π π 2 b b ] = [ π ( a) + π 2 b π a + π 2 ( b) ] and now we can solve this system of equations with the added restriction π + π 2 =. So we get [ ] [ ] π π 2 = b a a+b a+b [ as desired. ] And now we can easily note that the limiting distribution is lim n P n = π as desired. π Example Let P = [ 0 0 Show that π = (0.5, 0.5)is a stationary distribution. Does this chain converge? Solution All we are asked to show is that π = [ ] is a stationary distribution so: [ 0.5 ] [ ] = [ ] = [ ] as desired. However, the chain does not converge as it is clear that if we start with π we will have an equal probability of being in either state. This means that the period of each of the states is 2 and so the chain does not converge. 3 Markov Chain Monte Carlo Markov chain Monte Carlo integration is a method for integrating function that might not have a closed form using estimation. We demonstrate a very basic example of MCMC processes through Importance Sampling. Importance sampling is used in statistics as a variance reduction method. While the standard method that we describe below does not ] 7
8 necessarily optimize the variance, we will state the condition for minimal variance. Importance sampling allows us to estimate the distribution of a random variable using a dierent random variable. The idea behind the process is that during the simulation, due to weighing of the random variable from which we have the observations, we get a better, less biased idea of the parameter we are estimating. Thus the choice of the weight is very important. We will not go through the derivation of the best (in terms of minimizing the variance) weight, but just state the result here. In an importance sampling problem we are trying to estimate the distribution of I using Î = h(xi)f(x i) N g(x i).the optimal choise of g in this case is g (x) = h(x) f(x) h(s) f(s)ds. The problem is from Larry Wasserman's All of Statistics: A Concise Course in Statistical Inference. 3. Statement (From Larry Wasserman's All of Statistics, ) Let f X,Y (x, y)be a bivariate density and let (X, Y ),..., (X N, Y N ) f X,Y.. Let w (x) be an arbitrary probability density function. Let ˆf X (x) = N i= f X,Y (x, Y i ) w (x). f X,Y (X i, Y i ) Show that, for each x, ˆfX (x) p f X (x). Find an expression for the variance. 2. Let Y N (0, ) and X Y = y N ( y, + y 2). Use the method in () to estimate f X (x). 3.2 Solution. We consider each part of the sum to be its own random variable, and we note that they are all identically and independently distributed. Due to this we can consider just one of them for the following: [ ] fx,y (x, Y i ) w (x) fx,y (x, y) w (z) E = f X,Y (z, y) dzdy f X,Y (X i, Y i ) f X,Y (z, y) = f X,Y (x, y) w (z) dzdy = f X,Y (x, y) dy = f X (x) 8
9 and so we can apply the law of large numbers to note that N i= f X,Y (x, Y i ) w (x) f X,Y (X i, Y i ) p which is the same as ˆf X (x) p f X (x), as desired. E [ ] fx,y (x, Y i ) w (x) f X,Y (X i, Y i ) The variance calculation is fairly simple and we do not dwell on it, providing simply the easily veriable answer: [ ] var ˆf X (x) = f 2 X,Y (x, y) w 2 (z) dzdy fx 2 (x). N f X,Y (z, y) 2. We note that the marginal density of X is hard to evaluate: f X (x) = f X Y (x) f Y (y) dy = (x y) 2 2(+y e 2 ) e y2 2 dy. 2π ( + y2 ) 2π Thus it makes sense to employ importance sampling as in () in order to estimate f X (x). So we take the distribution of w (x) to be the Standard Normal as it seems like a reasonable one in this case. So we have: ˆf X (x) = N = N = N i= i= f X,Y (x, Y i ) w (x) f X,Y (X i, Y i ) f X Y (x Y i ) w (X i ) f X Y (X Y i ) { exp [(x 2π 2 ( + y 2 Y i ) 2 (X i Y i ) 2 + ( x ( + y 2)) ]} 2 ) i= using the above knowledge. 3.3 Further Discussion We see above a very basic application of Markov chain Monte Carlo methods which allows us to use a biased sampling distribution in order to estimate a random variable of interest. The above discussed method is a very basic and introductory one. We actually have multiple possible algorithms that we can apply in order to arrive at the best possible estimate, however they are a topic for another paper and will only be briey mentioned here. The most commonly used algorithms for MCMC are the Metropolis Hastings algorithms which use a conditional proposal distribution in order to construct a Markov chain with a stationary distribution f. It supposes that X 0 was chosen arbitrarily and then proceeds to use the proposal distribution in order to 9
10 generate candidates that are either added to the chain or are overlooked based on a specied probability distribution. 4 There are several dierent incarnations of this algorithm, with dierent suggested proposal distributions: the randomwalk M-H algorithm is the one that was described above (as if we do not accept or reject the generated value, all we are doing is simulating a random walk on the real line). In independence M-H we change the proposal distribution to a xed distribution which we believe to be an approximation of f. Another method that gets a lot of use is the Gibbs sampling algorithm which is simply an emblishment of the M-H algorithm. What this method does is take a multi-dimensional problem and turns it into several one-dimensional problems that can be easily estimated using the above method. So ( instead of simply ) getting X i+, Gibbs sampling allows for the estimation of X () i+,..., X(n) i+ for an n-dimensional model. This process works due to detailed balance of Markov Chains, which was briey mentioned in the previos section. For further information consult Wasserman's book. 4 For { basic Metropolis-Hastings, we have the probability be r (x, y) = f(y) q(x y) min }where f(x) q(y x), q is the proposal distribution. In the case that we have, we can easily choose a q such that q (x y) = q (y x) so obviously one part of the multiplication cancels out. 0
11 4 Appendix Theorem 4. Chapman Kolmogorov Equations. The n-step probabilities satisfy p ij (n + m) = p ik (m) p kj (n). k (We thus get P m+n = P m P n which is standard matrix multiplication) Proof We employ the law of conditional probability and the law of total probability to arrive at the following: p ij (m + n) = P (X m+n = j X 0 = i) = k P (X m+n = j, X m = k X 0 = i) = k = k = k P (X m+n = j X m = k, X 0 = i) P (X m = k X 0 = i) P (X m+n = j X m = k) P (X m = k X 0 = i) p ik (m) p kj (n) by denition, as desired.
Introduction to Markov Chain Monte Carlo
Introduction to Markov Chain Monte Carlo Monte Carlo: sample from a distribution to estimate the distribution to compute max, mean Markov Chain Monte Carlo: sampling using local information Generic problem
More informationLECTURE 4. Last time: Lecture outline
LECTURE 4 Last time: Types of convergence Weak Law of Large Numbers Strong Law of Large Numbers Asymptotic Equipartition Property Lecture outline Stochastic processes Markov chains Entropy rate Random
More informationHOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!
Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following
More information1. (First passage/hitting times/gambler s ruin problem:) Suppose that X has a discrete state space and let i be a fixed state. Let
Copyright c 2009 by Karl Sigman 1 Stopping Times 1.1 Stopping Times: Definition Given a stochastic process X = {X n : n 0}, a random time τ is a discrete random variable on the same probability space as
More informationPractice with Proofs
Practice with Proofs October 6, 2014 Recall the following Definition 0.1. A function f is increasing if for every x, y in the domain of f, x < y = f(x) < f(y) 1. Prove that h(x) = x 3 is increasing, using
More information1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
More informationAdaptive Search with Stochastic Acceptance Probabilities for Global Optimization
Adaptive Search with Stochastic Acceptance Probabilities for Global Optimization Archis Ghate a and Robert L. Smith b a Industrial Engineering, University of Washington, Box 352650, Seattle, Washington,
More informationI. GROUPS: BASIC DEFINITIONS AND EXAMPLES
I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called
More information2.3 Convex Constrained Optimization Problems
42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions
More informationLinear Algebra Notes for Marsden and Tromba Vector Calculus
Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of
More informationOn the mathematical theory of splitting and Russian roulette
On the mathematical theory of splitting and Russian roulette techniques St.Petersburg State University, Russia 1. Introduction Splitting is an universal and potentially very powerful technique for increasing
More informationMASTER S THESIS for stud.techn. Kristian Stormark. Faculty of Information Technology, Mathematics and Electrical Engineering NTNU
NTNU Norwegian University of Science and Technology Faculty of Information Technology, Mathematics and Electrical Engineering MASTER S THESIS for stud.techn. Kristian Stormark Faculty of Information Technology,
More information. (3.3) n Note that supremum (3.2) must occur at one of the observed values x i or to the left of x i.
Chapter 3 Kolmogorov-Smirnov Tests There are many situations where experimenters need to know what is the distribution of the population of their interest. For example, if they want to use a parametric
More information1 Homework 1. [p 0 q i+j +... + p i 1 q j+1 ] + [p i q j ] + [p i+1 q j 1 +... + p i+j q 0 ]
1 Homework 1 (1) Prove the ideal (3,x) is a maximal ideal in Z[x]. SOLUTION: Suppose we expand this ideal by including another generator polynomial, P / (3, x). Write P = n + x Q with n an integer not
More informationTutorial on Markov Chain Monte Carlo
Tutorial on Markov Chain Monte Carlo Kenneth M. Hanson Los Alamos National Laboratory Presented at the 29 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Technology,
More informationFalse-Alarm and Non-Detection Probabilities for On-line Quality Control via HMM
Int. Journal of Math. Analysis, Vol. 6, 2012, no. 24, 1153-1162 False-Alarm and Non-Detection Probabilities for On-line Quality Control via HMM C.C.Y. Dorea a,1, C.R. Gonçalves a, P.G. Medeiros b and W.B.
More informationUndergraduate Notes in Mathematics. Arkansas Tech University Department of Mathematics
Undergraduate Notes in Mathematics Arkansas Tech University Department of Mathematics An Introductory Single Variable Real Analysis: A Learning Approach through Problem Solving Marcel B. Finan c All Rights
More informationLimits and Continuity
Math 20C Multivariable Calculus Lecture Limits and Continuity Slide Review of Limit. Side limits and squeeze theorem. Continuous functions of 2,3 variables. Review: Limits Slide 2 Definition Given a function
More information11 Multivariate Polynomials
CS 487: Intro. to Symbolic Computation Winter 2009: M. Giesbrecht Script 11 Page 1 (These lecture notes were prepared and presented by Dan Roche.) 11 Multivariate Polynomials References: MC: Section 16.6
More informationChapter 7. Continuity
Chapter 7 Continuity There are many processes and eects that depends on certain set of variables in such a way that a small change in these variables acts as small change in the process. Changes of this
More informationData analysis in supersaturated designs
Statistics & Probability Letters 59 (2002) 35 44 Data analysis in supersaturated designs Runze Li a;b;, Dennis K.J. Lin a;b a Department of Statistics, The Pennsylvania State University, University Park,
More informationMATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column
More informationNOTES ON LINEAR TRANSFORMATIONS
NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all
More informationNotes 11: List Decoding Folded Reed-Solomon Codes
Introduction to Coding Theory CMU: Spring 2010 Notes 11: List Decoding Folded Reed-Solomon Codes April 2010 Lecturer: Venkatesan Guruswami Scribe: Venkatesan Guruswami At the end of the previous notes,
More informationTHE DIMENSION OF A VECTOR SPACE
THE DIMENSION OF A VECTOR SPACE KEITH CONRAD This handout is a supplementary discussion leading up to the definition of dimension and some of its basic properties. Let V be a vector space over a field
More informationMatrix-Chain Multiplication
Matrix-Chain Multiplication Let A be an n by m matrix, let B be an m by p matrix, then C = AB is an n by p matrix. C = AB can be computed in O(nmp) time, using traditional matrix multiplication. Suppose
More information4. CLASSES OF RINGS 4.1. Classes of Rings class operator A-closed Example 1: product Example 2:
4. CLASSES OF RINGS 4.1. Classes of Rings Normally we associate, with any property, a set of objects that satisfy that property. But problems can arise when we allow sets to be elements of larger sets
More informationit is easy to see that α = a
21. Polynomial rings Let us now turn out attention to determining the prime elements of a polynomial ring, where the coefficient ring is a field. We already know that such a polynomial ring is a UF. Therefore
More informationDynamic Assignment of Dedicated and Flexible Servers in Tandem Lines
Dynamic Assignment of Dedicated and Flexible Servers in Tandem Lines Sigrún Andradóttir and Hayriye Ayhan School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332-0205,
More informationCritical points of once continuously differentiable functions are important because they are the only points that can be local maxima or minima.
Lecture 0: Convexity and Optimization We say that if f is a once continuously differentiable function on an interval I, and x is a point in the interior of I that x is a critical point of f if f (x) =
More informationMATH PROBLEMS, WITH SOLUTIONS
MATH PROBLEMS, WITH SOLUTIONS OVIDIU MUNTEANU These are free online notes that I wrote to assist students that wish to test their math skills with some problems that go beyond the usual curriculum. These
More informationLecture 1: Systems of Linear Equations
MTH Elementary Matrix Algebra Professor Chao Huang Department of Mathematics and Statistics Wright State University Lecture 1 Systems of Linear Equations ² Systems of two linear equations with two variables
More information1.2 Solving a System of Linear Equations
1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables
More informationDiscrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2
CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2 Proofs Intuitively, the concept of proof should already be familiar We all like to assert things, and few of us
More informationM/M/1 and M/M/m Queueing Systems
M/M/ and M/M/m Queueing Systems M. Veeraraghavan; March 20, 2004. Preliminaries. Kendall s notation: G/G/n/k queue G: General - can be any distribution. First letter: Arrival process; M: memoryless - exponential
More informationDirichlet Processes A gentle tutorial
Dirichlet Processes A gentle tutorial SELECT Lab Meeting October 14, 2008 Khalid El-Arini Motivation We are given a data set, and are told that it was generated from a mixture of Gaussian distributions.
More informationPolynomial Invariants
Polynomial Invariants Dylan Wilson October 9, 2014 (1) Today we will be interested in the following Question 1.1. What are all the possible polynomials in two variables f(x, y) such that f(x, y) = f(y,
More informationa 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.
Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given
More informationPermanents, Order Statistics, Outliers, and Robustness
Permanents, Order Statistics, Outliers, and Robustness N. BALAKRISHNAN Department of Mathematics and Statistics McMaster University Hamilton, Ontario, Canada L8S 4K bala@mcmaster.ca Received: November
More informationMetric Spaces. Chapter 7. 7.1. Metrics
Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some
More informationThe Exponential Distribution
21 The Exponential Distribution From Discrete-Time to Continuous-Time: In Chapter 6 of the text we will be considering Markov processes in continuous time. In a sense, we already have a very good understanding
More informationEfficiency Improvement of Stochastic Simulation by Means of Subset Sampling
5. LS-DYNA Anwenderforum, Ulm 6 Robustheit / Optimierung I Efficiency Improvement of Stochastic Simulation by Means of Subset Sampling Martin Liebscher, Stephan Pannier, Jan-Uwe Sickert, Wolfgang Graf
More informationGaussian Processes to Speed up Hamiltonian Monte Carlo
Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu Lê Murray, Iain http://videolectures.net/mlss09uk_murray_mcmc/ Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo
More informationTHE CENTRAL LIMIT THEOREM TORONTO
THE CENTRAL LIMIT THEOREM DANIEL RÜDT UNIVERSITY OF TORONTO MARCH, 2010 Contents 1 Introduction 1 2 Mathematical Background 3 3 The Central Limit Theorem 4 4 Examples 4 4.1 Roulette......................................
More informationNotes on metric spaces
Notes on metric spaces 1 Introduction The purpose of these notes is to quickly review some of the basic concepts from Real Analysis, Metric Spaces and some related results that will be used in this course.
More informationBALANCED LOADS IN INFINITE NETWORKS. By Bruce Hajek. University of Illinois at Urbana-Champaign
BALANCED LOADS IN INFINITE NETWORKS By Bruce Hajek University of Illinois at Urbana-Champaign A set of nodes and a set of consumers are given, and to each consumer there corresponds a subset of the nodes.
More informationOverview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
More informationExact Nonparametric Tests for Comparing Means - A Personal Summary
Exact Nonparametric Tests for Comparing Means - A Personal Summary Karl H. Schlag European University Institute 1 December 14, 2006 1 Economics Department, European University Institute. Via della Piazzuola
More informationProt Maximization and Cost Minimization
Simon Fraser University Prof. Karaivanov Department of Economics Econ 0 COST MINIMIZATION Prot Maximization and Cost Minimization Remember that the rm's problem is maximizing prots by choosing the optimal
More informationIntroduction to Monte Carlo. Astro 542 Princeton University Shirley Ho
Introduction to Monte Carlo Astro 542 Princeton University Shirley Ho Agenda Monte Carlo -- definition, examples Sampling Methods (Rejection, Metropolis, Metropolis-Hasting, Exact Sampling) Markov Chains
More informationMonte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMS091)
Monte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMS091) Magnus Wiktorsson Centre for Mathematical Sciences Lund University, Sweden Lecture 5 Sequential Monte Carlo methods I February
More informationx a x 2 (1 + x 2 ) n.
Limits and continuity Suppose that we have a function f : R R. Let a R. We say that f(x) tends to the limit l as x tends to a; lim f(x) = l ; x a if, given any real number ɛ > 0, there exists a real number
More informationMonte Carlo Methods in Finance
Author: Yiyang Yang Advisor: Pr. Xiaolin Li, Pr. Zari Rachev Department of Applied Mathematics and Statistics State University of New York at Stony Brook October 2, 2012 Outline Introduction 1 Introduction
More informationSystems of Linear Equations
Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and
More informationMATH 22. THE FUNDAMENTAL THEOREM of ARITHMETIC. Lecture R: 10/30/2003
MATH 22 Lecture R: 10/30/2003 THE FUNDAMENTAL THEOREM of ARITHMETIC You must remember this, A kiss is still a kiss, A sigh is just a sigh; The fundamental things apply, As time goes by. Herman Hupfeld
More informationMarkov Chain Monte Carlo and Numerical Differential Equations
Markov Chain Monte Carlo and Numerical Differential Equations J.M. Sanz-Serna 1 Introduction This contribution presents a hopefully readable introduction to Markov Chain Monte Carlo methods with particular
More informationA Sublinear Bipartiteness Tester for Bounded Degree Graphs
A Sublinear Bipartiteness Tester for Bounded Degree Graphs Oded Goldreich Dana Ron February 5, 1998 Abstract We present a sublinear-time algorithm for testing whether a bounded degree graph is bipartite
More information1 Sufficient statistics
1 Sufficient statistics A statistic is a function T = rx 1, X 2,, X n of the random sample X 1, X 2,, X n. Examples are X n = 1 n s 2 = = X i, 1 n 1 the sample mean X i X n 2, the sample variance T 1 =
More informationOnline Appendix to Stochastic Imitative Game Dynamics with Committed Agents
Online Appendix to Stochastic Imitative Game Dynamics with Committed Agents William H. Sandholm January 6, 22 O.. Imitative protocols, mean dynamics, and equilibrium selection In this section, we consider
More informationIEOR 6711: Stochastic Models, I Fall 2012, Professor Whitt, Final Exam SOLUTIONS
IEOR 6711: Stochastic Models, I Fall 2012, Professor Whitt, Final Exam SOLUTIONS There are four questions, each with several parts. 1. Customers Coming to an Automatic Teller Machine (ATM) (30 points)
More informationCHAPTER 3. Methods of Proofs. 1. Logical Arguments and Formal Proofs
CHAPTER 3 Methods of Proofs 1. Logical Arguments and Formal Proofs 1.1. Basic Terminology. An axiom is a statement that is given to be true. A rule of inference is a logical rule that is used to deduce
More informationWRITING PROOFS. Christopher Heil Georgia Institute of Technology
WRITING PROOFS Christopher Heil Georgia Institute of Technology A theorem is just a statement of fact A proof of the theorem is a logical explanation of why the theorem is true Many theorems have this
More informationT ( a i x i ) = a i T (x i ).
Chapter 2 Defn 1. (p. 65) Let V and W be vector spaces (over F ). We call a function T : V W a linear transformation form V to W if, for all x, y V and c F, we have (a) T (x + y) = T (x) + T (y) and (b)
More informationA characterization of trace zero symmetric nonnegative 5x5 matrices
A characterization of trace zero symmetric nonnegative 5x5 matrices Oren Spector June 1, 009 Abstract The problem of determining necessary and sufficient conditions for a set of real numbers to be the
More informationChapter 1. Introduction
Chapter 1 Introduction 1.1. Motivation Network performance analysis, and the underlying queueing theory, was born at the beginning of the 20th Century when two Scandinavian engineers, Erlang 1 and Engset
More informationMath 4310 Handout - Quotient Vector Spaces
Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable
More information5 Homogeneous systems
5 Homogeneous systems Definition: A homogeneous (ho-mo-jeen -i-us) system of linear algebraic equations is one in which all the numbers on the right hand side are equal to : a x +... + a n x n =.. a m
More informationSection 4.2: The Division Algorithm and Greatest Common Divisors
Section 4.2: The Division Algorithm and Greatest Common Divisors The Division Algorithm The Division Algorithm is merely long division restated as an equation. For example, the division 29 r. 20 32 948
More informationSome probability and statistics
Appendix A Some probability and statistics A Probabilities, random variables and their distribution We summarize a few of the basic concepts of random variables, usually denoted by capital letters, X,Y,
More informationLecture 3: Finding integer solutions to systems of linear equations
Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture
More information11. Time series and dynamic linear models
11. Time series and dynamic linear models Objective To introduce the Bayesian approach to the modeling and forecasting of time series. Recommended reading West, M. and Harrison, J. (1997). models, (2 nd
More informationFluid models in performance analysis
Fluid models in performance analysis Marco Gribaudo 1, Miklós Telek 2 1 Dip. di Informatica, Università di Torino, marcog@di.unito.it 2 Dept. of Telecom., Technical University of Budapest telek@hitbme.hu
More informationContinuity of the Perron Root
Linear and Multilinear Algebra http://dx.doi.org/10.1080/03081087.2014.934233 ArXiv: 1407.7564 (http://arxiv.org/abs/1407.7564) Continuity of the Perron Root Carl D. Meyer Department of Mathematics, North
More informationChapter 3: Section 3-3 Solutions of Linear Programming Problems
Chapter 3: Section 3-3 Solutions of Linear Programming Problems D. S. Malik Creighton University, Omaha, NE D. S. Malik Creighton University, Omaha, NE Chapter () 3: Section 3-3 Solutions of Linear Programming
More informationThe last three chapters introduced three major proof techniques: direct,
CHAPTER 7 Proving Non-Conditional Statements The last three chapters introduced three major proof techniques: direct, contrapositive and contradiction. These three techniques are used to prove statements
More informationSECTION 10-2 Mathematical Induction
73 0 Sequences and Series 6. Approximate e 0. using the first five terms of the series. Compare this approximation with your calculator evaluation of e 0.. 6. Approximate e 0.5 using the first five terms
More informationRepresentation of functions as power series
Representation of functions as power series Dr. Philippe B. Laval Kennesaw State University November 9, 008 Abstract This document is a summary of the theory and techniques used to represent functions
More informationMATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.
MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α
More informationHydrodynamic Limits of Randomized Load Balancing Networks
Hydrodynamic Limits of Randomized Load Balancing Networks Kavita Ramanan and Mohammadreza Aghajani Brown University Stochastic Networks and Stochastic Geometry a conference in honour of François Baccelli
More informationProbability Generating Functions
page 39 Chapter 3 Probability Generating Functions 3 Preamble: Generating Functions Generating functions are widely used in mathematics, and play an important role in probability theory Consider a sequence
More informationMarkov random fields and Gibbs measures
Chapter Markov random fields and Gibbs measures 1. Conditional independence Suppose X i is a random element of (X i, B i ), for i = 1, 2, 3, with all X i defined on the same probability space (.F, P).
More informationSTABILITY OF LU-KUMAR NETWORKS UNDER LONGEST-QUEUE AND LONGEST-DOMINATING-QUEUE SCHEDULING
Applied Probability Trust (28 December 2012) STABILITY OF LU-KUMAR NETWORKS UNDER LONGEST-QUEUE AND LONGEST-DOMINATING-QUEUE SCHEDULING RAMTIN PEDARSANI and JEAN WALRAND, University of California, Berkeley
More informationTutorial: Stochastic Modeling in Biology Applications of Discrete- Time Markov Chains. NIMBioS Knoxville, Tennessee March 16-18, 2011
Tutorial: Stochastic Modeling in Biology Applications of Discrete- Time Markov Chains Linda J. S. Allen Texas Tech University Lubbock, Texas U.S.A. NIMBioS Knoxville, Tennessee March 16-18, 2011 OUTLINE
More informationA Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data
A Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data Faming Liang University of Florida August 9, 2015 Abstract MCMC methods have proven to be a very powerful tool for analyzing
More informationSUM OF TWO SQUARES JAHNAVI BHASKAR
SUM OF TWO SQUARES JAHNAVI BHASKAR Abstract. I will investigate which numbers can be written as the sum of two squares and in how many ways, providing enough basic number theory so even the unacquainted
More informationLectures on Stochastic Processes. William G. Faris
Lectures on Stochastic Processes William G. Faris November 8, 2001 2 Contents 1 Random walk 7 1.1 Symmetric simple random walk................... 7 1.2 Simple random walk......................... 9 1.3
More informationMarkov Chains And Existtime Stabilization
50 15. MARKOV CHAINS: DEFINITIONS AND EXAMPLES Along with the discussion of martingales, we have introduced the concept of a discrete-time stochastic process. In this chapter we will study a particular
More informationVerication by Finitary Abstraction Weizmann Institute of Sciences and Universite Joseph Fourier, Grenoble Fourth International Spin Workshop (SPIN'98) Paris 2.11.98 Joint work with: Y. Kesten Ben Gurion
More informationMarkov Chains. Chapter 4. 4.1 Stochastic Processes
Chapter 4 Markov Chains 4 Stochastic Processes Often, we need to estimate probabilities using a collection of random variables For example, an actuary may be interested in estimating the probability that
More informationIntroduction. Appendix D Mathematical Induction D1
Appendix D Mathematical Induction D D Mathematical Induction Use mathematical induction to prove a formula. Find a sum of powers of integers. Find a formula for a finite sum. Use finite differences to
More informationChapter 3. Cartesian Products and Relations. 3.1 Cartesian Products
Chapter 3 Cartesian Products and Relations The material in this chapter is the first real encounter with abstraction. Relations are very general thing they are a special type of subset. After introducing
More informationWinter Camp 2011 Polynomials Alexander Remorov. Polynomials. Alexander Remorov alexanderrem@gmail.com
Polynomials Alexander Remorov alexanderrem@gmail.com Warm-up Problem 1: Let f(x) be a quadratic polynomial. Prove that there exist quadratic polynomials g(x) and h(x) such that f(x)f(x + 1) = g(h(x)).
More informationOverview of Monte Carlo Simulation, Probability Review and Introduction to Matlab
Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab 1 Overview of Monte Carlo Simulation 1.1 Why use simulation?
More informationRow Echelon Form and Reduced Row Echelon Form
These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation
More informationSTAT 830 Convergence in Distribution
STAT 830 Convergence in Distribution Richard Lockhart Simon Fraser University STAT 830 Fall 2011 Richard Lockhart (Simon Fraser University) STAT 830 Convergence in Distribution STAT 830 Fall 2011 1 / 31
More informationSPECTRAL POLYNOMIAL ALGORITHMS FOR COMPUTING BI-DIAGONAL REPRESENTATIONS FOR PHASE TYPE DISTRIBUTIONS AND MATRIX-EXPONENTIAL DISTRIBUTIONS
Stochastic Models, 22:289 317, 2006 Copyright Taylor & Francis Group, LLC ISSN: 1532-6349 print/1532-4214 online DOI: 10.1080/15326340600649045 SPECTRAL POLYNOMIAL ALGORITHMS FOR COMPUTING BI-DIAGONAL
More informationThe Division Algorithm for Polynomials Handout Monday March 5, 2012
The Division Algorithm for Polynomials Handout Monday March 5, 0 Let F be a field (such as R, Q, C, or F p for some prime p. This will allow us to divide by any nonzero scalar. (For some of the following,
More informationResearch Article Two-Period Inventory Control with Manufacturing and Remanufacturing under Return Compensation Policy
Discrete Dynamics in Nature and Society Volume 2013, Article ID 871286, 8 pages http://dx.doi.org/10.1155/2013/871286 Research Article Two-Period Inventory Control with Manufacturing and Remanufacturing
More informationPUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.
PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include
More informationModern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh
Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem
More information