Renewal processes and Poisson process

Transcription

1 CHAPTER 3 Renewal processes and Poisson process 31 Definiion of renewal processes and limi heorems Le ξ 1, ξ 2, be independen and idenically disribued random variables wih P[ξ k > 0] = 1 Define heir parial sums S n = ξ ξ n, n N, S 0 = 0 Noe ha he sequence S 1, S 2, is increasing We call S 1, S 2, he renewal imes (or simply renewals) and ξ 1, ξ 2, he inerrenewal imes Definiion 311 The process {N : 0} given by N = is called he renewal process n=1 1 {Sn } Theorem 312 (Law of large numbers for renewal processes) Le m := Eξ 1 (0, ), hen Idea of proof N as 1, as m By he definiion of N we have he inequaliy Dividing his by N we obain (311) S N S N+1 S N N N S N +1 N + 1 N + 1 N We have N as since here are infiniely many renewals and hus, he funcion N (which is non-decreasing by definiion) canno say bounded By he law of large numbers, boh sides of (311) as converge o m as By he sandwich lemma, we have This proves he claim N as m, as Theorem 313 (Cenral limi heorem for renewal processes) Le m := Eξ 1 (0, ) and σ 2 := Var ξ 1 (0, ) Then, N m σ m 3/2 d N(0, 1), as 1

2 Idea of proof The usual cenral limi heorem for S n = ξ ξ n saes ha S n nm σ d N(0, 1) n n Denoing by N a sandard normal random variable we can wrie his as follows: For large n, we have an approximae equaliy of disribuions S n nm + σ nn This means ha he inerval [0, nm + σ nn] conains approximaely n renewals By he law of large numbers for renewal processes, see Theorem 312, i seems plausible ha he inerval [nm, nm + σ nn] conains approximaely σ nn/m renewals I follows ha he inerval [0, nm] conains approximaely n σ nn/m renewals Le us now inroduce he variable = nm Then, n is equivalen o Consequenly, for large in he inerval [0, ] we have approximaely m σ m N 3/2 renewals By definiion, his number of renewals is N This means ha for large N m σ m 3/2 N, Definiion 314 The renewal funcion H() is he expeced number of renewals in he inerval [0, ]: H() = EN, 0 Remark 315 Denoing by F k () = P[S k ] he disribuion funcion of S k, we have he formula H() = EN = E 1 Sk = E1 Sk = P[S k ] = F k () Theorem 316 (Weak renewal heorem) Le m := Eξ 1 (0, ) I holds ha H() lim as 1 m = 1 m Idea of proof By Theorem 312, N as In order o obain Theorem 316, we have o ake expecaion of boh sides and inerchange he limi and he expecaion The rigorous jusificaion will be omied Definiion 317 The random variables ξ k are called laice if here are a > 0, b R so ha ξ k wih probabiliy 1 akes values in he se az + b, ha is P[ξ k {an + b : n Z}] = 1 Theorem 318 (Blackwell renewal heorem) Assume ha ξ 1 is non-laice and le m := Eξ 1 (0, ) Then, for all s > 0, Proof Omied lim (H( + s) H()) = s m 2

3 32 Saionary processes and processes wih saionary incremens Consider a sochasic process {X, 0} For concreeness, we have chosen he index se T o be [0, ), bu similar definiions apply o sochasic processes wih index ses T = R, N, N 0, Z Definiion 321 The process {X : 0} is called saionary if for all n N, 0 1 n and all h 0, (X 1,, X n ) d = (X 1 +h,, X n+h) Example 322 Le {X : N 0 } be independen and idenically disribued random variables We claim ha he process X is saionary Le µ be he probabiliy disribuion of X, ha is µ(a) = P[X A], for all Borel ses A R Then, for all Borel ses A 1,, A n R, P[X 1 +h A 1,, X n+h A n ] = µ(a 1 ) µ(a n ) = P[X 1 A 1,, X n A n ] This proves ha X is saionary Example 323 Le {X : N 0 } be a Markov chain saring wih an invarian probabiliy disribuion λ Then, X is saionary Proof Le us firs compue he join disribuion of (X h, X h+1,, X h+m ) For any saes i 0,, i m E we have P[X h = i 0, X h+1 = i 1,, X h+m = i m ] = P[X h = i 0 ] p i0 i 1 p im 1 i m Since he iniial measure λ of he Markov chain is invarian, we have P[X h = i 0 ] = λ i0 We herefore obain ha P[X h = i 0, X h+1 = i 1,, X h+m = i m ] = λ i0 p i0 i 1 p im 1 i m This expression does no depend on h hus showing ha (X h, X h+1,, X h+m ) d = (X 0, X 1,, X m ) If we drop some componens in he firs vecor and he corresponding componens in he second vecor, he vecors formed by he remaining componens sill have he same disribuion In his way we can prove ha (X 1 +h, X 2 +h,, X n+h) has he same disribuion as (X 1, X 2,, X n ) Definiion 324 The process {X : 0} has saionary incremens if for all n N, h 0 and m, we have he following equaliy in disribuion: (X 1 +h X 0 +h, X 2 +h X 1 +h,, X n+h X n 1 +h) d = (X 1 X 0, X 2 X 1,, X n X n 1 ) Definiion 325 The process {X : 0} has independen incremens if for all n N and n, he random variables are independen X 0, X 1 X 0, X 2 X 1,, X n X n 1 Laer we will consider wo examples of processes which have boh saionary and independen incremens: he Poisson Process and he Brownian Moion 3

4 33 Poisson process The Poisson process is a special case of renewal process in which he inerrenewal imes are exponenially disribued Namely, le ξ 1, ξ 2, be independen idenically disribued random variables having exponenial disribuion wih parameer λ > 0, ha is Define he renewal imes S n by P[ξ k x] = 1 e λx, x 0 S n = ξ ξ n, n N, S 0 = 0 I s an exercise o show (for example, by inducion) ha he densiy of S n is given by f Sn (x) = λn x n 1 (n 1)! e λx, x 0 The disribuion of S n is called he Erlang disribuion wih parameers n and λ I is a paricular case of he Gamma disribuion Definiion 331 The Poisson process wih inensiy λ > 0 is a process {N : 0} defined by N = 1 {Sk } Noe ha N couns he number of renewals in he inerval [0, ] The nex heorem explains why he Poisson process was named afer Poisson Theorem 332 For all 0 i holds ha N Poi(λ) Proof We need o prove ha for all n N 0, Sep 1 Le firs n = 0 Then, P[N = n] = (λ)n e λ P[N = 0] = P[ξ 1 > ] = e λ, hus esablishing he required formula for n = 0 Sep 2 Le n N We compue he probabiliy P[N = n] By definiion of N we have P[N = n] = P[N n] P[N n + 1] = P[S n ] P[S n+1 ] Using he formula for he densiy of S n we obain ha ( ) λ n x n 1 P[N = n] = f Sn (x)dx f Sn+1 (x)dx = (n 1)! e λx λn+1 x n e λx dx 0 The expression under he sign of he inegral is equal o ( ) d (λx) n e λx dx Thus, we can compue he inegral as follows: ( ) (λx) n x= P[N = n] = e λx x=0 = (λ)n e λ,

5 where he las sep holds since we assumed ha n 0 Remark 333 From he above heorem i follows ha he renewal funcion of he Poisson process is given by H() = EN = λ For he nex heorem le U 1,, U n be independen random variables which are uniformly disribued on he inerval [0, ] Denoe by U (1) U (n) he order saisics of U 1,, U n Theorem 334 The condiional disribuion of he random vecor (S 1,, S n ) given ha {N = n} coincides wih he disribuion of (U (1),, U (n) ): (S 1,, S n ) {N = n} d = (U (1),, U (n) ) Proof We will compue he densiies of boh vecors and show hese densiies are equal Sep 1 The join densiy of he random variables (ξ 1,, ξ n+1 ) has (by independence) he produc form n+1 f ξ1,,ξ n+1 (u 1,, u n+1 ) = λe λu k, u 1,, u n+1 > 0 Sep 2 We compue he join densiy of (S 1,, S n+1 ) Consider a linear ransformaion A defined by A(u 1, u 2,, u n+1 ) = (u 1, u 1 + u 2,, u u n+1 ) The random variables (S 1,, S n+1 ) can be obained by applying he linear ransformaion A o he variables (ξ 1,, ξ n+1 ): (S 1,, S n+1 ) = A(ξ 1,, ξ n+1 ) The deerminan of he ransformaion A is 1 since he marix of his ransformaion is riangular wih 1 s on he diagonal By he densiy ransformaion heorem, he densiy of (S 1,, S n+1 ) is given by n+1 f S1,,S n+1 ( 1,, n+1 ) = λe λ( k k 1 ) = λ n+1 e λ n+1, where 0 = 0 < 1 < < n+1 Oherwise, he densiy vanishes Noe ha he formula for he densiy depends only on n+1 and does no depend on 1,, n Sep 3 We compue he condiional densiy of (S 1,, S n ) given ha N = n Le 0 < 1 < < n < Inuiively, he condiional densiy of (S 1,, S n ) given ha N = n is given by f S1,,S n ( 1,, n N = n) = lim ε 0 P[ 1 < S 1 < 1 + ε,, n < S 1 < n + ε N = n] ε n P[ 1 < S 1 < 1 + ε,, n < S n < n + ε, N = n] = lim ε 0 ε n P[N = n] P[ 1 < S 1 < 1 + ε,, n < S n < n + ε, S n+1 > ] = lim ε 0 ε n P[N = n] 5

6 Using he formula for he join densiy of (S 1,, S n+1 ) and noing ha his densiy does no depend on 1,, n, we obain ha P[ 1 < S 1 < 1 + ε,, n < S n < n + ε, S n+1 > ] λ n+1 e λ n+1 d n+1 = = ε n P[N = n] P[N = n], n where in he las sep we used ha N has Poisson disribuion wih parameer λ So, we have {, for 0 < f S1,,S n ( 1,, n N = n) = n 1 < < n <, 0, oherwise Sep 4 The join densiy of he order saisics (U (1),, U (n) ) is known (Sochasik I) o be given by {, for 0 < f U(1),,U (n) ( 1,, n ) = n 1 < < n <, 0, oherwise This coincides wih he condiional densiy of (S 1,, S n ) given ha N = n, hus proving he heorem Theorem 335 The Poisson process {N : 0} has independen incremens and hese incremens have Poisson disribuion, namely for all, s 0 we have N +s N Poi(λs) Proof Take some poins 0 = 0 1 n We deermine he disribuion of he random vecor (N 1, N 2 N 1,, N n N n 1 ) Take some x 1,, x n N 0 We compue he probabiliy P := P[N 1 = x 1, N 2 N 1 = x 2,, N n N n 1 = x n ] Le x = x x n By definiion of condiional probabiliy, P = P[N 1 = x 1, N 2 N 1 = x 2,, N n N n 1 = x n N n = x] P[N n = x] Given ha N n = x, he Poisson process has x renewals in he inerval [0, n ] and by Theorem 334 hese renewals have he same disribuion as x independen random variables which have uniform disribuion on he inerval [0, n ], afer arranging hem in an increasing order Hence, in order o compue he condiional probabiliy we can use he mulinomial disribuion: ( ) x! n ( k k 1 ) x k P = (λ n) x e λn x 1! x n! x! Afer making ransformaions we arrive a n ( (λ(k k 1 )) x k P = x k! x k n ) e λ( k k 1 ) From his formula we see ha he random variables N 1, N 2 N 1,, N n N n 1 are independen and ha hey are Poisson disribued, namely This proves he heorem N k N k 1 Poi(λ( k k 1 )) 6

7 Theorem 336 The Poisson process has saionary incremens Proof Take some h 0, and some n We have o show ha he disribuion of he random vecor (N 1 +h N 0 +h, N 2 +h N 1 +h,, N n+h N n 1 +h) does no depend on h However, we know from Theorem 335 ha he componens of his vecor are independen and ha N k +h N k 1 +h Poi(λ( k k 1 )), which does no depend on h 34 Laice renewal processes In his secion we show how he heory of Markov chains can be used o obain some properies of renewal processes whose inerrenewal imes are ineger Le ξ 1, ξ 2, be independen and idenically disribued random variables wih values in N = {1, 2, } Le us wrie We will make he aperiodiciy assumpion: r n := P[ξ 1 = n], n N (341) gcd{n N : r n 0} = 1 For example, his condiion excludes renewal processes for which he ξ k s ake only even values Define he renewal imes S n = ξ ξ n, n N Theorem 341 Le m := Eξ 1 be finie Then, lim P[ k N : S k = n] = 1 n m So, he probabiliy ha here is a renewal a ime n converges, as n, o 1 m Proof Sep 1 Consider a Markov chain defined as follows: Le X n = inf{ n : is renewal ime} n The random variable X n (which is called he forward renewal ime) represens he lengh of he ime inerval beween n and he firs renewal following n (Please hink why X n has he Markov propery) Noe ha a renewal imes we have X n = 0 The sae space of his chain is E = {0, 1,, M 1}, if M <, E = {0, 1, 2, }, if M =, where M is he maximal value which he ξ k s can aain: M = sup{i N : r i > 0} N { } The ransiion probabiliies of his Markov chain are given by p i,i 1 = 1 for i = 1, 2,, M 1, p 0,i = r i+1 for i = 1,, M 1 7

8 Sep 2 We prove ha he chain is irreducible Saring a any sae i E we can reach sae 0 by following he pah i i 1 i 2 0 So, every sae leads o sae 0 Le us prove ha conversely, sae 0 leads o every sae Le firs M be finie Saring in sae 0 we can reach any sae i E wih posiive probabiliy by following he pah 0 M 1 M 2 i If M is infinie, hen for every i E we can find some K > i such ha r K > 0 Saring a sae 0 we can reach sae i by following he pah 0 K 1 K 2 i We have shown ha every sae leads o 0 and 0 leads o every sae, so he chain is irreducible Sep 3 We prove ha he chain is aperiodic By irreducibiliy, we need o show ha sae 0 is aperiodic For every i such ha r i 0 we can go from 0 o 0 in i seps by following he pah 0 i 1 i 2 0 By (341) he greaes common divisor of all such i s is 1, so he period of sae 0 is 1 and i is aperiodic Sep 4 We claim ha he unique invarian probabiliy measure of his Markov chain is given by λ i = r i+1 + r i+2 +, i E m Indeed, he equaions for he invarian probabiliy measure look as follows: I follows ha λ j = M 1 i=0 We obain he following equaions: p ij λ i = p 0,j λ 0 + p j+1,j λ j+1 = r j+1 λ 0 + λ j+1 λ j λ j+1 = r j+1 λ 0 λ 0 λ 1 = r 1 λ 0, λ 1 λ 2 = r 2 λ 0, λ 2 λ 3 = r 3 λ 0, By adding all hese equaions saring wih he (j + 1)-s one, we obain ha λ j = (r j+1 + r j+2 + )λ 0 I remains o compue λ 0 By adding he equaions for all j = 0, 1,, M 1 we obain ha 1 = λ 0 + λ 1 + = (r 1 + 2r 2 + 3r 3 + )λ 0 = mλ 0 8

9 I follows ha λ 0 = 1 m This proves he formula for he invarian probabiliy disribuion Sep 5 Our chain is hus irreducible, aperiodic, and posiive recurren By he heorem on he convergence o he invarian probabiliy disribuion we have lim P[X n = 0] = λ 0 = 1 n m Recalling ha we have X n = 0 if and only if n is a renewal ime, we obain ha lim P[ k N : S n = k] = lim P[X n = 0] = 1 n n m, hus proving he claim of he heorem 9