Worked examples Random Processes

Transcription

1 Worked examples Random Processes Example 1 Consider patients coming to a doctor s office at random points in time. Let X n denote the time (in hrs) that the n th patient has to wait before being admitted to see the doctor. (a) Describe the random process X n, n 1. (b) Sketch a typical sample path of X n. Solution (a) The random process X n is a discrete-time, continuous-valued random process. The sample space is S X {x : x 0}. The index parameter set (domain of time) is (b) A sample path: I {1,,, }. Example Let {X n, n 1,, } be a sequence of independent random variables with S Xn {0, 1}, P X n 0], P X n 1] 1 n. Let Y n X i. (a) Find the 1 st order pmf of Y n. (b) Find m Y (n), R Y (n, n + k), C Y (n, n + k). Solution Note than X n is an iid random process and for each fixed n, X n is a Bernoulli random variable with p 1. We have EX n ] p 1 1

2 VARX n ] p(1 p) 9 EX n] VARX n ] + EX n ] 1. (a) For each fixed n, Y n is a binomial random variable with p 1. We obtain P Y n k] C n k ( ) k ( ) n k 1, k 0, 1,, n. (b) m Y (n) EY n ] np n n n+k R Y (n, n + k) EY (n)y (n + k)] E X i n n+k j1 EX i X j ] n n+k EX i ]EX j ] + EXi ] j1 j i n ( n + k ) 9 n n + k + 9 n (n + k + ). 9 j1 X j C Y (n, n + k) R Y (n, n + k) m Y (n)m Y (n + k) n(n + k + ) 9 One can deduce that C Y (n, m) min(n, m) 9. n n + k n 9. Example The number of failures N(t), which occur in a computer network over the time interval 0, t), can be described by a homogeneous Poisson process {N(t), t 0}. On an average, there is a failure after every hours, i.e. the intensity of the process is equal to λ 0.5h 1 ].

3 (a) What is the probability of at most 1 failure in 0, 8), at least failures in 8, 16), and at most 1 failure in 16, ) (time unit: hour)? (b) What is the probability that the third failure occurs after 8 hours? Solution (a) The probability p P N(8) N(0) 1, N(16) N(8), N() N(16) 1] is required. In view of the independence and the homogeneity of the increments of a homogeneous Poisson process, it can be determined as follows: Since p P N(8) N(0) 1]P N(16) N(8) ]P N() N(16) 1] P N(8) 1]P N(8) ]P N(8) 1]. P N(8) 1] P N(8) 0] + P N(8) 1] e e and P N(8) ] 1 P N(8) 1] 0.59, the desired probability is p (b) Let T be the time of arrival of the third failure. Then P T > 8] probability that there are two failures in the first 8 hours ( ) e (0.5 8) i i! i0 ( ) e ! + 5e ! Example Random Telegraph signal Let a random signal X(t) have the structure X(t) ( 1) N(t) Y, t 0,

4 where {N(t), t 0} is a homogeneous Poisson process with intensity λ and Y is a binary random variable with P (Y 1) P (Y 1) 1/ which is independent of N(t) for all t. Signals of this structure are called random telegraph signals. Random telegraph signals are basic modules for generating signals with a more complicated structure. Obviously, X(t) 1 or X(t) 1 and Y determines the sign of X(0). Since X(t) 1 < for all t 0, the stochastic process {X(t), t 0} is a secondorder process. Letting I(t) ( 1) N(t), its trend function is m(t) EX(t)] EY ]EI(t)]. Since EY ] 0, the trend function is identically zero: m(t) 0. It remains to show that the covariance function C(s, t) of this process depends only on t s. This requires the determination of the probability distribution of I(t). A transition from I(t) 1 to I(t) +1 or, conversely, from I(t) +1 to I(t) 1 occurs at those time points where Poisson events occur, i.e. where N(t) jumps. Hence the expected value of I(t) is P (I(t) 1) P (even number of jumps in 0, t]) e λt i0 (λt) i (i)! e λt cosh λt, P (I(t) 1) P (odd number of jumps in 0, t]) e λt i0 (λt) i+1 (i + 1)! e λt sinh λt. EI(t)] 1 P (I(t) 1) + ( 1) P (I(t) 1) e λt cosh λt sinh λt] e λt. Since C(s, t) COVX(s), X(t)] E(X(s)X(t))] EY I(s)Y I(t)] EY I(s)I(t)] E(Y )EI(s)I(t)] and E(Y ) 1, C(s, t) EI(s)I(t)]. Thus, in order to evaluate C(s, t), the joint distribution of the random vector (I(s), I(t)) must be determined. In view of the homogeneity of the increments of {N(t), t 0}, for

5 s < t, p 1,1 P (I(s) 1, I(t) 1) P (I(s) 1)P (I(t) 1 I(s) 1) e λs cosh λs P (even number of jumps in (s, t]) e λs cosh λs e λ(t s) cosh λ(t s) e λt cosh λs cosh λ(t s). Analogously, p 1, 1 P (I(s) 1, I(t) 1) e λt cosh λs sinh λ(t s) p 1,1 P (I(s) 1, I(t) 1) e λt sinh λs sinh λ(t s) p 1, 1 P (I(s) 1, I(t) 1) e λt sinh λs cosh λ(t s). Since EI(s)I(t)] p 1,1 + p 1, 1 p 1, 1 p 1,1, we obtain C(s, t) e λ(t s), s < t. Note that the order of s and t can be changed so that C(s, t) e λ t s. Example 5 A random process is defined by X(t) T + (1 t) where T is a uniform random variable in (0, 1). (a) Find the cdf of X(t). (b) Find m X (t) and C X (t 1, t ). Solution Given that X(t) T + (1 t), where T is uniformly distributed over (0, 1), we then have P X(t) x] P T x (1 t)]; { 0 y < 0 P T y] y 0 < y < 1. 1 y > 1 Write x (1 t) y, then { 0 x < 1 t F X (x) P X(t) x] P T x (1 t)] x (1 t) 1 t < x < t ; 1 x > t 5

6 Note that ET ] 1 and ET ] 1. f X (x) d { 1 1 t < x < t dx F X(x). 0 otherwise t t m X (t) x dx x t. 1 t Alternatively, m X (t) EX(t)] ET + (1 t)] 1 t + ET ] t. Define and observe R X (t 1, t ) E{T + (1 t 1 )}{T + (1 t )}] 1 t ET ] + (1 t t )ET ] + (1 t 1 )(1 t ) C X (t 1, t ) R X (t 1, t ) m X (t 1 )m X (t ) 1 + ( t 1 t ) (1 t 1 )(1 t ) ( ) ( ) t 1 t Example 6 Customers arrive at a service station (service system, queueing system) according to a homogeneous Poisson process {N(t), t 0} with intensity λ. The arrival of a customer is therefore a Poisson event. The number of servers in the system is assumed to be so large that an incoming customer will always find an available server. To cope with this situation the service system must be modeled as having an infinite number of servers. The service times of all customers are assumed to be independent random variables which are identically distributed as B. Let G(y) P (B y) be the distribution function of B and X(t) the random number of customers which gives the state of the system at time t, X(0) 0. The aim is to determine the state probabilities of the system According to the total probability rule, p i (t) P (X(t) i); i 0, 1, ; t 0. p i (t) P (X(t) i N(t) n) P (N(t) n) ni ni P (X(t) i N(t) n) (λt)n e λt. n! 6

7 A customer arriving at time x is with probability 1 G(t x) still in the system at time t, t > x, i.e. the service has not yet been finished by time t. Given N(t) n, the arrival times T 1, T,, T n of the n customers in the system are independent random variables, uniformly distributed over 0, t]. To calculate the state probabilities, the order of the T i is not important. Hence they can be assumed to be independent, unordered random variables, uniformly distributed over 0, t]. Thus, the probability that any one of those n customers is still in the system at time t is p(t) t 0 1 G(t x)] dx t 1 t t 0 1 G(x)] dx. Since the service times are independent of each other, we have ( ) n P X(t) i N(t) n] p(t)] i 1 p(t)] n i ; i 0, 1,, n. i The desired state probabilities can be obtained as follows: ( n p i (t) )p(t)] i 1 p(t)] n i (λt)n e λt i n! ni λtp(t)]i e λt i! k0 λt(1 p(t))] k λtp(t)]i e λt e λt1 p(t)] i! λtp(t)]i i! k! e λtp(t) ; i 0, 1,. Thus, {p i (t); i 0, 1, } is a Poisson distribution with intensity EX(t)] λtp(t). The trend function of the stochastic process {X(t), t 0} is, consequently, m(t) λ t 0 1 G(x)] dx. Let EY ] 1/λ be the expected interarrival time between two successive customers. Since EB] 0 1 G(x)] dx, we obtain EB] lim m(t) t EY ]. Thus, for t, the trend function and the state probabilities of the stochastic process {X(t), t 0} become constant. Letting ρ EB]/EY ], the stationary state probabilities of the system become p i lim t p i (t) ρi i! e ρ ; i 0, 1,. 7

8 In particular, if B is exponentially distributed with parameter µ, then In this case, ρ λ/µ. t m(t) λ e µx dx λ 0 µ (1 e µt ). Example 7 (a) EX (t)] Let X(t) be a Poisson process with parameter λ. Find (b) E{X(t) X(s)} ], for t > s. Solution (a) Note that X(t) is a Poisson random variable and we have EX(t)] VARX(t)] λt, t 0, EX (t)] VARX(t)] + EX(t)] λt + λ t. (b) Since X(t) has stationary increments, so X(t) X(s) and X(t s) have the same distribution, that is, Hence, EX(t) X(s)] EX(t s)] λ(t s) VARX(t) X(s)] VARX(t s)] λ(t s). E{X(t) X(s)} ] VARX(t) X(s)] + EX(t) X(s)] λ(t s) + λ (t s), t > s. Example 8 Patients arrive at the doctor s office according to a Poisson process with rate λ 1 minute. The doctor will not see a patient until at least three patients are in 10 the waiting room. (a) Find the expected wating time until the first patient is admitted to see the doctor. (b) What is the probability that nobody is admitted to see the doctor in the first hour? Solution 8

9 (a) Let Z n inter-event time time between arrival of the n th and (n 1) th patients. Then Z n s are iid exponential random variables with mean 1 λ, i.e., EZ n] 1 λ. Let T n arrival time of the n th patient n Z n. n ] ET n ] E Z n n EZ n ] n λ. The expected waiting time until the first patient is admitted to see the doctor is / 1 ET ] 10 0 minutes. (b) Let X(t) be the Poisson process with mean λt. Note that P X(t) k] (λt)k e λt, k! k 0, 1,. We have P {Nobody is admitted to see the docotr in the 1 st hr}] P {At most patient arrive in first 60 mins}] P X(t) over 0, 60]] P X(60) ] P X(60) 0] + P X(60) 1] + P X(60) ] ( ) 60 e 60/10 + e 60/ ( ) 60 e 60/ e 6 ( ) Example 9 A message requires N time units to be transmitted, when N is a geometric random variable with pmf P N (j) (1 a)a j 1, j 1,,. A single new message arrives during a time unit with probability p and no message arrive with probability 1 p. Let K be the number of new messages that arrive during the transmission of a single message. (a) Find the pmf of K. (b) Use conditional expectation to find EK] and EK ]. 9

10 Solution Using the law of total probabilities, we have P K k] P K k N n]p N (n) n1 Note that P K k N n] 0 for k > n since the number of messages arriving during the transmission time must be less than or equal to n. We have P K k N n] nc k p k (1 p) n k, n k, since this is a binomial experiment with n trials and probability of success in each trial is p. Hence P K k] Using the formula: we then obtain P K k N n]p N (n) nk (1 β) (k+1) P K k] nc k p k (1 p) n k (1 a)a n 1. nk nc k β n k, 0 < β < 1, nk ] k 1 a ap, k 1. a1 a(1 p)] 1 a(1 p) (b) We start with the conditional expectation formulas: EK] EK N n]p N (n) and EK ] n1 EK N n]p N (n). n1 Note that K is a binomial variable with parameters p and n; and so EK N n] np, EK N n] VARK N n] + EK N n] np(1 p) + n p. We obtain and Remark EK] np(1 a)a n 1 n1 EK ] np(1 p) + n p ](1 a)a n 1. n1 10

11 To find the sum, we use the following identities: x+x + x+x +x + d ( ) x dx 1 x x 1 x and 1+ x+ x + x + d dx ; consequently, 1+ x d ( )] x. dx 1 x Example 10 Two gamblers play the following game. A fair coin is flipped; if the outcome is heads, player A pays player B $1, and if the outcome is tails player B pays player A $1. The game is continued until one of the players goes broke. Suppose that initially player A has $1 and player B has $, so a total of $ is up for grabs. Let X n denote the number of dollars held by player A after n trials. a. Show that X n is a Markov chain. b. Sketch the state transition diagram for X n and give the one-step transition probability matrix P. c. Use the state transition diagram to help you show that for n even (i.e., n k), ( ) n 1 p ii (n) for i 1, p 10 (n) ( ) ] k 1 1 p (n). d. Find the n-step transition probability matrix for n even using part c. e. Find the limit of P n as n. Solution f. Find the probability that player A eventually wins. (a) Note that S {0, 1,, } and we need to verify P X n+1 j X n i, X n 1 x n 1,, X 0 x 0 ] P X n+1 j X n i] for all i, j S. Consider the following three cases: i 1, ; i 0 and i. When i 1,, P X n+1 j X n i, X n 1 x n 1,, X 0 x 0 ] P The outcome is H], if j i 1 P The outcome is T ], if j i + 1 0, if j i 1, i , if j i 1 1, if j i + 1 0, if j i 1, i + 1 P X n+1 j X n i]. 11

12 When i 0, P X n+1 j X n 0, X n 1 x n 1,, X 0 x 0 ] { P Xn+1 0 A has gone broke], if j 0 0, if j 0 { 1, if j 0 0, if j 0 P X n+1 j X n 0]. When i, P X n+1 j X n, X n 1 x n 1,, X 0 x 0 ] { P Xn 1 A has won], if j 0, if j { 1, if j 0, if j P X n+1 j X n ]. In summary, X n is a Markov Chain. (b) The transition probability matrix is (c) Note that for n k, which is even, so P The transition 1 followed {X k 1 X 0 1} by transition 1 and this is repeated k n times p 11 (n) (p 1 p 1 )(p 1 p 1 ) (p 1 p 1 ) }{{} k n pairs ( 1 1 ( ) n 1. 1 ) n

13 Similarly, p (n) (p 1 p 1 )(p 1 p 1 ) (p 1 p 1 ) }{{} k n pairs ( ) n 1 State 0 can only change to state 0 in every step during the period (i 1) + 1, n], so {X k 0 X 0 1} n k X (i 1) 1 X 0 1 followed by a one-step transition, where X (i 1)+1 0 X (i 1) 1 Similarly, p 10 (n) P X k 0 X 0 1] k P X (i 1) 1 X 0 1]P X (i 1)+1 0 X (i 1) 1] 1 1 k p 11 ((i 1))p 10 k k ( ) (i 1) 1 ( ) i 1 1 ) k 1 1 ( ( ) ] k 1 1. p (n) P X k X 0 ] k P X (i 1) X 0 ]P X (i 1)+1 X (i 1) ] 1

14 1 k p ((i 1))p k p 10 (n). ( ) (i 1) 1 (d) We compute the remaining n-step transition probabilities p ij (n) row by row for n being even. 1 st row, i 0: Since p 00 (n) 1 as state 0 always changes to state 0 p 0j (n) 1, so p 01 (n) p 0 (n) p 0 (n) 0 j0 nd row, i 1: p 1 (n) 0 as n is even p 1 (n) 1 p ij (n) 1 j0 { 1 ( ) ] k 1 + ( ) k 1 + 0} 1 1 ( ) ] k 1 rd row, i : p 1 (n) 0 as n is even p 0 (n) 1 p j (n) 1 j1 { 0 + ( ) k ( ) ]} k ( ) ] k 1 th row, i : Since p (n) 1 as state always changes to state p j (n) 1 so p 0 (n) p 1 (n) p (n) 0. j0 1

15 In summary, for n k P (n) ( ] 1 k ( 1 ) k 1 ) 0 1 ( ) ] 1 k 1 ( ) ] 1 k ( 0 1 ) k 1 ( ) ] 1 k (e) For n k + 1, that is, n is odd, 1 P (n) P (k + 1) P k P ( ) ] 1 k ( ) k ( ) k 1 1 ( ) ] 1 k P (n) 1 1 ( ) ] 1 k ( 1 1 ) k 0 1 ( ) ] 1 k ( k ) Note that ( ) k 1 0 as k, we have lim P (k) k lim P (k + 1), k so (f) Note that lim P (n) n lim p 1(k) lim p 1 1(k + 1) lim 1 k k k ( ) ] k 1 1 so lim n p 1(n) 1. P Player A eventually wins] lim P X n X 0 1] n lim P 1(n) n 1. 15