Notes 8 Autumn Some discrete random variables

Transcription

1 MAS 108 Probability I Notes 8 Autum 2005 Some discrete radom variables We ow look at five types of discrete radom variables, each depedig o oe or more parameters. We describe for each type the situatios i which it arises, ad give the p.m.f., the expected value, ad the variace. If the variable is tabulated i the New Cambridge Statistical Tables [1], we give the table umber, ad some examples of usig the tables. You should have a copy of the tables to follow the examples. A summary of this iformatio is give o the course iformatio sheet etitled Discrete radom variables. Make sure that you have a copy of this sheet too. Before we begi, a commet o the New Cambridge Statistical Tables [1]. They do t give the probability mass fuctio (or p.m.f.), but a closely related fuctio called the cumulative distributio fuctio. It is defied for a discrete radom variable as follows. Let X be a radom variable takig values a 1, a 2,..., a. We assume that these are arraged i ascedig order: a 1 < a 2 < < a. The cumulative distributio fuctio, or c.d.f., of X is give by F X (a i ) = P(X a i ). We see that it ca be expressed i terms of the p.m.f. of X as follows: F X (a i ) = P(X = a 1 ) + + P(X = a i ) = I the other directio, we c recover the p.m.f. from the c.d.f.: P(X = a i ) = F X (a i ) F X (a i 1 ). i P(X = a j ). j=1 Usually the values of a radom variable i the tables are itegers startig at 0 or sometimes 1. I this case, the equatios become 1

2 F X (k) = P(X k) = P(X = 0) + P(X = 1) + + P(X = k), P(X = k) = F X (k) F X (k 1), P(X k) = 1 F X (k 1), P(k X l) = F X (l) F X (k 1). For example, the last equatio holds because we obtai the probability that k X l by workig out the probability of the values 0, 1,..., l ad subtractig the oes we do t wat: 0, 1,..., k 1. We wo t use the c.d.f. of a discrete radom variable except for lookig up the tables. It is much more importat for cotiuous radom variables! Beroulli radom variable Beroulli(p) A Beroulli radom variable is the simplest type of all. It oly takes two values, 0 ad 1. So its p.m.f. looks as follows: x 0 1 P(X = x) q p Here, p is the probability that X = 1; it ca be ay umber betwee 0 ad 1. Necessarily q (the probability that X = 0) is equal to 1 p. So p determies everythig. For a Beroulli radom variable X, we sometimes describe the experimet as a trial, the evet X = 1 as success, ad the evet X = 0 as failure. For example, if a biased coi has probability p of comig dow heads, the the umber of heads that we get whe we toss the coi oce is a Beroulli(p) radom variable. More geerally, let A be ay evet i a probability space S. With A, we associate a radom variable I A (remember that a radom variable is just a fuctio o S) by the rule { 1 if s A; I A (s) = 0 if s / A. The radom variable I A is called the idicator variable of A, because its value idicates whether or ot A occurred. It is a Beroulli(p) radom variable, where p = P(A). (The evet I A = 1 is just the evet A.) Some people write 1 A istead of I A. This shows that Beroulli radom variables are essetially the same thig as evets, so that if we wated to we could do all probability theory with radom variables ad ever metio evets! 2

3 Calculatio of the expected value ad variace of a Beroulli radom variable is easy. Let X Beroulli(p). (Remember that meas has the same p.m.f. as or is distributed as.) E(X) = 0 q + 1 p = p; Var(X) = 0 2 q p p 2 = p p 2 = pq. (Remember that q = 1 p.) Biomial radom variable Bi(, p) Remember that for a Beroulli radom variable, we describe the evet X = 1 as a success. Now a biomial radom variable couts the umber of successes i idepedet trials each associated with a Beroulli(p) radom variable. For example, suppose that we have a biased coi for which the probability of heads is p. We toss the coi times ad cout the umber of heads obtaied. This umber is a Bi(, p) radom variable. A Bi(, p) radom variable X takes the values 0, 1, 2,...,, ad the p.m.f. of X is give by P(X = m) = C m q m p m = b(m;, p) for m = 0, 1, 2,...,, where q = 1 p. This is because there are C m differet ways of obtaiig m heads i a sequece of throws (the umber of choices of the m positios i which the heads occur), ad the probability of gettig m heads ad m tails i a particular order is q m p m. Note that we have give a formula rather tha a table here. For small values we could tabulate the results; for example, for Bi(3, p): m P(X = m) q 3 3q 2 p 3qp 2 p 3 Addig up the probabilities gives q 3 + 3q 2 p + 3qp 2 + p 3 = (q + p) 3 = 1, sice q = 1 p. Moreover, we fid that E(X) = 0 q q 2 p + 2 3qp p 3 = 3p(q 2 + 2qp + p 2 ) = 3p(q + p) 2 = 3p, Var(X) = 0 2 q q 2 p qp p 3 (3p) 2 = 3p(q 2 + 4qp + 3p 2 ) 9p 2 3

4 = 3p(q + p)(q + 3p) 9p 2 = 3p(q + 3p) 9p 2 = 3pq. For arbitrary, whe we add up all the probabilities i the table, we get C m q m p m = (q + p) = 1, m=0 as it should be: here we used the biomial theorem (x + y) = C m x m y m. m=0 (This argumet explais the ame of the biomial radom variable!) The geeral formula for expected value ad variace of X Bi(, p) is: E(X) = p, Var(X) = pq. There are three ways to prove this. The first method is straightforward but hard, takig the explicit calculatios that we did above for = 3 ad makig them work for geeral. The secod method is more sophisticated, but relatively easy; it also works for very may radom variables whose values are itegers. However, you ca skip it if you wish: I have set it i smaller type for this reaso. We will see a third way (yet more sophisticated but eve easier) whe we have doe joit distributios. The secod method uses a gadget called the probability geeratig fuctio. We oly use it here for calculatig expected values ad variaces, but if you lear more probability theory you will see other uses for it. Let X be a radom variable whose values are o-egative itegers. (We do t isist that it takes all possible values; this method is fie for the biomial Bi(, p), which takes values betwee 0 ad.) To save space, we write p m for the probability P(X = m). Now the probability geeratig fuctio of X is the power series G X (x) = p m x m. (The sum is over all values m take by X.) It may be abbreviated to G(x) if it is obvious which radom variable we are talkig about. We use the otatio [F(x)] x=1 for the result of substitutig x = 1 i the series F(x). Propositio Let G(x) be the probability geeratig fuctio of a radom variable X. The (a) [G(x)] x=1 = 1; (b) E(X) = [ d dx G(x) ] x=1 ; [ ] (c) Var(X) = d 2 G(x) dx + E(X) 2 x=1 E(X)2. 4

5 Part (a) is just the statemet that probabilities add up to 1: whe we substitute x = 1 i the power series for G(x) we just get p m. For part (b), whe we differetiate the series term-by-term (you will lear later i Aalysis that this is OK), we get d dx G(x) = mp m x m 1. Now puttig x = 1 i this series we get For part (c), differetiatig twice gives Now puttig x = 1 i this series we get mp m = E(X). d 2 dx 2 G(x) = m(m 1)p m x m 2. m(m 1)p m = m 2 p m mp m = E(X 2 ) E(X). Addig E(X) ad subtractig E(X) 2 gives E(X 2 ) E(X) 2, which by defiitio is Var(X). Now let us appply this to the biomial radom variable X Bi(, p). We have so the probability geeratig fuctio is p m = P(X = m) = C m q m p m, C m q m p m x m = (q + px), m=0 by the Biomial Theorem. Puttig x = 1 gives (q + p) = 1, i agreemet with the Propositio. Differetiatig oce, usig the Chai Rule, we get p(q + px) 1. Puttig x = 1 we fid that E(X) = p. Differetiatig agai, we get ( 1)p 2 (q + px) 2. Puttig x = 1 gives ( 1)p 2. Now addig E(X) E(X) 2, we get Var(X) = ( 1)p 2 + p 2 p 2 = p p 2 = pq. The biomial radom variable is tabulated i Table 1 of the Cambridge Statistical Tables [1]. As explaied earlier, the tables give the cumulative distributio fuctio. For example, suppose that the probability that a certai coi comes dow heads is If the coi is tossed 15 times, what is the probability of five or fewer heads? Turig to the page = 15 i Table 1 ad lookig at the row 0.45, you read off the aswer What is the probability of exactly five heads? This is P(5 or fewer) P(4 or fewer), ad from tables the aswer is =

6 The tables oly go up to p = 0.5. For larger values of p, use the fact that the umber of failures i Bi(, p) is equal to the umber of successes i Bi(,1 p). So the probability of five heads i 15 tosses of a coi with p = 0.55 is = More formally, if X Bi(15,0.55), ad Y = 15 X, the Y Bi(15,0.45). Aother iterpretatio of the biomial radom variable cocers samplig. Suppose that we have N balls i a box, of which M are red. We sample balls from the box with replacemet; let the radom variable X be the umber of red balls i the sample. What is the distributio of X? Sice each ball has probability M/N of beig red, ad differet choices are idepedet, X Bi(, p), where p = M/N is the proportio of red balls i the sample. What about samplig without replacemet? This leads us to our ext radom variable: Hypergeometric radom variable Hg(,M,N) Suppose that we have N balls i a box, of which M are red. We sample balls from the box without replacemet. Let the radom variable X be the umber of red balls i the sample. Such a X is called a hypergeometric radom variable Hg(,M,N). The radom variable X ca take ay of the values 0, 1, 2,...,. Its p.m.f. is give by the formula M C m N M C m P(X = m) = N. C For the umber of samples of balls from N is N C ; the umber of ways of choosig m of the M red balls ad m of the N M others is M C m N M C m ; ad all choices are equally likely. The expected value ad variace of a hypergeometric radom variable are as follows (we wo t go ito the proofs): E(X) = ( M N ), Var(X) = ( M N )( N M N )( ) N. N 1 You should compare these to the values for a biomial radom variable. If we let p = M/N be the proportio of red balls i the box, the E(X) = p, ad Var(X) is equal to pq multiplied by a correctio factor (N )/(N 1). I particular, if the umbers M ad N M of red ad o-red balls i the box are both very large compared to the size of the sample, the the differece betwee samplig with ad without replacemet is very small, ad ideed the correctio factor is close to 1. So we ca say that Hg(,M,N) is approximately Bi(,M/N) if is small compared to M ad N M. Cosider the example from the last otes of choosig five sheep from 24, of which 6 are shor. The umber X of shor sheep i the sample is a Hg(5,6,24) radom 6

7 variable. We calculated i the last otes that E(X) = ad Var(X) = , but oted that these figures were affected by roudig errors. The formulae above show that the exact values should be E(X) = 5 4 ad Var(X) = = Geometric radom variable Geom(p) The geometric radom variable is like the biomial but with a differet stoppig rule. We have agai a coi whose probability of heads is p. Now, istead of tossig it a fixed umber of times ad coutig the heads, we toss it util it comes dow heads for the first time, ad cout the umber of times we have tossed the coi. Thus, the values of the variable are the positive itegers 1, 2, 3,.... (I theory we might ever get a head ad toss the coi ifiitely ofte, but if p > 0 this possibility is ifiitely ulikely, i.e. has probability zero, as we will see.) We always assume that 0 < p < 1. More geerally, the umber of idepedet Beroulli trials required util the first success is obtaied is a geometric radom variable. The p.m.f. of a Geom(p) radom variable is give by P(X = m) = q m 1 p, where q = 1 p. For the evet X = m meas that we get tails o the first m 1 tosses ad heads o the mth, ad this evet has probability q m 1 p, sice tails has probability q ad differet tosses are idepedet. Let s add up these probabilities: q m 1 p = p + qp + q 2 p + = p(1 + q + q 2 + ) = p m=1 1 q = 1, sice the series i paretheses is a geometric progressio with first term 1 ad commo ratio q, where q < 1. (Just as the biomial theorem shows that probabilities sum to 1 for a biomial radom variable, ad gives its ame to the radom variable, so the geometric progressio does for the geometric radom variable.) We calculate the expected value ad the variace usig the probability geeratig fuctio. If X Geom(p), the result will be that E(X) = 1/p, Var(X) = q/p 2. We have G(x) = m=1 q m 1 px m = px 1 qx, 7

8 agai by summig a geometric progressio. Differetiatig, we get Puttig x = 1, we obtai d (1 qx)p + pxq p G(x) = dx (1 qx) 2 = (1 qx) 2. E(X) = Differetiatig agai gives 2pq/(1 qx) 3, so p (1 q) 2 = 1 p. Var(X) = 2pq p p 1 p 2 = q p 2. For example, if we toss a fair coi util heads is obtaied, the expected umber of tosses util the first head is 2 (so the expected umber of tails is 1); ad the variace of this umber is also 2. Poisso radom variable Poisso(λ) The Poisso radom variable, ulike the oes we have see before, is very closely coected with cotiuous thigs. Suppose that icidets occur at radom times, but at a steady rate overall. The best example is radioactive decay: atomic uclei decay radomly, but the average umber λ which will decay i a give iterval is costat. The Poisso radom variable X couts the umber of icidets which occur i a give iterval. So if, o average, there are 2.4 uclear decays per secod, the the umber of decays i oe secod startig ow is a Poisso(2.4) radom variable. Aother example might be the umber of telephoe calls a miute to a busy telephoe umber, or the umber of people joiig the queue at the bus-stop i the ext miute. The p.m.f. for a Poisso(λ) variable X is give by the formula λ λm P(X = m) = e m! for m = 0, 1,.... It is derived from the biomial distributio. I do ot expect you to reproduce this derivatio, so I am puttig it i small type. Suppose that the icidets happe at the rate of λ per miute. Choose large eough that it is very ulikely for two or more icidets to happe i oe (1/)-th of a miute. The the umber that happe i such a small iterval of time is approximately Beroulli(p) for some suitable p. If the icidets i the differet parts of the miute are mutually idepedet ad X is the umber of icidets i oe miute the X Bi(, p). The E(X) = p. But we kow that E(X) = λ so p = λ/. 8

9 Now, P(X = m) = C m ( λ = = ) m ( 1 λ ) m! λ m ( m!( m)! m 1 λ ) 1 ( ) m 1 λ ( 1) ( 2) ( m + 1) λ m m! ( 1 λ ) 1 ( ) m 1 λ Now let ted to. Each of the m ratios i the first term teds to 1. The secod term has o i it, so it stays as λ m /m!. For the third term, we eed to use the fact that ( 1 λ ) e λ as, which you will lear i Calculus. I the fourth term, λ/ 0, so (1 λ/) 1 so the whole term teds to 1/1 m, which is 1. Thus the limitig value of P(X = m) is ideed e λ λ m /m!. I the lectures I gave the example of of bristles fallig out of my hairbrush whe I brush my (thick ad tagly) hair. Iitially there are N bristles, so it is reasoable to suppose that the radom variable X, which couts how may bristles fall out whe I brush my hair, is Bi(N, p) for some probability p. But what happes whe there are oly 10 bristles left? The X should be a radom variable with first parameter 10, but what should the probability be? It seems reasoable to suppose that I apply costat force K whe I brush my hair. With N bristles, that force is spread over all of them, so p should be (proportioal to) K/N. Whe there are oly 10 bristles, the same force is spread over just those 10 bristles, so the probablity should be (proportioal to) K/10. Thus, i geeral, whe there are bristles left the X Bi (, K Replacig K by λ ad takig the limit as gives the same result as above. Let s check that these probabilities add up to oe. We get ) λ m e λ = e λ e λ = 1, m! ( m=0 sice the expressio i brackets is the sum of the expoetial series. By aalogy with what happeed for the biomial ad geometric radom variables, you might have expected that this radom variable would be called expoetial. Ufortuately, this ame has bee give to a closely-related cotiuous radom variable which we will meet later. However, if you speak a little Frech, you might use as a memoic the fact that if I go fishig, ad the fish are bitig at the rate of λ per hour o average, the the umber of fish I will catch i the ext hour is a Poisso(λ) radom variable. ). 9

10 The expected value ad variace of a Poisso(λ) radom variable X are give by E(X) = Var(X) = λ. Agai we use the probability geeratig fuctio. If X Poisso(λ), the G(x) = m=0 λ (λx)m e = e λ(x 1), m! agai usig the series for the expoetial fuctio. Differetiatio gives λe λ(x 1), so E(X) = λ. Differetiatig agai gives λ 2 e λ(x 1), so Var(X) = λ 2 + λ λ 2 = λ. The lie graphs o the ext page illustrate how the biomial distributio teds towards the Poisso whe λ = 5. The cumulative distributio fuctio of a Poisso radom variable is tabulated i Table 2 of the New Cambridge Statistical Tables. So, for example, we fid from the tables that, if 2.4 fish bite per hour o average, the the probability that I will catch o fish i the ext hour is , while the probability that I catch at five or fewer is (so that the probability that I catch six or more is ). There is aother situatio i which the Poisso distributio arises. Suppose I am lookig for some very rare evet which oly occurs oce i 1000 trials o average. So I coduct 1000 idepedet trials. How may occurreces of the evet do I see? This umber is really a biomial radom variable Bi(1000, 1/1000). But we have see above that this is Poisso(1), to a very good approximatio. So, for example, the probability that the evet does t occur is about 1/e. The geeral rule is: If is large, p is small, ad p = λ, the Bi(, p) ca be approximated by Poisso(λ). For example, if the lake cotais 2400 fish ad if each fish bites idepedetly with probability 1/1000 i a hour, the the umber of fish I catch i a hour is Bi(2400,1/1000), which is approximately Poisso(2.4). [1] D. V. Lidley ad W. F. Scott, New Cambridge Statistical Tables, Cambridge Uiversity Press. 10

11 Bi(10, 0.5) Bi(25, 0.2) Bi(50, 0.1) Poisso(5)