Lecture 3 Basic Probability and Statistics

Transcription

1 Lecture 3 Bsic Probbility nd Sttistics The im of this lecture is to provide n extremely speedy introduction to the probbility nd sttistics which will be needed for the rest of this lecture course. The mjority of mthemtics students should lredy be fmilir with this mteril. Definition 1. A smple spce is the set of ll possible outcomes from n experiment. For exmple, if we consider tossing two coins, the possible outcomes re HH, HT, TH nd TT. The smple spce my be discrete (s in the previous exmple) or continuous (for exmple mesurement of person s height in metres). Formlly, discrete smple spce is one with finite or countbly infinite number of possible vlues. A continuous smple spce is one which tkes vlues in one or more intervls. Exmple One: The number of bytes counted pst point in network in second is disrete smple spce with the possible outcomes between 0 nd the bndwidth of the link in bytes per second. Exmple Two: The number of bytes counted t the queues of n nodes in network is discrete smple spce with the possible outcomes in Z n +. Definition 2. An event is subset of smple spce md simple event is one member of the smple spce. Often n event is described in words rther thn by explicit enumertion of the subspce. For exmple, if the event is getting exctly one hed in two coin tosses then it would be the subset HT nd TT. An exmple of n event on continuous smple spce is mesuring height which is between 1.5 nd 2.0 metres. Definition 3. A probbility mesure P is rel-vlued set function defined on smple spce S which stisfies 1. 0 P [ A ] 1 for every event A 2. P [ S ] = 1 3. P [ A 1 A 2... ] = P [ A 1 ] + P [ A 2 ] +... for every finite or infinite sequence of disjoint events A 1, A 2,.... Exmple: The probbility of throwing two or more heds when throwing three unbised coins is the probbility of the event {HHH HHT HTH THH}. This is union of four disjoint simple events nd is (4/8 = 1/2) since there re eight possibilities in totl ech with equl probbility 1/8. Exmple: The Poisson distribution is given by P [ X = x ] = λx e λ x Z +. It is left s n exercise for the student to show tht the first nd second condition re both met. Tht is, 0 λx e λ 1 x Z +, nd λ x e λ = 1. x=0 1

2 The nottion P [ A, B ] refers to the probbility tht events A nd B both occur lso known s the joint probbility. The nottion P [ A B ] is the probbility tht event A occurs given tht event B occurs or the conditionl probbility. Exmple: If A is the event the first coin shows H nd B is the event more thn two heds re thrown then of the bove exmples, A nd B both occur for HHH, HHT nd HTH there re four possibilities which hve A true HTT is the remining one. Therefore P [ B A ] = 3/4 by coincidence P [ A B ] = 3/4 s well in this cse (this is not generlly true). In this cse P [ A, B ] = 3/8. Theorem 1. Byes theorem sttes tht P [ A, B ] = P [ B ] P [ A B ] The reson for this cn be trivilly seen. The probbility of A nd B is the probbility of B multiplied by the probbility of A given tht B hs occurred. Definition 4. A rndom vrible is rel-vlued function defined on smple spce. For exmple, X might be the number of heds in two coin tosses or the height of given mesurement in metres. The domin of X is the smple spce nd its rnge is within the rel numbers R. A discrete rndom vrible is rndom vrible defined on discrete smple spce nd continuous rndom vrible is rndom vrible defined on continuous smple spce for which the probbility is zero tht it will ssume ny given vlue in n intervl. It should lso be noted tht it follows from these definitions tht rel-vlued function of rndom vrible (or set of rndom vribles) is itself rndom vrible. Definition 5. The discrete density function f(x) for discrete rndom vrible X is given by the eqution f(x) = P [ X = x ]. The distribution function (sometimes clled the cumultive distribution function) F (x) for discrete rndom vrible X is given by F (x) = P [ X x ] = y x P [ X = y ]. Definition 6. The continuous density function f(x) for continuous rndom vrible X is uniquely determined by the following properties: 1. f(x) 0 for ll x R 2. f(x)dx = 1 3. b f(x)dx = P [ < x < b ] for ll, b R where b. Exmple: Consider the so clled flt or constnt distribution where the smple spce is some intervl (, b) nd P [ c < X < d ] (d c) where c d b. The third prt of the definition will give us tht { k < x < f(x) = 0 otherwise, where k is some constnt. From the second prt we get tht b kdx = 1, 2

3 nd hence k = 1/(b ). The distribution function (sometimes clled the cumultive distribution function) is the sum of the density function, where f(y) is the density function. F (x) = x f(y)dy, Exmple: For the flt distribution defined bove it is left s n exercise to the student to show tht 0 x F (x) = (x )/(b ) < x < b 1 x b. Often it is useful to del with more thn one rndom vrible t once. If two vribles X nd Y re considered then it is useful to know probbilities bout wht hppens with both vribles. Definition 7. The joint density function of two rndom vribles X nd Y is defined by f(x, y). In the discrete cse this is defined by the eqution f(x, y) = P [ X = x, Y = y ]. In the continuous cse it must possess the following properties: 1. f(x, y) 0 2. f(x, y)dxdy = 1 3. b d f(x, y)dxdy = P [ < X < b nd c < Y < d ], for ll b R nd c d R. c Definition 8. The rndom vribles X nd Y with density functions g(x) nd h(x) nd the joint density function f(x, y) re sid to be independent if nd only if for ll x nd y. f(x, y) = g(x)h(y), Definitions 7 nd 8 cn be extended in the obvious wy to more thn two vribles. It should be noted, however, tht merely becuse ech pir of events is independent does NOT men tht the entire set of events is independent. It is instructive to come up with exmples where this is not the cse. Definition 9. The expected vlue or expecttion of the function g(x) on discrete rndom vrible X is given by E [ g(x) ] = g(x i )f(x i ), i=1 where x i re ll the possible vlues of X (tht is ll the members of its smple spce) nd f(x) is the density fucntion for X. For continuous vrible the sum in the bove chnges to n integrl. 3

4 Definition 10. The expected vlue or expecttion of the function g(x) on continuous rndom vrible X is given by E [ g(x) ] = where f(x) is the density function for X. g(x)f(x)dx, It should be noted tht in Defintions 9 nd 10 there is no gurntee tht either the sum or the integrl converge. If they diverge then the expecttion is undefined. We cn extend the definition of expecttion to set of rndom vribles X 1,..., X n. Definition 11. For rndom vribles X 1,..., X n with density function f(x 1,..., x n ) then the expecttion vlue of function h(x 1,..., X n ) is given by: E [ h ] =... h(x 1,..., x n )f(x 1,..., x n )dx 1... dx n. Expecttion E is liner opertor. If g, g 1 nd g 2 re three functions of set of rndom vribles then the following properties follow from the previous definitions: E [ cg ] = ce [ g ] for ny constnt c. E [ g 1 + g 2 ] = E [ g 1 ] + E [ g 2 ]. E [ g 1 g 2 ] = E [ g 1 ] E [ g 2 ], if g 1 nd g 2 re independent. The first two properties follow trivilly from substituting h = cg nd h = g 1 + g 2 into Definition 11. The third property is derived s follows: E [ g 1 g 2 ] = g 1 g 2 f(g 1, g 2 )dg 1 dg 2, where f(g 1, g 2 ) is the joint density function of g 1 nd g 2. Since g 1 nd g 2 re independent then from Definition 8 g 1 g 2 f(g 1, g 2 )dg 1 dg 2 = g 1 g 2 f 1 (g 1 )f 2 (g 2 )dg 1 dg 2 = g 1 f 1 (g 1 )dg 1 g 2 f 2 (g 2 )dg 2 = E [ g 1 ] E [ g 2 ], where f 1 (g 1 ) nd f 2 (g 2 ) re the density functions of g 1 nd g 2 respectively. Using the Definitions 9 nd 10 for expecttion then men µ nd vrince σ 2 of rndom vrible X cn be defined. Definition 12. The men µ of rndom vrible X (either discrete or continuous) is given by µ = E [ X ]. Definition 13. The vrince σ 2 of rndom vrible X (either discrete or continuous) is denoted by vr ( X ) nd is given by σ 2 = vr ( X ) = E [ (X µ) 2 ]. 4

5 Exmples: Show tht the flt distribution s previously defined, hs men ( + b)/2 nd vrince (b ) 2 /12. Show tht the Poisson distribution hs men λ nd vrince λ. As previously noted, the expecttion is not gurnteed to converge nd, for some rndom vribles, µ nd σ 2 do not exist. Instructive Exmple: Wht is the expected pyout of coin-tossing gme defined s follows. The plyer tosses coin until they get hed. If the first hed occurs on throw n then they re pid 2 n. Clculte the expected pyout. 5