`Will it snow tomorrow?' `What are our chances of winning the lottery?' What is the time between emission of alpha particles by a radioactive source?

Transcription

1 Notes for Chapter of DeGroot and Schervsh The world s full of random events that we seek to understand. x. `Wll t snow tomorrow?' `What are our chances of wnnng the lottery?' What s the tme between emsson of alpha partcles by a radoactve source? The outcome of these questons cannot be exactly predcted n advance. But the occurrence of these phenomena s not completely haphazard. The outcome s uncertan, but there s a regular pattern to the outcome of these random phenomena. x. If I flp a far con, I do not know n advance whether the result wll be a head or tal, but I can say n the long run approxmately half the tme the result wll be heads. Probablty theory s a mathematcal representaton of random phenomena. A probablty s a numercal measurement of uncertanty. Applcatons: Quantum mechancs Genetcs Fnance Computer scence (analyze the performance of computer algorthms that employ randomzaton) Actuaral scence (computng nsurance rsks and premums) Provdes the foundaton for the study of statstcs. The theory of probablty s often attrbuted to the French mathematcans Pascal and Fermat (~650). Began by tryng to understand games of chance. Gamblng was popular. As the games became more complcated and the stakes ncreased, there was a need for mathematcal methods for computng chance. These notes adapted from Martn Lndqust s notes for STAT W405 (Probablty).

2 Through hstory there have been a varety of dfferent defntons of probablty. Classcal Defnton: Suppose a game has n equally lkely outcomes of whch m outcomes (m<n) correspond to wnnng. Then the probablty of wnnng s m/n. x. A box contans 4 blue balls and 6 red balls. If I randomly pck a ball from the box, what s the probablty t s blue? n=0, m=4, P(blue ball) = 4/0 = 2/5. The classcal defnton requres a game to be broken down nto equally lkely outcomes. Ideal stuaton wth very lmted scope. Relatve Frequency Defnton: Repeat a game a large number of tmes under the same condtons. The probablty of wnnng s approxmately equal to the number of wns n the repeated trals. x. Flp a con 00,000 tmes. We get approxmately 50,000 heads. P(Heads) = ½. Frequency approach s consstent wth the classcal approach. Subjectve probablty s a degree of belef n a proposton. Axomatc Approach: Kolmogorov developed the frst rgorous approach to probablty (~930). He bult up probablty theory from a few fundamental axoms. The book and ths course wll use these axoms to buld up the theory of probablty. Modern research n probablty s closely related to measure theory. Ths course avods most of these ssues and stresses ntuton.

3 Sample Space and vents One of the man objectves of statstcs and probablty theory s to draw conclusons about a populaton of objects by conductng an experment. We must begn by dentfyng the possble outcomes of an experment, or the sample space. Defnton: The set, S, of all possble outcomes from a random experment s called the sample space. x. Flp a con. Two possble outcomes: Heads (H) or Tals (T). Thus, S={H, T}. x. Roll a de. Sx possble outcomes: S={, 2, 3, 4, 5, 6}. x. Roll two dce. 36 possble outcomes: S={(,j) =,.6, j=,.6 }. x. Lfetme of a battery. S = {x: 0 x< } Defnton: An event s any collecton of possble outcomes of an experment, that s, any subset of S (ncludng S tself). If the outcome of the experment s contaned n an event, then we say that has occurred. x. Flp two cons. Four possble outcomes: HH, HT, TH, TT. Thus, S={HH, HT, TH, TT}. An event could be obtanng at least one H n the two tosses. Thus, ={HH, HT, TH}. Note does not contan TT. x. Battery example. = {x: x<3} (The battery lasts less than 3 hours.)

4 Set Theory Gven any two events (or sets) A and B, we have the followng elementary set operatons: Unon: The unon of A and B, wrtten A B, s the set of elements that belong to ether A or B or both: A B = {x: x A or x B} If A and B are events n an experment wth sample space S, then the unon of A and B s the event that occurs f and only f A occurs or B occurs. Venn dagram The sample space s all the outcomes n a large rectangle. The events are represented as consstng of all the outcomes n gven crcles wthn the rectangle. Intersecton: The ntersecton of A and B, wrtten A B or AB, s the set of elements that belong to both A and B: AB = {x: x A and x B} If A and B are events n an experment wth sample space S, then the ntersecton of A and B s the event that occurs f and only f A occurs and B occurs Complementaton: The complement of A, wrtten A C, s the set of all elements that are not n A: A C = {x: x A } If A s an event n an experment wth sample space S, then the complement of A s the event that occurs f and only f A does not occur. x. Select a card at random from a standard deck of cards, and note ts sut: clubs (C), damonds (D), hearts (H) or spades (Sp). The sample space s S={C,D,H,Sp}. Some possble events are A={C,D}, B={D,H,Sp} and C={H}. A B ={C,D,H,Sp} = S AB ={D} A C ={H,Sp} AC = (null event event consstng of no outcomes)

5 If AB = then A and B are sad to be mutually exclusve. For any two events A and B, f all of the outcomes n A are also n B, then we say that A s contaned n B. Ths s wrtten A B. x. Roll a far de. S = {,2,3,4,5,6} Let ={,3,5} and F={}, then F. Note that f A B then the occurrence of A mples the occurrence of B. Also f A B and B A then A=B. The operatons of formng unons, ntersectons and complements of events follow certan basc laws. Basc laws Commutatve Laws: F= F F=F Assocatve Laws: Dstrbutve Laws: ( F) G = (F G) (F)G=(FG) ( F) G = G FG (F) G=( G)(F G) DeMorgan s Laws (A B) C =A C B C (*) (AB) C =A C B C

6 Proof of (*): x (A B) C x A B x A and x B x A C and x B C x A C B C Hence, (A B) C A C B C x A C B C x A C and x B C x A and x B x A B x (A B) C Hence, A C B C (A B) C Snce (A B) C A C B C and A C B C (A B) C, t must hold that A C B C = (A B) C The proofs of the other laws are done n a smlar manner. Often we are nterested n unons and ntersectons of more than two events. U Defnton: A = { x x! Afor atleast one! I} and! I I! I Generalzed DeMorgan s Laws: " # (a) $ U A % = I ( A ) &! I '! I " # (b) $ I A % = U ( A ) &! I '! I C C A = { x x! Afor every! I} C C

7 Axoms of Probablty Consder an experment whose sample space s S. For each event of the sample space S we assume that a number P() s defned and satsfes the followng three axoms. Axom : 0 <= P() <= Axom 2: P(S) = Axom 3: For any sequence of mutually exclusve events, 2,,!! " # P$ U % = ( P( ). & = ' = We refer to P() as the probablty of the event. x. For any fnte sequence of mutually exclusve events, 2,.. n, n n! " P# U $ = ' P( ) % = & = Proposton : P( C ) = -P() Proof: and C are mutually exclusve and S= C. By Axom 2: P(S) = By Axom 3: P(S) = P( C ) = P() + P( C ) Hence, P( C ) = - P() x. Let =S. Then C =. P( ) = - P(S) = - = 0

8 x. Roll a far dce 3 tmes. What s the probablty of at least one sx, f we know that the probablty of gettng 0 sxes s 0.579? P(at least one sx) = - P(0 sxes) = 0.42 Proposton 2: If F, then P() <= P(F). Proof: Snce F, F= C F. and C F are mutually exclusve, Axom 3 gves P(F) = P() + P( C F). But P( C F)>=0 by Axom. Hence, P(F) >= P() Proposton 3: P( F) = P() + P(F) P(F) Proof: F = C F and C F are mutually exclusve. P( F) = P( C F) = P() + P( C F) P( C F) = P( F) P() F = F C F F and C F are mutually exclusve. P(F) = P(F C F) = P(F) + P( C F) P( C F) = P(F) P(F) Together, P(F) P(F) = P( F) P()

9 P( F) = P() + P(F) P(F) x. A store accepts ether VISA or Mastercard. 50% of the stores customers have VISA, 30% have Mastercard and 0% have both. (a) What percent of the stores customers have a credt card the store accepts? A = customer has VISA B = customer has Mastercard P(A B) = P(A) + P(B) P(AB) = = 0.70 (b) What percent of the stores customers have exactly one of the credt cards the store accepts? P(A B) P(AB) = = 0.60 (c) What percent of the stores customers doesn t have a credt card the store accepts? P(0 cards) = -P(at least one card) = = 0.30 Proposton 4: n r+ (! 2! L! ) = # ( ) " # ( ) ( ) ( ) + K+ " 2 # L 2 r = < < < L< P n P P P + L+ " n+ ( ) P( ) L 2 n < < L< 2 # n! 2 2 The summaton P( ) L 2 s taken over all (n r) possble subsets of sze r of r the set {,2,,n}. < 2 < L< r r x. Three events, F and G. P( F G) = P() + P(F) + P(G) P(F) P(G) P(FG) + P(FG)

10 Combnatoral Analyss Many problems n probablty theory can be solved by smply, countng the number of dfferent ways an event can occur. The mathematcal theory of countng s formally known as combnatoral analyss The Sum Rule: If a frst task can be done n m ways and a second task n n ways, and f these tasks cannot be done at the same tme, then there are m + n ways to do ether task x. A lbrary has 0 textbooks dealng wth statstcs and 5 textbooks dealng wth bology. How many textbooks can a student choose from f he s nterested n learnng about ether statstcs or bology? 0+5 = 25 x. At a cafe one can choose between three dfferent snacks: ce cream, cake or pe. Addtonally there are two dfferent beverages: coffee or tea. How many possble combnatons are there f we are allowed to choose one snack and one beverage? Tree dagram: IC, IT; CakeC, CakeT; PC,PT 6 dfferent possble combnatons. Basc prncple of countng: If two successve choces are to be made wth m choces n the frst stage and n choces n the second stage, then the total number of successve choces s n*m. (Ths s sometmes referred to as the product rule.) Proof: We can wrte the two choces as (,j), f the frst choce s and the second j. =, m and j=,.n numerate all the possble choces: (,), (,2), (,n) (2,), (2,2), (2,n) : : (m,), (m,2), (m,n) The set of possble choces conssts of m rows, each row consstng of n elements.

11 Hence m*n total choces. -- The generalzed basc prncple of countng: If r successve choces are to be made wth exactly n j choces at each stage, then the total number of successve choces s n*n2*..*nr x. A lcense plate conssts of 3 letters followed by 2 numbers. How many possble lcense plates are there? 26*26*26*0*0=,757,600 What f repettons of letters and numbers are not allowed? 26*25*24*0*9=,404,000

12 Permutatons An ordered arrangement of the objects n a set s called a permutaton of the set. x. How many ways can we rank three people: A, B and C? ABC, ACB, BAC, BCA, CAB, CBA ach arrangement s called a permutaton. There are 6 dfferent permutatons n ths case. _ We use the BPC to descrbe our selecton. We have three letters to choose from n fllng the frst poston, two letters reman to fll the second poston, and just one letter left for the last poston: 3x2x=6 dfferent orders are possble. Another way of wrtng ths s 3!. Defnton: For an nteger n>=0, n! = n(n-)(n-2)..3*2* 0! = Note that n! = n(n-)! The number of dfferent permutatons of n dstnct objects s n(n-)..=n! x. How many dfferent letter arrangements can be made of the word COLUMBIA. Ans. 8! x. How many ways can we choose a Presdent, Vce-Presdent and Secretary from a group consstng of fve people? Presdent can be chosen 5 dfferent ways Vce-Presdent can be chosen 4 dfferent ways, Secretary can be chosen 3 dfferent ways 5*4*3=20 The number of dfferent permutatons of r objects taken from n dstnct objects s n(n- ).(n-r+).

13 Note: n(n-).(n-r+) = n!/(n-r)! x. How many ways can we choose a Presdent, Vce-Presdent and Secretary from a group consstng of fve people f A wll only serve f he s Presdent? A serves *4*3=2 OR A doesn t serve 4*3*2=24 Total = 2+24 = 36 Fnally, what s the number of permutatons of a set of n objects when certan objects are ndstngushable from one another? x. How many permutatons of the letters n the set {M,O,M} can we make? There are 3! dfferent ways to arrange a three letter word, when all the letters are dfferent. But n ths case the two M s are ndstngushable from one another. Order the objects as f they were dstngushable. Rewrte as (M,O,M2) 3! dfferent ways to arrange these three letters. MOM2, MM2O, OMM2, OM2M, M2OM, M2MO But MOM2 s the same as M2OM when the subscrpts are removed. We need to compensate for ths. How many ways can we arrange the 2 M s n the two slots. (2!) Dvde out the arrangements that look dentcal. 3!/2!=3 dstnct arrangements. In general there are n!/n! nr! dfferent permutatons of n objects, of whch n are alke, n2 are alke, etc.. x. How many dfferent letter arrangements can be made from the letters MISSISSIPPI?

14 M, 4 I s, 4 S s, 2 P s (!/(!4!4!2!) = 34,650

15 Combnatons A permutaton s an ordered arrangement of r objects from a set of n objects. x. How many ways can we choose a Presdent, Vce-Presdent and Secretary from a group consstng of fve people? Presdent can be chosen 5 dfferent ways Vce- Presdent can be chosen 4 dfferent ways, Secretary can be chosen 3 dfferent ways 5!/2! A Combnaton s an unordered arrangement of r objects from a set of n objects. x. Suppose nstead of pckng a Presdent, Vce-Presdent and Secretary we would just lke to pck three people out of the group of fve to be on the leadershp board,.e. how many teams of three can we create from a larger group consstng of fve people? In ths case order does not matter. ach combnaton of three people s counted 3! tmes. We need to compensate for ths. 5!/(2!3!) When countng permutatons abc and cab are dfferent. When countng combnatons they are the same. Notaton: n choose r (n r) = n!/(n-r)!r! for 0<=r<=n. (n r) represent the number of ways we can choose groups of sze r from a larger group of sze n, when the order of selecton s not relevant. SHOW AT HOM: Property : (n r) = (n (n-r)). Property 2: (n r) = ((n-) r) + ((n-) (r-)). x. How many ways can I choose 5 cards from a deck of 52. (52 5) =2,598,960 (Total number of poker hands)

16 x. (Q.20 p.7) A person has 8 frends, of whom 5 wll be nvted to a party. (a) How many choces are there f 2 of the frends are feudng and wll not attend together? (b) How many choces f 2 of the frends wll only attend together? (a) One of the feudng frends attends and the other does not. (2 )(6 4) or Nether of the feudng frends attend. (6 5) Total = 2*5+6 = 36 (b) The two frends attend together. (6 3) = 20 or The two frends do not attend. (6 5) = 6 Total = 20+6 = 26 x. A commttee of 6 people s to be chosen from a group consstng of 7 men and 8 women. If the commttee must consst of at least 3 women and at least 2 men, how many dfferent commttees are possble? 3 women/ 3 men: (8 3)*(7 3) or 4 women/ 2 men: (8 4)*(7 2) Total = (8 3)*(7 3) + (8 4)*(7 2) = The symbol (n r) s sometmes called a bnomal coeffcent.

17 The bnomal theorem If x and y are varables and n s a postve nteger, then (x+y)^n = sum_{k=0}^{n} (n k) x^k y^(n-k) x. (x+y)^2 = (2 0) x^0y^2 + (2 ) x^y^ + (2 2) x^2y^0 = y^2 + 2xy + x^2 x. Prove that sum_{k=0}^{n} (n k) = 2^n. Set x= and y=. By bnomal theorem (+)^n = sum_{k=0}^{n} (n k) ^k ^(n-k) 2^n = sum_{k=0}^{n} (n k) Multnomal Coeffcents x. How many ways are there of assgnng 0 polce offcers to 3 dfferent tasks, f 5 must be on patrol, 2 n the offce and 3 on reserve? (0 5) (5 2) (3 3) = 0!/(5!2!3!) In general there wll n!/(n!n2! nr!) ways to dvde up n objects nto r dfferent categores so that there wll be n objects n category, n 2 n category 2, and so on, where (n+ nr=n). Notaton: If n+n2+ +nr=n, we defne (n n,n2,.nr) = n!/(n!.nr!) The symbol (n n,n2, nr) s sometmes called a multnomal coeffcent. The multnomal theorem (x+x2+ +xr)^n = sum_{(n,.,nr): n+.+nr=n} (n n,n2,.nr) x^n x2^n2 xr^nr That s, the sum s over all nonnegatve nteger-valued vectors (n,.,nr) such that n+.+nr=n.

18 Sample Spaces havng equally lkely outcomes The assgnment of probabltes to sample ponts can be derved from the physcal set-up of an experment. Consder the scenaro that we have N sample ponts, each equally lkely to occur. Then each has a probablty of /N. xample: a far con, a far dce or a poker hand. Assume an event consstng of M sample ponts. The probablty of the event occurrng s M/N. x. Flp two far cons. S = {HH, HT, TH, TT}. Our sample space conssts of 4 ponts, each of whch s equally lkely to occur. P(HH) = /4. Let = at least one head = {HH, HT, TH}. P() = ¾. Let F= exactly one head = {HT, TH}. P() = 2/4 = /2. x. Roll two dce. S={(,j) =,.6, j=,.6 } There are 36 possble outcomes. What s the probablty that the total of the two dce s less than or equal to 3? = sum of the two dce are less than or equal to 3. (,), (,2), (2,) P() = 3/36 = /2. x. Two cards are chosen at random from a deck of 52 playng cards. What s the probablty that they (a) are both aces? (b) are both spades? (a) Total number of ways to choose two cards - (52 2) Total number of ways to choose two aces - (4 2)

19 P(two aces) = (4 2)/(52 2) = /22 (b) Total number of ways to choose two cards - (52 2) Total number of ways to choose two spades - (3 2) P(two spades) = (3 2)/(52 2) = /7 x. Poker hands. Choose a fve-card poker hand from a standard deck of 52 playng cards. What s the probablty of four aces? There are (52 5) = 2,598,960 possble hands. Havng specfed that four cards are aces, there are 48 dfferent ways of specfyng the ffth card. P(four aces) = 48/2,598,960. What s the probablty of a full house (one 3 of a knd + one par)? There are 3 ways to pck ways to choose the denomnaton of the three of a knd. There are (4 3) dfferent ways to choose three cards of that denomnaton. There are 2 ways to pck ways to choose the denomnaton of the par. There are (4 2) dfferent ways to choose two cards of that denomnaton. P(full house) = 3*(4 3)*2*(4 2)/(52 5) = 3,744/2,598,960

20 Suppose you flp a con and bet that t wll come up tals. Snce you are equally lkely to get heads or tals, the probablty of tals s 50%. Ths means that f you try ths bet often, you should wn about half the tme. What f somebody offered to bet that at least two people n your math class had the same brthday? Would you take the bet? x. The Brthday Problem. What s the probablty that at least two people n ths class share the same brthday? Assumptons: 365 days n a year. (No leap year). ach day of the year has an equal chance of beng somebodes brthday (Brthdays are evenly dstrbuted over the year). = no two people share a brthday. C = at least two people share a brthday. Use Proposton from last tme: P(at least two people share brthday) = - P(no two people share brthday). What s the probablty that no two people wll share a brthday? The frst person can have any brthday. The second person's brthday has to be dfferent. There are 364 dfferent days to choose from, so the chance that two people have dfferent brthdays s 364/365. That leaves 363 brthdays out of 365 open for the thrd person. To fnd the probablty that both the second person and the thrd person wll have dfferent brthdays, we have to multply: (365/365) * (364/365) * (363/365) = If we want to know the probablty that four people wll all have dfferent brthdays, we multply agan: (365/365) *(364/365) * (363/365) * (362/365) = We can keep on gong the same way as long as we want. A formula for the probablty that n people have dfferent brthdays s ((365-)/365) * ((365-2)/365) * ((365-3)/365) *... * ((365-n+)/365) = 365! / ((365-n)! * 365^n). P(two people share brthday) = - P(no two people share brthday) = -365! / ((365-n)! * 365^n).

21 How large must a class be to make the probablty of fndng two people wth the same brthday at least 50%? It turns out that the smallest class where the chance of fndng two people wth the same brthday s more than 50% s a class of 23 people. (The probablty s about 50.73%.) n P( C )

22 x. Matchng Problem Suppose that each of N men at a party throws hs hat nto the center of the room. The hats are frst mxed up, and then each man randomly selects a hat. What s the probablty that none of the men select ther own hat? (a) Calculate the probablty of at least one man selectng hs own hat and use Proposton. Let = the event that the th man selects hs own hat, =, 2,, N P(at least one man selectng hs own hat) = ) ( ) ( ) ( ) ( ) ( ) ( ) ( N N n N N P P P P P n n L L L L L L + < < < + = <! + +! + +! = " " " # # # The outcome of ths experment can be seen as a vector of N elements, where the th element s the number of the hat drawn by the th man. For example (,2,3,...N) means that every man chooses hs own hat. The sample space S s the set of all permutatons of the set {, 2,..., N}. Hence S has N! elements. There are N! possble outcomes. P() = (N-)!/N! _ (N-)! P( 2 )=(N-2)!/N! _ 3 (N-2)! etc... P( 2... n ) = (N-n)!/N! Also, as there are (N n) terms n! < < < n n P L L 2 2 ) (, so

23 and thus P( < < L< " 2 N!( N! n)! P( L ) = = ( N! n)! n! N! n! 2 n n " 2 " L" N ) =! +! L+ (! ) 2! 3! N + N! Hence the probablty that none of the N men selects ther own hat, P N, s P N = ' P( ( 2 ( L( N & ) = ' $ ' % 2! + 3! ' L + (' ) N + N! #! " P N = /2! - /3! + /4! (-)^N /N! As N, P N /e 0.37, usng exp(x) = + x + x^2/2! + x^3/3!... b) What s the probablty that exactly k of the men select ther won hats? To obtan the probablty of exactly k matches, consder any fxed group of k men. The probablty that they, and only they, select ther own hats s calculated as follows: k _ P( 2... k )* P N-k = (/N)*(/(N-))*...*(/(N-(k-)))* P N-k = /(N*(N-)*...*(Nk+)* P N-k = ((n-k)!/n!)*p N-k P N-k s the probablty that the other n-k men, selectng among ther own hats, have no matches. As there are (n k) choces of a set of k men, the desred probablty of exactly k matches s (n k)*(n-k)!/n! P N-k = P N-k /k!

24 Strlng s approxmaton formula for n! It s useful to have a better sense of how n! behaves for large n. ln n! = ln + ln 2 + ln n = _{k=} ^n ln k _ ^n ln k dk = [k ln k k]_ ^n n ln n n + o(n) So n! may be approxmated as exp(n ln n - n) = (n^n) * exp(-n). More sophstcated methods gve better approxmaton: ln n! n ln n n + (/2) ln (2πn) +o(). x: n=0: ln n! = 5.04; n ln n n + (/2) ln (2πn) = x: back to the brthday problem. Let m=365; n=# of students. p(no match) = m!/[(m-n)! m^n] exp[m ln m m + (/2) ln (2πm) (m-n) ln (m-n) + m-n - (/2) ln (2π(mn)) n ln m] = exp[(m-n) ln (m/(m-n)) + (/2) ln (m/(m-n)) n] exp[(m-n) (n/m + (/2) (n/m)^2) + (/2) n/m n] exp[-n^2/2m + n/2m] (we used Taylor expanson ln (+x) x x^2/2 + o(x^2) for small x.) So, prob of no match becomes small when n>>sqrt(m). x: (2n n) = (2n)! / (n!)(n!) ~ exp[2n ln (2n) 2n 2(n ln n n)] = exp[2n ln n + 2n ln 2 2n 2n ln n + 2n] = exp[2n ln 2] = 2^(2n) x: What about (n an), where 0<a<? (n an) = n! / [(an)!((-a)n)]! ~ exp[n ln n n an ln an + an (-a)n ln (-a)n + (-a)n] = exp[nh(a)] h(a) = -a log a (-a) log (-a): often called the entropy functon. Key functon n nformaton theory. For what a does (n an) grow most quckly? Maxmze h(a): h (a) = -ln a + log(-a) + = ln [(-a)/a]. Settng a = ½ gves maxmum: corresponds to (2n n) case consdered above.