3 Basic Definitions of Probability Theory

Transcription

1 3 Basic Defiitios of Probability Theory 3defprob.tex: Feb 10, 2003 Classical probability Frequecy probability axiomatic probability Historical developemet: Classical Frequecy Axiomatic The Axiomatic defiitio ecompasses the Classical ad Frequecy defiitios of probability Note that i a umber of places i these otes I use axiom as a syomom for assumptio. Classical probability Defiitio Classical probability (MBG): If a radom experimet (process with a ucertai outcome) ca result i mutually exclusive ad equally likely outcomes, ad if A of these outcomes has a attribute A, the the probability of A is the fractio A /. This otio of probabilty had its coceptio i the study of games of chace; i particular, fair games of chace. E.g. if oe tosses a coi there are two mutually exclusive outcomes: head or tail. Of these two outcomes, oe is associated with the attribute heads; oe is associated with the attribute tails. If the coi is fair each outcome is equally likely. I which case, Pr head A 1, 2 where 2ad A is the umber of possible outcomes associated with a head (1). Cosider some other examples: 1. The roll of a die: There are 6 equally likely outcomes. The probability of each is 1/6. 2. Draw a card from a deck: There are 52 equally likely outcomes. 3. The roll of two die: There are 36 equally likely outcomes (6x6): 6 possibilities for the first die, ad 6 for the secod. The probability of each outcome is 1/36.

2 4. Drawig (with replacemet) four balls from a ur with a equal umber of red, white, ad blue balls: There are 81 possible outcomes (3x3x3x3 3 4 ). For example, {red, white, white, blue} is a outcome which is a differet outcome from {white, white, red, blue}. The probability associated with each outcome is 1/ The toss of two cois: The four possible outcomes are H,H, H,T, T,H ad TT.The probability of each is 1/4. 6. The draw of two cards: There are 52 2 possible outcomes. Terms to ote i the defiitio of classical probability are radom,, mutually exclusive, ad equally likely. Axiom A Basic assumptio i the defiitio of classical probability is that is a fiite umber; that is, there is oly a fiite umber of possible outcomes. If there is a ifiite umber of possible outcomes, the probability of a outcome is ot defied i the classical sese. Defiitio mutually exclusive: The radom experimet result i the occurace of oly oe of the outcomes. E.g. if a coi is tossed, the result is a head or a tail, but ot both. That is, the outcomes are defied so as to be mutually exclusive. Defiitio equally likely: Each outcome of the radom experimet has a equal chace of occurig. Defiitio radom experimet (Gujarati p23): A radom experimet is a process leadig to at least two possible outcomes with ucertaity as to which will occur. Defiitio sample space: The collectio of all possible outcomes of a experimet. If I had to guess, I would say it is called sample space because it is the collectio (set) of all possible samples. As importat thig to ote is that classical probabilities ca be deduced from kowledge of the sample space ad the assumptios. Nothig has to be observed i terms of outcomes to deduce the probabilities. Frequecy Probability What if is ot fiite? I that case, the Classical defiitio is ot applicable. What if the outcomes are ot equally likely? Agai, the Classical defitio of probability is ot applicable. I such cases, how might we defie the probablity of a outcome that has attribute A.

3 Defiitio We might take a radom sample from the populatio of iterest ad idetify the proportio of the sample with attribute A. That is, calculate umber of obser i the sample that possess attribute A Relative freq of A i the sample umber of obser i the sample The assume Relative freq of A i the sample is a estimate of Pr A The foudatio of this approach is that there is some Pr A. We caot deduce it, as i Classical probability, but we ca estimate it. For example, oe tosses a coi, which might or might ot be fair, 100 times ad observes heads o 52 of the tosses. Oe s estimate of the probability of a head is.52. Frequecy probability allows to estimate probabilities whe Classical probability provides o isight. Axiomatic Approach to Probability Put simply, the axiomatic approach build up probability theory from a umber of assumptios (axioms). Axiom there is some sample space, : the collectio of all possible outcomes of a experimet For example, if the experimet is tossig a coi H,T. If the experimet is rollig a die 1,2,3,4,5,6. If the experimet is tossig two cois H,H, H,T, T,H, T,T. If the experimet is takig a sample of oe from the U.S. populatio, populatio of the U.S.. If the experimet is drawig a radom iterger, is the set of all itergers (there are a ifiite umber of these). Defiitio Evet A (MGB p15): a subset of the sample space. The set of all evets associated with a experimet is defied as the evet space Note that the above is ot truly a defitio (it does ot idetify the ecc ad suff coditios). Beig a subset of (A ) is a ecessary but ot a sufficiet coditio for beig a evet. That is, for some sample spaces, there are subset that are ot evets; evet space ca be a strict subset of sample space. If the umber of outcomes i is fiite, the all subsets of are evets.

4 I geeral, oe of the outcomes is a evet, typically deoted, ad oe of the outcomes is a evet, deoted. Thatis, is both the sample space ad a evet, a certai evet. The evet set is difficult to formally defie. Some examples of evet sets: 1. the toss of a sigle coi: H,T. The possible evets are H, T, either H or T ( ), ad either H or T ( ). 2. the toss of two cois: H,H, H,T, T,H, T,T. How may evets are there? (H,H), (H,T), (T,H), (T,T), oe of the these outcomes, oe of these outcomes, at least oe head ( (H,H), (H,T) or(t,h)), at least oe tail ((H,T), (T,H) or(t,t)). Ca you thik of others? 3. drawig 4 cards from a deck: Evets iclude all spades, sum of the 4 cards is 20 (assumig face cards have a value of zero), a sequece of itegers, a had with a 2,3,4 ad 5. There are may more evets. 4. the example o page 16 of MGB: radomly choose a lightbulb to observe ad record time it takes to bur out. x : x 0 Defie evet A k,m as A x : k x m I ecoometrics we will be cocered with the probablility of evets, ad eed oly to defie those evets we care about. For example, we might be cocered with the probability that the wier is a female or that the sample mea is 5 give that the populatio mea is 0. Typically, oe uses capital letters to represet evets. E.g. A, B, C; ora 1, A 2, A 3, etc. I will use Ą to repeset the set of all possible evets. Axiom Ą. That is, the evet that oe of the outcomes is occurs i a evet. Note that Pr 1. Axiom if A Ą the Ā Ą where Ā is the complimet of A. That is, if A is a evet, the ot A is a evet Axiom if A 1 ad A 2 AtheA 1 A 2 Ą. That is two evets together is a evet. These three assumptios imply the followig: 1. Ą For example, this implicatio follows from the first two Axioms. 2. if A 1,A 2,...A Ą the i 1 A i ad i 1 A i Ą

5 Ay collectio of evets that fulfills the above three assumptios/axioms is called a Boolea algebra Defiitio The axiomatic defitio of probability: With these defiitios ad assumptios i mid, a probability fuctio Pr. maps evets i Ą oto the 0,1 iteval. That is, there exists some fuctio that idetifies the probabilty associated with ay evet i Ą. It fufills the followig axioms: Pr A 0 A Ą Pr 1 If A 1,A 2,...,A is a sequece of mutually exclusive evets (A i A j i,j i j) ad if i 1 A i Ą the Pr i 1 A i i 1 Pr A i. From this defiitio with its three axioms, it is possible to deduce a buch of properties that oe would expect a probability fuctio to have (MGB 24). Icludig Pr 0, ad Pr Ā 1 Pr A If A ad B Ą ad A B the Pr A Pr B If A 1,A 2,...A Ą the Pr A 1 A 2... A Pr A 1 Pr A 2... Pr A. This is called Boole s iequality. If A ad B Ą the Pr A Pr A B Pr A B. That is, Pr A Pr AB Pr AB If A ad B Ą the Pr A B Pr A Pr A B. Thatis,Pr A B Pr A Pr AB Put simply, the axiomatic defiitio builds up the otio of a probability fuctio from a umber of assumptios/axioms. The properties of that probablity fuctio follow from the assumptios. Note that if the umber of outcomes are fiite ad equally likely the oe has the Classical world of probability. Also ote that the Frequecy defitio assumes the existece of the probability fuctio Pr A. The axiomatic approach subsumes the Classical ad Frequecy approaches. Probability of evet A coditioal o evet B occurig Let Pr A B Probability of evet A coditioal o evet B occurig, assumig Pr B 0. Otherwise Pr A B is ot defied.

6 Cosider the followig: x,y :0 x 100, 0 y 100. Now defie two sets: A x,y :20 x 50, 40 y 60 ad B x,y :0 x 40, 50 y 100. Note that these two sets defie evets. Note that i this example the two sets partially itersect. Cosider other examples where A B or A B. Draw some pictures. Defiitio of coditioal probability (1) Pr A B Pr AB Pr B if Pr B 0. ReadPr AB Pr A B Pr A ad B. So (2) Pr B A Pr AB Pr A if Pr A 0. Oe ca rearrage (1) ad (2) to obtai (1a) ad (2a) ad Combiig (1a) ad (2a), oe obtais (3) Pr AB Pr A B Pr B Pr BA Pr B A Pr A Pr AB Pr A B Pr B Pr B A Pr A Pr BA What is the ituitio behid this beig the defiitio of coditioal probability? I coditioal probability, the sample space is effectively reduced to B; that is, what is the probability that A will happe give that oes lives i a world where B prevails. So, Pr A B is the probability that both A ad B occur, Pr AB, as a proportio of the probability of B, Pr B. Note the followig, which followig from (2) ad (3): Pr B A Pr AB Pr A Pr A B Pr B Pr A This says that if oe kows, Pr A B, Pr B, adpr A, oe ca determie Pr B A. What is this result called? Baye s theorem? Whe are evets A ad B idepedet?

7 Start by cosiderig a case where Pr A 0adPr B 0, but A B, sopr AB 0. If these assumptios hold, Pr A B Pr B A 0; that is, the two evets are mutually exclusive. Draw a Ve diagram. I this case, are A ad B idepedet? NO. Oe precludes the other. Whe do we say A ad B are idepedet? Defiitio A ad B are idepedet iff ay of the followig are true: Pr AB Pr A B Pr A Pr B Pr A B Pr A if Pr B 0 Pr B A Pr B if Pr A 0 Note that these are three equivalet statemets: each implies the other two. Ca you show this? The followig ca be deduced: Pr A 0, Pr B 0adA B A ad B are ot idepedet Pr A 0, Pr B 0adA ad B are idepedet A B Idepedece of evets is more complicated tha idepedece of 2 evets. Some studet examples of idepedece ad depedece Example 1 Suppose a survey classified the populatio as male or female, ad as favorig or opposig the death pealty. Suppose, the proportios i each category were Death ot Death Male There are four evets: male, female, favor death pealty, Female do t favor death pealty. I this case, the probability of a idividual favorig the death pealty, coditioal o beig male is Pr D M Pr D,M Pr M

8 ad Pr D , so the the probability of favorig the death pealty equals the probability of favorig the death pealty coditioal o oe beig male, so the two evets (favorig death ad beig male) are idepedet. However if the proportios were Death ot Death Male Female Pr D M Pr D,M Pr M ad Pr D , so the the probability of favorig the death pealty does ot equal the probability of favorig the death pealty coditioal o oe beig male, so the two evets (favorig death ad beig male) are depedet. Males are more likely to prefer the death pealty.