Unit 5 Outline. Stat 110. Unit 5: Continuous Random Variables Ch. 5 in the text. Definition: Continuous r.v. Probabilities for a Continuous r.v.

Transcription

1 9/9/04 Unit 5 Outlin Stat 0 Unit 5: Continuous Random Variabls Ch. 5 in th txt Probability Dnsity Functions Uniform Distribution Univrsality of th Uniform Normal Distribution Exponntial Distribution Poisson Procsss Dfinition: Continuous r.v. A r.v. has a continuous distribution if its CDF is diffrntiabl. W also allow thr to b ndpoints (or finitly many points) whr th CDF is continuous but not diffrntiabl, as long as th CDF is diffrntiabl vrywhr ls. For a continuous r.v. X with CDF F, th probability dnsity function (PDF) of X is th drivativ, f, of th CDF, givn by f(x) = F (x): Th support of X, and of its distribution, is th st of all x whr f(x) > 0. x F ( x) f ( t) dt So how do w calculat probabilitis for a continuous random variabl? 3 Probabilitis for a Continuous r.v. So how can w calculat th probability that a continuous r.v. X falls into an intrval (a,b)? Or [a,b)? (a,b]? [a,b]? By th dfinition of CDF and th fundamntal thorm of calculus, P ( a X b) F( b) F( a) dx Translation: to gt a dsird probability, intgrat th PDF ovr th appropriat rang. So how is probability rprsntd gomtrically? Not: W can b carfr about including or xcluding ndpoints as abov for continuous r.v.s (why?), but w must not b carlss about this for discrt r.v.s. b a 4

2 9/9/04 Continuous r.v. Exampl Lt X hav a Rayligh distribution, which has th CDF: F( x) Calculat th PDF of X. Find P(X > ). x /, x 0 5 Intrprtation of a PDF So what dos f(x) rprsnt anyway? W know f(x) is not a probability; for xampl, w could hav f(3) >, and w know P(X = 3) = 0. But thinking about th probability of X bing vry clos to 3 givs us a way to intrprt f(3). In gnral, w can think of f(x)dx as th probability of X bing in an infinitsimally small intrval containing x, of lngth dx. Spcifically, th probability of X bing in a tiny intrval of lngth ε, cntrd at 3, will ssntially b f(3)ε: P(3 / X 3 / ) 3 / 3 / dx f (3) f(x) is th dnsity of th distribution nar a spcific valu x. What ar th units on f(x)? 6 Expctd valu of a continuous r.v. Th xpctd valu (also calld th xpctation or man, μ) of a continuous r.v. X with PDF f is E ( X ) x dx Th intgral is takn ovr th ntir ral lin, but if th support of X is not th ntir ral lin, w can just intgrat ovr th support. 7 LOTUS and Varianc of a continuous r.v. LOTUS applis to continuous r.v.s as wll. That is: g( X ) E g( x) dx Th Varianc (σ ) of a continuous r.v. X with PDF f is ( X ) Var( X ) E X ( x ) dx Just lik for discrt r.v.s, this is somtims asir to calculat by using: Var( X ) E X X 8

3 9/9/04 Exampls: man of a continuous r.v. Lt X ~ Rayligh. Find E(X). Yay, intgration by parts! E( X ) x dx What is th intrprtation of this valu? Calculat Var(X). x x / dx Unit 5 Outlin Probability Dnsity Functions Uniform Distribution Univrsality of th Uniform Normal Distribution Exponntial Distribution Poisson Procsss 9 0 Story of th Uniform Distribution Considr a compltly random numbr (with ral valu) btwn th valus a and b, ach with qual liklihood. Lt th r.v. X b th valu of this compltly random numbr on th intrval (a,b). Thn X has th Uniform distribution with paramtrs a and b; w dnot this by X ~ Unif(a,b) What is X s distribution? That is, what is th probability dnsity function for X? Don t forgt to mntion X s support. Uniform Distribution Dfinition If X ~ Unif(a, b), thn th PDF of X is: if a x b b a 0 othrwis Or altrnativly writtn:, a x b b a Why is this a valid PMF? Th most common uniform r.v.? X ~ Unif(0,). This is somtims calld th standard uniform. 3

4 9/9/04 Plot of a standard Uniform PDF and CDF [Unif(0,)] Th Uniform Distribution: Man and Varianc Lt X ~ Unif(a,b). Intuitivly, what should b th man of X? What about it s varianc? Find E(X) and Var(X). 3 E(X) = (a+b)/ Var(X) = (b-a) / 4 Unit 5 Outlin Probability Dnsity Functions Uniform Distribution Univrsality of th Uniform Normal Distribution Exponntial Distribution Poisson Procsss 5 Univrsality of th Uniform Th Unif(0,) distribution has a rmarkabl proprty: givn a Unif(0,) r.v., w can construct any continuous distribution. Lt F b a CDF which is a continuous function and strictly incrasing on th support of th distribution. This nsurs that th invrs function F - xists, as a function from (0, ) to R. W thn hav th following rsults.. Lt U ~ Unif(0,) and X = F - (U). Thn X is an r.v. with CDF F.. Lt X b an r.v. with CDF F. Thn F(X) ~ Unif(0, ). Don t gt confusd by what th nd part is saying: 6 4

5 9/9/04 Univrsality of th Uniform (cont.) Th first part of th thorm says that if w start with U ~ Unif(0, ) and a CDF F, thn w can crat a r.v. whos CDF is F by plugging U into th invrs CDF F -. Th scond part of th thorm gos in th rvrs dirction, starting from an r.v. X whos CDF is F and thn crating a Unif(0, ) r.v. B carful with th nd part: it would b incorrct to say F(X) = P(X X) = ". Rathr, w should first find an xprssion for th CDF as a function of x, thn rplac x with X to obtain a random variabl. For xampl, if th CDF of X is F(x) = -x for x > 0, thn F(X) = -X. Exampl: Univrsality of th Rayligh Lt X hav a Rayligh distribution, which has th CDF: Calculat th quantil function F - (invrs of th CDF). So if U ~ Unif(0,), thn X = F - (U) = What s th support? F( x) / ln( U) ~ Rayligh. x, x Univrsality discrt r.v.s How dos th univrsality of th Uniform hold for discrt random variabls? Th CDF F of a discrt r.v. has jumps and flat rgions. So a closd form of F - dos not xist. But w can still construct any discrt distribution w want from th Uniform. How? Givn a PMF, chop up th intrval (0, ) into pics, with lngths givn by th PMF valus. It s bst illustratd with a pictur: 9 Why do w car about Univrsality? Th Univrsality of th Uniform is VERY usful whn running simulations. If w hav a mthod to gnrat ralizations from a Uniform distribution, thn w can asily crat ralizations from any spcific distribution. Ky: w nd to know th invrs CDF function (quantil function, F - ) to mak this asy. Lt s us an analogy: Random variabls ar lik houss Distributions ar lik bluprints Univrsality givs us a simpl rul for rmodling th Uniform hous into a hous with any othr bluprint 0 5

6 9/9/04 Unit 5 Outlin Probability Dnsity Functions Uniform Distribution Univrsality of th Uniform Normal Distribution Exponntial Distribution Poisson Procsss Story of th Normal Distribution Probably th most famous distribution of all is th Normal distribution (somtims calld th Gaussian). It shows up a lot in statistics bcaus of th cntral limit thorm (which w dfin mor dply at a latr tim), which says that undr vry wak assumptions, th sum (or avrag) of a larg numbr of i.i.d. random variabls has an approximatly Normal distribution, rgardlss of th distribution of th individual r.v.s. Exampl: Hights of individuals. Standardizd tst scors. Why can ths rasonably b assumd to b normally distributd? Standard Normal Distribution Dfinition W will start off with a spcial cas of th Normal distribution, calld th standard Normal, as it will allow to build som proprtis of Normal r.v.s asily. A continuous r.v. Z is said to hav th standard Normal distribution if its PDF φ is givn by: z / ( z), x Th standard normal CDF Φ is th accumulatd ara undr th PDF: z z t / ( z) ( t) dt dt 3 Plot of a standard Normal PDF and CDF [Normal(0,)] 4 6

7 9/9/04 Som Proprtis of th standard Normal Symmtry of PDF: φ(z) = φ( z). Symmtry of tail probabilitis: Φ(z) = Φ( z) Symmtry of Z and Z. W ll prov 3 facts about th standard Normal, and that will allow us to handl th gnral form of th Normal distribution: φ is a valid PDF. E(Z) = 0. Var(Z) =. 5 φ is a valid PDF To show φ is a valid PDF, w actually us a strang trick: w multiply th intgral twic, and thn convrt to polar coordinats (which xplains th / π): z / z / dz dz 0 0 x / r / dx x y rdrd / dxdy y / dy 6 φ is a valid PDF (cont.) Nxt stp: substitut u = r /, du = rdr. Which rsults in: r / rdrd d 0 0 Thrfor, sinc th product of two of th whol probabilitis is, thn ach whol probability must b. Nat trick, huh? u dud E(Z) = 0 Now w just nd to show E(Z) = 0 and Var(Z) =. Luckily ths arn t narly as tricky. Intuitivly, why must th E(Z) = 0? Math is asy too: E( Z) z z / z / Sinc g( z) z is an odd function, th ara from - to 0 cancls with th ara from 0 to. Thus, E(Z) = 0. dz 7 8 7

8 9/9/04 Var(Z) = Lastly w nd to show Var(Z) =. Rcall: Var( Z) E( Z ) Sinc E(Z) = μ Z = 0, w just nd to worry about E(Z ). z / E( Z ) z dz z / z dz 0 z / This is tru sinc z is an vn function. Now just us intgration by parts to show: Var( Z) E( Z ) 9 Z Normal Distribution Dfinition If X follows a normal distribution, X ~ N(μ, σ ), thn th PDF of X is (μ ϵ R, σ ϵ R + ): ( x ), x Lt Z ~ N(0,), and lt X = μ + σz. Thn X ~ N(μ, σ ). Show that th PDF of X is truly what is abov. *Hint: tak drivativ of Φ[(x- μ)/σ]. 30 Man and Varianc of N(μ,σ ) Rcall, if Z ~ N(0,) and X = μ + σz, thn X ~ N(μ, σ ). Using th proprtis of mans and variancs, find E(X) and Var(X). 3 Empirical Rul ( % rul) OK, so w saw that taking intgrals of th PDF for a Normal distribution is a littl difficult. So w will lav th probabilitis in class in trms of Φ. Or you could us a computr (or tabl). But it is nic to know th mpirical rul for Normally distributd r.v.s, which stats for X ~ N(μ, σ ): P P P X X X

9 9/9/04 Normal Distribution Exampl (hights of Amrican mals) Lt X ~ N(μ = 69, σ = 3 ). Find th following: P(66 < X < 7) P(X > 7) P(60 < X < 7) P(X > 68) What valu of X is ndd to b in th top.5% of th distribution? Unit 5 Outlin Probability Dnsity Functions Uniform Distribution Univrsality of th Uniform Normal Distribution Exponntial Distribution Poisson Procsss Story of th Exponntial Distribution Considr you ar waiting (in continuous tim) until a succss for som procss/xprimnt occurs, whr λ is th avrag # succsss pr unit of tim. Lt X b th amount of tim you hav to wait until this succss arrivs. Thn X has th Exponntial distribution with paramtr λ; w dnot this by X ~ Expo(λ) Th Exponntial distribution is th continuous countrpart to th Gomtric distribution. How so? Exampl: you ar waiting at a bus stop for th nxt bus to arriv, and th rat of bus arrivals is fixd (λ pr unit tim). 35 Exponntial Distribution Dfinition If X ~ Expo(λ), thn th PDF of X is (λ > 0): x, x 0 Th corrsponding CDF is: x F( x), x 0 Why is this a valid PDF? 36 9

10 9/9/04 Plot of an Exponntial PDF and CDF [Expo()] Exponntial Distribution Scaling Suppos X ~ Expo(). Thn: X Y ~ Expo( ) Intuitivly, why dos this mak sns? Using CDFs, show this to b tru Th Exponntial Distribution: Man and Varianc Lt X ~ Expo(). Find th man and varianc of X. *Not: you will nd to us intgration by parts. Now lt Y = X/λ. What distribution dos Y hav? Us th proprtis of xpctation and varianc to find th man and varianc of Y ~ Expo(λ). 39 Mmorylssnss proprty of th Exponntial Distribution A distribution is said to hav mmorylssnss proprty if a random variabl X from that distribution satisfis: P( X s t X s) P( X t) Hr s rprsnts th tim you'v alrady spnt waiting; th dfinition says that aftr you'v waitd s minuts, th probability you'll hav to wait anothr t minuts is xactly th sam as th probability of having to wait t minuts with no prvious waiting tim undr your blt. This implis that for X ~ Expo(λ): E( X X s) s E( X ) s 40 0

11 9/9/04 Mmorylssnss proprty of th Exponntial Distribution Lt X ~ Expo(λ). Show that th mmorylssnss proprty holds for X. *Hint: us th df. of conditional prob. and X s CDF. What ar th implications of this? Think about waiting at a bus stop if tim until nxt bus arrival is Exponntially distributd. Is this valid in rality? 4 Exponntial Distribution Exampls OK, so Exponntial distributions don t rally modl human and machin liftims. Thn why ar thy usful? Som physical phnomna (radioactiv dcay) do xhibit th mmorylssnss proprty Exponntials ar wll connctd to othr distributions (Poisson), and hav a shard intuition/story. Exponntial distribution can b usd as th basis for mor flxibl distributions (lik th Wibull dist.) whr rat of succsss can incras or dcras ovr tim. 4 Unit 5 Outlin Probability Dnsity Functions Uniform Distribution Univrsality of th Uniform Normal Distribution Exponntial Distribution Poisson Procsss Poisson Procss Dfinition A procss of arrivals in continuous tim is calld a Poisson procss with rat λ if th following two conditions hold:. Th numbr of arrivals that occur in an intrval of lngth t is a Pois(λt) random variabl.. Th numbrs of arrivals that occur in disjoint intrvals ar indpndnt of ach othr. For xampl, th numbrs of arrivals in th intrvals (0; 0); [0; ); and [5;9) ar indpndnt

12 9/9/04 Poisson Procss Exampl Suppos th mails arriv in your inbox according to a Poisson procss with rat paramtr λ (0.5 pr minut). In on hour, how many mails, X, will arriv? This follows a Pois(λ) distribution. What is E(X)? W could also ask: how long dos it tak, T, until th first mail arrivs? What distribution dos T follow? Saying that th waiting tim for th first mail is btwn 0 and t is th sam as saying no mails hav arrivd btwn 0 and t. So if N t is th numbr of mails that arriv at or bfor tim t, thn: (T > t) is th sam vnt as (N t = 0) 45 Poisson Procss Exampl From last slid: (T > t) is th sam vnt as (N t = 0) If two vnts ar th sam, thn thy hav th sam probability (and w know N t ~ Pois(λt)). So: t 0 ( t) t P( T t) P( Nt 0) 0! Thus P(T < t) = -λt, so T ~ Expo(λ). Wow! Th tim until th st arrival in a Poisson procss of rat λ has an Exponntial distribution with paramtr λ. Prtty cool! What about T - T? Sinc disjoint intrvals ar indpndnt, th past is irrlvant! Not: T, which is th sum of two indpndnt Expos is not xponntial (it s Gamma distributd). 46 Minimum of Exponntials is Lt X,, X n b indpndnt with X j ~ Expo(λ j ). Lt L = min(x,, X n ). Show that : L ~ Expo(λ + + λ n ). *Hint: considr th survival function of L: S(L) = F(L). Last Word: what is Normal? What dos this man intuitivly? 47 48