Joint Probability Distributions and Random Samples

Transcription

1 STAT5 Sprig 204 Lecture Notes Chapter 5 February, 204 Joit Probability Distributios ad Radom Samples 5. Joitly Distributed Radom Variables Chapter Overview Joitly distributed rv Joit mass fuctio, margial mass fuctio for discrete rv Joit desity fuctio, margial desity fuctio for cotiuous rv Idepedet radom variables Expectatio, covariace ad correlatio betwee two rvs Expectatio Covariace Correlatio Iterpretatios Statistics ad their distributios Distributio of the sample mea Distributio of a liear combiatio Joit Mass Fuctio of Two Discrete RVs Defiitio. Let X ad Y be two discrete rvs defied o the sample space S of a radom experimet. The joit probability mass fuctio p(x, y) is defied for each pair of umbers (x, y) by: p(x, y) = P (X = x ad Y = y) Let A be the set cosistig of pairs of (x, y) values, the the probability P [(X, Y ) A] is obtaied by summig the joit pmf pairs i A: P [(X, Y ) A] = p(x, y) (x,y) A Example of Joit PMF Example 5.. Exercise 5.3: A market has two check out lies. Let X be the umber of customers at the express checkout lie at a particular time of day. Let Y deote the umber of customers i the super-express lie at the same time. The joit pmf of (X, Y ) is give below: x =, y = What is P (X =, Y = 0)? What is P (X =, Y > 2)? What is P (X = Y )? Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

2 STAT5 Sprig 204 Lecture Notes 2 Margial Probability Mass Fuctio Defiitio 2. The margial probability mass fuctios of X ad Y, deoted p X (x) ad p Y (y), respectively, are give by p X (x) = P (X = x) = y p(x, y) p Y (y) = P (Y = y) = x p(x, y) Example of Margial Probability Mass Fuctio Example 5.. Now let s fid the margial mass fuctio. What is P (X = 3)? What is P (Y = 2)? x =, y = p(x) p(y) Joit Probability Desity Fuctio of Two Cotiuous RVs Defiitio 3. Let X ad Y be cotiuous rv s. The f(x, y) is the joit probability desity fuctio for X ad Y if for ay two-dimesioal set A: P [(X, Y ) A] = f(x, y)dxdy I particular, if A is the two-dimesioal rectagle {(x, y) : a x b, c x d}, P [(X, Y ) A] = A b d a c f(x, y)dydx Joit Probability Desity Fuctio of Two Cotiuous RVs P [(X, Y ) A] = Volume uder desity surface above A Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

3 STAT5 Sprig 204 Lecture Notes 3 Margial Probability Desity Fuctio Defiitio 4. The margial probability desity fuctio of X ad Y, deoted f X (x) ad f Y (y), respectively, are give by: f X (x) = f Y (y) = Idepedet Radom Variables f(x, y)dy, < X <. f(x, y)dx, < Y <. Defiitio 5 (Idepedece betwee X ad Y ). Two radom variables X ad Y are said to be idepedet if for every pair of x ad y values, whe X ad Y are discrete or whe X ad Y are cotiuous If X, Y are idepedet, we have: p(x, y) = p X (x) p Y (y) f(x, y) = f X (x) f Y (y) P (a < X < b, c < Y < d) = P (a < X < b) P (c < Y < d) If the coditios are ot satisfied for all (x, y) the X ad Y are depedet. Example of Idepedece Example 5..2 X follows a expoetial distributio with λ = 2, Y follows a expoetial distributio with λ = 3, X ad Y are idepedet, fid f(x, y). f(x, y) = f X (x) f Y (y) = 2e 2x 3e 3y = e (2x+3y), x 0, y 0 Example 5..3 Toss a fair coi, ad a die. Let X = if coi is head, let X = 0 if coi is tail. Let Y be the outcome of the die. if X ad Y are idepedet, fid p(x, y) ad fid the probability that the outcome of the die is greater tha 3 ad the coi is a head? x=,y= P (X =, Y > 3) = P (X = ) P (Y > 3) 2 2 Examples Cotiued Example 5..4 Give the followig p(x, y), is X ad Y idepedet? x =, y = p(x) p(y) Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

4 STAT5 Sprig 204 Lecture Notes 4 Example 5..5 Give f X (x) = 0.5x, 0 < x < 2, f Y (y) = 3y 2, 0 < y <, f(x, y) =.5xy 2, 0 < x < 2 ad 0 < y <, is X, Y idepedet? f X (x)f Y (y) = 0.5x 3y 2 =.5x y 2 = f(x, y) More Tha Two Radom Variables If X, X 2,, X are all discrete radom variables, the joit pmf of the variables is the fuctio p(x,, x ) = P (X = x,, X = x ) If the variables are cotiuous, the joit pdf is the fuctio f such that for ay itervals [a, b ],, [a, b ], P (a X b,, a X b ) = b a b a f(x,, x )dx dx Idepedece More Tha Two Radom Variables The radom variables X, X 2,, X are idepedet if for every subset X i, X i2,, X i of the variables, the joit pmf or pdf of the subset is equal to the product of the margial pmf s or pdf s. Coditioal Distributios Defiitio. Let X, Y be two cotiuous rv s with joit pdf f(x, y) ad margial pdfs f X (x) ad f Y (y). The for ay X value x for which f X (x) > 0, the coditioal probability desity fuctio of Y give that X = x is: f Y X (y x) = f(x, y), < y <. f X (x) If X ad Y are discrete, replace pdf s by pmf s i this defiitio. coditioal probability mass fuctio of Y whe X = x. That the gives Example of Coditioal Mass Example 5.. Joit mass is give below. What is the coditioal mass fuctio of Y, give X =? x =, y = p(x) p(y) Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

5 STAT5 Sprig 204 Lecture Notes 5 Example 5.. Give f(x, y) = 5 (x + y2 ), 0 x, 0 < y <. f X (x) = 5 x What is the coditioal desity of Y give X = 0? f Y X (y 0) = f(0, y) f X (0) = 5 y Expected Values, Covariace, ad Correlatio Expected Values Defiitio 7. Let X ad Y be joitly distributed rvs with pmf p(x, y) or pdf f(x, y) accordig to whether the variables are discrete or cotiuous. The the expected value of a fuctio h(x, Y ), deoted E[h(X, Y )] or µ h(x,y) is: µ h(x, Y ) = E [h(x, Y )] = Examples of Expected Values x y h(x, y) p(x, y), discrete; h(x, y) f(x, y)dxdy, cotiuous. Example 5.2. The joit pmf is give below. What is E(XY )? What is E[max(X, Y )]? p(x, y) y = x = Example Joit pdf of X ad Y is: f(x, y) = 4xy, 0 < x <, 0 < y <. What is E(XY )? Covariace Defiitio 8. Let E(X) ad E(Y ) deote the expectatios of rv X ad Y. covariace betwee X ad Y, deoted Cov(X, Y ) is defied as: The Cov(X, Y ) = E[(X E(X))(Y E(Y ))] i.e., = x y [x E(X)] [y E(Y )] p(x, y), discrete; (x E(X))(y E(Y ))f(x, y)dxdy, cotiuous. Properties of Covariace ad Shortcut Formula Cov(X, X) = V ar(x) Cov(X, Y ) = Cov(Y, X) Cov(aX, by ) = abcov(x, Y ), Cov(X + a, Y + b) = Cov(X, Y ), i.e., Cov(aX + b, cy + d) = accov(x, Y ) Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

6 STAT5 Sprig 204 Lecture Notes Shortcut formula: Cov(X, Y ) = E(XY ) E(X)E(Y ) If X ad Y are idepedet, the Cov(X, Y ) = 0. However, Cov(X, Y ) = 0 does ot imply idepedece. Iterpretatio of Covariace Similar to Variace, Covariace is a measure of variatio. Covariace measures how much two radom variables vary together. As opposed to variace: a measure of variatio of a sigle rv. If two rv s ted to vary together, the the covariace betwee the two variables will be positive. For example, whe oe of them is above its expected value, the the other variable teds to be above its expected value as well. If two rv s vary differetly, the the covariace betwee the two variables will be egative. For example, whe oe of them is above its expected value, the other variable teds to be below its expected value. Examples of Covariace Example 5.2. Exercise The joit pmf is give below. What s Cov(X, Y )? p(x, y) y = x = Example Joit pdf of X ad Y is: f(x, y) = 4xy, 0 < x <, 0 < y <. What s Cov(X, Y )? Example Give the pmf below, what s Cov(X, Y )? Correlatio p(x, y) y = 2 p X (x) x = p Y (y) Defiitio. The correlatio coefficiet of two rv s X ad Y, deoted Corr(X, Y ), ρ X,Y or just ρ is defied by: Corr(X, Y ) = ρ X,Y = Cov(X, Y ) V ar(x) V ar(y ) i.e., where σ X ad σ Y Corr(X, Y ) = ρ X,Y = Cov(X, Y ) σ X σ Y are the std dev s of X ad Y, respectively. Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

7 STAT5 Sprig 204 Lecture Notes 7 Properties ad Shortcut Formula of Correlatio For ay two rv s X ad Y, Corr(X, Y ) For a ad c both positive or both egative, Corr(aX + b, cy + d) = Corr(X, Y ) If X ad Y are liearly related, i.e., Y = ax + b, the Corr(X, Y ) = ± Shortcut formula: Corr(X, Y ) = E(XY ) E(X)E(Y ) E(X 2 ) (E(X)) 2 E(Y 2 ) (E(Y )) 2 If X ad Y are idepedet, Corr(X, Y ) = 0. However, Corr(X, Y ) = 0 does ot imply idepedece. Iterpretatio of Correlatio Correlatio is a stadardized measure. Correlatio coefficiet idicates the stregth ad directio of a liear relatioship betwee two rv s. X ad Y approximately positively liearly related, Corr(X, Y ) will be close to. X ad Y approximately egatively liearly related, Corr(X, Y ) will be close to. X ad Y ot liearly related, Corr(X, Y ) = 0. Especially, whe X, Y ot related, i.e., idepedet, Corr(X, Y ) = 0. Examples of Correlatio Example 5.2. Exercise The joit pmf is give below. Fid Cov(X, Y ). p(x, y) y = x = Example Joit pdf of X ad Y is: f(x, y) = 4xy, 0 < x <, 0 < y <. Fid Corr(X, Y ). Example Give the pmf below, what s Corr(X, Y )? p(x, y) y = 3 x = 0 Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

8 STAT5 Sprig 204 Lecture Notes Statistics ad their Distributios Statistic Defiitio 0 (Statistic). A statistic is ay quatity whose value ca be calculated from sample data. Or, a statistic is a fuctio of radom variables. We deote a statistic by a uppercase letter; a lowercase letter is used to represet the calculated or observed value of the statistic. Examples: Two rv X ad X 2, deote X = X +X 2 2, X is a statistic. 5 rv s X, X 2,, X 5, deote X max = max(x, X 2,, X 5 ), X max is a statistic. 2+ Idepedet RV s Defiitio (Joit pmf ad pdf for more tha two rv s). For rv s X, X 2,, X, the joit pmf is: p(x, x 2,, x ) = P (X = x, X 2, = x 2,, X = x ) ad the joit pdf is for ay itervals [a, b ],, [a, b ]: P (a X b,..., a X b ) = b a b a f(x,, x )dx dx Defiitio 2 (Idepedece of more tha two rv s). The radom variables X, X 2,, X are said to be idepedet if for ay subset of X i s, the joit pmf or pdf is the product of the margial pmf or pdf s. Radom Samples Defiitio 3 (Radom Sample). The rv s X, X 2,, X are said to form a (simple) radom sample of size if:. The X i s are idepedet rv s. 2. Every X i has the same probability distributio. Such X i s are said to be idepedet ad idetically distributed. Example: Let X, X 2,, X be a radom sample from stadard ormal, i.e., X follows N(0, ), X 2 follows N(0, ), X follows N(0, ) Ad, X, X 2,, X are idepedet. Examples of Statistic of a Radom Sample Example 5.3. Sample mea Take a radom sample of size from a specific distributio (say stadard ormal). X = X +X 2 + +X is the radom variable sample mea. X = If X = x, X 2 = x 2,, X = x, X + X2 + + X, is a statistic x = x + x2 + + x is a value of the rv X Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

9 STAT5 Sprig 204 Lecture Notes Example Sample variace Take a radom sample of size from a specific distributio(say stadard ormal). X = X +X 2 + +X is the radom variable sample mea. S 2 (Xi = X) 2, is a statistic If X = x, X 2 = x 2,, X = x, s 2 (xi x) 2 =, is a value of the rv S 2 Derivig the Samplig Distributio of a Statistic Defiitio 4 (Samplig Distributio). A statistic is a radom variable, the distributio of a statistic is called the samplig distributio of the statistic. We may use probability rules to obtai the samplig distributio of a statistic. Example Example 5.20 i textbook. A large automobile service ceter charges $40, $45, ad $50 for a tue-up of four-, six-, ad eight-cylider cars, respectively. 20% of the tue-ups are doe for four-cylider cars, 30% for six-cylider cars ad 50% for eight-cylider cars. Let X be the service charge for a sigle tue-up. The the distributio of X is: x p(x) Now let X ad X 2 be the service charges of two radomly selected tue-ups. Fid the distributio of:. X = X +X S 2 = (X X) 2 +(X 2 X) 2 2 Example x x 2 p(x, x 2 ) x s The Distributio of the Sample Mea Samplig Distributio of Sample Sum ad Sample Mea Propositio (Mea ad Std Dev of Sample Sum). Let X, X 2,..., X be a radom sample from a distributio with mea µ ad stadard deviatio σ. Sample sum is T o = X + X X. The: Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

10 STAT5 Sprig 204 Lecture Notes 0. E(T o ) = µ 2. V ar(t o ) = σ 2 ad σ To = σ Propositio (Mea ad Std Dev of Sample Mea). Let X, X 2,..., X be a radom sample from a distributio with mea µ ad stadard deviatio σ. Sample mea is X = X +X X. The:. E ( X) = µ 2. V ar ( X) = σ 2 σ ad σ X = Sample variace S 2 related samplig distributio will be itroduced i Chapter 7. The Case of Normal Populatio Distributio Propositio. Let X, X 2,..., X be a radom sample from a ormal distributio with mea µ ad stadard deviatio σ (N(µ, σ)). The T o ad X both follow ormal distributios:. T o N(µ, σ) 2. X N(µ, σ ) Example 5.4. Example Let X be the time it takes a rat to fid its way through a maze. X N(µ =.5, σ 2 = ) (i miutes). Suppose five rats are radomly selected. Let X, X 2,, X 5 deote their times i the maze. Assume X, X 2,, X 5 be a radom sample from N(µ =.5, σ 2 = ). Let total time T o = X + X X 5, average time X = X +X 2 + +X 5 5. What is the probability that the total time of the 5 rats is betwee ad 8 miutes? What is the probability that the average time is at most 2.0 miutes? Example 5.4. Cotiued... T 0 N(5.5 = 7.5, = 0.25) So, P ( < T o < 8) = P ( < Z < P ( < T o < 8) = Φ(0.4) Φ(.2) = 0.75 So, X N(.5, ) P ( X ) = P (Z < 0.35/ ) = P (Z 3.) 5 P ( X 2.0) = Φ(3.) = 0.3 Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

11 STAT5 Sprig 204 Lecture Notes Cetral Limit Theorem Theorem 5. Let X, X 2,, X be a radom sample from a distributio with mea µ ad variace σ 2. The if is sufficietly large, X has approximately a ormal distributio with µ X = µ ad σ 2 X = σ2. T o the has approximately a ormal distributio with µ To = µ ad σt 2 o = σ 2. The larger the value of, the better the approximatio. Rule of Thumb: If > 30, the Cetral Limit Theorem ca be used. Cetral Limit Theorem Cetral Limit Theorem Examples Example 5.5. Example 5.2 i textbook. The amout of a particular impurity i a batch of some chemical product is a radom variable with mea 4.0g ad stadard deviatio.5g. If 50 batches are idepedetly prepared, what is the (approximate) probability that the sample average amout of impurity X is betwee 3.5g ad 3.8g? Example Example 5.27 i textbook. The umber of major defects for a certai model of automobile is a radom variable with mea 3.2 ad stadard deviatio 2.4. Amog 00 radomly selected cars of this model, how likely is it that the average umber of major defects exceeds 4? How likely is it that the umber of major defects of all 00 cars exceeds 20? 5.5 The Distributio of a Liear Combiatio Liear Combiatios of RV s Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)

12 STAT5 Sprig 204 Lecture Notes 2 Defiitio. Give a collectio of radom variables X, X 2,, X ad umerical costats a, a 2,, a, the rv: Y = a X + a 2 X a X = a i X i i= is called a liear combiatio of the X i s.. Sample sum T o = X + X X is a liear combiatio with a = a 2 = = a =. 2. Sample mea X = X +X 2 + +X is a liear combiatio with a = a 2 = = a =. Mea ad Variace of Liear Combiatios Propositio. Let X, X 2,, X have expectatios µ, µ 2,, µ respectively ad variaces σ 2, σ2 2,, σ2 respectively. Let Y be the liear combiatio of X i s, Y = a X + a 2 X a X. The:. E(Y ) = a µ + a 2 µ a µ. 2. V ar(y ) = i= j= a ia j Cov(X i, X j ) 3. If X, X 2,, X are idepedet, V ar(y ) = a 2 σ2 + a2 2 σ a2 σ 2. Corollary 7. If X, X 2 are idepedet, the E(X X 2 ) = E(X ) E(X 2 ) ad V ar(x X 2 ) = V ar(x ) + V ar(x 2 ). The Case of Normal Radom Variables Propositio. If X, X 2,, X are idepedet,ormally distributed rv s, with meas µ, µ 2,, µ ad variaces σ 2, σ2 2,, σ2, the ay liear combiatio of the X i s Y = a X + a 2 X a X also has a ormal distributio. Y N(a µ + a 2 µ a µ, a 2 σ 2 + a 2 2σ a 2 σ 2 ) Example Example Example 5.30 i textbook. Three grades of gasolie are priced at $.20, $.35 ad $.50 per gallo, respectively. Let X, X 2 ad X 3 deote the amouts of these grades purchased (gallos) o a particular day. Suppose X i s are idepedet ad ormally distributed with µ = 000, µ 2 = 500, µ 3 = 00, σ = 00, σ 2 = 80, σ 3 = 50. The total reveue of the sale of the three grades of gasolie o a particular day is Y =.2X +.35X 2 +.5X 3. Fid the probability that total reveue exceeds $2500. Chapter5 prit.tex; Last Modified: February, 204 (W. Sharabati)