Joint Distributions. Lecture 5. Probability & Statistics in Engineering / Dr. P. s Clinic Consultant Module in.

Transcription

1 -3σ -σ -σ +σ +σ +3σ Joint Distributions Lecture / Dr. P. s Clinic Consultant Module in Probabilit & Statistics in Engineering

2 Toda in P&S -3σ -σ -σ +σ +σ +3σ Dealing with multile random variables at the same time: Jointl distributed random variables Indeendent random variables Eected values, Covariance and correlation of joint random variables 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

3 -3σ -σ -σ +σ +σ +3σ Jointl Distributed Random Variables Often times in articular in engineering we are interested in joint behavior of two or more random variables Effect of rocess technolog and laout densit in defect rate of a chi Effect of SAT scores, and GPA, and curricular involvement in estimating otential success Effect of temerature and humidit on a chemical reaction, etc. Sometimes we are interested in the behavior of sum/difference or combined effect of random variables in the outcome of an eeriment In such cases we need to know how these multile variables behave jointl, described b their joint robabilit mass/densit function These functions lace a robabilit value to airs / trilets / quadrulets of random variables The robabilit of a student randoml selected from a articular high school to have an SAT score of 1400 and a GPA of 3.7 ou would eect a high mf value How about an SAT score of 1570 and GPA of.3, or SAT score of 1190 and GPA of 3.97? The robabilit that a chi manufactured with a 0.05 (new) technolog with a densit of 1000 devices/cm (ver low)to be defective? How about a 0.5 (old) technolog and a densit of devices/cm (ver high)? The robabilit that a atient will get a heart attack in the net 5 ears if s/he has a HR of 110bm, cholesterol level of 40, BP of 100/140, age 65 (four random variables)? All these and countless similar random events are governed b joint robabilit distributions. 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

4 -3σ -σ -σ +σ +σ +3σ Two Variables, Discrete Case Let X and be two discrete r.v. s defined on the samle sace of an eeriment. The joint robabilit mass function (, ) is defined for each air of numbers (, ) b Let A be the set consisting of airs of (, ) values, then ( ) (, ) (, ) ( & ) P X X P X, A (, ) (, ) A E: Comuters bought on this camus during Fall 006: XChi seeds:.6, 3. GHz, RAM: 56, 51 and 104 MB.6 GHz 3. GHz 56 MB MB (X,) airs: (.6, 56),, (3., 104) P(.6, 51)0.1, P( 51) Marginal distribution GB 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

5 Marginal Distributions -3σ -σ -σ +σ +σ +3σ The distribution of an individual r.v. s of X and from a joint distribution of (X,) is the marginal robabilit mass functions, denoted X () and () To obtain a marginal distribution, we remove the other variable b summing it out ( ) (, ) ( ) (, ) X X.6 GHz 3. GHz 56 MB MB GB X () Sum over values () X ( ) ( ) 0.5.6, 3.GHz 0 otherwise , 51 MB 0.5 1GB 0 otherwise (, ) (, ) All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering Sum over X values j

6 -3σ -σ -σ +σ +σ +3σ Conditional Distributions Recall the definition of a conditional robabilities of two events A and B PAB ( ) ( I B) P A PB ( ) This can be easil adated to r.v. s X and The robabilit distribution (mass function) of ossible values of, given that X: P( X) (, ) ( ) (, ) ( ) X ( ) X ( ) > 0 X X ( ) ( ) > 0 X Joint distribution of X and Marginal distribution of 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

7 -3σ -σ -σ +σ +σ +3σ Conditional Distributions Proerties of r.v. s can easil be etended to conditional r.v. s as well: Conditional mean, or eected value of, given X E ( ) And conditional variance of given X ( ) ( ) ( ) σ V 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

8 -3σ -σ -σ +σ +σ +3σ Joint Probabilit Densit Functions For continuous r.v. s, the definition of the joint df follows directl from that of a single df: essentiall, the robabilit that a air of variables, lie in a certain interval is given as the double integral comuting the area under the two-dimensional surface curve. That is, Let X and be continuous rv s. Then f (, ) is a joint robabilit densit function for X and if for an two-dimensional set A ( ) P X, A f (, ) dd If the area in which the df is to be evaluated is rectangular, that is if then the robabilit { a bc d} (, ):,, ( ) P X, A f (, ) dd A bd ac 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

9 Cont. Joint Densit -3σ -σ -σ +σ +σ +3σ f (, ) A shaded rectangle (, ) P X A Volume under densit surface above A f (, ) f 0 (, ) All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

10 -3σ -σ -σ +σ +σ +3σ Marginal Probabilit Densit Functions The marginal robabilit densit functions of X and, denoted f X () and f (), are given b fx ( ) f(, ) d for < < f ( ) f(, ) d for < < f X () f () f (, ) 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

11 -3σ -σ -σ +σ +σ +3σ Indeendent Random Variables Often the observation of a articular r.v. ma rovide some information about the robabilit of another r.v. Consider our revious eamle: 56 MB 51 MB 1 GB.6 GHz GHz The marginal robabilit of chi seed at.6 GHz is 0.5, so is that of 3. GHz. That is, a randoml selected student is equall likel to bu a.6 GHz machine as s/he is a 3. GHz machine. However, if we are told that a student just bought a machine with a RAM of 56MB, then the robabilit that.6ghz is (0.) four times likel that 3. GHz (0.05). Conversel, if a machine with 1GB RAM is bought then robabilit that it is a 3. GHz machine is (0.3) 50% more then it is a.6 GHz machine (0.) Clearl these two variables are not indeendent!!! 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

12 -3σ -σ -σ +σ +σ +3σ Indeendent Random Variables Recall that two random events were defined as indeendent if their joint robabilit was the same as roduct of individual robabilities, that is, if P(A B)P(A).P(B). For eamle, the individual outcomes of two dice rolled are indeendent. A similar definition can be given for random variables: Two random variables X and are said to be indeendent if for ever air of and values (, ) X( ) ( ) when X and are discrete or f (, ) f ( ) f ( ) X when X and are continuous. If the conditions are not satisfied for all (, ) then X and are deendent. That is, two rv s are indeendent, if their joint distribution is equal to the roduct of their marginal distributions. 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

13 -3σ -σ -σ +σ +σ +3σ How About More than Two Random Variables? The concet of having a joint distribution for two random variables can be easil etended to more then two random variables: If X 1, X,,X n are all discrete random variables, the joint mf of the variables is the function (,..., ) P( X,..., X ) 1 n 1 1 n n If the variables are continuous, the joint df is the function f (.)such that for an n intervals [a 1,b 1 ],,[a n,b n ], Pa ( X b,..., a X b) n n n b a 1 1 b n... f(,..., ) d... d a n 1 n n All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

14 -3σ -σ -σ +σ +σ +3σ Multinomial Distribution Recall the binomial eeriment? An eeriment with onl two ossible outcomes, reeated n times giving successes was the binomial distribution Consider the following scenario: An eeriment consisting n indeendent and identical trials, in which each trial can result in an one of r ossible (instead of ) outcomes. Let i P the robabilit of outcome i on an given trial, and define X i the number of trials resulting in outcome i, i1,,r. Multinomial eeriment and the joint mf of X 1, X, X r is called the multinomial distribution 0 n! (, L, ) (!)(!) L(!) 1 r 1 r 1 r L r i 0,1,, L, r L 1 otherwise r n 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

15 -3σ -σ -σ +σ +σ +3σ Conditional Distributions (Continuous Case) Consider Suose X and denote the related arameters of two comonents of a comuter, sa the core temerature and the microrocessor seed. If we know that the cooling fan is running slow (the tem is high), does this give us an additional information about the microrocessor seed? Or how about, let X be the length of a fish and be the te of fish (seabass, salmon). Given the length of a fish is observed as cm, does that give us an additional information on what the fish ma be? Questions like this and countless man others can be answered b continuous conditional robabilit distributions Let X and be two continuous rv s with joint df f (, ) and marginal X df f X (). Then for an X value for which f X () > 0, the conditional robabilit densit function of given that X is f(, ) fx ( ) f ( ) < < If X and are discrete, relacing df s b mf s gives the conditional robabilit mass function of when X. X 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

16 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering +σ -σ -σ +σ +3σ -3σ Statistics of joint Distributions Statistics of joint Distributions Eected values and variances of individual random variables in joint distributions can be comuted similar to single variable cases: ( ) ( ) ( ) [ ] ( ) ( ) ( ) ( ) [ ] ( ) ( ) ( ) X X E E ), ( ), ( ), ( ), ( σ σ ( ) ( ) ( ) [ ] ( ) ( ) ( ) ( ) [ ] ( ) ( ) ( ) X X d f d d f E d f d d f E d f d d f d f d d f ), ( ), ( ), ( ), ( σ σ

17 An Eamle -3σ -σ -σ +σ +σ +3σ A universit offers both on-line and on-camus courses. On an given semester, let Xthe roortion of on-line classes are full and roortion of on-camus classes are full. Suose that the joint distribution of (,) is given as 6 f (, ) ( + ), 0 1, First, it is eas to verif that this is indeed a roer distribution: ( ) dd For eamle, the robabilit that neither te of classes are more than 5% full is P 0 X,0 ( + ) dd All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

18 Eamle (Cont.) -3σ -σ -σ +σ +σ +3σ The marginal robabilit of X, the robabilit that on-line classes are full, regardless of on-camus classes is then: And the marginal dist. of f X ( ) ( + ) d +, , 0 1 ( ) ( ) f + d We can then calculate such quantities as, sa robabilit that on-camus courses are between 5% and 75% full 3/ P d / All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

19 Eamle (cont.) -3σ -σ -σ +σ +σ +3σ How about conditional robabilities: What is the roortion of on-camus courses being full, given that on-line courses are 80% full? ( ) f ( 0.8, ) f ( 0.8) ( ) 0 1 f X ( 0.8) 1.( 0.8) Then, we can calculate, sa, robabilit that on-camus courses are at most halffull, given that on-line courses are 80% full: P ( 0.5 X 0.8 ) fx ( 0.8) d ( ) d Or, we can even calculate the eected roortion of on-camus classes being full, given that on-line classes are 80% full: 1 E [ X 0.8 ] fx ( 0.8) d ( ) d All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

20 -3σ -σ -σ +σ +σ +3σ Eected Values of Functions of Joint PDFs Recall that a function h() of a r.v. X is itself a random variable. However, we do not need to comute its df/mf to obtain its eected value, we can siml comute it as weighted sum of ()/f(). Similar eressions can be given for joint distributions: Let X and be jointl distributed rv s with mf (, ) or df f (, ) according to whether the variables are discrete or continuous. Then the eected value of a function h(x, ), denoted E[h(X, )] or h(x,) is (, ) E h h (, ) (, ) h (, ) f( dd, ) 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

21 Covariance -3σ -σ -σ +σ +σ +3σ If two variables are NOT indeendent, then we would like to know how strongl the are related, a quantit given b the covariance σ Cov( X, ) E[ ( X )( )] E ( X ) ( ) ( ) (, ) ( ) ( ) f (, ) d d [ ] discrete continuous Both X and increase or decrease together ositive correlation σ X >0 As X increases, decreases and vice versa negative correlation σ X <0 X and do not seem to be correlated with each other, near zero correlation σ X All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

22 Covariance Matri -3σ -σ -σ +σ +σ +3σ Note that we can also define σ σ, σ σ, and σ σ, all which can be reresented with a single matri, the covariance matri, denoted b Σ σ σ If X and are statisticall indeendent σ 0 If σ 0 then the variables are said to be uncorrelated Note that statistical indeendence is a stronger roert then correlation: σ σ statistical indeendence uncorrelated 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

23 Eamle -3σ -σ -σ +σ +σ +3σ Back to our comuter eamle: P(,) 56 MB 51 MB 1 GB.6 GHz GHz X.6 GHz 3. GHz 56MB 51MB 1GB P X () P () X X ( ) ( ).6* *0.5.9GHz 56* * * MB 0.56* * * GB Cov ( X, ) σ (.9)( 0.69) (, ) X (.6.9)(.56.69) (3..9)( ) (, ) (, ) ( ) ( ) ( ) ( ) σ (, ) X 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

24 -3σ -σ -σ +σ +σ +3σ Correlation Coefficient The roblem with the covariance is that it deends on the absolute magnitude of the numbers, more secificall to the units. For eamle, if we used the units of MB instead of GB in calculating the covariance, the actual number of would be 1000 times the revious number.. Clearl, the correlation between the two variables should not be deendent on the actual units used, but siml how the variables are related to each other. This can be achieved siml b normalizing or scaling the covariance, which ields the correlation coefficient: ρ X ( X, ) Cov σ X 1 ρ 1 σ σ σ σ X If ρ1, then the variables move together, i.e., the are linearl deendent ax+b, a>0 If ρ-1, then the variables are negativel correlated, one decreases as the other increases at the same rate, again the are linearl deendent with a negative sloe: ax+b, a<0 If ρ0 the variables are uncorrelated. The variation of one, has no effect on the other. Note however, uncorrelated does not necessaril mean indeendent For all ractical uroses, if ρ <0.05, the variables are considered to be uncorrelated. 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering

25 Homework -3σ -σ -σ +σ +σ +3σ Problems from Chater 5 10, 0, 34, 38, 48*,58* Bonus: 3, 78, 86* *: Requires that ou need additional sections of Chater 5 from our book. 006 All Rights Reserved, Robi Polikar, Rowan Universit, Det. of Electrical and Comuter Engineering