11. Conditional Density Functions and Conditional Expected Values

Transcription

1 Conditional Densit unctions and Conditional Epected Values As we have seen in section 4 conditional pobabilit densit unctions ae useul to update the inomation about an event based on the knowledge about some othe elated event ee to eample 47 In this section we shall analze the situation whee the elated event happens to be a andom vaiable that is dependent on the one o inteest om 4- ecall that the distibution unction o given an event B is B P ξ B P ξ B P B -

2 Suppose we let Substituting - into - we get whee we have made use o 7-4 But using 3-8 and 7-7 we can ewite -3 as To detemine the limiting case we can let and in -4 { } B ξ -3 - P P ξ ξ ξ dv v dudv v u -4 +

3 3 This gives and hence in the limit To emind about the conditional natue on the let hand side we shall use the subscipt instead o thee Thus Dieentiating -7 with espect to using 8-7 we get -5-6 lim du u du u dv v dudv v u du u

4 It is eas to see that the let side o -8 epesents a valid pobabilit densit unction In act and + d d whee we have made use o 7-4 om indeed epesents a valid pd and we shall ee to it as the conditional pd o the v given We ma also wite om -8 and - we have

5 5 and similal I the vs and ae independent then and educes to impling that the conditional pds coincide with thei unconditional pds This makes sense since i and ae independent vs inomation about shouldn t be o an help in updating ou knowledge about In the case o discete-tpe vs - educes to j j i j i P P P

6 Net we shall illustate the method o obtaining conditional pds though an eample Eample : Given detemine and Solution: The joint pd is given to be a constant in the shaded egion This gives Similal and k othewise -6 k dd k d d k d k d d k d k d k k ig

7 7 om we get and We can use to deive an impotant esult om thee we also have o But and using -3 in - we get d d -3

8 + d Equation -4 epesents the pd vesion o Baes theoem To appeciate the ull signiicance o -4 one need to look at communication poblems whee obsevations can be used to update ou knowledge about unknown paametes We shall illustate this using a simple eample 4 Eample : An unknown andom phase is unioml distibuted in the inteval π and + n whee n N σ Detemine Solution: Initiall almost nothing about the v is known so that we assume its a-pioi pd to be uniom in the inteval π 8

9 9 In the equation we can think o n as the noise contibution and as the obsevation It is easonable to assume that and n ae independent In that case since it is given that is a constant behaves like n Using -4 this gives the a-posteioi pd o given to be see ig b whee + n σ N -5 / / / π ϕ π σ π σ σ π e d e e d / π σ π ϕ d e -6 + n

10 Notice that the knowledge about the obsevation is elected in the a-posteioi pd o in ig b It is no longe lat as the a-pioi pd in ig a and it shows highe pobabilities in the neighbohood o π π a a-pioi pd o ig b a-posteioi pd o Conditional Mean: We can use the conditional pds to deine the conditional mean Moe geneall appling 6-3 to conditional pds we get

11 + E g B g B d -7 and using a limiting agument as in we get µ + E d to be the conditional mean o given Notice that E will be a unction o Also + µ E d -8-9 In a simila manne the conditional vaiance o given is given b Va σ E E E µ we shall illustate these calculations though an eample -3

12 Eample 3: Let Detemine E and E Solution: As ig 3 shows in the shaded aea and zeo elsewhee om thee and This gives and -3 othewise d d ig

13 Hence E d d E d d -35 It is possible to obtain an inteesting genealization o the conditional mean omulas in Moe geneall -8 gives + E g g d But E g g d g dd g dd g d d + E g d E E -36 E g { g } -37 3

14 Obviousl in the ight side o -37 the inne epectation is with espect to and the oute epectation is with espect to Letting g in -37 we get the inteesting identit E E{ E } whee the inne epectation on the ight side is with espect to and the oute one is with espect to Similal we have E E{ E } Using -37 and -3 we also obtain Va E Va

15 Conditional mean tuns out to be an impotant concept in estimation and pediction theo o eample given an obsevation about a v what can we sa about a elated v? In othe wods what is the best pedicted value o given that? It tuns out that i best is meant in the sense o minimizing the mean squae eo between and its estimate Ŷ then the conditional mean o given ie E is the best estimate o see Lectue 6 o moe on Mean Squae Estimation 5