Normal Distribution.

Transcription

1 Normal Distributio

2 Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued radom variables that ted to cluster aroud a sigle mea value. f µ e π where parameter μ is the mea locatio of the peak ad is the variace the measure of the width of the distributio. The distributio with μ 0 ad is called the stadard ormal.

3 The ormal distributio is cosidered the most promiet probability distributio i statistics There are several reasos for this: First, the ormal distributio is very tractable aalytically, that is, a large umber of results ivolvig this distributio ca be derived i eplicit form. Secod, the ormal distributio arises as the outcome of the cetral limit theorem, which states that uder mild coditios the sum of a large umber of radom variables is distributed approimately ormally. Fially, the bell shape of the ormal distributio make it a coveiet choice for modelig a large variety of radom variables ecoutered i practice. The ormal distributio is usually deoted by Nμ,. Commoly the letter N is writte i calligraphic fot typed as \mathcal{n} i LaTeX. Thus whe a radom variable X is distributed ormally with mea μ ad variace, we write: X ~ Ν µ, 3

4 Stadard ormal distributio The simplest case of a ormal distributio is kow as the stadard ormal distributio, described by the probability desity fuctio φ e π Notice that for a stadard ormal distributio, μ 0 ad. The parameter μ is at the same time the mea, the media ad the mode of the ormal distributio. The parameter is called the variace; as for ay radom variable, it describes how cocetrated the distributio is aroud its mea. The square root of is called the stadard deviatio ad is the width of the desity fuctio. µ µ f ;, e π φ µ 4

5 Cumulative distributio fuctio I The cumulative distributio fuctio cdf describes probabilities for a radom variable to fall i the itervals of the form, ]. The cdfof the stadard ormal distributio is deoted with the capital Greek letter Φ phi, ad ca be computed as a itegral of the probability desity fuctio: t / Φ e dt [ + erf π ] I mathematics, the error fuctio also called the Gauss error fuctio or probability itegral is a special fuctio o-elemetary of sigmoid shape which occurs i probability, statistics ad partial differetial equatios. It is defied as: erf e t π 0 dt 5

6 Cumulative distributio fuctio II This itegral ca oly be epressed i terms of a special fuctio erf, called the error fuctio. The umerical methods for calculatio of the stadard ormal cdfare discussed below. For a geeric ormal radom variable with mea μ ad variace > 0 the cdfwill be equal to F ; µ, Φ µ µ [+ erf ] µ The complemet of the stadard ormal cdf, Q Φ, is referred to as the Q-fuctio, especially i egieerig tets. This represets the tail probability of the Gaussia distributio, that is the probability that a stadard ormal radom variable X is greater tha the umber. Other defiitios of the Q-fuctio, all of which are simple trasformatios of Φ, are also used occasioally. 6

7 Stadardizig ormal radom variables It is possible to relate all ormal radom variables to the stadard ormal. For eample if X is ormal with mea μ ad variace, the Z Xµ has mea zero ad uit variace, that is Z has the stadard ormal distributio. Coversely, havig a stadard ormal radom variable Z we ca always costruct aother ormal radom variable with specific mea μ ad variace : X Z+µ This stadardizig trasformatio is coveiet as it allows oe to compute the pdfad especially the cdfof a ormal distributio havig the table of pdfad cdf values for the stadard ormal. They will be related via µ µ φ, f FX Φ 7

8 Stadard deviatio About 68% of values draw from a ormal distributio are withi oe stadard deviatio away from the mea; about 95% of the values lie withi two stadard deviatios; ad about 99.7% are withi three stadard deviatios. This fact is kow as the rule, or the empirical rule, or the 3-sigma rule. To be more precise, the area uder the bell curve betwee μ ad μ + is give by F µ + ; µ, F µ ; µ, Φ Φ erf Dark blue is less tha oe stadard deviatio from the mea. For the ormal distributio, this accouts for about 68% of the set, while two stadard deviatios from the mea medium ad dark blue accout for about 95%, ad three stadard deviatios light, medium, ad dark blue accout for about 99.7% 8

9 Estimatio of parameters I It is ofte the case that we do t kow the parameters of the ormal distributio, but istead wat to estimate them. That is, havig a sample,, from a ormal Nμ, populatio we would like to lear the approimate values of parameters μad. The stadard approach to this problem is the maimum likelihood method. Maimum likelihood estimates: ˆ i, i i i ˆ µ Estimator µˆ is called the sample mea, sice it is the arithmetic mea of all observatios. The estimator ˆ is called the sample variace, sice it is the variace of the sample,, Of practical importace is the fact that the stadard error of µˆ is proportioal to /sqrtn, that is, if oe wishes to decrease the stadard error by a factor of 0, oe must icrease the umber of poits i the sample by a factor of

10 Estimatio of parameters II To use statistical parameters such as mea ad stadard deviatio reliably, you eed to have a good estimator for them. The maimum likelihood estimates MLEs provide oe such estimator. However, a MLE might be biased, which meas that its epected value of the parameter might ot equal the parameter beig estimated. For eample, a MLE is biased for estimatig the variace of a ormal distributio. A ubiased estimator that is commoly used to estimate the parameters of the ormal distributio is the miimum variace ubiased estimatormvue. The MVUE has the miimum variace of all ubiased estimators of a parameter. The MVUEs of parameters µad for the ormal distributio are the sample mea ad variace. The sample mea is also the MLE for µ. The followig are two commo formulas for the variace. s s i i i i with i i The first equatio is the maimum likelihood estimator for, ad the secod equatio is MVUE. 0

11 Sigal to oise improvemet due to averagig Ideally it is assumed that: Sigal ad oise are ucorrelated. Sigal stregth is costat i the replicate measuremets. Noise is radom, with a mea of zero ad costat variace i the replicate measuremets. Uder these assumptios let the sigal stregth be deoted by Sad let the stadard deviatio of a sigle measuremet be ; this represets the oise i oe measuremet, N. If measuremets are added together the sum of sigal stregths will be *S. For the oise, the stadard error propagatio formula shows that the variace,, is additive. The variace of the sum is equal to. Hece the sigal-to-oise ratio, S/N, is give by S N S The equivalet epressio for sigal averagig is obtaied by dividig both umerator ad deomiator by. S N S Thus, i the ideal case S/N icreases with the square root of the umber of measuremets that are averaged. I practice, the assumptios may be ot be fully realized. This will result i a lower S/N improvemet tha i the ideal case, but i may cases ear-ideal S/N improvemet ca be achieved. S S

12 Radom umbers with a Gaussia distributio i Ecel This Ecel formula computes a radom umber from a Gaussia distributio with a mea of 0.0 ad a SD of.0. NORMSINVRAND The RAND fuctio calculates a radom umber from 0 to. the NORMSINV fuctio takes a fractio betwee 0 ad ad tells you how may stadard deviatios you eed to go above or below the mea for a cumulative Gaussia distributio to cotai that fractio of the etire populatio. Multiple by the stadard deviatio ad add a mea, ad you'll have radom umbers draw from a Gaussia distributio with that mea ad SD. For eample, use this formula to sample from a Gaussia distributio with a mea of 00 ad a SD of 5: NORMSINVRAND*5+00

13 Normal distributio i Matlab Normal probability desity fuctios are geerated usig fuctio ormpdf. Characteristic of a ormal distributio are mea ad stadard deviatio. ormpdf, mea, std : vector of rage icludig graularity -5:0.:5; mu 3; sigma 4; pdfnormal ormpdf, mu, sigma; plot, pdfnormal; 3

14 Radom umbers with a Gaussia ormrd- Normal radom umbers Syta R ormrdmu,sigma R ormrdmu,sigma,m,,... R ormrdmu,sigma,[m,,...] distributio i Matlab Descriptio R ormrdmu,sigma geerates radom umbers from the ormal distributio with mea parameter mu ad stadard deviatio parameter sigma. mu ad sigma ca be vectors, matrices, or multidimesioal arrays that have the same size, which is also the size of R. A scalar iput for mu or sigma is epaded to a costat array with the same dimesios as the other iput. R ormrdmu,sigma,m,,... or R ormrdmu,sigma,[m,,...] geerates a m-by--by-... array. The mu, sigma parameters ca each be scalars or arrays of the same size as R. Eamples ormrd:6,./: ormrd0,,[ 5] ormrd[ 3;4 5 6],0.,,

15 Sources