ECON 351* -- Note 3: Desirable Statistical Properties of Estimators. 1. Introduction

Transcription

1 ECON 351* -- NOTE 3 Desrable Statstcal Propertes of Estmators 1. Itroducto We derved Note 2 the OLS (Ordary Least Squares) estmators ˆβ j (j = 1, 2) of the regresso coeffcets β j (j = 1, 2) the smple lear regresso model gve by the populato regresso equato, or PRE Y = β 1 + β 2 X + u (1) where u s a d radom error term. The OLS estmators of the regresso coeffcets β 1 ad β 2 are: βˆ 2 = x y 2 x = ( X X)( Y Y) 2 ( X X) = XY NXY. 2 2 X NX (2.2) where β! = Y β! X. (2.1) 1 2 y Y Y ( = 1,..., N) x X X ( = 1,..., N) Σ Y Y = Σ Y N = N = the sample mea of the Y values. Σ X X = Σ X N = N = the sample mea of the X values. The reaso we use these OLS coeffcet estmators s that, uder assumptos A1-A8 of the classcal lear regresso model, they have a umber of desrable statstcal propertes. Note 4 outles these desrable statstcal propertes of the OLS coeffcet estmators.... Page 1 of 15 pages

2 Why are statstcal propertes of estmators mportat? These statstcal propertes are extremely mportat because they provde crtera for choosg amog alteratve estmators. Kowledge of these propertes s therefore essetal to uderstadg why we use the estmato procedures we do ecoometrc aalyss. 2. Dstcto Betwee a "Estmator" ad a "Estmate" Recall the dstcto betwee a estmator ad a estmate. Let some ukow populato parameter be deoted geeral as θ, ad let a estmator of θ be deoted as θˆ. 3. Two Categores of Statstcal Propertes There are two categores of statstcal propertes. (1) Small-sample, or fte-sample, propertes; The most fudametal desrable small-sample propertes of a estmator are: S1. Ubasedess; S2. Mmum Varace; S3. Effcecy. (2) Large-sample, or asymptotc, propertes. The most mportat desrable large-sample property of a estmator s: L1. Cosstecy. Both sets of statstcal propertes refer to the propertes of the samplg dstrbuto, or probablty dstrbuto, of the estmator θˆ for dfferet sample szes.... Page 2 of 15 pages

3 3.1 Small-Sample (Fte-Sample) Propertes! The small-sample, or fte-sample, propertes of the estmator θˆ refer to the propertes of the samplg dstrbuto of θˆ for ay sample of fxed sze, where s a fte umber (.e., a umber less tha fty) deotg the umber of observatos the sample. = umber of sample observatos, where <. Defto: The samplg dstrbuto of θˆ for ay fte sample sze < s called the small-sample, or fte-sample, dstrbuto of the estmator θˆ. I fact, there s a famly of fte-sample dstrbutos for the estmator θˆ, oe for each fte value of.! The samplg dstrbuto of θˆ s based o the cocept of repeated samplg. Suppose a large umber of samples of sze are radomly selected from some uderlyg populato. Each of these samples cotas observatos. Each of these samples geeral cotas dfferet sample values of the observable radom varables that eter the formula for the estmator θˆ. For each of these samples of observatos, the formula for θˆ s used to compute a umercal estmate of the populato parameter θ. Each sample yelds a dfferet umercal estmate of the ukow parameter θ. Why? Because each sample typcally cotas dfferet sample values of the observable radom varables that eter the formula for the estmator θˆ. If we tabulate or plot these dfferet sample estmates of the parameter θ for a very large umber of samples of sze, we obta the small-sample, or fte-sample, dstrbuto of the estmator θˆ.... Page 3 of 15 pages

4 3.2 Large-Sample (Asymptotc) Propertes! The large-sample, or asymptotc, propertes of the estmator θˆ refer to the propertes of the samplg dstrbuto of θˆ as the sample sze becomes deftely large,.e., as sample sze approaches fty (as ). Defto: The probablty dstrbuto to whch the samplg dstrbuto of θˆ coverges as sample sze becomes deftely large (.e., as ) s called the asymptotc, or large-sample, dstrbuto of the estmator θˆ.! The propertes of the asymptotc dstrbuto of θˆ are what we call the largesample, or asymptotc, propertes of the estmator θˆ.! To defe all the large-sample propertes of a estmator, we eed to dstgush betwee two large-sample dstrbutos of a estmator: 1. the asymptotc dstrbuto of the estmator; 2. the ultmate, or fal, dstrbuto of the estmator.... Page 4 of 15 pages

5 Nature of Small-Sample Propertes 4. Small-Sample Propertes! The small-sample, or fte-sample, dstrbuto of the estmator θˆ for ay fte sample sze < has 1. a mea, or expectato, deoted as E(θˆ ), ad 2. a varace deoted as Var(θˆ ).! The small-sample propertes of the estmator θˆ are defed terms of the mea E(θˆ ) ad the varace Var(θˆ ) of the fte-sample dstrbuto of the estmator θˆ.... Page 5 of 15 pages

6 S1: Ubasedess Defto of Ubasedess: The estmator θˆ s a ubased estmator of the populato parameter θ f the mea or expectato of the fte-sample dstrbuto of θˆ s equal to the true θ. That s, θˆ s a ubased estmator of θ f ( θˆ ) = θ E for ay gve fte sample sze <. Defto of the Bas of a Estmator: The bas of the estmator θˆ s defed as ( θˆ ) = E( θˆ ) θ Bas = the mea of θˆ mus the true value of θ. The estmator θˆ s a ubased estmator of the populato parameter θ f the bas of θˆ s equal to zero;.e., f ( θ ˆ ) = E( θˆ ) θ 0 ( θˆ ) = θ Bas = E. Alteratvely, the estmator θˆ s a based estmator of the populato parameter θ f the bas of θˆ s o-zero;.e., f Bas ( θ ˆ ) = E( θˆ ) θ 0 ( θˆ ) θ E. 1. The estmator θˆ s a upward based (or postvely based) estmator the populato parameter θ f the bas of θˆ s greater tha zero;.e., f ( θ ˆ ) = E( θˆ ) θ 0 ( θˆ ) > θ Bas > E. 2. The estmator θˆ s a dowward based (or egatvely based) estmator of the populato parameter θ f the bas of θˆ s less tha zero;.e., f ( θ ˆ ) = E( θˆ ) θ 0 ( θˆ ) < θ Bas < E.... Page 6 of 15 pages

7 Meag of the Ubasedess Property The estmator θˆ s a ubased estmator of θ f o average t equals the true parameter value θ. Ths meas that o average the estmator θˆ s correct, eve though ay sgle estmate of θ for a partcular sample of data may ot equal θ. More techcally, t meas that the fte-sample dstrbuto of the estmator θˆ s cetered o the value θ, ot o some other real value. The bas of a estmator s a verse measure of ts average accuracy. The smaller absolute value s Bas ( θˆ ), the more accurate o average s the estmator θˆ estmatg the populato parameter θ. Thus, a ubased estmator for whch ( ) 0 ( θˆ ) = θ Bas θ ˆ = -- that s, for whch E -- s o average a perfectly accurate estmator of θ. Gve a choce betwee two estmators of the same populato parameter θ, of whch oe s based ad the other s ubased, we prefer the ubased estmator because t s more accurate o average tha the based estmator.... Page 7 of 15 pages

8 S2: Mmum Varace Defto of Mmum Varace: The estmator θˆ s a mmum-varace estmator of the populato parameter θ f the varace of the fte-sample dstrbuto of θˆ s less tha or equal to the varace of the fte-sample dstrbuto of θ ~, where θ ~ s ay other estmator of the populato parameter θ;.e., f ( θˆ ~ ) Var() θ Var for all fte sample szes such that 0 < < where () ˆ = E[ θˆ E() θˆ ] 2 ~ ~ ~ () = E[ θ E() θ ] 2 Var θ = the varace of the estmator θˆ ; Var θ = the varace of ay other estmator θ ~. Note: Ether or both of the estmators θˆ ad θ ~ may be based. The mmum varace property mples othg about whether the estmators are based or ubased. Meag of the Mmum Varace Property The varace of a estmator s a verse measure of ts statstcal precso,.e., of ts dsperso or spread aroud ts mea. The smaller the varace of a estmator, the more statstcally precse t s. A mmum varace estmator s therefore the statstcally most precse estmator of a ukow populato parameter, although t may be based or ubased.... Page 8 of 15 pages

9 S3: Effcecy A Necessary Codto for Effcecy -- Ubasedess The small-sample property of effcecy s defed oly for ubased estmators. Therefore, a ecessary codto for effcecy of the estmator θˆ s that E (ˆ θ) = θ,.e., θˆ must be a ubased estmator of the populato parameter θ. Defto of Effcecy: Effcecy = Ubasedess + Mmum Varace Verbal Defto: If θˆ ad θ ~ are two ubased estmators of the populato parameter θ, the the estmator θˆ s effcet relatve to the estmator θ ~ f the varace of θˆ s smaller tha the varace of θ ~ for ay fte sample sze <. Formal Defto: Let θˆ ad θ ~ be two ubased estmators of the populato ~ parameter θ, such that E ( θˆ ) = θ ad E () θ = θ. The the estmator θˆ s effcet relatve to the estmator θ ~ f the varace of the fte-sample dstrbuto of θˆ s less tha or at most equal to the varace of the fte-sample dstrbuto of ~ θ ;.e. f ( θˆ ~ ~ Var ) Var() θ for all fte where E ( θˆ ) = θ ad ( θ) = θ E. Note: Both the estmators θˆ ad θ ~ must be ubased, sce the effcecy property refers oly to the varaces of ubased estmators. Meag of the Effcecy Property Effcecy s a desrable statstcal property because of two ubased estmators of the same populato parameter, we prefer the oe that has the smaller varace,.e., the oe that s statstcally more precse. I the above defto of effcecy, f θ ~ s ay other ubased estmator of the populato parameter θ, the the estmator θˆ s the best ubased, or mmum-varace ubased, estmator of θ.... Page 9 of 15 pages

10 Nature of Large-Sample Propertes 5. Large-Sample Propertes! The large-sample propertes of a estmator are the propertes of the samplg dstrbuto of that estmator as sample sze becomes very large, as approaches fty, as.! Recall that the samplg dstrbuto of a estmator dffers for dfferet sample szes --.e., for dfferet values of. The samplg dstrbuto of a gve estmator for oe sample sze s dfferet from the samplg dstrbuto of that same estmator for some other sample sze. Cosder the estmator θˆ for two dfferet values of, 1 ad 2. I geeral, the samplg dstrbutos of ˆθ ad ˆθ 1 are dfferet: they ca 2 have dfferet meas dfferet varaces dfferet mathematcal forms Desrable Large-Sample Propertes L1. Cosstecy; L2. Asymptotc Ubasedess; L3. Asymptotc Effcecy. We eed to dstgush betwee two large-sample dstrbutos of a estmator: 1. the asymptotc dstrbuto of the estmator; 2. the ultmate, or fal, dstrbuto of the estmator.... Page 10 of 15 pages

11 The Ultmate Dstrbuto of a Estmator! The asymptotc dstrbuto of a estmator θˆ s ot ecessarly the fal or ultmate form that the samplg dstrbuto of the estmator takes as sample sze approaches fty (as ).! For may estmators, the samplg dstrbuto collapses to a sgle pot as sample sze approaches fty. More specfcally, as sample sze, the samplg dstrbuto of the estmator collapses to a colum of ut probablty mass (or ut probablty desty) at a sgle pot o the real le. Such a estmator s sad to coverge probablty to some value. A dstrbuto that s completely cocetrated at oe pot (o the real le) s called a degeerate dstrbuto. Graphcally, a degeerate dstrbuto s represeted by a perpedcular to the real le wth heght equal to oe. Dstgushg Betwee a Estmator's Asymptotc ad Ultmate Dstrbutos! The ultmate dstrbuto of a estmator s the dstrbuto to whch the samplg dstrbuto of the estmator fally coverges as sample sze "reaches" fty.! The asymptotc dstrbuto of a estmator s the dstrbuto to whch the samplg dstrbuto of the estmator coverges just before t collapses (f deed t does) as sample sze "approaches" fty. If the ultmate dstrbuto of a estmator s degeerate, the ts asymptotc dstrbuto s ot detcal to ts ultmate (fal) dstrbuto. But f the ultmate dstrbuto of a estmator s o-degeerate, the ts asymptotc dstrbuto s detcal to ts ultmate (fal) dstrbuto.... Page 11 of 15 pages

12 L1: Cosstecy A Necessary Codto for Cosstecy Let ˆθ be a estmator of the populato parameter θ based o a sample of sze observatos. A ecessary codto for cosstecy of the estmator ˆθ s that the ultmate dstrbuto of ˆθ be a degeerate dstrbuto at some pot o the real le, meag that t coverges to a sgle pot o the real le. The Probablty Lmt of a Estmator -- the plm of a estmator Short Defto: The probablty lmt of a estmator ˆθ s the value -- or pot o the real le -- to whch that estmator's samplg dstrbuto coverges as sample sze creases wthout lmt. Log Defto: If the estmator ˆθ has a degeerate ultmate dstrbuto --.e., f the samplg dstrbuto of ˆθ collapses o a sgle pot as sample sze -- the that pot o whch the samplg dstrbuto of ˆθ coverges s called the probablty lmt of ˆθ, ad s deoted as plm θ ˆ or plm θˆ where "plm" meas "probablty lmt".... Page 12 of 15 pages

13 Formal Defto of Probablty Lmt: The pot θ 0 o the real le s the probablty lmt of the estmator ˆθ f the ultmate samplg dstrbuto of ˆθ s degeerate at the pot θ 0, meag that the samplg dstrbuto of ˆθ collapses to a colum of ut desty o the pot θ 0 as sample sze. Ths defto ca be wrtte cocsely as plm ˆ θ = θ0 or p θ = θ0 lm ˆ. The three statemets above are equvalet to the statemet ( θ ε θˆ θ + ε) = lm Pr( ε θˆ θ + ε) = lm Pr( θˆ θ ε) = 1 lm Pr where ε > 0 s a arbtrarly small postve umber ad θ ˆ θ0 deotes the absolute value of the dfferece betwee the estmator ˆθ ad the pot θ Ths statemet meas that as sample sze, the probablty that the estmator ˆθ s arbtrarly close to the pot θ 0 approaches oe. 2. The more techcal way of sayg ths s that the estmator ˆθ coverges probablty to the pot θ Page 13 of 15 pages

14 Defto of Cosstecy Verbal Defto: The estmator ˆθ s a cosstet estmator of the populato parameter θ f ts samplg dstrbuto collapses o, or coverges to, the value of the populato parameter θ as. Formal Defto: The estmator ˆθ s a cosstet estmator of the populato parameter θ f the probablty lmt of ˆθ s θ,.e., f p lm θˆ = θ or lm Pr( θˆ θ ε) = 1 Pr θ ˆ θ ε as. or ( ) 1 The estmator ˆθ s a cosstet estmator of the populato parameter θ f the probablty that ˆθ s arbtrarly close to θ approaches 1 as the sample sze or f the estmator ˆθ coverges probablty to the populato parameter θ. Meag of the Cosstecy Property As sample sze becomes larger ad larger, the samplg dstrbuto of ˆθ becomes more ad more cocetrated aroud θ. As sample sze becomes larger ad larger, the value of ˆθ s more ad more lkely to be very close to θ.... Page 14 of 15 pages

15 A Suffcet Codto for Cosstecy Oe way of determg f the estmator ˆθ s cosstet s to trace the behavor of the samplg dstrbuto of ˆθ as sample sze becomes larger ad larger. If as (sample sze approaches fty) both the bas of ˆθ ad the varace of ˆθ approach zero, the ˆθ s a cosstet estmator of the parameter θ. Recall that the bas of ˆθ s defed as ( θˆ ) = E( θˆ ) θ Bas. Thus, the bas of ˆθ approaches zero as f ad oly f the mea or expectato of the samplg dstrbuto of ˆθ approaches θ as : lm Bas ( θˆ ) = lm E( θˆ ) θ = 0 lm E ˆ. ( θ ) = θ! Result: A suffcet codto for cosstecy of the estmator ˆθ s that lm Bas ( θˆ ) = 0 or lm E( θ ) = θ ad lm Var( θ ) = 0 ˆ ˆ. Ths codto states that f both the bas ad varace of the estmator ˆθ approach zero as sample sze, the ˆθ s a cosstet estmator of θ.... Page 15 of 15 pages