Michael J. Rosenfeld, draft version 1.7 (under construction). draft November 5, 2015

Transcription

1 Mchael J. Rosefeld, draft verso 1.7 (uder costructo). draft November 5, 015 Notes o the Mea, the Stadard Devato, ad the Stadard Error. Practcal Appled Statstcs for Socologsts. A troductory word o phlosophy of the class: My goal ths class s to gve you a tutve uderstadg of some basc statstcal deas, ad to gve you practcal experece wth usg basc statstcs. Toward these pedagogcal eds, I dspese wth as much statstcal formalty as I possbly ca. We wll talk a lttle bt about lear algebra ad about calculus (two bedrocks of statstcal theory) ths class, but oly as asdes. Cosder a varable X. E(X) s the expected value of X or the mea of X. The formal defto of E(X) s E( X) xp( x) f X s a dscrete fucto, meag you sum up the dfferet outcomes weghted by how lkely each dfferet outcome s. If X s a cotuous fucto, the expectato s defed ths way: EX ( ) xf( xdx ) where f(x) s the probablty desty fucto. Expectato s a mportat dea, but t s somewhat abstract. If X s Normally dstrbuted (a assumpto that s actually qute reasoable the kds of data ad questos we wll be lookg at), ad we have a buch of x s to observe, the sample mea of our x s s actually equal to the expectato of X. Sce the sample mea s very cocrete ad tagble ad already famlar to you, I am gog to talk a lot about the sample mea ad ot so much about E(X). These are otes o the Sample mea, the Varace, the Stadard Devato, ad so o. I ths dscusso you wll have to kow a few basc thgs about summary otato: 1 X ( X X... X ) ax a X ( ax b ) a X b 1 1 1

2 I words, summary otato s just a sum of thgs. No bg deal. Whe you multply each value by a costat, t s the same as multplyg the sum by the same costat. If the sght of summary otato scares you, do t worry. Summary otato s just shorthad for a smple dea. 1) The Sample mea, or the average. If we have observatos, X 1, X,...X, the average of these s smply 1 Avg( X ) X 1 I other words, you take sum of your observatos, ad dvde by the umber of observatos. We wll geerally wrte ths more smply as 1 Avg( X ) X Ths s a formula you are all famlar wth. The smple formula has some terestg mplcatos. ) How the Average chages whe we add or multply the X s by costat values. I all the below equatos, a ad b are costats. 1 Avg( ax ) ax a X a( Avg( X )) Whe we take a varable ad double t, the average also doubles. That should be o surprse. To be slghtly more geeral: Avg( a bx ) a b( Avg( X )) (ths s easy eough to show. See Homework ) 3) Also, the Average of a sum of two varables s the sum of the Averages. More formally: Avg( X ) ( X ) ( X) ( ) Avg( X) Avg( )

3 If X s Jauary come, ad s February come, the the Average of Jauary plus February come s the same as the Average for Jauary plus the Average for February. No surprse there. 4) The sample varace, defed: 1 Var( X ) ( X Avg( X )) The Varace s bascally the average squared dstace betwee X ad Avg(X ). Varace ca t be egatve, because every elemet has to be postve or zero. If all of the observatos X are the same, the each X = Avg(X ) ad Varace=0. Varace has some dow sdes. For oe thg, the uts of Varace are squared uts. If X s measured dollars, the Var(X) s measured dollars squared. That ca be awkward. That s oe reaso we more usually use the stadard devato rather tha the varace s that the stadard devato (just the square root of the varace) puts the uts back to the uts of X. Sometmes the sample varace s calculated wth 1/(-1) rather tha 1/. Wth large eough samples, the dfferece s small. For smplcty s sake, we wll stck wth the 1/. 5) How does varace respod to chages scale? Var a bx b Var X ( ) ( ) If you move the bell curve over (dsplacemet by a), the varace does ot chage. If you crease the X by a factor of b, the varace creases by b. 6) How about the Varace of the combato of two varables? Var( X ) Var( X ) Var( ) Cov( X, ) Var( X ) Var( X ) Var( ) Cov( X, ) If X ad are depedet, the covarace(x,)=0, ad lfe becomes smple ad sweet. Varace of (X+) s smply Var(X)+ Var(). Also ote that Var(X-)= Var(X)+Var(), because you could thk of - as (-1). If you take the dstrbuto ad move t to the egatve umbers, the varace s stll the same. Of course we could just calculate the covarace (t s ot that hard). But most of the tme t s smpler ad cleaer to make the assumpto of depedece (ad sometmes, t s eve true!) 7) Stadard Devato StDev( X ) Var( X )

4 Stadard Devato s smply the square root of the varace. Stadard devato of X has the same uts as X, whereas varace has squared uts. Whe you wat to kow the stadard devato of the combato of two varables, the easest thg to do s frst calculate the varaces, ad the take the square root last. 8) Stadard Error of the Mea Usually socal statstcs we are terested ot oly dstrbuto of a populato (let s say, the come of Nurses), but also the mea ad the comparso of meas (do urses ear more tha socologsts? How sure are we?) So let s look at the varace ad stadard error of the mea. How sure are we about the mea eargs of urses? 1 1 Var( Avg( X )) Var( X ) Var( X ) 1 Because Var(bX )=b Var(X ). Now we take advatage of the fact that the X s are depedet, ad detcally dstrbuted, so that the covarace betwee them s zero: Var( X ) Var( X X... X ) ( ) Var( X ) ( ) Var( X ) 1 1 O the mportace of sample sze. Stadard Devato of the mea s usually called the Stadard Error: Stadard Error= Stdev( Avg( X )) Var( X ) What s ew here s the factor of square root of the deomator. What ths meas s the larger the sample sze, the smaller the stadard error of the mea. Hgh stadard error meas we are less sure of what we are tryg to measure ( ths case the average of X). Small stadard error mples that we are more sure. Sample sze s crucal socal statstcs, but f you wat a stadard error half as large, you eed a sample sze 4 tmes as bg (because of the square root). If you crease the sample sze, the populato varace of urse s come stays the same, but the stadard error of the mea of urse s come decreases. It s mportat to keep md the dfferece betwee stadard devato of the populato ad the stadard error of the mea. 9a) Now let s say we wat to compare two meas, X ad, say they are the comes of lawyers ad urses. The dfferece betwee the meas s easy to calculate- t s just average come of the lawyers mus average come of the urses. But what about the stadard error of the dfferece? ou take the stadard errors of the dvdual meas of X ad, ad you square them, to get the varace of the mea. The you add them together to get the varace of the dfferece (because Var(X-)= Var(X)+ Var() uder depedece), the you take the square root to get the stadard error of the dfferece.

5 StdError( Avg( X ) Avg( )) ( StdError( Avg( X ))) ( StdError( Avg( ))) j j Avg( X ) Avg( j) the dfferece T Statstc StdError( Avg( X ) Avg( )) stadard error of that dfferece j whch we wll compare to the Normal dstrbuto or sometmes to the T dstrbuto (a close ad slghtly fatter relatve of the Normal dstrbuto). The T-statstc s ut free, because t has uts of X the umerator ad deomator, whch cacel. The T-statstc s also mmue to chages scale. I kow all the algebra looks a lttle dautg, but the dea s smple, ad t s the bass for a lot of hypothess testg. See my Excel sheet for a example. Why does the average dvded by ts stadard error take a Normal (or close to Normal) dstrbuto? There s a famous theorem statstcs called the Cetral Lmt Theorem whch explas why. Ths Theorem requres a lot of advaced mathematcs to prove, but the basc pot s ths: No matter what shape the dstrbuto of the X s take- t could be flat, t could have three modes, etcetera, the mea of the X s approaches a Normal dstrbuto as grows large. 10a) The T-Statstc I descrbe above s the T-statstc whch ackowleges that the varace ad stadard devatos of sample X ad sample may be dfferet. Ths s called the T-Test wth uequal varaces, ad ca be wrtte ths way (where x s the sample sze of X, ad y s the sample sze of ). Note that the deomator we just have the square root of the sum of the varace of the mea of X ad the varace of the mea of : T Statstc Avg( X ) Avg( ) Var( X ) Var( ) x 10b) If we are comparg our mea to a costat, ote that costats have varace of zero, so Avg( X ) cost Avg( X ) cost Avg( X ) cost T Statstc x Var( X ) Std Error(Avg(X)) Var( X ) x Our basc T-statstc s proportoal to the square root of, ad takes -1 degrees of freedom.

6 11) Although the T-statstc wth uequal varace s the most tutve, the more commo T-statstc, whch s also the T-statstc we wll ecouter Ordary Least Squares (OLS) regresso s the T-Statstc whch assumes equal varaces X ad. The assumpto of equal varace s called homoskedastcty. I fact, real data very frequetly have heteroskedastcty, or uequal varaces dfferet subsamples. The equal varace or homoskedastc T-Statstc s: T Statstc Avg( X ) Avg( ) x 1 Var( X) 1 Var( ) 1 1 x x ou ca show that these two formulas (T-statstc for uequal varace ad T-statstc for equal varace) are the same whe Var(X)=Var(). Ad ote that the equal varace T statstc, the deomator s the square root of the weghted sum of the varaces of mea of X ad mea of. 1) Whe lookg up the T-statstc o a table or Stata, you eed to kow ot oly the T-statstc but also the degrees of freedom, or the of the test. For the equal varace T- statstc, the df of the test s x + y -. The degrees of freedom for the uequal varace T- statstc s gve by Satterthwate s formula, ad t s a lttle more complcated (you ca look the formula up the STATA documetato or ole, but you do t eed to kow t. For Satterthwate s formula, that s for the df of the uequal varace T-test, f (Var(X)/ x ) (Var()/ y ), meag that the stadard errors of our two meas are smlar, the for the uequal varace T-test df x + y, whch meas the df s smlar to the df we would get wth the equal varace test (whch makes sese sce the stadard errors are early equal, so the equal varace assumpto s vald). If (Var(X)/ x )>>(Var()/ y ), the for the uequal varace T-test the df x, because the (Var(X)/ x ) wll domate the combed varace, ad that meas that the Xs are determg the combed varace, the the sample sze of the Xs should determe our degrees of freedom for the T-statstc. But do t worry too much about the degrees of freedom of the T-test! Bg chages the df of the T-test may ot chage the substatve outcome of the test- the T dstrbuto chages wth chages df, but for df>10 the chages are farly subtle. Eve a chage from 10 to 1000 df mght ot result a dfferet substatve aswer. A T- statstc of.5 wll correspod to a oe-tal probablty of.4% wth 10 df ( tal probablty of 4.8%), whle the same statstc of.5 would result a oe-tal probablty of 1.% o 1,000 df ( tal probablty of.4%). So for 10 df or 1,000 df or ay hgher umber of df, a T-statstc of.5 yelds a -tal probablty of less tha 5%, meag we would reject the ull hypothess. 1.1) The Fte Populato Correcto ad the Samplg Fracto Above Secto 8, I defed the Stadard Error of the mea ths way:

7 Stadard Error= Stdev( Avg( X )) Var( X ) I fact, ths defto leaves somethg out: the Fte Populato Correcto. A more accurate formula for the Stadard Error of the mea s: Var( X ) 1 Stadard Error= Stdev( Avg( X )) 1 N 1 where s the sample sze of our sample (133,710 the CPS), ad N s the sample sze of the uverse that our sample s draw from (74 mllo people the US), /N s the samplg fracto (about 1/000 the CPS), ad 1 1 N 1 s the Fte Populato Correcto, or FPC. Note the followg: Whe <<N, whch s the stuato we usually face, FPC 1 whch s why we usually gore the FPC. Also ote that whe <<N, ad FPC 1, t s oly the small (sample sze of our sample) ad ot large N (sze of the uverse our sample was draw from) that matters the stadard error formula. What ths meas, practce, s that a 500 perso opo survey s just as accurate a strumet to test opos a small state lke Mae as a large state lke Calfora. As log as <<N, we do t really care how bg N s. Whe s 500 ad N s 100,000, FPC s Whe s 500 ad N s 1,000,000, FPC s Whe s 500 ad N s 35,000,000, FPC s Geerally we treat these FPC all as 1, ad we gore t. Whe =N, samplg fracto =1, ad FPC=0. Whe =N, the stadard error of the mea s zero, whch makes sese because f we have the etre sample uverse measured our had, there s o statstcal ucertaty left, ad our measured mea s the true mea. Whe you have the whole sample uverse your data, for stace the votes of all 100 seators or the data from all 50 states, you ca stll ru models, but you caot treat the stadard errors ad probabltes that STATA reports as real probabltes descrbg ucertaty what we kow about the sample uverse, sce there s o ucertaty. We kow what we kow. Whe.01<samplg fracto<.9, the we have to thk serously about the FPC, or let STATA thk serously about t.

8 13) Ordary Least Squares regresso, or OLS. OLS s the orgal kd of regresso ad the kd we wll be dealg wth ths class, s a method of comparg meas. Cost B1X 1BX... BX resdual or predcted Cost B1X 1BX... BX Where s the depedet varable,.e. the thg we are tryg to predct. It s mportat to ote that the predcted values of the model wll ot geeral equal the real values. The X s are the depedet, or predctor varables. The B s are the coeffcets for each varable whch are produced by the regresso. Each B wll have a stadard error ad a resultg T-statstc to tell us whether the B s sgfcatly dfferet from zero or ot. The resduals have a mea of zero. I theory, the resduals are supposed to be Normally dstrbuted ad well behaved, lke pure ose. I realty, the resduals are oly Normally dstrbuted pure ose f the model fts the data very well. If the model fts poorly, the resduals ca have all sorts of odd patters. The costat equals the average of predcted whe all the X s are zero. Sometmes zero values for the X s make sese (.e. correspod to real subgroups the populato). Sometmes zero values of the X s do t correspod to real subgroups the populato, ad that case the costat does ot have a useful or substatve terpretato. 13.1) Correlato, r, ad R-squared. 1 Cov( X, ) ( X Avg( X ))( Avg( )) ad ote that Cov(X,X)=Var(X). r, the correlato coeffcet, also kow as Pearso s correlato, s defed ths way: r xy, Cov( X, ) Cov( X, ) Corr( X, ) Var( X ) Var( ) SD( X ) SD( ) Pearso s correlato r rages from -1 (a perfect egatve correlato) to 1 (a perfect postve correlato). Whe r=0, there s o lear relatoshp betwee X ad. Wth a sgle predctor X 1, ad a OLS equato =a+bx 1, the regresso le slope wll Cov( X1, ) be: b, so the slope of the regresso le must be same drecto (postve Var( X ) or egatve) as Corr(X 1,), but the slope b s ot bouded the way the correlato r s. Furthermore, the slope b ca oly be zero f r s zero. Oce you kow b, the a Avg( ) b( Avg( X )) 1

9 Now cosder the followg sums of squares from a geeralzed OLS regresso model (whch ca have may predctor varables): SS Avg Var tot reg res ( ( )) ( ) ( predcted ( )) SS Avg SS resdual I OLS regresso, SS tot =SS reg +SS res The R-square, or R, also kow as the coeffcet of determato, s defed: R SSreg SSreg SS SS res reg 1 SS SS tot SStot tot Var( ) R-square ca be terpreted as the proporto of Var() (the varace of the outcome varable) that s explaed by the model. R-square vares betwee 0 ad 1; R-square s 0 f the model explas oe of Var(), ad the value of R-square s 1 f the model fts the data perfectly, so that all resduals are zero. I practce, wth models predctg socal outcomes, R-square of 0. or 0.3 s ofte as good as you ca expect to get wth a reasoable model. It s also worth otg that a model wth predctor varables, that s oe term for every observato the dataset, could be desged to ft the data exactly, so that resduals would be zero ad R-square would be 1. But f s large ( our CPS dataset, s typcally the tes of thousads), a model wth predctors would be useless to us because t would ot have smplfed the data at all. I a OLS regresso wth oe predctor varable, X 1, the R-square of the model =a+bx 1 s the square of the Corr(X 1,), so that R =(r). That s how R-square got ts ame. Oe thg to remember s that Cov(X,)=Cov(,X), ad that Corr(X,)=Corr(,X), whch meas the R-square for the OLS regresso =ax 1 +b wll be the same as the R-square for the OLS regresso X 1 =c+d. Wth regressos, however, usually oly oe of the models makes sese, we usually wat the predctor varables to be logcally or temporally pror to the depedet varable. Uts: Var(X) s uts of X. Std(X) s uts of X. Cov(X,) s uts of X. Corr(X,) s ut-free. The regresso le slope b s uts of /X. R-square s utfree.

10 The more depedet varables or predctors you put to the model, the closer your predcted should be to, ad the lower SS reg should be. New predctors caot make the model ft worse; the ew predctor would be exactly zero (ad ot chage the predcted values at all) f the ew predctor had o value at predctg at all (et of the other predctors already the model). R-square creases mootocally as you add terms to ested models, so we eed some other way to compare goodess of ft betwee models. Note: Two models are ested f they cota the same set of observatos, the same fuctoal form, the same depedet varable, ad oe model s predctor varables are a subset of the secod model s predctor varables. Whe comparg model fts ths class, we wll wat the models to be ested (though there are some who argue that Bayesa ways of comparg model fts, such as the BIC, do t requre the models to be ested). Frst approach: adjust the R-square for the umber of terms the model. The defto of the adjusted R-square s: k R R k 1 Adjusted R-Square= (1 ) where R s the regular R-square statstc, k s the umber of terms the model (ot coutg the costat term), ad s the sample sze of your dataset. The Adjusted R- square s ot bouded by [0,1] the way R-square s. Adjusted R-square s adjusted because the adjusted R-square pealzes you for addg more terms to the model; the larger k s, the larger the pealty s. Hgher values of adjusted R-square dcate better ft, but as far as I kow there s o statstcal test for the comparso of adjusted R-squares. ou ca smply exame the adjusted R-squares of ested models, ad whchever model has the hgher adjusted R-square s the better fttg model (by the metrc of adjusted R- square). For regular (uadjusted) R-square, you would ot wat to smply compare the R- squares of ested models, because the R-square of the model wth more terms would have to be hgher. The regular uadjusted R-square statstcs, whe derved from OLS regresso, o the other had, ca be compared wth a statstcal test, specfcally the F- test: Fm (, k1) B A B 1 R (k1) R R m Where R B s the R-squared (regular R-squared rather tha adjusted) from a larger model B, R s the R-squared from a smaller model A ested wth B, m, s the umber of A degrees of freedom dfferece betwee A ad B, ad k s the umber of predctor varables B (besdes the costat), ad s the sample of subjects both models. Because of some ce propertes of OLS regresso, ths goodess of ft test s the same

11 test as the F-test that the m ew terms model B are all jotly ozero, ad the test s easy to get Stata to perform after you ru model B. A very bref troducto to Logstc Regresso 14) Ordary Least Squares regresso, OLS regresso, has all sorts of ce features ad s the orgal kd of regresso, sort of lke the Rose Bowl s the orgal college football bowl game. But there are some kds of data that do t led themselves easly to OLS regresso. Take, for stace the case of depedet varable whch s a es/no kd of varable,.e. ca be coded 0 or 1. There are lots of varables lke ths. If we try to ru depedet varable through our OLS regressos, we mght get predcted values of that were greater tha 1 or less tha 0, because s assumed to be Normally dstrbuted, ad could theory take o ay values. But greater tha 1 or less tha zero mght be out of the acceptable rage, so we eed to mpose a trasformato o to keep all predcted values the (0,1) rage. The most commo ad popular trasformato s the logstc trasformato. The logt trasformato looks lke ths: Logt 1 ( ) L( ) The logt trasformato covers the etre real umbers whe predcted values of are betwee 0 ad 1, whch s what we wat. Here L s the atural logarthm, that s the logarthm wth base e (where e.7183) Logstc regresso has a smlar form o the rght had sde: (14.1) L( ) Cost B 1X1BX... BX 1 It wo t be vsble to you, but logstc regresso gets estmated recursvely, whereas OLS regresso gets solved drectly. Because we have trasformed the left had sde of our regresso equato, the coeffcets o the rght, the betas, eed to be terpreted dfferetly... If we expoetate both sdes ( ) e 1 ( CostB1X 1BX... BX ) ad the, because of a property of expoets that e (a+b) =e a e b

12 Cost BX 1 1 BX (14.) ( ) e e e... e 1 BX The left had sde s the odds of, So you ca thk of the expoetated betas as factors B1 the odds. If we crease X 1 by 1, we crease the odds by a factor of e Whch meas (14.3) ~ (X1 X 1) Cost ( ) e e e e e e e 1 ~ (X1 X 1) B1( X11) BX Cost B1X1 B1 BX The value of the predcted odds before cremetg X s: (14.4) ~ (X 1 X ) Cost ( ) e e e 1 ~ (X 1 X ) BX 1 1 BX So f we take the rato of equatos 14.3/14.4, we get that the expoetated coeffcet s actually a odds rato, the rato of the predcted odds wth X 1= X+1 to the odds whe X 1 =X. If you thk of X 1 as a categorcal varable, for marred (compared to umarred) B1 or for black (compared to whte), the e s just the rato of the predcted odds of the outcome varable for marred (compared to umarred) or for black (compared to whte). e B1 ~ (X1 X 1) ( ) 1 ~ (X1 X 1) ~ (X 1 X ) ( ~ ) (X 1 X ) 1 B1 Whch meas e s a odds rato, or the rato of two odds. I practce, we do t have to go through ths algebra very much. Also, the logstc regresso wll produce coeffcets wth famlar Normal dstrbutos ad Z-statstcs. I the coeffcet (the u-expoetated) verso of logstc regresso output, equato 14.1 above, the coeffcets are Normally dstrbuted wth 95% cofdece terval (coef-1.96se, coef SE). As OLS regresso, the ull hypothess for coeffcets s that the coeffcet s zero. I the expoetated verso, equato 14. above, the ull hypothess s that the expoetated coeffcet=1, because e 0 =1, ad because 1 s the

13 multplcatve detty. Addg zero does t chage the predcted values equato 14.1, ad multplyg by 1 does ot chage the predcted values equato 14.. Note that whle the uexpoetated coeffcets are ce ad Normally dstrbuted, wth a ce symmetrc 95% cofdece terval wth the coeffcet the mddle, the expoetated coeffcet s ot Normally dstrbuted ad ts cofdece terval s ot symmetrc ay more. The expoetated verso of the cofdece terval aroud the coeffcet s (e coef-1.96se, e coef SE ), ad sce the expoetato fucto creases large umbers more tha t creases small umbers, the cofdece terval s o loger symmetrc aroud the coeffcet. If the coeffcet s 5 ad ts stadard error s 1, the 95% CI for the coeffcet would be approxmately (3, 7). The coeffcet would be sgfcat because t s 5 stadard errors greater tha zero, so the Z-score would be 5. I expoetated terms, e 5 =148, ad the 95% cofdece terval would be (e 3, e 7 ), or (0,1097), whch as you ca see s ot symmetrc aroud 148. It s mportat to keep md that a odds rato of, for stace, does mea that the predcted probabltes wll be doubled. The whole pot of the logt trasformato s that the odds rato ca be as hgh as t wats wthout ever allowg the predcted probablty of the outcome to be as hgh as 1 (or as low as zero). If we start wth a odds rato as the rato of two predcted odds of a postve outcome, P 1 P e P 1 1 P1 Ad we solve ths for P, (because we wat to kow how great the probablty of success wll be gve startg probablty P 1 ), we get: P 1 e 1 P1 P P 1 1 e 1 P1 Let s say we go back to our equato 14.1, the (uexpoetated) coeffcet verso of loglstc regresso: L( ) Cost B 1X1BX... BX 1 The rght sde of the equato s very famlar, t s just a sum of coeffcets multpled by X values, wth the costat added o. So how would we get to actual predcted values, that s? Ths s smply a matter of solvg the above equato for. Let s start by

14 sayg that the total o the rght sde of the equato s W, ad W ca be ay value, postve or egatve. The solvg for we have: L( ) W 1 the expoetate both sdes, ( ) e 1 ad W e ad W W e e ad W W e e Factorg, W W e e ad (1 ) W (1 ) ( ) W e W (1 e ) ( ) If you look at the rght sde of that equato, regardless of W s value, e W must be postve. Ad sce 1+ e W must be greater tha e W, we kow that 0< < 1, whch s what we wat.

15 Some Commets o the Chsquare Dstrbuto: wth (teger) degrees of freedom, (1 / ) x e f( x) ( /) / ( /) 1 x/ for x 0 Mea= Varace= Stadard Devato a) The chsquare dstrbuto has a doma of all postve real umbers, meag x 0. The Greek letter Γ the deomator s just Gamma, dcatg the Gamma fucto. The Gamma fucto s othg more tha a facy way of extedg the factoral fucto to all the real umbers. Γ(x)= (x-1)!, whe x s a teger. b) χ (1), or the Chsquare dstrbuto wth oe degree of freedom s defed as the square of a Stadard Normal varable. I other words, f z has the famlar Normal(0,1) dstrbuto whose cumulatve dstrbuto s the source of tables the back of every statstcs text book (.e. Normal wth mea of zero ad varace of 1), ad f y=z, the y has a χ (1) dstrbuto. Ths also meas that f you have a statstc expressed a value from a χ (1) dstrbuto, you ca take the square root ad you wll have the famlar z-score. Whe swtchg back ad forth from χ (1) ad Normal(0,1) you do have to keep md that the Normal dstrbuto has two tals ( the postve ad egatve drectos), whereas as the Chsquare dstrbuto oly has the oe tal. c) Uder depedece, χ (a)+ χ (b)= χ (a+b). Aother way to look at ths s that χ (a)= χ (1)+ χ (1)+ χ (1)+... a tmes (wth each compoet beg depedet). Gve what we kow about the Cetral Lmt Theorem, you would expect that χ () would look more ad more lke the Normal dstrbuto, the larger gets (sce χ () s just combatos of depedet χ (1) varables). The examples of the chsquare dstrbuto wll verfy that that χ (16) looks qute Normal (ad ths case t approxmates Normal(16,3)). d) Oe property of the Chsquare dstrbuto that s relevat to evaluatg the relatve ft of dfferet models that are ft by lkelhood maxmzato (such as logstc regresso), s that f Model 1 has -loglkelhood, or -LL=V, ad Model adds m addtoal terms ad has -LL=U, the the comparso of Model 1 ad Model s χ (m)=v-u. Ths comparso oly works f Model 1 s ested wth Model (that s, f Model cotas Model 1). Ths s the Lkelhood Rato Test, or LRT, whch I descrbe a lttle more detal below.

16 e) The F-dstrbuto s defed as a rato of two chsquare dstrbutos. If P s dstrbuted as χ (m) ad Q s depedet from P ad s dstrbuted as χ (), ad Pm W, the W s has the F m, dstrbuto, that s the F dstrbuto wth m ad Q degrees of freedom. Lkelhood ad the Lkelhood Rato Test: Let s say that s the vector of all the parameters our model, the several rght-sded predctor varables we are tryg to fd the best values for. The lkelhood of a model s the jot pot probablty that every data pot our sample would have ther exact values of Lkelhood( ) f( y1, y, y3,..., y ) ad Lkelhood( ) f( y ) 1 There are a few thgs to kow about the lkelhood. Frst of all, the pot probablty p of ay sgle exact outcome value, eve gve the best fttg model, wll almost always be 0<p<1 (although probablty desty fuctos ca yeld pot values of equal to or greater tha 1 uder some uusual crcumstaces, thk of the uform dstrbuto, or the Chsquare(1) dstrbuto for small values of x). If s at all substatal sze (as t almost always s the datasets we use lke the CPS), the the product of the (already very small) lkelhoods s gog to be pheomeally small- ftesmal. We do t really care about the value of the lkelhood of the model, except that Stata wll choose to maxmze the lkelhood. We do t care about the value of the lkelhood tself, but we do care about the comparso of the lkelhoods of two ested models. Frst thgs frst: If we wat to fd the value (or values) of that wll maxmze the jot lkelhood, we do t wat to deal wth a eormous product of probabltes, because products of fuctos are dffcult to dfferetate. What we do stead s we take the atural log of the lkelhood, the log lkelhood: log lkelhood l( ) l( f ( y )) 1 Because the logarthm s a fucto that s mootoc, that s t has o maxma or mma of ts ow, the that maxmzes the log lkelhood s the same that maxmzes the lkelhood, ad the log lkelhood almost always s easer to work wth.

17 Lkelhoods are a kd of probablty, so geeral 0<lkelhood( )<1. Log lkelhood, or l( ) s gog to be egatve, because t s a sum of egatve elemets. Every tme we add ew varables to our model (keepg the same set of data pots), the lkelhood wll be larger (or the same f the added terms are all zero), ad the log lkelhood wll be greater (meag less egatve,.e. closer to zero). The -LL, or mus two tmes the log lkelhood wll be postve, ad wll get smaller ad smaller wth each ew term that mproves the ft of the model. The lkelhood rato test s the a test of the rato of the lkelhoods of two ested models, whch we covert to the dfferece of the -LL of the two ested models (because t s always easer to work wth the log lkelhood tha to work wth the lkelhood tself), ad the dfferece of the -LLs s chsquare wth m degrees of freedom, where the larger model has m addtoal terms beyod what the smaller model has. The Normal ad T dstrbutos: The probablty desty fucto of the Normal dstrbuto s defed ths way: f( x) 1 e ( x) for - <x< Expected value (or mea), E (X)=μ Varace (X)=σ ou ca see from the dstrbuto that t s symmetrcal aroud ts mea. If Z s dstrbuted as Normal(0,1)- the stadard Normal, ad U s dstrbuted as χ (), ad Z U ad Z are depedet, ad f T, the T s dstrbuted as the t-dstrbuto, U wth degrees of freedom.

18 OLS as a example of Maxmum Lkelhood: Recall the fuctoal form of our Normal dstrbuto: f( x) 1 e ( x) The Maxmum Lkelhood estmates for our lear regresso wll be OLS, that s Ordary Least Squares estmates f the errors are Normally dstrbuted,.e. f the errors are Gaussa (Gauss beg the mathematca credted wth descrbg the Normal dstrbuto). To get Stata s glm fucto to assume that the errors are Normally dstrbuted, you specfy the opto famly(gaussa). Now, f we assume that the errors are Normally dstrbuted, what would the Lkelhood fucto for the B 0 ad B 1 look lke, our regresso le of B B X resdual ( 0 1 ) B B X L ( B0, B1, ) e Where B 0 s smply our costat term, ad ( B0 B1X ) s the square of each resdual. Note that the errors of the true regresso model ad the resduals of the actual regresso model we use are ot the same, but the resduals are what we have to estmate the errors wth. We do t much care here about how s gog to be estmated, because wll be estmated by the square root of the sample varace of our data (wth a mor correcto that eed ot cocer us here). Here we are terested the estmates of B 0 ad B 1 that wll maxmze the Lkelhood, so let us press o a bt further. Remember that e e e e ( abc) a b c. We ca get rd of the Product otato, ad covert the expoetal fucto to a expoetal of a sum. 1 1 L ( B0, B1, ) e 1 ( B B X ) 0 1 I am ot gog to put you through the calculus of fdg the maxmum here, but the story should be clear from lookg at the lkelhood fucto. The sum the above equato s the sum of squared resduals. I order to maxmze the lkelhood, we eed to maxmze the expoetal fucto. I order to maxmze the expoetal fucto, whch s of the

19 f ( x) e form where f(x) 0, we eed to mmze f(x). I other words, the Maxmum Lkelhood soluto for B 0 ad B 1 s the soluto whch mmzes the sum of squared resduals, whch s the Least Squares soluto. OLS s the MLE soluto f we assume that the errors, ad therefore the resduals are Normally dstrbuted. The ext questo you should ask s: Is t reasoable to assume that the resduals wll be Normally dstrbuted? If the model s the true model, so that the data are perfectly predcted by the model except for ose, the of course the resduals wll be Normal. If the model s ot the true model (ad wth real lfe data, there are o true models), the the Normalty assumpto mght ot be vald. Plottg the resduals s a useful exercse, ad oe ofte wll fd that the resduals are very o-normal. O the other had, there s statstcal theory showg that OLS yelds the Best Lear Ubased Estmates (BLUE) for the coeffcets, eve f the errors are ot Normally dstrbuted. O Idepedece: Oe way you wll sometmes see the Chsquare dstrbuto voked s for tests of depedece betwee two varables. What does Idepedece mea ths cotext? If Varables X ad are Idepedet, the P(X )= P(X)P(). The just meas tersecto. ou ca read the above statemet as The probablty of X ad both happeg s the product of the probablty of X multpled by the probablty of. Let s say X s the probablty of husbads beg black, ad s the probablty of wves beg black. Let s further say that P(X)=P()=0.1, because blacks are about 10% of the US populato. So does that mea that P(X )= P(X)P()=0.1(0.1)=0.01? NO. I fact ths case P(X )= because husbad s race ad wfe s race are ot depedet from each other: there s a strog patter of selectve matg by race. If we compare crosstabulated data from two models, we ca geerate the chsquare statstc two ways. Here j are the cell couts from oe model (or the actual data), ad u j are the cell couts from the other model, or our case the cell couts from predcted values : Pearso Chsquare or ( ) j uj X u j Lkelhood Rato Chsquare or j G jl( ) u j

20 I Chsquare tests for depedece, the data tself has RC df, ad the depedece model has R+C-1 df, so the Chsquare test for depedece s o RC-(R+C-1) df.