Psych 5741 (Carey): 8/22/97 Parametric Statistics - 1

Transcription

1 Psych 5741 (Carey): 8//97 Parametrc Statstcs Parametrc Statstcs: Tradtonal Approach 11 Denton o parametrc statstcs: Parametrc statstcs assume that the varable(s) o nterest n the populaton(s) o nterest can be descrbed by one or more mathematcal unknowns Some types o parametrc statstcs make a stronger assumpton namely, that the varable(s) have a certan dstrbuton To llustrate, consder a smple problem: do male and emale US college students der n average heght? One type o parametrc approach s to assume that our mathematcal quanttes can descrbe heght n the populaton o college students the mean or emales, the mean or males, the standard devaton or emales, and the standard devaton or males A second parametrc approach mght assume that heght ollows a normal dstrbuton n both sexes 1 Key Terms: 11 Populaton: For learnng purposes, two types o populatons can be dstngushed n parametrc statstcs Ocally, statstcans never make ths dstncton, so the termnology gven below s not standard The two types o populaton are: 111 Specc populaton: Specc populatons consst o denable ndvdual observatons (eg, people, rats, ctes) where each ndvdual could be dented and enumerate, although t usually would be mpractcal to do so Examples o specc populatons would be: mule deer lvng n Rocky Mountans; US college sophomores; Amercan vctms o ncest The heght example would have two specc populatons, the populaton o all US male college students and the populaton o all US emale college students 11 Hypothetcal populaton: Hypothetcal populatons do not exst, hence the term hypothetcal They are magnary populatons o nnte sze and are used whenever parametrc statstcs assume that the varables are dstrbuted n a certan orm In the heght example, the hypothetcal populaton o US emale students would produce a perectly smooth hstogram whereas the specc populaton o US emale college students would produce a hstogram wth some small rregulartes When statstcans reer to the populaton, they usually speak about the hypothetcal populaton 1 Sample: A sample s a nte group o observatons taken rom a specc populaton There are numerous strateges or samplng Among them are:

2 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 11 Random Samplng In a random sample, each observaton n the specc populaton stands the same equal chance o beng selected nto the sample For obvous reasons, completely random samples are encountered very seldom n psychology In the heght example, t would be necessary to cover many derent campuses to obtan a completely random sample 1 Strated Random Samplng In strated random samplng, the specc populaton s dvded nto two or more groups (or strata) Each ndvdual wthn a strata stands the same, equal chance o beng sampled, but one or more strata are delberately oversampled A classc example would be a medcal study o ethnc derences n hypertenson that would sample an equal number o whte Amercans and Arcan-Amercans 13 Selected Samplng Selected samples occur when predened crtera are used to sample ndvduals rom the specc populaton For example, only chldren wth IQ scores less than 70 are sampled Selected samples have the advantage o ncreasng statstcal power (e, the ablty to detect eects that are really present) but the dsadvantage o lmted generalzaton 14 Convenence Samplng Convenence samplng occurs when the ndvduals are not truly representatve o the specc populaton but are easy to collect It s the most requently encountered samplng technque n psychology Convenence samplng always makes the assumpton that those varables on whch the sample s unrepresentatve o the specc populaton are uncorrelated (or have very low correlatons) wth those varables that are actually measured and studed For the heght example, a convenence sample would be to take a certan number o Unversty o Colorado men and women Ths strategy makes the reasonable assumpton that attendng the Unversty o Colorado, as opposed to other colleges, s not assocated wth sex derences n heght The assumpton would be volated, or example, tall women would preerentally choose Colorado over other nsttutons 13 Parameter A parameter s a mathematcal unknown n the populaton, ether the specc or the hypothetcal populaton It s customary to denote parameters wth Greek letters Some requently encountered parameters are µ (populaton mean), σ (populaton standard devaton), and ρ (populaton correlaton) The heght example could have our parameters: µ m and σ m (the mean and standard devaton or male college students) and µ and σ (the respectve parameters or emales)

3 Psych 5741 (Carey): 8//97 Parametrc Statstcs Statstc There are two types o statstcs used n parametrc statstcs They are: 141 Descrptve statstcs A descrptve statstc s an estmate o a populaton parameter, almost always obtaned through sample data For example, the sample mean s used to estmate the populaton mean Usually, Roman letters are used to denote descrptve statstcs such as X (sample mean), s (sample standard devaton), and r (sample correlaton) In the heght example, the estmates o the our parameters mght be: X m (the sample mean or males whch s an estmate o µ m, the populaton mean or males), and sm, X,and s (the three statstcs or the other populaton parameters) 14 Test Statstcs A test statstc s used to make nerences about one or more descrptve statstcs Usually, a test statstc does not drectly measure a populaton parameter, although n some cases t may be mathematcally manpulated to do so Ether Roman or Greek characters are used or test statstcs Examples o test statstcs would be usng a t test statstc to test whether two sample means der, usng an F test statstc to test whether two or more sample means der, and usng a χ test statstc to test whether two or more sample proportons der Unortunately or the begnnng statstcs student, t s customary to drop the word test rom test statstc Eg, most textbooks call the t test statstc the t statstc In the heght example, one could use the our descrptve statstcs ( X m, sm, X,and s ) and the sample szes to construct a t statstc that could gve normaton about average heght derences between males and emales 15 Probablty densty uncton The probablty densty uncton s also reerred to as pd or smply densty uncton The pd s a mathematcal uncton used to descrbe two mportant phenomena: (1) the dstrbuton o a varable(s) n the hypothetcal populaton; and () the dstrbuton o test statstcs Mathematcally, the pd ullls two condtons: (1) t gves the relatve requency (or probablty) o observng a value as a uncton o that value; and () the area under the curve between two values gves the probablty o randomly selectng a number between those two values The unknowns n a pd are parameters The probablty densty uncton most oten encountered n behavoral research s the normal probablty densty uncton (e, the normal curve) I (X) s the probablty o observng a value wthn a tny nterval o X, then the normal probablty densty uncton o X s X µ ( X ) = exp π σ σ

4 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 4 Here, the quantty π s the constant 314 and the term exp denotes the exponental uncton (e, the natural exponent, e, rased to the power o the stu n the parentheses) The two parameters or the normal curve are µ (populaton mean) and σ (populaton standard devaton Other mportant probablty densty unctons are the Posson, bnomal, multnomal, lognormal, exponental, and Webull Important probablty densty unctons or test statstcs are the t pd (or the t test statstc), the F pd (or the F test statstc), and the χ pd (or the χ test statstc) The pd or a test statstc s called the samplng dstrbuton o the statstc 16 Probablty dstrbuton uncton The probablty dstrbuton uncton s also reerred to as the dstrbuton uncton, cumulatve dstrbuton uncton, or cd Lke the pd, the cd s used or both hypothetcal populatons and or test statstcs The probablty dstrbuton uncton s the mathematcal ntegral o the probablty densty uncton (Hence, conversely, the probablty densty uncton s the dervatve o the probablty dstrbuton uncton) The cd gves the probablty o observng a partcular value o X that alls n between two values, say, X 1 and X So, or example, the probablty dstrbuton or the normal curve s X X µ Prob( X1 < X < X) = exp dx π σ σ X1 For example, X denotes IQ scores whch are assumed to be normally dstrbuted wth a mean o 100 and a standard devaton o 15, then the probablty dstrbuton uncton gves the probablty o randomly pckng an IQ score that alls somewhere n the nterval between X 1 and X I X 1 = 10 and X = 118, then ths normal probablty dstrbuton uncton gves the probablty o randomly selectng an IQ score between 10 and 118 or Prob( 10 < X < 118) = exp π σ 10 X µ dx σ Test statstcs have ther own cd s Hence, the cd or a t test statstc gves the area under the curve or a specc t dstrbuton 17 Samplng dstrbuton The samplng dstrbuton s the probablty densty uncton o a test statstc For completely perverted reasons (e, to delberately conuse the begnnng graduate student), statstcans always say the samplng dstrbuton o the t statstc, and never call t what t really s the probablty densty uncton o the t test statstc 18 Standard error For the same perverted reasons, statstcans reer to the standard devaton o the probablty densty uncton o a test statstc as the standard error o the statstc

5 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 5 Puttng 17 together wth 18, statstcans dene the standard error as the standard devaton o the samplng dstrbuton o a statstc 19 Mathematcal Model A mathematcal model o data conssts o one or more mathematcal equatons that gves the predcted or expected value or a partcular varable o an observaton For example, suppose that we assumed that the average heght or males was 8 centmeters (cm) greater than the mean heght or emales Then one could construct a mathematcal model that states the ollowng: I the observaton s a emale then the predcted heght s an unknown, say µ I the observaton s a male, then the predcted heght s µ + 8 The value or an observaton predcted by the mathematcal model s called the predcted value o the observaton 110 Error or Resdual The terms error and resdual are synonymous n statstcs Error (or a resdual) s the mathematcal derence between the actual observed value or an observaton and the observaton s predcted value (e, value predcted by a mathematcal model) 13 Steps n parametrc statstcs 131 Estmate the parameter(s) Gven a sample, one or more mathematcal ormula are used to obtan descrptve statstcs o the parameters and to then test hypotheses about these descrptve statstcs 1311 Methods or estmaton There are several derent methods that can be used to estmate parameters o sngle method s best, so the choce depends largely on the nature o the problem For the technques n ths course, all the methods oten gve the same answers or when they do not, they gve very smlar answers The Sum o Least Squared Error The sum o least squared error s most oten reerred to as smply least squares Least squares s the most requently encountered crteron or estmaton n the behavoral scences, so let s take some tme to explore t n detal Least squares begns wth the observed value or the rst observaton n the sample It develops a predcted value or ths observaton based on a mathematcal model The mathematcal model s dened as one or more equatons based on populaton parameters and other aspects o the observed data Least squares then subtracts the predcted value rom the observed value gvng what s called the predcton error (or more oten, error or resdual) or the rst observaton It then squares ths predcton error Usually, the observed value or the rst observaton s denoted algebracally as Y 1 (the subscrpt denotes the order o the observaton or 1 n ths case)

6 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 6 and the predcted value s denoted by the same symbol wth a hat (^) placed over t-- Y ˆ 1 The predcton error s usually denoted as e 1 Hence, algebracally, the predcton error or the rst observaton s e 1 = Y 1 Y ˆ ( 1 ) and the squared error or the rst observaton s ( ) e 1 = Y 1 ˆ Y 1 Least squares then goes to the second observaton and computes ts error and ts squared error Least squares contnues wth ths procedure or all the observatons n the sample The nal step s to add up all the squared errors gvng the sum o squared errors or SSE Algebracally, the ollowng two equatons are dentcal ways o wrtng SSE: and SSE = e 1 + e + + e = e SSE = (Y 1 Y ˆ 1 ) +(Y Y ˆ ) + + (Y Y ˆ ) = =1 (Y Y ˆ ) We are now n a poston to dene the sum o least squared error crteron or parameter estmaton Least squares statstcs are those estmates o the populaton parameters that mnmze the sum o squared predcton errors Mathematcally, least squares statstcs are those parameter estmates than generate all the Y ˆ s so that SSE s mnmzed To llustrate consder the emales n the heght example and suppose we would lke to nd the least squares statstc or µ, the mean heght or the populaton o emale college students We can wrte the model by lettng the predcted value or heght be the mean or Y = Y ˆ + e = µ +e What now s the descrptve statstc that best estmates the parameter µ n terms o mnmzng e =1? We wll eschew all the mathematcs that proves the obvous the best estmate o µ s Y, the mean heght n the sample o US college women = Maxmum Lkelhood(*) Maxmum lkelhood s another method or estmatng parameters Maxmum lkelhood estmators are those that maxmze the probablty o observng the data when the varable(s) n the hypothetcal populaton s(are) assumed to have a certan probablty densty uncton For example, the varable n the hypothetcal populaton s assumed to ollow a normal dstrbuton, then the maxmum lkelhood estmator o the populaton mean, µ, s that numercal value that maxmzes the probablty o observng

7 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 7 the sample data Lke least squares, the maxmum lkelhood estmator o µ s the sample mean Bayesan Estmaton(*) Baysan estmators are yet another type o estmators These estmators assume that a descrptve statstcs s tssel sampled rom a hypothetcal populaton o descrptve statstcs that has a certan dstrbuton called the pror dstrbuton It them nds the most lkely estmator gven the data Lke least squares and maxmum lkelhood, the Baysan estmator o µ s the sample mean Robust Estmators(*) Robust estmators are a whole class o estmators that are relatvely nsenstve to departures rom the assumptons that go nto the mathematcal model behnd ther estmaton An example o a robust estmator would be a trmmed mean whch deletes, say, the hghest three scores and the lowest three scores and then takes the mean o the remanng data ponts There s no specc crteron(a) developed to determne whether an estmator s robust or not robust 131 Crtera or Good Estmators(*) ot all estmators o a populaton parameter are good estmators Statstcans have establshed our propertes o good estmators They are: 1311 Unbasedness(*) Techncally, an unbased estmator s one whose expected value equals the populaton parameter To see what ths statement means, examne the sample mean heght or males as an estmator o the populaton mean heght or males Suppose that we were to draw a random sample o observatons rom the specc populaton o college males, compute the sample mean, and then enter ths sample mean nto a data set We then gathered another sample o observatons rom the specc populaton, calculated ts sample mean, and add ths new sample mean to the data set We contnue wth ths process untl we gathered an nnte number o sample means n the data set I the mean o all the sample means equaled the mean n the specc populaton, then the sample mean s an unbased estmator o the populaton mean 131 Consstency(*) A consstent estmator s one whose value gets closer and closer to the populaton parameter as the sample sze ncreases 1313 Ecency (*) Ecency s a relatve concept Gven two derent estmators o a populaton parameter, the estmator wth the greater ecency s the one wth the smaller standard error or ts samplng dstrbuton Return to the denton o unbasedness n 1311 Suppose or each sample drawn, we calculated two statstcs, the sample mean (whch s

8 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 8 entered nto one data set) and the sample medan (whch s entered nto a second data set) We could then calculate the mean and standard devaton o all the sample means as well as the mean and standard devaton o all the sample medans We would nd that both means equal the populaton mean Ths ndcates that both the sample mean and the sample medan are unbased estmators o the populaton mean However, we would nd that the standard devaton o the sample means s smaller than the standard devaton o the sample medans Ths ndcates that the sample mean s a more ecent estmator than the sample medan 1314 Sucency(*) A sucent estmator s one that cannot be mproved upon by usng any other aspects o the data other than the actual numbers that went nto the calculaton o the statstc For example, suppose that we used the sample mean or emales to estmate the populaton mean heght or emales Then the sample mean would be a sucent estmator no other aspect o the data (eg, eye color, GPA, astrologcal sgn, etc) could mprove upon estmatng the populaton mean 13 Test hypotheses The second major step n parametrc statstcs s to test hypotheses and make nerences about the descrptve statstcs Ths area s reerred to as nerental statstcs and t uses test statstcs Inerental statstcs are dened as that group o test statstcs used to determne the relatve probablty that a mathematcal model o the hypothess s true or alse The logc behnd ths s on the complcated sde and wll be descrbed later n the course For now, let us consder the heght example to obtan an ntal taste o the logc behnd hypothess testng Inerental statstcs assume that the hypothess can: (1) be stated n precse mathematcal terms (e, a mathematcal model); and () the mathematcal model s developed wthout lookng at the data The heght example asks whether the average heght ders or male and emale college students Lookng at the hypothetcal populaton, there are three logcal possbltes: (1) mean emale heght exceeds mean male heght or µ > µ m ; () mean emale and mean male heght are the same or µ = µ m ; and (3) mean emale heght s less than mean male heght or µ < µ m For reasons that wll become clear later n the course (hopeully!), possbltes (1) and (3) are not mathematcally precse because they cannot specy (or be mathematcally manpulated to specy) a concrete number For example, possblty (1) states that the emale mean s greater than the male mean but does not specy the concrete number o unts separatng the two means Possblty (), however, can be manpulated to gve a concrete number µ = µ m, then µ - µ m = 0 Hence, the mathematcal model that can be tested s µ - µ m = 0 Logcally, t seems arly smple to test ths I the sample mean s the best estmator o the populaton mean, then just substtute the sample means nto the equaton, gvng Y Y = 0 Consequently, a good test statstc would be the derence between the two m

9 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 9 sample means Let δ equal ths quantty so that the equaton or the test statstc becomes δ = Y Y I δ = 0, then the hypothess o no average derence n heght m between males and emales s supported by the data But there s an obvous problem wth ths logc Sample means may be the best estmators o the populaton means but sample means wll not always exactly equal the populaton means In act, the lkelhood that a sample mean equals the populaton mean to the nth decmal place s vanshngly small Consequently, we expect δ to be small and close to 0, but t wll very seldom equal 0 exactly So not how do we test the mathematcal model? The answer s that we calculate the probablty densty uncton or the samplng dstrbuton o δ (Remember that statstcans call the probablty densty uncton o a test statstc the samplng dstrbuton o the statstc) The samplng dstrbuton o δ wll provde us wth the probablty o observng δs that are close to 0 and the probabltes o observng δs that are ar away rom 0 Hence, we can compare the observed δ rom the data wth ts samplng dstrbuton and have some reasonable, albet probablstc, way o determnng whether the mathematcal model s plausble Parametrc Statstcs: The Chck and Gary Approach 1 Denton o Parametrc Statstcs Chck and Gary s denton o parametrc statstcs s the same as that gven n 11 above Key Terms Chck and Gary s approach contans all the dentons o the tradtonal approach wth one mnor twst Chck and Gary dstngush two types o mathematcal models whereas above, n 19, we gave a generc denton o a mathematcal model The two types o mathematcal models dened by C&G are termed the compact model and the augmented model ote that the two terms are relatve That s, the same mathematcal model that s the augmented model n one stuaton may be the compact model n another stuaton The techncal dentons are: 1 Compact Model A compact mathematcal model s any model that gves the predcted values or all data observatons n p unknown populaton parameters

10 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 10 Augmented model An augmented mathematcal model s any model that gves the predcted values or all data observatons n the same p unknown populaton parameters as the compact model plus one or more addtonal populaton parameters 3 Steps n C&G s approach to parametrc statstcs The steps are n C&G s approach are the same as those n the tradtonal approach wth a derent twst, agan nvolvng the compact and augmented mathematcal models The steps n C&G s approach are: 31 Estmate the parameters o the compact model Ths ollows all the conventons o the tradtonal approach except that t s appled only to the compact model 3 Estmate the parameters o the augmented model Agan, ths ollows all the conventons o the tradtonal approach but the conventons are appled only to the augmented model 33 Assess the relatve t o the two models The t o a model to data s determned by the extent to whch the predcted values generated by the mathematcal model agree wth the observed values o the data The better the predcted values agree wth the observed values, the better the t o the model C&G stress model comparsons by askng the ollowng queston, How much better does the augmented model t the data than the compact model? I the augmented model ts the data much better than the compact model, then one must conclude that the extra parameters n the augmented model are mportant I the augmented model does not t the data much better than the compact model, then the extra parameters n the augmented model are not very mportant 3 An example o the tradtonal and the C&G approach The orgnal problem n the heght example was to determne whether the mean heghts or male and emale US were dentcal For the present, we wll make no assumptons about the dstrbuton o heght n ether males or emales Instead, we wll assume that the specc populaton o US male students has an unknown mean heght o µ m and an unknown standard devaton o σ m and that the specc populaton o US emale college students has an unknown mean heght o µ and an unknown standard devaton o σ We wll urther assume that the specc populaton that conssts o both male and emale students has an overall mean o µ and a standard devaton o σ We may obtan estmates o all sx quanttes-- µ, µ m, µ, σ, σ m, and σ rom the sample data, gvng respectvely Y, Y, Y, s, s m, and s m In the tradtonal approach, we would construct a precse mathematcal model that relates the two means In secton 13, we saw that a good mathematcal model would be

11 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 11 µ = µ so that µ µ = 0 Thus, we could calculate the test statstc δ = Y Y m m whch should be close to 0 the mathematcal model s correct We would then go to a statstcan and ask hm/her to construct the samplng dstrbuton (e, pd) o δ or us We would then plot out the samplng dstrbuton The possble values o δ, rangng rom negatve nnty to postve nnty would be on the horzontal axs and the requency o δ (e, relatve probablty or relatve lkelhood) would consttute the vertcal axs We would then take the observed value o δ and use the graph to nd the relatve probablty o observng ths δ O course, the value o δ would be large and negatve and the relatve probablty o observng a large, negatve δ would be very small Hence, we would conclude that the mathematcal model µ = µ s very poor whch, translated nto Englsh, means that US college men and US college women do not have the same average heght In C&G s approach, we would develop two mathematcal models, a compact model and an augmented model A good compact model would be Y = µ + e In other words, the predcted heght or any observaton, male or emale, s the overall populaton mean In the augmented model, we would lke to have one addtonal populaton parameter to relect sex derences n heght We could do ths by constructng another varable n the data set to relect sex Let us denote ths varable as X and code t n the ollowng way: the observaton s a male then X = +1; the observaton s a emale, then X = -1 We could then wrte an augmented model as: Y = µ + β X + e ote how the augmented model looks exactly lke the compact model wth the addton o one extra populaton parameter, β Ths parameter quantes sex derences I β s large and postve, then college men are taller than college women I β s large and negatve then college women are taller than college men I β s close to 0, then the average heght o college men equals that o college women The problem now s to see whether the augmented model, Y = µ + β X + e, ts the data better than the compact model, Y = µ + e To do ths, we ollow the steps n parametrc statstcs outlned above n secton 13 Frst, we must obtan a good estmates o the populaton parameters µ n the compact model and µ and β n the augmented model Let us use the least squares crteron or estmaton because that s the crteron used most oten n psychologcal research So we take the problem to a statstcan who tells us the ollowng: In the compact model, the best estmate o µ s Y, the overall sample mean In the augmented model, the best estmate o µ s stll Y, the overall sample mean, and the best estmate o β s b whch equals m m

12 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 1 = 1 X( Y Y ) X = 1 (OTE: In estmatng these quanttes, t s assumed that the number o males n the sample equals the number o emales And or the moment, let us punt on the ssue o how the statstcan arrved at these results) ow that we have obtaned the parameter estmates, we use nerental statstcs to test the hypothess about sex derences n heght There are two ways o dong ths, both o whch result n the same mathematcal result but express the normaton n derent ways The rst s to use b, the estmate o β, as a test statstc Re-examne the equatons or the compact and augmented model: and Y = µ + e Y = µ + β X + e Obvously, there are no sex derences then β = 0 and the augmented model equals the compact model The only other two logcal possbltes or β are β > 0 (men are taller than women) or β < 0 (women are taller than men) However, these two possbltes are untestable because they are mathematcally mprecse they do not provde a specc number or β Consequently, we want to test the hypothess that β = 0 usng the estmate o β, e b, as the test statstc Once agan, we cannot smply look at b and see t s 0 or not because o samplng error Hence, we take the problem to our avorte statstcan and request a plot o the samplng dstrbuton o b Ths would plot all the mathematcal possbltes o b on the horzontal axs and the relatve lkelhood o b on the vertcal axs The observed value o b wll be large and postve (Recall that X = 1 or males but X = -1 or emales) The relatve lkelhood o observng a large postve value o b would be very small accordng to the plot Hence, we would conclude that t s very unlkely that the populaton parameter β s really 0 Men are ndeed taller than women, A second, mathematcally equvalent way o testng whether mean heght n males equals that n emales s to compute another test statstc based on the predcton errors I the model Y = µ + e s true then the quantty β n the augmented model, Y = µ + β X + e, should be 0 and the predcton errors n the augmented model should equal those n the compact model To do ths, we would estmate µ n the compact model Y = µ + e, calculate the predcted value or each observaton (whch would equal µ), and then calculate the resdual (e, error) or each observaton n the sample We could then calculate the sum o squared errors or the compact model Let us denote the sum o squared errors or the compact model as SSE C Then

13 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 13 SSE C = e C1 + e C + + e C = e C (The subscrpt C s used to sgny that the errors have been computed rom the compact model) We would then estmate µ and β rom the augmented model, Y = µ + β X + e and calculate the predcted value or each observaton n the sample Here, the predcted value would equal µ + β the observaton s male but µ - β the observaton s emale We would then calculate the predcton error (e, resdual) or each observaton Fnally, we would calculate the sum o squared predcton errors, say SSE A, or the augmented model: SSE A = e A1 + e A + + e A = e A I the augmented model s really a better model, then SSE A should be much smaller than SSE C I the augmented model s not that much better then SSE A should be roughly equal to SSE C So let us develop a test statstc based on the two sums o squared error, SSE A and SSE C At rst glance, a good test statstc mght be the smple derence between the two or SSE C - SSE A I the augmented model s not better than the compact model then SSE C - SSE A should be close to 0 But the augmented model s superor to the compact model, then SSE C - SSE A should be large and postve There are two problems, however, wth SSE C - SSE A that make ths a poor test statstc (1) t depends on sample sze; the larger the sample sze, the larger the sum o squared error; and () t depends on the measurement metrc; or example, sum o squared error wll be larger or heght measured n centmeters than or heght measured n nches To get around these problems, C&G suggest that the derence SSE C - SSE A be dvded by SSE C Hence, the test statstc s SSEC SSEA SSE Ths s an mportant test statstc that C&G call PRE or the Proportonal Reducton n Error Mathematcally, t can be shown that PRE has a lower lmt o 0 (when SSE A = SSE C ) 1 and an upper lmt o 1 (when SSE A = 0) Hence, there s no mean derence n heght, then the squared error or the augmented model wll equal than or the compact model and PRE should be close to 0 I the augmented model s superor, then SSE C should be greater than SSE A and PRE should be postve Hence, we could take PRE to our avorte statstcan and request the samplng dstrbuton o PRE under the assumpton that the augmented model s really the same as the compact model (Remember, we cannot nd the dstrbuton o PRE under the assumpton that the augmented model s better than the compact model because, once agan, we would have to come up wth a specc number or b n the augmented model) The statstcan would then gve us the samplng dstrbuton o PRE Ths would plot the value o PRE, rangng rom 0 to 1, on the horzontal axs and the relatve lkelhood o C =1 =1 1 For techncal reasons, SSE A must always be less than SSE C Hence, SSE C - SSE A can never be negatve

14 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 14 observng that value on the vertcal axs Because men are taller than women, the actual value o PRE would be large and t would be very unlkely to observe a large value o PRE under the hypothess that men and women have the same heght 4 Four Important Descrptve Statstcs and Ther Formula Both the tradtonal and the C&G approach use our basc descrptve statstcs Almost all other ormula can be expressed as a uncton o one or more o these statstcs The our statstcs are: 41 Arthmetc Mean The arthmetc mean s smply the arthmetc average It s computed by summng the scores on a varable over all observatons and then dvdng by the number o observatons For a specc populaton, the ormula s X µ = = 1 The sample mean s an unbased estmator o the populaton mean Hence, µ ˆ =1 = X = The hat (^) over the µ denotes an estmator o µ Thus, the Englsh meanng o the symbol µ ˆ s an estmate o the populaton mean There are types o means other than the arthmetc mean (the geometrc mean and the harmonc mean) However, these are seldom encountered n tradtonal parametrc statstcs, so the term the mean s always taken as the arthmetc mean The arthmetc mean s a measure o central tendency That s, t answers the queston, Around whch number are the scores clustered? 4 Varance The varance s a measure o varablty It answers the queston, How dspersed are the scores around ther central tendency? There are two types o varance 41 Varance o a Specc Populaton The varance o a specc populaton s the average squared devaton rom the mean One would rst calculate the arthmetc mean or µ or the specc populaton For each observaton n the specc populaton, one would then calculate the devaton rom the mean For the th observaton, the devaton rom the mean s smply X µ One would then square each devaton rom the mean gvng ( ) X µ or the th observaton The varance o the specc populaton s then the arthmetc average o these squared devatons Consequently, the ormula or the varance o a specc populaton s X

15 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 15 σ ( X µ ) = 1 = 4 Sample Varance as an Estmator o the Varance o a Specc or Hypothetcal Populaton Because t s usually mpractcal to enumerate all the observatons n a specc populaton and because t s mpossble to calculate any parameter or a hypothetcal populaton, sample data are used to estmate the populaton varance When sample data are used to estmate the populaton varance the denton o a varance changes slghtly Here, the varance s dened as the sum o the squared devatons rom the mean dvded by the degrees o reedom let n the data We have encountered the sum o squared devatons rom the mean above n dscussng the varance o a specc populaton Here, we just note some mportant termnology The sum o squared devatons rom the mean s usually just called the sum o squares and s abbrevated as SS We have not encountered the term degrees o reedom beore In Englsh, the degrees o reedom or an estmator s the amount o normaton requred over and above the estmator to gure out all the scores n a sample For example, suppose we had 6 scores and calculated the mean Then, gven that we know the mean, how many o the sx scores would we have to know beore we could gure out the remanng scores? The answer s 5 Once we know the mean, then we only need to know any 5 scores to gure out the remanng score Consequently, once the mean s known the data have 5 degrees o reedom let In general, a mean s calculated rom scores, then there wll be - 1 degrees o reedom let n the data Consequently, the ormula or usng sample data to estmate a populaton varance becomes σ ˆ = s = SS ( X X ) d = =1 1 Once agan, the hat (^) over σ denotes an estmator o σ ; translated nto Englsh, the notaton σ ˆ means an estmate o the populaton varance Because every ndvdual squared devaton rom the mean must be postve, the sum o squared devatons rom the mean must also be postve Lkewse, the degrees o reedom must always be postve Consequently, the varance wll always be postve (or 0, all the scores are the same number) I you ever calculate the varance and arrve at a negatve number, then you must have made a computatonal error Closely related to the varance s ts sblng statstc, the standard devaton The standard devaton s smply the square root o the varance Consequently, because σ = σ, the symbol σ s used to denote a populaton standard devaton For analogous reasons, the symbol s s oten used to denote the estmate o a populaton standard

16 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 16 devaton rom sample data Because a varance s always postve, ts standard devatons s always taken as the postve root 43 Covarance The covarance s a statstc that measures the extent to whch two varables vary together I we let X denote one varable and Y denote the other varable, then the ormula or the covarance n a specc populaton s ( X X )( Y Y ) = 1 cov( X, Y) = Unlke the mean (where µ customarly denotes the populaton parameter) and the varance (where σ customarly denotes the populaton parameter), there s no conventonal Greek letter used to denote the covarance When sample data are used to estmate the populaton covarance, the denomnator s replaced by the degrees o reedom or - 1 Thus, the ormula or the estmate o a populaton covarance s (X X )(Y Y ) =1 cˆ o v(x,y ) = 1 There are two attrbutes o a covarance (1) the sgn o the covarance and () the absolute value o the covarance The sgn o a covarance ndcates the drecton o the relatonshp between X and Y A postve covarance means that hgh scores on X are assocated wth hgh scores on Y and, conversely, that low scores on X are assocated wth low scores on Y An example o a postve covarance would be the relatonshp between heght and weght People who are taller than average also tend to wegh more than the average person Smlarly, people who are smaller than average tend to wegh less than the average person A negatve covarance denotes an nverse relatonshp Here hgh scores on X are assocated wth hgh scores on Y whle, conversely, low scores on X are assocated wth low scores on Y An example o a negatve covarance would be the relatonshp between grades and the amount o tme spent partyng Folks who party a lot, and presumably do not have as much tme to study, tend to have low grades whle students who sacrce play or study wll tend to get hgher grades The absolute value o a covarance s a measure o the magntude o the relatonshp A covarance o 0 denotes no statstcal assocaton between X and Y As the covarance departs rom 0 n ether a postve or negatve drecton, the relatonshp between X and Y becomes stronger 44 Correlaton Although most major statstcal procedures operate wth the covarance, the covarance s a very poor descrptve statstc or communcaton among scentsts The

17 Psych 5741 (Carey): 8//97 Parametrc Statstcs - 17 reason s that the tellng someone that the covarance s 0 s akn to tellng hm/her that the prce o an ce cream cone s 0 herns wthout specyng how much a hern s I a hern s close to an Englsh pound, then t s a very expensve ce cream cone (about $35) But a hern s close to an Italan lra, then the ce cream cone s qute the bargn (about 6 cents) Smlarly, a covarance o 0 mght ndcate a very strong relatonshp or one that s hardly worth carng about t all depends on the measurement unts or X and Y To overcome ths dculty, statstcans convert a covarance nto the statstcal equvalent o a common currency, the correlaton coecent The advantage o the correlaton coecent s that t has all the propertes o a covarance (e, the sgn denotes drecton and absolute value denotes strength o assocaton) but t has a mathematcal lower bound o -10 and a mathematcal upper bound o 10 When a correlaton equals - 10 or 10, then one can perectly predct the value o Y rom any X value That s, there are no predcton errors or all observatons As the correlaton coecent gets closer to 0, the statstcal relatonshp between the two varables becomes weaker Lke a covarance o 0, a correlaton o 0 denotes no statstcal assocaton between two varables Consequently, t s possble to compare the magntude o correlatons whle t s not possble to compare the magntude o covarances For example, the correlaton between varables X and Y s 0 whle the correlaton between varables A and B s 35, then t s possble to state that the varables A and B are more strongly assocated than varables X and Y The same could not be made or covarances The Greek letter rho or ρ usually denotes a populaton correlaton The estmate o the populaton correlaton s gven by the Roman equvalent, r, although the uppercase R s requently encountered The ormula or the populaton correlaton s X Y ρ XY = cov(, ) σ σ For sample data, the sample statstcs are replaced nto the above equaton, gvng ρ ˆ XY = r XY = cˆ o v(x,y ) s X s Y One mportant property about correlatons s that the square o the correlaton gves the proporton o varance n one varable attrbutable or predcted by the other varable For example, the correlaton between heght and weght s 60, then 36% o the varance n weght s predctable rom knowng heght Smlarly, 36% o the varance n heght s predctable rom knowng weght It s essental never to thnk o the phrases "varance atrbutable" or "varance predcted by" n causal terms A correlaton coecent only denotes a statstcal relatonshp The statstcal assocaton may or may not be causal, but a correlaton coecent alone s not sucent to deteremne causalty X Y There are actually several derent types o correlaton coecents The one outlned here s ocally called the Pearson product-moment correlaton ater a amous statstcan, Karl Pearson, who developed many o ts prncples around the turn o the century