Part 1: quick summary 5. Part 2: understanding the basics of ANOVA 8

Transcription

1 Statstcs Rudolf N. Cardnal Graduate-level statstcs for psychology and neuroscence NOV n practce, and complex NOV desgns Verson of May 4 Part : quck summary 5. Overvew of ths document 5. Background knowledge 5.3 Quck summary: choosng and performng an NOV 5 Part : understandng the bascs of NOV 8. he basc logc and assumptons of NOV 8.. he underlyng model 8.. n example: data and a structural model 8..3 he null hypothess 9..4 he assumptons of NOV 9..5 he logc of NOV..6 Expected mean squares (EMS). he calculatons behnd a smple one-way NOV (one between-subjects factor).. Calculatons wth means (conceptual) or totals (for manual calculaton only).. Calculatng SS total, SS treatment, and SS error..3 Degrees of freedom 3..4 Mean squares 3..5 he F test 3..6 NOV summary table 3..7 SS treatment for unequal sample szes 4..8 Pctoral representaton 4..9 Relatng SS calculatons to the structural model 5.3 Regresson NOV: the other way to understand the basc logc 6.3. Lnear regresson n terms of sums of squares 6.3. Lnear regresson as an NOV 8.4 Factors versus covarates 9.5 ssumptons of NOV nvolvng covarates 9.6 NOV wth two between-subjects factors.6. Structural model and termnology (man effects, nteractons, smple effects).6. Expected mean squares.6.3 Degrees of freedom.6.4 Sums of squares.6.5 Relatng SS calculatons to the structural model NOV table 3.7 Wthn-subjects (repeated measures) NOV 3.7. Structural model 3.7. Degrees of freedom Sums of squares EMS and NOV summary table 5.8 ssumptons of wthn-subjects NOV: Mauchly, Greenhouse Gesser, etc Short verson 5.8. Long verson 6.9 Mssng data n desgns nvolvng wthn-subjects factors 8. Mxed NOV (wth both between-subjects and wthn-subject factors) 8.. Structural model 8.. Degrees of freedom 9..3 Sums of squares 9..4 NOV table 3. Fxed and random factors 3

2 Part 3: practcal analyss 3 3. Remnder: assumptons of NOV 3 3. Remnder: assumpton of NOV wth wthn-subject factors Consequences of volatng the assumptons of NOV Exploratory data analyss, transformatons, and resduals Plot your data Outlers ransformatons Plot your resduals Further analyss: man effects, nteractons, post hoc tests, smple effects Smple effects Determnng the effects of a factor wth > levels Post-hoc tests: the problem he specal case of three groups: multple t tests are OK Otherwse a varety of post hoc tests Drawng pctures: error bars for dfferent comparsons Error bars for t tests: between-subjects comparsons Error bars for t tests: wthn-subjects comparsons Error bars for an NOV Summarzng your methods: a gude for thess-wrtng and publcaton 46 Part 4: ptfalls and common ssues n expermental desgn me n wthn-subjects (repeated measures) desgns nalyss of pre-test versus post-test data Observng subjects repeatedly to ncrease power It s sgnfcant n ths subject Should I add/remove a factor? Full versus reduced models Should I add/remove/collapse over levels of a factor? ddng and removng levels by addng new observatons Collapsng over or subdvdng levels 5 Part 5: usng SPSS for NOV Runnng NOVs usng SPSS Interpretng the output 56 p: parwse comparsons for nteractons Further analyss: selectng cases he ntercept, total, and corrected total terms 65 Part 6: advanced topcs harder thngs about NOV Rules for calculatng sums of squares Parttonng sums of squares General rule for calculatng sums of squares Rules for calculatng degrees of freedom Nasty bt: unequal group szes and non-orthogonal sums of squares Proportonal cell frequences Dsproportonate cell frequences a problem Expected mean squares (EMS) and error terms Rules for obtanng expected mean squares (EMS) Choosng an error term Poolng error terms 75

3 6.5 Contrasts Lnear contrasts ype I error rates wth planned contrasts Orthogonal contrasts Lnear contrasts n SPSS Contrasts n multfactor desgns an overvew rend analyss: the effects of quanttatve factors rends rend analyss n SPSS How trend analyss relates to multple regresson or polynomal NCOV How computers perform complex NOVs: the general lnear model (GLM) he basc dea of a GLM, llustrated wth multple regresson Usng a GLM for smple NOV: the desgn matrx Example of a GLM for a one-way NOV GLM for two-way NOV and beyond n overvew of GLM desgns hnt at multvarate analyss: MNOV Lnear contrasts wth a GLM GLMs n SPSS Effect sze Effect sze n the language of multple regresson Effect sze n the language of NOV Part 7: specfc desgns 5 7. One between-subjects factor 6 7. wo between-subjects factors hree between-subjects factors 7.4 One wthn-subjects factor 7.5 wo wthn-subjects factors hree wthn-subjects factors One between- and one wthn-subjects factor 7.8 wo between-subjects factors and one wthn-subjects factor One between-subjects factor and two wthn-subjects factors 3 7. Other NOV desgns wth between and/or wthn-subjects factors One between-subjects covarate (lnear regresson) One between-subjects covarate and one between-subjects factor he covarate and factor do not nteract he covarate and factor nteract One between-subjects covarate and two between-subjects factors wo or more between-subjects covarates (multple regresson) wo or more between-subjects covarates and one or more between-subjects factors One wthn-subjects covarate One wthn-subjects covarate and one between-subjects factor he covarate and factor do not nteract he covarate and factor nteract Herarchcal desgns: two or more levels of relatedness n measurement Subjects wthn groups wthn treatments (S/G/) Groups versus ndvduals ddng a further wthn-group, between-subjects varable (S/GB/) ddng a wthn-subjects varable (US/GB/) 6 3

4 Nestng wthn-subjects varables, such as V/US/ he splt-splt plot desgn hree levels of relatedness Latn square desgns Latn squares n expermental desgn he analyss of a basc Latn square B nteractons n a sngle Latn square More subjects than rows: (a) usng several squares More subjects than rows: (b) usng the same square several tmes (replcatng a sngle Latn square) Between-subjects desgns usng Latn squares (fractonal factoral desgns) Several-squares desgn wth a between-subjects factor Replcated-squares desgn wth a between-subjects factor grcultural termnology and desgns, compared to psychology 86 8 Mathematcs, revson and advanced Matrces Matrx notaton Matrx algebra he nverse of a matrx Matrx transposton Calculus 8... Dervatves 8... Smple, non-trgonometrc dervatves Rules for dfferentaton Dervatves of a vector functon Partal dervatves he chan rule for partal dervatves Illustratons of partal dervatves 8.3. Solvng a GLM (an overdetermned system of equatons) (advanced) Sngular value decomposton to solve GLMs (very advanced) Egenvectors and egenvalues Sngular value decomposton n underdetermned set of equatons: the role of expectatons Random varables, means, and varances Summaton Random varables; defnton of mean and varance Contnuous random varables Expected values he sample mean and SD are unbased estmators of µ and σ Varance laws Dstrbuton of a set of means: the standard error of the mean 8.6 he harmonc mean 3 9 Glossary 4 Further readng Bblography 4

5 : Quck summary 5 Part : quck summary I m not a statstcs expert, so caveat emptor. If you spot any mstakes or have suggestons to make ths document more useful, please let me know (at rudolf@pobox.com). hanks to Mke tken (MRF) for helpful comments!. Overvew of ths document Frst, n Part, we ll summarze what most people want to know to get gong how to choose and perform an NOV. Nobody reads NOV theory before startng to analyse, much as statstcans may complan about ths, so we mght as well be pragmatc. hs can be combned wth Part 3, whch talks about common thngs that are requred n NOV analyss, and Part 5, whch shows how to perform an NOV n SPSS. hen, n Part, we ll cover what NOV does and what t assumes thngs people should have known before runnng an NOV but probably ddn t. In Part 3, we ll walk through what most people need to do to complete an NOV analyss. In Part 4, we ll look at expermental desgn and analyss ssues, such as how to analyse changes from baselne, and when and how to perform post hoc tests. In Part 5, we ll look at how to use SPSS to perform dfferent NOVs. In Part 6, we ll cover complex theory that most people wll never need. In Part 7, we ll look at a varety of NOV models that can be used for dfferent expermental desgns. hese wll range from the very smple (one-way NOV) through the very useful (mxed desgns wth both between- and wthn-subject factors) to the very complcated. hs materal s for reference. In Part 8, we ll revse mathematcs that s touched on occasonally elsewhere, and cover very advanced mathematcs that underpns computer calculatons of complex NOVs. In Part 9, there s a glossary.. Background knowledge hs handout s amed at graduate students who need to perform analyss of varance (NOV). Coverng the theory of NOV s one thng; puttng t nto practce n psychology and neuroscence research unfortunately means usng the technque at a level at whch even statstcans debate the proper methods. hs s depressng to the begnner; I hope ths handout helps. It s also a remnder to me of nformaton I ve collected about dfferent NOV desgns. It covers smple NOV and also some complex technques that are not often used but rather powerful. It assumes a basc knowledge of statstcs. Explct coverage of the background knowledge can be found n my NS IB Psychology handouts, avalable at and coverage of exploratory data analyss (ED) and NOV can be found n Mke tken s NS II Psychology handouts, avalable at foxfeld.psychol.cam.ac.uk/stats/default.html.3 Quck summary: choosng and performng an NOV We ll presume your experment was sensbly desgned and free of confounds. No amount of analyss wll fx a bad desgn. Now, the purpose of NOV s to predct a sngle dependent varable on the bass of one or more predctor varables,

6 : Quck summary 6 and to establsh whether those predctors are good predctors or not. herefore you need to do the followng: Identfy your dependent varable. Identfy your predctor varables. Establsh whether your predctor varables are dscrete (e.g. sham/leson, sham/core/shell, day //3) or contnuous (e.g. body mass). We wll call dscrete varables factors, and contnuous varables covarates. he number of dscrete values that a factor can take s known as the number of levels of that factor. For most psychology desgns, the key unt of relatedness s usually the subject. It then suffces to establsh whether your predctor varables are betweensubjects varables (e.g. operatve group; every subject s only measured at one level of the factor, such as leson ) or wthn-subjects varables (e.g. test day; each subject s measured at more than one level of the factor, such as day, day, and so on). You should now be able to dentfy your desgn (e.g. one between-subjects factor and two wthn-subjects factors ) usng ths document. he sectons gvng detal on each desgn also gve the SPSS syntax. You should check that the assumptons of NOV are met for example, do you need to transform your dependent varable (by takng the square root, arcsne, logarthm, etc.) before analyss? Run the NOV. If your NOV has wthn-subjects factors, check Mauchly s test of sphercty, whch your software should have done for you. If the Mauchly test s sgnfcant (small p value), one of the assumptons behnd wthn-subjects NOV has been volated. Don t use the normal df; use the corrected df ether wth the Greenhouse Gesser (conservatve) or Huynh Feldt (better; Myers & Well, 995, p. 48) correcton. Your software should provde both. Interpret the results. You may need to perform further analyses post hoc to explan man effects or nteractons that you fnd. I use a notaton for descrbng NOV models n whch factors are wrtten wth ther number of levels as a subscrpt, covarates are wrtten wth cov as a subscrpt, S denotes subjects, factors/covarates n brackets wth S are wthn-subjects predctors, and unbracketed factors/covarates are between-subjects predctors. n NOV wth one between-subjects factor () and two wthn-subjects factors (U, V) mght be wrtten lke ths: dependent varable (U V S) s a more concrete example of ths notaton, suppose you measured locomotor actvty (dependent varable) n two groups of rats (sham/leson). Each rat was tested on sx occasons: followng one of three drug treatments (salne/low-dose cocane/hgh-dose cocane), and n one of two rooms (hot/cold). We assume the testng order for wthn-subjects factors was approprately counterbalanced to avod order effects (see handouts at We could wrte ths desgn as: locomotor actvty Group (Drug 3 Room S) In ths document, I wll try to use, B, C as labels for between-subjects factors and U, V, W as labels for wthn-subjects factors, snce t gets hard to read otherwse when there are both between- and wthn-subjects factors n a desgn. Desgns wth both between-subjects and wthn-subjects factors are called mxed or nested desgns (Keppel, 99, p. 563): varablty due to subjects s nested wthn varablty due to the between-subjects factor(s), because each subject s only tested at one level of the between-subjects factor(s).

7 : Quck summary 7 If you have unts of relatedness other than subject (e.g. plot of land ), but you only have one level of relatedness, you can merely thnk of your desgn n the same between-/wthn-subject terms. If you have multple levels of relatedness, you wll need a complex or herarchcal desgn (Myers & Well, 995, chapter ); you should am to understand the prncples behnd the desgns dscussed n ths document. t the end we ll cover some herarchcal desgns, but ths s hard stuff.

8 : NOV bascs 8 Part : understandng the bascs of NOV. he basc logc and assumptons of NOV.. he underlyng model fter Howell (997, ch. ). Suppose that the average heght of UK adults s 75 cm, that of adult females s 7 cm, and that of adult males s 8 cm. So maleness contrbutes, on average, +5 cm to an adult s heght (compared to the mean of all adults), and femaleness contrbutes, on average, 5 cm. Suppose we take a gven adult male. We could break hs heght down nto three components: 75 cm for beng an adult, 5 cm for beng a male, and some other component that represents ths ndvdual s unqueness, snce there s of course varaton n the heghts of adult men. We could wrte ths model as or n more general terms heght 75 cm + 5 cm + unqueness heght ndvdual male µ + τ + ε male ndvdual where µ s the overall mean (75 cm), τ male s the contrbuton for beng a male, and ε ndvdual s a partcular ndvdual s unque contrbuton. We have wrtten an expresson for our dependent varable (heght) n terms of predctor varables (the grand mean and a factor, sex) and unpredcted varablty. Let s extend that prncple... n example: data and a structural model Suppose 5 subjects are assgned to fve groups. Each group reads a lst of words n a dfferent way: one was asked to count the number of letters n each word, one to thnk of a rhyme for each word, one to gve an adjectve that could be used wth each word, one to form a vvd mage of each word, and one to memorze each word for later recall. Later, all groups were asked to recall all the words they could remember. In NOV termnology, we have a sngle factor Group wth fve levels (Group, Group, Group 5 ). Here are some results (Howell, 997, p. 3): No. words Group Group Group 3 Group 4 Group 5 otal recalled Countng Rhymng djectve Imagery Memorze One 9 7 number, one subject total x 53 n n n n 3 n 4 n 5 N 5 mean x 7 x 6.9 x 3 x x 5 x.6 SD s.83 s.3 s 3.49 s s varance s 3.33 s 4.54 s 3 6. s 4.7 s 5 4 s 6.6 For ths data, we can specfy a model, just as we dd before. Let j represent the score of person j n condton (group) µ represent the overall mean score µ represent the mean of scores n condton τ represent the degree to whch the mean of condton devates from the overall mean (the contrbuton of condton ),.e. µ µ τ

9 : NOV bascs 9 ε j represent the amount by whch person j n condton devates from the mean of hs or her group (the unqueness of person j n condton ),.e. εj j µ Snce t s obvous that t follows that µ + µ µ ) + ( µ ) j ( j µ + τ + ε j j..3 he null hypothess We wll test the null hypothess that there s no dfference between the varous groups (condtons). We can state that null hypothess lke ths: H : µ µ µ µ µ µ In other words, the null hypothess s that all means are equal to each other and to the grand mean (µ), and that all treatment (group) effects are zero...4 he assumptons of NOV If µ represents the populaton mean of condton and σ represents the populaton varance of ths condton, analyss of varance s based on certan assumptons about these populaton parameters.. Homogenety of varance We assume that each of our populatons has the same varance: σ σ 3 σ 4 σ 5 σ e σ e he termσ (where e stands for error) represents the error varance the varance unrelated to any treatment (condton) dfferences. We would expect homogenety of varance f the effect of any treatment s to add or subtract a constant to everyone s score wthout a treatment the varance would be σ e, and f you add a constant to a varable, the varance of that varable doesn t change.. Normalty We assume that the scores for each condton are normally dstrbuted around the mean for that condton. (Snce ε j represents the varablty of each person s score around the mean of that condton, ths assumpton s the same as sayng that error s normally dstrbuted wthn each condton sometmes referred to as the assumpton of the normal dstrbuton of error.) 3. Independence of error components ( ndependence of observatons) We also assume that the observatons are ndependent techncally, that the error components (e j ) are ndependent. For any two observatons wthn an expermental treatment, we assume that knowng how one of these observatons stands relatve to the treatment (or populaton) mean tells us nothng about the other observaton. Random assgnment of subjects to groups s an mportant way of achevng ths. o deal wth observatons that are not ndependent for example, observatons that are correlated because they come from the same subjects we need to account specfcally for the sources of relatedness to make sure that the resdual error components

10 : NOV bascs are ndependent; ths s why we need wthn-subjects (repeated measures) desgns for ths sort of stuaton. But we ll gnore that for the moment...5 he logc of NOV Snce we have assumed that the dstrbuton of the scores for each condton have the same shape (are normally dstrbuted) and have the same varance (homogenety of varance), they can only dffer n ther means. Now f we measure the varance of any one condton, such as s, that varance wll be an estmate of the common populaton varance σ e (remember, we assumed σ σ σ 3 σ 4 σ 5 σ e, that s, homogenety of varance). In each case, our sample varance estmates a populaton varance: s σ ; σ s ; σ 5 s5 (where denotes s estmated by ). Because of our homogenety of varance assumpton, each of these sample varances s an estmate of σ e : σ e s ; σ e s ; σ e s 5 o mprove our estmate of σ e, we can pool the fve estmates by takng ther mean (f n n n 3 n 4 n 5 n), and thus σ s e s s a where a s the number of treatments n ths case, 5. (If the sample szes were not equal, we would stll average the fve estmates, but we would weght them by the number of degrees of freedom for each sample, so varance estmates from larger samples would get more weght.) hs gves us an estmate of the populaton varance that s referred to as MS error ( mean square error ), sometmes called MS wthn, or MS subjects wthn groups, or MS S /groups ( mean square for subjects wthn groups ). hs s true regardless of whether H s true or false. For the example above, our pooled estmate of σ e wll be σ e Now let us assume that H s true. In ths case, our fve samples of cases may be thought of as fve ndependent samples from the same populaton (or, equvalently, fve samples from fve dentcal populatons). he Central Lmt heorem (see handouts at states that the varance of means drawn from the same populaton s equal to the varance of the populaton dvded by the sample sze. If H s true, therefore, the varance of our fve sample means estmates σ n : and so e / e s x σ n σ e ns x σ e hs s therefore a second estmate of, referred to as MS treatment or MS group. On the other hand, f H s false, ths wll not be a good estmate of σ e. So we have found that MS error estmates σ e whether H s true or false, but MS treatment only estmates σ e f H s true. herefore, f our two estmates of σ e, MS treatment and MS error, are smlar, ths s evdence that H s true; f they are very dfferent, ths s evdence

11 : NOV bascs that H s false. We wll compare the two varance estmates wth an F test, whch s desgned specfcally for comparng two varances (see handouts at F MS treatment /MS error. If our F statstc s very dfferent from, we wll reject the null hypothess that the two varances (MS treatment and MS error ) are the same, and hence reject the null hypothess of the NOV...6 Expected mean squares (EMS) Let s formalze that. We ve defned the treatment effect τ as µ µ, the dfference between the mean of treatment (µ ) and the grand mean (µ). We wll also defne σ τ as the varance of the true populaton s means (µ, µ, µ a ): σ τ ( µ µ ) τ a a echncally, ths s not actually the varance snce we are workng wth parameters, not statstcs, we should have dvded by a rather than a f we wanted the varance. However, we can thnk of t as a varance wthout much problem. We can then defne, wthout proof, the expected value of the mean square terms: E(MS E(MS error treatment ) σ ) σ e e n τ + a σ e + n σ τ where σ e s the varance wthn each populaton and σ τ s the varance of the populaton means (µ j ). So f H s true, σ τ, so E ( MStreatment ) E(MSerror ), but f H s false, E MS ) > E(MS ). ( treatment error. he calculatons behnd a smple one-way NOV (one between-subjects factor) Let s go back to the results n the table we saw earler and conduct an NOV... Calculatons wth means (conceptual) or totals (for manual calculaton only) Most NOV calculatons are based on sums of squares. Remember that a varance s a sum of squared devatons from the mean (a sum of squares ) dvded by the number of degrees of freedom. We work wth sums of squares because they are addtve, whereas mean squares and varances are only addtve f they are based on the same number of degrees of freedom. Purely for convenence, Howell (997) tends to do the calculatons n terms of treatment totals rather than treatment means. In the table above, we have defned as the total for treatment. otals are lnearly related to means ( nx ). If you multple a varable by a constant, you multply the varance of that varable by the square of the constant. So snce nx, we can see that We saw earler that f H s true, σ s x s n s x e ns x σ e s n ; therefore, f H s true, s n

12 : NOV bascs On the other hand, though calculatng sums of squares may be easer n terms of treatment totals, conceptually t s much easer to thnk n terms of means. We ll present both for a whle frst the defnton n terms of means, and then, n brackets, the formula n terms of totals. Ignore what s n brackets unless you re dong the calculatons by hand. Eventually we ll just show the calculatons n terms of means. fter all, you ll be usng a computer for the hard work... Calculatng SS total, SS treatment, and SS error Frst, we calculate SS total ( total sum of squares ) the sum of squares of all the observatons (the summed squared devatons of each observaton from the overall mean), regardless of whch treatment group the observatons came from. SS total ( x x) x ( x) N Now we calculate SS treatment. hs represents the summed squared devatons of the treatment mean from the mean of all treatment means, summed over each data pont. (Or, n terms of totals, the summed squared devatons of each total [ j ] from the mean of the treatment totals [ ], all dvded by the number of observatons per total.) SS treatment n( x x) ( ) n n n n n ( ) n ( ) a ( ) n n na ( x) n N where a s the number of treatments, n s the number of observatons per treatment, and N s the total number of observatons ( na). Now we can calculate SS error. hs represents the sum of the squared devatons of each pont from ts group mean. Snce SS total SS treatment + SS error, the quck way to obtan SS error s by subtracton: SS error ( x x ) SS total SS treatment lternatvely, we could have calculated SS error by workng out an SS for each group separately and addng them up: SS SS group group SS error ( x x ) SS group ( x x ) + SS (9 7) group (7 6.9) + (8 7) + + SS + (9 6.9) group (7 7) + + (7 6.9) Both approaches gve the same answer.

13 : NOV bascs 3..3 Degrees of freedom If there are N observatons n total, df total N. If there are a treatments, df treatment a. We can calculate the degrees of freedom for error lke ths: df error df total df treatment lternatvely, we could calculate df error as the sum of the degrees of freedom wthn each treatment; f there are n observatons n each of a treatments, there are n degrees of freedom wthn each treatment, and so df error a(n ). hs gves the same answer (snce df total df treatment [N ] [a ] [na ] [a ] na a a[n ])...4 Mean squares Mean squares are easy; just dvde each SS by the correspondng number of df...5 he F test From the defntons of EMS above, E(MS E(MS treatment error We can therefore calculate an F statstc ) σ ) MS F MS error e treatment + nσ and t s dstrbuted as F a, a(n ) that s, as F treatment df, error df, under the null hypothess. So we can look up crtcal value of F n tables. If t s sgnfcant (unlkely gven the null hypothess), we reject the null hypothess and say that the treatment dd nfluence our dependent varable. σ e τ very complcated asde: f H s true and σ τ, although E(MS E(MS treatment error e e τ ) σ + nσ and therefore under the null hypothess ) σ E(MStreatment ) MStreatment, and so you d thnk E E(MSerror ) MS, the expected value of F error dferror under the null hypothess s actually E ( F ) (Frank & lthoen, 994, pp. df error 47, 53). I don t fully understand that; I suspected that the dfference was that E(MStreatment ) MS treatment E because E ( Y ) E( ) E( Y ) only f and Y are E(MSerror ) MSerror ndependently dstrbuted. MRF has snce ponted out the real reason: under the null hypothess, MS error s asymmetrcally dstrbuted. For asymmetrc dstrbutons, E( ) E( ), so E( MSerror ) E(MSerror ). It s akn to the reasonng behnd usng a t test rather than a Z test when you estmate the populaton standard devaton σ usng the sample standard devaton s: even though E ( s) E( σ ), ( s) E( σ ) E...6 NOV summary table NOV results are presented n a summary table lke ths:

14 : NOV bascs 4 Source d.f. SS MS F reatment a SS treatment SS treatment /df treatment MS treatment /MS error Error (S/treatments) a(n ) SS error SS error /df error otal N an SS total SS total /df total [ s ] Remember that S/treatments denotes subjects wthn treatments ; ths s the source of all our error varablty n ths example. nyway, for our example, we can now calculate all the SS: SS SS SS ) 69 (7 ) ( ) ( SS ) 8 (9 ) ( ) ( SS treatment total error treatment total N x n x x n N x x x x so our NOV table looks lke ths: Source d.f. SS MS F reatment Error (S/treatments) otal Our F has (4, 45) degrees of freedom. We could wrte F 4,45 9.9, and look ths up to fnd an assocated p value (p.)...7 SS treatment for unequal sample szes What f our group szes were not equal? Prevously we had defned N x n na n n a n n nx nx n x x n x x n treatment ) ( ) ( ) ( ) ( ) ( ) ( ) ( SS whch apples when all groups have n observatons. If the group szes are unequal, we smply multply the devaton of each score from ts treatment mean by the number of scores n that treatment group (so the larger one sample s, the more t contrbutes to SS treatment ): N x n n n x n x n n x x n x x n treatment ) ( ) ( ) ( ) ( ) ( SS..8 Pctoral representaton What the NOV technque has done s to partton total varaton from the overall mean (SS total ) nto varaton from the overall mean accounted for or predcted by the treatment or group dfference (SS treatment or SS groups ) and further varaton wthn the

15 : NOV bascs 5 groups due to nter-subject varablty (SS error or SS S/groups ). If the varaton attrbutable to the model s large, compared to the error varablty, we wll reject the null hypothess. SS total ( x x) df total N he sum of squares s the sum of the squared lengths of the vertcal lnes (devatons from the mean). SStreatment n ( x x) dftreatment a Do you see now why we ve been multplyng the devatons by the group sze to fnd SS treatment? SSerror ( x x ) df N a error nother way to look at NOV s ths: the hypothess test we have performed effectvely compares two models (Myers & Well, 995, p. 44-): one (restrcted) model allows for the effects of a mean only all other varablty s error ; the other (full) model allows for the effects of a mean and a treatment (and everythng else s error). If the full model accounts for sgnfcantly more varablty than the restrcted model, we reject the null hypothess that the treatment has no effect...9 Relatng SS calculatons to the structural model Note that our structural model was ths: j j µ + τ + ε τ µ µ ε j µ j and our SS were these: SS SS total treatment SS error ( x x) n( x x) ( x x ) SS total SS treatment See the smlarty? We can prove that the one follows from the other. hs s not somethng we have to do routnely, but t demonstrates how the sums of squares (SS) are derved drectly from the model. Our model was ths: or j µ + τ + ε µ + µ µ ) + ( µ ) j ( j j Rearrangng to express the left-hand sde as a devaton of each score from the overall mean: µ µ µ ) + ( µ ) j ( j

16 : NOV bascs 6 Squarng each sde: Summng over and j: a ( µ ) ( µ µ ) + ( µ ) + ( µ µ )( µ ) j a a j ( µ ) n ( µ µ ) + ( µ ) + ( µ µ )( µ ) n j j he far-rght term s actually zero: a ( µ µ )( n j j n j a j µ ) ( µ µ ) ( µ ) a ( µ µ ) snce the sum of devatons of all observatons about ther mean s zero. So we re left wth: n j a n j j j j a SS ( µ ) n ( µ µ ) + ( µ ) n j j total a SS + SS he degrees of freedom are smlarly related: error a n j j df total df + df error.3 Regresson NOV: the other way to understand the basc logc.3. Lnear regresson n terms of sums of squares Suppose that n some way we can measure the rsk of a heart attack (call t Y) n many 5-year-old men. If we then want to predct the rsk of a heart attack n an unknown 5-year-old man, our best guess would be the mean rsk of a heart attack ( y ). If we call our predcted varable Ŷ, and a predcted ndvdual value ŷ, then our best guess could be wrtten We could also wrte t lke ths: y ˆ y y y + ε where ε represents error varablty or natural varaton. he error n our best guess would be the same as the natural varablty n Y t would be descrbed by some way by the standard devaton of Y, s Y, or the varance, s Y. he sample varance (whch estmates the populaton varance), remember, s s Y ( y y) n hs varance, lke any varance, s the sum of squared devatons about the mean dvded by the number of degrees of freedom that the varance estmate s based on. Because they are convenently addtve, we could wrte the varablty n our estmate just n terms of the sum of squared devatons about the mean the sum of squares:

17 : NOV bascs 7 SSY SStotal ( y y) hs s the total varablty n cholesterol, so t s sometmes wrtten SS total. Now suppose we also measure cholesterol levels () for each of our subjects. We now have (x, y) pars (cholesterol and heart attack rsk) for each subject. We could predct Y from usng lnear regresson. We would call the predcted varable Ŷ, and we d call an ndvdual predcted value ŷ. standard lnear regresson (see handouts at wll gve us ths equaton: Y ˆ a + b where a s the ntercept and b s the slope. We could also wrte our model lke ths: y yˆ + ε a + bx + ε Now our best guess of the heart attack rsk of a new subject should be rather better than y ˆ y ; f we measure our new subject s cholesterol as well, we can make what should be a better predcton: y ˆ a + bx he error n ths predcton wll related to the devatons between the predcted value, ŷ, and the actual value, y. We could wrte ths ether n terms of a varance or as a sum of squares: resdual s ( y yˆ) n SSresdual ( y yˆ ) If cholesterol s somehow lnearly related to heart attack rsk, the error n our predcton, whch was SS total, has now been reduced to SS error. herefore, the amount of varablty n Y that we have accounted for by predctng t from, whch we can wrte as SS regresson or SS model or SSY ˆ, s based on the dfference between the predcted values and the overall mean: It s also true that SSmodel ( yˆ y) and that ( y y) ( yˆ y) + ( y yˆ) SS SS + SS df total total df model model n + ( n ) + df resdual resdual Snce we have already calculated the overall mean, and the regresson lne always passes through the overall mean, the regresson model has one df (ts slope). hat s, people vary n ther cholesterol levels (SS ), they vary n ther heart attack rsk (SS Y SS total ), a certan amount of the varablty n ther heart attack rsk s predctable from ther cholesterol ( SS ˆ SS Y model ), and a certan amount of varablty s left over after you ve made that predcton (SS resdual SS error ). Incdentally, the propor-

18 : NOV bascs 8 ton of the total varablty n Y that s accounted for by predctng t from s also equal to r : r SS Yˆ SS Y SS SS model total We can llustrate SS total, SS model, and SS resdual lke ths: SS df total total ( y y) N SS df model model ( yˆ y) SS df error error ( y yˆ) N What would t mean to alter SS model and SS resdual? If you pulled all of the scores further away from the regresson lne (f a pont s above the regresson lne, move t up; f t s below, move t down) wthout changng the slope of the regresson lne, you d ncrease SS error wthout alterng SS model. If you altered the slope of the regresson lne but moved the ndvdual scores up or down to keep them the same dstance from the lne, you d ncrease SS model wthout changng SS resdual..3. Lnear regresson as an NOV We can use ths way of wrtng a lnear regresson model to express lnear regresson as an NOV. If there s no correlaton between and Y, then predctng Y from won t be any better than usng y as our estmate of a value of y. So we could obtan a measure of the total varablty n Y: MS and we could smlarly obtan ( y y) Y MStotal s Y n SS df total total MS model ( yˆ y) SS s Y ˆ df model model MS resdual MS error ( y yˆ) SS s resdual n df resdual resdual If the null hypothess s true and there s no correlaton between and Y, then some of the varaton n Y wll, by chance, ft a lnear model, and contrbute to SS model. he rest wll not, and wll contrbute to SS resdual. he correspondng MS values, once we have dvded by the df, wll be measurng the same thng the varablty of Y. hat s, under the null hypothess, E(MS model ) E(MS error ). On the other hand, f there s a correlaton, and Y vares consstently wth, then SS model wll contan varaton due to ths effect as well as varaton due to other thngs (error), but SS resdual wll only contan varaton due to other thngs (error). herefore, f the null hypothess s false, E(MS model ) > E(MS error ). We can therefore compare MS model to MS error wth an F test;

19 : NOV bascs 9 f they are sgnfcantly dfferent, we reject the null hypothess. Our NOV table would look lke ths: Source d.f. SS MS F Model SS model SS model /df model MS model /MS error Error (resdual) N SS error SS error /df error otal N SS total where N s the total number of (x, y) observatons. o calculate a regresson NOV by hand, SS total can be calculated as s Y ( n ) and SS model can be calculated as r SS. total.4 Factors versus covarates We ve seen that we can perform an NOV to predct our dependent varable usng a dscrete varable, or factor, that has several levels as when we asked whether word recall dffered between fve groups that had read the same word lst n dfferent ways. We saw a pctoral representaton of a three-group example. We ve also seen that we can perform an NOV to predct our dependent varable usng a contnuous varable, or covarate, as n our lnear regresson example, and we ve seen a pctoral representaton of that. he mathematcal technque of NOV does not care whether our predctor varables are dscrete (factors) or contnuous (covarates). We ll see that n Part 6 when we look at the dea of a general lnear model (p. 84). However, the way most people use covarates s slghtly dfferent from the way they use factors. If you are runnng an experment, you do not generally assgn subjects to dfferent values of a contnuous varable (covarate) you assgn subjects to dfferent levels of a factor, wth several subjects per level (group). herefore, real-lfe covarates are generally thngs that you measure rather than thngs that you manpulate. s a consequence, most people use covarates and analyss of covarance (NCOV) as a way to ncrease the power of NOV f you can account for some of your error varablty by usng a covarate to predct your dependent varable, there s less error varablty and therefore there may be more power to detect effects of the factors that you re nterested n..5 ssumptons of NOV nvolvng covarates ake a common desgn nvolvng covarates: a desgn wth one between-subjects factor and one between-subjects covarate. Suppose you have chldren at your dsposal. You measure ther IQ. hen you randomly assgn 5 chldren to receve the standard method of maths teachng, and 5 chldren to receve a new method. hs represents the between-subject factor Method, wth two levels. fter some tme, you measure ther mathematcal problem-solvng ablty. But you suspect that ther IQ may also play a part n determnng ther fnal score, not just the teachng method IQ may be contrbutng to the error (unmeasured) varablty n the scores of your two groups. So you enter IQ as a covarate nto your NOV model. hs covarate may therefore account for some of the prevously-unmeasured varablty, reducng your error term, and ncreasng the power to detect an effect of teachng method. If you use NCOV n ths way, there are a few assumptons (Myers & Well, 995, pp ; Howell, 997, p. 587): that the relatonshp between the covarate and the dependent varable s lnear; that the regresson slopes relatng the covarate to the dependent varable are the same n both groups homogenety of regresson.

20 : NOV bascs hs s the desgn dscussed n 7.. (p. 38). he second assumpton s drectly testable, and the method for testng t s dscussed n 7.. (p. 44). fnal assumpton n ths sort of desgn s ths: that the covarate and the treatment are ndependent (Myers & Well, 995, p. 45). If ths s not the case, nterpretaton s very dffcult. Usng as a covarate removes the component of Y predctable from. If the treatment nfluences or s otherwse predctable from, performng an NCOV wll not smply remove nusance varablty from Y; t wll remove part of the effect of the treatment tself. For example, f you had measured IQ at the end of the experment and the teachng method actually nfluenced IQ, nterpretaton would be very hard; smlarly, t would be hard to nterpret f you had assgned hgh-iq students to one teachng method and low-iq students to another. hs can also be a problem n stuatons when you are usng (for example) patent groups and IQ (f the patents have a dfferent IQ to the controls), or sex and body mass (males have a hgher body mass)..6 NOV wth two between-subjects factors We can extend our basc one-way NOV to two factors. Suppose we have two factors, one wth two levels and one wth fve levels; ths desgn would be called a 5 factoral. Suppose we repeat our prevous experment (Howell, 997, p. 43) but for young and old subjects. Factor s age (young versus old); factor B s task type (countng, rhymng, adjectve, magery, ntentonal). Suppose our results look lke ths: No. words B B B 3 B 4 B 5 otal recalled Countng Rhymng djectve Imagery Memorze 9 7 old total B 7 B 69 B3 B4 34 B young hs dotted lne encloses one cell. 5 7 hs s a very mportant term to B 65 B 76 B3 48 B4 76 B understand! B 35 B 45 B3 58 B4 3 B5 33 Σx 6 Note our defnton of cell one partcular (, B) condton, such as B (shown here wth a dotted lne around t)..6. Structural model and termnology (man effects, nteractons, smple effects) Our NOV must allow for the effects of factor, and factor B. It should also allow the possblty that and B nteract that the effect of factor depends on the level of factor B, or vce versa. For example, suppose that young people are generally better, regardless of task type; we would call ths a man effect of factor (age). man effect s an effect of a factor regardless of (gnorng) the other factor(s). Suppose that the memorze condton gves better recall than the countng condton, regardless of age; we would call ths a man effect of factor B (task type).

21 : NOV bascs On the other hand, perhaps young people have a partcular advantage n the memorze condton but not n other condtons; ths would be an nteracton between and B, wrtten B or sometmes B. We may also defne, for later, the term smple effect: ths s an effect of one factor at only one level of another factor. For example, f the memorze condton gves better performance than the adjectve condton consderng young people only, ths s a smple effect of factor B (task type) at the young level of factor (age). We can specfy a model, just as we dd before: µ + α + β + αβ + ε jk j j jk where jk the score of person k n condton B j µ the overall mean score α the degree to whch the mean of condton devates from the overall mean ( the contrbuton of condton ),.e. α µ µ. By ths defnton, α. β j the degree to whch the mean of condton devates from the overall mean ( the contrbuton of condton B j ),.e. β µ µ. By ths defnton, β j. αβ j the degree to whch the mean of condton B j devates from what you d expect based on the overall mean and the separate contrbutons of and B j ( the nteracton B),.e. αβ µ µ + α + β ). By ths defnton, αβ αβ. j j j j B j j B j ( j ε jk the error or amount by whch person k n condton B j devates from the mean of hs or her group (the unqueness of person k n condton B j ),.e. ε jk jk ( µ j + α + β j + αβj ). By our usual assumpton of normal dstrbuton of error, ε jk s normally dstrbuted wth mean and varance σ e..6. Expected mean squares lthough we won t derve t, the EMS terms are: Source E(MS) σ e + nbσ B σ e + naσ B B ( B) σ e + nσ B Error e σ (Note that these EMS values assume that the factors are fxed factors; see p. 3.) So we should be able to form F ratos based on the error term. For example, f the null hypothess that factor has no effect s true, µ µ, so σ and E(MS ) E(MS error ). If ths null hypothess s false, E(MS ) > E(MS error ). So the rato E(MS E(MS error e ) σ + nbσ ) σ can be tested usng an F test wth df and df error degrees of freedom..6.3 Degrees of freedom here are n subjects per (, B) condton (per cell), so N observatons n all. herefore, df total 99. By our usual rules, df and df B 4 (one less than the e

22 : NOV bascs number of levels). he nteracton term, wrtten B or B, represents the possblty that the effects of factors and B represent each other. he df for an nteracton term B s always the product of df and df B n our example, 4. So our total df are parttoned lke ths: df total df + df B + df df so we have 9 error df n our example..6.4 Sums of squares Smlarly, total B B error B + df error SS SS + SS + SS + SS SS total s calculated exactly as before: the sum of squared devatons of every observaton from the grand mean. ( x) SStotal ( x x) x n he SS for factor s calculated exactly as we would f ths were a one-way NOV wthout the other factor. he same s true for SS B. hat s, we take the sum of the squares of the devatons of the means of each condton (, ) from the overall mean, summed over every observaton. (In terms of totals, t s the sum of the squares of the devatons of the totals of each condton,, from the overall mean total, dvded by the number of observatons on whch each mean was based.) In our example, snce there are condtons and each s made up of n observatons per cell and 5 cells ( b levels of B) per condton, there are nb observatons contrbutng to each condton mean. So: SS SS B nb( x x) na( x x) B ( ) nb ( B ) na nb na error B ( x) N ( x) N o fnd the nteracton term SS B, we calculate an ntermedate value, SS cells, whch measures the varablty of the cell means. Snce cell varablty can be due to, B, or B, we can see that SS cells SS + SS B + SS B, and therefore calculate SS B ths way. SS cells s the sum of the squares of the devatons of ndvdual cell means from the grand mean, summed over each observaton. (In terms of totals, t s the sum of the squares of the devatons of ndvdual cell totals from the grand mean total, dvded by the number of observatons that contrbuted to each cell mean.e. the number of observatons per cell.) Whew. SS SS cells B ( n( xb x) SS (SS + SS ) cells B ) n B B nb ( x) N o fnd the error term, we know that SS SS + SS + SS + SS SS + SS total B B so we can fnd SS error by subtracton. lternatvely, we could calculate SS error as the grand sum of the sums of the squares of the devatons of ndvdual observatons from ther cell means. error cells error

23 : NOV bascs Relatng SS calculatons to the structural model Note that our structural model was ths: and our SS were these: See the smlarty?.6.6 NOV table SS SS ε jk j αβ µ SS We ve ended up wth ths: j jk µ + α + β + αβ + ε α µ µ β µ total SS SS B B error B B j jk j µ ( x x) j ( µ + α + β ) ( µ + α + β + αβ ) nb( x x) na( x x) j j j jk n( xb x) (SS + SSB) SS (SS + SS + SS ) total B B Source of varaton d.f. SS MS F Between cells df +df B +df B SS cells a SS SS /df MS /MS error B b SS B SS B /df B MS B /MS error B ( B) (a )(b ) SS B SS B /df B MS B /MS error Wthn cells ( error S/cells) ab(n ) SS error SS error /df error otal N abn SS total SS total /df total [ s ] j B.7 Wthn-subjects (repeated measures) NOV Prncple: f a set of measurements are more correlated than we would expect by chance, we must account for ths correlaton. We can say that these measurements come from the same subject (n psychologcal termnology), or that ths measure was repeated. Suppose we have one wthn-subjects factor. Call t U. Let s suppose we ve measured all our subjects n three condtons (U hot, U warm, U 3 cold), once each, and have counterbalanced approprately to avod nasty order effects. ll we have to do s to partton the sum of squares so as to account for the fact that we ve measured subjects several tmes each.7. Structural model Our structural model s ether one of these two: j j µ + π + α + ε (Model : addtve ) j j j j µ + π + α + πα + ε (Model : nonaddtve ) j where j s the dependent varable for subject n condton U j µ s the overall mean

24 : NOV bascs 4 π s the contrbuton from a partcular person or subject (subject, or S ): π µ S µ. α j s the contrbuton from a partcular level (level j) of the factor U: α j µ U j µ. πα j s the contrbuton from the nteracton of subject wth treatment j: π µ S U ( µ + π + α j ). hs nteracton would reflect that the subjects responded dfferently to the dfferent levels of U. j ε j s everythng else (the expermental error assocated wth subject n condton j). In Model, ths wll be ε j j ( µ + π + α j ). In Model, ths wll be ε j j ( µ + π + α j + παj ). hese two models dffer n the presence or absence of πα j, the nteracton of U wth a partcular person (Howell, 997, pp ). Includng t makes for a realstc model t s lkely that subjects do not all respond equally to all condtons (levels of U). However, f we measure each person n each condton once, we wll not be able to measure dfferences n the way subjects respond to dfferent condtons ndependently of other sources of error such as measurement error. (o do that, we d need to measure subjects more than once per condton, and then we d need a dfferent model agan!) hs s another way of sayng that the S U nteracton s confounded wth s! the error term..7. Degrees of freedom We partton the df lke ths: df df total wthn subjects df df between subjects U + df error S U + df wthn subjects herefore df df df total total between subjects df df U error df N s u df between subjects total df + df U between subjects + df where s s the number of subjects, u s the number of levels of U, and N s the total number of observatons ( su). We could also wrte df between subjects as df S, whch you sometmes see..7.3 Sums of squares Smlarly, we can partton the SS lke ths: SS SS total wthn subjects SS total SS SS SS between subjects U + SS error S U between subjects + SS + SS U error wthn subjects + SS error We can defne our SS as usual SS SS total SS U between subjects ( x x) s( xu x) u( x x) S

25 : NOV bascs 5 where s s the number of subjects and u s the number of levels of U. x U represents the mean for a partcular level of U (across subjects), and x S represents the mean for a partcular subject (across levels of U). Our total number of observatons wll be N su..7.4 EMS and NOV summary table he EMS depend on whch model we use: Source of varaton Model : E(MS) Model : E(MS) Between subjects (S) σ + σ + e uσ S U σ e + sσ U Error e σ e uσ S e + σus e σ US σ + sσ σ + hs means that n Model t s rather hard to do a proper F test for the between subjects factor, snce there s no term whose E(MS) s dentcal to E(MS between subjects ) except for the presence of σ S, the relevant varance for the between-subjects factor. On the other hand, who cares. If ths term were sgnfcant, all t would tell us s that subjects are dfferent, whch s hardly earth-shatterng. Ether way, we have no problem testng U: the proper way to test for an effect of U s to do an F test comparng MS U to MS error. If Model s true f subjects respond equally to the treatments; f the effects are addtve we wll have more power to detect effects of U, snce f the null hypothess (that U has no effect) s false, U E(MS E(MS Umodel errormodel ) σ e + sσ ) σ e U σ > e + σ σ US e + σ + sσ US U E(MS E(MS Umodel ) ) errormodel and the bgger the rato of MS U to MS error, the bgger the F rato, and the more lkely the effect s to be sgnfcant (Myers & Well, 995, p. 44; Howell, 997, pp ). You may be thnkng the calculatons for the two models are exactly the same n practce, so why all ths fuss? You d be rght unless you wanted to estmate the proporton of varance accounted for by a partcular term (Myers & Well, 995, pp. 4, 5-55). See p...8 ssumptons of wthn-subjects NOV: Mauchly, Greenhouse Gesser, etc..8. Short verson. ny NOV nvolvng wthn-subjects factors has a potental problem. here s an assumpton known as sphercty [of the covarance matrx]. If ths assumpton s volated, ype I error rates wll be nflated (f the null hypothess s true, you wll get too many results that you wll declare sgnfcant than you should). Mauchly s test of sphercty checks for ths. sgnfcant Mauchly s test means that the assumpton s lkely to have been volated. But t s not a very good test (see below), so we should probably gnore t.. Correct the df for any term nvolvng a wthn-subjects factor, and the correspondng error df, by multplyng them both by a correcton factor. he correcton factor s known as epslon (ε). If the sphercty assumpton s not volated, ε (so applyng the correcton changes nothng). You do not need to correct any terms that have only between-subjects factors. nd you can never