Part 1: quick summary 5. Part 2: understanding the basics of ANOVA 8

Transcription

1 Statstcs Rudolf N. Cardnal Graduate-level statstcs for psychology and neuroscence NOV n practce, and complex NOV desgns Verson of May 4 Part : quck summary 5. Overvew of ths document 5. Background knowledge 5.3 Quck summary: choosng and performng an NOV 5 Part : understandng the bascs of NOV 8. he basc logc and assumptons of NOV 8.. he underlyng model 8.. n example: data and a structural model 8..3 he null hypothess 9..4 he assumptons of NOV 9..5 he logc of NOV..6 Expected mean squares (EMS). he calculatons behnd a smple one-way NOV (one between-subjects factor).. Calculatons wth means (conceptual) or totals (for manual calculaton only).. Calculatng SS total, SS treatment, and SS error..3 Degrees of freedom 3..4 Mean squares 3..5 he F test 3..6 NOV summary table 3..7 SS treatment for unequal sample szes 4..8 Pctoral representaton 4..9 Relatng SS calculatons to the structural model 5.3 Regresson NOV: the other way to understand the basc logc 6.3. Lnear regresson n terms of sums of squares 6.3. Lnear regresson as an NOV 8.4 Factors versus covarates 9.5 ssumptons of NOV nvolvng covarates 9.6 NOV wth two between-subjects factors.6. Structural model and termnology (man effects, nteractons, smple effects).6. Expected mean squares.6.3 Degrees of freedom.6.4 Sums of squares.6.5 Relatng SS calculatons to the structural model NOV table 3.7 Wthn-subjects (repeated measures) NOV 3.7. Structural model 3.7. Degrees of freedom Sums of squares EMS and NOV summary table 5.8 ssumptons of wthn-subjects NOV: Mauchly, Greenhouse Gesser, etc Short verson 5.8. Long verson 6.9 Mssng data n desgns nvolvng wthn-subjects factors 8. Mxed NOV (wth both between-subjects and wthn-subject factors) 8.. Structural model 8.. Degrees of freedom 9..3 Sums of squares 9..4 NOV table 3. Fxed and random factors 3

2 Part 3: practcal analyss 3 3. Remnder: assumptons of NOV 3 3. Remnder: assumpton of NOV wth wthn-subject factors Consequences of volatng the assumptons of NOV Exploratory data analyss, transformatons, and resduals Plot your data Outlers ransformatons Plot your resduals Further analyss: man effects, nteractons, post hoc tests, smple effects Smple effects Determnng the effects of a factor wth > levels Post-hoc tests: the problem he specal case of three groups: multple t tests are OK Otherwse a varety of post hoc tests Drawng pctures: error bars for dfferent comparsons Error bars for t tests: between-subjects comparsons Error bars for t tests: wthn-subjects comparsons Error bars for an NOV Summarzng your methods: a gude for thess-wrtng and publcaton 46 Part 4: ptfalls and common ssues n expermental desgn me n wthn-subjects (repeated measures) desgns nalyss of pre-test versus post-test data Observng subjects repeatedly to ncrease power It s sgnfcant n ths subject Should I add/remove a factor? Full versus reduced models Should I add/remove/collapse over levels of a factor? ddng and removng levels by addng new observatons Collapsng over or subdvdng levels 5 Part 5: usng SPSS for NOV Runnng NOVs usng SPSS Interpretng the output 56 p: parwse comparsons for nteractons Further analyss: selectng cases he ntercept, total, and corrected total terms 65 Part 6: advanced topcs harder thngs about NOV Rules for calculatng sums of squares Parttonng sums of squares General rule for calculatng sums of squares Rules for calculatng degrees of freedom Nasty bt: unequal group szes and non-orthogonal sums of squares Proportonal cell frequences Dsproportonate cell frequences a problem Expected mean squares (EMS) and error terms Rules for obtanng expected mean squares (EMS) Choosng an error term Poolng error terms 75

3 6.5 Contrasts Lnear contrasts ype I error rates wth planned contrasts Orthogonal contrasts Lnear contrasts n SPSS Contrasts n multfactor desgns an overvew rend analyss: the effects of quanttatve factors rends rend analyss n SPSS How trend analyss relates to multple regresson or polynomal NCOV How computers perform complex NOVs: the general lnear model (GLM) he basc dea of a GLM, llustrated wth multple regresson Usng a GLM for smple NOV: the desgn matrx Example of a GLM for a one-way NOV GLM for two-way NOV and beyond n overvew of GLM desgns hnt at multvarate analyss: MNOV Lnear contrasts wth a GLM GLMs n SPSS Effect sze Effect sze n the language of multple regresson Effect sze n the language of NOV Part 7: specfc desgns 5 7. One between-subjects factor 6 7. wo between-subjects factors hree between-subjects factors 7.4 One wthn-subjects factor 7.5 wo wthn-subjects factors hree wthn-subjects factors One between- and one wthn-subjects factor 7.8 wo between-subjects factors and one wthn-subjects factor One between-subjects factor and two wthn-subjects factors 3 7. Other NOV desgns wth between and/or wthn-subjects factors One between-subjects covarate (lnear regresson) One between-subjects covarate and one between-subjects factor he covarate and factor do not nteract he covarate and factor nteract One between-subjects covarate and two between-subjects factors wo or more between-subjects covarates (multple regresson) wo or more between-subjects covarates and one or more between-subjects factors One wthn-subjects covarate One wthn-subjects covarate and one between-subjects factor he covarate and factor do not nteract he covarate and factor nteract Herarchcal desgns: two or more levels of relatedness n measurement Subjects wthn groups wthn treatments (S/G/) Groups versus ndvduals ddng a further wthn-group, between-subjects varable (S/GB/) ddng a wthn-subjects varable (US/GB/) 6 3

4 Nestng wthn-subjects varables, such as V/US/ he splt-splt plot desgn hree levels of relatedness Latn square desgns Latn squares n expermental desgn he analyss of a basc Latn square B nteractons n a sngle Latn square More subjects than rows: (a) usng several squares More subjects than rows: (b) usng the same square several tmes (replcatng a sngle Latn square) Between-subjects desgns usng Latn squares (fractonal factoral desgns) Several-squares desgn wth a between-subjects factor Replcated-squares desgn wth a between-subjects factor grcultural termnology and desgns, compared to psychology 86 8 Mathematcs, revson and advanced Matrces Matrx notaton Matrx algebra he nverse of a matrx Matrx transposton Calculus 8... Dervatves 8... Smple, non-trgonometrc dervatves Rules for dfferentaton Dervatves of a vector functon Partal dervatves he chan rule for partal dervatves Illustratons of partal dervatves 8.3. Solvng a GLM (an overdetermned system of equatons) (advanced) Sngular value decomposton to solve GLMs (very advanced) Egenvectors and egenvalues Sngular value decomposton n underdetermned set of equatons: the role of expectatons Random varables, means, and varances Summaton Random varables; defnton of mean and varance Contnuous random varables Expected values he sample mean and SD are unbased estmators of µ and σ Varance laws Dstrbuton of a set of means: the standard error of the mean 8.6 he harmonc mean 3 9 Glossary 4 Further readng Bblography 4

5 : Quck summary 5 Part : quck summary I m not a statstcs expert, so caveat emptor. If you spot any mstakes or have suggestons to make ths document more useful, please let me know (at [email protected]). hanks to Mke tken (MRF) for helpful comments!. Overvew of ths document Frst, n Part, we ll summarze what most people want to know to get gong how to choose and perform an NOV. Nobody reads NOV theory before startng to analyse, much as statstcans may complan about ths, so we mght as well be pragmatc. hs can be combned wth Part 3, whch talks about common thngs that are requred n NOV analyss, and Part 5, whch shows how to perform an NOV n SPSS. hen, n Part, we ll cover what NOV does and what t assumes thngs people should have known before runnng an NOV but probably ddn t. In Part 3, we ll walk through what most people need to do to complete an NOV analyss. In Part 4, we ll look at expermental desgn and analyss ssues, such as how to analyse changes from baselne, and when and how to perform post hoc tests. In Part 5, we ll look at how to use SPSS to perform dfferent NOVs. In Part 6, we ll cover complex theory that most people wll never need. In Part 7, we ll look at a varety of NOV models that can be used for dfferent expermental desgns. hese wll range from the very smple (one-way NOV) through the very useful (mxed desgns wth both between- and wthn-subject factors) to the very complcated. hs materal s for reference. In Part 8, we ll revse mathematcs that s touched on occasonally elsewhere, and cover very advanced mathematcs that underpns computer calculatons of complex NOVs. In Part 9, there s a glossary.. Background knowledge hs handout s amed at graduate students who need to perform analyss of varance (NOV). Coverng the theory of NOV s one thng; puttng t nto practce n psychology and neuroscence research unfortunately means usng the technque at a level at whch even statstcans debate the proper methods. hs s depressng to the begnner; I hope ths handout helps. It s also a remnder to me of nformaton I ve collected about dfferent NOV desgns. It covers smple NOV and also some complex technques that are not often used but rather powerful. It assumes a basc knowledge of statstcs. Explct coverage of the background knowledge can be found n my NS IB Psychology handouts, avalable at and coverage of exploratory data analyss (ED) and NOV can be found n Mke tken s NS II Psychology handouts, avalable at foxfeld.psychol.cam.ac.uk/stats/default.html.3 Quck summary: choosng and performng an NOV We ll presume your experment was sensbly desgned and free of confounds. No amount of analyss wll fx a bad desgn. Now, the purpose of NOV s to predct a sngle dependent varable on the bass of one or more predctor varables,

6 : Quck summary 6 and to establsh whether those predctors are good predctors or not. herefore you need to do the followng: Identfy your dependent varable. Identfy your predctor varables. Establsh whether your predctor varables are dscrete (e.g. sham/leson, sham/core/shell, day //3) or contnuous (e.g. body mass). We wll call dscrete varables factors, and contnuous varables covarates. he number of dscrete values that a factor can take s known as the number of levels of that factor. For most psychology desgns, the key unt of relatedness s usually the subject. It then suffces to establsh whether your predctor varables are betweensubjects varables (e.g. operatve group; every subject s only measured at one level of the factor, such as leson ) or wthn-subjects varables (e.g. test day; each subject s measured at more than one level of the factor, such as day, day, and so on). You should now be able to dentfy your desgn (e.g. one between-subjects factor and two wthn-subjects factors ) usng ths document. he sectons gvng detal on each desgn also gve the SPSS syntax. You should check that the assumptons of NOV are met for example, do you need to transform your dependent varable (by takng the square root, arcsne, logarthm, etc.) before analyss? Run the NOV. If your NOV has wthn-subjects factors, check Mauchly s test of sphercty, whch your software should have done for you. If the Mauchly test s sgnfcant (small p value), one of the assumptons behnd wthn-subjects NOV has been volated. Don t use the normal df; use the corrected df ether wth the Greenhouse Gesser (conservatve) or Huynh Feldt (better; Myers & Well, 995, p. 48) correcton. Your software should provde both. Interpret the results. You may need to perform further analyses post hoc to explan man effects or nteractons that you fnd. I use a notaton for descrbng NOV models n whch factors are wrtten wth ther number of levels as a subscrpt, covarates are wrtten wth cov as a subscrpt, S denotes subjects, factors/covarates n brackets wth S are wthn-subjects predctors, and unbracketed factors/covarates are between-subjects predctors. n NOV wth one between-subjects factor () and two wthn-subjects factors (U, V) mght be wrtten lke ths: dependent varable (U V S) s a more concrete example of ths notaton, suppose you measured locomotor actvty (dependent varable) n two groups of rats (sham/leson). Each rat was tested on sx occasons: followng one of three drug treatments (salne/low-dose cocane/hgh-dose cocane), and n one of two rooms (hot/cold). We assume the testng order for wthn-subjects factors was approprately counterbalanced to avod order effects (see handouts at We could wrte ths desgn as: locomotor actvty Group (Drug 3 Room S) In ths document, I wll try to use, B, C as labels for between-subjects factors and U, V, W as labels for wthn-subjects factors, snce t gets hard to read otherwse when there are both between- and wthn-subjects factors n a desgn. Desgns wth both between-subjects and wthn-subjects factors are called mxed or nested desgns (Keppel, 99, p. 563): varablty due to subjects s nested wthn varablty due to the between-subjects factor(s), because each subject s only tested at one level of the between-subjects factor(s).

7 : Quck summary 7 If you have unts of relatedness other than subject (e.g. plot of land ), but you only have one level of relatedness, you can merely thnk of your desgn n the same between-/wthn-subject terms. If you have multple levels of relatedness, you wll need a complex or herarchcal desgn (Myers & Well, 995, chapter ); you should am to understand the prncples behnd the desgns dscussed n ths document. t the end we ll cover some herarchcal desgns, but ths s hard stuff.

8 : NOV bascs 8 Part : understandng the bascs of NOV. he basc logc and assumptons of NOV.. he underlyng model fter Howell (997, ch. ). Suppose that the average heght of UK adults s 75 cm, that of adult females s 7 cm, and that of adult males s 8 cm. So maleness contrbutes, on average, +5 cm to an adult s heght (compared to the mean of all adults), and femaleness contrbutes, on average, 5 cm. Suppose we take a gven adult male. We could break hs heght down nto three components: 75 cm for beng an adult, 5 cm for beng a male, and some other component that represents ths ndvdual s unqueness, snce there s of course varaton n the heghts of adult men. We could wrte ths model as or n more general terms heght 75 cm + 5 cm + unqueness heght ndvdual male µ + τ + ε male ndvdual where µ s the overall mean (75 cm), τ male s the contrbuton for beng a male, and ε ndvdual s a partcular ndvdual s unque contrbuton. We have wrtten an expresson for our dependent varable (heght) n terms of predctor varables (the grand mean and a factor, sex) and unpredcted varablty. Let s extend that prncple... n example: data and a structural model Suppose 5 subjects are assgned to fve groups. Each group reads a lst of words n a dfferent way: one was asked to count the number of letters n each word, one to thnk of a rhyme for each word, one to gve an adjectve that could be used wth each word, one to form a vvd mage of each word, and one to memorze each word for later recall. Later, all groups were asked to recall all the words they could remember. In NOV termnology, we have a sngle factor Group wth fve levels (Group, Group, Group 5 ). Here are some results (Howell, 997, p. 3): No. words Group Group Group 3 Group 4 Group 5 otal recalled Countng Rhymng djectve Imagery Memorze One 9 7 number, one subject total x 53 n n n n 3 n 4 n 5 N 5 mean x 7 x 6.9 x 3 x x 5 x.6 SD s.83 s.3 s 3.49 s s varance s 3.33 s 4.54 s 3 6. s 4.7 s 5 4 s 6.6 For ths data, we can specfy a model, just as we dd before. Let j represent the score of person j n condton (group) µ represent the overall mean score µ represent the mean of scores n condton τ represent the degree to whch the mean of condton devates from the overall mean (the contrbuton of condton ),.e. µ µ τ

9 : NOV bascs 9 ε j represent the amount by whch person j n condton devates from the mean of hs or her group (the unqueness of person j n condton ),.e. εj j µ Snce t s obvous that t follows that µ + µ µ ) + ( µ ) j ( j µ + τ + ε j j..3 he null hypothess We wll test the null hypothess that there s no dfference between the varous groups (condtons). We can state that null hypothess lke ths: H : µ µ µ µ µ µ In other words, the null hypothess s that all means are equal to each other and to the grand mean (µ), and that all treatment (group) effects are zero...4 he assumptons of NOV If µ represents the populaton mean of condton and σ represents the populaton varance of ths condton, analyss of varance s based on certan assumptons about these populaton parameters.. Homogenety of varance We assume that each of our populatons has the same varance: σ σ 3 σ 4 σ 5 σ e σ e he termσ (where e stands for error) represents the error varance the varance unrelated to any treatment (condton) dfferences. We would expect homogenety of varance f the effect of any treatment s to add or subtract a constant to everyone s score wthout a treatment the varance would be σ e, and f you add a constant to a varable, the varance of that varable doesn t change.. Normalty We assume that the scores for each condton are normally dstrbuted around the mean for that condton. (Snce ε j represents the varablty of each person s score around the mean of that condton, ths assumpton s the same as sayng that error s normally dstrbuted wthn each condton sometmes referred to as the assumpton of the normal dstrbuton of error.) 3. Independence of error components ( ndependence of observatons) We also assume that the observatons are ndependent techncally, that the error components (e j ) are ndependent. For any two observatons wthn an expermental treatment, we assume that knowng how one of these observatons stands relatve to the treatment (or populaton) mean tells us nothng about the other observaton. Random assgnment of subjects to groups s an mportant way of achevng ths. o deal wth observatons that are not ndependent for example, observatons that are correlated because they come from the same subjects we need to account specfcally for the sources of relatedness to make sure that the resdual error components

10 : NOV bascs are ndependent; ths s why we need wthn-subjects (repeated measures) desgns for ths sort of stuaton. But we ll gnore that for the moment...5 he logc of NOV Snce we have assumed that the dstrbuton of the scores for each condton have the same shape (are normally dstrbuted) and have the same varance (homogenety of varance), they can only dffer n ther means. Now f we measure the varance of any one condton, such as s, that varance wll be an estmate of the common populaton varance σ e (remember, we assumed σ σ σ 3 σ 4 σ 5 σ e, that s, homogenety of varance). In each case, our sample varance estmates a populaton varance: s σ ; σ s ; σ 5 s5 (where denotes s estmated by ). Because of our homogenety of varance assumpton, each of these sample varances s an estmate of σ e : σ e s ; σ e s ; σ e s 5 o mprove our estmate of σ e, we can pool the fve estmates by takng ther mean (f n n n 3 n 4 n 5 n), and thus σ s e s s a where a s the number of treatments n ths case, 5. (If the sample szes were not equal, we would stll average the fve estmates, but we would weght them by the number of degrees of freedom for each sample, so varance estmates from larger samples would get more weght.) hs gves us an estmate of the populaton varance that s referred to as MS error ( mean square error ), sometmes called MS wthn, or MS subjects wthn groups, or MS S /groups ( mean square for subjects wthn groups ). hs s true regardless of whether H s true or false. For the example above, our pooled estmate of σ e wll be σ e Now let us assume that H s true. In ths case, our fve samples of cases may be thought of as fve ndependent samples from the same populaton (or, equvalently, fve samples from fve dentcal populatons). he Central Lmt heorem (see handouts at states that the varance of means drawn from the same populaton s equal to the varance of the populaton dvded by the sample sze. If H s true, therefore, the varance of our fve sample means estmates σ n : and so e / e s x σ n σ e ns x σ e hs s therefore a second estmate of, referred to as MS treatment or MS group. On the other hand, f H s false, ths wll not be a good estmate of σ e. So we have found that MS error estmates σ e whether H s true or false, but MS treatment only estmates σ e f H s true. herefore, f our two estmates of σ e, MS treatment and MS error, are smlar, ths s evdence that H s true; f they are very dfferent, ths s evdence

11 : NOV bascs that H s false. We wll compare the two varance estmates wth an F test, whch s desgned specfcally for comparng two varances (see handouts at F MS treatment /MS error. If our F statstc s very dfferent from, we wll reject the null hypothess that the two varances (MS treatment and MS error ) are the same, and hence reject the null hypothess of the NOV...6 Expected mean squares (EMS) Let s formalze that. We ve defned the treatment effect τ as µ µ, the dfference between the mean of treatment (µ ) and the grand mean (µ). We wll also defne σ τ as the varance of the true populaton s means (µ, µ, µ a ): σ τ ( µ µ ) τ a a echncally, ths s not actually the varance snce we are workng wth parameters, not statstcs, we should have dvded by a rather than a f we wanted the varance. However, we can thnk of t as a varance wthout much problem. We can then defne, wthout proof, the expected value of the mean square terms: E(MS E(MS error treatment ) σ ) σ e e n τ + a σ e + n σ τ where σ e s the varance wthn each populaton and σ τ s the varance of the populaton means (µ j ). So f H s true, σ τ, so E ( MStreatment ) E(MSerror ), but f H s false, E MS ) > E(MS ). ( treatment error. he calculatons behnd a smple one-way NOV (one between-subjects factor) Let s go back to the results n the table we saw earler and conduct an NOV... Calculatons wth means (conceptual) or totals (for manual calculaton only) Most NOV calculatons are based on sums of squares. Remember that a varance s a sum of squared devatons from the mean (a sum of squares ) dvded by the number of degrees of freedom. We work wth sums of squares because they are addtve, whereas mean squares and varances are only addtve f they are based on the same number of degrees of freedom. Purely for convenence, Howell (997) tends to do the calculatons n terms of treatment totals rather than treatment means. In the table above, we have defned as the total for treatment. otals are lnearly related to means ( nx ). If you multple a varable by a constant, you multply the varance of that varable by the square of the constant. So snce nx, we can see that We saw earler that f H s true, σ s x s n s x e ns x σ e s n ; therefore, f H s true, s n

12 : NOV bascs On the other hand, though calculatng sums of squares may be easer n terms of treatment totals, conceptually t s much easer to thnk n terms of means. We ll present both for a whle frst the defnton n terms of means, and then, n brackets, the formula n terms of totals. Ignore what s n brackets unless you re dong the calculatons by hand. Eventually we ll just show the calculatons n terms of means. fter all, you ll be usng a computer for the hard work... Calculatng SS total, SS treatment, and SS error Frst, we calculate SS total ( total sum of squares ) the sum of squares of all the observatons (the summed squared devatons of each observaton from the overall mean), regardless of whch treatment group the observatons came from. SS total ( x x) x ( x) N Now we calculate SS treatment. hs represents the summed squared devatons of the treatment mean from the mean of all treatment means, summed over each data pont. (Or, n terms of totals, the summed squared devatons of each total [ j ] from the mean of the treatment totals [ ], all dvded by the number of observatons per total.) SS treatment n( x x) ( ) n n n n n ( ) n ( ) a ( ) n n na ( x) n N where a s the number of treatments, n s the number of observatons per treatment, and N s the total number of observatons ( na). Now we can calculate SS error. hs represents the sum of the squared devatons of each pont from ts group mean. Snce SS total SS treatment + SS error, the quck way to obtan SS error s by subtracton: SS error ( x x ) SS total SS treatment lternatvely, we could have calculated SS error by workng out an SS for each group separately and addng them up: SS SS group group SS error ( x x ) SS group ( x x ) + SS (9 7) group (7 6.9) + (8 7) + + SS + (9 6.9) group (7 7) + + (7 6.9) Both approaches gve the same answer.

13 : NOV bascs 3..3 Degrees of freedom If there are N observatons n total, df total N. If there are a treatments, df treatment a. We can calculate the degrees of freedom for error lke ths: df error df total df treatment lternatvely, we could calculate df error as the sum of the degrees of freedom wthn each treatment; f there are n observatons n each of a treatments, there are n degrees of freedom wthn each treatment, and so df error a(n ). hs gves the same answer (snce df total df treatment [N ] [a ] [na ] [a ] na a a[n ])...4 Mean squares Mean squares are easy; just dvde each SS by the correspondng number of df...5 he F test From the defntons of EMS above, E(MS E(MS treatment error We can therefore calculate an F statstc ) σ ) MS F MS error e treatment + nσ and t s dstrbuted as F a, a(n ) that s, as F treatment df, error df, under the null hypothess. So we can look up crtcal value of F n tables. If t s sgnfcant (unlkely gven the null hypothess), we reject the null hypothess and say that the treatment dd nfluence our dependent varable. σ e τ very complcated asde: f H s true and σ τ, although E(MS E(MS treatment error e e τ ) σ + nσ and therefore under the null hypothess ) σ E(MStreatment ) MStreatment, and so you d thnk E E(MSerror ) MS, the expected value of F error dferror under the null hypothess s actually E ( F ) (Frank & lthoen, 994, pp. df error 47, 53). I don t fully understand that; I suspected that the dfference was that E(MStreatment ) MS treatment E because E ( Y ) E( ) E( Y ) only f and Y are E(MSerror ) MSerror ndependently dstrbuted. MRF has snce ponted out the real reason: under the null hypothess, MS error s asymmetrcally dstrbuted. For asymmetrc dstrbutons, E( ) E( ), so E( MSerror ) E(MSerror ). It s akn to the reasonng behnd usng a t test rather than a Z test when you estmate the populaton standard devaton σ usng the sample standard devaton s: even though E ( s) E( σ ), ( s) E( σ ) E...6 NOV summary table NOV results are presented n a summary table lke ths:

14 : NOV bascs 4 Source d.f. SS MS F reatment a SS treatment SS treatment /df treatment MS treatment /MS error Error (S/treatments) a(n ) SS error SS error /df error otal N an SS total SS total /df total [ s ] Remember that S/treatments denotes subjects wthn treatments ; ths s the source of all our error varablty n ths example. nyway, for our example, we can now calculate all the SS: SS SS SS ) 69 (7 ) ( ) ( SS ) 8 (9 ) ( ) ( SS treatment total error treatment total N x n x x n N x x x x so our NOV table looks lke ths: Source d.f. SS MS F reatment Error (S/treatments) otal Our F has (4, 45) degrees of freedom. We could wrte F 4,45 9.9, and look ths up to fnd an assocated p value (p.)...7 SS treatment for unequal sample szes What f our group szes were not equal? Prevously we had defned N x n na n n a n n nx nx n x x n x x n treatment ) ( ) ( ) ( ) ( ) ( ) ( ) ( SS whch apples when all groups have n observatons. If the group szes are unequal, we smply multply the devaton of each score from ts treatment mean by the number of scores n that treatment group (so the larger one sample s, the more t contrbutes to SS treatment ): N x n n n x n x n n x x n x x n treatment ) ( ) ( ) ( ) ( ) ( SS..8 Pctoral representaton What the NOV technque has done s to partton total varaton from the overall mean (SS total ) nto varaton from the overall mean accounted for or predcted by the treatment or group dfference (SS treatment or SS groups ) and further varaton wthn the

15 : NOV bascs 5 groups due to nter-subject varablty (SS error or SS S/groups ). If the varaton attrbutable to the model s large, compared to the error varablty, we wll reject the null hypothess. SS total ( x x) df total N he sum of squares s the sum of the squared lengths of the vertcal lnes (devatons from the mean). SStreatment n ( x x) dftreatment a Do you see now why we ve been multplyng the devatons by the group sze to fnd SS treatment? SSerror ( x x ) df N a error nother way to look at NOV s ths: the hypothess test we have performed effectvely compares two models (Myers & Well, 995, p. 44-): one (restrcted) model allows for the effects of a mean only all other varablty s error ; the other (full) model allows for the effects of a mean and a treatment (and everythng else s error). If the full model accounts for sgnfcantly more varablty than the restrcted model, we reject the null hypothess that the treatment has no effect...9 Relatng SS calculatons to the structural model Note that our structural model was ths: j j µ + τ + ε τ µ µ ε j µ j and our SS were these: SS SS total treatment SS error ( x x) n( x x) ( x x ) SS total SS treatment See the smlarty? We can prove that the one follows from the other. hs s not somethng we have to do routnely, but t demonstrates how the sums of squares (SS) are derved drectly from the model. Our model was ths: or j µ + τ + ε µ + µ µ ) + ( µ ) j ( j j Rearrangng to express the left-hand sde as a devaton of each score from the overall mean: µ µ µ ) + ( µ ) j ( j

16 : NOV bascs 6 Squarng each sde: Summng over and j: a ( µ ) ( µ µ ) + ( µ ) + ( µ µ )( µ ) j a a j ( µ ) n ( µ µ ) + ( µ ) + ( µ µ )( µ ) n j j he far-rght term s actually zero: a ( µ µ )( n j j n j a j µ ) ( µ µ ) ( µ ) a ( µ µ ) snce the sum of devatons of all observatons about ther mean s zero. So we re left wth: n j a n j j j j a SS ( µ ) n ( µ µ ) + ( µ ) n j j total a SS + SS he degrees of freedom are smlarly related: error a n j j df total df + df error.3 Regresson NOV: the other way to understand the basc logc.3. Lnear regresson n terms of sums of squares Suppose that n some way we can measure the rsk of a heart attack (call t Y) n many 5-year-old men. If we then want to predct the rsk of a heart attack n an unknown 5-year-old man, our best guess would be the mean rsk of a heart attack ( y ). If we call our predcted varable Ŷ, and a predcted ndvdual value ŷ, then our best guess could be wrtten We could also wrte t lke ths: y ˆ y y y + ε where ε represents error varablty or natural varaton. he error n our best guess would be the same as the natural varablty n Y t would be descrbed by some way by the standard devaton of Y, s Y, or the varance, s Y. he sample varance (whch estmates the populaton varance), remember, s s Y ( y y) n hs varance, lke any varance, s the sum of squared devatons about the mean dvded by the number of degrees of freedom that the varance estmate s based on. Because they are convenently addtve, we could wrte the varablty n our estmate just n terms of the sum of squared devatons about the mean the sum of squares:

17 : NOV bascs 7 SSY SStotal ( y y) hs s the total varablty n cholesterol, so t s sometmes wrtten SS total. Now suppose we also measure cholesterol levels () for each of our subjects. We now have (x, y) pars (cholesterol and heart attack rsk) for each subject. We could predct Y from usng lnear regresson. We would call the predcted varable Ŷ, and we d call an ndvdual predcted value ŷ. standard lnear regresson (see handouts at wll gve us ths equaton: Y ˆ a + b where a s the ntercept and b s the slope. We could also wrte our model lke ths: y yˆ + ε a + bx + ε Now our best guess of the heart attack rsk of a new subject should be rather better than y ˆ y ; f we measure our new subject s cholesterol as well, we can make what should be a better predcton: y ˆ a + bx he error n ths predcton wll related to the devatons between the predcted value, ŷ, and the actual value, y. We could wrte ths ether n terms of a varance or as a sum of squares: resdual s ( y yˆ) n SSresdual ( y yˆ ) If cholesterol s somehow lnearly related to heart attack rsk, the error n our predcton, whch was SS total, has now been reduced to SS error. herefore, the amount of varablty n Y that we have accounted for by predctng t from, whch we can wrte as SS regresson or SS model or SSY ˆ, s based on the dfference between the predcted values and the overall mean: It s also true that SSmodel ( yˆ y) and that ( y y) ( yˆ y) + ( y yˆ) SS SS + SS df total total df model model n + ( n ) + df resdual resdual Snce we have already calculated the overall mean, and the regresson lne always passes through the overall mean, the regresson model has one df (ts slope). hat s, people vary n ther cholesterol levels (SS ), they vary n ther heart attack rsk (SS Y SS total ), a certan amount of the varablty n ther heart attack rsk s predctable from ther cholesterol ( SS ˆ SS Y model ), and a certan amount of varablty s left over after you ve made that predcton (SS resdual SS error ). Incdentally, the propor-

18 : NOV bascs 8 ton of the total varablty n Y that s accounted for by predctng t from s also equal to r : r SS Yˆ SS Y SS SS model total We can llustrate SS total, SS model, and SS resdual lke ths: SS df total total ( y y) N SS df model model ( yˆ y) SS df error error ( y yˆ) N What would t mean to alter SS model and SS resdual? If you pulled all of the scores further away from the regresson lne (f a pont s above the regresson lne, move t up; f t s below, move t down) wthout changng the slope of the regresson lne, you d ncrease SS error wthout alterng SS model. If you altered the slope of the regresson lne but moved the ndvdual scores up or down to keep them the same dstance from the lne, you d ncrease SS model wthout changng SS resdual..3. Lnear regresson as an NOV We can use ths way of wrtng a lnear regresson model to express lnear regresson as an NOV. If there s no correlaton between and Y, then predctng Y from won t be any better than usng y as our estmate of a value of y. So we could obtan a measure of the total varablty n Y: MS and we could smlarly obtan ( y y) Y MStotal s Y n SS df total total MS model ( yˆ y) SS s Y ˆ df model model MS resdual MS error ( y yˆ) SS s resdual n df resdual resdual If the null hypothess s true and there s no correlaton between and Y, then some of the varaton n Y wll, by chance, ft a lnear model, and contrbute to SS model. he rest wll not, and wll contrbute to SS resdual. he correspondng MS values, once we have dvded by the df, wll be measurng the same thng the varablty of Y. hat s, under the null hypothess, E(MS model ) E(MS error ). On the other hand, f there s a correlaton, and Y vares consstently wth, then SS model wll contan varaton due to ths effect as well as varaton due to other thngs (error), but SS resdual wll only contan varaton due to other thngs (error). herefore, f the null hypothess s false, E(MS model ) > E(MS error ). We can therefore compare MS model to MS error wth an F test;

19 : NOV bascs 9 f they are sgnfcantly dfferent, we reject the null hypothess. Our NOV table would look lke ths: Source d.f. SS MS F Model SS model SS model /df model MS model /MS error Error (resdual) N SS error SS error /df error otal N SS total where N s the total number of (x, y) observatons. o calculate a regresson NOV by hand, SS total can be calculated as s Y ( n ) and SS model can be calculated as r SS. total.4 Factors versus covarates We ve seen that we can perform an NOV to predct our dependent varable usng a dscrete varable, or factor, that has several levels as when we asked whether word recall dffered between fve groups that had read the same word lst n dfferent ways. We saw a pctoral representaton of a three-group example. We ve also seen that we can perform an NOV to predct our dependent varable usng a contnuous varable, or covarate, as n our lnear regresson example, and we ve seen a pctoral representaton of that. he mathematcal technque of NOV does not care whether our predctor varables are dscrete (factors) or contnuous (covarates). We ll see that n Part 6 when we look at the dea of a general lnear model (p. 84). However, the way most people use covarates s slghtly dfferent from the way they use factors. If you are runnng an experment, you do not generally assgn subjects to dfferent values of a contnuous varable (covarate) you assgn subjects to dfferent levels of a factor, wth several subjects per level (group). herefore, real-lfe covarates are generally thngs that you measure rather than thngs that you manpulate. s a consequence, most people use covarates and analyss of covarance (NCOV) as a way to ncrease the power of NOV f you can account for some of your error varablty by usng a covarate to predct your dependent varable, there s less error varablty and therefore there may be more power to detect effects of the factors that you re nterested n..5 ssumptons of NOV nvolvng covarates ake a common desgn nvolvng covarates: a desgn wth one between-subjects factor and one between-subjects covarate. Suppose you have chldren at your dsposal. You measure ther IQ. hen you randomly assgn 5 chldren to receve the standard method of maths teachng, and 5 chldren to receve a new method. hs represents the between-subject factor Method, wth two levels. fter some tme, you measure ther mathematcal problem-solvng ablty. But you suspect that ther IQ may also play a part n determnng ther fnal score, not just the teachng method IQ may be contrbutng to the error (unmeasured) varablty n the scores of your two groups. So you enter IQ as a covarate nto your NOV model. hs covarate may therefore account for some of the prevously-unmeasured varablty, reducng your error term, and ncreasng the power to detect an effect of teachng method. If you use NCOV n ths way, there are a few assumptons (Myers & Well, 995, pp ; Howell, 997, p. 587): that the relatonshp between the covarate and the dependent varable s lnear; that the regresson slopes relatng the covarate to the dependent varable are the same n both groups homogenety of regresson.

20 : NOV bascs hs s the desgn dscussed n 7.. (p. 38). he second assumpton s drectly testable, and the method for testng t s dscussed n 7.. (p. 44). fnal assumpton n ths sort of desgn s ths: that the covarate and the treatment are ndependent (Myers & Well, 995, p. 45). If ths s not the case, nterpretaton s very dffcult. Usng as a covarate removes the component of Y predctable from. If the treatment nfluences or s otherwse predctable from, performng an NCOV wll not smply remove nusance varablty from Y; t wll remove part of the effect of the treatment tself. For example, f you had measured IQ at the end of the experment and the teachng method actually nfluenced IQ, nterpretaton would be very hard; smlarly, t would be hard to nterpret f you had assgned hgh-iq students to one teachng method and low-iq students to another. hs can also be a problem n stuatons when you are usng (for example) patent groups and IQ (f the patents have a dfferent IQ to the controls), or sex and body mass (males have a hgher body mass)..6 NOV wth two between-subjects factors We can extend our basc one-way NOV to two factors. Suppose we have two factors, one wth two levels and one wth fve levels; ths desgn would be called a 5 factoral. Suppose we repeat our prevous experment (Howell, 997, p. 43) but for young and old subjects. Factor s age (young versus old); factor B s task type (countng, rhymng, adjectve, magery, ntentonal). Suppose our results look lke ths: No. words B B B 3 B 4 B 5 otal recalled Countng Rhymng djectve Imagery Memorze 9 7 old total B 7 B 69 B3 B4 34 B young hs dotted lne encloses one cell. 5 7 hs s a very mportant term to B 65 B 76 B3 48 B4 76 B understand! B 35 B 45 B3 58 B4 3 B5 33 Σx 6 Note our defnton of cell one partcular (, B) condton, such as B (shown here wth a dotted lne around t)..6. Structural model and termnology (man effects, nteractons, smple effects) Our NOV must allow for the effects of factor, and factor B. It should also allow the possblty that and B nteract that the effect of factor depends on the level of factor B, or vce versa. For example, suppose that young people are generally better, regardless of task type; we would call ths a man effect of factor (age). man effect s an effect of a factor regardless of (gnorng) the other factor(s). Suppose that the memorze condton gves better recall than the countng condton, regardless of age; we would call ths a man effect of factor B (task type).

21 : NOV bascs On the other hand, perhaps young people have a partcular advantage n the memorze condton but not n other condtons; ths would be an nteracton between and B, wrtten B or sometmes B. We may also defne, for later, the term smple effect: ths s an effect of one factor at only one level of another factor. For example, f the memorze condton gves better performance than the adjectve condton consderng young people only, ths s a smple effect of factor B (task type) at the young level of factor (age). We can specfy a model, just as we dd before: µ + α + β + αβ + ε jk j j jk where jk the score of person k n condton B j µ the overall mean score α the degree to whch the mean of condton devates from the overall mean ( the contrbuton of condton ),.e. α µ µ. By ths defnton, α. β j the degree to whch the mean of condton devates from the overall mean ( the contrbuton of condton B j ),.e. β µ µ. By ths defnton, β j. αβ j the degree to whch the mean of condton B j devates from what you d expect based on the overall mean and the separate contrbutons of and B j ( the nteracton B),.e. αβ µ µ + α + β ). By ths defnton, αβ αβ. j j j j B j j B j ( j ε jk the error or amount by whch person k n condton B j devates from the mean of hs or her group (the unqueness of person k n condton B j ),.e. ε jk jk ( µ j + α + β j + αβj ). By our usual assumpton of normal dstrbuton of error, ε jk s normally dstrbuted wth mean and varance σ e..6. Expected mean squares lthough we won t derve t, the EMS terms are: Source E(MS) σ e + nbσ B σ e + naσ B B ( B) σ e + nσ B Error e σ (Note that these EMS values assume that the factors are fxed factors; see p. 3.) So we should be able to form F ratos based on the error term. For example, f the null hypothess that factor has no effect s true, µ µ, so σ and E(MS ) E(MS error ). If ths null hypothess s false, E(MS ) > E(MS error ). So the rato E(MS E(MS error e ) σ + nbσ ) σ can be tested usng an F test wth df and df error degrees of freedom..6.3 Degrees of freedom here are n subjects per (, B) condton (per cell), so N observatons n all. herefore, df total 99. By our usual rules, df and df B 4 (one less than the e

22 : NOV bascs number of levels). he nteracton term, wrtten B or B, represents the possblty that the effects of factors and B represent each other. he df for an nteracton term B s always the product of df and df B n our example, 4. So our total df are parttoned lke ths: df total df + df B + df df so we have 9 error df n our example..6.4 Sums of squares Smlarly, total B B error B + df error SS SS + SS + SS + SS SS total s calculated exactly as before: the sum of squared devatons of every observaton from the grand mean. ( x) SStotal ( x x) x n he SS for factor s calculated exactly as we would f ths were a one-way NOV wthout the other factor. he same s true for SS B. hat s, we take the sum of the squares of the devatons of the means of each condton (, ) from the overall mean, summed over every observaton. (In terms of totals, t s the sum of the squares of the devatons of the totals of each condton,, from the overall mean total, dvded by the number of observatons on whch each mean was based.) In our example, snce there are condtons and each s made up of n observatons per cell and 5 cells ( b levels of B) per condton, there are nb observatons contrbutng to each condton mean. So: SS SS B nb( x x) na( x x) B ( ) nb ( B ) na nb na error B ( x) N ( x) N o fnd the nteracton term SS B, we calculate an ntermedate value, SS cells, whch measures the varablty of the cell means. Snce cell varablty can be due to, B, or B, we can see that SS cells SS + SS B + SS B, and therefore calculate SS B ths way. SS cells s the sum of the squares of the devatons of ndvdual cell means from the grand mean, summed over each observaton. (In terms of totals, t s the sum of the squares of the devatons of ndvdual cell totals from the grand mean total, dvded by the number of observatons that contrbuted to each cell mean.e. the number of observatons per cell.) Whew. SS SS cells B ( n( xb x) SS (SS + SS ) cells B ) n B B nb ( x) N o fnd the error term, we know that SS SS + SS + SS + SS SS + SS total B B so we can fnd SS error by subtracton. lternatvely, we could calculate SS error as the grand sum of the sums of the squares of the devatons of ndvdual observatons from ther cell means. error cells error

23 : NOV bascs Relatng SS calculatons to the structural model Note that our structural model was ths: and our SS were these: See the smlarty?.6.6 NOV table SS SS ε jk j αβ µ SS We ve ended up wth ths: j jk µ + α + β + αβ + ε α µ µ β µ total SS SS B B error B B j jk j µ ( x x) j ( µ + α + β ) ( µ + α + β + αβ ) nb( x x) na( x x) j j j jk n( xb x) (SS + SSB) SS (SS + SS + SS ) total B B Source of varaton d.f. SS MS F Between cells df +df B +df B SS cells a SS SS /df MS /MS error B b SS B SS B /df B MS B /MS error B ( B) (a )(b ) SS B SS B /df B MS B /MS error Wthn cells ( error S/cells) ab(n ) SS error SS error /df error otal N abn SS total SS total /df total [ s ] j B.7 Wthn-subjects (repeated measures) NOV Prncple: f a set of measurements are more correlated than we would expect by chance, we must account for ths correlaton. We can say that these measurements come from the same subject (n psychologcal termnology), or that ths measure was repeated. Suppose we have one wthn-subjects factor. Call t U. Let s suppose we ve measured all our subjects n three condtons (U hot, U warm, U 3 cold), once each, and have counterbalanced approprately to avod nasty order effects. ll we have to do s to partton the sum of squares so as to account for the fact that we ve measured subjects several tmes each.7. Structural model Our structural model s ether one of these two: j j µ + π + α + ε (Model : addtve ) j j j j µ + π + α + πα + ε (Model : nonaddtve ) j where j s the dependent varable for subject n condton U j µ s the overall mean

24 : NOV bascs 4 π s the contrbuton from a partcular person or subject (subject, or S ): π µ S µ. α j s the contrbuton from a partcular level (level j) of the factor U: α j µ U j µ. πα j s the contrbuton from the nteracton of subject wth treatment j: π µ S U ( µ + π + α j ). hs nteracton would reflect that the subjects responded dfferently to the dfferent levels of U. j ε j s everythng else (the expermental error assocated wth subject n condton j). In Model, ths wll be ε j j ( µ + π + α j ). In Model, ths wll be ε j j ( µ + π + α j + παj ). hese two models dffer n the presence or absence of πα j, the nteracton of U wth a partcular person (Howell, 997, pp ). Includng t makes for a realstc model t s lkely that subjects do not all respond equally to all condtons (levels of U). However, f we measure each person n each condton once, we wll not be able to measure dfferences n the way subjects respond to dfferent condtons ndependently of other sources of error such as measurement error. (o do that, we d need to measure subjects more than once per condton, and then we d need a dfferent model agan!) hs s another way of sayng that the S U nteracton s confounded wth s! the error term..7. Degrees of freedom We partton the df lke ths: df df total wthn subjects df df between subjects U + df error S U + df wthn subjects herefore df df df total total between subjects df df U error df N s u df between subjects total df + df U between subjects + df where s s the number of subjects, u s the number of levels of U, and N s the total number of observatons ( su). We could also wrte df between subjects as df S, whch you sometmes see..7.3 Sums of squares Smlarly, we can partton the SS lke ths: SS SS total wthn subjects SS total SS SS SS between subjects U + SS error S U between subjects + SS + SS U error wthn subjects + SS error We can defne our SS as usual SS SS total SS U between subjects ( x x) s( xu x) u( x x) S

25 : NOV bascs 5 where s s the number of subjects and u s the number of levels of U. x U represents the mean for a partcular level of U (across subjects), and x S represents the mean for a partcular subject (across levels of U). Our total number of observatons wll be N su..7.4 EMS and NOV summary table he EMS depend on whch model we use: Source of varaton Model : E(MS) Model : E(MS) Between subjects (S) σ + σ + e uσ S U σ e + sσ U Error e σ e uσ S e + σus e σ US σ + sσ σ + hs means that n Model t s rather hard to do a proper F test for the between subjects factor, snce there s no term whose E(MS) s dentcal to E(MS between subjects ) except for the presence of σ S, the relevant varance for the between-subjects factor. On the other hand, who cares. If ths term were sgnfcant, all t would tell us s that subjects are dfferent, whch s hardly earth-shatterng. Ether way, we have no problem testng U: the proper way to test for an effect of U s to do an F test comparng MS U to MS error. If Model s true f subjects respond equally to the treatments; f the effects are addtve we wll have more power to detect effects of U, snce f the null hypothess (that U has no effect) s false, U E(MS E(MS Umodel errormodel ) σ e + sσ ) σ e U σ > e + σ σ US e + σ + sσ US U E(MS E(MS Umodel ) ) errormodel and the bgger the rato of MS U to MS error, the bgger the F rato, and the more lkely the effect s to be sgnfcant (Myers & Well, 995, p. 44; Howell, 997, pp ). You may be thnkng the calculatons for the two models are exactly the same n practce, so why all ths fuss? You d be rght unless you wanted to estmate the proporton of varance accounted for by a partcular term (Myers & Well, 995, pp. 4, 5-55). See p...8 ssumptons of wthn-subjects NOV: Mauchly, Greenhouse Gesser, etc..8. Short verson. ny NOV nvolvng wthn-subjects factors has a potental problem. here s an assumpton known as sphercty [of the covarance matrx]. If ths assumpton s volated, ype I error rates wll be nflated (f the null hypothess s true, you wll get too many results that you wll declare sgnfcant than you should). Mauchly s test of sphercty checks for ths. sgnfcant Mauchly s test means that the assumpton s lkely to have been volated. But t s not a very good test (see below), so we should probably gnore t.. Correct the df for any term nvolvng a wthn-subjects factor, and the correspondng error df, by multplyng them both by a correcton factor. he correcton factor s known as epslon (ε). If the sphercty assumpton s not volated, ε (so applyng the correcton changes nothng). You do not need to correct any terms that have only between-subjects factors. nd you can never

26 : NOV bascs 6 volate the sphercty assumpton for a wthn-subjects factor that has only levels. 3. Use ether the Greenhouse Gesser or the Huynh Feldt epslon. he Greenhouse Gesser one (sometmes wrtten εˆ ) s probably a bt too conservatve; the Huynh Feldt one (sometmes wrtten ε ~ ) s better (Myers & Well, 995, p. 48; Howell, 997, p. 465) but more detal below. SPSS reports Mauchly s test and both the G G and H F correctons whenever you run a wthn-subjects NOV usng ts menus. Just to confuse you, there are actually several dfferent approaches: NO HE BES: Look at the results of Mauchly s test; apply a correcton (G G or H F) f and only f Mauchly s test s sgnfcant for a factor that s part of the term n queston, ndcatng a volaton of the sphercty assumpton. hs s not deal, because Mauchly s test sn t very relable (Myers & Well, 995, p. 46; Howell, 997, p. 466, and see below). NO HE BES: lways use the Greenhouse Gesser correcton. oo conservatve. Good and smple: lways use the Huynh Feldt correcton. hs s not totally deal because the H F procedure tends to overestmate sphercty (be a bt too optmstc) (see refs n Feld, 998), but t s pretty good; Myers & Well (995, p. 48) recommend t. OK but awkward: use the average of the εˆ and ε ~. Good: Look at the estmated epslons (G G εˆ and H F ε ~ ); f they re n the regon of.75 or hgher (n some textbooks, f εˆ.75) use the H F ε ~ ; f below, use the G G εˆ (Howell, 997, pp ). Of course, f you really want to avod ype I errors, you d be predsposed to usng the G G correcton (conservatve); f you d rather avod ype II errors, you d be predsposed to usng the H F correcton (more lberal)..8. Long verson Sphercty s the assumpton of homogenety of varance of dfference scores (Myers & Well, 995, p. 44-5); see also www-staff.lboro.ac.uk/~hutsb/spherc.htm. Suppose we test 5 subjects at three levels of. We can therefore calculate three sets of dfference scores ( 3 ), ( ), and ( 3 ), for each subject. Sphercty s the assumpton that the varances of these dfference scores are the same. Here are two examples: Data set : exhbtng sphercty (homogenety of varance of dfference scores) 3 dfference dfference dfference Subject 3 3 S S S S S mean varance

27 : NOV bascs 7 Data set B: exhbtng nonsphercty 3 dfference dfference dfference Subject 3 3 S S S S S mean varance a( a ) In general, f there are a treatment levels, there are s assumed that they all have the same varance. dfference scores, and t Obvously, the sphercty assumpton cannot be volated f the wthn-subjects factor has less than 3 levels. he sphercty assumpton wll be met f there s no S nteracton (f there s addtvty). In ths case, any dfference score s exactly the same over subjects, so there s zero varance n the dfference scores. However, sphercty can be met wthout addtvty, as shown above (that s to say, addtvty s suffcent but not necessary for sphercty). nother condton that s suffcent (but not necessary) for sphercty s compound symmetry. hs requres homogenety of the populaton treatment varances: σ σ and homogenety of the populaton covarances: ρ σ σ ρ, 3σ σ 3 ρ, 3σ σ 3, where ρ, s the populaton correlaton between the and scores, and ρ, σ σ s ther covarance (see handouts at pobox.com/~rudolf/psychology). he varance sum law tells us that the varance of a dfference between two varables s Y Y σ σ + σ ρ and so f the two condtons above are met, the varances of the dfference scores wll all be the same. Howell (997, p. 455) explans why the term compound symmetry s appled to ths stuaton, usng a matrx that llustrates varances and covarances between,, and 3 (ths s llustrated under covarance matrx n the Glossary on p. 4). However, the explanaton s not as clear as Myers & Well s. Yet data set shown above exhbts sphercty wthout compound symmetry (that s, although the varances of dfference scores are dentcal,.e. sphercty s true, the varances of the ndvdual scores are not the same and nor are the covarances for pars of treatments). Myers & Well (995, p. 46) don t lke Mauchly s test because t tends to gve sgnfcant results (suggestng a problem) even n stuatons when sphercty holds that s, usng Mauchly s test s a conservatve approach. he three thngs you can do about volatons of sphercty are () the usual F test wth adjusted degrees of freedom, as suggested above (after Box, 954); () multvarate NOV (MNOV) (see p. 9); (3) tests of planned contrasts (see p. 75). See Myers & Well (995, pp. 46-5). Y σ σ Y

28 : NOV bascs 8.9 Mssng data n desgns nvolvng wthn-subjects factors If some data are lost for a partcular subject, you have a problem. You can ether assume the addtve model dscussed above that the effect of wthn-subjects factors are the same for all subjects and estmate the mssng value (Myers & Well, 995, p. 56-8). Every tme you estmate a value, you reduce the df for the relevant error term by. If you don t assume the addtve model, you can t estmate the value, and you may then have to throw out all data for that subject. SPSS does the latter by default.. Mxed NOV (wth both between-subjects and wthn-subject factors) We wll llustrate the smplest mxed desgn here: one between-subjects factor and one wthn-subjects factor. General prncples of more complcated wthn-subjects models are gven by Keppel (99, pp ), and lad out n Part 7. Suppose we take three groups of rats, n 8 subjects per group (s 4 subjects total). We gve one group treatment, one group treatment, and one group treatment 3 (a 3). One subject only experences one treatment. Note that s an. hen we measure every subject s performance at sx tme ponts U U 6 (u 6). We have N su anu observatons n total. We frst partton the total varaton nto between-subjects varablty and wthnsubjects varablty. he between-subjects varablty can be attrbuted to ether the effect of the treatment group (), or dfferences between subjects n the same group ( S wthn or S/ ). (hs notaton ndcates that there s a dfferent group of subjects at each level of the between-subjects factor, ; we could not measure smply subject varaton ndependent of the effects of snce no subjects ever serve n more than one group, or level of.) So we have these sources of between-subjects varablty: S/ he wthn-subjects varablty can be attrbuted to ether the effects of the tme pont (U), or an nteracton between the tme pont and the drug group (U ), or an nteracton between the tme pont and the subject-to-subject varablty, whch agan we can only measure wthn a drug group (U S/). So we have these sources of wthn-subject varablty: U U U S/.. Structural model Followng Myers & Well (995, p. 95-6): jk + α + π j / + β k + αβk + πβ jk / µ + ε jk where jk s the dependent varable for subject j n group and condton U k µ s the overall mean α s the contrbuton from a partcular level (level ) of factor : α µ µ. By ths defnton, α. π j/ s the contrbuton from a partcular person or subject (subject j), who only serves wthn condton ( subject wthn group, or S/): π µ µ j / S j /.

29 : NOV bascs 9 (here s no straghtforward nteracton of wth S: every subject s only measured at one level of, so ths term would be ndstngushable from the subject-only effect π j/.) β k s the contrbuton from a partcular level (level k) of factor U: β k µ µ. By ths defnton, β j. U k αβ k s the contrbuton from the nteracton of and U k : αβ µ µ + α + β ). By ths defnton, αβ αβ. k U k ( k πβ jk/ s the contrbuton from the nteracton of U k wth subject j, whch can only be measured wthn one level of (t s the SU/ term): πβ µ µ + π + β ). By ths defnton, πβ. jk / S / ( ju k j / k k k k k jk / (here s no straghtforward three-way U S nteracton: every subject s only measured at one level of, so ths term would be ndstngushable from the SU/ effect πβ jk/.) ε jk s everythng else (the expermental error assocated wth measurng person j who always experences treatment n condton U k ): ε jk jk ( µ + α + π j / + β k + αβk + πβ jk / ). Note that we cannot actually measure ε jk ndependent of the SU/ term f we only have one measurement per subject per level of U; ths term smply contrbutes to the wthn-subjects error term (see below)... Degrees of freedom We can partton the df lke ths: df total between subjects df df wthn subjects df df df between subjects U + df + df S/ U + + df df wthn subjects U S/ So now we can calculate all our df. (Often, df between subjects and SS between subjects are smply wrtten df S and SS S.) df total U U between subjects df df..3 Sums of squares df df df df S/ U S/ N a u df s df df ( df df between subjects wthn subjects total U df df ( df + df between subjects he parttonng s always exactly the same as for the df: SS SS SS total between subjects wthn subjects SS SS SS between subjects U + SS + SS S/ U + SS U ) ) ( df + SS U S/ + df wthn subjects U ) So SS total SS + SS S/ + SS U + SS U + SS U S/ We have two dfferent error terms, one for the between-subjects factor and one for the wthn-subjects factor (and ts nteracton wth the between-subjects factor), so

30 : NOV bascs 3 we can t just label them SS error. But we could rewrte the total lke ths f we wanted: SS total SS + SS error-between + SS U + SS U + SS error-wthn Now we can calculate the SS. Remember, each SS must be made up of N components, because there are N observatons. ake the example of SS : we calculate ths by summng over a means (namely x, x, x ). But each mean s based on (or, a f you prefer, contrbutes to) N/a su/a anu/a nu ndvdual scores; we therefore multply our devatons by nu to get the total SS. SS SS SS between subjects SS SS total SS S/ SS U U U S/ ( x x) u( x x) nu( x x) SS s( x x) n( x x) SS (SS S between subjects U U wthn subjects total SS SS (SS U between subjects + SS U ) ) (SS Just to make t clear how many scores each mean s based on: U + SS U ) Subject U U U 3 U 4 U 5 U 6 u 6 S datum datum datum datum datum datum x means are S datum datum datum datum datum datum based on nu S 3 datum datum datum datum datum datum 48 scores S 4 datum datum datum datum datum datum S 5 datum datum datum datum datum datum S 6 datum datum datum datum datum datum S 7 datum datum datum datum datum datum S 8 datum datum datum datum datum datum S 9 datum datum datum datum datum datum S datum datum datum datum datum datum S datum datum datum datum datum datum x means are S S datum datum datum datum datum datum based on u 6 S 3 datum datum datum datum datum datum scores S 4 datum datum datum datum datum datum S 5 datum datum datum datum datum datum S 6 datum datum datum datum datum datum 3 S 7 datum datum datum datum datum datum x means are U S 8 datum datum datum datum datum datum based on n 8 S 9 datum datum datum datum datum datum scores S datum datum datum datum datum datum S datum datum datum datum datum datum S datum datum datum datum datum datum S 3 datum datum datum datum datum datum S 4 datum datum datum datum datum datum x means are based a 3 s 4 au 4 U on an s 4 scores N su anu 44

31 : NOV bascs 3..4 NOV table Source d.f. SS F Between subjects (S): s an a SS MS /MS S/ error S/ (an ) (a ) a(n ) SS S/ Wthn subjects: (N ) (s ) an(u ) U u SS U MS U /MS U S/ U (u )(a ) SS U MS U /MS U S/ error U S/ a(u )(n ) SS U S/ otal N aun SS total where a s the number of levels of factor, etc., N s the total number of observatons ( aun), n s the number of subjects per group (per level of ), and s s the total number of subjects.. Fxed and random factors When we consder NOV factors we must dstngush fxed factors, whch contan all the levels we are nterested n (e.g. sex: male v. female) and random factors, where we have sampled some of the possble levels at random (e.g. subjects). Random factors can be thought of as those whose levels mght change; f we repeated the experment, we mght pck dfferent subjects. Sometmes the fxed/random dstncton s pretty much nherent n the factor Subjects s usually a random factor, for example. But sometmes whether a factor s fxed or random really does depend on the study. Howell (997, p. 33) uses the example of pankllers. If we are asked to study the relatve effcacy of the UK s four most popular over-the-counter pankllers, we have no choce n whch pankllers we study. If we were asked to repeat the study, we would use the same four pankllers. Pankllers would be a fxed factor. If, on the other hand, we were asked to compare several pankllers to see f one brand s as good as the next, we mght select a few pankllers randomly from the dozens on offer. In ths case, where our sample s ntended to be representatve of pankllers n general but where t s an arbtrary and non-exclusve sample, we would consder Pankller to be a random factor. Further examples are gven by Keppel (99, p. 485 and ppendx C), and by Myers & Well (995, pp. 7-). When we test effects nvolvng a random factor, we often have to test effects aganst an nteracton term. Examples are gven n the consderaton of wthn-subjects desgns (whch nvolve random factors, snce Subjects s a random factor). he determnaton of approprate error terms s dscussed later n the secton on expected mean squares (EMS) (p. 73), whch are dfferent for fxed and random factors.

32 3: Practcal analyss 3 Part 3: practcal analyss 3. Remnder: assumptons of NOV. Homogenety of varance We assume that each of our groups (condtons) has the same varance. How to check? In SPSS, Levene s test (Levene, 96) checks ths assumpton. o obtan t, choose Optons Homogenety tests from the NOV dalogue box. If Levene s test produces a sgnfcant result, the assumpton s volated there s heterogenety of varance. hs s a Potentally Bad hng. Consder transformaton of the data (see below, p. 34). You can also plot the standard devaton (and varances) versus the means of each level of a factor by choosng Optons Spread vs. level plot. Unequal ns exaggerate the consequences of heterogenety of varance a Bad hng (p. 33) (see also Myers & Well, 995, p. 5-6).. Normalty We assume that the scores for each condton are normally dstrbuted around the mean for that condton. (hs assumpton s the same as sayng that error s normally dstrbuted wthn each condton.) How to check? You can nspect the data to get an dea whether the data are normally dstrbuted n each condton. In SPSS, choose nalyze Descrptve Statstcs Explore. hs gves you get stem-and-left plots, boxplots, and so on. In the dalogue box, tck Both to get statstcs and plots; clck Plots Normalty plots wth tests. hs produces a Q Q plot a plot of each score aganst ts expected z value (the value t would have f the dstrbuton were normal calculated as the devaton of the score from the mean, dvded by the standard devaton of the scores). If ths produces a straght lne, the data are normally dstrbuted. You also get the Kolmogorov Smrnov test wth Lllefors correcton (Lllefors, 967) appled to a normal dstrbuton, and the Shapro Wlk test (Shapro & Wlk, 965) f these are sgnfcant, your data are not normally dstrbuted a Bad hng. Consder transformaton of the data (see below, p. 34). 3. Independence of observatons We also assume that the observatons are ndependent techncally, that the error components (ε) are ndependent. For any two observatons wthn an expermental treatment, we assume that knowng how one of these observatons stands relatve to the treatment (or populaton) mean tells us nothng about the other observaton. Random assgnment of subjects to groups s an mportant way of achevng ths. We must account for any non-ndependence of observatons for example, observatons that are correlated because they come from the same subjects by addng factors (e.g. Subject) that account for the non-ndependence. Introducng factors to account for non-ndependence of observatons makes the error terms ndependent agan, and we re OK. However, these desgns known as wthn-subject or repeated measures desgns have ther own assumptons, lsted below. statstcs package can t check ths assumpton for you! It depends on your experment.

33 3: Practcal analyss Remnder: assumpton of NOV wth wthn-subject factors Sphercty ny NOV nvolvng wthn-subjects factors assumes sphercty (dscussed earler). If ths assumpton s volated, ype I error rates wll be nflated (f the null hypothess s true, you wll get too many results that you wll declare sgnfcant than you should). smple plan of acton: Look at Mauchly s test of sphercty. sgnfcant Mauchly s test means that the assumpton s lkely to have been volated. When the assumpton has been volated for a partcular wthn-subjects factor, correct the df for any term nvolvng the wthn-subjects factor, and the correspondng error df, by multplyng them both by epslon (ε). Use ether the Greenhouse Gesser or the Huynh Feldt epslon. he Greenhouse Gesser one (sometmes wrtten εˆ ) s probably a bt too conservatve; the Huynh Feldt one (sometmes wrtten ε ~ ) s better (Myers & Well, 995, p. 48; Howell, 997, p. 465). SPSS reports Mauchly s test and both the G G and H F correctons whenever you run a wthn-subjects NOV usng ts menus. You never need to correct any terms that have only between-subjects factors. nd you can never volate the sphercty assumpton for a wthn-subjects factor that has only levels. 3.3 Consequences of volatng the assumptons of NOV Independence of observatons. If there are correlatons between scores that are not taken account of by the NOV model, ype I error rates can be nflated (Myers & Well, 995, p. 69, ). Normalty. he ype I error rate s not affected much by samplng from nonnormal populatons unless the samples are qute small and the departure from normalty extremely marked (Myers & Well, 995, pp. 69, ). hs s the effect of the central lmt theorem: the dstrbuton of means and ther dfferences wll tend to be normal as n ncreases, even when the dstrbuton of the parent populaton s not. hngs are pretty OK even when the dependent varable s dscretely (rather than contnuously) dstrbuted (Myers & Well, 995, p. ). However, there are nonparametrc alternatves to NOV whch may sometmes be better when the normalty assumpton s volated such as the Kruskal Walls H test (Myers & Well, 995, p. -5). For repeatedmeasures desgns, there are others: Fredman s ch-square ( χ ), the ranktransformaton F test (F r ), the Wlcoxon sgned-rank test, and Cochran s Q test (Myers & Well, 995, pp. 7-8). Homogenety of varance. If the two sample szes are equal, there s lttle dstorton to ype I error rates unless n s very small and the rato of the varances s qute large. here s generally not a problem f the rato of the largest to the smallest varance s no more than 4:, and sometmes even bgger dscrepances can be tolerated. However, when ns are unequal, there s more of a problem. Whether the ype I error rate goes up or down depends on relatonshp between the sample sze and the populaton varance: f the larger group has the larger varance, the test s conservatve, but f the smaller group has the larger varance, the test s lberal too many ype I errors and sometmes the ype I error rate gets really hgh (Myers & Well, 995, pp. 69-7, 5-). he two strateges are to use an alternatve test or to transform the data to mprove the homogenety of varance (see p. 34). he alternatve tests nclude the Welch and Brown Forsythe modfed F tests (Myers & Well, 995, pp. 6-9; Howell, 997, pp. 3-33). F

34 3: Practcal analyss 34 Sphercty. Volatons are a problem. We ve dscussed the solutons elsewhere (p. 5). 3.4 Exploratory data analyss, transformatons, and resduals Plot your data It s a very good dea to plot your data before analysng t. lthough there are formal tests for thngs lke homogenety of varance, and the other assumptons of an NOV, the tests won t descrbe the dstrbuton of your data or show you f there are outlers. See also Howell (997, chapter ). In SPSS, you can choose nalyze Descrptve Statstcs Descrptves to get smple descrptve statstcs, or nalyze Descrptve Statstcs Explore for a very comprehensve set of optons, ncludng descrptve statstcs, hstograms, stemand-leaf plots, Q Q plots (see p. 9; are the data normally dstrbuted?), boxplots (also known as box-and-whsker plots, showng outlers), and so on wth your data broken down by a factor. For example, to analyse Post scores broken down by levels of the factor, I mght do ths: Outlers Outlers can cause substantal problems wth parametrc statstcal tests (e.g. Myers & Well, 995, p. 5). If you fnd one, check that the datum has been entered correctly f not, re-enter t, or f you can t, throw t out. If t was entered correctly, you may consder removng the outler. here s a danger here we can t smply throw away data we don t lke (see Myers & Well, 995, p. 49), and maybe ths s a vald measurement, n whch case we shouldn t be chuckng t out. But sometmes t represents somethng we re not nterested n. If a reason for the outler can be establshed (data ms-entered, equpment broken, subject fell asleep, etc.) then t may be corrected or removed as approprate. We can always use nonparametrc tests, whch are much less senstve to outlers. How do we defne an outler? Wth a boxplot, one conventon s to regard ponts more than 3 box wdths (3 nterquartle range) from the box as outlers. nother s to consder ponts outsde the whskers as outlers (ukey s orgnal suggeston), but ths throws away many too many data ponts. Fnally, another approach s to defne outlers as ponts > standard devatons from the group mean.

35 3: Practcal analyss 35 nother technque related to outler removal s the use of trmmed samples. Rather than transformng your data to acheve homogenety of varance (see below), another approach to heavy-taled samples (farly flat dstrbutons wth a lot of observatons n the tal posh name platykurtc) s to trm the sample. For example, wth 4 cases per sample, a 5% trmmed sample s the sample wth the top two and the bottom two observatons removed (5% removed from each tal). When comparng several groups, as n NOV, each sample would be trmmed by the same percentage. However, there s a specal technque requred for the NOV: the MS error should be based on the varance of the correspondng Wnsorzed sample one n whch the values you removed are replaced by copes of the next-most-extreme datum (Howell, 997, p. 39). o my knowledge, ths sn t a very common technque ransformatons ransformatons can be used () to transform skewed dstrbutons nto somethng closer to a normal dstrbuton; () to reduce heterogenety of varance; (3) to remedy non-addtvty n wthn-subject desgns. transformaton that acheves one purpose well may not be equally suted to other purposes, although transformatons that equate varances do tend to gve more normally dstrbuted scores (Myers & Well, 995, p. 9). We ll focus on transformatons desgned to acheve homogeneous varances. Such transformatons can be derved f the relatonshp between µ (the group mean) and σ j (the group varance) s known. Here are some examples, and a general rule (Myers & Well, 995, pp. 9-; Howell, 997, pp ): If the data are proportons, such as percent correct, the scores n the populaton are bnomally dstrbuted; the varance can be wrtten as σ j k µ j ( µ j ) where k s a constant. he approprate transformaton s the arcsne transformaton: Y arcsn Y. For example, f a datum (Y value) s.5, the transformed value Y s arcsn.5 45 ; you would use the value 45 for analyss. (Or you could do t n radans; t doesn t matter: π radans 8.) Your data should be n the range ; f your data are percentages (97%), analyse them as proportons (.97). he arcsne transformaton stretches out both tals (numbers near to and ) relatve to the mddle (numbers near to.5). In general Plot log( ˆ σ ), the log of the standard devaton of each group, j aganst log( Y j ), the log of the mean of each group. If ths relaton s approxmately a straght lne, fnd ts slope. he approprate transformaton would be ( slope) Y Y. If slope, take the log of each score nstead of rasng t to a power. If the data are markedly skewed, or the standard devaton s proportonal to the mean, you wll often fnd that the slope s and a log transformaton s approprate. Reacton tmes may be amenable to ths transformaton. It s also applcable when the scores themselves are standard devatons. You can use any base for the logarthm ( s smple, but you could use e or anythng else). You can t fnd logarthms of zero or of negatve numbers, so f your data are negatve t s permssble to add a constant before takng the log: Y log( Y + k). If you have near-zero values, use Y log( Y +) rather than Y log(y ). Varance s proportonal to the mean. Consder takng the square root of each.5 datum: Y Y Y. he square-root transformaton compresses the upper tal of the dstrbuton. If your scores are small (e.g. <), you may fnd that Y Y +.5 or even Y Y + Y + works better for equatng varances. j he recprocal transformaton, Y Y, s also useful f the data are Y postvely skewed (a few very large values at the upper end of the dstrbuton).

36 3: Practcal analyss 36 Indeed, t may often make a lot of sense to use t partcularly n the example of transformng reacton tmes or latences to reacton speeds. Don t apply a transformaton unless you need to, or t makes theoretcal sense. major problem wth transformatons s nterpretng subsequent analyses. Sometmes transformatons make excellent sense, such as n the tme-to-speed transformaton. Or you mght have a theoretcal reason to thnk that Y s a power functon of some varable : Y a b. hen analysng log(y) and log() would make sense, because ther relatonshp would then be lnear and NOV technques are bult around lnear relatonshps. nd f there s a clear relatonshp between group means and standard devatons, the approprate transformaton wll tend to gve a more powerful statstcal test. But sometmes transformatons that mprove heterogenety of varance don t help you theoretcally you may dscover that group makes more lever presses +.5 than group, and then have to nterpret that n terms of the real world of lever presses. nd f relatve dstances between means are of nterest, problems can crop up: Myers & Well (995, p. ) gve the example of comparng two teachng methods for hgh- and low-ablty subjects. Even f the dfference between the two teachng methods were the same for the hgh- and low-ablty groups on the orgnal data scale, the transformaton mght well produce a dfferent result on the new scale; conversely one method mght have more of an advantage for lowablty subjects on the orgnal data scale, but agan the results mght be qute dfferent on the transformed scale. If you transform your data, t s only far that you plot the transformed data n your fgures, snce that s what you analysed (especally f your fgures show ndces of varablty, such as error bars, and/or make some clams as to sgnfcant dfferences between condtons). You may also choose to report the group means converted back to real unts. But be aware that ths can be a lttle msleadng. For example, f a group of sx rats makes (6, 8, 38, 96, 55, 5) lever presses (mean 39.67) and you analyse the square-root transformed data (4, 5.9, 6.6, 9.8, 7.4,.4), you wll fnd that the mean of the transformed data s 5.8. But 5.8 s the square root of so convertng the mean of the transformed data back to the orgnal scale (by applyng the reverse of the transformaton) doesn t gve you the untransformed mean. If you do need to transform, t s perfectly permssble to shop around, tryng out several transformatons untl you fnd one that does a good job of reducng heterogenety of varance (Howell, 997, p. 39). But t s not permssble to shop around untl you fnd a transformaton that gves you a sgnfcant result! You are tryng to optmze the data so that the NOV s vald you are not tryng to optmze the NOV result Plot your resduals You should always plot the dstrbuton of the resduals from any analyss y yˆ, or what s left over after you ve predcted your dependent varable. re the resduals normally dstrbuted? If not, you should do somethng. Remember that an assumpton of NOV was that the error varance was normally dstrbuted (p. 9). If your resduals are not normally dstrbuted, your p values don t mean what you hope. You can transform your dependent varable (p. 35) add another predctor (p. 5) Why mght non-normal resduals suggest that addng another predctor would be a good dea? Well, normal (Gaussan) resduals are what you d expect f error was n fact made up of a load of ndependent thngs of roughly equal mportance (e.g. measurement error, room temperature fluctuatons, background nose varatons, tme of day varatons, subject alertness ); remember that the central lmt theorem tells us that the dstrbuton of a sum of a set of dentcally dstrbuted random varables approaches the normal dstrbuton. For a gven standard devaton, the normal dstrbuton has the maxmum uncertanty (n nformaton-theoretc terms, conveys

37 3: Practcal analyss 37 the maxmum nformaton). So normally dstrbuted resduals help to suggest that there s no other major non-normally-dstrbuted predctor you should add n. In SPSS, you can choose Optons Resdual plot for any NOV: he three types of plot you get are: observed Y values (y) aganst predcted Y values ( ŷ ) there ll be a correlaton f your model s any good; observed Y values (y) aganst resduals ( y yˆ ) there ll be a correlaton, snce the two aren t ndependent (snce y yˆ + resdual ); predcted Y values ( ŷ ) aganst resduals ( y yˆ ) the two should be ndependent. hese plots are not terrbly helpful. SPSS uses standardzed resduals n these plots. ( standardzed resdual s a resdual dvded by an estmate of ts standard devaton.) In SPSS s output there are two copes of each plot one arranged wth one varable on the x axs and the other on the y axs, and the other flpped (mrrored around the x y lne). Here I ve faked some data where Y depends on two other varables and (both contnuous, for ths example,.e. covarates, but they could equally be factors), whch happen themselves to correlate. hs s what SPSS s resdual plots look lke: Not a very useful plot. Here, only s used as a predctor varable (model: Y constant + b ). here s a correlaton between observed and predcted Y values, meanng that our model s dong some predctng. he man thng to look at s the standardzed resdual versus predcted Y plot. Does that look lke a random scatterplot? No. hat suggests there s a further relatonshp between and Y that s not captured by a lnear relatonshp. Not a very useful plot. When we mprove our model by ncludng as a predctor (model: Y constant + b + b ), the standardzed resdual versus predcted Y plot looks more lke a scatterplot. Whatever part of Y s not predcted (the resdual) now appears to be uncorrelated wth the predcted part, whch s good our model s dong a better job.

38 3: Practcal analyss 38 However, ths resdual analyss s not deal t doesn t gve us a very clear ndcaton of whether the resduals are normally dstrbuted. What you can also do s to save the resduals from any NOV. Choose the Save dalogue box and choose the approprate opton, such as the unstandardzed (raw) resduals: When the NOV s run, new column(s) are created wth the resduals n. (If you run an NOV wth wthn-subjects factors n the usual way usng nalyze General Lnear Model Repeated Measures, wth one subject per row, you get one resdual column for every data column n your nput. hs dalogue box can also be used to save the predcted values. In syntax, you can specfy /SVE PRED RESID to get both.) Once you ve obtaned your resduals, you can check them for normalty: nalyze Descrptve Statstcs Explore; tck Both to get statstcs and plots; clck Plots Normalty plots wth tests. hs produces a Q Q plot (f ths produces a straght lne, the data are normally dstrbuted) and the Kolmogorov Smrnov and Shapro Wlk tests (f sgnfcant, your resduals are non-normal); see p. 3 for explanaton of these. o examne the resdual dstrbuton for several groups separately, enter the groupng factor nto the Factor Lst: s an example, I created some data n whch the data were created from the sum of contrbutons from factor (two levels), factor B (two levels), and random nose. If we analyse wth an NOV that only has factor n t, savng and plottng the resduals as descrbed above, we get a Q Q plot of the resduals that looks lke the left-hand sde of the fgure below clearly not normal. hs mght suggest to us that we need to nclude another predctor. If we now nclude factor B n the NOV and replot the new resduals, we get the rght-hand verson, n whch the resduals are normally dstrbuted. hat meets the assumptons of the NOV, and we can feel a bt happer that we haven t left anythng out of the analyss.

39 3: Practcal analyss 39 Dependent varable was caused by factors and B, but only factor was entered nto the analyss. Resduals are not normally dstrbuted. Factors and B are both entered nto the analyss. Resduals are normally dstrbuted. Fnally, resduals that are outlers (large) for a group reflect data ponts that are outlers, so resdual plots are another way to spot outlers (see also Myers & Well, 995, p. 44). 3.5 Further analyss: man effects, nteractons, post hoc tests, smple effects Plot your data. Wth any reasonably complex experment, you can t nterpret the data untl you ve plotted t Smple effects remnder of what man effects, nteractons, and smple effects refer to (see p. ). It s easest to vsualze wth a two-factor NOV. man effect of means that the means (,, a ) are not all equal. Smlarly for a man effect of B. n nteracton means that the effects of are not the same at all levels of B (equvalently, that the effects of B are not the same at all levels of ). Suppose we have a between-subjects factor (group: control, drugged) and a wthn-subjects factor U (task condton: U hot room, U cold room). We analyse our data and fnd an nteracton. We may want to ask questons about smple effects: was there an effect of drug on performance n a hot room (smple effect of at U, also wrtten /U )? Was there an effect of drug on performance n a cold room (/U )? Was there an effect of room temperature on the control group (smple effect of U at, wrtten U/ )? On the drugged group (U/ )? here are two ways of runnng smple effects analyss. he frst and smplest s only to analyse the data that s relevant. So to ask about /U, we d only analyse the U (cold room) data, ths tme wth a one-factor NOV we ve dropped out the U factor. Smlarly, f we had started wth a three-way NOV ( B C), we would have run a two-way NOV to establsh effects such as /C, B/C, and

40 3: Practcal analyss 4 B/C (the last one s sometmes called a smple nteracton ). hs s easy and generally recommended (Myers & Well, 995, p. 34). It can be appled to betweenand wthn-subjects factors. It s possble to obtan a more powerful test of the smple effects. hs nvolves calculatng the MS for the smple effect just as before, but testng t not aganst the MS error for the sub-analyss (the one-factor NOV n our U example), but aganst the MS error for the orgnal, full NOV known as the pooled error term. If you want to do ths, you have to do t by hand: F df-factor/df-pooled-error MS factor/ms pooled error. Smlarly, you can use the pooled error term for multple comparsons between treatment means, f your factors have > levels. However, you shouldn t do ths for wthn-subjects smple effects, as correctons for volatons of the sphercty assumpton are nadequate (Howell, 997, p. 468). Furthermore, f there s some heterogenety of varance, there can also be substantal problems (Myers & Well, 995, pp , 34-35). So t s smplest and probably best to gnore ths technque just run a smpler NOV on a subset of your data Determnng the effects of a factor wth > levels If you dscover that you have a sgnfcant man effect of a factor wth levels, you know what t means: µ µ. So you only have to look at the means to work out f µ > µ or µ < µ. But f you have fve levels, a sgnfcant man effect merely means that the null hypothess H µ : µ µ 3 µ 4 µ 5 has been rejected. But what does that mean? here are all sorts of alternatves: µ µ µ µ µ... µ µ µ µ µ µ µ µ µ µ hs s where we would use post hoc comparsons among treatment means. here are two types of post hoc tests. One knd tests all possble parwse comparsons. For 5 levels, we can compare µ and µ, µ and µ 3 up to µ 4 and µ 5. For 5 comparsons, there are C possble parwse comparsons. he other type of test groups 5 the means nto homogeneous subsets, and tells you somethng lke µ, µ 3, and µ 4 fall nto one subset [are all the same] µ and µ 5 fall nto another subset [are the same] the subsets dffer from each other. But we must be careful Post-hoc tests: the problem he upshot: f you collect your data, look at t, and wonder are those two ponts sgnfcantly dfferent?, you need to use a post hoc test because your eye has already selected partcular ponts to compare, whch nfluences the lkelhood of fndng a sgnfcant dfference It s beautfully put by Sometmes we fnd effects n an experment that were not expected. Even though n most cases a creatve expermenter wll be able to explan almost any pattern of means, t would not be approprate to analyse and evaluate that pattern as f one had predcted t all along. he problem here s one of captalzng on chance when performng multple tests post hoc, that s, wthout a pror hypotheses. o llustrate ths pont, let us consder the followng experment. Imagne we were to wrte down a number between and on peces of paper. We then put all of those peces nto a hat and draw samples (of peces of paper) of 5 observatons each, and compute the means (from the numbers wrtten on the peces of paper) for each group. How lkely do you

41 3: Practcal analyss 4 thnk t s that we wll fnd two sample means that are sgnfcantly dfferent from each other? It s very lkely! Selectng the extreme means obtaned from samples s very dfferent from takng only samples from the hat n the frst place, whch s what the test va the contrast analyss [known as an a pror test or planned contrast] mples. Wthout gong nto further detal, there are several so-called post hoc tests that are explctly based on the frst scenaro (takng the extremes from samples), that s, they are based on the assumpton that we have chosen for our comparson the most extreme (dfferent) means out of k total means n the desgn. hose tests apply correctons that are desgned to offset the advantage of post hoc selecton of the most extreme comparsons. Whenever one fnds unexpected results n an experment one should use those post hoc procedures to test ther statstcal sgnfcance. In general, we can defne the per-test ype I error rate (α, also called the error rate per contrast) and the famlywse ype I error rate (α FW ), the probablty of makng at least one ype I error rate when performng a famly of multple comparsons he specal case of three groups: multple t tests are OK here s a specal case n whch multple uncorrected t tests are OK when you have three groups (Howell, 997, p. 37) and you have a sgnfcant man effect for your factor. hs sn t wdely apprecated. he NOV F test assesses the null hypothess: H : µ µ µ 3 If we have a sgnfcant man effect, then we ve already rejected ths null hypothess. hat means that one of the followng must be true: µ µ µ µ µ µ 3 3 µ µ µ If we run a complete set of (3) uncorrected t tests, we wll choose one of these three conclusons. But no concluson nvolves us judgng that there are more than two nequaltes (sgnfcant dfferences between ndvdual means). nd we know that there s at least one nequalty, snce we ve rejected the overall null hypothess. So we can make at most one ype I error. herefore, the probablty of makng that ype I error (choosng µ µ µ 3 when one of the other two s correct) s the plan α for each test, and no further correcton s necessary Otherwse a varety of post hoc tests For between-subjects factors, SPSS provdes too many optons n ts Post Hoc box: Equal varances assumed * LSD (Fsher s least sgnfcant dfference). Uncorrected multple t tests, except that the test s only performed when an NOV has rejected the overall null hypothess,.e. shown that somethng s gong on (Myers & Well, 995, p. 88; Howell, 997, p ). α FW ( α) k when k ndependent tests are performed, and α FW ( α) k when the tests are not ndependent (Myers & Well, 995, p. 77). Only sutable for 3 levels of a factor n whch case t s the most powerful test but don t use t otherwse. * Bonferron t procedure. Occasonally called the Dunn procedure. Makes use of the Bonferron nequalty: α FW kα, or more generally, α FW α where α s the probablty of a ype I error for the th contrast (Myers & Well, 995, p. 79). hs s derved from the proper verson, α FW ( α) k, by notng that for small values of α (and.5 s small), ( α) k kα. herefore, each contrast s tested at α α FW /k. For example, f four tests are to be performed (k 4) and we desre α FW.5, then each test s per- 3

42 3: Practcal analyss 4 formed at α.5. Quck to do. ddtonally, we don t have to have all the αs equal. If we re much more nterested n one of our four comparsons, we could allocate α.3 to t, and α.67 to each of the others (Myers & Well, 995, p. 8). Can be used for testng k planned contrasts. * Šdák (or Dunn Šdák, or Sdak). Snce α FW ( α) k, ths procedure solves for α [α ( α FW ) / k ] so as to get α FW to be what you want (typcally.5). Lke the Bonferron correcton, but more accurate (.e. t s correct). See also Howell (997, p. 364). * Scheffé. See Myers & Well (995, p. 83) and Howell (997, p. 379). Controls α FW aganst all possble lnear contrasts (see p. 75), not just parwse contrasts. Consequently, very conservatve. REGWF (Ryan Enot Gabrel Welsch F test). No dea; somehow smlar to the REGWQ. REGWQ (Ryan Enot Gabrel Welsch) range test. compromse between the Newman Keuls (lberal) and ukey HSD (conservatve) (Howell, 997, p. 378). hs test does not requre the overall F for groups to be sgnfcant as t controls the famlywse error rate ndependently and test dfferent hypotheses from the overall NOV, wth dfferent power (Howell, 997, p. 35). Recommended (Howell, 997, p. 378) except for unequal cell szes (SPSS help). SNK (Student Newman Keuls, a.k.a. Newman Keuls). Not often used. Poor control of α FW (Myers & Well, 995, p. 88; Howell, 997, p ) unless there are only three means to be compared, n whch case t s OK. * ukey HSD (honestly sgnfcant dfference). Smlar to the Newman Keuls test except that t fxes α FW properly (Howell, 997, p. 377). ukey s-b. ukey s test as a range test? Not sure. Duncan s multple range test. Not often used. Poor control of α FW (Myers & Well, 995, p. 88). * Hochberg s G. Less powerful varant of ukey s; see SPSS help. * Gabrel s parwse comparsons test. more powerful verson of Hochberg s G when cell szes are unequal; may become lberal when the cell szes vary (SPSS help). Waller Duncan t test. Uses a Bayesan approach. Uses the harmonc mean of the sample sze when the sample szes are unequal (SPSS help). hat doesn t tell you much. Dunnett s test for comparng treatment groups wth a control group. Sometmes we are nterested n comparng each of the a treatment groups to a control group, and less nterested n comparng them to each other. For ths case, snce no two of the set of contrasts are orthogonal, the Bonferron approach would be conservatve (see pp. 76, 77). hs test does not requre the overall F for groups to be sgnfcant as t controls the famlywse error rate ndependently and test dfferent hypotheses from the overall NOV, wth dfferent power (Howell, 997, p. 35). Equal varances not assumed * amhane s. Conservatve parwse comparsons, based on a t test (SPSS help). * Dunnett s 3. No dea. Range test. * Games Howell. Sometmes lberal (SPSS help). * Dunnett s C. No dea. Range test. * Parwse comparson test. Homogeneous subset test. range test s one based on a Studentzed range statstc q, a modfcaton of the t statstc (Howell, 997, p ). he mportant tests are summarzed by Myers & Well (995, p. 86). You can do most of what you want wth the Sdak correcton for parwse comparsons, Dunnett s test when you re comparng treatment groups to a control group, and perhaps the REGWQ as a homogeneous subset test.

43 3: Practcal analyss 43 Pck your post hoc tests n advance: t s not vald to run all sorts of tests and then pck the most sgnfcant. I suggest uncorrected t tests (Fsher s LSD) for three groups, the Sdak correcton for >3 groups, and Dunnett s test for comparng treatment groups to a control group f you re more nterested n that comparson than n dfferences between the treatment groups. If you would lke a homogeneous subset test, then the ukey HSD test s popular but the REGWQ s perhaps better. ukey s HSD, REGWQ, Dunnett s, and the Sdak test don t even requre the overall F test from the NOV to be sgnfcant (Howell, 997, pp. 35, 364, 377), although the 3-group Fsher LSD does. SPSS doesn t let you perform many of those tests on wthn-subjects factors, for good reason many of them aren t vald (see Howell, 997, p. 47). However, you can choose Dsplay means for n the Optons box and tck Compare man effects wth ether no correcton for multple comparsons (LSD) only vald f the factor has only 3 levels or a Bonferron or Sdak correcton. he faclty to compare means wth a Sdak correcton and to run further NOVs on subsets of your data s enough to analyse any between/wthn-subjects desgn, unless you also want to run specfc contrasts (see p. 75). 3.6 Drawng pctures: error bars for dfferent comparsons Much of ths s reproduced from except the secton on NOV Error bars for t tests: between-subjects comparsons In bref: he standard error of the mean (SEM) conveys the precson wth whch the populaton mean was estmated. (It depends on the SD and the sample sze, n.) Every mean (e.g. every group) has ts own SEM. It s approprate to use t as an error bar for between-subjects comparsons. It s the most common error bar you see publshed. he conventon s to plot ± SEM that s, your error bars extend above the mean by SEM and below the mean by SEM. lternatves nclude the standard devaton (SD), whch measures the varablty of observatons about ther mean and s ndependent of n, and confdence ntervals (CI); these show the range of values wthn whch the populaton mean probably les, and depend on the SD and n. he SEM s frequently used as an ndex of varaton when people publsh data. hey may quote a measurement of 5.4 ±. g, or dsplay a datum on a graph wth a value of 5.4 unts and error bars that are each. unts long. hese varaton ndces could be one of several thngs mean ± SD, mean ± 95% CI, mean ± SEM he paper should state somewhere whch one s beng used, but usually t s the SEM. Why? Frst, t s smaller than the SD, so t conveys an mpresson of mproved precson (remember that accuracy s how close a measurement s to a true value and precson s how well t s defned; thus,.53 8 m s s a more precse but far less accurate measurement of the speed of lght than 3. 8 m s ). In fact, usng the SEM s perfectly far and correct: the precson of an estmator s generally measured by the standard error of ts samplng dstrbuton (Wner, 97, p. 7). Secondly more mportantly f the SEM error bars of two groups overlap, t s very unlkely that the two groups are sgnfcantly dfferent. (hs s explaned somewhat n the fgure.) he opposte sn t necessarly true, though just because two sets of error bars don t overlap doesn t mean they are sgnfcantly dfferent (they have to not overlap by a certan amount, and that depends on the sample sze, and so on).

44 3: Practcal analyss Error bars for t tests: wthn-subjects comparsons In bref: SEMs are msleadng for wthn-subjects comparsons. Use the standard error of the dfference (SED) for the relevant comparson nstead. SEDs are also approprate for between-subjects comparsons. SEDs are not attached to a partcular mean, so the conventon s to plot a free-floatng error bar that s SED long, and label t. (he reader can use t as a mental ruler to make comparsons between the relevant means.) For wthn-subjects comparsons, SEMs calculated for each condton are hghly msleadng (see fgure below). For ths comparson ndeed, for any comparson the SED s an approprate ndex of comparson, because that s what the t test s based on (t dfference between means / SED). So f the dfference between two means s greater than twce the SED, t >. nd for a healthy n, t > s sgnfcant at the two-taled α.5 level (have a quck glance at your tables of crtcal values of t). he SED s therefore a very good ndex of varaton that can be used to make vsual comparsons drectly, partcularly f you draw error bars that are SED long f the means to be compared are further apart than the length of ths bar, there s a good chance the dfference s sgnfcant. However, t s a bt more work to calculate the SED, whch s why you don t see t very often.

45 3: Practcal analyss 45 If you want to work out an SED, just choose the approprate t test and calculate the denomnator of the t test. For between-group comparsons where the group SEMs are SEM and SEM, you ll see that SED (SEM + SEM ). o summarze, for wthn-subject changes:. he mean wthn-subject change equals the dfference of the group means.. he varance of the wthn-subject change may dffer greatly from the varance of any one condton (group). 3. Present wthn-subject changes when the baselne vares a lot, or you want to show varance of the wthn-subject measure. 4. Present group means when the baselne matters Error bars for an NOV In bref: SEDs are always approprate. MS Use SED error f all groups are the same sze. n MS error Use error MS SED + f there are two groups beng compared and n n they are of unequal sze. hs means there may be a dfferent SED for each comparson of two means. In SPSS, you can obtan these usng parwse comparsons for nteracton effects (see p. 6). However, most people want to plot a sngle SED. For ths purpose, f there are > groups of unequal sze, I thnk the most approprate one to use s SED error MS where n h s the harmonc mean of n h the group szes (see p. 3 and also p. 7). For two groups, that reduces to the formula above. In an NOV wth several factors, there may be are several dfferent SEDs, correspondng to several dfferent MS error terms. lthough you can plot the most relevant one(s), the most common conventon s to plot the SED from the hghest nteracton shown n your graph (so f your graph shows factors and B, you would plot the SED from the B nteracton). he conventon s to plot a free-floatng error bar that s SED long, and label t as such. t test s drectly related to an NOV: ths general formula:, k tk F and t k F, k. nd a t test has quantty t standard error of that quantty For example, a one-sample t test has the formula and a two-sample t test has the formula mean test value t standard error of the mean (SEM) mean mean t standard error of the dfference between means (SED) For a sngle sample, the SEM (the standard devaton of all sample means of a gven sample sze n) s

46 3: Practcal analyss 46 σ σ σ x wth correspondng varance σ x n n For two ndependent samples, the SED (the standard devaton of the set of dfferences between pars of sample means) s σ x x σ n SEM σ + n + SEM σ σ wth correspondng varance σ x x + n n In an NOV wth one factor and two groups, snce we assume homogenety of varance, our best estmate of the varances of two groups σ and σ s a weghted ( pooled ) average of the two group varances (Myers & Well, 995, pp ): df ˆ σ df ˆ σ SS + SS SS ˆ error pooled + df + df df + df df + df dferror σ MS error So MS error s an approxmaton to σ. In fact, we knew that already (see pp. 9, ). In general, the standard error of an estmate (Myers & Well, 995, pp. 5-), σˆ e, whch estmates the standard devaton of the error varablty ε, s SS error MS ˆ σ e error, or ˆ σ e MSerror dferror and therefore for a comparson between two groups, the SED s gven by σ MS error MS error x x + n n For equal group szes, wth n observatons per group, ths smplfes: σ x x MS error n SPSS provdes SEM and SED estmates for any gven comparson when you choose Optons Estmated Margnal Means for a factor or set of factors, or f you use the /EMMENS BLES(factor) syntax (see llustrated example on p. 56 ). But note that there s no one SED approprate for all comparsons. If you have > groups, and ther szes are unequal, the SED for comparng group to group may be dfferent for that for comparng group to group 3. nd n a mult-factor NOV, the SED for comparsons nvolvng factor wll dffer from the SED for comparsons between B subgroups. s we saw above, the conventon s to plot the SED from the hghest-order nteracton. 3.7 Summarzng your methods: a gude for thess-wrtng and publcaton he followng s an extract from my PhD thess methods (whch proved perfectly publshable: e.g. Cardnal et al., 3), wth comments n square brackets. Data were analysed wth [computer package, e.g. SPSS], usng prncples based on Howell (997) [or other approprate textbook]. Graphcal output was provded by [computer package, e.g. Excel 97 and Sgmaplot ]. ll graphs show group means and error bars are ± SEM unless otherwse stated. ransformatons. Skewed data, whch volate the dstrbuton requrement of analyss of varance, were subjected to approprate transformatons (Howell, 997, secton.9). Count data ([e.g.] lever presses and locomotor actvty counts), for whch varance

47 3: Practcal analyss 47 ncreases wth the mean, were subjected to a square-root transformaton. Homogenety of varance was verfed usng Levene s test. nalyss of varance. Behavoural data were subjected to analyss of varance (NOV) usng a general lnear model, usng SPSS s ype III sum-of-squares method. Mssng values were not estmated but excluded from analyss [ subjects for whom some data were mssng were omtted entrely; SPSS s default]. ll tests of sgnfcance were performed at α.5; full factoral models were used unless otherwse stated. NOV models are descrbed usng a form of Keppel s (98) notaton; that s, dependent varable (B 5 S) where s a between-subjects factor wth two levels and B s a wthn-subjects factor wth fve levels; S denotes subjects. For repeated measures analyses, Mauchly s (94) test of sphercty of the covarance matrx was appled and the degrees of freedom corrected to more conservatve values usng the Huynh Feldt epslon ε ~ (Huynh & Feldt, 97) for any terms nvolvng factors n whch the sphercty assumpton was volated. [Better approach, now I ve learned more (see p. 5): Degrees of freedom for terms nvolvng wthn-subjects factors were corrected usng the Greenhouse Gesser epslon εˆ (Greenhouse & Gesser, 959) where the sphercty assumpton was volated substantally ( εˆ <.75) or the Huynh Feldt epslon ε ~ (Huynh & Feldt, 97) when the sphercty assumpton was volated mnmally ( εˆ.75).] [Pretty good and smple approach (see p. 5): Degrees of freedom for terms nvolvng wthn-subjects factors were corrected usng the Huynh Feldt epslon ε ~ (Huynh & Feldt, 97).] hus, the same analyss wth and wthout sphercty correcton would be reported as follows: Uncorrected: F,6.47, p.3 Corrected: F 4.83, , εˆ.483, p.84 [Journals used to qubble about non-nteger df because they were gnorant; such qubblng s less common these days. If you quote non-nteger df, though, state the correcton factor so people can work out the orgnal df.] Post-hoc tests. Sgnfcant man effects of nterest were nvestgated usng parwse comparsons wth a Sdak correcton. hs s based on the observaton that α famlywse ( α each test ) n when n tests are performed; the correcton was made such that α famlywse.5. Where man effects were found for between-subjects factors wth three or more levels, post hoc comparsons were performed wth the REGWQ range test (famlywse α.5), or Dunnett s test n stuatons where several expermental treatments were compared wth a sngle control group. hese tests do not requre the overall F for groups to be sgnfcant as they control the famlywse error rate ndependently and test dfferent hypotheses from the overall NOV, wth dfferent power (Howell, 997, p. 35). [I was clearly ramblng a bt here!] Where sgnfcant nteractons were found followng factoral analyss of varance, smple effects of a pror nterest were calculated by one-way NOV and tested by hand aganst the pooled error term (F MS factor /MS pooled error ; crtcal values of F based on df factor and df pooled error ). Multple comparsons for smple effects were performed as descrbed above but usng the pooled error term. Where sgnfcant nteractons were found followng repeated measures analyss, a pooled error term was used to test between-subjects smple effects of a pror nterest, but separate error terms (.e. plan one-way NOV) were used for wthn-subjects factors as sphercty correctons are nadequate f a pooled error term s used (Howell, 997, p. 468). [hese days I wouldn t use the pooled error term at all, and would just use plan one-way NOV; see Myers & Well (995, pp. 34-5).] dd any other specal procedures you used! For example, you mght add ths: dependent varables were checked for normalty by nspecton of Q Q plots (whch plot scores aganst ther expected values under a normal dstrbuton) and usng the Kolmogorov Smrnov test wth Lllefors correcton (Lllefors, 967) [and/or] Shapro Wlks test (Shapro & Wlk, 965).

48 4: Expermental desgn 48 Part 4: ptfalls and common ssues n expermental desgn 4. me n wthn-subjects (repeated measures) desgns here s nothng nherently specal about tme as a wthn-subjects factor you only get that mpresson from books that dstngush repeated measures (mplyng tme) from desgns that are logcally equvalent to wthn-subjects desgns, e.g. n agrculture. s always, the sphercty assumpton should be checked; tme also represents a contnuous factor, so that trend analyss (p. 8) nvolvng t may be meanngful. nd counterbalancng s often vtal to avod order effects. hat s about t. 4. nalyss of pre-test versus post-test data very common desgn s as follows. Subjects are randomly assgned to groups (levels of ), such as and. hey are tested; the treatment ( or ) s appled; they are retested. Snce subjects were randomly assgned to groups, there are no systematc group dfferences n the pre-test scores. he post-test scores reflect the effects of the treatment. here are several ways to analyse ths sort of desgn (Myers & Well, 995, pp , p. 454; also Howell, 997, p. 66-7):. nalyss of covarance (p. 38). When ts assumptons are met, ths s the most powerful. Bascally, ths assumes that the post-test scores are lnear functons of the pre-test scores. (It s often also assumed that the slopes of these functons are the same at each level of, but see p. 38). he analyss takes advantage of ths relatonshp, reducng error varablty n the post-test scores by removng varablty accounted for by the pre-test scores.. nalyss of dfference scores. For each subject, the pre-test score s subtracted from the post-test scores; a one-factor NOV (usng factor ) s then performed on these scores. he approach assumes that the effect of each treatment s to add a constant to the pretest score. Because ths model s less lkely to be true than that assumed by the analyss of covarance, t wll generally be a less powerful test. 3. nalyss of post-test scores only. hs approach s vald, but gnores nformaton (the pre-test scores) that could help to reduce error varance, and therefore wll be less powerful than those above. 4. nalyss usng a mxed desgn: as a between-subjects factor, P as pre-test versus post-test. hs s frequently done. However, t wll be a very conservatve test of the man effect of t doesn t take account of the nformaton that the pre-test scores cannot be affected by. better test for would be that gven by the P nteracton, whch s dentcal to that obtaned by performng a one-way NOV on the dfference scores and as we saw above, an analyss of covarance s generally better. If the subjects haven t been randomly assgned to levels of, then the analyss (or the nterpretaton) can be much more dffcult. If you don t understand the prncples of multple regresson wth correlated varables, don t go there just analyse the post-test scores (Myers & Well, 995, p. 36). Or understand the trcky stuff (Parts 6 & 7) 4.3 Observng subjects repeatedly to ncrease power n example: low-n experment where subjects are precous. he dependent varable s change n blood pressure n response to a condtoned stmulus (CS). wo CSs are used: one sgnallng a hgh-ncentve, tasty food, and the other sgnallng a low-

49 4: Expermental desgn 49 ncentve, less-preferred food. Furthermore, subjects are tested followng admnstraton of a drug or placebo. he response of each subject to each CS s observed 6 tmes, to reduce the measurement error or ncrease power somehow (the expermenter feels that more observatons should gve more power, but can t verbalze exactly how). Presentaton order s sutably counterbalanced. he orgnal data layout s shown below. How should ths be analysed to maxmze power? Subject (S) Incentve () Drug (B) Observaton (C) Dependent varable Low Placebo datum Low Placebo datum Low Placebo 3 datum Low Placebo 4 datum Low Placebo 5 datum Low Placebo 6 datum Hgh Placebo datum Hgh Placebo datum Hgh Placebo 3 datum Low Drug datum Low Drug datum Hgh Drug datum Hgh Drug datum Low Placebo datum Low Placebo datum 3 subjects levels levels 6 observatons per level 7 observatons We have these possble factors, even f we do not use them all: subject (S), whch s a random factor (see p. 3); ncentve (), whch s a fxed factor; drug (B), whch s a fxed factor; perhaps observaton (C), whch we ll consder to be a fxed factor. We seek to test the effects of (does the response to a hgh ncentve CS dffer from that to a low ncentve CS?), B (does the response of a drugged subject dffer from that of a non-drugged subject?), and B (does the effect of ncentve alter as a result of the drug?) wth maxmum power. Consder the optons:. and B are used as factors. No subject term s entered, so t s effectvely a between-subjects desgn. Wrong. hs s pseudoreplcaton; we are pretendng that we have 8 ndependent observatons per B combnaton. In fact, we have 3 subjects per B combnaton wth 6 observatons per subject and those observatons are lkely to be correlated, because they come from the same subject. We must take account of ths fact. Indeed, to do so s lkely to mprove our power, by accountng for dfferences between subjects. Remember the key assumpton of NOV: that the error components (ε) are ndependent.., B, and S are used as factors. hs s a desgn wth two wthn-subjects factors. here are 6 observatons per cell (per BS combnaton). We are assumng that there s no correlaton between observatons beyond that attrbutable to them comng from the same subject//b combnaton. Somewhat related to wthn-subjects NCOV (Bland & ltman, 995a) (see p. 5). Vald. 3., B, C, and S are used as factors. hs s a desgn wth three wthn-subjects factors. We have a factor of observaton number. hs may mean very lttle to us (we wouldn t be nterested n effects attrbutable to t), but we nclude t n the hope that t removes some varablty, reducng our error varablty and mprovng our power. We have one observaton per cell. Vald. 4. We take the mean of the 6 observatons per subject per B combnaton. We now have N observatons rather than N 7, but we expect the means to

50 4: Expermental desgn 5 be more accurate estmators of the true effect on each subject. We analyse them wth, B, and S as factors. We have one observaton per cell. Vald. So of desgns 4, whch s optmal? hey ll all gve dentcal answers! Observng a subject more than once n the same condton smply mproves the precson wth whch the subject s measured n that condton. You can use that more precse mean drectly (desgn 4), or let the NOV maths work out the means for each condton (desgns and 3). he varablty that you reduce by measurng the subject repeatedly s the varablty about the mean for that subject n that condton, not the varablty assocated wth measurng the effect of factors or B. ry t and see. See also the CRD wth subsamplng and RCB wth subsamplng agrcultural desgns (p. 86 ). 4.4 It s sgnfcant n ths subject Words to strke fear nto your heart. he scenaro runs lke ths. n expermenter usng precous subjects assgns them to sham or leson groups. Each s measured repeatedly for ts response to a stmulus pared wth food (CS + ) and to a neutral stmulus (CS ). Let s say we have ten CS + observatons and ten CS observatons per subject. It s, of course, completely vald to perform a t test or equvalent NOV to ask whether the effect of CS (CS + versus CS ) s sgnfcant for that subject. Note that you mght use an unpared ( between-subjects ) analyss, snce the CS + data and the CS data are not related beyond the fact that they come from the same subject (whch s now your expermental unverse ) unless there s some further factor that pars data ponts wthn each subject. (One such factor mght be tral par, f one tral par has one CS + and one CS presented close to each other n tme.) However, the conclusons of such a test apply only to that subject. You could not generalze t to others ( subjects n general wth such-and-such a leson ). I ve seen arguments that run lke ths: We compared a CS + and a CS for each subject to obtan a measurement of CS reactvty [a sngle number per subject]. We compared these CS reactvty scores pre-operatvely and post-operatvely. he leson sgnfcantly reduced CS reactvty scores n out of 4 lesoned subjects [note wthn-one-subject sgnfcance tests]. None of the 4 sham-operated subjects showed a sgnfcant change n CS reactvty scores. he mplcaton that one s presumably meant to draw s the leson reduced CS reactvty. here are at least two fallaces here: he sgnfcance tests for ndvdual subjects don t tell you that the change was sgnfcant for a group. (Ignorng the prevous pont for a moment ) he change n reactvty scores was sgnfcant for group but not for group B; therefore group dffered from group B. hs s a common statstcal fallacy. here mght have been a decrease n scores for one group (p.4) but not the other (p.6) that does not mean that the two groups dffered. hat test would requre examnaton of the leson (pre-post) nteracton or, better (as we saw above), an analyss of covarance wth pre-operatve scores as the covarate. Even f you used sgnfcant or not as a dchotomy and t would be an artfcal dchotomy (usng a crteron p value as a cut-off, rather than a genune dchotomy such as sex; see Howell, 997, p. 86), the test across groups would then be a χ contngency test wth two varables (sham versus leson; changed versus unchanged). For ths specfc example, χ. 67, p., NS.

51 4: Expermental desgn Should I add/remove a factor? Full versus reduced models Omttng relevant varables and ncludng rrelevant varables can both alter your estmate of effects of other varables (Myers & Well, 995, pp. 59-5). Includng rrelevant varables sn t too bad ths doesn t bas the estmate of the proporton of varablty accounted for by your other predctors, but t does use up error degrees of freedom, reducng the power to detect effects of other varables. Omttng relevant varables s worse; t can substantally bas the estmates of the effects of the other terms. s a smple example, suppose your data contan a man effect of and a man effect of B, but no nteracton. If you were to analyse these data usng a model wth just and B terms (and no B term), you ve omtted a relevant varable, and you can get a spurous nteracton. here are varous formal ways to work out the best set of predctor varables to use f you have a lot of potental predctors (e.g. forward selecton, backward elmnaton, and stepwse regresson; see Myers & Well, 995, p ), but they are prmarly of use n descrptve (correlatve, non-expermental) research and none of them removes the need to thnk carefully about your expermental desgn. People commonly neglect potentally mportant predctors (Myers & Well, 995, pp. -, 49-5), such as who tested the subjects, because they re not of nterest, or they weren t thought about. hese are poor reasons. good reason to remove a predctor from an NOV s that you have evdence that t sn t contrbutng to the predcton. If so, then by removng t, you may ncrease the power to detect other effects. good rule s to nclude all the potentally relevant predctors ntally, and consder removng a term f (a) you have a pror reason to thnk t s rrelevant and (b) the term s not sgnfcant at the α.5 level (Myers & Well, 995, pp. -, 5). Note that a non-normal dstrbuton of resduals (p. 36) may also suggest the need to add another predctor (or to transform the dependent varable). For example, suppose we have a three-way NOV (factors, B, and C). he expermenter s prmarly nterested n the effects of and B. he analyss shows that none of the C, B C, B C terms are sgnfcant at the α.5 level, but the man effect of C s sgnfcant at α.5. he plan would then be to drop out those nteractons, so you re left wth, B, B, and C. Droppng out terms that are genunely not contrbutng helps, because t ncreases the error df (whch ncreases power); the df and any varablty attrbutable to the term jons (s pooled wth ) the error df and error varablty. You hope that the error df go up but the error varablty doesn t whch should be the case f the term wasn t contrbutng to the predcton. But f your term s actually contrbutng, then poolng ts varablty as part of the error term also ncreases the E(MS) of the error term, negatvely basng all your other F tests makng t less lkely that you ll detect other effects that you re nterested n (Myers & Well, 995, pp. 49-5). hs argument also apples to the expermental desgn technque of blockng (Myers & Well, 995, pp ). Suppose we want to test the effect of dfferent types of teachng method () on readng skll (Y) n chldren, and subjects are randomly assgned to the levels of. If there s consderable ndvdual varaton (varablty among subjects wthn groups the error term for the NOV) we may have low power to detect effects of. One way to deal wth ths s to block the subjects. We would dvde them nto groups based on ther performance on some varable, (perhaps IQ?), that we beleve to be hghly correlated wth Y. Suppose we used fve blocks: block B would contan the chldren wth the hghest scores, block B would have the next-hghest scores, and so on. hen we would randomly assgn the members of block B to the dfferent condtons. We have made our one-factor NOV nto a two-factor NOV; we hope that ths reduces the wthn-block nter-subject varablty, and therefore ncreases the power to detect effects of. In general, blockng s ntended to reduce error varablty (whch ncreases power). Of course, t uses up error df (whch reduces power). herefore, to get the best power,

52 4: Expermental desgn 5 you should choose the number of blocks based on N, a, and the correlaton (ρ) between and Y (see Myers & Well, 995, pp ). 4.6 Should I add/remove/collapse over levels of a factor? he key thng to remember s ths: MS F MS predctor error SS SS predctor error df df error predctor he more levels a factor has, the larger ts df predctor, so on ts own ths wll reduce the F statstc, and therefore the power to detect the effect of ths factor. On the other hand, f addng a level ncreases SS predctor, power goes up. nd, all other thngs beng equal, addng more observatons ncreases power because t ncreases df error. Let s llustrate ths wth a couple of examples: ddng and removng levels by addng new observatons akng new observatons at further levels of a factor can reduce power: Left: the dependent varable s measured at only two levels of (n 5 per group). here s a sgnfcant effect of (MS 4.349, MS error.793, F, , p.47). Rght: three more groups have been measured. Even though the orgnal data s unchanged, the effect of s now not sgnfcant (MS.58, MS error.88, F 4,.958, p.4). Vertcal lnes represent contrbutons to SS ; specfcally, SS s the sum of the squares of these vertcal lnes (devatons of group means from the overall mean). Equally, t s very easy to magne a stuaton n whch a non-sgnfcant effect wth a few levels becomes a sgnfcant effect when subjects are measured at more levels a very easy example would be a drug measured at and. mg doses, where. mg s below the effectve dose; f and mg doses are added to the study, the effect of the drug mght emerge Collapsng over or subdvdng levels Collapsng over levels wth smlar means ncreases power:

53 4: Expermental desgn 53 Left: there s not a sgnfcant effect of (SS 8.34, MS.775, MS error.6, F 3,6.8, p.8). Rght: f we collapse over levels by combnng levels and, and levels 3 and 4, there s a sgnfcant effect of (SS 6.8, MS 6.8, MS error.4, F,8 5.8, p.37). But collapsng over levels can have the opposte effect, f you collapse over levels wth dssmlar means: Left: there s a sgnfcant effect of (SS 7.549, MS 59.63, MS error.57, F 3,6 7.65, p.3). Rght: f we collapse over levels n the same way as before, we reduce the sum of squares and there s no longer a sgnfcant effect of (SS 7.5, MS 7.5, MS error.5, F,8 3.34, p.84).

54 5: Usng SPSS 54 Part 5: usng SPSS for NOV 5. Runnng NOVs usng SPSS 5.. nalyss of varance You can perform NOVs from the nalyze General Lnear Model menu (below). Unvarate analyses a sngle dependent varable. It wll easly handle betweensubjects desgns. Fll n your between-subjects factors as fxed factors and add any betweensubjects covarates (by default these wll not nteract wth any factors). It wll also handle wthn-subjects desgns f your data s n a one column, one varable format smply enter Subject as a random factor and enter all the real factors as fxed factors. However, ths way of dong wthn-subjects analyss may be slow and wll not nclude Mauchly s test or the Greenhouse Gesser or Huynh Feldt correctons (explaned above; see p. 5). Furthermore, t wll get the analyss of mxed models (models that have both between-subjects and wthn-subjects factors) wrong unless you enter a custom model n the Models dalogue box. he easer way of analysng smple desgns that nclude wthn-subjects factors s wth the Repeated Measures opton; ths requres that your data s n a one row, one subject format. hs opton also allows you to nclude between-subjects factors and between-subjects covarates. he Multvarate opton s used for analysng multple dependent varables (multvarate analyss of varance: MNOV), and we wll not cover t. 5.. Syntax Whenever you see an OK button to begn an analyss n SPSS, there wll also be a Paste button that wll not run the analyss, but wll copy the syntax for the analyss nto a syntax wndow (openng one f necessary). hs allows you to edt the syntax, f you want to do somethng complcated; t also allows you to save syntax so that you can run large mult-step analyss tme after tme wth the mnmum of effort. You can even nclude syntax to load data from a fle, or retreve t from an ODBCcompatble database. he Run menu of a syntax wndow allows you to run all of a syntax fle, or part that you have selected.

55 5: Usng SPSS Plots SPSS can produce sketch plots along wth ts NOV output. Clck the Plots opton of an NOV dalogue box and fll n the gaps. Clck dd to add your plot to the lst once you ve assembled ts components Optons, ncludng homogenety-of-varance tests ll the NOV dalogue boxes also allow you to set Optons. By default, no optons are tcked: I fnd t useful to nclude descrptve statstcs (ncludng means and SEMs for all levels of factors and nteractons). I tend reflexvely to compare man effects usng a Sdak correcton. It s certanly worthwhle ncludng homogenety tests to check the assumptons of the NOV; SPSS wll perform Levene s test for homogenety of varance (sgnfcant heterogeneous a problem) f you tck ths box. he optons menu for the Unvarate analyss looks slghtly dfferent:

56 5: Usng SPSS Post hoc tests SPSS wll allow you to specfy post hoc tests for between-subjects factors n the Post hoc dalogue box: It won t allow you to specfy post-hoc tests for wthn-subjects factors, manly because most post hoc tests are not sutable for use wth wthn-subjects factors (see Howell, 997, p. 47). SPSS tres hard not to let you do somethng daft. he smplest and usually best thng to do s to run a separate wthn-subjects NOV for the data you want to perform a wthn-subjects post hoc test on. 5. Interpretng the output Let s look at a real and farly complcated analyss. It nvolves four factors. Rats were receved ether lesons of the nucleus accumbens core (cbc) or sham surgery. Each group was further dvded nto three (delay,, or s). ll rats were placed n operant chambers wth two levers present throughout each sesson. One lever (Inactve) dd nothng. he other (ctve) delvered a sngle food pellet. In the delay group, that pellet was delvered mmedately. For the delay s group, the pellet was delvered after a s delay, and for the delay s group, after s. hey were traned for 4 sessons each. hese are our factors: Factor Between-subjects (B) or wthn-subjects (W) Number of levels Levels Leson B sham, cbc Delay B 3,, s Lever W actve, nactve Sesson W 4 4

57 5: Usng SPSS 57 he data s entered nto SPSS n one subject, one row format (see p. ), lke ths: Subject Leson Delay S_ctve S_Inactve S_ctve S_Inactve O sham datum datum datum datum O sham datum datum datum datum O9 sham datum datum datum datum O48 cbc datum datum datum datum We have wthn-subjects factors and we have the data n one-subject-one-row format, so we choose nalyze General Lnear Model Repeated Measures: We declare the wthn-subjects factors: Now we fll n the between-subjects factors and assgn ndvdual columns to approprate levels of the wthn-subjects factors: It s mportant to ensure that the wthn-subjects level assgnments are correct so s5_nact s labelled as (5,), and the dalogue box tells us that ths refers to (sesson, lever) so t s gong to be level 5 of sesson and level of lever. hs s correct. So we proceed to set approprate optons. I m gong to tck loads of thngs so we can nterpret a farly full output:

58 5: Usng SPSS 58 I wouldn t normally tck estmates of effect sze, observed power, or parameter estmates. We can also set up some plots: SPSS doesn t do very good graphs, and t ll only plot three factors at once. So ths plot has sesson on the horzontal axs, delay on separate lnes, and lever on separate plots. (he data wll be collapsed across leson, whch means ths graph won t gve us any ndcaton of how the sham/leson groups dffered not very helpful!) OK. Now we could Paste the syntax for ths command nto the syntax edtor to save t and/or fddle wth t, or just clck OK to run the analyss. Let s run the analyss. We get a lot of stuff It s huge! Let s look at them one by one. tle. Says General Lnear Model. Notes. None.

59 5: Usng SPSS 59 Warnngs. ells you what t couldn t do. Sometmes ths nformaton s helpful; here, t s not very comprehensble and we gnore t. Wthn-subjects factors. hs tells you what you told t. It lsts all your wthnsubjects factors and tells you whch column of data has been matched to each level of the factor(s). If ths s wrong, the rest of your analyss wll be meanngless, so t s worth checkng. Between-subjects factors. he same, but for between-subjects factors. It also gves you the number of subjects n each condton. Check ths t may not always be what you expect. If a subject has mssng data somewhere, SPSS wll default to chuckng the subject out completely. Descrptve statstcs. Snce we asked for ths n the Optons, we get a long lst of cell means: Multvarate tests. Ignore em; we re not analysng multple dependent varables. We re analysng one (lever-pressng), predcted by four predctors (factors). So skp ths. Mauchly s test of sphercty. For every wthn-subjects factor and nteracton of wthn-subjects factors, SPSS performs Mauchly s test of sphercty. If t s sgnfcant ( Sg. column p <.5), then you should multply your df by the Huynh Feldt epslon ~ ε lsted by t. For example, the Sesson factor has volated the sphercty assumpton and wll have ts df multpled by ~ ε. 87, whle the Sesson Lever nteracton wll have ts df multpled by ~ ε he Lever factor has not volated the sphercty assumpton. Sometmes you can tell because the Sg. column has a p value that s >.5. Here, there s no p value but snce ~ ε, we know that there s no problem anyway. ests of wthn-subjects effects. hs s one of the mportant bts. here s a set of rows for every wthn-subjects factor, or nteracton nvolvng a wthnsubjects factor.

60 5: Usng SPSS 6 here s a columns correspondng to the SS ( ype III Sum of Squares SPSS has a few ways of calculatng the SS and you almost certanly want ype III, whch s the default). It gves you the df. he top row ( sphercty assumed ) gves you the normal df. he subsequent rows gve you the df multpled by the varous correcton factors lsted n the results of Mauchly s test, ncludng the Huynh Feldt epslon ~ ε. he MS s the SS dvded by the df. he F rato s the MS for the term dvded by the MS for the correspondng error term. It s always the same, no matter whether you use the Huynh Feldt correcton or not. For example, the F for Sesson (8.) s the MS for SES- SION (9.45, 38.99, 3.475, or 7.584, dependng on the df correcton) dvded by the MS for Error(SESSION) (.3,.487,.393, or.468, dependng on the df correcton). he Sg. column s the p value for the F rato, assessed on the relevant number of degrees of freedom. It may vary dependng on whether or not you need to use the Huynh Feldt correcton. In ths example, Lever doesn t requre any correcton, so we would report F,38 678, p <. for the effect of Lever. However, Sesson requres a Huynh Feldt correcton, as we saw above, so we would report F 3.736, , ~ ~ ε.87, p <.. If you correct the df, t s good practce to report ε so readers can work out the orgnal df (whch tells them somethng about your analyss). Partal eta-squared s a column that only appeared because we tcked Estmates of effect sze n the Optons. For detals of η partal, see p.. Noncent(ralty) parameter and Observed power only appeared because we tcked Observed power. he observed power s the probablty that the F test would detect a populaton dfference between the two groups equal to that mpled by the sample dfference (SPSS,, p. 476). he noncentralty parameter s used to calculate ths (Howell, 997, pp ).

61 5: Usng SPSS 6 ests of wthn-subjects contrasts. Well, we ddn t ask for ths explctly and we re not nterested n any specfc contrasts at the moment, so we ll gnore ths. Levene s test of equalty of error varances. more mportant one: tests whether the varances of the varous data columns dffers across groups (defned by the between-subjects factors). hs tests the homogenety of varance assumpton of NOV. he results here aren t deal we have a few volatons of ths assumpton (where p <.5). For example, the varablty of sesson, actve lever responses sn t the same across all sx between-subjects groups (sham-, sham-, sham-, cbc-, cbc-, cbc-). hese data have n fact already been square-root transformed to try to mprove matters, but there s stll a volaton of the homogenety of varance assumpton n 7 of the 8 data columns. We have to make a judgement about the robustness of NOV n these crcumstances (and the alternatve analytcal technques avalable); although sgnfcant, the varances don t n fact dffer by huge amounts f you look at the descrptve statstcs (for example, the sesson /actve lever responses have SDs that range from.47 to. a.5-fold dfference, whch sn t the end of the world as NOV s reasonably robust to that level of volaton; see p. 33). ests of between-subjects effects. he other mportant bt that everyone wll want to look at. nd very easy to nterpret. We can see that there s a sgnfcant effect of delay (F, , p <.) and although there s no man effect of leson (F <, NS), there s a leson delay nteracton (F, , p.6). Of course, we d want to nterpret all the wthn-subjects factors and the

62 5: Usng SPSS 6 complex nteractons too (for example, ths data set has a 4-way sesson lever leson delay nteracton). Parameter estmates. Not really very useful unless we re dong some regresson analyss, so t probably wasn t worth tckng t for ths analyss! Estmated margnal means. hese can be useful. SPSS gves the means for the varous levels of each factor (or nteracton). I also tcked Compare man effects wth a Sdak adjustment n the Optons. hs gves us some quck posthoc tests. If you have a factor wth only two levels (e.g. Leson), ths tells you nothng more than the NOV dd. But for factors wth > levels, t can be useful. Here are the means for Delay, whch t s certanly vald to perform post hoc tests on (snce t was sgnfcant n the NOV, above). We see the mean (across all other varables) for Delay ( Estmates ), and then t compares pars of delays ( v., v., v. ) ( Parwse comparsons ). We also get the standard error of the mean (SEM) for each mean and the standard error of the dfference between means (SED) for every parwse comparson (see p. 43 ). Fnally, t repeats the overall F test from the NOV (not very helpfully; Unvarate ests ). p: parwse comparsons for nteractons op tp: by default, SPSS only performs parwse comparsons for factors, and not nteractons. If we were to Paste the syntax for ths analyss, we d see ths sort of thng: /EMMENS BLES(leson) COMPRE DJ(SIDK) /EMMENS BLES(delay) COMPRE DJ(SIDK) /EMMENS BLES(sesson) COMPRE DJ(SIDK) /EMMENS BLES(lever) COMPRE DJ(SIDK) /EMMENS BLES(leson*delay) /EMMENS BLES(leson*sesson) /EMMENS BLES(delay*sesson) Note that the man effects have COMPRE and DJ(SIDK) on them, but the nteractons don t. If you want, you can add that n syntax! Lke ths: /EMMENS BLES(leson) COMPRE DJ(SIDK) /EMMENS BLES(delay) COMPRE DJ(SIDK)

63 5: Usng SPSS 63 /EMMENS BLES(sesson) COMPRE DJ(SIDK) /EMMENS BLES(lever) COMPRE DJ(SIDK) /EMMENS BLES(leson*delay) COMPRE(leson) DJ(SIDK) /EMMENS BLES(leson*delay) COMPRE(delay) DJ(SIDK) You can t just put COMPRE, because SPSS wouldn t know whether to compare Leson dfferences for each level of Delay, or Delay dfferences for each level of Leson. So you specfy one other thng; for example, COMPRE(leson) would compare Leson groups at each level of Delay. You can specfy both knds of comparson, as I dd above. he output also gves you the standard error of the dfference for each comparson (see p. 45). Fnally, you can specfy a Sdak correcton to the tests by addng DJ(SIDK), or smlarly for Bonferron f you really want to. hs can be extended to hgher-order nteractons; you specfy the factor you want to be compared at all possble combnatons of the other factors. Observed * predcted * std. resdual plots. SPSS s resdual plots are a lttle bt ncomprehensble; see p. 36 for explanatons. Profle plots. Fnally, we get some not-so-pretty graphs:

64 5: Usng SPSS Further analyss: selectng cases In ths stuaton, we d want to do further analyss, especally snce we have a hugely complex 4-way nteracton. We mght want to fnd out f there are effect of Leson or Delay f we only consder ctve lever responses easy, we just run another repeated-measures NOV on the ctve lever data only, wthout the Lever factor. We mght also want to see f there s an effect of delay/sesson/lever n the shams alone. For ths we mght want to restrct the cases analysed by SPSS. Choose Data Select cases: hen clck If We only want to select cases f the leson varable s equal to sham :

65 5: Usng SPSS 65 Clck Contnue and the condton s entered nto the prevous dalogue box: Clck OK. You ll now fnd that all cases (rows) that don t match your crteron are crossed out, and won t be analysed: 5.4 he ntercept, total, and corrected total terms When you run an NOV wth SPSS, by default t ncludes the ntercept term. o turn ths on/off wth the menus, clck on the Model button:

66 5: Usng SPSS 66 You can then choose to Include ntercept n model or not. In syntax, you can add the command /INERCEP ECLUDE or /INERCEP INCLUDE What does ths do? Let s llustrate wth some sample data for an NOV wth a sngle between-subjects factor wth two levels: Dependent varable x Overall mean x ( x x) ( x x) ( x x ) x x x a n per group N an SS total as usually calculated SS 338. SS error SS ntercept SS total wth ntercept ncluded SS model wth ntercept as part of model If you run ths analyss wth the ntercept ncluded, SPSS prnts ths: ests of Between-Subjects Effects Dependent Varable: DEPVR ype III Sum Source of Squares df Mean Square F Sg. Corrected Model a Intercept Error otal Corrected otal a. R Squared.6 (djusted R Squared.83) Here, ts SS total s x ; ts df total s N. he ntercept tself (the grand mean) has SS ntercept N x wth df ntercept. he corrected total, SS corrected total SS total SS ntercept s what we normally thnk of as SS total, namely ( x x), wth the usual df of N. he effect of s gven by SS n( x x), df a. he corrected model models the effects of the factor(s),, gnorng the effect of the ntercept (the grand mean). If you have more than one factor, the corrected model term s the sum of all ther effects: SS corrected model SS total SS ntercept SS error. he error s calculated as usual: SSerror ( x x), df error (N ) (a ).

67 5: Usng SPSS 67 Incdentally, the F test on the ntercept term (MS ntercept /MS error ) tests the null hypothess that the grand mean s zero. If you run an NOV wth no factors other than the ntercept (or wth a factor wth only one level, whch SPSS wll let you do), t s equvalent to a one-sample t test comparng all N observatons to zero; as for any t test, F t and t k F, k., k k If you don t nclude the ntercept, you get ths: ests of Between-Subjects Effects Dependent Varable: DEPVR ype III Sum Source of Squares df Mean Square F Sg. Model a Error otal a. R Squared.89 (djusted R Squared.8) In other words, when you exclude the ntercept, the model models the effects of the factor(s),, and the ntercept, together, wthout dstngushng the two. In ths case, t calculates SS total s x ; ts df total s N. he model (ntercept plus effect of ) has SSmodel n x, df a (df because there are two x means and one overall x mean). SS s calculated wthout consderng the dfference between the effect of and the grand mean as we would usually do, so SS SS model for ths onefactor case. he error s calculated as usual: SSerror ( x x), df error (N ) (a ). It should be farly clear that you probably want to nclude the ntercept when runnng NOVs n SPSS. hs s the default.

68 6: dvanced topcs 68 Part 6: advanced topcs harder thngs about NOV 6. Rules for calculatng sums of squares 6.. Parttonng sums of squares Sums of squares are parttoned exactly as degrees of freedom (see below, p. 68). hs requres a structural model. We ve seen several examples of ths, and many more are dscussed n Part General rule for calculatng sums of squares Every SS corresponds to a term n the structural model that represents the dfference between two quanttes P and Q. Every SS s the summed squared devaton of P from Q. If a term contrbutng to the SS s based on n observatons, multply ts contrbuton by n. For example, for two between-subjects factors and B, the structural model s Y µ + α + β + αβ + ε jk j j jk and f there are a levels of, b levels of B, and n subjects (ndependent observatons) per B combnaton, the SS are erm µ ntercept Sum of squares SS Ny generally gnored α µ µ SS nb( y y) each y mean based on nb scores β j µ B j µ SSB na( y B y) each y B mean based on na scores αβ j µ B ( µ + α + β j ) j SSB n( y B y) (SS + SSB) each y B mean based on n scores ε jk Yjk ( µ + α + β j + αβj ) SS ( y y) (SS + SS + SS ) SS (SS + SS SS ) error B B total B + total SS ( y y) SS + SS + SS + SS B SS SS + SS y generally gnored grand total ncludng ntercept total B ntercept We frst saw the general technque for dervng these SS equatons on p. 5 (and another s on p. 59): we rearrange the structural model to gve Y jk µ on the lefthand sde, expand out the defnton of all the terms, smplfy, square both sdes of the equaton (so we have SS total on the left-hand sde), and elmnate a number of terms that sum to zero. he expected value of the squared terms n the structural model are drectly related to the E(MS), dscussed below (p. 73); for example, E ) σ ; E ( α ) σ e + nbσ. error ( ε jk e B 6. Rules for calculatng degrees of freedom From Keppel (99, pp. 7-4). For any source of varance: he df equal the number of dfferent observatons on whch each sum of squares s based, mnus the number of constrants operatng on these observatons. (hs s the defnton of df n general: the number of ndependent

69 6: dvanced topcs 69 observatons, or the number of observatons mnus the number of constrants.) For between-subjects desgns: he man effect of a factor wth a levels has a df. So df a and df B b. he man effect of a covarate has df (snce ts effect s represented by a straght lne, whch can be determned by two parameters, but the lne s constraned to pass through the overall mean, so the one df represents the lne s slope; t s thus akn to a factor wth two levels). he df for an B nteracton, where has a levels and B has b levels, s the product of the two separate dfs,.e. df B (a )(b ). he total number of dfs s the number of observatons N mnus,.e. (N ). he error or resdual df s df total mnus the sum of everythng else. We partton dfs n exactly the same way as SSs. For example, for an B S desgn, SS total SS + SS B + SS B + SS error df total df + df B + df B + df error For wthn-subjects and mxed desgns, most of the above stll holds, but we don t have just a sngle error term. akng groups to refer to groups of subjects defned by between-subjects factors: df between subjects total subjects df wthn subjects df total df between subjects df subjects wthn groups df between subjects df groups df WS factor subjects wthn groups df wthn subjects df WS factor df WS factor groups If a group s defned by the between-subjects factor, we would wrte subjects wthn groups as S/. For example, f we have the desgn (U S) wth a between-subjects factor wth 3 levels, n 8 subjects per group (4 subjects total), and a wthn-subjects factor U wth 6 levels, we would be able to calculate: df total N anu (3 8 6) 43 df between subjects total subjects 4 3 df a 3 df S/ df between subjects df 3 df wthn subjects df total df between subjects 43 3 df U u 6 5 df U df U df 5 df U S/ df wthn subjects df U df U 5 5 We partton sums of squares n exactly the same way as dfs (descrbed for ths partcular desgn n more detal later), lke ths: SS total SS between subjects + SS wthn subjects SS between subjects SS + SS S/ SS wthn subjects SS U + SS U + SS U S/ You can see that ths exactly mrrors the df parttonng shown above (wth sutable smple arthmetc rearrangement). 6.3 Nasty bt: unequal group szes and non-orthogonal sums of squares hs can be very complcated. So far we ve assumed that equal-szed expermental groups have been sampled from equal-szed treatment populatons. If ths s not the case, we can have problems. Frstly, unequal ns exaggerate the problem of heteroge-

70 6: dvanced topcs 7 nety of varance (see Myers & Well, 995, pp. 5-6) (and see p. 33). Secondly, they can really screw up an NOV Proportonal cell frequences If we have unequal populaton szes and the sample szes reflect the ratos of ther szes and, f there s > factor, the nequaltes are n consstent proportons across those factors we re OK. For example, suppose (Myers & Well, 995, p. 5) we know that Labour, Conservatve, and Lberal Democrat supporters are present n our populaton n the rato 4:3:3, and we know that two-thrds of each group voted n the last electon. We could qute reasonably run experments on them wth the followng numbers of subjects: Labour Conservatve Lb Dem Voted Dd not vote 9 9 No huge problem here. Suppose we use two between-subjects factors and B agan, as above. Suppose there there are a levels of and b levels of B. But now suppose there are n observatons for condton, n j observatons for condton B j, and n j observatons for condton B j. Snce every SS has a contrbuton from every observaton t s based on, the formulae are stll very smple: erm µ ntercept Sum of squares SS Ny generally gnored α µ µ SS n ( y y snce ) B n j ( yb y) j j β j µ µ SS snce B j αβ j µ B ( µ + α + β j ) SS n ( y y) (SS SS ) j ε jk Yjk ( µ j + α + β j + αβj ) B j B + j j y s based on n scores y B j s based on n j scores B snce y B s based on n j j scores SSerror ( y y) (SS + SSB + SSB) SStotal (SS + SSB + SS total SS ( y y) SS + SS + SS + SS SS grand total ncludng ntercept 6.3. Dsproportonate cell frequences a problem SS B total + SS B ntercept Here s an example (from Howell, 997, p. 43): expermenters test the number of errors made by sober and by drunk people on a smulated drvng test. wo experments dvde up the work, testng half the subjects n ther Mchgan lab and half n ther rzona lab. hey have absolutely no reason to thnk that the choce of state makes any dfference. hese are ther results: error B y generally gnored ) Number of errors Sober Drunk Mchgan 3, 5, 4, 6, 8,,, 9,, 3, 7, 8,, Mchgan mean 8. (n 5, mean 4) rzona 3, 5, 8, 4,,, 6, 7, 5,, 4 (n, mean ) 4, 5, 7, 6, 8 rzona mean 5.9 (n, mean 4) (n 5, mean ) Sober mean 4 Drunk mean It appears that drunk subjects make more errors than sober subjects, whch makes sense, but t also looks lke Mchgan subjects make more errors than rzona subjects. But clearly that s an lcohol effect masqueradng as a State effect the Mchgan lab tested a hgher proporton of ts subjects whle drunk. he two factors

71 6: dvanced topcs 7 are correlated, thanks to dsproportonate cell frequences f you knew whether a subject was drunk or sober, you could guess better than chance whch state the subject came from. What can we do? We can use unweghted means. When we calculated the Mchgan mean, we calculated t as a weghted mean (where M Mchgan, S sober, D drunk n the formula): y M y n M, S M, S y + y n M M, S + n n M M, D M, D y M, D hs s weghted n the sense that the contrbuton of ndvdual cell means ( y M, S and y M, D ) s weghted by the sample szes ( n M, S and n M, D ). n unweghted mean (or, more properly, an equally weghted mean) s what you get when you smply average the cell means, gnorng the number of subjects n each cell. hat would gve us a Mchgan mean of (4 + )/ 7, and an rzona mean exactly the same. In an unweghted-means analyss, each cell mean contrbutes equally to the calculaton of each of the sums of squares. In the calculaton, we calculate an average cell sze (the harmonc mean of the cells szes; see revson maths chapter, p. 3) and use that average n as f every cell had that many subjects (Howell, 997, pp ). hs s a specfc example of a general problem when the effects of two or more effects (or nteractons) are not fully ndependent. he example shown above s farly common (the effects of one factor, State, are partly correlated wth the effects of another, lcohol, because one state tested a hgher proporton of drunks). It may be easer to vsualze the problem wth an even more extreme example one n whch two factors and B are completely correlated. Consder ths partcularly stupd set of data collected as part of an B S desgn (Myers & Well, 995, p. 53): B no observatons B no observatons Let s calculate the SS. Each observaton makes one contrbuton to the SS, as usual, so we should defne n as the number of observatons at level, n j as the number of observatons at level B j, and n j as the number of observatons at j B j. hen

72 6: dvanced topcs 7 SS SS SS total SS SS B B error ( y y) n ( y y) n ( y y) j n ( y y) (SS j SS j j total B j B j (SS SS B + SS + SS ) 4.5 B B ) Pretty stupd; we have a negatve SS B! he problem s that the effects of and B n ths desgn are not orthogonal; the man effects of and B are perfectly correlated (smply because there are only observatons for B and B ; the effects of and B are confounded). If we added two B and two B observatons, the effects of and B are now not perfectly correlated, but they are stll correlated. he problem can be llustrated lke ths: If we calculate SS n the usual way, t conssts of t+u+v+w. On the other hand, f we adjust t for the contrbuton of the other man effect B, t would consst of t+w. Or we could adjust t for the contrbuton of B and B, n whch case the adjusted SS would consst only of t. Smlar optons exst for the other sources of varance. he approprate choce probably depends on the mportance the expermenter attaches to the varous factors (Myers & Well, 995, p. 55). See also Howell (997, pp ). hs also means that the order you enter terms nto a computer analyss can affect the results. On some packages, an NOV wth the sources of varance beng, B, and B gves you a dfferent answer from an NOV wth the sources of varance beng B,, and B. he default method n SPSS does not care about the order t s what SPSS refers to as the ype III sum of squares. I thnk (Myers & Well, 995, p. 55) that ths method uses area t for SS, area x for SS B, and z for SS B. hs s probably what you want t s certanly approprate for the case when there s chance varaton n cell frequences, such as when subjects drop out at random (Myers & Well, 995, p. 55). It s also the method approxmated by the unweghted (equally weghted) means soluton descrbed above (Howell, 997, p. 58). In general, whenever cell frequences (ns n each cell) are equal or proportonal (meanng that for each cell, n j n n j /N), the sums of squares are orthogonal (unless the experment tself has been ms-desgned and confounds two varables). But whenever cell frequences are dsproportonate, the sums of squares are nonorthogonal (Myers & Well, 995, p. 54; Howell, 997, pp and ). hs problem occurs whenever predctor varables are themselves correlated (see also Myers & Well, 995, pp ). NOV wth equal cell frequences s exactly equvalent to multple regresson wth uncorrelated categorcal varables (Myers & Well, 995, p. 536), and NOV wth dsproportonate cell frequences mples that the factors are correlated. hs s easy to see: f our utsm Sex ex-

73 6: dvanced topcs 73 perment has 8 male autstcs, female autstcs, male controls, and 8 female controls (dsproportonate cell frequences), you can make a better-than chance guess as to whether a subject s male or female f you know whether they re autstc or not the two factors are correlated. It s, of course, possble to have a mddle ground unequal but proportonate cell frequences (see above, p. 7, for an example), whch stll nvolves orthogonal sums of squares. 6.4 Expected mean squares (EMS) and error terms Frst we need to consder the samplng fracton for fxed and random factors (fxed and random factors are defned on p. 3). If we have factor wth a levels and t s a fxed factor, we have sampled all the levels. We can say that the maxmum number of levels of s a max a, and the samplng fracton a/a max. On the other hand, f our factor s a random factor, a max s lkely to be very large, so a/a max, approxmately. ake the example of subjects: we presume that our s subjects are sampled from a very large populaton, s max, so the samplng fracton s/s max. It s possble to have samplng fractons between and (Howell, 997, p. 43) but you wll have to work out some messy EMSs yourself. Software packages such as SPSS assume that the samplng fracton s for fxed factors and for random factors Rules for obtanng expected mean squares (EMS) From Myers & Well (995, p. 99). Let s lst the rules wth an llustratve example. Suppose we have one between-subjects factor wth 3 levels. here are 6 subjects per level of the between-subjects factor (n 6). here are 4 levels of a wthnsubjects factor B.. Decde for each ndependent varable, ncludng Subjects, whether t s fxed or random. ssgn a letter to desgnate each varable. ssgn another letter to represent the number of levels of each varable. (In our example, the varables are desgnated, B, and S; the levels are a, b, and n respectvely. and B are fxed and S s random.). Determne the sources of varance (SS) from the structural model. (We ve already seen what ths produces for our example desgn, when we dscussed t earler: SS total s made up of SS + SS S/ + SS B + SS B + SS SB/. hese are our sources of varance.) 3. Lst σ e as part of each EMS. 4. For each EMS, lst the null hypothess component that s, the component correspondng drectly to the source of varance under consderaton. (hus, we add nbσ to the EMS for the lne, and bσ S / to the EMS for the S/ lne.) Note that a component conssts of three parts: coeffcent representng the number of scores at each level of the effect (for example, nb scores at each level of, or b scores for each subject). σ [Myers & Well (995, pp. 99) use σ f s a random factor, and θ f s a fxed factor; Howell (997, p. 43) doesn t, and I thnk t s clearer not to.] s subscrpts, those letters that desgnate the effect under consderaton. 5. Now add to each EMS all components whose subscrpts contan all the letters desgnatng the source of varance n queston. (For example, snce the subscrpt SB/ contans the letters S and, add σ SB / to the EMS for the S/ lne.)

74 6: dvanced topcs Next, examne the components for each source of varance. If a slash (/) appears n the subscrpt, defne only the letters to the left of the slash as essental. If there are several slashes, only the letters precedng the leftmost slash are essental. If there s no slash, all letters are essental. 7. mong the essental letters, gnore any that are necessary to desgnate the source of varance. (If the source of varance s, for example, then when consderng nσ B, gnore the. If the source s S/, then when consderng the σ SB / component, S and B are essental subscrpts and S s to be gnored.) If any of the remanng (non-gnored) essental letters desgnate fxed varables, delete the entre component from the EMS. n example: erm EMS so far Step : dentfy varables and numbers of levels., a (between-subjects factor) B, b (wthn-subjects factor) S, n (number of subjects per group) Step : dentfy sources of varance. Step 3: Lst S/ B B SB/ σ e as part of each EMS. S/ B B SB/ e σ e σ σ e e σ e σ Step 4: lst the null hypothess component. σ e + nbσ S/ σ e + bσ S / B σ e + anσ B B σ e + nσ B SB/ σ e + σ SB / Step 5: add all components whose subscrpts contan all the letters desgnatng the source of varance n queston. σ e + nbσ + bσ S / + nσ B + σ SB / S/ σ e + bσ S / + σ SB / B σ e + anσ B + nσ B + σ SB / B σ e + nσ B + σ SB / SB/ σ e + σ SB /

75 6: dvanced topcs 75 Steps 6 and 7: for each component, defne essental letters; gnore any that are part of the desgnaton of the source of varance; f any remanng essental letters contan fxed factors, delete the component. σ e + nbσ + bσ S / S/ σ e + bσ S / B σ e + anσ B + σ SB / B σ e + nσ B + σ SB / SB/ σ e + σ SB / 6.4. Choosng an error term mean square qualfes as an error term for testng an effect f ts E(MS) matches the E(MS effect ) n all respects except the null-hypothess component (Keppel, 99, p. 568). In our example above, therefore, we d test MS aganst MS S/, and we d test both MS B and MS B aganst MS SB/ Poolng error terms When we have random factors n a model, mportant varables are often tested aganst an nteracton term. Snce nteracton terms have few df (and snce power depends on F beng large when the null hypothess s false, and snce F s the rato of MS effect to MS error, and snce MS error s SS error /df error ), ths means we may have poor power to detect such effects. One possblty s to test nteracton terms n a full model wth a conservatve crteron, lke ths (Howell, 997, p. 45). If there s an nteracton (p <.5), we declare that there s an nteracton. If there sn t (.5 < p <.5), we just look at the results for other terms. But f there s no nteracton (p >.5), we remove the nteracton term from the model. In the example above, f we found that the B nteracton was not sgnfcant (p >.5), we could remove any terms ncludng t and ts df would contrbute to the wthn-subjects error term, whch mght ncrease power to detect effects of B (see p. 5). 6.5 Contrasts See Howell (997, pp ); Myers & Well (995, chapter 6) Lnear contrasts Lnear contrasts are comparsons between lnear combnatons of dfferent groups. Suppose we want to know whether students are more bored on Wednesdays than other weekdays, because Wednesday s statstcs day, and whether they re more bored on weekdays than weekends. We could measure ther boredom on all days of the week, and use DayOfWeek as a factor (wth 7 levels) n an NOV. If ths turned up sgnfcant, we would know that all days were not the same but t wouldn t answer our orgnal questons. We can do that wth lnear contrasts. In general, a lnear contrast s a lnear combnaton of a set of treatment means. Each mean µ j s weghted by a weght w j : L w µ + w µ + + w k µ w µ such that w j j In our example, suppose µ s the Monday mean, µ s the uesday mean, and so on. Our Wednesdays versus other weekdays queston can be wrtten as a lnear contrast: k j j j

76 6: dvanced topcs 76 or µ L Mon + µ ue + µ 4 hu + µ Fr µ Wed L + µ Mon + µ ue µ Wed + µ hu + µ Fr + µ Sat µ Sun Equvalently (multply everythng up to get whole numbers): L + µ + µ Mon + µ ue 4µ Wed + µ hu + µ Fr + µ Sat If the Wednesday mean s the same as the mean of the other weekdays, we expect that L. So our null hypothess s that L. If a statstcal test rejects ths null hypothess (shows that L devates from more than chance alone would predct), we would conclude that Wednesdays were dfferent from other weekdays. Our weekdays versus weekends queston could be wrtten as a dfferent lnear contrast: L + µ Mon + µ ue + µ Wed + µ hu + µ Fr µ Sat µ gan, f the null hypothess (weekdays the same as weekends) s true, the expected value of L s. Comparsons between ndvdual pars of means can also be accomplshed wth lnear contrasts for example, Sunday versus Monday (the back to work effect?): L + µ µ Mon + µ ue + µ Wed + µ hu + µ Fr µ Sat Sun Sun Sun For any contrast, SS contrast L w j n j j ll lnear contrasts have df per contrast. he sgnfcance test of a contrast s gven by F MS contrast /MS error ype I error rates wth planned contrasts If we ran parwse comparson post hoc tests on our days-of-the-week example, 7 we d make C parwse comparsons, so f we used α.5 per comparson, our famlywse α FW would be a huge.66. We d run the rsk of falsely declarng all sorts of dfferences sgnfcant. But our experment was only desgned to answer three questons: Wednesdays v. other weekdays, weekdays v. weekends, and Sundays v. Mondays. So f we only ask these questons, whch we had n mnd a pror, we could never declare the Monday v. uesday dfference sgnfcant. sk fewer questons, less chance of a ype I error. In general, the methods of controllng for ype I errors are the same n prncple for a pror and post hoc tests. he dfferences are smply () that we generally ask fewer questons a pror, and () when we perform post hoc tests we often focus on the dfferences that look bggest whch s logcally equvalent to performng all possble comparsons (vsually) and then selectng the bggest for statstcal testng. Snce ths has a hgh lkelhood of a ype I error, such data-guded post hoc tests must be corrected as f we were makng all possble comparsons (because actually we are). s Myers & Well (995, p. 79) put t, the effectve sze of a famly of post hoc contrasts s determned not by the number of contrasts actually tested but by those that concevably mght have been tested, had the data suggested t was worth dong so. When we specfy n advance (a pror) whch comparsons we re nterested n, we can specfy the ype I error rate per contrast (EC or α) or per famly of contrasts (EF or α FW ). What should consttute a famly of contrasts? ll the contrasts an exper-

77 6: dvanced topcs 77 menter ever runs? ll that are publshed n a sngle paper? Most people would say no; although that would result n a very low ype I error rate, t would lead to a hgh ype II error rate (low power) mssng real dfferences. here are two serous canddates for a famly (Myers & Well, 995, p. 78). hey are () all the contrasts made n a sngle experment; () all the contrasts assocated wth a sngle source of varance n a sngle experment. Suppose your experment has three factors,, B, and C. By the frst crteron, all contrasts n your B C desgn together consttute one famly. By the second crteron, there are seven famles (nvolvng, B, C, B, C, BC, and BC). Myers & Well (995, p. 78) recommend the second crteron as a reasonable compromse between ype I and ype II errors. Once you ve decded how many contrasts are n a famly, you can reduce your EC (α), or ncrease your p values, to obtan the desred EF (α FW ). For example, you could use the Bonferron or Sdak correctons dscussed above; these are smple (though the Bonferron s over-conservatve, so I prefer the Sdak). If you run k contrasts that are ndependent (orthogonal, see p. 77), α FW ( α) k, so the Sdak correcton s spot on. If your contrasts are not ndependent, α FW < ( α) k (Myers & Well, 995, p. 77) but t s hard to calculate α FW exactly, so just use the Šdák or Bonferron correcton and at worst your tests wll be conservatve. Planned contrasts may be conducted whether or not the overall F tests from the NOV are sgnfcant (Myers & Well, 995, p. 79). In fact, you could run them nstead of the usual NOV, but you are recommended to run the NOV too (Myers & Well, 995, pp. 79, 96). Why? () Because our theores are rarely good enough that we are wllng to forgo checkng whether unantcpated effects are present n the data wth post hoc tests, sutably controlled for ype I error. () he NOV carres addtonal nformaton, for example about the effect sze; see p. 97. Note also that the NOV may gve a dfferent result from a famly of post hoc tests, snce the power of the NOV s that of the maxmum contrast (Myers & Well, 995, p. 96), whch may not be obvous or nterestng (e.g. t may reflect a lnear combnaton of groups that you wouldn t have thought about n advance, such as.3 Mon +.7 ue.4 Wed.6 Sat) Orthogonal contrasts Contrasts are orthogonal f the questons they ask are ndependent. hs s one set of 6 orthogonal contrasts for our days-of-the-week example, showng how you can break down a set of means nto a set of orthogonal contrasts: (Mon, ue, Wed, hu, Fr) v. (Sat, Sun) (Mon, ue) v. (Wed, hu, Fr) (Sat) v. (Sun) (Mon) v. (ue) (Wed) v. (hu, Fr) (hu) v. (Fr) ll these are ndependent of each other. But these two are not ndependent: (Mon) v. (ue) (Mon) v. (Wed) here are many possble sets of orthogonal contrasts (some of them nvolvng odd fractonal combnatons of day means, whch mght not be very meanngful expermentally!). For any complete set of orthogonal contrasts, SS treatment SScontrast, and df treatment dfcontrast. So for our days-of-the-week example, we would need 6 orthogonal contrasts for a complete set; the set of 6 shown above s one complete set. Formally, two contrasts L () µ and L () µ are orthogonal f, for w j j j w j j j j( ) w j () equal sample szes, w (Howell, 997, p. 36). he more general con- w j() w j() dton, for unequal sample szes, s (Myers & Well, 995, p. 76). n j j j

78 6: dvanced topcs 78 here s no reason that we should test only orthogonal contrasts we test the contrasts that ask the questons we re nterested n Lnear contrasts n SPSS In SPSS, to run lnear contrasts other than very specfc ones (such as comparng all groups separately to the last one), you need to specfy the desgn n syntax usng the /CONRS()SPECIL() or /LMRI command. For a between-subjects NOV of a dependent varable (depvar) wth one factor (Day, 7 levels), you can specfy your contrasts lke ths: UNINOV depvar BY day /CONRS (day)specal( ) /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /PRIN ES(LMRI) /DESIGN day. he /PRIN command makes your custom contrast matrx appear under the headng Custom Hypothess ests, followed by the results (sgnfcance values for each test), followed by the sum of squares for the contrast. In ths example you can see that contrast L s Wednesdays v. other weekdays, L s weekdays v. weekends, and L3 s Sundays v. Mondays. ll are sgnfcant n ths example. lternatvely, you can use the LMRI syntax, whch allows you to specfy any lnear combnaton of any number of factors or nteractons (SPSS,, pp ). It may help to read the GLM secton to understand ths (p. 84, especally p. 93 ). For our smple example the syntax would be: UNINOV

79 6: dvanced topcs 79 depvar BY day /LMRI "Wed_v_otherweekday" day /LMRI "weekday_v_wkend" day /LMRI "sun_v_mon" day - /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /PRIN ES(LMRI) /DESIGN day. or to put all the tests nto one matrx as before, UNINOV depvar BY day /LMRI day ; day ; day - /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /PRIN ES(LMRI) /DESIGN day. If you want to obtan separate sums of squares for each contrast (reasons for whch are gven below), you can use the verson wth several /LMRI commands you get one est Results box wth one sum of squares for each /LMRI command. (It s also possble to work out SS contrast from the contrast estmate L gven n the results and the weght coeffcents prnted n the L matrx, usng SS contrast, L w j n j but ths s rather a pan.) If you specfy nonorthogonal contrasts, lke ths: UNINOV depvar BY a /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /LMRI "contrast" a - + /LMRI "contrast" a - + /LMRI "bothtogether" a - + ; a - + /PRIN ES(LMRI) /DESIGN a. then you wll fnd that SS contrast + SS contrast SS bothtogether. For a dscusson of correlated (nonorthogonal) predctors, see above and pp. 7 and Contrasts n multfactor desgns an overvew he same prncples can be appled to any contrast, even nvolvng multple factors (Myers & Well, 995, pp ). Suppose we have two factors: therapy type (: control CON, analytc therapy, behavour therapy B, cogntve therapy C) and patent dagnoss (B: unpolar depresson D, schzophrena S, manc depresson M). We measure some sort of dependent varable. We fnd a man effect of, a man effect of B, and an B nteracton. We can therefore reject these null hypotheses: α CON α CON,D α B α C C,M βd βs βm αβ αβ We can ask further questons usng contrasts. Does the mean of control subjects dffer from the mean of all the therapy populatons? hat would be a sngle contrast: j

80 6: dvanced topcs 8 L : µ CON µ + µ 3 B + µ Call ths (control) versus (all other treatments) the treatment effect. Does the treatment effect vary over clncal populatons? hat would nvolvng seeng f three contrasts dffer: D M S µ µ µ µ CON CON,D CON,S CON,S µ µ µ µ,d,s,m C + µ B + µ C 3 + µ B,D + µ 3 + µ B,S + µ 3 + µ B,M + µ 3 H : D S M C,D C,S C,M hs s harder but possble (Myers & Well, 995, pp. 9-5); t nvolves testng a sum of squares based on the devatons of D, S, and M from the overall treatment effect. nd so on. n SPSS-based example of ths sort of thng s gven on p rend analyss: the effects of quanttatve factors rends Factors are categorcal varables. But some categores are qualtatve (male/female; bpolar/schzophrenc/control) and some are quanttatve (sesson //3, stmulus heght 7 cm/9 cm/ cm ). How can we ask quanttatve questons about the relatonshp between stmulus heght and our dependent varable? Well, f the predctor varables are contnuous (covarates), you can ask thngs lke s my dependent varable a lnear functon of the predctor? (smple NCOV lnear regresson, see p. 35) or s my dependent varable a quadratc functon of the predctor? (polynomal NCOV, see p. 88 ). But wth categorcal predctors (factors), you use trend analyss (see Myers & Well, 995, chapter 7; Howell, 997, pp ). Obvously, ths technque requres that the levels of the factor are n some sort of order. We can accomplsh ths usng contrasts but wth partcular weghts for our contrast coeffcents. For example, returnng to our days-of-the-week example, takng just the weekdays, we can ask: Do people get happer durng the week? lnear trend. re people happer n the mddle of the week? quadratc trend, an example of a non-lnear (curved) trend. Mon ue Wed hu Fr So for our lnear example, we could test the contrast L µ µ + µ + µ + µ Mon ue H Wed : L he contrast coeffcents shown above would be vald f () the values of the factor are equally spaced, as they are for days of the week, and () each mean s based on the same number of scores. If not, see below. hu Fr

81 6: dvanced topcs 8 One common approach to trend testng s to ask what set of trends explan the data well (Myers & Well, 995, pp. 9-6). Here we would be guded by our theores. Suppose (Myers & Well, 995, p. 4) we are performng a generalzaton experment; we tran subjects that an stmulus predcts electrc shock. We mght expect that an stmulus would elct a substantal skn conductance response, whch would generalze somewhat to 9 and 3 stmul, but less so to 7 and 5 stmul. hs would be an nverted-u-shaped curve, and such a curve can be descrbed by a quadratc equaton (y a + bx, where b < ). So the responses to 7/9//3/5 stmul mght be somethng lke /4/9/4/ unts. We mght also expect that larger stmul cause more of a response a straght lne relatonshp between stmulus sze and response, whch can be descrbed by a lnear equaton (y a + bx). So f ths were the only thng nfluencng respondng, respondng for the 7/9//3/5 stmul mght be somethng lke //3/4/5 unts. Overall, f these two effects are ndependent, we mght expect an asymmetrc nverted-u curve, the sum of the other two effects (y b + b x + b x ) n ths example, /6//8/6 unts. We can perform an NOV to ask f the stmul dffer. Suppose they do the effect of the factor s sgnfcant. We know that takng full account of our factor,, can explan a certan amount of varance: SS, such that SS total SS + SS error. pplyng Occam s razor, t s common to ask frst whether a straght lne (a lnear trend) can explan the data well. Suppose we obtan a sum of squares for our lnear contrast, SS lnear. We can see f ths accounts for a sgnfcant amount of varablty: F lnear MS lnear /MS error. So does the effect of nclude somethng over and above a lnear component? Well, SS SS lnear + SS nonlnear (and, of course, df df lnear + df nonlnear + df nonlnear ). So we can calculate an F test to see f there s anythng substantal n that nonlnear component: F df-nonlnear/df-error MS nonlnear /MS error. hs s an F test for the lack of ft of the lnear model (see also Myers & Well, 995, p. 4) we know how much varablty accounts for overall; the queston s, what component of that s lnear and what s not. If ths sn t sgnfcant, our lnear model does a good enough job we stop. If t s, we can add n a quadratc trend. We would now have SS SS lnear + SS quadratc + SS hgher-order. We can test SS hgher-order to see f we should add any other predctors (SS cubc ) and carry on untl the leftovers no longer contan anythng sgnfcant. However, f your theory predcts certan components (e.g. lnear and quadratc), you shouldn t perform tests that you re not nterested n (Myers & Well, 995, p. 6). If you have a groups, then you can ft at most a polynomal of order a. So f you have 5 groups, you can only ft a lnear (x ), quadratc (x ), cubc (x 3 ), and quartc (x 4 ) trend; you haven t got enough data to ft a quntc (x 5 ) trend. So n general, the most complex polynomal equaton that can be ftted wth a groups s Y ˆ j b + b j + b j + + b p p j + + b a a j o apply ths technque, the trends (SS lnear, SS quadratc, ) have to be ndependent of each other, or orthogonal, so that ther sums of squares add up to SS. If () the values of the factor are equally spaced, as they are for days of the week, and () each mean s based on the same number of scores, coeffcents can easly be generated for a set of orthogonal polynomals (Myers & Well, 995, pp. 9-6 and able D7; Howell, 997, p. 39 and ppendx Polynomal). It s possble to derve coeffcents when the ns are not equal and/or the groups are not equally spaced (Myers & Well, 995, pp., 7-9) but t s much smpler to use standard lnear and/or nonlnear regresson technques, treatng the predctor as a contnuous varable (see pp. 8, 88, 35) rend analyss n SPSS In SPSS, polynomal contrasts can be done easly. In the example of an NOV wth a factor, specfy ths: UNINOV depvar BY a

82 6: dvanced topcs 8 /CONRS (a)polynomal /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /PRIN ES(LMRI) /DESIGN a. You can also specfy the contrast coeffcents by hand. For a factor wth fve levels, equally spaced, wth equal n, you could use: UNINOV depvar BY a /LMRI "a lnear" a - - /LMRI "a quadratc" a /LMRI "a cubc" a - - /LMRI "a quartc" a /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /PRIN ES(LMRI) /DESIGN a. o use the NOV dalogue box (nalyze General Lnear Model ), choose Contrasts, set the contrast for your factor to Polynomal, and clck change. o get the LMRI prntout, choose Optons Contrast coeffcent matrx. Other forms of contrast (and the lack-of-ft test descrbed above for s there anythng sgnfcant left over that needs to be accounted for? ) can be specfed by hand usng the syntax outlned above (p. 77). For one-way NOV, better output s obtaned from nalyze Compare means One-way NOV. Clck Contrasts Polynomal, and enter the order of the polynomal. You may also want Optons Means plot. he output looks lke ths: How trend analyss relates to multple regresson or polynomal NCOV rend analyss descrbed how well lnear, quadratc, etc., components ft the means of each group. Suppose s your factor (fve levels: 7, 9,, 3, and 5 stmul). Your dependent varable s Y: you have observatons per level ( subjects). You could treat as a factor, as we ve seen, or as a contnuous varable. If you performed a lnear regresson or NCOV wth your predctor varable havng one of the values 7, 9,, 3, and 5 for all the subjects, and your ndependent varable beng Y, you would fnd that your SS regresson was the same as the SS lnear from the NOV contrast. If you performed a lnear regresson wth your predctor varable havng the values 7, 9,, 3, and 5 and your ndependent varable beng Y, you would not obtan SS quadratc. rend analyss assumes that the centre of the quadratc functon s n the mddle of your groups. In our example, the mddle value s. So the quadratc trend analyss calculates y ˆ a + b( x ), not y ˆ a + bx. If you obtan SS regresson wth (x ) as your predctor, you wll obtan SS quadratc from the NOV contrast. You ll also obtan the same answer f you use the quadratc trend coeffcents (,,,, ) as your predctor (x ) values. You d thnk that SS cubc s the SS regresson you obtan wth the regresson 3 model y ˆ a + b( x ). But the cubc trend analyss coeffcents for fve

83 groups are (,,,, ), so that s what you need to use as your predctor values to obtan SS cubc. I d ntally thought they d be somethng lke ( 8,,,, 8) but the problem s that these values are not orthogonal to the other (lnear, quadratc) components. Specfcally, ( 8,,,, 8) s not orthogonal to the lnear component. If you use (,,,, ) your cubc predctor values, you do obtan SS cubc. hs s the cubc component over and above any lnear component. If you put all these predctors nto a multple regresson, you get the correct SS for each component as long as the predctors are orthogonal; otherwse, they start to reduce each other s SS (see pp. 7 and 97). Each of the SSs should be compared to the overall MS error from the NOV to get the same F values for all the components. he multple regresson approach can never measure the group means for dfferent values of n the way the NOV does, so t can never measure the lack of ft. Its SS errormultple-regresson reduces towards SS error-nov as you put n more predctors and the predcton gets better. o work out whether t s worth puttng another predctor n, you would have to compare the multple regresson R values for models wth and wthout the extra predctor (see p. 86 ). hs s one advantage of trend analyss you begn by knowng how much varablty the group means of your factor account for (whch the multple regresson doesn t), and you try to work out what polynomal components contrbute to that. 6: dvanced topcs 83

84 6: dvanced topcs How computers perform complex NOVs: the general lnear model (GLM) 6.7. he basc dea of a GLM, llustrated wth multple regresson Followng Howell (997, p. 567), suppose we want to solve a multple-regresson equaton to predct Y wth three predctors (varables,, and 3 ). Our equaton s: Y ˆ b + b + b + b or, wrtten out for an ndvdual observaton: Y + e b + b, + b, + b3, 3 where stands for a partcular observaton (labelled from to n) and e s the error assocated wth each observaton. We could wrte that usng vector (matrx) notaton (see revson chapter on matrces, p. 96): y b + bx + bx + b3x 3 + where y, x, x, x 3 are n vectors of data, e s an n vector of errors, and b s an n vector whose elements are the ntercept. hs can be further reduced to y b + e where there are p predctor varables, s an n (p + ) matrx of predctors, the frst column of whch contans only ones, and b s a (p + ) matrx of regresson coeffcents lke ths: 3 3 e y,,, n,,,, n,,3,3,3 n,3 b e b + e b b3 en Solvng a multple regresson equaton then becomes the problem of solvng y b + e for b so as to mnmze the sum of squares of the resduals, ( Yˆ ) or e. When ths s solved, b contans the correct regresson coeffcents. hree thngs are worth notng. Frstly, the multple regresson coeffcent R s the correlaton between Y and Ŷ, and ts square s the proporton of varablty accounted for by the overall regresson: Y R SS regresson SSY Secondly, the contrbuton of ndvdual predctors may be easy to specfy (f the predctors themselves aren t correlated, n whch case r represents the each predctor proporton of total varaton explaned by each predctor and R reach predctor ) or rather trcky to specfy (f the predctors are correlated); see Myers & Well (995, pp ). nd just as r for a sample can be adjusted (to r adj ) to provde a better estmate of ρ for the underlyng populaton, R can also be adjusted accordng to the sample sze (Myers & Well, 995, p. 58-9) (see p. 98). Other ssues regardng multple regresson are dscussed by Howell (997, ch. 5).

85 6: dvanced topcs 85 hrdly, the method of solvng ths matrx equaton s pretty damn complcated when there are several predctors. It s llustrated for lnear regresson (and even more complcated cases) at and a general proof s gven on p. 4, but we ll just leave SPSS to do t for us Usng a GLM for smple NOV: the desgn matrx How can we represent NOV n ths way? Suppose we take our old favourte, the one-way NOV wth a sngle between-subjects factor,. Our equaton for ths s Y µ + τ + ε j j where τ s the effect of level of factor. hs symbol τ represents τ, τ, τ 3 τ a (f there are a levels of factor ) but for one subject we are only nterested n the contrbuton of one level of. We can accomplsh ths wth somethng called a desgn matrx. he desgn matrx,, wll have a + columns and as many rows as there ae subjects. Suppose (after Howell, 997, p. 567) there are 6 subjects, 3 levels of, and subjects per level. hen our desgn matrx looks lke ths (the S or Subject column s purely for explanaton and sn t part of the matrx): S µ So subjects and experenced treatment, subjects 3 and 4 experenced treatments, and subjects 5 and 6 experenced treatments 3. ll subjects experenced the effect of the overall mean, so the frst column s full of ones. We can now defne our treatment matrx and wrte the whole thng n matrx form: y y τ + e e µ e τ e + τ e τ 3 e e,,,, 3, 3, Solvng ths equaton for τ so as to mnmze e gves us the treatment effects (µ, τ, τ, τ 3 ) we re after. However, for practcal use, t s common to alter the desgn matrx slghtly. Frstly, the µ column has no varance, so t can t go nto a standard multple regresson analyss, so we remove t. Secondly, the 3 column s redundant: f a subject sn t n or, we know t s n 3 (.e. there are only df for ), so we remove that too. Fnally, to make our treatment effects matrx gve treatment effects that are relatve to the overall mean (µ), the mean of each column must be zero (correspondng to the NOV requrement that τ ). We can acheve ths by scorng a subject n column f the subject s a member of treatment, scorng f the subject s a member of the last (ath) treatment, and scorng otherwse. (hs s sometmes

86 6: dvanced topcs 86 called sgma-restrcted parameterzaton, snce the columns sum to zero, whle the orgnal form s called the overparameterzed model, snce t contans redundant nformaton. It s possble to analyse usng the overparameterzed model; see nyway, ths process gves us ths revsed desgn matrx, whch carres all the necessary nformaton: Example of a GLM for a one-way NOV So suppose we have these data (one datum per subject): o analyse them wth a GLM, we use a set of matrces lke ths (one row per subject): e τ Y e b Y e τ τ τ he regresson coeffcent matrx can be called b (as t was for multple regresson) or τ (as t was for NOV). he overall R wll represent total model SS SS, and testng t for sgnfcance s the same as testng the effect of for sgnfcance. he ntercept n the regresson model wll equal the grand mean (Howell, 997, p. 57-) GLM for two-way NOV and beyond Let s move up to a two-way NOV, wth between-subjects factors and B (Howell, 997, p. 57). Our full model s jk j j jk Y ε αβ β α µ We can easly deal wth the α and β j terms n a desgn matrx. o represent the nteracton, αβ j, we use the fact that an nteracton represents a multplcatve effect of

87 6: dvanced topcs 87 the two varables. Let s start wth a desgn. Our desgn matrx, once coded usng sgma-restrcted parameterzaton, would look lke ths: a b ab a b a b B B hs matrx has one row per B combnaton, but n actual use we d have to replcate the rows so that there was one row per subject. So f there were fve subjects n the a b condton, for example, there d have to be fve rows whose coeffcents were [ ]. In ths matrx the frst column represents the man effect of, as t dstngushes those subjects who receved treatment and those who receved. he second column represents the man effect of B, dstngushng B from B. he thrd column s the B nteracton. Its elements are obtaned by multplyng the correspondng elements of the frst two columns. s always, we have as many columns per effect as we have degrees of freedom for that effect (df ; df B ; df B ). here are no entres because wth only two levels of each varable, a subject s ether n the frst or the last ( ) level. Now consder a 3 factoral ( B 3 ). We now have df, df B, and df B. So for the full model, we obtan the followng matrx (agan, we d need to ensure that we had one row per subject n the real thng ): a b a b ab 3 a b a b a b 3 B B B B hs smply apples the prncples outlned above for the, B, and B columns; the B column s the product of the column and the B column, whle the B column s the product of and B. Runnng an NOV lke ths gves us an overall R. Snce we know that SS regresson SS model SS Y R SS + SS B + SS B, and SS resdual SS error SS Y ( R ), we can calculate our SS model and SS error, we know our df model ( df + df B + df B ) and df error, and therefore we can calculate an F test for the whole model ( MS model /MS error ). However, ths doesn t tell us what proporton of the effect s attrbutable to, B, or B. o partton the varance, we must recalculate the regresson for a number of reduced models. We mght call the sum of squares for the full model that we ve just looked at SS regresson α,β,αβ. If we dropped the nteracton columns (B and B ), we d be deletng the predctors contanng nformaton about the nteracton but we d retan the predctors contanng nformaton about α and β; we d call the resultng sum of squares SS regresson α,β. If we only used the, B and B columns, our model would only account for α and αβ; we d obtan SS regresson α,αβ. If we only used the B, B, B and B columns, our model would only account for β and αβ; we d obtan SS regresson β,αβ. Once we ve calculated these, we can say that SS SS SS B B SS SS SS regresson regresson regresson α,β,αβ α,β,αβ α,β,αβ SS SS SS regresson regresson regresson α,β β,αβ α,αβ For example, f the nteracton term accounts for any of the varance n Y, then removng the nteracton term should lead to a decrease n the varaton n Y account-

88 6: dvanced topcs 88 able for; that decrease s equal to the varaton attrbutable to the nteracton. nd so on for the other terms. Note that f the predctors are n some way ntercorrelated, these sums of squares may not add up to SS total (see secton above on dsproportonate cell means wth a Venn dagram, p. 7); that s fne (Howell, 997, p ). hs method s the one that assgns SS area t, SS B area x, SS B area z n the Venn dagram above (see p. 7 ), whch s often what you want (Myers & Well, 995, p.55). Fnally, to test these effects (to obtan F statstcs for the effects of, B, and B), we need to know how to compare one model to another. nd ths s very smple (Myers & Well, 995, p. 44 and 5-4; Howell, 997, p.578). We can use any of the followng equvalent statements. If we have a Full and a Reduced model (wth f and r predctors respectvely), F( df df ), error(r) F( df df ), model(f) error(f) dferror(f) model(r) dferror(f) F ( SS SS ) ( df df ) error(r) SS error(f) error(f) df error(r) error(f) error(f) ( SS SS ) ( df df ) f r, N f model(f) model(r) SS error(f) df ( N f )( R f Rr ) ( f r)( R ) f model(f) error(f) model(r) he second formulaton s perhaps the clearest from the pont of vew of NOV; the thrd s the most useful when you have a multple regresson coeffcent R for each model. So to test the effect of, we calculate the full model to obtan SS regresson α,β,αβ, a reduced model to obtan SS regresson β,αβ, and test the dfference between them as above. But ths smplfes a bt for example, take the effect of : SS SS SS regresson α,β,αβ regresson(full) SS SS regresson β,αβ regresson(reduced) F( df df ), model(f) model(r) F df, df df error(f) error(f) ( SS SS ) ( df df ) model(f) SS SS error(f) MS MS error(f) df df model(r) SS error(f) error(f) df model(f) error(f) model(r) n overvew of GLM desgns We ve seen that a one-way NOV uses ths desgn matrx: 3 hs form of the matrx keeps the frst grand mean column ( ) but uses sgmarestrcted codng for the factor. s usual, the duplcaton of rows necessary to get one datum, one row, s not shown f there were one subject (S) n condton, two subjects (S, S3) n condton, and one subject (S4) n condton 3 that would make the fnal matrx look lke ths:

89 6: dvanced topcs 89 3,S4,S3,S,S If we used the overparameterzed model to represent, the matrx s smpler. hs s the reduced form (gnorng the fact that we ll eventually need one row per subject): 3 two-way NOV wth no nteracton terms mght look lke ths (left-hand verson n sgma-parameterzed form; rght-hand verson n overparameterzed verson): B B B B or B B B B two-way NOV wth the usual nteracton term looks lke ths, wth an 3 column (the nteracton term) that s the product of the () and (B) columns: 3 B B B B or B B B B In the overparameterzed form, there s a grand mean column, then two columns for the two levels of, then two columns for the two levels of B, then four columns for the possble values of the B nteracton. In a fractonal factoral desgn, columns are omtted from a full factoral desgn. We saw an example above, n whch the nteracton was omtted from a factoral desgn. Smlarly, you mght choose to run a NOV but to gnore the 3-way nteracton. he approprate matrx s shown below (overparameterzed verson); t has grand mean column, columns for, columns for B, columns for C, 4 columns for B, 4 columns for C, and 4 columns for BC. C B C B B C B C C B C B B C B C In a nested desgn, varablty due to one factor s nested wthn varablty due to another factor. For example, f one were to admnster four dfferent tests to four school classes (.e. a between-groups factor wth four levels), and two of those four classes are n school, whereas the other two classes are n school B, then the levels of the frst factor (four dfferent tests) would be nested n the second factor (two df-

90 6: dvanced topcs 9 ferent schools). In the desgn, nested varables never appear as man effects. For example, f we have a factor (3 levels) and a factor B ( levels) nested wthn, our overparameterzed matrx has one grand mean column, 3 columns for, and 6 columns for the effect of B nested wthn [ B/ or B() ]. B B B B 3 3 B B Overparameterzed models are always used to represent nested desgns, as the sgma-restrcted codng method has dffculty dealng wth the desgn (see smple regresson desgn, wth a sngle contnuous predctor varable, s easy to code. If there were three Y data ponts (dependent varable) and the correspondng values of the predctor varable were 7, 4, and 9, then the desgn matrx for the regresson Y b + b would be: smple quadratc regresson such as Y b + b would be coded smply by squarng the relevant values: Multple regressons, such as Y b + b P + b Q + b 3 R, are coded just as smple regressons. In a factoral regresson desgn, combnatons (products) of the predctors are ncluded n the desgn. If the predctors are P and Q, then the full factoral desgn would nclude P, Q, and ther nteracton (P by Q), represented by the product of P and Q scores for each case. So the equaton would be Y b + b P + b Q + b 3 PQ. Factoral regresson desgns can also be fractonal, n whch you omt some of the hgher-order effects from the desgn. n example would be a desgn wth three predctors that omtted the three-way nteracton: Y b + b P + b Q + b 3 R + b 4 PQ + b 5 PR + b 6 QR. Polynomal regressons contan man and hgher-order effects for the predctors but do not nclude nteractons. For example, the seconddegree polynomal desgn for three predctors would nclude man (frst-order) effects, quadratc (second-order) effects, but not nteractons: Y b + b P + b P + b 3 Q + b 4 Q + b 5 R + b 6 R. here are many other possble desgns. nalyss of covarance refers to a desgn contanng both categorcal predctors (factors) and contnuous predctors (covarates). radtonally, however, the term has referred specfcally to desgns n whch the frst-order effects (only) of one or more contnuous predctors are taken nto account when assessng the effects of one or more factors. For example, suppose a researcher wants to assess the nfluence of a factor wth 3 levels on some outcome, and measurements on a contnuous predctor C, known to covary wth the outcome, are avalable. If the data are:

91 6: dvanced topcs group C then the desgn matrx would be or In the left-hand (sgma-restrcted) model, the equaton s Y b + b + b + b 3 3 and the coeffcents b and b 3 represent the effects of, controllng for the effects of C. he b coeffcent represents the effects of C controllng for. hs tradtonal analyss s napproprate when the categorcal and contnuous predctors nteract n nfluencng the dependent varable. he approprate desgn s the separate slope desgn, whch ncludes the factor covarate nteracton. For the stuaton above, the overparameterzed matrx that ncludes the man effect of and the C nteracton would be: Separate slope desgns omt the man effects of C. Overparameterzed matrces are always used for separate slope desgns, snce the sgma-restrcted model runs nto problems ( he homogenety of slopes desgn can be used to test whether the covarate and factor nteract, and thus whether the tradtonal NCOV or the separate slope desgn s better. hs one does nclude the man effect of C: Mxed NOV and NCOV models are those that contan random effects, rather than fxed effects, for one or more factors. he dfference s only n how effects are tested. When computers perform tests for desgns that nclude random (rather than fxed) factors, they have to work out the approprate error term for every effect n the model. In a fxed-effect desgn, between-subjects effects are always tested usng the mean squared resdual as the error term. But n mxed-model de-

92 6: dvanced topcs 9 sgns, between-subjects effects are tested usng relevant error terms based on the covaraton of random sources of varaton n the desgn. Computers do ths wth somethng called the denomnator synthess approach of Satterthwate (946); detals are at [covers much GLM theory] Remember, a mean square qualfes as an error term for testng an effect f ts E(MS) matches the E(MS effect ) n all respects except the null-hypothess component (Keppel, 99, p. 568). Wthn-subjects (repeated measures) desgns can be analysed by codng Subject as a set of columns (Myers & Well, 995, pp ). If there are n subjects, there must be n S columns (sgma-restrcted parameterzaton form of the matrx) or n columns (overparameterzed form); smlarly, any nteractons nvolvng S can be coded. Wthn-subjects (repeated measures) desgns can also be analysed by constructng new dependent varables for example, f subjects are tested at tme and tme, a new dfference between the two tmes varable can be constructed and analysed. hese technques can be extended to multple levels of a wthn-subjects factor and multple factors usng specal technques based on multvarate analyss (see below), or by consderng Subjects as a (random) factor n ts own rght and workng out the relatonshp between the other factors. For example, a very common example s a desgn wth one between-subjects factor and one wthn-subjects factor, wrtten (U S); varaton due to subjects s nested wthn varaton due to (or, for shorthand, S s nested wthn ), because each subject s only tested at one level of the between-subjects factor. he dsadvantage wth the latter technque s that t does not take account of the potentally major problem of correlaton between dfferences between levels of a wthn-subjects factor, known as the sphercty problem (see below and p. 5 ) hnt at multvarate analyss: MNOV he Y matrx, so far an n vector of n observatons of a sngle Y varable, can be replaced by an n m matrx of n observatons of m dfferent Y varables. In ths case, the b vector smlarly has to be replaced by a matrx of coeffcents. he advantage s that you can then analyse lnear combnatons of several dependent varables, whch may themselves be correlated; one applcaton s to measure the strength of the relatonshps between predctor and dependent varables ndependent of the dependent varable nterrelatonshps. For example, f we gve students one of two textbooks and measure ther performance on maths and physcs (two dependent varables), we mght want to ask whether the textbooks affected performance, and f so, whether a textbook mproved maths, physcs, or both yet students performance on maths and physcs tests may be related. Some of the theory s dscussed at [general GLM theory] multvarate approach can also be used for wthn-subjects (repeated measures) desgns. he bonus s that the sphercty problem (q.v.) s bypassed altogether. Essentally, the problem of sphercty relates to the fact that the comparsons nvolved n testng wthn-subjects factors wth > levels may or may not be ndependent of each other, and f they re not, then the NOV results wll be wrong unless we account for ths. For example, f subjects learn some materal and are tested at tmes,, and 3, then subjects who learn most between tme and tme (contrast: tme tme ) may learn least between tme and tme 3 (contrast: tme 3 tme ), so the two contrasts are not ndependent. NOV assumes that all contrasts are ndependent (orthogonal). It s easy to see what that means f you had a factor : male or not and a factor B: female or not f you entered both factors nto an NOV, both factors would account for equal varance (snce they ask the same queston are not orthogonal) and f you parttoned out ths varance you d get the wrong answer

93 6: dvanced topcs 93 (snce you d be parttonng out the same thng twce). hs s the problem that wthn-subjects contrasts can run nto. Correcton procedures such as the Greenhouse Gesser and Huynh Feldt procedure attempt to deal wth ths. But a multvarate analyss automatcally deals wth correlatons between dependent varables, so you don t have to worry about the problem. Sometmes MNOV can t be used because t requres a bt more data. Sometmes repeated-measured NOV and MNOV gve dfferent answers but ths means that the dfferences between levels of the repeated-measures factors (e.g. tme v. tme ; tme v. tme 3) are correlated across subjects n some way, and that may tself be of nterest Lnear contrasts wth a GLM GLMs make t easy to specfy lnear combnatons of effects to test as contrasts. For example, f you had measured subjects on each of the 7 days of the week, and you wanted to ask whether the dependent varable was dfferent on weekdays and weekends, you could use the contrast Mon 5 ue Wed hu Fr Sat + Sun hs contrast would be zero f the mean weekend score and the mean weekday score were the same, so t s an approprate contrast. If your desgn matrx looked lke ths: Mon ue Wed hu Fr Sat Sun then a sutable contrast matrx mght look lke ths: L hs would be equally approprate: L [ 5 5] j It works lke ths: you solve the usual GLM, Y b + e, to fnd the parameter estmates b. hen you calculate L Lb to estmate the value of your contrast. You can then test t for sgnfcance; ts sum of squares s gven by the usual L SS contrast where w j are the weghts n the L matrx and n j are the corre- w j n j spondng group szes, and MS contrast SS contrast s compared to MS error. For detals, see GLMs n SPSS If you run an NOV n SPSS, how can you see the desgn matrx? SPSS doesn t show you ths drectly, but t wll show you parameter estmates that s, the b matrx. nd t labels each row of the b matrx wth a descrpton of the relevant column of the correspondng desgn matrx (). o obtan ths, ether use the /PRIN PRMEER

94 6: dvanced topcs 94 opton or, from the menus, Optons Parameter estmates. You ll get somethng lke ths: he desgn matrx s specfed by the /DESIGN command try clckng Paste nstead of OK when youre about to run any NOV and you wll see the /DESIGN command t was gong to use. Smlarly, f you add /PRINES(LMRI), you see a contrast for every term n the desgn matrx, whch shows you the columns present n the desgn matrx. For example, wth a two-way NOV, B, you get ths: Rather than smply usng the null hypothess Lb, SPSS can also test custom hypotheses wth non-zero expected values for the contrast: Lb k, or for multple contrasts smultaneously, wth more than one row for the L matrx, Lb K. hs can be specfed wth the /LMRI and /KMRI subcommands (SPSS,, p ). For an 3 B 3 NOV, the default contrasts look lke ths: he contrasts shown above the default contrasts that examne the man effects of and B and the B nteracton could be specfed by hand lke ths: GLM DEPVR BY B /LMRI "Intercept" all /3 /3 /3 /3 /3 /3 /9 /9 /9 /9 /9 /9 /9 /9 /9

95 6: dvanced topcs 95 /LMRI "" /LMRI "B" a - b a*b /3 /3 /3 -/3 -/3 -/3; a - b a*b /3 /3 /3 -/3 -/3 -/3 a b - a*b /3 -/3 /3 -/3 /3 -/3; a b - a*b /3 -/3 /3 -/3 /3 -/3 /LMRI "xb" a b a*b - - ; a b a*b - - ; a b a*b - - ; a b a*b - - /DESIGN, B, *B. (You have to use /3 rather than.333 to avod roundng errors; f the coeffcents don t add up to for each contrast matrx you won t get an answer.) Havng seen how the general technque works, we can test advanced contrasts: /LMRI "B vs B at " B - *B - hs would be more powerful than just analysng the data and applyng a B v. B contrast the MS contrast would be the same, but the contrast specfed above uses the overall (pooled) MS error, makng t more powerful (more error df). /LMRI "B vs (B+B3)" B -/ -/ *B /3 -/6 -/6 /3 -/6 -/6 /3 -/6 -/6 Fnally, a really complex one. Suppose B s a control condton and B and B 3 are two dfferent selectve serotonn reuptake nhbtor drugs. herefore, (B ) versus (B and B 3 ) mght represent an SSRI treatment effect (call t ) that we re nterested n. Suppose that and are depressves and schzophrencs. If we want to compare the SSRI effect between depressves ( ) and schzophrencs ( ), we could follow ths logc:

96 6: dvanced topcs 96 H : µ B, µ B, µ µ H B, B, H + µ : : B3, µ µ µ B, B, + µ B, + µ B3, B3, µ + B, + µ B3, Havng calculated our null hypothess, we can specfy the contrast: /LMRI "(B vs (B+B3)) at versus " *B -/ -/ - / / Hope I ve got that rght; t seems to work.

97 6: dvanced topcs Effect sze Whether a contrbuton s sgnfcant or not does not tell you whether that sgnfcant contrbuton s large. If you have hgh power (large n), you may be able to measure sgnfcant small effects. nd f you have lower power (small n), you may mss (fal to declare as sgnfcant) large effects. o ask about effect sze s to ask not just whether the effect of a predctor varable s statstcally sgnfcant, but how bg (mportant) ts effect s. In general, when we are predctng a dependent varable Y by one or more predctor varables, be they contnuous (NCOV, multple regresson) or dscrete (NOV factors), we can ask to what extent a gven term (man effect, nteracton, etc.) contrbutes to the predcton of the dependent varable. We ve already seen that ths can be complcated, especally f the predctors are themselves correlated effect sze s a farly complex topc (Wner, 97, pp ; Keppel, 99, pp , - 4, ; Myers & Well, 995, pp. -3, 5-56, 54-59; Howell, 997, pp , 46-49, ). We ll start by examnng effect sze n the context of multple regresson (predctng Y from,, and so on), because t s the smplest conceptually. In general, effect sze can refer to the sze of the change n Y that follows a certan change n a predctor (regresson slope) or the proporton of varaton n Y explcable by a predctor (equvalent to r n smple lnear regresson) Effect sze n the language of multple regresson remnder of what the sgnfcance of a predctor means ssumng you use the usual (SPSS ype III) way of parttonng sums of squares wth correlated predctors, the sgnfcance test of each predctor reflects whether that predctor contrbutes to the predcton over and above all the other predctors n the model (see also sectons on correlated predctors earler: p. 7 and p. 86 ). hs s not effect sze. Interpretng the effects of ndvdual predctors: the regresson slope, b he computerzed results wll gve us ndvdual slope parameters for each of the effects n our model (n SPSS, tck Optons Parameter estmates). Remember that a multple regresson equaton looks lke ths: Yˆ b + b + b + y b + e he parameters are the values of b. he frst, b, s the ntercept (grand mean). he others reflect the effects of all the other predctors. However, there are problems of nterpretaton of the ndvdual slope parameters b j (Myers & Well, 995, p. 5; Howell, 997, pp and ). It s temptng to thnk that f we were to change j by one unt, Y would change by b j unts ths would be true of smple lnear regresson (wth one predctor). However, a regresson coeffcent b j does not reflect the total effect of j on Y. Rather, t reflects the drect effect of j on Y the rate of change of Y wth j holdng all of the other varables n the equaton constant. If the varous predctors (,, ) are mutually correlated (known as collnearty or multcollnearty), t may often not make a great deal of sense to ask ths queston for example, f we were predctng car crash fataltes by drvers annual mleage and drvers annual fuel consumpton, t s not clear what t would mean to change annual mleage whle holdng fuel consumpton constant. When we ask about the consequences of changng j, we must be concerned not only wth the drect effect but also wth the ndrect effects the effects on Y that occur because of changes n the other varables. Gven a vald causal model, path analyss can be used to calculate the total effect (drect + ndrect effects) of changng a varable. However, f the model s ncomplete or nvald, we would have to establsh the ef-

98 6: dvanced topcs 98 fects of changng j expermentally, by manpulatng t wthout confoundng t wth the other varables, and observng the results. Standardzed regresson slope, β ( r) he standardzed regresson slope, β j, s smply the b j that you would obtan f both the dependent varable Y and the predctor j were standardzed that s, transformed so that they have a mean of and a standard devaton of (Howell, 997, pp. 44, 57-8, 544-6). If b.75, then a one unt ncrease n would be reflected n an.75 unt ncrease n Y. If β.75, then a one standard devaton ncrease n would be reflected n an.75 standard devaton ncrease n Y. It s easy to calculate β. If b j and s j are the regresson slope and standard devaton of a predctor j, then bjs β j s Bear n mnd that slopes are related to r: for smple lnear regresson, s b r s and so b r when both varables are standardzed (Howell, 997, p. 4), and β r at all tmes. However, wth multple predctors, the problem wth β j s just the same as for b j : t reflects the change n Y assocated wth a change n j holdng all other predctors constant, and f the predctors are correlated ths may not make much sense. Y Y j Overall R and R adj : how good s the whole model? he computerzed results of an NOV, NCOV, or other GLM wll gve an overall R, whch reflects the proporton of total Y varance predcted by all the predctors together,.e. SS regresson /SS total. (lternatvely, we could say that R s the correlaton between the dependent varable and the best lnear combnaton of the predctors.) R can also be adjusted (downwards) to gve R adj, a better estmate of the correspondng populaton parameter (Myers & Well, 995, p ; Howell, 997, p. 5), and SPSS wll do that automatcally. If there are N observatons and p contnuous predctors: N R adj ( R ) N p If you are usng predctors wth > df per predctor (e.g. factors wth > levels), you need a more general form of ths equaton, whch I beleve s R adj ( R df ) df ssessng the mportance of ndvdual predctors: total error r sempartal a good one Let s move on to a better measure (Myers & Well, 995, pp ; Howell, 997, pp , ). When we predct Y from p predctor varables, Y., p R (or smply R ) s the proporton of varablty n Y accounted for by the regresson on all p predctors. If the p predctors are not mutually correlated (Myers & Well, 995, p. 55), SS regresson can be parttoned nto nonoverlappng components from each of the predctors:

99 6: dvanced topcs 99 SS regresson SS r Y. Y. SS + SS Y + r Y. Y. + + SS SS Y Y. p + + r Y. p SS Y where SS Y. s the proporton of varablty of Y accounted for by the predctor j, and r Y. j s the correlaton between j and Y. Snce SS regresson R Y., p SS Y t follows that for uncorrelated predctors, R Y., p ry. + ry. + ry. p r If the predctors are correlated, we must use ths (Myers & Well, 995, p. 56): j Y. j R Y., p Y. j r b j ˆ σ j j ˆ σ where b j s the regresson coeffcent of j n the multple regresson equaton and σˆ and σˆ are the standard devatons of j and Y, respectvely. he ncrease n R j Y when s added to a regresson equaton that already contans s r Y.( ), the square of the sempartal correlaton coeffcent (Myers & Well, 995, p. 486 and 57). Here s a vsual nterpretaton, n whch the area of each crcle represents the total varablty of a gven varable: Y R R R Y. Y. Y., a + b b + c a + b + c Y.( ) r Y.( ) r c a You could also say that the sempartal correlaton ry.( ) s the correlaton of Y wth that part of that s ndependent of (Howell, 997, p. 58). In general, Y.( p, p) r + s the ncrease n R that follows from addng p+ to a regresson equaton that already ncludes,, p. hat s, Y., p+ Y.( p+, p) r R R R Y., p + r Y., p+ Y.( p+, p) RY., p hs would seem to be a useful measure. Howell (997, p ) agrees, statng that when the man goal s predcton rather than explanaton, ths s probably the best

100 6: dvanced topcs measure of mportance. If the computer package doesn t gve t drectly (and SPSS doesn t), t can easly be calculated (Howell, 997, p. 546): Y.(, everythng except p) r p F ( RY., N p where p s the total number of predctors, r Y.(, everythng except p) s the squared sempartal correlaton for predctor, F s the F test for predctor (use F t f your stats package reports t nstead), R Y., p s the overall R (wth predctor ncluded), and N s the total number of observatons. Note that ths means that the F statstcs n an NOV are n the same order as the the squared sempartal correlaton coeffcents (wthn an NOV, you could say that bgger F more mportant ). If you re usng factors as predctors (.e. predctors wth > df per predctor), I rather suspect that Howell s formula should be rewrtten lke ths: ) Y.(, everythng except p) r F ( R df Y., p error ) But f you re havng trouble workng out a formula, you can always fall back to the poston of runnng the NOV wth and wthout a partcular term, and calculatng the dfference between the two overall R values. Partal and sempartal correlatons It s easy to be confused by the dfference between partal and sempartal correlatons. We ve just seen what the sempartal correlaton s (Howell, 997, pp ). Let s go back to the Venn dagram: he squared sempartal correlaton coeffcent r Y.( ) s the proporton of the varablty n Y explaned by over and above what s explaned by. he squared partal correlaton coeffcent r Y. s the proporton of the varablty n Y explaned by relatve to that not explaned by. In our Venn dagram, the two look lke ths: Overall predcton of models Squared sempartal Squared partal RY. a + b c RY. b + c ry c ry..( ) c + d RY., a + b + c ry.( ) a a ry. a + d R d Y., Y. Y.( ) Suppose that R. 4, the squared sempartal r. and the squared partal Y. r.3. hat would mean that explans 4% of the varablty n Y f t s the only predctor, that explans % of the varablty n Y once has been taken nto account (sempartal), and that explans 3% of the varablty n Y that

101 6: dvanced topcs faled to explan (partal). hat would reflect ths stuaton (areas denote varablty; fgures are proportons of the total varablty of Y; sorry f t s not qute to scale): nother defnton of partal correlaton If r xy s the correlaton between and Y, then r xy z, the partal correlaton between and Y wth the effects of Z partaled out, s the correlaton between Z and Y Z, where Z ˆ s the resdual that results when s regressed on Z, and Y Z Y Yˆ s the resdual that results when Y s regressed on Z (Myers & Well, 995, p. 483). It s possble to obtan r xy z from the smple correlatons between each of the varables: r xy z r xy ( r r xz xz r yz )( r yz ) For example, suppose we look at 48 US states and measure populaton, motor vehcle deaths, and whether or not the state has enacted seat belt legslaton the last beng a dchotomous varable, but that s OK (Myers & Well, 995, p. 483). here s a postve correlaton between deaths and belt legslaton (+.39), whch mght seem worryng. However, r deaths,populaton +.98 and r belts,populaton larger states have more deaths, and larger states are more lkely to have seat belt legslaton. he partal correlaton r deaths,belts populaton.3, ndcatng a small but negatve relatonshp between seat belt laws and motor vehcle deaths once the effects of populaton have been partalled out. nother defnton of sempartal correlaton he sempartal correlaton coeffcent r y(x z) s the correlaton between Y and Z, where Z ˆ s the resdual that results when s regressed on Z. It too can be calculated from the smple regresson coeffcents: r y( x z) r xy r xz ( r r xz yz ) Effect sze n the language of NOV he effect sze n the context of NOV s the same thng as the effect sze n multple regresson (snce both are smply nstances of a GLM), but people tend to use dfferent termnology. helpful dscusson of some dfferent measures of effect sze s gven at web.uccs.edu/lbecker/spss/glm_effectsze.htm. Dfference between level means hs s smple. If you have a factor (e.g. Sex: Male/Female) and you establsh through an NOV that ts effect s sgnfcant, you have an nstant measure of ts effect: the dfference between µ male and µ female. You can extend ths approach to multple factors and to nteractons. For example, for the data shown below, we can state

102 6: dvanced topcs the effect szes very smply. Overall mean: he overall mean ( beng a male or female - or -year-old ) s 37 cm. Man effects: Maleness contrbutes +3.5 cm and femaleness contrbutes 3.5 cm (or, maleness contrbutes +7 cm compared to femaleness). Beng a -year-old contrbutes 33.5 cm; beng a -year-old contrbutes cm (or, beng a -year-old contrbutes +67 cm relatve to beng a - year-old). Interactons: f the overall mean contrbutes 37 cm, beng a male contrbutes +3.5 cm, and beng a -year-old contrbutes cm, we d expect - year-old males to be 74 cm, but n fact they re 77 cm, so the nteracton term (beng a male -year-old) contrbutes an extra +3 cm on top of the man effects. nd so on. Heght Male Female mean -year-old 4 cm 3 cm 3.5 cm -year-old 77 cm 64 cm 7.5 cm mean 4.5 cm 33.5 cm 37 cm Effect sze measures related to the dfference between means perhaps best to skp ths bt! here are lots of these, most desgned to facltate calculaton of power. For a stuaton wth two groups wth the same standard devaton, we can measure the dfference between means µ µ. We can standardze that by dvdng by the standard devaton to produce d, often called the effect sze : µ µ d σ hs number d can be combned wth knowledge of the sample sze n to calculate δ d n, whch n turn can be used to calculate power (Myers & Well, 995, pp. 3-6; Howell, 997, p. 6-6). Cohen (988) more or less arbtrarly called d. a small effect (the means dffer by. of a standard devaton),.5 a medum effect, and.8 a large effect. Smlar prncples can be appled to NOV (Howell, 997, p ), but the notaton s a bt dfferent. If there are k levels for a factor, the standardzed measure of effect sze s φ f σ σ treatment error ( µ µ ) k σ error hs can then be combned wth knowledge of the sample sze to calculate φ φ n, whch n turn can be used to calculate power. hs can be extended to factoral desgns (Myers & Well, 995, pp ). nd just as correlaton slopes b were related to r n the language of regresson, φ (also wrtten f) s related to η (see below) n the language of NOV (Wner et al., 99, p. 4): f η η δ and φ are also known as noncentralty parameters (Wner et al., 99, pp. 6-4; Howell, 997, pp., 334-5). hs refers to the fact that f there s an effect (f the null hypothess s false), the dstrbuton of F statstcs sn t the plan F dstrbuton (as t would be f the null hypothess were true), but a shfted (noncentral) F dstrbuton. he noncentralty parameters measure effect sze by how much the dstrbuton s shfted. ssessng the mportance of ndvdual predctors: η Eta-squared s gven by

103 6: dvanced topcs 3 SS η SS η represents the proporton of total varaton accounted for by a factor; equvalently, the proporton by whch error s reduced when you use the factor to predct the dependent varable (Howell, 997, p. 33). Eta tself, η, s called the correlaton rato (Wner et al., 99, p. 3), although η s also sometmes called the correlaton rato (Howell, 997, p. 33). If you only have one predctor, η R. If you have more than one predctor and they re correlated, η depends on how you calculate SS effect. ssumng you use the usual (SPSS ype III) method (see p. 7 ), the SS for predctor n the dagram below s area c, and SS total (SS Y ) s area a + b + c + d. effect total So for our usual (SPSS ype III) sums-of-squares method, the η for s SS η SS effect total c a + b + c + d c r Y.( ) so η s the squared sempartal correlaton coeffcent, t seems to me. If you calculate η by hand n SPSS, remember that what we normally refer to as SS total, ( y y), s labelled corrected total by SPSS. (Its total s y, whch we re not nterested n.) ssessng the mportance of ndvdual predctors: η partal not very helpful One measure of the mportance of ndvdual predctors s the partal eta-squared coeffcent, whch s somethng that SPSS gves you (tck Optons Estmates of effect sze). We ve just seen what η s (above). he partal eta-squared s an overestmate of the effect sze n an F test (SPSS,, p. 475). Specfcally, t s ths: η partal df SS dfeffect F F + df effect effect SS + SS error effect error term for that effect he top formula s from SPSS (, p. 475) and the second from web.uccs.edu/lbecker/spss/glm_effectsze.htm. I m not sure f t s partcularly useful, especally as the partal eta-squared terms sum to more than one ( η partal > ), whch s pretty daft. In terms of our Venn dagram, the η partal for s: SStreatment c η partal r Y. SS + SS c + d treatment error

104 6: dvanced topcs 4 so η partal s the squared partal correlaton coeffcent, t seems to me. herefore, I ll gnore t. nother one: ω When a factor predcts a dependent varable Y, omega-squared (ω ) for s defned as the proporton of the total varance n Y attrbutable to the effects of (Myers & Well, 995, p. 3). In general, the estmated ω, wrtten ˆω, s ˆ σ ˆ ω ˆ σ For a fxed (not a random) effect, ω s estmated by ˆ ω SS df MS error Y MS + SS (Formula from web.uccs.edu/lbecker/spss/glm_effectsze.htm.) For random effects, such as n wthn-subjects (repeated measures) desgns, the defnton of ˆω depends on the specfc NOV model (Myers & Well, 995, pp. 5-56; Howell, 997, pp ), and sometmes t cannot be estmated exactly (Myers & Well, 995, p. 54). total error nd another: the ntraclass correlaton ρ I he ntraclass correlaton coeffcent s a measure of assocaton between the ndependent and dependent varables for a random-effects model (Howell, 997, p. 334) (web.uccs.edu/lbecker/spss/glm_effectsze.htm); for an effect, t s ρ I MS MS MS + df MS error error he squared ntraclass correlaton, ρ I, s a verson of ω for the random model. Whch one to use? lthough η s perhaps the smplest, t does have a problem (Howell, 997, pp ). When t s appled to populaton data, t s correct; when appled to samples (as we normally do), t s based as a measure of the underlyng populaton effect sze. So ˆω s generally preferred when we want an estmate of the effect sze n the populaton the way t s calculated takes account of sample sze approprately (so ˆω wll always be smaller than η or η partal ). On the other hand, SPSS doesn t produce t, whch s a bt of a shame, and t s laborous to calculate by hand. So for a quck dea, η s perhaps easest. hs also has an advantage over η partal n that t s addtve (the η values sum to, whle the η partal values can sum to >) and s therefore perhaps easer to conceptualze and nterpret.

105 7: Specfc desgns 5 Part 7: specfc desgns For desgns 7, all factors other than subject factors are assumed to be fxed. If you wsh to use other random factors, see Myers & Well (995, p. 6) or just tell SPSS that s what you want and trust t to sort out the maths. Desgn (BS between-subjects; WS wthn-subjects) One BS factor Includes step-by-step nstructons for performng between-subjects analyss n SPSS Descrpton (n most economcal format; S subjects; cov subscrpt covarate) Between-subjects factor(s) or covarate(s) S wo BS factors B S, B Wthn-subjects factors(s) or covarates) 3 hree BS factors B C S, B, C 4 One WS factor (U S) U 5 wo WS factors (U V S) U, V 6 hree WS factors (U V W S) U, V, W 7 One BS and one WS factor Includes step-by-step nstructons for performng wthn-subjects (repeated measures) analyss n SPSS (U S) U 8 wo BS factors and one WS factor B (U S), B U 9 One BS factor and two WS factors (U V S) U, V Hgher-order desgns along the same prncples and summary of desgns 9 See text See text See text One BS covarate C cov S C cov (lnear regresson) One BS covarate and one BS factor C cov S C cov, 3 One BS covarate and two BS factors C cov B S C cov,, B 4 wo or more BS covarates (multple regresson) C cov D cov S C cov, D cov, 5 wo or more BS covarates and one or more BS factors e.g. C cov D cov B S C cov, D cov,, B, etc. 6 One WS covarate (C cov S) C cov 7 One WS covarate and one BS factor (C cov S) C cov 8 Herarchcal desgns See text (complex) See text (complex) See text (complex) 9 Latn square desgns See text (complex) See text (complex) See text (complex) grcultural desgns See text (complex) See text (complex) See text (complex)

106 7: Specfc desgns 6 7. One between-subjects factor lternatve names One-way NOV Completely randomzed desgn (CRD) Example Subjects are assgned at random to drug treatments,, or 3 (completely randomzed desgn; sngle factor wth three levels) and ther reacton tme s measured on some task (dependent varable). Does the drug treatment affect performance? researcher wshes to test the effectveness of four fertlzers (,, 3, 4). He dvdes hs feld nto sxteen plots (equvalent to subjects or replcatons ) and randomly assgns fertlzer to four replcatons, to four replcatons, and so on. Notes Model descrpton (S subjects) For two levels of the factor, ths s equvalent to an unpared (ndependent sample) t test. reatments (levels of the factor) are assgned at random to subjects (replcatons). For full detals, see Howell (997, chapter ). depvar S Model Yj µ + α + ε j where Y j s the dependent varable for subject j experencng level of the factor µ s the overall mean α s the contrbuton from a partcular level (level ) of the factor: α µ µ and α. he null hypothess s that all values of α are zero. ε j s everythng else (the unqueness of subject j n condton, error, ndvdual varaton, etc.): ε j Y j µ. We assume ε j s normally dstrbuted wth mean and varance σ e. Sources of varance nalyss of varance dscards constant terms (lke µ) and examnes the sources of varablty (varance). Wrtng ths n terms of sums of squares (SS), SS total SS + SS error where SS total s the total varablty, SS factor s the varablty attrbutable to the factor, and SS error s the error varablty (everythng that s left over). lternatvely, we could wrte SS total SS + SS S/ because our total varablty s made up of varablty due to factor, and varablty due to nter-subject dfferences wthn each level of ( S wthn, or S/ ). NOV table In all cases, the mean square (MS) s the sum of squares (SS) for a partcular row dvded by the degrees of freedom (d.f.) for the same row. ssumng the same number of subjects n for each level of factor, we have Source d.f. SS F a SS MS /MS error Error (S/) a(n ) SS error otal N an SS total where a s the number of levels of factor, N s the total number of observatons (subjects), and n s the number of subjects (or replcatons ) per level of factor. Note that the error s sometmes wrtten S/,.e. subjects wthn. SPSS technque Data layout:

107 7: Specfc desgns 7 depvar datum datum datum datum datum level_ level_ level_ level_ level_ Syntax: UNINOV depvar BY /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN. Usng the menus, choose nalyze General Lnear Model Unvarate. We now see ths: Our dependent varable s depvar; our (fxed) factor s : Once everythng else s OK, clck OK to run the analyss, or Paste to copy the syntax for the analyss to a syntax wndow.

108 7: Specfc desgns 8 7. wo between-subjects factors lternatve names Example Notes Model descrpton (S subjects) wo-way NOV Factoral NOV a b factoral NOV (where a and b are the number of levels of factors and B; e.g. 5 factoral ) Factoral, completely randomzed desgn NOV Subjects are assgned at random to a hgh-arousal () or a low-arousal () stuaton, and are also gven drug (B) or placebo (B) (completely randomzed desgn; factoral NOV). her performance s measured on a task (dependent varable). Does the arousal stuaton () or the drug (B) affect performance, and does the effect of the drug depend on arousal ( B nteracton)? factoral desgn s one n whch every level of every factor s pared wth every level of every other factor (Howell, 997, p. 4). depvar B S Model Yjk µ + α + β + αβ + ε j j jk where Y jk s the dependent varable n condton, B j for subject k µ s the overall mean α s the contrbuton from level of factor ( ): α µ µ and α. β j s the contrbuton from level j of factor B (B j ): β j µ µ and β j. αβ j s the contrbuton from the nteracton of level of factor and level j of factor B that s, the degree to whch the mean of condton B j devates from what you d expect based on the overall mean and the separate contrbutons of and B j ( the nteracton B),.e. αβ µ µ + α + β ). By ths defnton, αβ αβ. j B j ( j ε jk s everythng else (the unqueness of subject k n condton of factor and condton j of factor B, error, ndvdual varaton, etc.): ε jk Yjk ( µ j + α + β j + αβj ). By our usual assumpton of normal dstrbuton of error, ε jk s normally dstrbuted wth mean and varance σ e. B j j j j Sources of varance s before, we consder only the sources of varaton for the NOV analyss: SS total SS + SS B + SS B + SS error where SS total s the total varablty SS s the varablty attrbutable to factor SS B s the varablty attrbutable to factor B SS B s the varablty attrbutable to the nteracton SS error s the error varablty (everythng that s left over). hs s sometmes wrtten SS S/B (ndcatng varablty due to nter-subject varaton wthn B combnatons). NOV table In all cases, the mean square (MS) s the sum of squares (SS) for a partcular row dvded by the degrees of freedom (d.f.) for the same row. ssumng the same number of subjects n for each cell (combnaton of one level of factor and one level of factor B), we have Source d.f. SS F a SS MS /MS error B b SS B MS B /MS error B (a )(b ) SS B MS B /MS error Error (S/B) ab(n ) SS error

109 7: Specfc desgns 9 otal N abn SS total where a s the number of levels of factor, N s the total number of observatons (subjects), and n s the number of subjects (or replcatons ) per cell. SPSS technque Data layout: depvar B datum level_ level_ datum level_ level_ datum level_ level_ datum level_ level_ datum level_ level_ datum level_ level_ datum level_ level_ datum level_ level_ Syntax: UNINOV depvar BY a b /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN a b a*b. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter and B as between-subjects factors.

110 7: Specfc desgns 7.3 hree between-subjects factors lternatve names a b c factoral NOV (where a, b and c are the number of levels of factors, B, and C; e.g. 5 3 factoral ) Factoral, completely randomzed desgn NOV Example Subjects have ther prefrontal cortex destroyed () or not () or have a specal prefrontal cortex augmenter ftted (3), are assgned at random to a hgh-arousal (B) or a low-arousal (B) stuaton, and are also gven drug (C) or placebo (C) (completely randomzed desgn; 3 factoral NOV). her performance s measured on a task (dependent varable). Do factors, B, or C affect performance? Do they nteract? Notes Model descrpton (S subjects) depvar B C S Model Yjkl µ + α + β + γ + αβ + αγ + βγ + αβγ + ε j k j k jk jk jkl where jkl s the dependent varable n condton, B j, C k for subject l µ s the overall mean α s the contrbuton from level of factor : α µ µ β j s the contrbuton from level j of factor B: β µ µ γ k s the contrbuton from level k of factor C: γ µ µ αβ j s the contrbuton from the nteracton of level of factor and level j of factor B: αβ j µ B ( µ + α + β j ) j αγ k s the contrbuton from the nteracton of level of factor and level k of factor C: αγ k µ C ( µ + α + γ k ) k βγ jk s the contrbuton from the nteracton of level j of factor B and level k of factor C: βγ jk µ B C ( µ + β j + γ k ) j k ε jkl s everythng else (the unqueness of subject l n condton of factor and condton j of factor B and condton k of factor C, error, ndvdual varaton, etc.): ε jk Yjk ( µ + α + β j + γ k + αβ j + αγ k + βγ jk ). j k B j C k Sources of varance s before, we consder only the sources of varaton for the NOV analyss: SS total SS + SS B + SS C + SS B + SS C + SS B C + SS B C + SS error where SS total s the total varablty SS s the varablty attrbutable to factor SS B s the varablty attrbutable to factor B SS C s the varablty attrbutable to factor C SS B s the varablty attrbutable to the B nteracton SS C s the varablty attrbutable to the C nteracton SS B C s the varablty attrbutable to the B C nteracton SS B C s the varablty attrbutable to the B C nteracton SS error s the error varablty (everythng that s left over). hs s sometmes wrtten SS S/BC (ndcatng varablty due to nter-subject varaton wthn B C combnatons). NOV table In all cases, the mean square (MS) s the sum of squares (SS) for a partcular row dvded by the degrees of freedom (d.f.) for the same row. ssumng the same number of subjects n for each cell (combnaton of one level of factor, one level of factor B, and one level of factor C) we have

111 7: Specfc desgns Source d.f. SS F a SS MS /MS error B b SS B MS B /MS error C c SS C MS C /MS error B (a )(b ) SS B MS B /MS error C (a )(c ) SS C MS C /MS error B C (b )(c ) SS B C MS B C /MS error B C (a )(b )(c ) SS B C MS B C /MS error Error (S/BC) abc(n ) SS error otal N abcn SS total where a s the number of levels of factor (etc.), N s the total number of observatons (subjects), and n s the number of subjects (or replcatons ) per cell. SPSS technque Data layout: depvar B C datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ datum level_ level_ level_ Syntax: UNINOV depvar BY a b c /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN a b c a*b a*c b*c a*b*c. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter, B, C as between-subjects factors.

112 7: Specfc desgns 7.4 One wthn-subjects factor lternatve names Repeated-measures NOV (wth one factor) Randomzed complete block (RCB) desgn (wth one factor) Sngle-factor wthn-subjects desgn Examples wenty students have ther dgt span tested on dry land (U) and then those same students have a further dgt span test when they are dvng n a dry sut n the Pacfc Ocean (U). Does ther locaton affect performance? researcher wshes to test the effectveness of four fertlzers (U, U, U3, U4). He dvdes hs orchard nto four blocks (equvalent to subjects ) to account for varatons across the orchard (e.g. southern sunny block, northern cool block, eastern mornng sun block, western evenng sun block). He dvdes each block nto four plots and assgns fertlzers U U4 to each plot at random, so that each block has all four fertlzers n t. Notes Descrbed n detal by Howell (997, chapter 4). otal varaton s frst parttoned nto varaton between subjects and varaton wthn subjects. Varaton wthn subjects s then subdvded nto varaton between treatments (levels of our factor) and error. We re not partcularly nterested n varaton between subjects, but accountng for t allows us to solate the effect of our factor more accurately. If our factor has only two levels, ths s equvalent to a two-sample pared t test. Model descrpton (S subjects) Model depvar (U S) Ether Yj µ + π + α + ε (addtve model) j where Y j s the dependent varable for subject n condton U j µ s the overall mean π s the contrbuton from a partcular person or subject (subject, or S ): π µ µ α j s the contrbuton from a partcular level (level j) of the factor U: α µ µ ε j s everythng else (the expermental error assocated wth subject n condton j): ε j j ( µ + π + α j ). j j U j S or, perhaps better, Yj µ + π + α + πα + ε (nonaddtve model) where πα j s the contrbuton from the nteracton of subject wth treatment j: n ths case, ε j would be redefned as ε j Yj ( µ + π + α j + πα j ). j j j However, f we measure each person n each condton once, we wll not be able to measure dfferences n the way subjects respond to dfferent condtons (πα j ) ndependently of other sources of error (ε j ). (o do that, we d need to measure subjects more than once, and then we d need a dfferent model agan!) hs s another way of sayng that the S U nteracton s confounded wth s! the error term. herefore, the calculatons do not dffer for the two models (Myers & Well, 995, p. 4); the only dfference s f you want to estmate ω, the proporton of varance accounted for by a partcular term (Myers & Well, 995, pp. 5-55). Sources of varance nalyss of varance dscards constant terms (lke µ) and examnes the sources of varablty (varance). Wrtng ths n terms of sums of squares (SS), SS total SS subjects + SS U + SS error

113 7: Specfc desgns 3 where SS total s the total varablty, SS U s the varablty attrbutable to the (wthn-subjects) factor U, and SS error s the error varablty (everythng that s left over). hs equaton can be used to represent both models descrbed above (wth or wthout the subject factor nteracton), snce, to repeat, the subject factor nteracton s the error term n ths desgn (wth only one score per cell) and cannot be separated from error ; see Howell (997, p. 45-4). NOV table In all cases, the mean square (MS) s the sum of squares (SS) for a partcular row dvded by the degrees of freedom (d.f.) for the same row. ssumng one observaton per cell, we have Source d.f. SS F Between subjects (S) n SS subjects MS subjects /MS error U u SS U MS U /MS error Error (S U) (n )(u ) SS error otal N un SS total where u s the number of levels of factor U, N s the total number of observatons ( un), and n s the number of subjects. SPSS technque One row, one subject: Ulevel Ulevel Ulevel3 datum datum datum datum datum datum datum datum datum Syntax: GLM u u u3 /WSFCOR u 3 Polynomal /MEHOD SSYPE(3) /CRIERI LPH(.5) /WSDESIGN u. SPSS won t report the between-subjects effects (the one based on SS subjects, whch we re not partcularly nterested n). It ll report somethng else (I m not sure what ) as Between- Subjects Effects: Intercept, and the wthn-subjects effect that we are nterested n as Wthn- Subjets: U. It wll also report Mauchly s test of sphercty of the covarance matrx, together wth Greenhouse Gesser and Huynh Feldt correctons for use f the assumpton of sphercty s volated. Usng the menus, choose nalyze General Lnear Model Repeated Measures. Defne the wthn-subjects factor (wth ts number of levels). hen you can assgn ndvdual varables (e.g. Ulevel) to approprate levels of the factor. For a worked example, see p.. SPSS technque One column, one varable: depvar subject U datum subj_ level_ datum subj_ level_ datum subj_ level_3 datum subj_ level_ datum subj_ level_ datum subj_ level_3 datum subj_3 level_ datum subj_3 level_ datum subj_3 level_3

114 7: Specfc desgns 4 Syntax: GLM depvar BY subject u /RNDOM subject /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN subject u. SPSS wll report the wthn-subjects effect as Between-Subjects: U (snce t doesn t know that anythng s a wthn-subjects effect!). It ll report the SS subjects term (the dfference between subjects) as Between-Subjects: SUBJEC. It ll report the same Intercept term as before. Mauchly s test s not reported; nether are the G G and H F correctons. o obtan these, use technque nstead. You could also use ths: GLM depvar BY subject u /RNDOM subject /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN subject u subject*u. but as we ve sad, the Subject U nteracton s confounded wth error n ths desgn, and SPSS smply won t gve you a result for t. ll other answers wll be the same. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter U as a fxed factor; enter Subject as a random factor.

115 7: Specfc desgns wo wthn-subjects factors lternatve names Repeated-measures NOV (wth two factors) Randomzed complete block (RCB) desgn (wth two factors) wo-factor wthn-subjects desgn Splt-block desgn Example wenty students have ther dgt span tested on dry land when sober (U V) and then those same students have a further dgt span test when they re on dry land and sober (U V), when they are dvng n a dry sut n the Pacfc Ocean and sober (U V) and when they re drunk and dvng (U V). Don t try ths at home, kds. Does ther locaton or sobrety affect performance? Do these two factors nteract? researcher wshes to test the effectveness of three fertlzers (U, U, U3) and three tree thnnng technques (V, V, V3). He dvdes hs natonal park forest nto four blocks (equvalent to subjects ) to account for varatons across the park (e.g. mountanous confers, lowland decduous, tmber-harvested forest, volcanc ash area). He dvdes each block nto nne plots and assgns fertlzers U U3 and thnnng technques V V3 to each plot at random but such that every block contans every combnaton of fertlzer and thnnng treatment once. Notes Model descrpton depvar (U V S) Model here are two alternatve models (see Howell, 997, p.486-7, whch descrbes the problem for three wthn-subjects factors; ths s merely a smpler case). he frst, smpler model, s ths, n whch the Subject term doesn t nteract wth anythng: Yjk µ + α + β + αβ + π + ε where Y jk s the dependent varable for subject k n condton U, V j µ s the overall mean α s the contrbuton from a partcular level (level ) of factor U: α µ µ β j s the contrbuton from a partcular level (level j) of factor V: β µ µ π k s the contrbuton from a partcular person or subject (subject k): π µ µ j ε jk s everythng else (the expermental error assocated wth subject k n condton U V j ): ε jk Yjk ( µ + α + β j + αβj + π k ) j k jk j k U V j S k he second, probably better model, s ths, whch allows the Subject term to nteract wth the other varables (.e. accounts for the fact that dfferent treatments may affect dfferent subjects n dfferent ways): Yjk µ + α + β j + αβ j + π k + απ k + βπ jk + αβπ jk + ε jk where απ k s the contrbuton from the nteracton of subject k wth treatment U : απ k µ S U ( µ + α + π k ) k βπ jk s the contrbuton from the nteracton of subject k wth treatment V j : βπ jk µ S V ( µ + β j + π k ) k j αβπ jk s the contrbuton from the nteracton of subject k wth the treatment combnaton U V j : αβπ jk µ S U V ( µ + α + β j + αβj + π k + απ j + βπ jk ) k j n ths case, we would redefne the error term: ε jk Yjk ( µ + α + β j + αβ j + π k + απ k + βπ jk + αβπ jk ) However, ths more complex model does have a problem: snce we have ncluded the Subject term as a varable that nteracts wth everythng, we now only have one score per cell, and we have no resdual left for estmatng error (ε jkl ). However, as t happens (Howell, 997, pp. 487-

116 7: Specfc desgns 6 8), we can use the sum of squares for the U S term (απ l ) as an error estmate for the U term (α ), the sum of squares for V S as an error estmate for the V term, and so on. he full model s usually preferable (Howell, 997, p. 487). Sources of varance Ether the reduced model SS total SS subjects + SS U + SS V + SS U V + SS error or the full model SS total SS subjects + SS U + SS V + SS U V + SS U S + SS V S + SS U V S NOV table In all cases, the mean square (MS) s the sum of squares (SS) for a partcular row dvded by the degrees of freedom (d.f.) for the same row. ssumng one observaton per cell, we have ether Source d.f. SS F Between subjects n SS subjects U u SS U MS U /MS error V v SS V MS V /MS error U V (u )(v ) SS U V MS U V /MS error Error (n )(uv ) SS error otal N uvn SS total or, wth the full model: Source d.f. SS F Between subjects (S) n SS S U u SS U MS U /MS U S error U S (u )(n ) SS U S V v SS V MS V /MS V S error V S (v )(n ) SS V S U V (u )(v ) SS U V MS U V /MS U V S error U V S (u )(v )(n ) SS U V S otal N uvn SS total where u s the number of levels of factor U, etc., N s the total number of observatons ( uvn), and n s the number of subjects. SPSS technque One row, one subject: UV UV UV UV datum datum datum datum datum datum datum datum datum datum datum datum Syntax: GLM uv uv uv uv /WSFCOR u Polynomal v Polynomal /MEHOD SSYPE(3) /CRIERI LPH(.5) /WSDESIGN u v u*v. hs wll gve you the full model answer (see above), n whch the Subject factor s allowed to nteract wth everythng n full. Usng the menus, choose nalyze General Lnear Model Repeated Measures. Defne the wthn-subjects factors (wth ther numbers of levels). hen you can assgn ndvdual varables (e.g. UV) to approprate levels of the factors. For a worked example, see p.. SPSS technque One column, one varable:

117 7: Specfc desgns 7 Subject U V depvar datum datum datum datum datum datum datum datum o get the reduced model (see above): GLM depvar BY subject u v /RNDOM subject /MEHOD SSYPE(3) /CRIERI LPH(.5) /DESIGN u v u*v subject. o get the full model, matchng SPSS s usual wthn-subjects technque (see above): GLM depvar BY subject u v /RNDOM subject /MEHOD SSYPE(3) /CRIERI LPH(.5) /DESIGN u v u*v subject u*subject v*subject u*v*subject. s usual wth ths technque, Mauchly s test s not reported; nether are the G G and H F correctons. o obtan these, use technque nstead. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter U, V as fxed factors; enter Subject as a random factor.

118 7: Specfc desgns hree wthn-subjects factors lternatve names Repeated-measures NOV (wth three factors) Randomzed complete block (RCB) desgn (wth three factors) hree-factor wthn-subjects desgn Example Notes Oh, t gets borng makng these up. set of subjects are all tested n every combnaton of three treatments (U U u, V V v, W W w ). In the agrcultural verson, ths s what an RCB desgn mght look lke: In our termnology, the agrcultural block s the psychologcal subject : each subject experences each combnaton of the factors U, V, and W. Model descrpton depvar (U V W S) Model here are two alternatve models (see Howell, 997, p.486-8). he frst, smpler model, s ths, n whch the Subject term doesn t nteract wth anythng: Yjkl µ + α + β + γ + αβ + αγ + βγ + αβγ + π + ε j k j k jk jk l jkl where Y jkl s the dependent varable for subject l n condton U, V j, W k µ s the overall mean α s the contrbuton from a partcular level (level ) of factor U β j s the contrbuton from a partcular level (level j) of factor V γ k s the contrbuton from a partcular level (level k) of factor W αβ j, αγ k, βγ jk, and αβγ jk are the contrbutons from the UV, UW, VW, and UVW nteracton terms π l s the contrbuton from a partcular person or subject (subject l) ε jkl s everythng else (the expermental error assocated wth subject l n condton U V j W k ). he second, probably better model, s ths, whch allows the Subject term to nteract wth the other varables (.e. accounts for the fact that dfferent treatments may affect dfferent subjects n dfferent ways): Y jkl αβπ µ + α + β + γ + αβ + αγ jl + αγπ kl j + βγπ k jkl j + αβγπ jkl k + ε + βγ jkl jk + αβγ jk + π + απ + βπ l l jk + γπ + kl

119 7: Specfc desgns 9 where απ l s the contrbuton from the nteracton of subject l wth treatment U βπ jk s the contrbuton from the nteracton of subject l wth treatment V j γπ kl s the contrbuton from the nteracton of subject l wth treatment W k αβπ jl s the contrbuton from the nteracton of subject l wth the treatment combnaton U V j αγπ kl s the contrbuton from the nteracton of subject l wth the treatment combnaton U W k βγπ jkl s the contrbuton from the nteracton of subject l wth the treatment combnaton V j W k αβγπ jkl s the contrbuton from the nteracton of subject l wth the treatment combnaton U V j W k For exact specfcaton of each of these components (e.g. α µ U µ ) see the prevous model (p. 5 ); t s just the same but wth more terms. However, ths more complex model does have a problem: snce we have ncluded the Subject term as a varable that nteracts wth everythng, we now only have one score per cell, and we have no resdual left for estmatng error (ε jkl ). However, as t happens (Howell, 997, pp ), we can use the sum of squares for the U S term (απ l ) as an error estmate for the U term (α ), the sum of squares for V S as an error estmate for the V term, and so on. he full model s usually preferable (Howell, 997, p. 487). Sources of varance Ether the reduced model SS total SS subjects + SS U + SS V + SS W + SS U V + SS U W + SS V W + SS U V W + SS error or the full model SS total SS subjects + SS U + SS V + SS W + SS U V + SS U W + SS V W + SS U V W + SS U S + SS V S + SS W S + SS U V S + SS U W S + SS V W S + SS U V W S NOV table In all cases, the mean square (MS) s the sum of squares (SS) for a partcular row dvded by the degrees of freedom (d.f.) for the same row. ssumng one observaton per cell, we have ether Source d.f. SS F Between subjects n SS subjects U u SS U MS U /MS error V v SS V MS V /MS error W w SS W MS W /MS error U V (u )(v ) SS U V MS U V /MS error U W (u )(w ) SS U W MS U W /MS error V W (v )(w ) SS V W MS V W /MS error U V W (u )(v )(w ) SS U V W MS U V W /MS error Error (n )(uvw ) SS error otal N uvwn SS total or n the full verson: Source d.f. SS F Between subjects n U u SS U MS U /MS U S error U S (u )(n ) SS U S V v SS V MS V /MS V S error V S (v )(n ) SS V S W w SS W MS W /MS W S error W S (w )(n ) SS W S U V (u )(v ) SS U V MS U V /MS U V S error U V S (u )(v )(n ) SS U V S U W (u )(w ) SS U W MS U W /MS U W S error U W S (u )(w )(n ) SS U W S

120 7: Specfc desgns V W (v )(w ) SS V W MS V W /MS V W S error V W S (v )(w )(n ) SS V W S U V W (u )(v )(w ) SS U V W MS U V W /MS U V W S error U V W S(u )(v )(w )(n ) SS U V W S otal N uvwn SS total where u s the number of levels of factor U, etc., N s the total number of observatons ( uvwn), and n s the number of subjects. SPSS technque One row, one subject: UVW UVW UVW UVW UVW UVW UVW UVW (etc.) datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum Syntax: GLM uvw uvw uvw uvw uvw uvw uvw uvw /WSFCOR u Polynomal v Polynomal w Polynomal /MEHOD SSYPE(3) /CRIERI LPH(.5) /WSDESIGN u v w u*v u*w v*w u*v*w. hs wll gve you the full model answer (see above), n whch the Subject factor s allowed to nteract wth everythng n full. hs layout doesn t allow you to use the reduced model, as far as I can see. Usng the menus, choose nalyze General Lnear Model Repeated Measures. Defne the wthn-subjects factors (wth ther numbers of levels). hen you can assgn ndvdual varables (e.g. UVW) to approprate levels of the factors. For a worked example, see p.. SPSS technque One column, one varable: Subject U V W depvar datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum o get the reduced model (see above): GLM depvar BY subject u v w /RNDOM subject /MEHOD SSYPE(3) /CRIERI LPH(.5) /DESIGN u v w u*v u*w v*w u*v*w subject. o get the full model, matchng SPSS s usual wthn-subjects technque (see above): GLM depvar BY subject u v w

121 7: Specfc desgns /RNDOM subject /MEHOD SSYPE(3) /CRIERI LPH(.5) /DESIGN [subject] u u*subject v v*subject w w*subject u*v u*v*subject u*w u*w*subject v*w v*w*subject u*v*w subject u*v*w*subject. s usual wth ths technque, Mauchly s test s not reported; nether are the G G and H F correctons. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter, B, C as fxed factors; enter Subject as a random factor.

122 7: Specfc desgns 7.7 One between- and one wthn-subjects factor lternatve names Splt-plot desgn (Keppel, 99) Mxed two-factor wthn-subjects desgn (Keppel, 99) Repeated measures analyss usng a splt-plot desgn (SPSS,, p. 464) Unvarate mxed models approach wth subject as a random effect (SPSS,, p. 464) Example Notes We take three groups of rats, n 8 per group (s 4). We gve one group treatment, one group treatment, and one group treatment 3. (One subject only experences one treatment.) hen we measure every subject s performance at sx tme ponts U U6. We frst partton the total varaton nto between-subjects varablty and wthn-subjects varablty. Model descrpton depvar (U S) he between-subjects varablty can be attrbuted to ether the effect of the treatment group (), or dfferences between subjects n the same group ( S wthn or S/ ). (hs notaton ndcates that there s a dfferent group of subjects at each level of the between-subjects factor, ; we could not measure smply of subject varaton ndependent of the effects of snce no subjects ever serve n more than one group, or level of. SPSS uses the alternatve notaton of S().) So we have these sources of between-subjects varablty: S/ he wthn-subjects varablty can be attrbuted to ether the effects of the tme pont (U), or an nteracton between the tme pont and the drug group (U ), or an nteracton between the tme pont and the subject-to-subject varablty, whch agan we can only measure wthn a drug group (U S/). So we have these sources of wthn-subject varablty: U U U S/ Model Followng Myers & Well (995, p. 95-6): Yjk + α + π j / + β k + αβ k + πβ jk / µ + ε jk where Y jk s the dependent varable for subject j n group and condton U k µ s the overall mean α s the contrbuton from a partcular level (level ) of factor : α µ µ π j/ s the contrbuton from a partcular person or subject (subject j), who only serves wthn condton ( subject wthn group, or S/): π µ µ j / S j / (here s no straghtforward nteracton of wth S: every subject s only measured at one level of, so ths term would be ndstngushable from the subject-only effect π j/.) β k s the contrbuton from a partcular level (level k) of factor U: β µ µ αβ k s the contrbuton from the nteracton of and U k : αβ k µ U ( µ + α + β k ) k πβ jk/ s the contrbuton from the nteracton of U k wth subject j, whch can only be measured wthn one level of (t s the SU/ term): πβ jk / µ S U / ( µ + π j / + βk ) j k (here s no straghtforward three-way U S nteracton: every subject s only measured at one level of, so ths term would be ndstngushable from the SU/ effect πβ jk/.) ε jk s everythng else (the expermental error assocated wth measurng person j who always experences treatment n condton U k ): ε jk Yjk ( µ + α + π j / + β k + αβ k + πβ jk / ). Note that we cannot actually measure ε jk ndependent of the SU/ term f we only have one measurement per subject per level of U. k U k

123 7: Specfc desgns 3 Sources of varance SS total SS between subjects + SS wthn subjects SS between subjects SS + SS S/ SS wthn subjects SS U + SS U + SS U S/ So SS total SS + SS S/ + SS U + SS U + SS U S/ We have two dfferent error terms, one for the between-subjects factor and one for the wthnsubjects factor (and ts nteracton wth the between-subjects factor), so we can t just label them SS error. But we could rewrte the total lke ths f we wanted: SS total SS + SS error-between + SS U + SS U + SS error-wthn NOV table Source d.f. SS F Between subjects (S): s an a SS MS /MS S/ error S/ (an ) (a ) a(n ) SS S/ Wthn subjects: (N ) (s ) an(u ) U u SS U MS U /MS U S/ U (u )(a ) SS U MS U /MS U S/ error U S/ a(u )(n ) SS U S/ otal N aun SS total where a s the number of levels of factor, etc., N s the total number of observatons ( aun), n s the number of subjects per group (per level of ), and s s the total number of subjects ( an). SPSS technque One subject, one row: U U datum datum datum datum datum datum datum datum datum datum datum datum Usng the menus, choose nalyze General Lnear Model Repeated Measures. Defne the wthn-subjects factor (wth ts number of levels). hen you can assgn ndvdual varables (e.g. U) to approprate levels of the factors, and assgn the between-subjects factor. Here s where we fll n the lst of wthn-subjects factors and the number of levels. ype them n and clck dd.

124 7: Specfc desgns 4 hey appear n the lst. If we had more wthn-subjects factors, we could add them too. Once we ve fnshed, we clck Defne. We can now fll n the varables (U, U) correspondng to the levels of factor U; we can also defne as a between-subjects factor.

125 7: Specfc desgns 5 Once everythng else s OK, clck OK to run the analyss, or Paste to copy the syntax for the analyss to a syntax wndow. hs analyss produces the followng syntax: GLM u u BY a /WSFCOR u Polynomal /MEHOD SSYPE(3) /CRIERI LPH(.5) /WSDESIGN u /DESIGN a. SPSS technque One column, one varable: Subject U depvar datum datum datum datum 3 datum 3 datum 4 datum 4 datum 5 datum 5 datum Syntax: GLM depvar BY subject U /RNDOM subject /DESIGN subject* U U* U*subject*. or alternatvely GLM depvar BY subject U /RNDOM subject /DESIGN subject() U U* U*subject(). (hs syntax s an example on page 464 of the SPSS. Syntax Reference Gude PDF.) It tests MS aganst MS subject, and t tests the others (MS U and MS U ) aganst what t smply calls MS error. s usual wth ths technque, Mauchly s test s not reported; nether are the G G and H F correctons. he underlned bt s optonal, snce ths s the same as the resdual error and won t be fully calculated, but ncludng t won t change the answers for any other factor.

126 7: Specfc desgns 6 Not entrely trval to accomplsh wth the SPSS menus. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter, U as fxed factors; enter Subject as a random factor. Snce SPSS wll get the model wrong for mxed models (by ncludng S and U S terms), you then need to edt the Model drectly before runnng the analyss. Untck Full factoral by tckng Custom. Enter the desred terms (n ths case the between subjects term, the error term S/ whch you enter as S, the wthn-subject bts U, U, and f you want, the error term U S/ whch you enter as U S, though that s optonal).

127 7: Specfc desgns wo between-subjects factors and one wthn-subjects factor lternatve names Example Notes Fat men, thn men, fat women, and thn women ( B, B, B, and B ) all have ther blood pressure measured n the mornng (U ) and n the evenng (U ). Does blood pressure depend on any of these factors, or on a combnaton of them? Obesty and sex are betweensubjects varables; tme of day s a wthn-subject varable. We frst partton the total varaton nto between-subjects varablty and wthn-subjects varablty. he between-subjects varablty can be attrbuted to ether the effect of the between-subjects factors (, B, B), or dfferences between subjects n the same group ( S wthn group, or n Keppel s notaton, snce a group s specfed by a unque combnaton of and B, S/B ). So we have these sources of between-subjects varablty: B B S/B (between-subjects error) he wthn-subjects varablty can be attrbuted to ether the effects of the wthn-subjects factor (U), or some form of nteracton between U and the between-subjects factors (U, U B, U B), or an nteracton between U and the subject-to-subject varablty, whch agan we can only measure wthn a group (U S/B). So we have these sources of wthn-subject varablty: U U U B U B U S/B (wthn-subjects error) Model descrpton depvar B (U S) Model I made ths up, but I got t rght for a change (Myers & Well, 995, p. 38): Y jkl µ + α + β + αβ + π l + γ + αγ + βγ l j jl j + αβγ k / j jl + πγ kl / j + ε jkl where Y jkl s the dependent varable for subject k n condton, B j, U k µ s the overall mean α s the contrbuton from a partcular level (level ) of factor : α µ µ β j s the contrbuton from a partcular level (level j) of factor B: β µ µ αβ j s the contrbuton from the nteracton of and B j : αβ j µ B ( µ + α + β j ) j π k/j s the contrbuton from a partcular person or subject (subject k), who s measured only n condton B j (ths s the S/B term): π µ µ k / j S / B γ l s the contrbuton of level l of factor U: γ µ µ αγ l, βγ jl, and αβγ jl represent the /U l, B j /U l, and /B j /U l nteracton contrbutons, respectvely: αγ l µ U ( µ + α + γ l ) ; βγ ( ) l jl µ B ju µ + β l j + γ l ; and αβγ jl µ B U ( µ + α + β j + αβj + γ l + αγ l + βγ jl ). j l πγ kl/j represents the nteracton of U l wth subject k (who only experences condton B j ) the U S/B term: πγ kl / j µ S U / B ( µ + α + β j + αβj + π k / j + γ l + αγ l + βγ jl + αβγ jl ) k l j ε jk s everythng else (the expermental error assocated wth measurng person k, who al- l U l j j j B j

128 7: Specfc desgns 8 ways experences treatment, n condton U j ): ε jkl Yjkl ( µ + α + β j + αβ j + π k / j + γ l + αγ l + βγ jl + αβγ jl + πγ kl / j ). Of course, ths cannot be measured ndependently of the U S/B term (snce there s only one observaton n condton B j S k U l ). Sources of varance SS total SS between-subjects + SS wthn-subjects SS between-subjects SS + SS B + SS B + SS error-between SS wthn-subjects SS U + SS U + SS U B + SS U B + SS error-wthn NOV table Source d.f. SS F Between subjects (S): abn a SS MS /MS S/B B b SS B MS B /MS S/B B (a )(b ) SS B MS B /MS S/B error S/B ab(n ) SS S/B Wthn subjects: abn(u ) U u SS U MS U /MS U S/B U (u )(a ) SS U MS U /MS U S/B U B (u )(b ) SS U B MS U B /MS U S/B U B (u )(a )(b ) SS U B MS U B /MS U S/B error U S/B ab(u )(n ) SS U S/B otal N abun SS total where a s the number of levels of factor, etc., N s the total number of observatons ( abun), and n s the number of subjects per group (where a group s defned by the combnaton of factors and B). SPSS technque One subject, one row: B U U U3 datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum Syntax: GLM u u u3 BY a b /WSFCOR u 3 Polynomal /MEHOD SSYPE(3) /CRIERI LPH(.5) /WSDESIGN u /DESIGN a b a*b. Usng the menus, choose nalyze General Lnear Model Repeated Measures. Defne the wthn-subjects factor (wth ts number of levels). hen you can assgn ndvdual varables (e.g. U) to approprate levels of the factors, and assgn the between-subjects factors. SPSS technque One column, one varable: B Subject U depvar datum datum 3 datum datum datum 3 datum

129 7: Specfc desgns 9 3 datum 3 datum 3 3 datum Syntax: GLM depvar BY a b subject u /RNDOM subject /DESIGN a b a*b subject*a*b u u*a u*b u*a*b u*subject*a*b. n alternatve syntax s ths: GLM depvar BY a b subject u /RNDOM subject /DESIGN a b a*b subject(a*b) u u*a u*b u*a*b u*subject(a*b). s usual wth ths technque, Mauchly s test s not reported; nether are the G G and H F correctons. he underlned bt s optonal, snce ths s the same as the resdual error and won t be fully calculated, but ncludng t won t change the answers for any other factor. Not entrely trval to accomplsh wth the SPSS menus. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter, B, U as fxed factors; enter Subject as a random factor. Snce SPSS wll get the model wrong for mxed models (by ncludng S and all sorts of terms n whch the between-subjects factors nteract wth S), you then need to edt the Model drectly before runnng the analyss. Untck Full factoral by tckng Custom. Enter the desred terms (n ths case the between-subjects bts, B, B, the error term S/B whch you enter as S B, the wthn-subjects bts U, U, U B, U B, and optonally the error term U S/B whch you enter as U S B).

130 7: Specfc desgns One between-subjects factor and two wthn-subjects factors lternatve names Example Notes Rats are gven a bran leson ( ) or a sham operaton ( ). hey are repeatedly offered two levers; one delvers small, mmedate reward, and the other delvers large, delayed reward. her preference for the large, delayed reward s assessed (dependent varable) at dfferent delays (U, U, U 5 ). Furthermore, they are tested hungry (V ) or sated (V ). ll subjects experence all combnatons of U and V, sutably counterbalanced, but one subject s only ever n one group. We frst partton the total varaton nto between-subjects varablty and wthn-subjects varablty. he between-subjects varablty can be attrbuted to ether the effect the between-subjects factor (), or dfferences between subjects n the same group ( S wthn group, or S/ ). So we have these sources of between-subjects varablty: S/ (between-subjects error) he wthn-subjects varablty can be attrbuted to ether the effects of the wthn-subjects factors (B, C, B C), or some form of nteracton between the wthn-subjects factors and the between-subjects factor (B, C, B C ), or an nteracton between the wthn-subjects factors and the subject-to-subject varablty (B S/, C S/, B C S/) where S/ agan refers to subject varablty wthn a group (defned by the between-subjects factor, ). So we have these sources of wthn-subject varablty: U U U S/ (wthn-subjects error term for the precedng two factors) V V V S/ (wthn-subjects error term for the precedng two factors) U V U V U V S/ (wthn-subjects error term for the precedng two factors) Model descrpton depvar (U V S) Model Sources of varance hs would be rather tedous to wrte out (see Myers & Well, 995, p. 3); follow the prncples n the prevous model, whch was for (U S). he models always start wth the overall mean (µ). hen the between-subject factors (here, α), and ther nteractons (here, none), are added. hen there s subject term (π), whch s nested wthn levels of. hen there are the wthn-subject factors (β, γ), and ther nteractons (βγ). hen for the full model all wthnsubject factors and nteractons nteract wth the subject term, whch tself s nested wthn (to gve βπ, γπ, βγπ). Fnally there s the ε term. SS total SS between-subjects + SS wthn-subjects SS between-subjects SS + SS error-between SS wthn-subjects SS U + SS U + SS U S/ + SS V + SS V + SS V S/ + SS U V + SS U V + SS U V S/ NOV table Source d.f. SS F Between subjects: an a SS MS /MS S/ error S/ a(n ) SS S/ Wthn subjects: an(uv ) U u SS U MS U /MS U S/ U (u )(a ) SS U MS U /MS U S/ error U S/ a(u )(n ) SS U S/

131 7: Specfc desgns 3 V v SS V MS V /MS V S/ V (v )(a ) SS V MS V /MS V S/ error V S/ a(v )(n ) SS V S/ U V (u )(v ) SS U V MS U V /MS U V S/ U V (v )(a )(u ) SS U V MS U V /MS U V S/ error U V S/ a(u )(v )(n ) SS U V S/ otal N auvn SS total where a s the number of levels of factor, etc., N s the total number of observatons ( auvn), and n s the number of subjects per group (where group s defned by factor ). SPSS technque One row, one subject: UV UV UV UV datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum Syntax: GLM uv uv uv uv BY a /WSFCOR u Polynomal v Polynomal /MEHOD SSYPE(3) /PRIN DESCRIPIVE HOMOGENEIY /CRIERI LPH(.5) /WSDESIGN u v u*v /DESIGN a. Usng the menus, choose nalyze General Lnear Model Repeated Measures. Defne the wthn-subjects factors (wth ther numbers of levels). hen you can assgn ndvdual varables (e.g. UV) to approprate levels of the factors, and assgn the between-subjects factor. SPSS technque One column, one varable: Subject U V depvar datum datum datum datum datum datum datum datum 3 datum 3 datum 3 datum 3 datum Syntax: UNINOV depvar BY a subject u v /RNDOM subject /MEHOD SSYPE(3) /INERCEP INCLUDE /PRIN DESCRIPIVE HOMOGENEIY /CRIERI LPH(.5) /DESIGN a subject*a u u*a u*subject*a

132 7: Specfc desgns 3 v v*a v*subject*a u*v u*v*a u*v*subject*a. Incdentally, the notaton Subject() wll be accepted as equvalent to Subject* n these sorts of desgns; feel free to use ths alternatve form f t seems clearer: UNINOV depvar BY a subject u v /RNDOM subject /MEHOD SSYPE(3) /INERCEP INCLUDE /PRIN DESCRIPIVE HOMOGENEIY /CRIERI LPH(.5) /DESIGN a subject(a) u u*a u*subject(a) v v*a v*subject(a) u*v u*v*a u*v*subject(a). Not entrely trval to accomplsh wth the SPSS menus. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter, B, U as fxed factors; enter Subject as a random factor. Snce SPSS wll get the model wrong for mxed models (by ncludng S and all sorts of terms n whch the between-subjects factors nteract wth S), you then need to edt the Model drectly before runnng the analyss. Untck Full factoral by tckng Custom. Enter the desred terms, as lsted above; the method s explaned further n the secton on the two-between, one-wthn model.

133 7: Specfc desgns Other NOV desgns wth between and/or wthn-subjects factors he models above can be extended along the same prncples. See Keppel (99), pp full map of all the error terms s gven on p. 493; an expanded verson showng all terms s presented here. For any term, the approprate error term s the next error term n the lst. he dfferent error terms needed for partal and full wthn-subjects models are dscussed by Howell (997, pp ). Only full models are presented for desgns nvolvng between-subject factors. Desgn: Between-subjects factors None factor () factors (, B) 3 factors (, B, C) S B S B C S Wthn-subjects factors None erms: error [ S/] B B error [ S/B] B C B C B C B C error [ S/BC] Desgn: (U S) (U S) B (U S) B C (U S) factor (U) erms: between subjects term [S] U error [ U S] between subjects: error S/ wthn subjects: U U error U S/ between subjects: B B error S/B wthn subjects: U U U B error U S/B between subjects: B C B C B C B C error S/BC wthn subjects: U U U B U C U B U C U B C U B C error U S/BC Desgn: (U V S) (U V S) B (U V S) B C (U V S) factors (U, V) erms: smpler model: between-subjects term [S] U V U V error full model (preferable): between-subjects term [S] (no correspondng error term) U error U S V error V S U V error U V S between subjects: error S/ wthn subjects: U U error U S/ V V error V S/ U V U V error U V S/ between subjects: B B error S/B wthn subjects: U U U B U B error U S/B V V V B V B error V S/B U V U V U V B U V B error U V S/B between subjects: B B B C error S/BC wthn subjects: U U U B U B error U S/B V V V B V B error V S/B U V U V U V B U V B error U V S/B

134 7: Specfc desgns 34 Desgn: (U V W S) (U V W S) B (U V W S) B C (U V W S) 3 factors (U, V, W) erms: smpler model: between-subjects term [S] U V U V U W V W U V W error full model (preferable): between-subjects term [S] (no correspondng error term) U error U S V error V S W error W S U V error U V S U W error U W S V W error V W S U V W error U V W S between subjects: error S/ wthn subjects: U U error U S/ V V error V S/ W W error W S/ U V U V error U V S/ U W U W error U W S/ V W V W error V W S/ U V W U V W error U V W S/ between subjects: B B error S/B wthn subjects: U U U B U B error U S/B V V V B V B error V S/B W W W B W B error W S/B U V U V U V B U V B error U V S/B U W U W U W B U W B error U W S/B V W V W V W B V W B error V W S/B U V W U V W U V W B U V W B error U V W S/B between subjects: B C B C B C error S/BC wthn subjects: U U U B U C U B U C U B C U B C error U S/BC V V V B V C V B V C V B C V B C error V S/BC W W W B W C W B W C W B C W B C error W S/BC U V U V U V B U V C U V B U V C U V B C U V B C error U V S/BC U W U W U W B U W C U W B U W C U W B C U W B C error U W S/BC V W V W V W B V W C V W B V W C V W B C V W B C error V W S/BC U V W U V W U V W B U V W C U V W B U V W C U V W B C U V W B C error U V W S/BC

135 7: Specfc desgns One between-subjects covarate (lnear regresson) lternatve names nalyss of covarance (NCOV) though tradtonally ths term sn t appled to a desgn wth no other factors Lnear regresson Example You measure subjects ncome (dependent varable) and want to predct t n the bass of ther IQ. Every subject contrbutes an sngle (IQ, ncome) par of values. hs s basc lnear regresson. In regresson termnology we would be tryng to predct the dependent varable Y from the another, predctor varable.e. solvng the regresson equaton where Y ˆ b + a cov sy b r r s s Y and a y bx SS SS Y where Ŷ s the predcted value of Y (see also Myers & Well, 995, p. 387). lternatvely, we could wrte ths: Y b + a + ε where ε symbolzes the error or resdual. he equaton represents, of course, ths: Or we could lay out the equaton so as to be extensble to multple regresson (whch we ll look at later): ˆ b + b Y Y b + b In NCOV termnology, the predctor varable s the covarate, whch we ll call C. So we could frst rewrte the smple lnear regresson equaton wth the letters we ll use from now on: Y ˆ a + bc where a Y b C and now wrte t as a predcton for specfc values of Y and C, namely Y and C : + ε Y a + bc + ε where a Y bc and now wrte t terms of the means of Y ( Y µ ) and C ( C ): Y µ bc + bc + ε µ + b( C C ) + ε (Compare Myers & Well, 995, p. 436.) We ll use ths below. It helps to dstngush between the predcted value of Y based on the covarate [whch s Yˆ a + bc µ + b( C C ) ] and the contrbuton of the covarate, whch s the devaton of the covarate-predcted value of Y

136 7: Specfc desgns 36 from the overall mean of Y [whch s therefore c b( C C ) ]. Obvously, c Yˆ µ. Note also that the proporton of the total varablty n Y that s accounted for by predctng t from C s equal to r : SS Yˆ SSmodel r SS SS Y and the SS attrbutable to the model (SS model or SS regresson or SS reg ) can be wrtten SS reg Y b SS C total ( Yˆ Y ) r SS Notes Model descrpton depvar C cov + S (I ve made that up, as Keppel doesn t have a specfc notaton for models ncludng covarates.) Model Y µ + + ε c where Y s the dependent varable for subject µ s the overall mean c s the contrbuton of the covarate for subject : c b( C C ) Yˆ µ where b s the regresson coeffcent, C s the value of the covarate for subject, C s the overall mean value of the covarate, and Yˆ s the value of Y predcted by on the bass of the covarate. ε s everythng else (the error, resdual, ndvdual varaton, etc.): ε Y µ + c ) ( Sources of varance SS total SS reg + SS error he SS reg s gven by SSreg ( c ) ( ) ( ˆ b C C Y µ ) r SSY b SSC (Myers & Well, 995, p. 393). It s the sum of the squared contrbutons of the covarate, whch s to say the sum of the squared devatons between the covarate-predcted value and overall mean. NOV table Covarates have degree of freedom. Source d.f. SS F C cov (regresson) SS C MS C /MS error Error N SS error otal N SS total where N s the number of subjects. SPSS technque Data layout: C datum datum datum depvar datum datum datum Ether run the analyss as a regresson: REGRESSION /MISSING LISWISE /SISICS COEFF OUS R NOV /CRIERIPIN(.5) POU(.) /NOORIGIN

137 7: Specfc desgns 37 /DEPENDEN depvar /MEHODENER c. or as an NCOV (note use of WIH for covarates, rather than BY for factors): UNINOV depvar WIH c /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /PRIN PRMEER /DESIGN c. hs wll also gve you r for the model. he /PRIN PRMEER syntax also gves you b; you can combne r wth the sgn of b to calculate r. Usng the menus, choose nalyze General Lnear Model Unvarate; enter the dependent varable and the covarate n the approprate boxes. o get parameter (b) estmates as well, choose Optons Parameter estmates.

138 7: Specfc desgns One between-subjects covarate and one between-subjects factor 7... he covarate and factor do not nteract lternatve names nalyss of covarance (NCOV) nalyss of covarance (NCOV) assumng homogenety of regresson radtonal NCOV Example fter Howell (997, p. 585). Suppose we are nterested n whether small cars are easer to handle. We can compare drvng profcency usng three cars: small, medum, and large (,, 3 ). One drver s tested n only one car. We have three groups of drvers to test, but they vary consderably n ther drvng experence (C cov ). We have arranged matters so the mean drvng experence s the same n each group. If drvng experence has a very large effect on performance, we may be unable to detect an effect of car type. So we can partal out the effect of drvng experence (C cov ), ncreasng our power to detect an effect of car type (). More controversally, suppose that the mean level of drvng experence was not the same for the three groups. hen performng an analyss of covarance s lke askng what the effect of car type was had the groups not dffered on the covarate. hs may not make sense; see Howell (997, pp ) and Myers & Well (995, pp ). For example, f you measure the effect of a drug on three-year-old and fve-year-old chldren and covary for body weght, t may make lttle sense to ask what the effect on three-year-olds would be f they weghed the same as fve-year-olds they don t. Statstcally controllng for the covarate s not the same as expermentally controllng for the covarate (Myers & Well, 995, p. 45). Even worse s the stuaton when you measure the covarate after the treatment (factor) has been appled and the treatment has affected the covarate; t s then pretty dffcult to nterpret an analyss of covarance meanngfully. See Howell (997, pp ). Notes Howell tends to refer to covarates as thngs that are accounted for or partalled out n advance of consderaton of other factors (Howell, 997, p. 587; p. 66). hs mples that the covarate factor nteracton s not ncluded, except to check the assumpton of homogenety of regresson. hs s a tradtonal meanng of NCOV; see the GLM secton (p. 88 ) for a full explanaton. SPSS refers to covarates n the sense of contnuous predctor varables (as opposed to factors, whch are dscrete predctor varables) but does follow Howell s approach when you use covarates n ts full model mode. I wll refer to covarates n the sense of contnuous predctor varables and try to make t explct when covarates nteract wth factors or do not. hs model assumes that the covarate s ndependent of the expermental treatments. (If not, see the nteractng verson below.) Let s take these data: depvar (Y) We mght run a one-way NOV on t, usng our standard parttonng of varance:

139 7: Specfc desgns 39 But suppose we also have nformaton about a covarate C: C depvar (Y) We mght be able to get a much more powerful test of the effects of f we removed the effect of C. We could, for example, correlate Y wth C for all data ponts, obtan predcted values of Y based on C, obtan the resduals and see what effect has on those. We could therefore splt the SS lke ths: hat d look lke ths: SS total SS regresson(overall) + SS resdual where SS resdual SS + SS error hs s almost what one-way NCOV does. However, the regresson lne used s not qute the overall regresson (Myers & Well, 995, pp ). o see why, consder these data: C depvar (Y) Here, f we calculated the regresson lne usng all the data lumped together, we wouldn t get as good a ft as f we ftted separate regresson lnes for each group (one lne for, another for ). But the NCOV model we are usng assumes homogenety of regresson that s, that the and data may have dfferent ntercepts but they have the same slope. How do we estmate ths slope? pparently (Myers & Well, 995, p. 438) the best estmate of what s called the pooled wthn-group slope s ths: b S / SS SS C / S / ( C) b SS SS C / S / ( C) b SS + SS C / S / ( C) b + where and b s the slope calculated just for observatons n group SS C / s the varance of C for observatons n group

140 7: Specfc desgns 4 For example, wth the data set above, and SS / ( ) SS S C C / b.95 ; b. 8 SSC 4 ; SSC / / S / ( C SS ) b S /.997 We can then calculate the sum of squares for lnear regresson wthn groups, SS reg(s/), by summng the varabltes accounted for by the regressons wth the common slope n each of the groups (Myers & Well, 995, p. 439): SS reg( S / ) S / S / SS C / C / ( C) S / b SS + b SS + b C / n ths case, SS reg(s/) (.997) Snce the wthn-group regresson lne wll pass through the wthn-group mean ponts { C, Y }, we can sketch the stuaton: Fnally, we can partton the varance lke ths: whch looks lke ths (!): SS total SS overall regresson + SS adjusted total SS total SS + SS S/ SS S/ SS wthn-group regresson, reg(s/) + SS adjusted S/ SS adjusted total SS adjusted + SS adjusted S/

141 7: Specfc desgns 4 s a result, the quoted SS covarate ( SS reg(s/) ), quoted SS ( SS,adjusted ), and quoted error ( SS adjusted S/ ) won t add up to SS total. Model descrpton depvar C cov + S (I ve used the notaton + to separate out thngs that don t nteract wth anythng ths seems reasonably consstent.) Model Essentally, the model s Y µ + + α + ε j c where Y j s the dependent varable for subject n condton j µ s the overall mean of Y c s the contrbuton of the covarate for subject α j s the contrbuton from a partcular level (level j) of factor ε j s everythng else (the error n measurng subject n condton j, resdual, ndvdual varaton, etc.): ε Y µ + c + α ) ( j j j nd everyone clams ths s ther model (Myers & Well, 995, p. 436; Howell, 997, pp ); see also Keppel (99, pp ). However, what s actually gong on s a bt more sophstcated there s are two defntons for c and α j, dependng on what we want to test. What actually happens s ths (best explaned by Myers & Well, 995, pp ; but also by Howell, 997, pp. 59-): We can vew any NOV hypothess test as a comparson of two models. For example, a smple one-way NOV s a comparson of a full model that ncorporates the effect of a

142 7: Specfc desgns 4 factor ( Yj µ + α + εj ) wth a restrcted model that doesn t n ths case, the restrcted model s µ + ε. Y Contrastng two models. he correct way of contrastng a full (F) model and a restrcted (R) model s to use ths F test (Myers & Well, 995, p. 44): F( df df ), error(r) error(f) dferror(f) ( SS SS ) ( df df ) error(r) SS error(f) error(f) df error(r) error(f) error(f) Or, we could rewrte that, snce SS total SS model + SS error and df total df model + df error : F( df df ), model(f) model(r) dferror(f) ( SS SS ) ( df df ) model(f) model(r) SS error(f) df model(f) error(f) model(r) For a one-way NOV, ths formula reduces to F MS /MS S/, our usual formula for testng the effects of see p. 86 n the secton on GLMs. n alternatve formulaton uses the R values for each model (Howell, 997, p. 578): f f and r are the number of predctors n the full and reduced models, ( N f )( R f Rr ) Ff r, N f. ( f r)( R f ) Now we apply that prncple to NCOV. o test the effects of the factor, one model s calculated testng just the effect of the covarate C. hat model s our usual regresson NOV model, Y µ + c + ε, where µ s the overall mean and c s the contrbuton of the covarate, calculated usng the overall regresson ( c b( C C ) ) snce n ths model we have no nformaton about whch level of a gven subject s at, so we can t calculate the pooled wthn-groups slope yet. hen we calculate another model ncludng the factor. hat model s Yj µ + c + α j + εj, where α j s the extra contrbuton of the factor. nd knowledge of that factor allows us to mprove our regresson as well, because t allows us to calculate two regresson lnes wth the same slope (the pooled wthn-groups slope, b S/ ) but dfferent ntercepts (Myers & Well, 995, p. 44). So the extra contrbuton s α j µ + bs / ( Cj C ) ( µ + c ). We j j compare those two models. o test the effects of the covarate C, one model s calculated testng just the effect of the factor. hat model s our usual one-way NOV model Yj µ + α j + εj, where µ s the overall mean and α j s the contrbuton from a partcular level (level j) of the factor ( α µ µ ). hen we calculate another model ncludng the covarate C. hat model s j j Yj µ + c + α j + εj, where c s the extra contrbuton of the covarate, usng the pooled wthn-groups slope (.e. usng the nformaton about whch subject s at whch level of factor ),.e. c bs / ( Cj C ). We compare those two models. he complcated pcture above shows ths. he top row parttonng SS total nto SS overall regresson, SS (adjusted), and an error term, corresponds to testng the effects of over and above those of the covarate. he mddle row parttonng SS total nto SS, SS wthn-group regresson, and an error term, corresponds to testng the effects of C over and above those of the factor. Snce the covarate and the factor may be correlated (provde mutual nformaton), the questons what does do? and what does C do? are not ndependent; we therefore ask what does do, over and above the effects of C? and what does C do, over and above the effects of? Sources of varance See above.

143 7: Specfc desgns 43 NOV table Covarates account for degree of freedom. Source d.f. SS F C cov SS reg(s/) MS reg(s/) /MS S/,adjusted a SS, adjusted MS,adjusted /MS S/,adjusted Error N a SS S/,adjusted otal N SS total where N s the number of subjects and a the number of levels of factor. Note that the SS components for C,, and error do not add up to SS total. hs s confusng; the method of parttonng s descrbed above. Correlaton coeffcent from NCOV SPSS technque See dscusson under the one wthn-subjects covarate model (p. 5) for detals of how to obtan correlaton coeffcents (r, r ) and parameter estmates (b) from NCOV. Data layout: C depvar datum datum datum datum datum datum datum datum datum datum datum datum Syntax: UNINOV depvar BY a WIH c /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN c a. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter as a fxed factor. Enter C cov as a covarate. Note that the nteracton term (C cov ) s not ncluded n ths model see below for a verson wth the nteracton.

144 7: Specfc desgns he covarate and factor nteract lternatve names nalyss of covarance (NCOV) allowng covarate factor nteracton nalyss of covarance (NCOV): full model to check homogenety of regresson Homogenety of slopes desgn NCOV (see p. 88 ) Example Rats receve sham surgery ( ) or lesons of the nucleus accumbens core ( ). hey are then traned n a task n whch they may press a lever freely; each lever press produces a pellet some tme later. For each rat, we measure the mean tme between pressng the lever and recevng the pellet (C cov ; one value per subject). hs s a contnuous varable. We also measure ther learnng speed (dependent varable). Does the learnng speed depend on the delay each rat experenced (man effect of C cov )? Does the learnng speed depend on the group they were n (man effect of )? Does the way the learnng speed depends on the delay depend n turn on whch group they were n (C cov nteracton)? Note the nterpretatve dffcultes (dscussed above) that can plague any NCOV f you don t thnk thngs through very carefully. Notes llows the covarate to nteract wth the factor that s, allows for the possblty that the effects of the factor dffer dependng on the value of the covarate, or (equvalently) that the effects of the covarate dffer dependng on the level of the factor. See above for a non-nteracton verson. Howell (997, pp ) dscusses the approach to a standard NCOV that assumes homogenety of regresson (that the regresson coeffcents are equal across levels of the factor,.e. that there s no covarate factor nteracton). We dscussed ths reduced model NCOV n above (p. 38). Howell (997, pp ) uses the full model, whch ncludes the nteracton term, to test the assumpton of homogenety of regresson before usng the reduced model. However, there are tmes when we are nterested n the nteracton term for ts own sake (see Example above). Model descrpton Model depvar C cov S Y j µ + c + α + cα + ε j j j where Y j s the dependent varable for subject n condton j µ s the overall mean c s the contrbuton of the covarate for subject α j s the contrbuton from a partcular level (level j) of factor cα j s the nteracton of the covarate for subject wth level j of factor ε j s everythng else (the error n measurng subject n condton j, resdual, ndvdual varaton, etc.): ε Y µ + c + α + cα ) ( j j Just as before, we can t defne c, α j and so on n just one way, snce they may be correlated. We ll have to ask what the covarate contrbutes over and above the factor, and so on. he test for the nteracton term (Myers & Well, 995, p. 447; Howell, 997, p ) nvolves the comparson of a full model n whch the regresson slopes can dffer for each group, or level of (so the regresson slopes are b ): Yj µ + α j + b ( C j j C ) + ε j j and a restrcted model n whch each group has the same slope: Y µ + α + b( C C ) + ε j j j pproach : testng the homogenety of regresson assumpton. est the nteracton term as above (.e. perform an NCOV ncludng the factor covarate assumpton). If the nteracton term s not sgnfcant, the slopes don t dffer. Drop the nteracton term out of the model and perform your usual NCOV (factor, covarate, no nteracton) safe n the knowledge that the j j j

145 7: Specfc desgns 45 assumpton of homogenety of regresson s vald. hs s why most textbooks test ths nteracton (Myers & Well, 995, p. 45; Howell, 997, p ). pproach : askng about the factor covarate assumpton for ts own sake. Perform the full analyss wth the nteracton; nterpret that drectly. Interpretaton of any man effects n the presence of an nteracton may be trcky, as t s n factoral NOV (Myers & Well, 995, p. 45). Sources of varance NOV table SS C, SS, SS C, SS error but these may not be ndependent, so they won t necessarly add up to SS total see above. Covarates account for degree of freedom. Source d.f. SS F C cov SS C MS C /MS error a SS MS /MS error C cov a SS C MS C /MS error Error N a SS error otal N SS total where N s the number of subjects and a the number of levels of factor. Correlaton coeffcent from NCOV SPSS technque See dscusson under the one wthn-subjects covarate model (p. 5) for detals of how to obtan correlaton coeffcents (r, r ) and parameter estmates (b) from NCOV. Data layout: C depvar datum datum datum datum datum datum datum datum datum datum datum datum Syntax: UNINOV depvar BY a WIH c /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN c a c*a. Note that the nteracton term (C cov ) s ncluded. Not entrely trval to accomplsh wth the SPSS menus. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter C as a covarate. Enter as a fxed factor. By default, SPSS wll not nclude the C cov nteracton. So you need to edt the Model drectly before runnng the analyss. Untck Full factoral by tckng Custom. Enter the desred terms (n ths case C,, C ).

146 7: Specfc desgns One between-subjects covarate and two between-subjects factors lternatve names Factoral analyss of covarance (factoral NCOV) Example Notes Suppose we are agan nterested n whether small cars are easer to handle. We can compare drvng profcency usng three cars: small, medum, and large (,, 3 ). One drver s tested n only one car. We have three groups of male drvers (B ), and three groups of female drvers (B ), whch we assgn to our three cars n a standard factoral desgn. We also want to account for varaton n drvng experence (C cov ; one value per subject). here s nothng to stop you ncludng covarate factor nteractons n your model, though we won t present them here. he general lnear model wll also be perfectly happy for you to nclude covarate covarate nteractons, f you thnk that s meanngful. hnk carefully, though; ths would be a complex desgn! We won t present that here. More detaled dscusson of ths desgn s gven by Myers & Well (995, pp ). Model descrpton (S subjects) Model depvar C cov + B S Y µ + + α + β + αβ + ε jk c j k jk jk where Y jk s the dependent varable for subject n condton j, B k µ s the overall mean c s the contrbuton from covarate C for subject α j s the contrbuton from a partcular level (level j) of factor β k s the contrbuton from a partcular level (level k) of factor B αβ jk s the contrbuton from the nteracton of level j of factor and level k of factor B ε jk s everythng else (the unqueness of subject n condton j of factor and condton k of factor B, error, ndvdual varaton, etc.). However, snce the predctors may be correlated, there s no unque way to defne the contrbutons of each of these components (see above). Sources of varance s the sources of varance may not be ndependent, the components (SS C, SS, SS B, SS B, SS error) may not add up to SS total ; see above. NOV table Source d.f. SS F C cov SS C MS C /MS error a SS MS /MS error B b SS B MS B /MS error B (a )(b ) SS B MS B /MS error Error ab(n ) SS error otal N abn SS total where a s the number of levels of factor, etc., N s the total number of observatons (subjects), and n s the number of subjects (or replcatons ) per cell. Correlaton coeffcent from NCOV See dscusson under the one wthn-subjects covarate model (p. 5) for detals of how to obtan correlaton coeffcents (r, r ) and parameter estmates (b) from NCOV.

147 7: Specfc desgns 47 SPSS technque Data layout: depvar B C datum level_ level_ datum datum level_ level_ datum datum level_ level_ datum datum level_ level_ datum datum level_ level_ datum datum level_ level_ datum datum level_ level_ datum datum level_ level_ datum Syntax: UNINOV depvar BY a b WIH c /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN c a b a*b. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter and B as between-subjects factors. Enter C as a covarate.

148 7: Specfc desgns wo or more between-subjects covarates (multple regresson) lternatve names Multple regresson Multple lnear regresson Example Suppose we want to predct marks n undergraduate exams on the bass of -Level ponts ( cov ) and IQ (B cov ). Notes See Howell (997, p. 5 on) for a dscusson of multple regresson, and Howell (997, pp ) for a dscusson of the use of multple covarates. standard multple regresson solves the equaton Y ˆ b + b + b + + b p p where b s the ntercept and b, b, b p represent the regresson coeffcents (slopes) for the predctors,, p respectvely. In general, as for lnear regresson, ths equaton s solved so as to perform least-squares regresson,.e. to mnmze ( Y Yˆ) However, f the two covarates are themselves correlated, there wll be a problem of nterpretaton of effects nvolvng one or other of them (because we wll have non-orthogonal sums of squares, as dscussed earler n the context of unequal group szes; see p. 7 and p. 97 ). Model descrpton (S subjects) Model C cov + D cov + S For the two-covarate case, C cov + D cov + S. o acheve standard multple regresson, n the two-predctor case, the multple regresson equaton above leads us to ths model n our usual NOV notaton: Y µ + c + d + ε where Y s the dependent varable for subject µ s the overall mean c s the contrbuton from covarate C for subject d j s the contrbuton from covarate D for subject ε s everythng else (the error n measurng subject, resdual, ndvdual varaton, etc.). However, snce the predctors may be correlated, there s no unque way to defne the contrbutons of each of these components (see above). he C cov D cov nteracton s not ncluded for conventonal multple lnear regresson. Sources of varance For the two-covarate case, f the covarates are ndependent, then SS total SS C + SS D + SS error. But f the covarates are themselves correlated, the contrbutons of each won t necessarly add up to the total (Myers & Well, 995, pp ). NOV table Covarates account for degree of freedom each. For the two-covarate case, Source d.f. SS F C cov SS C MS C /MS error D cov SS D MS D /MS error Error N 3 SS error otal N SS total where N s the number of subjects.

149 7: Specfc desgns 49 Correlaton coeffcents and parameter estmates SPSS technque See dscusson under the one wthn-subjects covarate model (p. 5) for detals of how to obtan correlaton coeffcents (r, r ) and parameter estmates (b) from NCOV. See dscusson of effect sze (p. 97 ) to see how to nterpret them. Data layout: C D depvar datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum datum Ether perform the analyss as a multple lnear regresson (nalyze Regresson Lnear; enter C and D as the ndependent varables), whch gves ths syntax: REGRESSION /MISSING LISWISE /SISICS COEFF OUS R NOV /CRIERIPIN(.5) POU(.) /NOORIGIN /DEPENDEN depvar /MEHODENER c d. Or run t as an NOV (nalyze General Lnear Model Unvarate; enter C and D as covarates), whch gves ths syntax: UNINOV depvar WIH c d /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN c d. Note that the nteracton term (C cov D cov ) s not ncluded. You could nclude t f you wanted the software won t complan but you d have to thnk very carefully about ts nterpretaton.

150 7: Specfc desgns wo or more between-subjects covarates and one or more between-subjects factors lternatve names Factoral analyss of covarance (factoral NCOV) wth multple covarates Example Notes Modfyng Howell s (997, pp. 65-6) example slghtly, suppose we want to look at the effect of two teachng styles () and two classroom temperatures (B) on student performance usng a factoral desgn. We mght also want to partal out the effect of age (C cov ) and IQ (D cov ). No problem statstcally, at least. here s nothng to stop you ncludng covarate factor nteractons n your model, though we won t present them here. he general lnear model wll also be perfectly happy for you to nclude covarate covarate nteractons, f you thnk that s meanngful. hnk carefully, though; ths would be a complex desgn! We won t present that here. s n the prevous desgn, f the two covarates are correlated, there wll be a problem of nterpretaton (because we wll have non-orthogonal sums of squares, as dscussed earler n the context of unequal group szes; see p. 7 and p. 97 ). Desgns wth more than one covarate are brefly dscussed by Myers & Well (995, p. 459), as s polynomal NCOV (Myers & Well, 995, p. 46); see also p. 88. Model descrpton (S subjects) Followng our example, we ll llustrate a two-covarate, two-factor model: depvar C cov + D cov + B S Model Y jk µ + c + d + α + β + αβ + ε j k jk jk where Y jk s the dependent varable for subject n condton j, B k µ s the overall mean c s the contrbuton from covarate C for subject d s the contrbuton from covarate D for subject α j s the contrbuton from a partcular level (level j) of factor β k s the contrbuton from a partcular level (level k) of factor B αβ jk s the contrbuton from the nteracton of level j of factor and level k of factor B ε jk s everythng else (the unqueness of subject n condton j of factor and condton k of factor B, error, ndvdual varaton, etc.). However, snce the predctors may be correlated, there s no unque way to defne the contrbutons of each of these components (see above). Sources of varance s the sources of varance may not be ndependent, the components (SS C, SS D, SS, SS B, SS B, SS error ) may not add up to SS total ; see above. NOV table Source d.f. SS F C cov SS C MS C /MS error D cov SS D MS D /MS error a SS MS /MS error B b SS B MS B /MS error B (a )(b ) SS B MS B /MS error Error ab(n ) SS error otal N abn SS total where a s the number of levels of factor, etc., N s the total number of observatons (subjects), and n s the number of subjects (or replcatons ) per cell. Correlaton coeffcents and effect szes See dscusson under the one wthn-subjects covarate model (p. 5) for detals of how to obtan correlaton coeffcents (r, r ) and parameter estmates (b) from NCOV. See dscusson of effect sze above (p. 97 ) to see how to nterpret them.

151 7: Specfc desgns 5 SPSS technque Data layout: depvar B C D datum level_ level_ datum datum datum level_ level_ datum datum datum level_ level_ datum datum datum level_ level_ datum datum datum level_ level_ datum datum datum level_ level_ datum datum datum level_ level_ datum datum datum level_ level_ datum datum Syntax: UNINOV depvar BY a b WIH c d /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN c d a b a*b. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter and B as between-subjects factors. Enter C and D as covarates.

152 7: Specfc desgns One wthn-subjects covarate lternatve names Multple regresson wth the covarate and Subject as predctors Example We measure gastrc ph and PaCO (partal pressure of arteral carbon doxde) for a group of 8 subjects, makng several measurements on each subject so we end up wth 47 measurements (see Bland & ltman, 995a). Is there a relatonshp between PaCO and ph? We must not analyse ths as f there were 47 ndependent observatons. Subjects may vary wdely n ther gastrc ph and arteral PaCO, yet there may be a consstent relatonshp wthn each subject between the two, and ths s what we want to look at. Notes I ve largely made up the model and sources of varance here, so I hope t s correct. It does match Bland & ltman s answer. Note that t s logcally dentcal to the model we looked at earler wth one between-subjects covarate and one between-subjects factor (the verson n whch the covarate and the factor do not nteract), except that our factor s now subjects tself; the only dfference s that subjects s a random, not a fxed, factor. Data from Bland & ltman (995a); orgnally from Boyd et al. (993). Model descrpton (S subjects) Model depvar (C cov + S) Y µ + + π + ε c where Y s the dependent varable for subject µ s the overall mean c s the contrbuton from covarate C for subject π s the average contrbuton from a partcular subject (subject ) ε jk s everythng else (measurement error, ntra-subject varaton, etc.). Sources of varance SS total SS subjects + SS C + SS error NOV table Source d.f. SS F Between subjects s SS subjects MS subjects /MS error C SS C MS C /MS error Error N s SS error otal N SS total where N s the total number of observatons and s s the number of subjects.

153 7: Specfc desgns 53 Correlaton coeffcent from NCOV Note also that snce f we are predctng a varable Y (callng the predcton Ŷ ) we can express r n terms of sums of squares: r SS SS Yˆ Y SS Yˆ SS Yˆ + SS resdual (see Correlaton & Regresson handout at If we rewrte ths for our present case, C s the thng that makes the predcton. he total wthn-subjects varaton s what we re left wth after we ve accounted for between-subjects varaton ( SS total SS subjects SS C + SS error ) and the varaton accounted for by the predcton from C s SS C. So the proporton of the wthn-subjects varaton accountable for by C s: r SS C SSC + SS hs allows us to work out the wthn-subjects correlaton coeffcent from the NCOV table. o obtan r tself, take the square root of r and combne t wth the sgn of the regresson coeffcent. o obtan regresson coeffcents n SPSS, tck Parameter estmates n the NOV Optons dalogue box, or add /PRIN PRMEER to your SPSS syntax. he regresson coeffcent (slope) wll appear n the B column and the row correspondng to the covarate. error SPSS technque Data layout: subject C depvar datum datum datum datum datum datum datum datum datum datum 3 datum datum 3 datum datum 3 datum datum 3 datum datum 3 datum datum 4 datum datum Syntax: UNINOV depvar BY subject WIH c /RNDOM subject /MEHOD SSYPE(3) /PRIN PRMEER /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN c subject. Usng the menus, select nalyze General Lnear Model Unvarate. Enter Subject as a random factor and C as a covarate.

154 7: Specfc desgns One wthn-subjects covarate and one between-subjects factor he covarate and factor do not nteract lternatve names Example We gve a drug ( ) or placebo ( ) to two groups of subjects to see f t affects ther secreton of growth hormone (dependent varable). he drug s effects are known to last for days, and we know that tme of day (C) also affects growth hormone secreton we beleve there s a lnear relatonshp between tme of day measured n a certan way and growth hormone levels. Each subject only experences ether the drug or the placebo, but we measure each subject repeatedly at several dfferent tme ponts. We wsh to partal out the effects of tme of day to have a better chance of fndng an effect of the drug. (Note that our expermental desgn must ensure that there s no systematc relatonshp between and C, or nterpretaton wll be well-ngh mpossble for example, t would be vtal not to measure the drug group n the evenng and the placebo group n the mornng.) Notes Model descrpton (S subjects) Model depvar (C cov + S) [but wth no C cov term n the model] I would guess ether ths: Y j + α + π j / µ + c + ε where Y j s the dependent varable for subject j n condton µ s the overall mean α s the contrbuton from level of factor π j/ s the average contrbuton from a partcular subject (subject j), who s only measured n condton c jk s the contrbuton from covarate C for subject j ε j s everythng else (measurement error, ntra-subject varaton, etc.). j j or ths: Y j + α + π j / + c j + πc j / µ + ε where πc j/ s the nteracton of the covarate C wth subject j (who s only measured n condton ) ε j s redefned as everythng else n ths new model Should we nclude the subject covarate nteracton, C S/ (allowng a dfferent regresson slope for the covarate for each subject)? Maybe that depends on the stuaton. Obvously, to nclude t, we must have multple measurements for each subject. One approach, I suppose, would be to test the full model and proceed to the smpler model f the subject covarate nteracton doesn t contrbute sgnfcantly. Includng t wll mprove the power to detect effects of C probably at the expense of power to detect effects of (see below). j Sources of varance NOV table he sources of varaton (SS, SS error-between, SS C, perhaps SS C S/, and SS error-wthn ) may not be ndependent and may therefore not add up to SS total. If the effects of and C are uncorrelated, the NOV table wll look lke ths: Source d.f. SS F Between subjects: s an a SS MS /MS S/ error (S/) a(n ) SS S/ Wthn subjects: (N ) (s ) N s C cov SS C MS C /MS error-wthn

155 7: Specfc desgns 55 error-wthn N s SS error-wthn lternatve for wthn subjects (n the model wth the C S/ term): C cov SS C MS C /MS error-wthn C cov S/ a(n ) SS C S/ MS C S/ /MS error-wthn error N s a(n ) SS error-wthn otal N SS total where a s the number of levels of factor and N s the total number of observatons ( aun), n s the number of subjects per group (where group s defned by factor ), and s s the total number of subjects. SPSS technque Data layout: Subject C depvar datum datum datum datum datum datum datum datum datum datum datum datum 7 datum datum 7 datum datum 7 datum datum 7 datum datum 8 datum datum 8 datum datum Syntax (usng the notaton subject(a) rather than the functonally equvalent subject*a for the term S/): GLM depvar BY a subject WIH c /RNDOM subject /MEHOD SSYPE(3) /INERCEP INCLUDE /PRIN DESCRIPIVE HOMOGENEIY /CRIERI LPH(.5) /DESIGN a subject(a) c c*subject(a). or not Choose whether or not to nclude the C S/ term If you do nclude t, the C S/ term s calculated and tself assessed aganst the resdual MS error, whereas otherwse C S/ s part of the error term. hs nevtably reduces the resdual MS error and wll therefore mprove power to detect effects of C (ether as an effect of C or a C S/ nteracton), probably at the expense of power to detect the effect of. One thng worth notcng: SPSS assesses MS aganst a lnear combnaton of MS S/ and the resdual (what t calls MS error ). You mght thnk that t should be assessed only aganst MS S/ and ths s what t wll do f and C are totally uncorrelated. It s possble to force SPSS to do ths at any tme wth a custom hypothess test usng the syntax /ES a VS subject(a). But ths may not be a good dea, because f and C are partally correlated, SPSS tres to sort thngs out. It calculates ts error terms usng Satterthwate s (946) denomnator synthess approach. If and C are pretty much uncorrelated, you ll fnd that the lnear combnaton t uses as ts error term s heavly weghted towards MS S/ (e.g..97 MS S/ +.3 MS error ). If they re correlated, the weghtng wll change (e.g..39 MS S/ +.76 MS error ). nd f and C are substantally correlated, your nterpretaton may be very dffcult n any case. In any case, the easest way to thnk about the calculatons gong on n ths sort of analyss s to vew each test as a comparson of two models (see secton on GLMs, p. 84 ). For example, assumng we re usng the usual method (SPSS s ype III sums of squares) for partallng out

156 7: Specfc desgns 56 the effects of mutually correlated predctors, the test of the effect of C s a test of the dfference between a full model, ncludng C, and a restrcted model ncludng every effect but C, and so on for all the other terms. Not entrely trval to accomplsh wth the SPSS menus. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter as a fxed factor; enter Subject as a random factor; enter C as a covarate. Snce SPSS wll not gve you the correct model by default (t wll nclude S), you then need to edt the Model drectly before runnng the analyss. Untck Full factoral by tckng Custom. Enter the desred terms as lsted n the NOV table he covarate and factor nteract lternatve names Examples One of my examples, so I do hope t s approprate. Subjects are assgned to two groups ( bran leson, sham). hey are gven a task n whch they have a choce between two levers. Lever delvers a sngle food pellet wth probablty p. Lever B delvers four pellets, but wth a probablty that ranges from to.65; the probablty changes n steps and the rats have an opportunty to experence the probablty currently n force before they choose between levers (small, certan reward) and B (large, uncertan reward). he dependent varable s the proporton of trals on whch they choose lever B. We could analyse these wth two factors: (group: leson/sham; between subjects) and B (probablty: /.5/.5/.5/.65; wthn subjects). But snce delvery of the large reward s under the control of a random process, the probablty experenced by the subjects may not always match the programmed probablty (e.g. f they have trals and the programmed probablty s.5, t s perfectly possble that they get 3 rewarded and 7 unrewarded trals, gvng an experenced probablty of only.3). So rather than usng programmed probablty as a wthn-subjects factor, we could use experenced probablty as a wthn-subjects covarate (call t C). We can then ask whether the probablty nfluenced choce (man effect of C), whether the leson nfluenced choce (man effect of ), and whether the leson nfluenced the effect of probablty ( C nteracton). Subjects are assgned to two groups ( bran leson, sham). hey respond freely on two levers, left and rght, to receve food pellets. Both levers delver food wth an element of randomness. Rats are tested for several sessons. cross sessons, the relatve number of pellets delvered by each lever vares. For each sesson, we calculate the proporton of responses allocated to the left lever the relatve response dstrbuton (dependent varable) and the proporton of the total number of pellets that were earned by respondng on the left lever the relatve renforcer dstrbuton (C). Both are contnuous, rather than dscrete, varables. Dd the renforcer dstrbuton nfluence respondng (man effect of C)? Dd the leson nfluence respondng (man effect of )? Dd the leson nfluence the way the anmals responded to the renforcer dstrbuton (nteracton between C and )? Notes Model descrpton (S subjects) Model In terms of the model, ths s logcally equvalent to the one between-subjects factor, one wthn-subjects factor desgn dscussed earler (q.v.). he computerzed NOV process used by SPSS, based on a general lnear model (GLM), does not care whether predctor varables are dscrete (factors) or contnuous (covarates), except n the way that t bulds ts default model (whch we need to overrde here). depvar (C cov S) Well, I m makng ths up agan; I would guess the full model would be essentally the same as the one between, one wthn desgn dscussed earler (q.v.): Y jk + α + π j / + ck + αck + πc jk / µ + ε jk where Y jk s the dependent varable for subject j n condton µ s the overall mean α s the contrbuton from a partcular level (level ) of factor

157 7: Specfc desgns 57 π j/ s the contrbuton from a partcular person or subject (subject j), who only serves wthn condton ( subject wthn group, or S/) (here s no straghtforward nteracton of wth S: every subject s only measured at one level of, so ths term would be ndstngushable from the subject-only effect π j/.) c k s the contrbuton from the covarate C for subject j (call t C j for the moment) αc k s the contrbuton from the nteracton of and C j πc jk s the contrbuton from the nteracton of C k wth subject j (who only serves wthn condton ) f you choose to nclude t (see above) ε jk s everythng else (the expermental error assocated wth measurng person j, who always experences treatment, wth covarate contrbuton C k ). Sources of varance If and C are uncorrelated, we could partton the varance lke ths: SS total SS between subjects + SS wthn subjects SS between subjects SS + SS S/ SS wthn subjects SS C + SS C + SS C S/ + SS error or f you don t nclude SS C S/, you d just wrte the wthn-subjects bt lke ths: SS wthn subjects SS C + SS C + SS error However, f and C are correlated, the sources of varance wll not be ndependent and wll not add up to SS total. NOV table If and C are uncorrelated, the NOV table would look lke ths: Source d.f. SS F Between subjects: s an a SS MS /MS S/ error S/ a(n ) SS S/ Wthn subjects: N s C cov SS C MS C /MS error-wthn C cov a SS C MS C /MS error-wthn C cov S/ a(n ) SS C S/ MS C S/ /MS error-wthn error-wthn N s an SS error-wthn Wthn subjects n a model that doesn t nclude C cov S/: C cov SS C MS C /MS error-wthn C cov a SS C MS C /MS error-wthn error-wthn N s a SS error-wthn otal N SS total where a s the number of levels of factor and N s the total number of observatons ( aun), n s the number of subjects per group (where group s defned by factor ), and s s the total number of subjects. SPSS technque Data layout: Subject C depvar datum datum datum datum datum datum datum datum datum datum datum datum 7 datum datum 7 datum datum 7 datum datum 7 datum datum

158 7: Specfc desgns 58 8 datum datum 8 datum datum Syntax: GLM depvar BY a subject WIH c /RNDOM subject /MEHOD SSYPE(3) /INERCEP INCLUDE /PRIN DESCRIPIVE HOMOGENEIY /CRIERI LPH(.5) /DESIGN a subject(a) c c*a c*subject(a). or not Choose whether or not to nclude the C S/ term If you do nclude t, the C S/ term s calculated and tself assessed aganst the resdual MS error, whereas otherwse C S/ s part of the error term. hs nevtably reduces the resdual MS error and mproves power to detect terms nvolvng C (that s, C, C, and C S/), probably at the expense of power to detect the effect of. Note that SPSS calculates ts error terms usng approprate lnear combnatons to deal wth any correlaton between and C (see above). Not entrely trval to accomplsh wth the SPSS menus. Usng the menus, choose nalyze General Lnear Model Unvarate. Enter as a fxed factor; enter Subject as a random factor. Snce SPSS wll not gve you the correct model by default (t wll nclude S and not nclude C ), you then need to edt the Model drectly before runnng the analyss. Untck Full factoral by tckng Custom. Enter the desred terms as above.

159 7: Specfc desgns Herarchcal desgns: two or more levels of relatedness n measurement Subjects wthn groups wthn treatments (S/G/) lternatve names Splt-splt plot desgn Double-splt desgn Doubly-nested desgn Herarchcal desgn Bloody complcated Example he smplest herarchcal desgn (Myers & Well, 995, pp. 3): subjects (S) are tested n groups (G). Dfferent groups are assgned to dfferent levels of some treatment (). One subject s only ever n one group, and one group s only ever n one treatment. hs desgn can be wrtten S/G/ (subjects wthn groups wthn treatments). Specfc examples: Prmary school pupls are taught n classes. We assgn several classes to one teachng method, several other classes to a dfferent teachng method, and so on. One pupl s only ever n one class; one class only ever uses one teachng method. he desgn s pupls wthn classes wthn teachng methods. Pupl and class are random factors. Dfferent methods of rearng rats mght be compared, wth each rearng method beng appled to several ltters of rats. (Rats wthn a ltter are related genetcally, so we should take nto account ths potental source of correlaton between scores of rats from the same ltter. Statng the same thng n a dfferent way, two randomly-selected rats may dffer not only because they are dfferent ndvduals, or because they experenced dfferent treatments, but because they come from dfferent ltters.) he desgn s rats wthn ltters wthn rearng methods. Rat and ltter are random factors. Model n ndvdual score mght be represented as Y jk, where,, a (number of treatment levels) j,, g (number of groups wthn a treatment level) k,, n (number of subjects n a group) hen or Y jk Y jk Yjk µ + α + γ + ε j jk µ + ( µ µ ) + ( µ µ ) + ( Y µ ( µ µ ) + ( µ µ ) + ( Y j j jk jk µ ) µ ) where Y jk s the dependent varable n condton, G j for subject k µ s the overall mean α s the contrbuton from level of factor ( ): α µ µ and α. γ j s the contrbuton from level j of group G n condton (G j ) relatve to the mean of : γ µ and γ. j G j µ j ε jk s everythng else (the devaton of subject k from ts group mean G j ): εjk Y jk µ j. If we sum and square both sdes (and elmnate cross-product terms that sum to zero), we get: j j SS total SS + SS G/ ( Y µ ) ng ( µ µ ) + n ( µ µ ) + ( Y µ ) j k jk + SS j S/G/ j j k jk j Sources of varance Subject and group are random factors; s a fxed factor. We d wrte the model lke ths: so SS total SS between-groups + SS wthn-groups SS between-groups SS + SS G/ SS wthn-groups SS S/G/ SS total SS + SS G/ + SS S/G/

160 7: Specfc desgns 6 We would state G/ as group wthn, and S/G/ as subject wthn group wthn, or smply subject wthn group. Smlarly, df total df + df G/ + df S/G/ NOV table Source d.f. SS F Between groups: ag a SS MS /MS G/ G/ a(g ) SS G/ MS G/ /MS S/G/ Wthn groups: S/G/ ag(n ) SS S/G/ otal N agn SS total where N s the total number of observatons and a, g, and n are as defned above. Note that the error term for s G/, and the error term for G/ s S/G/. SPSS technque G Subject depvar datum datum 3 datum 4 datum 5 datum 6 datum 7 datum 8 datum 3 9 datum 3 datum 3 datum 3 datum 4 3 datum 4 4 datum 4 5 datum 4 6 datum It doesn t matter f you use the same dentfers to code groups wthn dfferent levels of. For example, you can call the groups and and the groups 3 and 4, as I ve done above or you can call the groups and and the groups and agan. Snce the desgn knows that groups are nested wthn levels of, t doesn t care about how you label them. (Of course, each group must have a unque name wthn each level of.) GLM depvar BY a g subject /RNDOM g subject /DESIGN a g(a) subject(g(a)). Further notes It s a common mstake to use an experment wth ths knd of desgn but not to put the Group factor nto the analyss. People often analyse these knds of data only takng nto account the factor. hat wll generally overestmate the F rato for (gve a lower p value than t should) (Myers & Well, 995, pp. 35-7). On the other hand, f Group has few df, the value of MS G ( SS G / df G ) wll be large and we wll have low power to detect effects of. he alternatve model s to gnore the effect of G (what most people do wthout thnkng about t): SS total SS + SS S/ where SS S/ s the pool of G/ and S/G/. hs s what you get when you run a one-way NOV, gnorng the effect of G. In general, E(MS S/ ) s less than E(MS G/ ), so you re more lkely to fnd a sgnfcant effect of (Myers & Well, 995, p. 36). Myers & Well (995, pp. 5, 37) recommend that you only pool (gnore the effect of G) when you ve already run an analyss wth G ncluded and ths prelmnary test of the effect of G was not sgnfcant at the α.5 level, and you have pror reason to beleve that the thngs you re poolng over reflect only chance varablty (n ths case, that you have pror reason to thnk that groups don t dffer systematcally).

161 7: Specfc desgns 6 s Myers & Well (995, p. 339) put t, wshng some varance component [e.g. G] to be zero does not make t so, and the prce of wrongly assumng that the component s zero s ordnarly a ype error n testng treatment effects of nterest [.e. declarng the effect of to be sgnfcant when t sn t]. If you can t legtmately pool, then you need to have a hgh value of g (many groups), so you get hgh df G/ and therefore low MS G/, and therefore good power to detect effects of (whch uses MS G/ as ts error term). hs should be farly obvous, although many people fal to realze t: f one prmary school class s taught usng one method and another s taught usng another method, s a dfference n class means due to dfferent methods () or to dfference n the personal nteractons wthn the two classes (G)? hey re confounded Groups versus ndvduals If you need to compare the effects of beng n a group ( group condton) to the effect of not beng n a group ( ndvdual condton), there s a specal analytcal technque (Myers & Well, 995, pp. 37-9). For example, f 5 students study a topc ndvdually, whle another 5 students study the topc n fve dscusson groups of three, you can analyse the effect of beng n a group. hs s a farly common problem n socal psychology ddng a further wthn-group, between-subjects varable (S/GB/) Example Model Subjects (S) are part of groups (G). Wthn each group, subjects are ether anxous or not (anxety: B). Sets of groups are gven dfferent treatments (). So G s crossed wth B (all groups have anxous and non-anxous subjects; anxous and non-anxous subjects are found n all groups) but subjects are nested wthn GB (a subject s only part of one group and s ether anxous or not) and groups are nested wthn treatments. he model can be wrtten S/GB/ (or S/BG/). n ndvdual score mght be represented as Y jkl, where,, a (number of treatment levels) j,, b (number of B levels wthn a group, or wthn a treatment level) k,, g (number of groups wthn a treatment level) l,, n (number of subjects n a group) hen Yjk + α + γ j / + β j + αβj + γβkj / µ + ε jkl here are no nteractons nvolvng subjects (because subjects cross wth none of the other three varables: one subject only ever experences one level of G, B, and ). G does not cross wth, so there s no G or BG term. Sources of varance Subject and group are random factors; and B are fxed factors. We d wrte the model lke ths: so SS total SS between-groups + SS wthn-groups SS between-groups SS + SS G/ SS wthn-groups SS B + SS B + SS GB/ + SS S/GB/ SS total SS + SS G/ + SS B + SS B + SS GB/ + SS S/GB/ We would state G/ as group wthn, and S/G/ as subject wthn group wthn, or smply subject wthn group. Smlarly, df total df + df G/ + df B + df B + df GB/ + df S/GB/

162 7: Specfc desgns 6 NOV table Source d.f. SS F Between G: ag a SS MS /MS G/ G/ a(g ) SS G/ Wthn G: ag(bn ) B b SS B MS /MS GB/ B (a )(b ) SS B MS B /MS GB/ GB/ a(g )(b ) SS GB/ MS GB/ /MS S/GB/ S/GB/ gba(n ) SS S/GB/ otal N agbn SS total where N s the total number of observatons and a, g, and n are as defned above. Note that the error term for s G/, and the error term for G/ s S/G/. SPSS technque G B Subject depvar datum datum 3 datum 4 datum 5 datum 6 datum 7 datum 8 datum 3 9 datum 3 datum 3 datum 3 datum 4 3 datum 4 4 datum 4 5 datum 4 6 datum See the notes about group codng above. GLM depvar BY a g b subject /RNDOM g subject /DESIGN a g(a) b a*b g*b(a) subject(g*b(a)). hat seems to work (Myers & Well, 995, p. 33, but note ther typo for the F value for the effect of B) ddng a wthn-subjects varable (US/GB/) Example Model Sources of varance We take the prevous model to begn wth: subjects (S) are part of groups (G). Wthn each group, subjects are ether anxous or not (anxety: B). Sets of groups are gven dfferent treatments (). Now we measure each subject four tmes (tral: U). U s crossed wth S (snce every subject experences all four trals). So our desgn can be wrtten US/GB/ (Myers & Well, 995, p. 333). See sources of varance below, whch follow drectly from the model and are easer to grasp. he prevous model descrbes the between-subjects varablty. We just need to add wthnsubjects terms U, and the nteracton of U wth each of the between-subjects sources from the last model: SS total SS between-groups + SS wthn-groups SS between-groups SS + SS G/ SS wthn-groups SS wthn-groups-between-subjects + SS wthn-subjects SS wthn-groups-between-subjects SS B + SS B + SS GB/

163 7: Specfc desgns 63 SS wthn-subjects SS U + SS U + SS UG/ + SS UB + SS UB + SS UGB/ + SS US/GB/ NOV table We have g groups at each of a levels of. Wthn each group, there are b levels of B and n subjects at each of those levels. So we have bn subjects n each of ag groups, for a total of agbn subjects. Each subject provdes one score at u levels of U agbnu scores n all. Source d.f. SS F Between G: ag a SS MS /MS G/ G/ a(g ) SS G/ Wthn G, between S: ag(bn ) B b SS B MS /MS GB/ B (a )(b ) SS B MS B /MS GB/ GB/ a(g )(b ) SS GB/ MS GB/ /MS S/GB/ S/GB/ gba(n ) SS S/GB/ Wthn S: agbn(u ) U u SS U MS U /MS UG/ U (u )(a ) SS U MS U /MS UG/ UG/ (u )a(g ) SS UG/ MS UG/ /MS UGB/ UB (u )(b ) SS UB MS UB /MS UGB/ UB (u )(a )(b ) SS UB MS UB /MS UGB/ UGB/ (u )a(g )(b ) SS UGB/ MS UGB/ /MS US/GB/ US/GB/ (u )gba(n ) SS US/GB/ otal N agbnu SS total op tp: to check your df add up to the total, t s quck to use Mathematca. For example, Smplfy[(u-) + (u-)(a-) + (u-)a(g-) + (u-)(b-) + (u-)(a-)(b-) + (u- )a(g-)(b-) + (u-)g*b*a(n-)] gves a b g n (- + u). When you really can t work out the approprate error terms, you can enter the model nto SPSS and see what t used. SPSS technque G B Subject U depvar datum datum 3 datum 4 datum datum datum 3 datum 4 datum 3 datum and so on. Just the same as the prevous example but wth the new U column. See the notes about group codng above. GLM depvar BY a g b subject u /RNDOM g subject /DESIGN a g(a) b a*b g*b(a) subject(g*b(a)) u u*a u*g(a) u*b u*a*b u*g*b(a) u*subject(g*b(a)) Nestng wthn-subjects varables, such as V/US/ Example Model Sources of varance We have fve experenced subjects and fve novce subjects (factor for experence; betweensubjects factor; fxed factor; a ; n 5; total of an subjects). Every subject s requred to solve problems, of whch 4 are easy, 4 are of ntermedate dffculty, and 4 are hard (factor U for dffculty; factor V for problem; u 3; v 4). hs s almost the same as a one between, two wthn desgn except that V s nested wthn U, not crossed wth t (Myers & Well, 995, p ). See sources of varance below, whch follow drectly from the model and are easer to grasp. We start by parttonng nto between-subjects and wthn-subjects varablty:

164 7: Specfc desgns 64 SS total SS between-subjects + SS wthn-subjects SS between-subjects SS + SS S/ o partton the wthn-subjects varablty, we can frst vew the desgn as nvolvng uv levels of stmul. hat s, n general, we begn parttonng wthn-subjects varablty by usng our smallest expermental unts. We also cross stmul wth all the between-subject sources: SS wthn-subjects SS stmul + SS stmul + SS stmul S/ We now partton the varablty due to stmul and ts nteractons: SS stmul SS U + SS V/U and cross those wth and S/ n turn: SS stmul SS U + SS V/U SS stmul S/ SS SU/ + SS SV/U We can partton the df n the same way. ctual values for the dfs are n square brackets: df total [abcn ] df between-subjects [an ] + df wthn-subjects [an(uv )] df between-subjects df [a ] + df S/ [a(n )] df wthn-subjects df stmul [uv ] + df stmul [(a )(uv )] + df stmul S/ [a(n )(uv )] df stmul df U [u ] + df V/U [u(v )] df stmul df U [(a )(u )] + df V/U [u(a )(v )] df stmul S/ df SU/ [a(n )(u )] + df SV/U [au(n )(v )] way of checkng the desgn s to lst all factors, random and fxed, notng any nestng. We have four:, S/, U, V/U. Now we consder all possble cross products of these factors. We wrte no next to them f t s not legtmate to cross them for example, f S s nested n, t cannot also cross wth t. S/ U V/U S/ U S/ V/U C V/U No U V/U SU/ SV/U No he four factors we started wth plus the four cross-products generated above are the terms of nterest. We should also consder crossng more than two factors, but n ths desgn no legtmate terms would turn up (for example, U V/U s not legtmate because V cannot be nested wthn U and stll cross wth t). Once we ve specfed our factors, we can enter them nto SPSS s desgn syntax. NOV table Source d.f. SS F Between S: an a SS MS /MS S/ S/ a(n ) SS S/ MS S/ /MS SU/ Wthn S: an(uv ) U u SS U MS U /MS SU/ U (a )(u ) SS U MS U /MS SU/ V/U u(v ) SS V/U MS V/U /MS SV/U V/U u(a )(v ) SS V/U MS V/U /MS SV/U SU/ a(n )(u ) SS SU/ MS SU/ /MS SV/U SV/U au(n )(v ) SS SV/U otal N abcn SS total he F ratos depend on whch factors are treated as fxed and whch as random (because that determnes the EMS values); the ratos presented above are for when S s random and, V, and U are all fxed. ctually, our example suggests that V, whch we wrte n full as V/U ( specfc

165 7: Specfc desgns 65 problem of a certan dffculty ) should be random; n that stuaton, the approprate error term must be syntheszed as a lnear combnaton of other terms. It seems that SPSS and BDMP8V do ths n slghtly dfferent ways (Myers & Well, 995, p. 337, versus SPSS analyss of the same data). SPSS technque Subject U V depvar datum datum 3 datum 4 datum datum datum 3 datum 4 datum 3 datum 3 datum 3 3 datum 3 4 datum datum datum 3 datum 4 datum 6 datum 6 datum 6 3 datum 6 4 datum GLM depvar BY a subject u v /RNDOM subject /DESIGN a subject(a) u v(u) a*u a*v(u) subject*u(a) subject*v(a*u). If V s a random factor too, you d want /RNDOM subject v, and so on he splt-splt plot desgn lternatve names Splt-splt plot, completely randomzed desgn Pretty awful Example () n agrcultural example (Wner et al., 99, pp ). n orchard s dvded nto plots. Each level of factor s assgned at random to n plots, so there are an plots n total. Each of the plots s then dvded nto b subplots, and the b levels of factor B are assgned to them at random. Fnally, each of the subplots s dvded nto c sub-subplots, and the c levels of factor C are assgned to them at random. hus the expermental unt for s the whole plot, the expermental unt for B s the subplot, and the expermental unt for C s the subsubplot. Snce the sub-subplots are nested wthn the subplots, and the subplots are nested wthn the whole plots, factor C s nested under the subplots and factor B s nested under the whole plots. Factor s partally confounded wth groups of whole plots. () rat example. Rats are mplanted wth dalyss probes n ether the medal prefrontal cortex ( ) or orbtofrontal cortex ( ). hey are then assgned to trplets. One rat n each trplet chooses between two levers offerng alternatve renforcers n a task (B ). nother (B ) s offered only the lever chosen by the master rat. thrd (B 3 ) s gven the renforcer chosen by the master rat, wthout any opportunty to press a lever. Fnally, all rats are dalysed at fve tme ponts (C C 5 ). Data from dfferent levels of factor (probe ste) are unrelated. Data from dfferent levels of factor B (choce type) may be related to each other, because they all come from the same trplet. Data from dfferent levels of factor C (tme) may be related to each other, because

166 7: Specfc desgns 66 they all come from the same rat. However, we cannot wholly dstngush rat ndvdualty from the effects of choce type. hs desgn s equvalent to the agrcultural one: rplet Plot, and Rat Subplot. s before, (leson) s the whole-plot factor (a trplet ether gets medal prefrontal or orbtofrontal probes), B (choce type) s the subplot factor (wthn a trplet, a rat s ether a master, lever-yoked or renforcer-yoked rat), and C (tme) s the sub-subplot factor (every rat gets dalysed at fve tme ponts, so the sub-subplot s the combnaton of a partcular rat at a partcular tme). Model Yjkm µ + ε + α + π m( ) + β j + αβj + π m( j) + γ k + αγ k + βγk + αβγ jk + π m( jk) jkm where Y jkm s the value of an observaton n condton, plot m, B j, and C k µ s the grand mean α s the contrbuton of β j s the contrbuton of B j γ k s the contrbuton of C k αβ j, αγ k, βγ jk and s the contrbuton of the B j, C k, and B j C k nteractons, respectvely m() π s the contrbuton of plot m (whch only ever experences ) π m(j ) s the contrbuton of the subplot n plot m that experences B j π s the contrbuton of the sub-subplot n plot m that experences B j C k m(jk ) ε jkm s everythng else (error) Sources of varance For our rat example, we d call trplet plot and rat subplot (and consder them as random factors, whle the others are fxed factors). We d wrte the model lke ths: SS total SS between-plots + SS wthn-plots SS between-plots SS + SS plot SS wthn-plots SS between-subplots-wthn-plots + SS wthn-subplots SS between-subplots-wthn-plots SS B + SS B + SS B plot/ SS wthn-subplots SS C + SS C + SS C B + SS C B + SS wthn-subplot error C plot/b NOV table Source d.f. SS F Between plots: an a SS MS /MS plot error plot a(n ) SS plot ( whole-plot resdual ) Wthn plots, between subplots:an(b ) B b SS B MS B /MS B plot/ B (b )(a ) SS B MS B /MS B plot/ error B plot/ a(b )(n ) SS B plot/ ( subplot resdual ) Wthn subplots: abn(c ) C c SS C MS C /MS C plot/b C (c )(a ) SS C MS C /MS C plot/b C B (c )(b ) SS C B MS C B /MS C plot/b C B (c )(a )(b ) SS C B MS C B /MS C plot/b error C plot/b ab(c )(n ) SS C plot/b ( sub-subplot resdual ) otal N abcn SS total where a s the number of levels of factor, etc., N s the total number of observatons ( abcn), and n s the number of subjects. he F ratos above assume that Plot s random and, B, C are fxed. For the rat example, smply read trplet nstead of plot and rat nstead of subplot.

167 7: Specfc desgns 67 Result! grees wth Wner (99, p. 369, although there are typos n hs NOV table; wthn sub-subplots s certanly a mstake). SPSS technque Plot B C depvar datum datum 3 datum 4 datum 5 datum datum datum 3 datum 4 datum 5 datum 3 datum 3 datum 3 3 datum 3 4 datum 3 5 datum datum datum 3 datum 4 datum 5 datum 8 datum 8 datum 8 3 datum 8 4 datum 8 5 datum It doesn t matter whether you specfy unque labels for nested factors or not what I mean by ths s that you can code plot from, for the condton and carry on countng (8, 9, ) for the condton, or you can start numberng plot from agan n the condton. Snce the desgn knows that plot s nested wthn (one plot only gets one level of ), t won t get confused. GLM depvar BY plot a b c /RNDOM plot /DESIGN a plot*a b b*a b*plot(a) c c*a c*b c*a*b. op tp: when fakng data to analyse complex models, ensure that you don t over- or underspecfy your model! MRF ponted out that I had been stupd n my ntal attempt at ths example, whch ncluded a rat (subplot) term: because a trplet B [chocetype] combnaton unquely specfes a rat n ths example, there s no room n the desgn for a rat term hree levels of relatedness lternatve names Splt-splt plot, randomzed complete block (RCB) desgn Horrendous Examples () he standard agrcultural example: a randomzed complete block desgn (RCBD) wth blocks (also known as replcates), plots (), subplots (B), and sub-sub-plots (C). Suppose has two levels, B has two levels, and C has three levels. hs would be a descrpton of a feld lad out lke ths:

168 7: Specfc desgns 68 Compare the RCB three-factor agrcultural desgn llustrated n our consderaton of the three-wthn-subject-factor desgn (U V W S) (p. 8). Smlarty or relatedness n agrculture often refers to geographcal nearness; n the (U V W S) desgn dscussed earler, adjacent mn-plots of land were lkely to be smlar by vrtue of comng from the same block, but there was no other consstent relatonshp between geographcal nearness and the factors U, V, or W. hs desgn s a bt dfferent. You can see here that two adjacent ndvdual expermental unts (the sub-sub-plots) are most lkely be related by vrtue of comng from the same Block, qute lkely to be related by vrtue of havng the same value of the factor, not qute as lkely to be related on the B factor, and least lkely to be related on the C factor. nother way of puttng t: blocks are crossed wth (all blocks experence all levels of ). Plots are nested wthn (one plot only gets one level of ). Plots are crossed wth B (all plots experence all levels of B). Subplots are nested wthn B (one subplot only gets one level of B). Sub-subplots are nested wthn C (one sub-subplot only experences one level of C). () nother agrcultural example (Prescott et al., 999). Four blocks were used, spread across a forest (top-level factor: Block); the experment was replcated across these blocks. Each block was dvded nto four plots, whch were each fertlzed wth a dfferent fertlzer, assgned to the plots at random. Small bags of leaf ltter are placed n these plots (ltter placement factor, or fertlzer that the ltter s placed n :,, 3, 4 ). he bags themselves came ether from the same plot or one of the other three plots n the same block (ltter source factor, or fertlzer that the ltter came from : B, B, B 3, B 4 ). he ltter mass s then measured at dfferent tme ponts (C C 5 ). Notes hs s dfferent to a splt-splt plot desgn based on a completely randomzed desgn (CRD), whch doesn t have the block factor. See the only worked example I ve been able to fnd. hat also says: he splt-splt plot arrangement s especally suted for three-or-more-factor experments where dfferent levels of precson are requred for the factors evaluated. hree plot szes correspond to the three factors: the largest plot for the man factor, the ntermedate sze plot for the subplot factor, and the smallest plot for the sub-subplot factor. here are three levels of precson wth the man plot factor recevng the lowest precson, and the sub-subplot factor recevng the hghest precson. Sources of varance Let s call blocks (replcates) R, the plot treatment, the subplot treatment B, and the subsubplot treatment C. Replcate wll be a random factor; the others wll be fxed. We d wrte the model lke ths: SS total SS between-replcates + SS wthn-replcates

169 7: Specfc desgns 69 SS between-replcates SS R SS wthn-replcates SS between-plots-wthn-replcates + SS wthn-plots SS between-plots-wthn-replcates SS + SS R SS wthn-plots SS between-subplots-wthn-plots + SS wthn-subplots SS between-subplots-wthn-plots SS B + SS B + SS R B/ SS wthn-subplots SS C + SS C + SS C B + SS C B + SS wthn-subplot-error C R/B NOV table Source df SS F Between replcates (R): R r SS R MS R /MS R Wthn replcates, between plots: a SS MS /MS R error R (r )(a ) SS R Wthn plots, between subplots: B b SS B MS B /MS R B/ B (b )(a ) SS B MS B /MS R B/ error R B/ a(r )(b ) SS R B/ Wthn subplots: C c SS C MS C /MS R C/B C (c )(a ) SS C MS C /MS R C/B C B (c )(b ) SS C B MS C B /MS R C/B C B (c )(a )(b ) SS C B MS C B /MS R C/B error R C/B ab(r )(c ) SS R C/B otal rabc SS total SPSS technque Rep B C depvar datum datum 3 datum datum datum 3 datum datum datum 3 datum datum datum 3 datum datum datum You don t even need explct plot, subplot, or sub-subplot labels; all that nformaton s contaned n the desgn and the /B/C factor labels. GLM depvar BY r a b c /RNDOM r /DESIGN r a r*a b b*a b*r(a) c c*a c*b c*a*b.

170 7: Specfc desgns Latn square desgns here are two approaches to Latn squares. One (the smplest) s to use a Latn square as an expermental desgn technque to ensure that some factor (e.g. tme, order) s not confounded wth expermental treatments. he other (more advanced but far preferable) s to do ths, but also to use nformaton about ths factor (e.g. tme, order) n the analyss to take account of varablty attrbutable to ths factor to reduce the error varablty and ncrease the power to detect effects of the treatment of nterest. hs can be much more complcated than I frst thought! For ths secton, I wll abandon my prevous conventon of, B representng between-subjects factors and U, V representng wthn-subjects factors, because ths makes t easer to compare complex desgns to the orgnal sources Latn squares n expermental desgn Here s an example of the novce (expermental desgn only) approach that I ve used (e.g. Cardnal et al., 3). Rats had ntracranal cannulae mplanted n ther nucleus accumbens. hey responded on a lever that delvered a stmulus prevously pared wth renforcement (a condtoned renforcer). Before the sesson, they were gven ntra-accumbens amphetamne at one of four doses (, 3,, µg per hemsphere). s I put t: Doses were counterbalanced n a Latn square desgn to elmnate dfferental carryover effects and separated by 4 h. he Latn square was of a dgrambalanced desgn (Keppel, 99, p. 339), n whch each condton mmedately precedes and follows the other condtons once (e.g. 34, 34, 43, 43). What I meant was that f represents one dose ( µg), represents the second, 3 the thrd, and 4 the fourth, the desgn looked lke ths: Day Day Day 3 Day 4 Pattern 3 4 Pattern 3 4 Pattern Pattern here were more than 4 subjects, so I allocated them to the four patterns at random. he dea s that the order of treatments 4 was counterbalanced approprately. he square s a Latn square an n by n grd contanng the numbers to n arranged n such a way that no row and no column contans the same number twce. If I had gven all the subjects the treatments n the order 4, 3,,, and I found that treatment 4 gave hgher respondng than treatment, I wouldn t know f that was due to the dfference n drug doses or the fact that wth tme, respondng declnes generally (extncton), or some other effect left over from the prevous day s dose. So the Latn square counterbalances for order. here are good and bad Latn squares. he one above s dgram-balanced, whch s good every condton mmedately precedes and follows the other condtons once. he one below s cyclc, whch sn t so good: Day Day Day 3 Day 4 Pattern 3 4 Pattern 3 4 Pattern Pattern because n ths desgn dose s nearly always preceded by dose 4, and nearly always followed by dose 4 clearly not as good as the dgram-balanced one. he dgrambalanced verson controls for sequence effects better. However, dgram balancng can only be done f there s an even number of treatment condtons (Keppel, 99,

171 7: Specfc desgns 7 p. 339). Otherwse, there are procedures for selectng a random Latn square (Wner et al., 99, p. 674; Myers & Well, 995, p. 346). nyway, back to the example. When I analysed these data, I gnored the day factor. I smply took all the dose scores, all the dose scores, and so on, and entered the data wth a wthn-subjects factor of Dose. hs wasn t optmal I could have used nformaton about the Day factor as well. hat could be more effcent (Myers & Well, 995, p. 35), because t would remove varablty attrbutable to Days to gve better power to detect effects of Dose. Let s see how that can be done he analyss of a basc Latn square Example We test fve monkeys (Myers & Well, 995, p. 344) on dscrmnaton learnng under fve dfferent drug doses on fve dfferent test days. We use ths Latn square (S subject R row, C column day n ths example, drug dose). C C C 3 C 4 C 5 S S S S S Notes See Myers & Well (995, chapter ); Wner (99, chapter ). he Latn square analyss s potentally more effcent than the smple wthn-subjects analyss (gnorng Day) for the followng reasons (Myers & Well, 995, p. 35). he error term for the wthn-subjects ( S) analyss, MS S, wll be larger than the error term for the Latn square analyss as long as MS C s larger than the Latn-square error term MS error. However, the Latn square error term has fewer df, whch reduces power. he relatve contrbuton of the two effects can be calculated (Myers & Well, 995, pp. 35-). When usng Latn squares to counterbalance for order, t s vtal that the poston n the order (Day, n the example) does not nteract wth the treatment (Drug, n the example) (Keppel, 99, p ; Wner et al., 99, p. 68). If one dose has a dfferent effect when t s gven frst n the order to when t s gven thrd n the order, we d have to be very careful of the nterpretaton. It s worth plottng treatment means aganst order to check ths assumpton. If the effect of dfferent doses reverses on dfferent days, t s very hard to analyse or nterpret (Keppel, 99, p. 338) and we may be reduced to analysng only the data from the frst test, whch s uncontamnated by any pror effects, but whch may have rather low statstcal power. We ve seen that one major use of Latn squares s to counterbalance order effects, as shown here. But they have other uses. Latn squares were frst used n agrculture to control for two nusance varables (assgned to the rows and columns, wth the assumpton that the treatment effects do not nteract wth the row and column effects) (Wner et al., 99, p. 68). hey may be extended to deal wth three nusance varables usng a Greco Latn (Graeco Latn) square, n whch two orthogonal Latn squares (Wner et al., 99, p. 674) are used; one s gven Greek letters, the other Roman (Latn) letters, and the two are supermposed (Wner et al., 99, pp. 68-). hs prncple can be extended to four or more nusance varables. It s also possble to use Latn squares to extract partal nformaton from confounded factoral desgns (Wner et al., 99, p. 68). Latn squares are a specal case of fractonal factoral desgns (Wner et al., 99, pp. 585, 683), n whch not all the treatment condtons of a factoral desgn are examned (see also GLM notes about fractonal factoral desgns, p. 88 ). Model n addtve model assumes that man effects are addtve, and don t nteract.e. that the and C do not nteract. he model s: Yjk µ + η + α + γ + ε j k jk where µ s the grand mean, η s the effect of row (n ths example, subject S ), α j s the

172 effect of treatment j, and γ k s the effect of column k (n ths example, day k). 7: Specfc desgns 7 Sources of varance NOV table For ths addtve model, SS total SS row + SS column + SS + SS error Snce the number of rows, columns, and treatments s the same, Source d.f. SS F Row (subject) a SS R MS R /MS error Column a SS C MS C /MS error a SS MS /MS error Error (a )(a ) SS error otal N a SS total SPSS technque Data layout: S C depvar datum datum 3 4 datum 4 3 datum 5 5 datum 3 datum datum 3 5 datum 4 datum 5 4 datum Syntax: UNINOV depvar BY s c a /RNDOM s /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN s c a. If C s a random factor, smply add t to the /RNDOM lst. In general, substtute Row for Subject for any sutable Latn square. Mssng values Nonaddtve model If we assume the addtve model, then t s possble to estmate mssng scores (Myers & Well, 995, p. 35) to allow analyss. Of course, our error df are reduced when we do that. If the addtvty assumpton (above) sn t realstc, you can use a nonaddtve model. he full model adds n the S C, S, C, and S C terms. However, t s what we mght call very complex ndeed (Myers & Well, 995, pp ); I certanly don t understand t.

173 7: Specfc desgns B nteractons n a sngle Latn square Example We assgn not only an treatment but also a B treatment to each cell of the Latn square. hs can be analysed provded that all possble B combnatons appear exactly once n each row and column. For example (Myers & Well, 995, pp ): C C C 3 C 4 S B B B B S B B B B S 3 B B B B S 4 B B B B Notes Model Yjkm µ + η + α + β + αβ + γ + ε j k jk m jkm where µ s the grand mean, η s the effect of row (n ths example, subject S ), α j s the effect of treatment j, the effect of column m. β k s the effect of treatment B k, αβ jk s the B nteracton, and γ m s Sources of varance NOV table SS total SS row + SS column + SS + SS B + SS B + SS error Snce the number of rows, columns, and B condtons s the same, Source d.f. SS F Row (subject) ab SS R MS R /MS error Column ab SS C MS C /MS error a SS MS /MS error B b SS B MS B /MS error B (a )(b ) SS B MS B /MS error Error (a )(a ) SS error otal N (ab) SS total SPSS technque Data layout: S C B depvar datum datum 3 datum 4 datum datum datum 3 datum 4 datum Syntax: UNINOV depvar BY s c a b /RNDOM s /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN s c a b a*b. If C s a random factor, smply add t to the /RNDOM lst.

174 7: Specfc desgns More subjects than rows: (a) usng several squares Example In the frst example above, we had fve treatments and were therefore lmted to fve rows (subjects). If we want to run more subjects, whch wll ncrease power, one way s to use several dfferent squares. hs approach has an advantage: f there are nteractons wth order, usng several dfferent squares ncreases the chance that postve and negatve nteracton effects wll cancel each other. Suppose (Myers & Well, 995, p. 357) we have subjects beng tested on four tasks ( 4 ) requrng dfferent types of motor skll. Each task s performed on a dfferent day (C). hree 4 4 Latn squares are constructed (see Myers & Well, 995, pp ), and subjects are assgned at random to the rows. he desgn looks lke ths: Square C C C 3 C 4 S 3 4 Q S 3 4 S S C C C 3 C 4 S Q S S S C C C 3 C 4 S Q 3 S 3 4 S 4 3 S 3 4 Notes Model Subjects (S) are nested wthn squares (Q). We assume that S and Q are random factors, whle and C are fxed. Ether ths model: Yjkm + η / m µ + α + γ + π + απ + γπ + ε j k where µ s the grand mean, η / m s the effect of subject (wthn square m), α j s the effect of j, γ k s the effect of column k, απ jm allows for the possblty that treatment effects depend on the square (Q nteracton), and γπ km allows for the possblty that column effects depend on the square (CQ nteracton) m jm km jkm or, f the full model produces no evdence for Q or CQ nteractons, ths reduced model, whch pools the Q and CQ terms nto the error term to ncrease power: Y µ + η + α + γ + π + ε jkm / m j k m jkm Sources of varance Ether ths (for the frst model): or ths (for the reduced model): SS total SS S/Q + SS + SS C + SS Q + SS Q + SS CQ + SS error SS total SS S/Q + SS + SS C + SS Q + SS error NOV table For the full model: Source d.f. SS F Squares (Q) q SS Q MS Q /MS S/Q S/Q q(a ) SS S/Q MS S/Q /MS error C a SS C MS C /MS error a SS MS /MS error

175 7: Specfc desgns 75 C Q (a )(q ) SS CQ MS CQ /MS error Q (a )(q ) SS Q MS Q /MS error Error q(a )(a ) SS error otal N qa SS total For the reduced model: Source d.f. SS F Squares (Q) q SS Q MS Q /MS S/Q S/Q q(a ) SS S/Q MS S/Q /MS error C a SS C MS C /MS error a SS MS /MS error Error (qa )(a ) SS error otal N qa SS total SPSS technque Data layout: Q S C depvar datum 3 datum 3 4 datum 4 datum 3 datum 4 datum 3 datum 4 datum 5 datum 5 datum datum datum Syntax for the full model: UNINOV depvar BY q c a s /RNDOM s q /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN q s(q) c a c*q a*q. For the reduced model: UNINOV depvar BY q c a s /RNDOM s q /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN q s(q) c a.

176 7: Specfc desgns More subjects than rows: (b) usng the same square several tmes (replcatng a sngle Latn square) Example s above, but now you decde to use a sngle 4 4 square and assgn n subjects to each row of the square. If n 3, your desgn mght look lke ths: Subjects C C C 3 C 4 S, S, S S 4, S 5, S S 7, S 8, S S, S, S 3 4 Notes See Myers & Well (995, pp , ), who pont out that ths desgn s frequently used but frequently analysed mproperly. Should you use replcated squares, or several squares (as on p. 74)? Myers & Well (995, p. 37) suggest that several squares s better expermenters tend to replcate squares purely for smplcty. nyway, let s see how you analyse replcated squares now. Model hs s the smple way: Yjkm + π m + η / m µ + α + γ + ε j k jkm where µ s the grand mean, π m s the effect of row m, η / m s the effect of subject (wthn row m), α j s the effect of j, and γ k s the effect of column k. Sources of varance hat would gve these sources of varance: SS total SS between-subjects + SS wthn-subjects SS between-subjects SS row + SS subjects-wthn-row(s/r) SS wthn-subjects SS + SS C + SS S/R Complcated bt However, there are some extra fnesses: we can partton the data another way. here are a cell means n the Latn square. If you account for man effects of, C, and R, you re left wth what s called the between-cells error or resdual. It has (a ) (a ) (a ) (a ) (a )(a ) df. hen you have the wthn-cells resdual, whch s equvalent to S (nested wthn R), or S C (nested wthn R), whch are the same thng (snce wthn one row, a subject s level of completely determnes ts level of C). hs has a(n )(a ) df. Now Varaton among row means (SS R ) reflects dfferent effects of C combnatons. In other words, f there s an C nteracton, part of ts effect wll be reflected n MS R. Part of any C nteracton effect wll also be reflected n what s left n the cell mean varablty after you ve accounted for man effects of, C, and R the between-cells error (MS bce ). So any C nteracton would contrbute to MS bce. So both MS R and MS bce partally reflect effects of C. hs pcture would gve ths model: Yjkm + η / m µ + α + γ + αγ + ε j k jk jkm and ths parttonng: SS total SS between-subjects + SS wthn-subjects SS between-subjects SS subjects-wthn-row(s/r) + some-part-of-ss C SS wthn-subjects SS + SS C + some-part-of-ss C + SS S/R NOV table hs s for the smple way of dong thngs:

177 7: Specfc desgns 77 Source d.f. SS F R a SS R MS Q /MS S/R S/R n(a ) SS S/R MS S/R /MS error C a SS C MS C /MS error a SS MS /MS error Error (a )(an ) SS error otal N na SS total hs s for the complex way: Source d.f. SS F R (C ) a SS R MS Q /MS S/R S/R n(a ) SS S/R MS S/R /MS wce C a SS C MS C /MS wce a SS MS /MS wce Between-cells error (a )(a ) SS bce MS bce /MS wce (C ) Wthn-cells error a(n )(a ) SS wce S /R S C/R otal N na SS total he rows labelled C gve estmates for the effect of the C nteracton, based on partal nformaton. he between-cells error SS bce s calculated as SS RC SS (that s, calculate the row column nteracton and subtract SS ). SPSS technque Data layout: R S C depvar datum 3 datum 3 4 datum 4 datum datum 3 datum 3 4 datum 4 datum 5 datum 5 datum datum datum smple but less powerful way Run ths: UNINOV depvar BY r c a s /RNDOM s /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN r s(r) c a. hat s t. complex and more powerful way hs s pretty complcated. Frst, run ths to get the R C nteracton sum of squares (all the sums of squares are correct, but ths s the only one you need). UNINOV depvar BY r c s /RNDOM s /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5)

178 7: Specfc desgns 78 /DESIGN r s(r) c r*c. hen, run ths to get everythng else. (hs gves you correct answers for all sums of squares, dfs, and MSs. But you can mprove on the F ratos by usng a dfferent error term ) UNINOV depvar BY r c a s /RNDOM s /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN r s(r) c a. Next, calculate SS between-cells-error SS R C SS SS wthn-cells-error SS error-from-second-nov-n-whch--was-ncluded SS between-cells-error Calculate the correspondng MS by hand. he df for these error terms (whch you need to work out the MS) are n the NOV table above. Fnally, test MS C and MS aganst MS wthn-cells-error by hand. If you want, you can also test MS R (aganst MS S/R ) and MS between-cells-error (aganst MS wthn-cellserror) as estmates of the effect of the C nteracton, based on partal nformaton. Complex caveat If C s a random, rather than a fxed factor (Myers & Well, 995, pp ), thngs become more complcated, snce C should be tested aganst MS wce but must be tested aganst MS bce, but ths has poor power; Myers & Well recommend that f the effect of MS bce sn t sgnfcant tself that you use MS wce or the pooled MS error to test and C.

179 7: Specfc desgns Between-subjects desgns usng Latn squares (fractonal factoral desgns) Example Suppose (Wner et al., 99, p. 687; Myers & Well, 995, p. 37) we want to compare the effects of three teachng methods ( 3 ). o ncrease the power, we decde to block subjects on the bass of prevous experence wth the subject (R) and on the bass of ablty as measured by a pretest (C). For ths full-factoral desgn, we would need cells wth n subjects n each. Instead, we reduce the labour by usng a Latn-square desgn wth only 9 cells: R would be the rows, and C the columns. he desgn mght look lke ths, wth n subjects per cell: C C C 3 R 3 R 3 R 3 3 Notes hs s very smlar to the usual agrcultural use of Latn squares. See also Wner (99, p ). Model If t assumed that there are no nteractons between R, C, and : Yjkm µ + α + β + γ + ε j k m jkm where µ s the grand mean, and γ m s the effect of column m. α j s the effect of treatment j, β k s the effect of treatment R k, Sources of varance SS total SS + SS R + SS C + SS between-cell-error + SS wthn-cell-resdual where SS between-cell-error ncludes all sources of varaton due to treatment effects whch are not predctable from the sum of man effects (e.g. nteractons whch you hope aren t there; see below). NOV table Source d.f. SS F R a SS R MS R /MS wce C a SS C MS C /MS wce a SS MS /MS wce Between-cells error (a )(a ) SS bce MS bce /MS wce Wthn-cells error a (n ) SS wce S/BC otal N na SS total he between-cells error SS bce s calculated as SS RC SS (that s, calculate the row column nteracton and subtract SS ). Caveat hs model s approprate f addtvty can be assumed (f there are no nteractons between R, C, and ). nd f so, SS between-cell-error wll not be substantally larger than SS wce. One way to test ths (Wner et al., 99, p ) s to look at the F test on MS bce. If t s sgnfcant, then the assumptons behnd the model are not approprate, and f ths s not an approprate model f there are nteracton effects then t s very hard to analyse the data (Wner et al., 99, p. 69; Myers & Well, 995, p. 373). SPSS technque Data layout (wth an unnecessary Subject column to make thngs clearer): R C S depvar datum datum 3 datum 4 datum 3 5 datum 3 6 datum 3 7 datum

180 7: Specfc desgns datum 3 3 datum 3 4 datum 3 5 datum Run ths: UNINOV depvar BY r c a /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN r c a. hat gves you SS R, SS C, SS, and SS total. But to get SS bce and SS wce, you have to run ths to obtan SS RC : UNINOV depvar BY r c /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN r c r*c. hen calculate SS bce SS RC SS and SS wce SS error-from-frst-nov-ncludng--factor SS bce by hand and complete the NOV table.

181 7: Specfc desgns Several-squares desgn wth a between-subjects factor Example We saw how to use a desgn wth several Latn squares above (p. 74). We had a wthnsubjects factor. Let s add a between-subjects factor B wth b levels. We have q squares per level of B, and a subjects n each squares wth a scores for each subject. If a 4, b and q, we mght have ths: Square C C C 3 C 4 S 4 3 B Q S 3 4 S S C C C 3 C 4 S B Q S S S C C C 3 C 4 S B Q 3 S 3 4 S 4 3 S 3 4 C C C 3 C 4 S B Q 4 S S S hs example based on Myers & Well (995, p. 36), though ther orgnal has several numercal errors n ther fourth square, whch sn t even Latn (some values appear twce n a column). Notes Model Yjkm + βk + π p / k + η / p / k + α j + γ m + αβ jk + βγ km + απ jp / k + γπ mp / k µ + ε jkmp where ndex subjects (wthn squares wthn levels of B), j ndexes the level of, k ndexes the level of B, m ndexes the level of C, and p ndexes the square (wthn a level of B). Subject and Square are assumed to be random;, B, and C are assumed to be fxed effects. Sources of varance I thnk t s ths (based on Myers & Well, 995, p. 36): SS total SS between-squares + SS wthn-squares SS between-squares SS B + SS Q/B SS wthn-squares SS between-subjects-wthn-squares + SS wthn-subjects SS between-subjects-wthn-squares SS S/Q/B SS wthn-subjects SS + SS C + SS B + SS BC + SS Q/B + SS CQ/B + SS wthn-subject-error We could also note that SS between-subjects SS B + SS Q/B + SS S/Q/B (Myers & Well, 995, p. 363). But f p >.5 for the nteracton terms Q/B and CQ/B, t would be reasonable to pool those error terms: SS wthn-subjects SS + SS C + SS B + SS BC + SS pooled-wthn-subject-error NOV table For the full model (note that Myers & Well, 995 cock the df rght up):

182 7: Specfc desgns 8 Source d.f. SS F B b SS B MS B /MS Q/B Q/B b(q ) SS Q/B MS Q/B /MS S/Q/B S/Q/B bq(a ) SS S/Q/B MS S/Q/B /MS error a SS MS /MS Q/B C a SS C MS C /MS CQ/B B (a )(b ) SS B MS B /MS Q/B BC (b )(a ) SS BC MS BC /MS CQ/B Q/B b(a )(q ) SS Q/B MS Q/B /MS error CQ/B b(a )(q ) SS CQ/B MS CQ/B /MS error Error bq(a )(a ) SS error otal bqa SS total For the pooled error model: Source d.f. SS F B b SS B MS B /MS Q/B Q/B b(q ) SS Q/B MS Q/B /MS S/Q/B S/Q/B bq(a ) SS S/Q/B MS S/Q/B /MS error a SS MS /MS error C a SS C MS C /MS error B (a )(b ) SS B MS B /MS error BC (b )(a ) SS BC MS BC /MS error Error (pooled) b(a )(aq ) SS error otal bqa SS total SPSS technque Snce Myers & Well s (995, pp. 36-3) numercal example s wrong, I have no way of verfyng ths aganst some gold standard. Data format: B Q S C depvar 4 datum datum 3 3 datum 4 datum 3 datum datum 3 4 datum 4 datum 5 4 datum 5 3 datum 5 3 datum 5 4 datum 3 9 datum datum datum datum Full model syntax: UNINOV depvar BY b q s c a /RNDOM q s /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN b q(b) s(q(b)) a c a*b b*c a*q(b) c*q(b). Pooled error model syntax, I presume, s ths:

183 UNINOV depvar BY b q s c a /RNDOM q s /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN b q(b) s(q(b)) a c a*b b*c. 7: Specfc desgns 83

184 7: Specfc desgns Replcated-squares desgn wth a between-subjects factor Example We saw how to use a desgn wth a replcated Latn square above (p. 76). We had a wthnsubjects factor. Let s add a between-subjects factor B wth b levels. We have one Latn square wth a rows, wth n subjects for each row and therefore bn subjects per level of B. If a 4, b and n, we mght have ths: B B Subjects Subjects C C C 3 C 4 S, S S 9, S 4 3 S 3, S 4 S, S S 5, S 6 S 3, S S 7, S 8 S 5, S Notes Model See Myers & Well (995, pp , ), who pont out that ths desgn s frequently used but frequently analysed mproperly. Yjkm + η / kp µ + α + β + γ + αβ + αγ + βγ + αβγ j k m jk jm km jkm ε jkmp where ndexes the subject (wthn a row B combnaton; n), j ndexes the level of (j a), m ndexes the level of C (m a), and p ndexes the row wthn the square (p a). Subject s assumed to be a random factor; the rest are fxed. Sources of varance SS total SS between-subjects + SS wthn-subjects SS between-subjects SS B + SS R + SS BR + SS S/BR SS wthn-subjects SS C + SS + SS B + SS BC + SS between-cell-resdual + SS B between-cell-resdual + SS wthn-cell-resdual where SS between-cell-resdual SS CR SS and SS B between-cell-resdual SS BCR SS B. NOV table Source d.f. SS F B b SS B MS B /MS S/BR R (C ) a SS R MS R /MS S/BR BR (BC ) (b )(a ) SS BR MS BR /MS S/BR S/BR ab(n ) SS S/BR MS S/BR /MS wce C a SS C MS C /MS wce a SS MS /MS wce B (a )(b ) SS B MS B /MS wce BC (a )(b ) SS BC MS BC /MS wce Between-cells error (a )(a ) SS bce MS bce /MS wce (C ) B betw.-cells error (a )(a )(b ) SS B bce MS B bce /MS wce (BC ) Wthn-cells error ab(a )(n ) SS wce S /BR S C/BR otal N bna SS total s before, some terms gve estmates of nteractons based on partal nformaton; they re labelled wth a prme ( ) symbol above. gan, there s a df error n Myers & Well (995, p. 37). SPSS technque Data layout: B R S C depvar 4 datum datum 3 datum 4 3 datum

185 7: Specfc desgns 85 4 datum datum 3 datum 4 3 datum 3 3 datum 3 datum datum 3 4 datum 9 4 datum 9 datum 9 3 datum datum SPSS syntax: UNINOV depvar BY b r c a s /RNDOM s /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN b r b*r s(b*r) c a a*b b*c. hat ll gve you all the SS except SS bce, SS B bce, and SS wce. o get those, obtan SS CR and SS BCR from ths syntax: UNINOV depvar BY b r c s /RNDOM s /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN b r b*r s(b*r) c c*b c*r b*c*r. and calculate and Fnally, calculate SS bce SS CR SS SS B bce SS BCR SS B SS wce SS error-from-frst-nov-ncludng- SS bce SS B bce and use t to test the relevant terms by hand.

186 7: Specfc desgns grcultural termnology and desgns, compared to psychology In psychology, the most mportant factor n expermental desgn s often the subject, because ths accounts for much correlaton between observatons. If you have two groups of subjects and gve the two groups the two treatments, you account for much (you hope all) of the expected correlaton between any two subjects by specfyng the treatment factor n your analyss. (Of course, that may not be the case f one group were all men and the other all women, you d have confounded sex and treatment; another way of sayng that s that correlatons between ndvdual subjects scores may be due to them beng members of the same sex rather than havng experenced a partcular treatment.) On the other hand, f you measure subjects more than once, you can expect hgh correlatons between observatons from the same subject much more so than between observatons from dfferent subjects. So you need to account for ntra-subject correlaton, whch you do by specfyng a Subject factor (by performng a wthn-subjects analyss). Much psychologcal research bols down to askng s ths a between-subjects or a wthn-subjects factor? However, many NOV technques orgnated n agrcultural research, so t often happens that when you want an example of an advanced desgn, the only ones you fnd are agrcultural. nd n agrculture, sources of correlaton don t come from subjects, but from thngs lke geographcal proxmty. If you want to see whether fertlzer works better than fertlzer B, you d want to gve fertlzer to a set of plants (obvously not just one) and fertlzer B to another set of plants. But t would be pretty daft to spray fertlzer on the sunny south-facng sde of your feld and to fertlzer B under the shady oak tree. grcultural desgns and analyses revolve around these sorts of deas. hs overvew of agrcultural temnology s prncpally from angren (). Completely randomzed desgn (CRD) Your smallest expermental unt (sometmes called the subject or replcaton ) s a small plot of land wth a plant or plants n t. Each expermental unt produces a sngle value of the dependent varable. You have four fertlzers ( D; factor for treatment; t 4). You gve each to four expermental unts ( subjects ) (n 4 per group) at random. djacent subjects could potentally have the same treatment. Here s one possble layout, where D are treatments and 4 are subjects wthn each treatment (a sngle subject s underlned): B C D 3 D C B D3 C3 B3 C4 4 B4 D4 he approprate NOV s equvalent to a desgn wth one between-subjects factor (p. 6). If t s the number of treatments and r s the number of replcatons per treatment: Source df SS F t SS MS /MS error error t(r ) SS error otal tr SS total CRD wth subsamplng he same as a CRD, except that you take three samples per plant (or small plot of plants, or whatever your prevous basc unt was; plant replcaton). reatments are assgned at random to the plants. For example, f the treatments are D, the plants (replcatons) are 4 and the subsamples are a c, we could get ths: a b c Ba Bb Bc C3a C3b C3c B4a B4b B4c Ba Bb Bc a b c Ca Cb Cc 4a 4b 4c Ca Cb Cc B3a B3b B3c 3a 3b 3c C4a C4b C4c sngle plant/plot/whatever s underlned. he dea s that you get a better dea of your measurement error (wthn-plant varablty), so you can remove ths to get a better est-

187 mate of your between-plant varablty. he NOV looks lke ths: 7: Specfc desgns 87 SS total SS + SS between-plant-varablty + SS wthn-plant-varablty Source df SS F t SS MS /MS E expermental error E t(r ) SS E MS E /MS S replcaton/ samplng error S tr(s ) SS S error otal trs SS total where r s the number of replcatons per treatment and s s the number of subsamples per replcaton. For example, see No routne psychology equvalent? Except that t s a way to analyse stuatons n whch you have one between-subjects factor and you have multple observatons per subject. o run ths analyss n SPSS, the data can be lad out lke ths: Rep depvar subsample datum subsample datum subsample_3_datum subsample datum subsample datum subsample_3_datum 5 subsample datum 5 subsample datum 5 subsample_3_datum and analysed usng ths syntax: UNINOV depvar BY trt rep /RNDOM rep /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN trt rep(trt). o acheve ths usng the SPSS menus, you have to enter a custom model (because you don t want the Replcaton factor n there as a man effect; you just want replcaton/). You mght thnk ths was a good way to analyse desgns n whch you measure a subject (replcaton) several tmes. nd ndeed, ths s a vald way to analyse such data. Except ths desgn gves dentcal answers to takng a mean for every subject (replcaton) and analysng those means by one-way NOV usng as the only factor! See p. 48. Randomzed complete block (RCB) desgn he standard desgn for agrcultural experments (angren, ). he orchard s dvded nto unts called blocks to account for any varaton across the feld (sunny bt, shady bt, etc.). reatments are then assgned at random to the plants n the blocks, one treatment per plant (or small plot of plants). Each block experences each treatment. If the blocks are I IV and the treatments are D, we mght have ths: Block I B C D Block II D B C Block III B D C Block IV C B D

188 Source df SS F Block B b SS B MS B /MS E reatment t SS MS /MS E error E (t )(b ) SS E otal tb SS total 7: Specfc desgns 88 Equvalent to a desgn wth one wthn-subjects factor (p. ) (Block Subject; reatment WS factor). RCB wth subsamplng he layout s the same as an RCB, but each plant (or plot) s sampled several tmes. For example (a sngle plant subsampled basc unt s underlned): a b c Ba Bb Bc Ca Cb Cc Ba Bb Bc Ba Bb Bc a b c Ba Bb Bc a b c Ca Cb Cc Ca Cb Cc a b c Ca Cb Cc Block I Block II Block III Block IV Source df SS F Block B b SS B MS B /MS E reatment t SS MS /MS E expermental error E (t )(b ) SS E MS E /MS S sample error S tb(s ) SS S otal tb SS total where b s the number of blocks, t s the number of treatments and s s the number of subsamples per plot. For example, see No routne psychology equvalent? Except that t s a way to analyse stuatons n whch you have one wthn-subjects factor (p. ) (Block Subject; reatment WS factor) and you have multple observatons per level of the wthn-subjects factor per subject. o run ths analyss n SPSS, the data can be lad out lke ths: Block depvar subsample datum subsample datum subsample_3_datum subsample datum subsample datum subsample_3_datum 5 subsample datum 5 subsample datum 5 subsample_3_datum and analysed usng ths syntax: UNINOV depvar BY t block /RNDOM block /MEHOD SSYPE(3) /INERCEP INCLUDE /CRIERI LPH(.5) /DESIGN t block t*block. o acheve ths usng the SPSS menus, choose nalyze General Lnear Model Unvarate. Enter as a fxed factor and Block as a random factor. You mght thnk ths was a good way to analyse desgns n whch you measure a subject (replcaton) several tmes at each level of a wthn-subjects factor. nd ndeed, ths s a

189 7: Specfc desgns 89 vald way to analyse such data. Except ths desgn gves dentcal answers to takng a mean for every subject/factor combnaton and analysng those means usng a straghtforward wthn-subjects desgn wth as the only factor! Compare p. 48. Latn square Used to control for varaton n two dfferent drectons, the row drecton and the column drecton. Each treatment appears once per row and once per column. here are the same number of rows as columns as treatments (call that number r). For example: Column 3 4 Row I B C D Row II C D B Row III D C B Row IV B D C Source df SS F Row R r SS R MS R /MS E Column C r SS C MS C /MS E reatment r SS MS /MS E expermental error E (r )(r ) SS E otal r SS total Drectly equvalent to Latn square desgns used n psychology (p. 7 ). CRD factoral wo treatments are combned for example, fertlzer (of type or B) and pestcde (of type a or b) are combned to gve treatment combnatons a, b, Ba, Bb. Each combnaton s then randomly assgned to replcatons, wth r replcatons per treatment combnaton. For example, wth a desgn and 4 replcatons (plants, plots, whatvever) per treatment, you mght have the followng layout (a sngle plant/plot s underlned): a Ba b a Bb a3 Bb b Ba Bb3 b3 Ba3 b4 a4 Ba4 Bb4 Equvalent to a desgn wth two between-subjects factors (p. 8). So the table s obvous: Source df SS F frst factor F f SS F MS F /MS E second factor S s SS S MS S /MS E F S (f )(s ) SS FS MS FS /MS E error E fs(r ) SS E otal fsr SS total RCB factoral Orchard s dvded nto blocks. Every block gets all possble combnatons of the two factors, as above (assgned at random wthn each block). For example: Block IV Block III Block II Block I a Ba b Bb Bb a Ba b Ba Bb b a b a Ba Bb Equvalent to a desgn wth two wthn-subjects factors (p. 5) (Block Subject; reatment and reatment B are WS factors). Source df SS F Block B b SS B MS B /MS E frst factor F f SS F MS F /MS E second factor S s SS S MS S /MS E F S (f )(s ) SS FS MS FS /MS E error E (fs )(b ) SS E otal fsb SS total

190 7: Specfc desgns 9 RCB 3-way factoral Smply an extenson of an RCB -way factoral (see above) to 3 factors. herefore equvalent to a desgn wth three wthn-subjects factors (p. 8) (Block Subject). If our factor levels are C (frst factor), (second factor), a b (thrd factor), we mght have ths: Block I Block II Block III Ca Ba Ca Ca Ba Ba Ba a Cb Ba Bb a b a Bb a b a Bb Bb b a Cb Cb b Ca Bb b Ca b Cb Cb Cb Ba Bb Ca Source df SS F Block B b SS B MS B /MS E frst factor x SS MS /MS E second factor Y y SS Y MS Y /MS E thrd factor Z z SS Z MS Z /MS E Y (x )(y ) SS Y MS Y /MS E Z (x )(z ) SS Z MS Z /MS E Y Z (y )(z ) SS YZ MS YZ /MS E Y Z (x )(y )(z ) SS YZ MS YZ /MS E error E (xyz )(b ) SS E otal xyzb SS total Here s a pcture (partly for comparson to a splt splt plot, see below): Splt plot on a CRD he man expermental unts of a CRD (termed man plots) are dvded further nto subplots to whch another set of treatments are assgned at random. For example, suppose we have pestcdes C (man treatment), four plots (replcatons) per treatment ( plots n total), each dvded nto three subplots, and three fertlzers a c (subplot treatment). We could have ths: a b c Bc Bb Ba b c a Ca Cc Cb Cc Ca Cb 3b 3c 3a Bc Ba Bb C3b C3a C3c B3b B3c B3a 4a 4c 4b C4c C4a C4b B4a B4b B4c One plot (a plot s underlned) only experences one man treatment, but experences all three subplot treatments. Source df SS F plot treatment t SS MS /MS Em error, man plots (Em) t(r ) SS Em

191 7: Specfc desgns 9 subplot treatment S s SS S MS S /MS Es S (t )(s ) SS S MS S /MS Es error, subplots (Es) t(r )(s ) SS Es otal trs SS total Equvalent to a desgn wth one between-subjects factor and one wthn-subjects factor (p. ) (plot treatment BS factor, subplot treatment WS factor; plot Subject). Splt plot on an RCB he orchard s dvded nto unts called blocks to account for any varaton across the feld (sunny bt, shady bt, etc.). he blocks are then dvded nto plots. reatments (e.g. pestcdes) are then assgned at random to the plots n the blocks, one treatment per plot. Each block experences each treatment. he plots are then dvded nto subplots and a further set of treatments (e.g. fertlzer) are appled to the subplots, assgned at random. If the blocks are I IV, the man plot treatments are C, and the subplot treatments are a c, we mght have ths: Block-I Block-II Block-III Block-IV a b c Bc Bb Ba b c a Ca Cc Cb Cc Ca Cb b c a Bc Ba Bb a c b Bb Bc Ba Cb Ca Cc Cc Ca Cb Ba Bb Bc man plot s underlned. he number of blocks s the number of replcatons. Source df SS F block B b SS B MS B /MS Em plot treatment t SS MS /MS Em error, man plots (Em) (t )(b ) SS Em subplot treatment S s SS S MS S /MS Es S (t )(s ) SS S MS S /MS Es error, subplots (Es) t(b )(s ) SS Es otal tbs SS total hs s a herarchcal desgn (p. 59 ). he relatedness factors are Block (plots are related f they come from the same block) and Plot (subplots are related f they come from the same plot). Splt-splt plot on an RCB he orchard s dvded nto blocks. he blocks are then dvded nto plots. reatments (, e.g. pestcdes) are then assgned at random to the plots n the blocks, one treatment per plot. Each block experences each treatment. he plots are then dvded nto subplots (or splt plots) and a further set of treatments (S, e.g. fertlzer) are assgned at random to the subplots. he subplots are then further subdvded nto splt-subplots (or sub-subplots, or splt-splt plots) and a thrd set of treatments (U, e.g. prunng technque) are assgned at random to the splt-subplots. If the blocks are I III, the man plot treatments are B, the subplot treatments are, and the splt-subplot treatments are a c, we mght have ths: Block I reatment + reatment B a b c c b a + b c a a c b Block II reatment B + reatment c a b b c a + c a b b a c Block III reatment + reatment B b c a a c b + c a b a b c Here s a pcture:

192 7: Specfc desgns 9 Compare ths to an RCB 3-way factoral (see above). Source df SS F Between blocks: block B b SS B MS B /MS Em Wthn blocks, between plots: plot treatment t SS MS /MS Em error, man plots (Em) (t )(b ) SS Em Wthn plots, between subplots: subplot treatment S s SS S MS S /MS Es S (s )(t ) SS S MS S /MS Es error, subplots (Es) t(b )(s ) SS Es Wthn subplots: splt-subplot treatment U u SS U MS U /MS Eu U (u )(t ) SS U MS U /MS Eu U S (u )(s ) SS S MS S /MS Eu U S (u )(s )(t ) SS US MS US /MS Eu error, splt-subplots (Eu) ts(b )(u ) SS Es otal tbs SS total hs s a herarchcal desgn wth three levels of relatedness (p. 59 ). hey are Block (plots are related f they come from the same block), Plot (subplots are related f they come from the same plot), and Subplot (splt-subplots are related f they come from the same subplot). hs s one herarchcal level more than the basc splt-splt plot desgn (based on a CRD rather than an RCB), dscussed above. Splt block wo sets of treatments are randomzed across each other n strps n an otherwse RCB desgn. So the orchard s dvded nto blocks, and the blocks are dvded n an North South drecton and an East West drecton. One treatment (pestcde, C) s assgned randomly to the blocks n the North South drecton, so each block experences all treatments. he other treatment (fertlzer, ) s assgned randomly to the blocks n an East West drecton; agan, each block experences all treatments. It mght look lke ths: Block I Block II Block III C C B B B B C C C C B B Source df SS F block B b SS B MS B /MS B treatment t SS MS /MS B

193 7: Specfc desgns 93 B (t )(b ) SS B cross-treatment C c SS C MS C /MS C B C B (c )(b ) SS C B C (c )(t ) SS C MS C /MS E error, E (t )(c )(b ) SS E otal tcb SS total Equvalent to a desgn wth two wthn-subjects factors (Block Subject). Compare the full model for two wthn-subjects factors dscussed earler (p. 5). Pseudoreplcaton he researcher apples one treatment to all the trees n a row, the next treatment to all the trees n the next row, etc. here are (say) 4 trees per row. Row I Row II Row III Row IV B B B B C C C C D D D D he researcher hoped that the experment was beng replcated by havng four trees per row, but the researcher has cocked up. Row s confounded wth treatment, so we can t analyse ths. (he splt-block desgn s one way of applyng treatments to whole rows, properly.) here are frequent psychology equvalents, but that s not a good thng. Regresson appled to a CRD he orchard s dvded nto plots. Each plot has a certan amount of fertlzer appled treated as a contnuous varable. reatments are assgned to plots at random. If the expermenter uses,,, and 5 kg of fertlzer, and has four plots per fertlzer condton (replcates a d), the orchard mght look lke ths: 5a a a 5b a 5c b b b c c c d 5d d d he researcher expects a lnear relatonshp between fertlzer amount and the dependent varable. Source df SS F regresson R SS R MS R /MS E error E tr SS E otal tr SS total My comment: of course, there s no absolute requrement to have four plots wth kg, four plots wth kg, and so on; you could have one plot wth kg, one wth.5 kg, one wth kg Equvalent to smple (.e. between subjects) lnear regresson (see p. 35). Regresson, comparng trends from dfferent treatments (appled to a CRD) he orchard s dvded nto plots. One treatment factor (pestcde or B) s crossed wth a contnuously-measured treatment (fertlzer:,, 5 kg). here are four plots (replcatons a d) per pestcde/fertlzer combnaton. So we mght have ths: 5a a 5Ba 5b Ba Ba Bb a Bb b Bc b 5c Bc c c 5Bb 5Bc d 5Bd 5d Bd Bd d Source df SS F treatment t SS MS /MS E regresson R SS R MS R /MS E R t SS R MS R /MS E error E t(qr ) SS E

194 7: Specfc desgns 94 otal tqr SS total where q s the number of levels of the contnuously-measured thng that you re usng as a lnear predctor (fertlzer, n ths example). he R nteracton measures whether the regresson slope dffers across treatments. Equvalent to NCOV wth one between-subjects covarate and one between-subjects factor, n whch the covarate and factor nteracton s ncluded (p. 44). NCOV (appled to a CRD) he orchard s dvded nto plots, and treatments D are appled to the plots at random (ths s a CRD). hen an ndependent factor (e.g. sol ntrogen) wll be measured for each plot. Suppose there are four replcatons ( 4; four plots for each level of the treatment). We mght then have ths layout: B C D 3 D C B D3 C3 B3 C4 4 B4 D4 We ll also measure the covarate (ntrogen) n each plot. hs s the NOV table: Source df SS F covarate C SS C MS C /MS E adjusted treatment t SS MS /MS E error E t(r ) SS E otal tr SS total where r s the number of replcatons per treatment. he treatment effect s adjusted for the effects of the covarate. Equvalent to NCOV wth one between-subjects covarate and one between-subjects factor, n whch the covarate and factor nteracton s not ncluded (p. 38). RCB repeated at locatons We have three orchards, wdely separated a locaton factor. We dvde each orchard nto blocks. We assgn the levels of our treatment to plots wthn those blocks, each treatment once per block. (he number of blocks s the number of replcatons.) For example, f our treatments are C, we mght have ths: Locaton Locaton Block I II III Block I II III B C B C B B B B C C C C Locaton 3 Block I II III C C B C B B Source df SS F locaton L l SS L MS L /MS El error for locatons, El l(b ) SS El treatment t SS MS /MS E L (t )(l ) SS L MS L /MS E error E l(t )(b ) SS E otal ltb SS total Comment: ths s agan equvalent to a desgn wth one between-subjects factor and one wthn-subjects factor (p. ) (Block Subject). Locaton s the between-blocks factor and reatment s the wthn-blocks factor. herefore, t s also analytcally equvalent to the splt plot on a CRD desgn above.

195 7: Specfc desgns 95 RCB repeated n tme Merely one example of repeatng a desgn n tme he orchard s dvded nto blocks; treatments are assgned at random to plots wthn those blocks (each treatment once per block, so the number of blocks s the number of replcatons) and everythng s measured three tmes. t Block I B C D E F Block II F E B D C Block III C B F D E t Block I B C D E F Block II F E B D C Block III C B F D E t3 Block I B C D E F Block II F E B D C Block III C B F D E he approprate NOV depends on the effect of tme. he followng assumes that there s no more correlaton between samples taken closer together n tme than between those taken further apart n tme (a splt-plot n tme ). Source df SS F block B b SS B MS B /MS Em treatment t SS MS /MS Em error, man (Em) (t )(b ) SS Em tme Z z SS Z MS Z /MS E tme block (Z B) (z )(b ) SS ZB MS ZB /MS E tme treatment (Z ) (z )(t ) SS Z MS Z /MS E error, E (z )(t )(b ) SS E otal btz SS total where z s the number of tmes that measurements are taken. Comment : equvalent to a desgn wth two wthn-subjects factors (p. 5). If Block Subject, then reatment s a wthn-subjects factor and me s another wthn-subjects factor. he error, man (Em) term s reatment Block, and the error, E term s me reatment Block. he desgn above s the same as the full model for two wthnsubjects factors dscussed earler (p. 5), except that the agrcultural desgn as quoted here (angren, ) tests Z aganst Z B rather than Z B, whch s a bt odd. Compare the splt-block desgn above. Comment : the assumpton that there s no more correlaton between samples taken closer together n tme than between those taken further apart n tme s a (strong) verson of the assumpton of sphercty that we ve met before n the context of wthn-subjects desgns (p. 5). me s a wthn-subjects factor that frequently leads to volatons of the sphercty assumpton.

196 8: Maths, revson and advanced 96 8 Mathematcs, revson and advanced 8. Matrces Before we can examne a general lnear model, t helps to understand matrx notaton. 8.. Matrx notaton OK, a quck remnder hs s mostly from Myers & Well (995, ppendx C) wth some addtonal notes from plan number, or a symbol that represents one, s called a scalar (e.g., 3.5, c, x). vector s a onedmensonal array of elements, e.g u or v [ ] Here, we would call u a column vector and v a row vector. matrx s a twodmensonal array: 3 Y (More generally, a scalar s a -rank tensor; a vector s a -rank tensor, havng one ndex ; a matrx s a -rank tensor; and so on.) Matrces are frequently denoted wth bold-face type. he number of rows and columns s referred to as the order of the matrx; the matrx Y has order 5 3 (rows columns). So u s a 5 matrx and v s a 6 matrx. We can refer to an element by usng subscrpts n the format element row,column. For example, f we take ths matrx: a a am,,, a a a,, m, a r, c a a a, n, n m, n then a r,c refers to the element n the rth row and the cth column of. Sometmes the comma s mssed out (a rc ). he transpose of matrx s wrtten or. he transpose of a matrx s obtaned by swappng the rows and columns. So the transpose of Y s Y

197 8: Maths, revson and advanced 97 matrx wth equal numbers of rows and columns s called a square matrx. matrx such that s called a symmetrc matrx. In a symmetrc matrx, lke ths: for every element, a j a j. If ths s true, then the elements are symmetrcal about the major (leadng) dagonal of the matrx, whch s the dagonal that extends from the top left to the bottom rght. Matrces that have nonzero elements along ther major dagonals but only zeros as off-dagonal elements are called dagonal matrces. he dentty matrx s a specal square, dagonal matrx that has s along the major dagonal and s elsewhere, such as the 3 3 dentty matrx: I 8.. Matrx algebra Equalty. B f a j b j for all and j. hat s, for two matrces to be equal they must have the same order and dentcal elements. ddton. wo matrces may be added f and only f they are of the same order. C + B f c j a j + b j for all and j. For example, l f k e j d c h b g a l k j h g f e d c b a Subtracton. wo matrces may be added f and only f they are of the same order. C B f c j a j b j for all and j. For example, l f k e j d c h b g a l k j h g f e d c b a Scalar multplcaton. o multply a matrx by a scalar, multple every element n the matrx by the scalar. For example, x xh xg xf xe xd xc xb xa h g f e d c b a x It s not possble to add a scalar to a matrx or to subtract a scalar from a matrx. Matrx multplcaton. o multply matrx by matrx B, gvng the result B B, there must be the same number of columns n as there are rows n B. he smplest case s multplyng a row by a column vector, whch gves a scalar product: [ ] cf be ad f e d c b a + +

198 8: Maths, revson and advanced 98 In general, the product C of two matrces and B s defned by c k a j b jk where j s summed over for all possble values of and k (ths short-hand notaton s known as Ensten summaton). We could expand that formula: j jk j k b a c he number of columns n must equal the number of rows n B. If you multple an x y matrx by a y z matrx, you get an x z matrx. For example, o hm gk n hl gj fo em dk fn el dj co bm ak cn bl aj o n m l k j h g f e d c b a Not all matrces may be multpled by each other. Matrx multplcaton s not commutatve: B s not necessarly the same as B. (If and B are nterpreted as lnear transformatons, then B s the lnear transformaton n whch B s appled frst, and then.) In fact, f B s defned, B may not even be defned, f the number of rows and columns do not match approprately. However, matrx multplcaton s assocatve: (BC) (B)C BC. Matrx multplcaton s also dstrbutve: (B+C) B + C. Multplcaton by the dentty matrx leaves the orgnal matrx unchanged: I I. Note that the order of the dentty matrx that premultples (I) does not have to be the same as the order of the dentty matrx that postmultples t (I), as n ths example: f e d c b a f e d c b a f e d c b a Matrx multplcaton s useful n expressng systems of smultaneous equatons. Suppose D k x z y x then the matrx equaton Dx k ndcates that z y x z y x z y x so the matrx equaton represents a set of three smultaneous scalar equatons.

199 8: Maths, revson and advanced 99 More obscure ways of multplyng matrces. here are, of course, other ways to multply matrces; the one dscussed above s the ordnary matrx product ( nother s the Hadamard product. For two matrces of the same dmenson (m n), the Hadamard product B s gven by ( B),j,j B,j. It s rarely used n lnear algebra. here s another, too; f s an n p matrx and B s an m q matrx, the Kronecker product B (also known as the drect product or the tensor product) s an mn pq matrx: a, B a, B a, pb a B a B a B, B an,b We won t menton these further he nverse of a matrx a, n, B, p an, pb Dvdng a scalar b by another scalar a s equvalent to multplyng b by /a or a, the recprocal or nverse of a. he product of a and ts nverse, a a a a. nalogously, a square matrx s sad to have an nverse f we can fnd a matrx such that I hs s handy for solvng systems of smultaneous equatons; f the equaton x k represents a system of scalar equatons (dscussed above), then we can solve the equatons by premultplyng both sdes of the equaton by : x k Ix k x k Not all matrces have nverses. Matrces that have nverses are called nonsngular; matrces that do not have nverses are called sngular. Only square matrces can have nverses, but not all square matrces do. matrx wll have an nverse only f ts rows and columns are lnearly ndependent. hs s true f no row can be expressed as a lnear combnaton of the other rows, and no column can be expressed as a lnear combnaton of the other columns. (If one row s twce another, for example, the rows are lnearly dependent and the matrx wll have no nverse.) Calculatng the nverse of a matrx can be hard. o fnd the nverse of a matrx, there s a smple formula: a b c d d b d b c a ad bc c a where s called the determnant of ; clearly, the nverse s only defned f the determnant s non-zero. So a matrx s sngular f ts determnant s zero. o fnd the determnant or nverse of a 3 3 matrx or hgher, see Matrx transposton See planetmath.org/encyclopeda/ranspose.html

200 8: Maths, revson and advanced s we saw above, the transpose of a matrx s what you get when you swap all elements a j wth a j. ( ) (c) c where c s a constant If s nvertble, then ( ) ( ) ( + B) + B Pretty obvous: C + B f c j a j + b j for all and j. herefore, C has c j a j + b j. But has members a j and B has members b j, so D + B has members d j a j + b j. Swap the letters and j over, and the defnton of D s the same as that of C ; therefore, ( + B) + B. ( B) B ; the transpose of a product s the product of the transposes n reverse order. Proof: ( B j b k a ) ( b ) jk a b k jk k ( B) ( B) j j ( a ) where Ensten summaton has been used to sum over repeated ndces mplctly; n Ensten s notaton, for example, kj and a a a k a j a a a k a j (see B ( B ) and results, snce B ( B ). hese follow drectly from the precedng B (B (B ) ) 8.. Calculus 8... Dervatves Remember that a dervatve of a functon f(x), wrtten n one of these ways: df f '( x) dx d dx f ( x) s the rate of change of f wth respect to whatever parameters t may have (that s, wth respect to x). Formally, f ( x + h) f ( x) f '( x) : lm h h

201 8: Maths, revson and advanced 8... Smple, non-trgonometrc dervatves d ax dx d ln dx d e dx d a dx n x x x x anx e x d dx n Rules for dfferentaton e x ln a (the power rule) d e dx x ln a (ln a) e Dervatves of sums are equal to the sum of dervatves: xln a (ln a) a x d dx f ( x) + + h( x) f '( x) + + h'( x) If c s a constant, he product rule: he chan rule: d dx d cf dx ( x) cf '( x) f ( x) g( x) f ( x) g'( x) + f '( x) g( x) dy dx dy du. du dx dy du dx du Dervatves of a vector functon he dervatve of a vector functon f( x) f( x) F( x) fk ( x) s gven by df dx df df dx dx df k dx Partal dervatves If a functon has several parameters, such as f(x,y), we can defne the partal dervatve. hs s the dervatve when all parameters except the varable of nterest are held constant durng the dfferentaton. he partal dervatve of f(x,y) wth respect to x s wrtten f f ( x, y) f x x x Formally,

202 8: Maths, revson and advanced h f he f f a h a a f h a f f D h n h ) ( ) ( lm ) ( lm ) ( a a a a + + where e s called the standard bass vector of the th varable (ths s a vector wth a n poston and zeros n every other poston, I would nfer). Calculatng partal dervatves wth respect to x s easy: you treat everythng except x as beng a constant. For example, f z y y xy x f then 3 3 y z f z y y x y f y x x f he chan rule for partal dervatves he general form of the chan rule, usng partal dervatves, s: ds dx x f ds df See planetmath.org/encyclopeda/dervatve.html planetmath.org/encyclopeda/partaldervatve.html phy.asu.edu/phy5-shumway/notes/lec.pdf Illustratons of partal dervatves Suppose we have the functon 3 xy x f +. Its partal dervatves wth respect to x and y are: xy y f y x f We can llustrate the whole functon:

203 8: Maths, revson and advanced 3 Plot of f x + 3xy and some partal dervatves: f f + 3y, shown at y 5, where 77 x x f f 6xy, shown at x, where y y f f + 3y, shown at y +3, where 9 x x f f 6xy, shown at x 7, where 4y y y If you re wonderng how you d fnd the drecton n whch a ball would roll down ths slope (the drecton n whch the gradent s maxmum), and the gradent n that drecton, that s gven by the vector gradent ( grad ), denoted f grad( f ). Detals at

204 8: Maths, revson and advanced Solvng a GLM (an overdetermned system of equatons) (advanced) Solvng a GLM s the the problem of solvng e b y + for b so as to mnmze the sum of squares of the resduals, ) ( Yˆ Y or e. When ths s solved, b contans the correct regresson coeffcents. Note that we can wrte an expresson for e: b y e e b y + he error (resdual) sum of squares can be wrtten lke ths: [ ] error error SS SS n n n e e e e e e e e e e e e e e e We can also wrte t lke ths: b b y b y y b b y b y b y y b b y b y) (b y y b b y b y) ((b) y y b b y (b) (b) y y y b y b y b y b y e e ) ( ) ( ) )( ) ( ( ) ( ) ( SS error from defnton of e above usng B B + + ) ( multplyng out usng B B ) ( usng B B ) ( twce because each term n the sum s a real number, and hence equal to ts transpose o mnmze the sum of squares, we solve so that the partal dervatve of the sum of squares wth respect to the model parameters (b) s zero. o do ths, we wll need to use an partal dervatve analogue of the product rule for dfferentaton, whch s ) ( ) '( ) '( ) ( ) ( ) ( x g x f x g x f x g x f dx d + he vector b s a set of parameters b, b, b b n. We dfferentate wth respect to each b. he partal dervatve of b wth respect to b s a vector wth a n the th poston and n every other poston (see secton on partal dervatves, and dervatves of a vector functon). We call that vector e (the standard bass vector ); I wll use ths notaton to avod confuson wth the error vector e. n e b b b b b b b Smlarly,

205 8: Maths, revson and advanced 5 b e b rmed wth ths notaton, we can obtan the partal dervatve of SS error, whch we wsh to be equal to zero: ( ) ( ) ( ) ( ) ( ) ( ) SS SS error error b y b b y b b y b b y b b y b b y b b y b b y b y y b b y b y y e e e e e e e e e e e e e e e e e e e e b b Let s do ths n full. he frst term (y y) contans no terms nvolvng b so s treated as a constant. he second s smple. he thrd has two terms nvolvng b, namely b and b, so we use the product rule, dfferentatng wth respect to each n turn. For the expanson of the rghthand term, we use B B ) ( n the form B B ) (. Next, from B B ) ( t follows that ) (. Each term s a real number, and therefore equal to ts transpose. Rearrangng: y b y b b y e e e e e e + hs says that the th element of b s equal to the th element of y. Snce that s true for all values of, we have the equalty y b hese equatons (snce the thngs n the expresson above are matrces, they represent more than one equaton) are known as the normal equatons of the lnear leastsquares problem. If we can fnd the nverse of, and rememberng that matrx multplcaton s assocatve (BC) (B)C BC we can derve ths expresson for b: y b y b ) ( ) ( ) ( herefore, our optmal model b that mnmzes the SS error s gven by y b ) ( Magc. Of course, the soluton can only be found f s nvertble (whch may not be the case f your desgn matrx contans lnearly dependent columns, as wth overparametrzed desgn matrces). For terse versons of these dervatons, see

206 8: Maths, revson and advanced Sngular value decomposton to solve GLMs (very advanced) Sngular value decomposton (SVD) s a method that can solve arbtrary GLMs ones n whch we have more nformaton that we need (as s the case n NOV), and also ones n whch we have exactly the rght amount of nformaton, and ones n whch we have nsuffcent nformaton. When we solve a GLM, we normally have more measurements than we d need to determne the values of our predctors the model s y b + e, t s overdetermned (e ), and we solve t by mnmzng the sum of squares of the errors (e e). We can often solve t usng the normal equatons gven above ( b y, or b ( ) y ). When we solve a smple set of equatons that are exactly determned, we solve y b (gvng b y). hs s equvalent to the method for an overdetermned problem, except that e (our predctons are exact and there s no resdual error). What happens f we don t have enough nformaton? hen our model y b s underdetermned. Yet f we make assumptons about the world, we can stll get useful nformaton out. For example, suppose we re performng a C scan. We scan a sngle slce of the body. We want to fnd a set of -ray absorbances b, one absorbance per voxel. We know whch voxels each -ray beam passes through (), and we know the sum of absorbances for each beam (y), assumng some radaton manages to get through (f the -ray beam s completely absorbed, the maths s harder, whch may be why metal causes funny streaky shadows on C scans). I would guess that C scans are normally overdetermned, or perhaps exactly determned (though I reckon probably not t d be easer to desgn a machne that made overdetermned scans and the results would probably better, although the prce s a bt of tme and a bt of unnecessary -ray radaton). What happens f we had an undetermned stuaton lke tryng to nterpret 3D structure from an antero-posteror (P) and a lateral chest - ray only? Or lke shootng a C scan from too few drectons? We could assume that tssue s homogeneous unless we receve better nformaton. hat corresponds to mnmzng the sum of squares of b ( b ). very smple example: suppose x + y. hs has an nfnte number of solutons. But the one that mnmzes x + y s x 5, y 5. In general, we may wsh to mnmze both e and b. general technque for ths s called sngular value decomposton (SVD). I won t present t n full, because I don t understand t n full, but t goes lke ths Egenvectors and egenvalues If s a matrx and f there s a column vector such that or ( λi) R R λ R where I s the dentty matrx for some scalar λ, then λ s called the egenvalue of wth the correspondng (rght) egenvector R. (German: egen approprate, nnate, own, pecular.) hat s, an egenvector s a vector whose drecton s unchanged by the transformaton ; t s merely stretched by a factor (the egenvalue). For example, f the matrx represents rotaton, t has no egenvectors. If t represents reflecton n a plane, then every vector lyng n that plane s an egenvector, wth egenvalue, and any vector perpendcular to the plane wll be an egenvector wth egenvalue ; these are the only egenvectors. If the matrx represents D reflecton (reflecton n a lne), then vectors lyng along that lne wll be egenvectors wth egenvalue, and vectors perpen-

207 dcular to that lne wll be egenvectors wth egenvalue ; these are the only egenvectors. If the matrx represents smultaneous enlargement parallel to the axs by a factor of a, parallel to the Y axs by a factor of b, and parallel to the Z axs by a factor of c, wth a b c, so the matrx looks lke a b c then vectors along ether the axs, the Y axs, or the Z axs wll be egenvectors (and these are the only egenvectors), and ther egenvalues wll be a, b, and c respectvely. o fnd egenvalues, note that f ( λi) R and R then ( λi) must be sngular, so solve det ( λ I) to get the egenvalues, and thus the egenvectors. Less commonly used: he left egenvector s a row vector that satsfes L 8: Maths, revson and advanced 7 λ or ( λ I) L, where I s the dentty matrx. he egenvalues for the left and rght egenvectors are the same, although the left and rght egenvectors themselves need not be. When people use the term egenvector on ts own they generally mean rght egenvector. square matrx can often be decomposed ( dagonalsed ) nto ts egenvalues and egenvectors, whch are lnearly ndependent. hat s, L PDP where P s a matrx of egenvectors and D s a dagonal matrx of egenvalues. See Sngular value decomposton ny m n matrx can be decomposed nto where USV U s an m m orthogonal matrx (a matrx M s orthogonal f MM I,.e. f M M ); the columns of U are the egenvectors of. V s an n n orthogonal matrx; the columns of V are the egenvectors of. S s an m n matrx contanng a dagonal matrx (a matrx that has nonzero elements along ts major dagonal but only zeros elsewhere) wth real, nonnegatve elements σ (where s from to the mnmum of m and n) n descendng order: σ > σ > > σ mn( m, n) > he σ elements themselves (the sngular values ) are square roots of egenvalues from or. o create S, we frst create a dagonal matrx contanng these σ elements:

208 8: Maths, revson and advanced 8 σ Σ σ σ mn( m, n ) hen we pad t wth zeros to make S an m n matrx: Σ S [ Σ ] f m n f m n and S < Once we ve found the matrces such that USV, we can then solve our problem. Snce U and V are orthogonal, ther nverses are equal to ther transposes. Snce S s dagonal, ts nverse s the dagonal matrx whose elements are the recprocal of the elements of S. ( USV ) ( V ) S U VS U where the dagonal elements of S are /S [that s, S dag(/s )]. herefore, snce y b, we have b y and hence b VS It s possble to solve equatons even f the matrces are sngular or close to sngular usng ths technque: when you obtan S, by takng the values /S, f S s smaller than a threshold value (the sngularty threshold) you replace /S wth. hat s, SVD fnds the least squares best compromse soluton of the lnear equaton system. For detals and proof, see Press et al. (99, pp. 59-7, ) and rkb.home.cern.ch/rkb/n6pp/node65.html n underdetermned set of equatons: the role of expectatons (RNC, prl 4.) lternatvely, we mght have pror expectatons n our radologcal example, we expect to fnd a heart, we expect that rbs curve round the sde, and so on. We mght say that we d lke to nterpret the data to ft our expectatons as far as possble. If our pror expectatons are p, then ths would correspond to mnmzng the sum of squares of (b p). We can say that b p + d, where d represents the devaton from pror expectatons. hus, U y y b (p + d) p + d y p d he usual sngular value decomposton USV s used to solve y b for b, mnmzng the sum of squares of b when the system s underdetermned; the soluton s gven by b VS U y. In the present case, we use the same decomposton of and smply rewrte to solve for d, mnmzng ts sum of squares: d VS U (y p) and therefore snce b p + d, b p + VS U (y p)

209 8: Maths, revson and advanced Random varables, means, and varances 8.5. Summaton If we have n scores we could denote them x, x, x n. her sum can be wrtten n the followng ways: x + x + + x n n x x he followng are easly proved. If c s a constant, then cx c x n c nc he summaton sgn operates lke a multpler on quanttes wthn parentheses. For example: n ( x n y ) x n y ( x y) x + y + xy 8.5. Random varables; defnton of mean and varance random varable (RV) s a measurable or countable quantty that can take any of a range of values and whch has a probablty dstrbuton assocated wth t,.e. there s a means of gvng the probablty of the varable takng a partcular value. If the values an RV can take are real numbers (.e. an nfnte number of possbltes) then the RV s sad to be contnuous; otherwse t s dscrete. he probablty that a dscrete RV has the value x s denoted P(x). We can then defne the mean or expected value: E[ ] xp( x) and the varance: Var[ ] E[( x E[ ]) ( x E[ ]) P( x) E[ ] ] ( x xe[ ] + ( E[ ]) ) P( x) x P( x) xp( x) E[ ] + ( E[ ]) P( x) x P( x) E[ ] E[ ] + ( E[ ]) P( x) x P( x) ( E[ ]) ( E[ ]) + ( E[ ]) and the standard devaton, σ: σ Var[ ] Contnuous random varables For a contnuous random varable, the probablty P(x) of an exact value x occurrng s zero, so we must work wth the probablty densty functon (PDF), f(x). hs s defned as P( a x b) f ( x) dx b a

210 8: Maths, revson and advanced f ( x) dx x : f ( x) ( x means for all values of x ). he mean or expected value E[] s defned as he varance, Var[] s gven by E[ ] xf ( x) dx Var[ ] x f ( x) dx ( E[ ]) he cumulatve dstrbuton functon (CDF, also known as the dstrbuton functon or cumulatve densty functon ), F(a), s gven by F( a) a f ( x) dx.e. F( a) P( x a) P( a x b) F( b) F( a) Expected values If s a random varable and c s a constant, E() denotes the expected value of. E() acts lke a multpler. For example: E ( c) c E ( c ) ce( ) E( + Y ) E( ) + E( Y ) E( + c) E( ) + E( c) E( ) + c E[( + Y ) ] E( ) If and Y are ndependently dstrbuted, then + E( Y ) E ( Y ) E( ) E( Y ) + E( Y ) he sample mean and SD are unbased estmators of µ and σ We wll use to denote the random varable, x for an ndvdual value of that random varable, x for the sample mean, s for the sample varance (sometmes wrt- ten ˆ σ ), µ for the populaton mean, and σ for the populaton varance. Frst, the mean: x E( x) E E( x) E( ) ne( ) E( ) µ n n n n Now the standard devaton (Myers & Well, 995, p. 59). Consder frst the numerator (the sum of squares) [N.B. lne 3 uses the fact ( x µ ) n( x µ ) ]:

211 8: Maths, revson and advanced [ ] ) ( ) ( ] ) ( ) ( [ ] ) ( ) ( ) ( [ )] ( ) ( ) ( ) ( [ ) ( ) ( ] ) ( [ µ µ µ µ µ µ µ µ µ µ µ µ µ + + x ne x E x n x E x n x n x E x x x x E x x E x x E he average squared devaton of a quantty from ts average s a varance; that s, ) ( x E σ µ and, by the Central Lmt heorem, n x E ) ( σ σ µ herefore, ) ( ] ) ( [ n n n n x x E σ σ σ Hence ) ( ) ( s E n x x E σ Varance laws If and Y are two random varables wth varances V() σ and V(Y) Y σ, and c s a constant, then ) ( ) ( ) ( ) ( ) ( c V c c V V c V c V σ σ + Y Y Y Y Y Y Y Y Y V Y V σ σ ρ σ σ σ σ σ ρ σ σ σ ) ( ) ( where ρ s the correlaton between and Y; Y Y σ σ ρ s also known as the covarance: Y Y Y σ σ ρ cov herefore, f and Y are ndependent, ) ( ) ( cov Y Y Y Y Y Y Y V Y V σ σ σ σ σ σ ρ Dstrbuton of a set of means: the standard error of the mean See Frank & lthoen (994, pp. 8-89). Let,, N be a set of sample means. hen s the mean of all those sample means. Frst we derve the densty functon of.

212 8: Maths, revson and advanced If we sample n values from a random varable, callng them x, x x n, then ther mean s x x n or x ( x + x + + xn ) n Lkewse, for a set of n random varables, n, n ( n n n n n) n Let then W n W + W + + W n If, n are ndependent and dentcally dstrbuted, as when observatons are ndependent, then W, W W n are lkewse ndependent and dentcally dstrbuted. he mean can therefore be expressed as the sum of n ndependent, dentcally dstrbuted random varables, W. he Central Lmt heorem tells us that f W, W, W n are ndependent, dentcally dstrbuted random varables and Y W + W + + W n, then the probablty densty functon of Y approaches the normal dstrbuton ( yµ y ) σ Y e πσ Y as n. Next we derve the expected value of the sample mean, E ( ). (We saw one dervaton above; ths s a fuller verson.) Snce t follows that ( n n ( ) E ( ) E( + + ) E + n n n n From the lgebra of Expectatons, the expected value of a sum s equal to the sum of the expected values. So f E( ) µ, E( ) µ, E( n ) µ n, etc., then ( ) E( µ + µ + + ) E µ n n Let us suppose the populaton mean s µ. Snce the dstrbutons of,, n are all dentcal to the populaton dstrbuton, t follows that all n random varables have the same expected value: µ µ µ µ n )

213 8: Maths, revson and advanced 3 So E ( ) E( µ + µ + + µ ) nµ µ n n So the expected value of the sample mean (the mean of a set of sample means) s equal to the populaton mean. How about the varance of? ( n n ) So V ( ) V ( ) n n When you factor a constant out of a varance, t s squared: ( ) V ( + + ) V + n he varance of a sum of n ndependent random varables s the sum of the ndvd- V σ, then ual varances. If V ( ), V ( ),, ( n ) n σ V σ n ( ) σ + σ + + σ n n so V ( ) ( σ + σ + + σ ) n n and snce the varables representng our n observatons all have the same dstrbuton as the parent populaton, they must all have the same varance, namely σ, the populaton varance. So σ V ( ) ( Nσ ) n n So for samples of n ndependent observatons, the varance of the sample means s equal to the populaton varance dvded by the sample sze: σ σ and so the standard devaton of the sample means (the standard error of the mean) s σ σ n n 8.6 he harmonc mean he harmonc mean of n observatons x, x, x n s h + x x n + + x n

214 9: Glossary 4 9 Glossary Symbols: mples s equvalent to x mean of a set of values of x ε error εˆ ~ Greenhouse Gesser correcton (see p. 5) ε Huynh Feldt correcton (see p. 5) µ mean ρ populaton correlaton r sample correlaton r xy or r x. y correlaton between x and y r y. a, b, c multple correlaton between y and (a, b, c) ry.( x z) sempartal correlaton between y and x, havng partalled out z (see p. ) r y. x z partal correlaton between y and x, havng partalled out z (see p. ) sum of (see p. 9) σ populaton standard devaton of s sample standard devaton of σ populaton varance of s sample varance of ddtve model. In wthn-subjects NOV, a structural model that assumes the effects of wthn-subjects treatments are the same for all subjects. NCOV. nalyss of covarance: an NOV that uses a covarate as a predctor varable. NOV. nalyss of varance. See p. 8 for an explanaton of how t works. pror tests. ests planned n advance of obtanng the data; compare post hoc tests. Balanced NOV. n NOV s sad to be balanced when all the cells have equal n, when there are no mssng cells, and f there s a nested desgn, when the nestng s balanced so that equal numbers of levels of the nested factor appear n the levels of the factor(s) that they are nested wthn. hs greatly smplfes the computaton. Between-subjects (factor or covarate). If each subject s only tested at a sngle level of an ndependent varable, the ndependent varable s called a betweensubjects factor. Compare wthn-subjects. Carryover effects. See wthn-subjects. Categorcal predctor varable. varable measured on a nomnal scale, whose categores dentfy class or group membershp, used to predct one or more dependent varables. Often called a factor. Contnuous predctor varable. contnuous varable used to predct one or more dependent varables. Often called a covarate. Covarance matrx. If you have three varables x, y, z, the covarance matrx, x y z x σ x covxy cov xz denoted, s Σ where cov y cov xy σ y cov xy s the covarance of yz z cov cov xz yz σ z x and y ( ρ xy σ x σ y where ρ xy s the correlaton between x and y and σ x s the varance of x). Obvously, covxx σ x. It s sometmes used to check for compound symmetry of the covarance matrx, whch s a fancy way of sayng

215 9: Glossary 5 x y z σ σ σ (all numbers on the leadng dagonal the same as each other). and cov xy covyz covxz (all numbers not on the leadng dagonal the same as each other). If there s compound symmetry, there s also sphercty, whch s what s mportant when you re runnng NOVs wth wthn-subjects factors. On the other hand, you can have sphercty wthout havng compound symmetry; see p. 5. Conservatve. pt to gve p values that are too large. Contrast. See lnear contrast. Covarate. contnuous varable (one that can take any value) used as a predctor varable. Degrees of freedom (df). Estmates of parameters can be based upon dfferent amounts of nformaton. he number of ndependent peces of nformaton that go nto the estmate of a parameter s called the degrees of freedom (d.f. or df). Or, the number of observatons free to vary. For example, f you pck three numbers at random, you have 3 df but once you calculate the sample mean, x, you only have two df left, because you can only alter two numbers freely; the thrd s constraned by the fact that you have fxed x. Or, the number of measurements exceedng the amount absolutely necessary to measure the object (or parameter) n queston. o measure the length of a rod requres measurement. If measurements are taken, then the set of measurements has 9 df. In general, the df of an estmate s the number of ndependent scores that go nto the estmate mnus the number of parameters estmated from those scores as ntermedate steps. For example, f the populaton varance σ s estmated (by the sample varance s ) from a random sample of n ndependent scores, then the number of degrees of freedom s equal to the number of ndependent scores (n) mnus the number of parameters estmated as ntermedate steps (one, as µ s estmated by x ) and s therefore n. Dependent varable. he varable you measure, but do not control. NOV s about predctng the value of a sngle dependent varable usng one or more predctor varables. Desgn matrx. he matrx n a general lnear model that specfes the expermental desgn how dfferent factors and covarates contrbute to partcular values of the dependent varable(s). Doubly-nested desgn. One n whch there are two levels of nestng (see nested desgn). Some are descrbed on p. 59. Error term. o test the effect of a predctor varable of nterest wth an NOV, the varablty attrbutable to t (MS varable ) s compared to varablty attrbuted to an approprate error term (MS error ), whch measures an approprate error varablty. he error term s vald f the expected mean square for the varable, E(MS varable ), dffers from E(MS error ) only n a way attrbutable solely to the varable of nterest. Error varablty (or error varance, σ e ). Varablty among observatons that cannot be attrbuted to the effects of the ndependent varable(s). May nclude measurement error but also the effects of lots of rrelevant varables that are not measured or consdered. It may be possble to reduce the error varablty by accountng for some of them, and desgnng our experment accordngly. For example, f we want to study the effects of two methods of teachng readng on chldren s readng performance, rather than randomly assgnng all our students to teachng method or teachng method, we could splt our chldren nto groups wth low/medum/hgh ntellgence, and randomly allocate students from each level of ntellgence to one of our two teachng methods. If ntellgence accounts for some of the varablty n readng ablty, accountng for t n ths way wll reduce our error varablty. Wthn-subjects desgns take ths prncple further (but are susceptble to carryover effects). Expected mean square (EMS). he value a mean square (MS) would be expected to have f the null hypothess were true. F rato. he rato of two varances. In NOV, the rato of the mean square (MS) for a predctor varable to the MS of the correspondng error term.

216 9: Glossary 6 Factor. dscrete varable (one that can take only certan values) used as a predctor varable. categorcal predctor. Factors have a certan number of levels. Factoral NOV. n NOV usng factors as predctor varables. he term s often used to refer to NOVs nvolvng more than one factor (compare oneway NOV). Factoral desgns range from the completely randomzed desgn (subjects are randomly assgned to, and serve n only one of several dfferent treatment condtons,.e. completely between-subjects desgn), va mxed desgns (both between-subjects and wthn-subjects factors) to completely wthnsubjects desgns, n whch each subject serves n every condton. Fxed factor. factor that contans all the levels we are nterested n (e.g. the factor sex has the levels male and female). Compare random factor and see p. 3. Gaussan dstrbuton. Normal dstrbuton. General lnear model. general way of predctng one or more dependent varables from one or more predctor varables, be they categorcal or contnuous. Subsumes regresson, multple regresson, NOV, NCOV, M- NOV, MNCOV, and so on. Greenhouse Gesser correcton/epslon. If the sphercty assumpton s volated n an NOV nvolvng wthn-subjects factors, you can correct the df for any term nvolvng the WS factor (and the df of the correspondng error term) by multplyng both by ths correcton factor. Often wrtten εˆ, where < εˆ. Orgnally from Greenhouse & Gesser (959). Heterogenety of varance. Opposte of homogenety of varance. When varances for dfferent treatments are not the same. Herarchcal desgn. One n whch one varable s nested wthn a second, whch s tself nested wthn a thrd. doubly-nested desgn (such as the spltsplt plot desgn) s the smplest form of herarchcal desgns. hey re complex. Homogenety of varance. When a set of varances are all equal. If you perform an NOV wth a factor wth a levels, the homogenety of varance assumpton s that σ σ σ a σ e, where σ e s the error varance. Huynh Feldt correcton/epslon. If the sphercty assumpton s volated n an NOV nvolvng wthn-subjects factors, you can correct the df for any term nvolvng the WS factor (and the df of the correspondng error term) by multplyng both by ths correcton factor. Often wrtten ~ ε, where < ~ ε. Orgnally from Huynh & Feldt (97). Independent varable. he varables thought to be nfluencng the dependent varable(s). In experments, ndependent varables are manpulated. In correlatonal studes, ndependent varables are observed. (he advantage of the experment s the ease of makng causal nferences.) Interacton. here s an nteracton between factors and B f the effect of factor depends on the level of factor B, or vce versa. For example, f your dependent varable s engne speed, and your factors are presence of spark plugs (Y/N) () and presence of petrol (Y/N) (B), you wll fnd an nteracton such that factor only nfluences engne speed at the petrol present level of B; smlarly, factor B only nfluences engne speed at the spark plugs present level of B. hs s a bnary example, but nteractons need not be. Compare man effect, smple effect. Intercept. he contrbuton of the grand mean to the observatons. See p. 65. he F test on the ntercept term (MS ntercept /MS error ) tests the null hypothess that the grand mean s zero. Level (of a factor). One of the values that a dscrete predctor varable (factor) can take. For example, the factor Weekday mght have fve levels Monday, uesday, Wednesday, hursday, Frday. We mght wrte the factor as Weekday 5 n descrptons of NOV models (as n edum Drowsness Weekday 5 S ), or wrte the levels themselves as Weekday Weekday 5. Levene s test (for heterogenety of varance). Orgnally from Levene (96). ests the assumpton of homogenety of varance. If Levene s test produces a sgnfcant result, the assumpton of homogenety of varance cannot be made (ths s generally a Bad hng and suggests that you mght need to transform your data to mprove the stuaton; see p. 34).

217 9: Glossary 7 Lberal. pt to gve p values that are too small. Lnear contrasts. Comparsons between lnear combnatons of dfferent groups, used to test specfc hypotheses. See p. 75. Lnear regresson. Predctng Y from usng the equaton of a straght lne: Y ˆ b + a. May be performed wth regresson NOV. Logstc regresson. See Howell (997, pp ). logstc functon s a sgmod (see If your dependent varable s dchotomous (categoral) but ordered ( flght on tme versus flght late, for example) and you wsh to predct ths (for example, by plot experence), a logstc functon s often better than a straght lne. It reflects the fact that the dchotomy mposes a cutoff on some underlyng contnuous varable (e.g. once your flght delay n seconds contnuous varable reaches a certan level, you classfy the flght as late dchotomous varable). Dchotomous varables can be converted nto varables sutable for lnear regresson by convertng the probablty of fallng nto one category, P(flght late), nto the odds of fallng nto that category, usng P( ) odds, and then nto the log odds, usng the natural (base e) logarthm P( ) log e (odds) ln(odds). he probablty s therefore a logstc functon of the log ln(odds) e odds: probablty, so performng a lnear regresson on the log ln(odds) + e odds s equvalent to performng a logstc regresson on probablty. hs s pretty much what logstc regresson does, gve or take some procedural wrnkles. Odds ratos (lkelhood ratos), the odds for one group dvded by the odds for another group, emerge from logstc regresson n the way that slope estmates emerge from lnear regresson, but the statstcal tests nvolved are dfferent. Logstc regresson s a computatonally teratve task; there s no smple formula (the computer works out the model that best fts the data teratvely). Man effect. man effect s an effect of a factor regardless of the other factor(s). Compare smple effect; nteracton. MNCOV. Multvarate analyss of covarance; see MNOV and NCOV. MNOV. Multvarate NOV NOV that deals wth multple dependent varables smultaneously. Not covered n ths document. For example, f you thnk that your treatment has a bgger effect on dependent varable Y than on varable Y, how can you see f that s the case? Certanly not by makng categorcal decsons based on p values (sgnfcant effect on Y, not sgnfcant effect on Y ths wouldn t mean that the effect on Y and Y were sgnfcantly dfferent!). Instead, you should enter Y and Y nto a MNOV. Mauchly s test (for sphercty of the covarance matrx). Orgnally from Mauchly (94). See sphercty, covarance matrx, and p. 5. Mean square (MS). sum of squares (SS) dvded by the correspondng number of degrees of freedom (df), or number of ndependent observatons upon whch your SS was based. hs gves you the mean squared devaton from the mean, or the mean square. Effectvely, a varance. Mxed model. n NOV model that ncludes both between-subjects and wthn-subjects predctor varables. lternatvely, one that ncludes both fxed and random factors. he two uses are often equvalent n practce, snce Subjects s usually a random factor. Multple regresson. Predctng a dependent varable on the bass of two or more contnuous varables. Equvalent to NOV wth two or more covarates. Nested desgn. n NOV desgn n whch varablty due to one factor s nested wthn varablty due to another factor. For example, f one were to admnster four dfferent tests to four school classes (.e. a between-groups factor wth four levels), and two of those four classes are n school, whereas the other two classes are n school B, then the levels of the frst factor (four dfferent tests) would be nested n the second factor (two dfferent schools). very common example s a desgn wth one between-subjects factor and one wthnsubjects factor, wrtten (U S); varaton due to subjects s nested wthn varaton due to (or, for short-hand, S s nested wthn ), because each subject s only tested at one level of the between-subjects factor(s). We mght wrte ths S/ ( S s nested wthn ); SPSS uses the alternatve notaton of S(). See also doubly-nested desgn.

218 9: Glossary 8 Nonaddtve model. In wthn-subjects NOV, a structural model that allows that the effects of wthn-subjects treatments can dffer across subjects. Null hypothess. For a general dscusson of null hypotheses, see handouts at In a one-way NOV, when you test the man effect of a factor wth a levels, your null hypothess s that µ µ µ a. If you reject ths null hypothess (f your F rato s large and sgnfcant), you conclude that the effects of all a levels of were not the same. But f there are > levels of, you do not yet know whch levels dffered from each other; see post hoc tests. One-way NOV. NOV wth a sngle between-subjects factor. Order effects. See wthn-subjects. Overparameterzed model. way of specfyng a general lnear model desgn matrx n whch a separate predctor varable s created for each group dentfed by a factor. For example, to code Sex, one varable would be created n whch males score and females score, and another varable would be created n whch males score and females score. hese two varables contan mutually redundant nformaton: there are more predctor varables than are necessary to determne the relatonshp of a set of predctors to a set of dependent varables. Compare sgma-restrcted model. Planned contrasts. Lnear contrasts run as a pror tests. Polynomal NCOV. n NCOV n whch a nonlnear term s used as a predctor varable (such as x, x 3, rather than the usual x). See Myers & Well (995, p. 46). Post hoc tests. Statstcal tests you run after an NOV to examne the nature of any man effects or nteractons you found. For example, f you had an NOV wth a sngle between-subjects factor wth three levels, sham/core/shell, and you found a man effect of ths factor, was ths due to a dfference between sham and core subjects? Sham and shell? Shell and core? re all of them dfferent? here are many post hoc tests avalable for ths sort of purpose. However, there are statstcal ptfalls f you run many post-hoc tests; you may make ype I errors (see handouts at smply because you are runnng lots of tests. Post hoc tests may nclude further NOVs of subsets of your orgnal data for example, after fndng a sgnfcant Group Dffculty nteracton, you mght ask whether there was a smple effect of Group at the easy level of the Dffculty factor, and whether there was a smple effect of Group at the dffcult level of the Dffculty factor (see pp., 39 ). Power of an NOV. Complex to work out. But thngs that ncrease the expected F rato for a partcular term f the null hypothess s false ncrease power, MSpredctor SSpredctor dferror and F. Bgger samples contrbute to a larger MSerror SSerror dfpredctor df for your error term; ths therefore decreases MS error and ncreases the expected F f the null hypothess s false, and ths therefore ncreases your power. he larger the rato of E(MS treatment ) to E(MS error ), the larger your power. Sometmes two dfferent structural models gve you dfferent EMS ratos; you can use ths prncple to fnd out whch s more powerful for detectng the effects of a partcular effect (see p. 73 ). For references to methods of calculatng power drectly, see p.. Predctor varable. Factors and covarates: thngs that you use to predct your dependent varable. Pseudoreplcaton. What you do when you analyse correlated data wthout accountng for the correlaton. Bad hng entrely Wrong. For example, you could take 3 subjects, measure each tmes, and pretend that you had 3 ndependent measurements. No, no, no, no, no. ccount for the correlaton n your analyss (n ths case, by ntroducng a Subject factor and usng an approprate NOV desgn wth a wthn-subjects factor). Random factor. factor whose levels we have sampled at random from many possble alternatves. For example, Subjects s a random factor we pck our subjects out of a large potental pool, and f we repeat the experment, we may use dfferent subjects. Compare fxed factor and see p. 3.

219 9: Glossary 9 Regresson NOV. Performng lnear regresson usng NOV. smple lnear regresson s an NOV wth a sngle covarate (.e. NCOV) and no other factors. Repeated measures. Same as wthn-subjects. Repeated measures s the more general term wthn-subjects desgns nvolve repeated measurements of the same subject, but thngs other than subjects can also be measured repeatedly. In general, wthn-subjects/repeated-measures analyss s to do wth accountng for relatedness between sets of observatons above that you d expect by chance. Repeated measurement of a subject wll tend to generate data that are more closely related (by vrtue of comng from the same subject) than data from dfferent subjects. Robust. test that gves correct p values even when ts assumptons are volated to some degree ( ths test s farly robust to volaton of the normalty assumpton ). Sequence effects. See wthn-subjects. Sgma-restrcted model. way of specfyng a general lnear model n whch a categorcal varable wth k possble levels s coded n a desgn matrx wth k varables. he values used to code membershp of partcular groups sum to zero. For example, to code Sex, one varable would be created n whch males score + and females. Compare overparameterzed model. Smple effect. n effect of one factor consdered at only one level of another factor. smple effect of at level of factor B s wrtten at B or /B. See man effect, nteracton, and pp., 39. Source of varance (SV). Somethng contrbutng to varaton n a dependent varable. Includes predctor varables and error varablty. Sphercty assumpton. n mportant assumpton applcable to wthn-subjects (repeated measures) NOV. Sphercty s the assumpton of homogenety of varance of dfference scores. Suppose we test 5 subjects at three levels of. We can therefore calculate three sets of dfference scores ( 3 ), ( ), and ( 3 ), for each subject. Sphercty s the assumpton that the varances of these dfference scores are the same. See p. 5. Standard devaton. he square root of the varance. Structural model. n equaton gvng the value of the dependent varable n terms of sources of varablty ncludng predctor varables and error varablty. Sum of squares (SS). In full, the sum of the squared devatons from the mean. See varance. Sums of squares are used n preference to actual varances n NOV, because sample sums of squares are addtve (you can add them up and they stll mean somethng) whereas sample varances are not addtve unless they re based on the same number of degrees of freedom. t test, one-sample. Equvalent to testng MS ntercept /MS error wth an NOV wth no other factors (odd as that sounds). F, k t k and t k F, k. See ntercept. t test, two-sample, pared. Equvalent to NOV wth one wthn-subjects factor wth two levels. F t and t k F, k., k k t test, two-sample, unpared. Equvalent to NOV wth one betweensubjects factor wth two levels. F, k t k and t k F, k. Varance. o calculate the varance of a set of observatons, take each observaton and subtract t from the mean. hs gves you a set of devatons from the mean. Square them and add them up. t ths stage you have the sum of the squared devatons from the mean, also known as the sum of squares (SS). Dvde by the number of ndependent observatons you have (n for the populaton varance; n for the sample varance; or, n general, the number of degrees of freedom) to get the varance. See the Background Knowledge handouts at Wthn-subjects (factor or covarate). See also repeated measures. If a score s obtaned for every subject at each level of an ndependent varable, the ndependent varable s called a wthn-subjects factor. See also between-subjects. he advantage of a wthn-subjects desgn s that the dfferent treatment condtons are automatcally matched on many rrelevant varables all those that

220 are relatvely unchangng characterstcs of the subject (e.g. ntellgence, age). However, the desgn requres that each subject s tested several tmes, under dfferent treatment condtons. Care must be taken to avod order, sequence or carryover effects such as the subject gettng better through practce, worse through fatgue, drug hangovers, and so on. If the effect of a treatment s permanent, t s not possble to use a wthn-subjects desgn. You could not, for example, use a wthn-subjects desgn to study the effects of parachutes (versus no parachute) on mortalty rates after fallng out of a plane. 9: Glossary

221 : Further readng Further readng very good statstcs textbook for psychology s Howell (997). belson (995) examnes statstcs as an technque of argument and s very clear on the logcal prncples and some of the phlosophy of statstcs. Keppel (99) s a farly hefty tome on NOV technques. Wner (99) s another monster reference book. Nether are for the fant-hearted. Myers & Well (995) s another excellent one. Less fluffy than Howell (997) but deals wth the ssues head on. here s also an excellent seres of Statstcs Notes publshed by the Brtsh Medcal Journal, mostly by Bland and ltman. lst s avalable at and the artcles themselves are avalable onlne from hs seres ncludes the followng: he problem of the unt of analyss (ltman & Bland, 997). Correlaton and regresson when repeated measurements are taken, and the problem of pseudoreplcaton (Bland & ltman, 994a). he approach one should take to measure correlaton wthn subjects (Bland & ltman, 995a) and correlaton between subjects (Bland & ltman, 995b). Why correlaton s utterly napproprate for assessng whether two ways of measurng somethng agree (Bland & ltman, 986). Generalzaton and extrapolaton (ltman & Bland, 998). Why to randomze (ltman & Bland, 999b), how to randomze (ltman & Bland, 999a), and how to match subjects to dfferent expermental groups (Bland & ltman, 994b). Blndng (Day & ltman, ; ltman & Schulz, ). bsence of evdence s not evdence of absence about power (ltman & Bland, 995). Multple sgnfcance tests: the problem (Bland & ltman, 995c). Regresson to the mean (Bland & ltman, 994e; Bland & ltman, 994d). One-taled and two-taled sgnfcance tests (Bland & ltman, 994c). ransformng data (Bland & ltman, 996b) and how to calculate confdence ntervals wth transformed data (Bland & ltman, 996c; Bland & ltman, 996a). NOV, brefly (ltman & Bland, 996), and the analyss of nteracton effects (ltman & Matthews, 996; Matthews & ltman, 996a; Matthews & ltman, 996b). Comparng estmates derved from separate analyses (ltman & Bland, 3). Dealng wth dfferences n baselne by NCOV (Vckers & ltman, ). Fnally, there s an excellent on-lne textbook (StatSoft, ):

222 : Bblography Bblography belson, R. P. (995). Statstcs s Prncpled rgument, Lawrence Erlbaum, Hllsdale, New Jersey. ltman, D. G. & Bland, J. M. (995). bsence of evdence s not evdence of absence. Brtsh Medcal Journal 3: 485. ltman, D. G. & Bland, J. M. (996). Comparng several groups usng analyss of varance. Brtsh Medcal Journal 3: ltman, D. G. & Bland, J. M. (997). Statstcs notes. Unts of analyss. Brtsh Medcal Journal 34: 874. ltman, D. G. & Bland, J. M. (998). Generalsaton and extrapolaton. Brtsh Medcal Journal 37: ltman, D. G. & Bland, J. M. (999a). How to randomse. Brtsh Medcal Journal 39: ltman, D. G. & Bland, J. M. (999b). Statstcs notes. reatment allocaton n controlled trals: why randomse? Brtsh Medcal Journal 38: 9. ltman, D. G. & Bland, J. M. (3). Interacton revsted: the dfference between two estmates. Brtsh Medcal Journal 36: 9. ltman, D. G. & Matthews, J. N. (996). Statstcs notes. Interacton : Heterogenety of effects. Brtsh Medcal Journal 33: 486. ltman, D. G. & Schulz, K. F. (). Statstcs notes: Concealng treatment allocaton n randomsed trals. Brtsh Medcal Journal 33: Bland, J. M. & ltman, D. G. (986). Statstcal methods for assessng agreement between two methods of clncal measurement. Lancet : Bland, J. M. & ltman, D. G. (994a). Correlaton, regresson, and repeated data. Brtsh Medcal Journal 38: 896. Bland, J. M. & ltman, D. G. (994b). Matchng. Brtsh Medcal Journal 39: 8. Bland, J. M. & ltman, D. G. (994c). One and two sded tests of sgnfcance. Brtsh Medcal Journal 39: 48. Bland, J. M. & ltman, D. G. (994d). Regresson towards the mean. Brtsh Medcal Journal 38: 499. Bland, J. M. & ltman, D. G. (994e). Some examples of regresson towards the mean. Brtsh Medcal Journal 39: 78. Bland, J. M. & ltman, D. G. (995a). Calculatng correlaton coeffcents wth repeated observatons: Part --Correlaton wthn subjects. Brtsh Medcal Journal 3: 446. Bland, J. M. & ltman, D. G. (995b). Calculatng correlaton coeffcents wth repeated observatons: Part --Correlaton between subjects. Brtsh Medcal Journal 3: 633. Bland, J. M. & ltman, D. G. (995c). Multple sgnfcance tests: the Bonferron method. Brtsh Medcal Journal 3: 7. Bland, J. M. & ltman, D. G. (996a). ransformatons, means, and confdence ntervals. Brtsh Medcal Journal 3: 79. Bland, J. M. & ltman, D. G. (996b). ransformng data. Brtsh Medcal Journal 3: 77. Bland, J. M. & ltman, D. G. (996c). he use of transformaton when comparng two means. Brtsh Medcal Journal 3: 53. Box, G. E. P. (954). Some theorems on quadratc forms appled n the study of analyss of varance problems: II. Effect of nequalty of varance and of correlaton of errors n the two-way classfcaton. nnals of Mathematcal Statstcs 5: Boyd, O., Mackay, C. J., Lamb, G., Bland, J. M., Grounds, R. M. & Bennett, E. D. (993). Comparson of clncal nformaton ganed from routne blood-gas analyss and from gastrc tonometry for ntramural ph. Lancet 34: Cardnal, R. N., Parknson, J.., Djafar Marbn, H., oner,. J., Bussey,. J., Robbns,. W. & Evertt, B. J. (3). Role of the anteror cngulate cortex n the control over behavour by Pavlovan condtoned stmul n rats. Behavoral Neuroscence 7: Cohen, J. (988). Statstcal power analyss for the behavoral scences. Frst edton, cademc Press, New York. Day, S. J. & ltman, D. G. (). Statstcs notes: blndng n clncal trals and other studes. Brtsh Medcal Journal 3: 54. Feld,. P. (998). bluffer's gude to sphercty. Newsletter of the Mathematcal, Statstcal and computng secton of the Brtsh Psychologcal Socety 6: 3-. Frank, H. & lthoen, S. C. (994). Statstcs: Concepts and pplcatons, Cambrdge, Cambrdge Unversty Press. Greenhouse, S. W. & Gesser, S. (959). On methods n the analyss of profle data. Psychometrka 4: 95-. Howell, D. C. (997). Statstcal Methods for Psychology. Fourth edton, Wadsworth, Belmont, Calforna. Huynh, H. & Feldt, L. S. (97). Condtons under whch mean square ratos n repeated measures desgns have exact F-dstrbutons. Journal of the mercan Statstcal ssocaton 65: Keppel, G. (98). Desgn and analyss: a researcher's handbook. Second edton, Englewood Clffs: Prentce-Hall, London. Keppel, G. (99). Desgn and analyss: a researcher's handbook. hrd edton, Prentce-Hall, London. Levene, H. (96). Robust tests for the equalty of varance. In Contrbutons to probablty and statstcs (Okln, I., ed.). Stanford Unversty Press, Palo lto, Calforna. Lllefors, H. W. (967). On the Kolmogorov-Smrnov test for normalty wth mean and varance unknown. Journal of the mercan Statstcal ssocaton 6: Matthews, J. N. & ltman, D. G. (996a). Interacton 3: How to examne heterogenety. Brtsh Medcal Journal 33: 86. Matthews, J. N. & ltman, D. G. (996b). Statstcs notes. Interacton : Compare effect szes not P values. Brtsh Medcal Journal 33: 88. Mauchly, J. W. (94). Sgnfcance test for sphercty of a normal n- varate dstrbuton. nnals of Mathematcal Statstcs : 4-9. Myers, J. L. & Well,. D. (995). Research Desgn and Statstcal nalyss, Lawrence Erlbaum, Hllsdale, New Jersey. Prescott, C. E., Kabzems, R. & Zabek, L. M. (999). Effects of fertlzaton on decomposton rate of Populus tremulodes folar ltter n a boreal forest. Canadan Journal of Forest Research 9: Press, W. H., eukolsky, S.., Vetterlng, W.. & Flannery, B. P. (99). Numercal Recpes n C. Second edton, Cambrdge Unversty Press, Cambrdge, UK. Satterthwate, F. E. (946). n approxmate dstrbuton of estmates of varance components. Bometrcs Bulletn : 4. Shapro, S. S. & Wlk, M. B. (965). n analyss of varance test for normalty (complete samples). Bometrka 5: SPSS (). SPSS. Syntax Reference Gude (spssbase.pdf). StatSoft (). Electronc Statstcs extbook ( ulsa, OK. angren, J. (). Feld Gude o Expermental Desgns ( 4). Washngton State Unversty, ree Frut Research and Extenson Center. Vckers,. J. & ltman, D. G. (). Statstcs notes: nalysng controlled trals wth baselne and follow up measurements. Brtsh Medcal Journal 33: 3-4. Wner, B. J. (97). Statstcal prncples n expermental desgn. Second edton, McGraw-Hll, New York. Wner, B. J., Brown, D. R. & Mchels, K. M. (99). Statstcal Prncples n Expermental Desgn, McGraw-Hll, New York, NY.