Fixed vs. Random Effects

Transcription

1 Statistics 203: Introduction to Regression and Analysis of Variance Fixed vs. Random Effects Jonathan Taylor - p. 1/19

2 Today s class Implications for Random effects. One-way random effects ANOVA. Two-way mixed & random effects ANOVA. Sattherwaite s procedure. Mixed effects Two-way mixed effects - p. 2/19

3 Two-way ANOVA Implications for Mixed effects Two-way mixed effects Second generalization: more than one grouping variable. Two-way ANOVA : observations: (Y ijk ), 1 i r, 1 j m, 1 k n ij : r groups in first grouping variable, m groups ins second and n ij samples in (i, j)- cell : Y ijk = µ + α i + β j + (αβ) ij + ε ijk, ε ijk N(0, σ 2 ). Constraints: r i=1 α i = 0 m j=1 β j = 0 m j=1 (αβ) ij = 0, 1 i r r i=1 (αβ) ij = 0, 1 j m. - p. 3/19

4 Random vs. fixed effects Implications for Mixed effects Two-way mixed effects In ANOVA examples we have seen so far, the categorical variables are well-defined categories: below average fitness, long duration, etc. In some designs, the categorical variable is subject. Simplest example: repeated measures, where more than one (identical) measurement is taken on the same individual. In this case, the group effect α i is best thought of as random because we only sample a subset of the entire population of subjects. - p. 4/19

5 When to use random effects? Implications for Mixed effects Two-way mixed effects A group effect is random if we can think of the levels we observe in that group to be samples from a larger population. Example: if collecting data from different medical centers, center might be thought of as random. Example: if surveying students on different campuses, campus may be a random effect. - p. 5/19

6 Example: sodium content in Implications for Mixed effects Two-way mixed effects How much sodium is there in North American? How much does this vary by brand? Observations: for 6 brands of, researchers recorded the sodium content of 8 12 ounce bottles. Questions of interest: what is the grand mean sodium content? How much variability is there from brand to brand? Individuals in this case are brands, repeated measures are the 8 bottles. - p. 6/19

7 One-way random effects Implications for Mixed effects Two-way mixed effects Suppose we take n identical measurements from r subjects. Y ij µ + α i + ε ij, 1 i r, 1 j n ε ij N(0, σ 2 ), 1 i r, 1 j n α i N(0, σ 2 µ), 1 i r. We might be interested in the population mean, µ : CIs, is it zero? etc. Alternatively, we might be interested in the variability across subjects, σ 2 µ: CIs, is it zero? - p. 7/19

8 Implications for Implications for Mixed effects Two-way mixed effects In random effects, the observations are no longer independent (even if ε s are independent). In fact Cov(Y ij, Y i j ) = σ 2 µδ i,i + σ 2 δ j,j. In more complicated mixed effects s, this makes MLE more complicated: not only are there parameters in the mean, but in the covariance as well. In ordinary least squares regression, the only parameter to estimate is σ 2 because the covariance matrix is σ 2 I. - p. 8/19

9 One-way random ANOVA Implications for Mixed effects Two-way mixed effects Source SS df E(M S) Treatments SST R = P r 2 i=1 n Y i Y r 1 σ 2 + nσ µ 2 Error SSE = P r P nj=1 i=1 (Y ij Y i ) 2 (n 1)r σ 2 Only change here is the expectation of SST R which reflects randomness of α i s. ANOVA is still useful to setup tests: the same F statistics for fixed or random will work here. Under H 0 : σ 2 µ = 0, it is easy to see that MST R MSE F r 1,(n 1)r. - p. 9/19

10 Inference for µ Implications for Mixed effects Two-way mixed effects We know that E(Y ) = µ, and can show that Therefore, Var(Y ) = nσ2 µ + σ 2. rn Y µ SST R (r 1)rn t r 1 Why r 1 degrees of freedom? Imagine we could record an infinite number of observations for each individual, so that Y i µ i. To learn anything about µ we still only have r observations (µ 1,..., µ r ). Sampling more within an individual cannot narrow the CI for µ. - p. 10/19

11 Estimating σ 2 µ Implications for Mixed effects Two-way mixed effects From the ANOVA σ 2 µ = Natural estimate: E(SST R/(r 1)) E(SSE/((n 1)r)) n S 2 µ = SST R/(r 1) SSE/((n 1)r) n Problem: this estimate can be negative! One of the difficulties in random effects.. - p. 11/19

12 Example: productivity study Implications for Mixed effects Two-way mixed effects Imagine a study on the productivity of employees in a large manufacturing company. Company wants to get an idea of daily productivity, and how it depends on which machine an employee uses. Study: take m employees and r machines, having each employee work on each machine for a total of n days. As these employees are not all employees, and these machines are not all machines it makes sense to think of both the effects of machine and employees (and interactions) as random. - p. 12/19

13 Two-way random effects Implications for Mixed effects Two-way mixed effects Y ijk µ + α i + β j + (αβ) ij + ε ij, 1 i r, 1 j m, 1 k n ε ijk N(0, σ 2 ), 1 i r, 1 j m, 1 k n α i N(0, σ 2 α), 1 i r. β j N(0, σβ 2 ), 1 j m. (αβ) ij N(0, σαβ 2 ), 1 j m, 1 i r. Cov(Y ijk, Y i j k ) = δ ii σ 2 α +δ jj σ 2 β +δ ii δ jj σ 2 αβ +δ ii δ jj δ kk σ 2. - p. 13/19

14 ANOVA s: Two-way Implications for Mixed effects Two-way mixed effects SS df E(SS) SSA = nm P r 2 i=1 Y i Y r 1 σ 2 + nmσ α 2 + nσ2 αβ SSB = nr P m 2 j=1 Y j Y m 1 σ 2 + nrσ β 2 + nσ2 αβ SSAB = n P r P mj=1 2 i=1 Y ij Y i Y j + Y (m 1)(r 1) σ 2 + nσ αβ 2 SSE = P r P mj=1 P nk=1 i=1 (Y ijk Y ij ) 2 (n 1)ab σ 2 To test H 0 : σα 2 = 0 use SSA and SSAB. To test H 0 : σαβ 2 = 0 use SSAB and SSE. - p. 14/19

15 Mixed effects Implications for Mixed effects Two-way mixed effects In some studies, some factors can be thought of as fixed, others random. For instance, we might have a study of the effect of a standard part of the brewing process on sodium levels in the example. Then, we might think of a in which we have a fixed effect for brewing technique and a random effect for. - p. 15/19

16 Two-way mixed effects Implications for Mixed effects Two-way mixed effects Y ijk µ + α i + β j + (αβ) ij + ε ij, 1 i r, 1 j m, 1 k n ε ijk N(0, σ 2 ), 1 i r, 1 j m, 1 k n α i N(0, σ 2 α), 1 i r. β j, 1 j m are constants. (αβ) ij N(0, (m 1)σαβ 2 /m), 1 j m, 1 i r. Constraints: m j=1 β j = 0 r i=1 (αβ) ij = 0, 1 i r. Cov ((αβ) ij, (αβ) i j ) = σαβ 2 /m Cov(Y ( ijk, Y i j k ) = ) δ jj σβ 2 + δ ii m 1 m σ2 αβ (1 δ ii ) 1 m σ2 αβ + δ ii δ kk σ2 - p. 16/19

17 ANOVA s: Two-way Implications for Mixed effects Two-way mixed effects SS df E(M S) SSA r 1 σ 2 + nmσα 2 P mj=1 SSB m 1 σ 2 β i 2 + nr + nσ m 1 αβ 2 SSAB (m 1)(r 1) σ 2 + nσ αβ 2 SSE = P r P mj=1 P nk=1 i=1 (Y ijk Y ij ) 2 (n 1)ab σ 2 To test H 0 : σ 2 α = 0 use SSA and SSE. To test H 0 : β 1 = = β m = 0 use SSB and SSAB. To test H 0 : σ 2 αβ use SSAB and SSE. - p. 17/19

18 Confidence intervals for Implications for Mixed effects Two-way mixed effects Consider estimating σβ 2 in the two-way random effects ANOVA. A natural estimate is What about CI? σ 2 β = nr(msb MSAB). A linear combination of χ 2 but not χ 2. To form a confidence interval for σ β 2 we need to know distribution of a linear combination of MS s, at least approximately. - p. 18/19

19 Sattherwaite s procedure Implications for Mixed effects Two-way mixed effects Given k independent M S s Then where df T = L k c i MS i i=1 df T L E( L) χ 2 df T. ( k i=1 c ims i ) 2 k i=1 c2 i MS2 i /df i where df i are the degrees of freedom of the i-th MS. (1 α) 100% CI for E( L): L L = df T L χ 2 df T ;1 α/2, L U = df T L χ 2 df T ;α/2 - p. 19/19