Comparing Group Means: The T-test and One-way ANOVA Using STATA, SAS, and SPSS

003-005, The Trustees of Indiana University Comparing Group Means: 1 Comparing Group Means: The T-test and One-way ANOVA Using STATA, SAS, and SPSS Hun Myoung Park This document summarizes the method of comparing group means and illustrates how to conduct the t-test and one-way ANOVA using STATA 9.0, SAS 9.1, and SPSS 13.0. 1. Introduction. Univariate Samples 3. Paired (dependent) Samples 4. Independent Samples with Equal Variances 5. Independent Samples with Unequal Variances 6. One-way ANOVA, GLM, and Regression 7. Conclusion 1. Introduction The t-test and analysis of variance (ANOVA) compare group means. The mean of a variable to be compared should be substantively interpretable. A t-test may examine gender differences in average salary or racial (white versus black) differences in average annual income. The lefthand side (LHS) variable to be tested should be interval or ratio, whereas the right-hand side (RHS) variable should be binary (categorical). 1.1 T-test and ANOVA While the t-test is limited to comparing means of two groups, one-way ANOVA can compare more than two groups. Therefore, the t-test is considered a special case of one-way ANOVA. These analyses do not, however, necessarily imply any causality (i.e., a causal relationship between the left-hand and right-hand side variables). Table 1 compares the t-test and one-way ANOVA. Table 1. Comparison between the T-test and One-way ANOVA T-test One-way ANOVA LHS (Dependent) Interval or ratio variable Interval or ratio variable RHS (Independent) Binary variable with only two groups Categorical variable Null Hypothesis µ 1 = µ µ 1 = µ = µ 3 =... Prob. Distribution * T distribution F distribution * In the case of one degree of freedom on numerator, F=t. The t-test assumes that samples are randomly drawn from normally distributed populations with unknown population means. Otherwise, their means are no longer the best measures of central tendency and the t-test will not be valid. The Central Limit Theorem says, however, that

003-005, The Trustees of Indiana University Comparing Group Means: the distributions of y 1 and y are approximately normal when N is large. When n 1 + n 30, in practice, you do not need to worry too much about the normality assumption. You may numerically test the normality assumption using the Shapiro-Wilk W (N<=000), Shapiro-Francia W (N<=5000), Kolmogorov-Smirnov D (N>000), and Jarque-Bera tests. If N is small and the null hypothesis of normality is rejected, you my try such nonparametric methods as the Kolmogorov-Smirnov test, Kruscal-Wallis test, Wilcoxon Rank-Sum Test, or Log-Rank Test, depending on the circumstances. 1. T-test in SAS, STATA, and SPSS In STATA, the.ttest and.ttesti commands are used to conduct t-tests, whereas the.anova and.oneway commands perform one-way ANOVA. SAS has the TTEST procedure for t-test, but the UNIVARIATE, and MEANS procedures also have options for t- test. SAS provides various procedures for the analysis of variance, such as the ANOVA, GLM, and MIXED procedures. The ANOVA procedure can handle balanced data only, while the GLM and MIXED can analyze either balanced or unbalanced data (having the same or different numbers of observations across groups). However, unbalanced data does not cause any problems in the t-test and one-way ANOVA. In SPSS, T-TEST, ONEWAY, and UNIANOVA commands are used to perform t-test and one-way ANOVA. Table summarizes STATA commands, SAS procedures, and SPSS commands that are associated with t-test and one-way ANOVA. Table. Related Procedures and Commands in STATA, SAS, and SPSS STATA 9.0 SE SAS 9.1 SPSS 13.0 Normality Test.sktest;.swilk; UNIVARIATE EXAMINE.sfrancia Equal Variance.oneway TTEST T-TEST Nonparametric.ksmirnov;.kwallis NPAR1WAY NPAR TESTS T-test.ttest TTEST; MEANS T-TEST ANOVA.anova;.oneway ANOVA ONEWAY GLM * GLM; MIXED UNIANOVA * The STATA.glm command is not used for the T test, but for the generalized linear model. 1.3 Data Arrangement There are two types of data arrangement for t-tests (Figure 1). The first data arrangement has a variable to be tested and a grouping variable to classify groups (0 or 1). The second, appropriate especially for paired samples, has two variables to be tested. The two variables in this type are not, however, necessarily paired nor balanced. SAS and SPSS prefer the first data arrangement, whereas STATA can handle either type flexibly. Note that the numbers of observations across groups are not necessarily equal.

003-005, The Trustees of Indiana University Comparing Group Means: 3 Figure 1. Two Types of Data Arrangement Variable Group Variable1 Variable x x y y 0 0 1 1 The data set used here is adopted from J. F. Fraumeni s study on cigarette smoking and cancer (Fraumeni 1968). The data are per capita numbers of cigarettes sold by 43 states and the District of Columbia in 1960 together with death rates per hundred thousand people from various forms of cancer. Two variables were added to categorize states into two groups. See the appendix for the details. x x y y

003-005, The Trustees of Indiana University Comparing Group Means: 4. Univariate Samples The univariate-sample or one-sample t-test determines whether an unknown population mean µ differs from a hypothesized value c that is commonly set to zero: H 0 : µ = c. The t statistic y c follows Student s T probability distribution with n-1 degrees of freedom, t = ~ t( n 1), s y where y is a variable to be tested and n is the number of observations. 1 Suppose you want to test if the population mean of the death rates from lung cancer is 0 per 100,000 people at the.01 significance level. Note the default significance level used in most software is the.05 level..1 T-test in STATA The.ttest command conducts t-tests in an easy and flexible manner. For a univariate sample test, the command requires that a hypothesized value be explicitly specified. The level() option indicates the confidence level as a percentage. The 99 percent confidence level is equivalent to the.01 significance level.. ttest lung=0, level(99) One-sample t test Variable Obs Mean Std. Err. Std. Dev. [99% Conf. Interval] ---------+-------------------------------------------------------------------- lung 44 19.65318.6374133 4.81 17.9359 1.37108 mean = mean(lung) t = -0.5441 Ho: mean = 0 degrees of freedom = 43 Ha: mean < 0 Ha: mean!= 0 Ha: mean > 0 Pr(T < t) = 0.946 Pr( T > t ) = 0.589 Pr(T > t) = 0.7054 STATA first lists descriptive statistics of the variable lung. The mean and standard deviation of the 44 observations are 19.653 and 4.8, respectively. The t statistic is -.544 = (19.653-0) /.6374. Finally, the degrees of freedom are 43 =44-1. There are three t-tests at the bottom of the output above. The first and third are one-tailed tests, whereas the second is a two-tailed test. The t statistic -.544 and its large p-value do not reject the null hypothesis that the population mean of the death rate from lung cancer is 0 at the.01 level. The mean of the death rate may be 0 per 100,000 people. Note that the hypothesized value 0 falls into the 99 percent confidence interval 17.935-1.371. y 1 i ( ) y =, = yi y s s, and standard error s y =. n n 1 n The 99 percent confidence interval of the mean is y tα s = 19.653.695 *. 6374, where the.695 is ± y ± the critical value with 43 degree of freedom at the.01 level in the two-tailed test.

003-005, The Trustees of Indiana University Comparing Group Means: 5 If you just have the aggregate data (i.e., the number of observations, mean, and standard deviation of the sample), use the.ttesti command to replicate the t-test above. Note the hypothesized value is specified at the end of the summary statistics.. ttesti 44 19.65318 4.81 0, level(99). T-test Using the SAS TTEST Procedure The TTEST procedure conducts various types of t-tests in SAS. The H0 option specifies a hypothesized value, whereas the ALPHA indicates a significance level. If omitted, the default values zero and.05 respectively are assumed. PROC TTEST H0=0 ALPHA=.01 DATA=masil.smoking; VAR lung; RUN; The TTEST Procedure Statistics Lower CL Upper CL Lower CL Upper CL Variable N Mean Mean Mean Std Dev Std Dev Std Dev Std Err lung 44 17.935 19.653 1.371 3.994 4.81 5.7989 0.6374 T-Tests Variable DF t Value Pr > t lung 43-0.54 0.589 The TTEST procedure reports descriptive statistics followed by a one-tailed t-test. You may have a summary data set containing the values of a variable (lung) and their frequencies (count). The FREQ option of the TTEST procedure provides the solution for this case. PROC TTEST H0=0 ALPHA=.01 DATA=masil.smoking; VAR lung; FREQ count; RUN;.3 T-test Using the SAS UNIVARIATE and MEANS Procedures The SAS UNIVARIATE and MEANS procedures also conduct a t-test for a univariate-sample. The UNIVARIATE procedure is basically designed to produces a variety of descriptive statistics of a variable. Its MU0 option tells the procedure to perform a t-test using the hypothesized value specified. The VARDEF=DF specifies a divisor (degrees of freedom) used in

003-005, The Trustees of Indiana University Comparing Group Means: 6 computing the variance (standard deviation). 3 The NORMAL option examines if the variable is normally distributed. PROC UNIVARIATE MU0=0 VARDEF=DF NORMAL ALPHA=.01 DATA=masil.smoking; VAR lung; RUN; The UNIVARIATE Procedure Variable: lung Moments N 44 Sum Weights 44 Mean 19.6531818 Sum Observations 864.74 Std Deviation 4.81167 Variance 17.877019 Skewness -0.104796 Kurtosis -0.94960 Uncorrected SS 17763.604 Corrected SS 768.711555 Coeff Variation 1.5136751 Std Error Mean 0.63741333 Basic Statistical Measures Location Variability Mean 19.65318 Std Deviation 4.81 Median 0.3000 Variance 17.87701 Mode. Range 15.6000 Interquartile Range 6.53000 Tests for Location: Mu0=0 Test -Statistic- -----p Value------ Student's t t -0.5441 Pr > t 0.589 Sign M 1 Pr >= M 0.8804 Signed Rank S -36.5 Pr >= S 0.675 Tests for Normality Test --Statistic--- -----p Value------ Shapiro-Wilk W 0.967845 Pr < W 0.535 Kolmogorov-Smirnov D 0.086184 Pr > D >0.1500 Cramer-von Mises W-Sq 0.063737 Pr > W-Sq >0.500 Anderson-Darling A-Sq 0.38105 Pr > A-Sq >0.500 Quantiles (Definition 5) Quantile Estimate 100% Max 7.70 3 The VARDEF=N uses N as a divisor, while VARDEF=WDF specifies the sum of weights minus one.

003-005, The Trustees of Indiana University Comparing Group Means: 7 99% 7.70 95% 5.950 90% 5.450 75% Q3.815 50% Median 0.30 5% Q1 16.85 Quantiles (Definition 5) Quantile Estimate 10% 14.110 5% 1.10 1% 1.010 0% Min 1.010 Extreme Observations -----Lowest---- ----Highest---- Value Obs Value Obs 1.01 39 5.45 16 1.11 33 5.88 1 1.1 30 5.95 7 13.58 10 6.48 18 14.11 36 7.7 8 The third block of the output above reports a t statistic and its p-value. The fourth block contains several statistics of normality test. Since N is less than,000, you should read the Shapiro-Wilk W, which suggests that lung is normally distributed (p<.535) The MEANS procedure also conducts t-tests using the T and PROBT options that request the t statistic and its two-tailed p-value. The CLM option produces the two-tailed confidence interval (or upper and lower limits). The MEAN, STD, and STDERR respectively print the sample mean, standard deviation, and standard error. PROC MEANS MEAN STD STDERR T PROBT CLM VARDEF=DF ALPHA=.01 DATA=masil.smoking; VAR lung; RUN; The MEANS Procedure Analysis Variable : lung Lower 99% Upper 99% Mean Std Dev Std Error t Value Pr > t CL for Mean CL for Mean ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ 19.6531818 4.8117 0.6374133 30.83 <.0001 17.935878 1.3710758 ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

003-005, The Trustees of Indiana University Comparing Group Means: 8 The MEANS procedure does not, however, have an option to specify a hypothesized value to anything other than zero. Thus, the null hypothesis here is that the population mean of death rate from lung cancer is zero. The t statistic 30.83 is (19.653-0)/.6374. The large t statistic and small p-value reject the null hypothesis, reporting a consistent conclusion..4 T-test in SPSS The SPSS has the T-TEST command for t-tests. The /TESTVAL subcommand specifies the value with which the sample mean is compared, whereas the /VARIABLES list the variables to be tested. Like STATA, SPSS specifies a confidence level rather than a significance level in the /CRITERIA=CI() subcommand. T-TEST /TESTVAL = 0 /VARIABLES = lung /MISSING = ANALYSIS /CRITERIA = CI(.99).

003-005, The Trustees of Indiana University Comparing Group Means: 9 3. Paired (Dependent) Samples When two variables are not independent, but paired, the difference of these two variables, di = y1 i yi, is treated as if it were a single sample. This test is appropriate for pre-post treatment responses. The null hypothesis is that the true mean difference of the two variables is D 0, H : D 0 µ d = 0. 4 The difference is typically assumed to be zero unless explicitly specified. 3.1 T-test in STATA In order to conduct a paired sample t-test, you need to list two variables separated by an equal sign. The interpretation of the t-test remains almost unchanged. The -1.871 = (-10.1667-0)/5.4337 at 35 degrees of freedom does not reject the null hypothesis that the difference is zero.. ttest pre=post0, level(95) Paired t test Variable Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- pre 36 176.078 6.5973 39.17834 16.7717 189.838 post0 36 186.1944 7.86777 46.96066 170.305 0.0836 ---------+-------------------------------------------------------------------- diff 36-10.16667 5.433655 3.60193-1.19757.864387 mean(diff) = mean(pre post0) t = -1.8711 Ho: mean(diff) = 0 degrees of freedom = 35 Ha: mean(diff) < 0 Ha: mean(diff)!= 0 Ha: mean(diff) > 0 Pr(T < t) = 0.0349 Pr( T > t ) = 0.0697 Pr(T > t) = 0.9651 Alternatively, you may first compute the difference between the two variables, and then conduct one-sample t-test. Note that the default confidence level, level(95), can be omitted.. gen d=pre post0. ttest d=0 3. T-test in SAS In the TTEST procedure, you have to use the PAIRED instead of the VAR statement. For the output of the following procedure, refer to the end of this section. PROC TTEST DATA=temp.drug; PAIRED pre*post0; RUN; t d D 4 = 0 ~ t( n 1) d sd, where d n d =, i s d ( ) = di d n 1, and s d = sd n

003-005, The Trustees of Indiana University Comparing Group Means: 10 The PAIRED statement provides various ways of comparing variables using asterisk (*) and colon (:) operators. The asterisk requests comparisons between each variable on the left with each variable on the right. The colon requests comparisons between the first variable on the left and the first on the right, the second on the left and the second on the right, and so forth. Consider the following examples. PROC TTEST; PAIRED pro: post0; PAIRED (a b)*(c d); /* Equivalent to PAIRED a*c a*d b*c b*d; */ PAIRED (a b):(c d); /* Equivalent to PAIRED a*c b*c; */ PAIRED (a1-a10)*(b1-b10); RUN; The first PAIRED statement is the same as the PAIRED pre*post0. The second and the third PAIRED statements contrast differences between asterisk and colon operators. The hyphen ( ) operator in the last statement indicates a1 through a10 and b1 through b10. Let us consider an example of the PAIRED statement. PROC TTEST DATA=temp.drug; PAIRED (pre)*(post0-post1); RUN; The TTEST Procedure Statistics Lower CL Upper CL Lower CL Upper CL Difference N Mean Mean Mean Std Dev Std Dev Std Dev Std Err pre - post0 36-1. -10.17 0.864 6.443 3.60 4.57 5.4337 pre - post1 36-30.43-0.39-10.34 4.077 9.685 38.73 4.9475 T-Tests Difference DF t Value Pr > t pre - post0 35-1.87 0.0697 pre - post1 35-4.1 0.000 The first t statistic for pre versus post0 is identical to that of the previous section. The second for pre versus post1 rejects the null hypothesis of no mean difference at the.01 level (p<.000). In order to use the UNIVARIATE and MEANS procedures, the difference between two paired variables should be computed in advance. DATA temp.drug; SET temp.drug; d1 = pre - post0; d = pre - post1; RUN;

003-005, The Trustees of Indiana University Comparing Group Means: 11 PROC UNIVARIATE MU0=0 VARDEF=DF NORMAL; VAR d1 d; RUN; PROC MEANS MEAN STD STDERR T PROBT CLM; VAR d1 d; RUN; PROC TTEST ALPHA=.05; VAR d1 d; RUN; 3.3 T-test in SPSS In SPSS, the PAIRS subcommand indicates a paired sample t-test. T-TEST PAIRS = pre post0 /CRITERIA = CI(.95) /MISSING = ANALYSIS.

003-005, The Trustees of Indiana University Comparing Group Means: 1 4. Independent Samples with Equal Variances You should check three assumptions first when testing the mean difference of two independent samples. First, the samples are drawn from normally distributed populations with unknown parameters. Second, the two samples are independent in the sense that they are drawn from different populations and/or the elements of one sample are not related to those of the other sample. Finally, the population variances of the two groups, σ 1 and σ are equal. 5 If any one of assumption is violated, the t-test is not valid. An example here is to compare mean death rates from lung cancer between smokers and nonsmokers. Let us begin with discussing the equal variance assumption. 4.1 F test for Equal Variances The folded form F test is widely used to examine whether two populations have the same sl variance. The statistic is ~ F( n 1, 1) L ns, where L and S respectively indicate groups ss with larger and smaller sample variances. Unless the null hypothesis of equal variances is rejected, the pooled variance estimate s pool is used. The null hypothesis of the independent sample t-test is H : µ µ = D. 0 1 0 ( y1 y ) D0 t = ~ t( n1 + n 1 1 s pool + n1 n ( ) ( y1 i y1 + y s = n + n ), where y j ( n1 1) s1 + ( n 1) s pool =. 1 n1 + n ) When the assumption is violated, the t-test requires the approximations of the degree of freedom. The null hypothesis and other components of the t-test, however, remain unchanged. Satterthwaite s approximation for the degree of freedom is commonly used. Note that the approximation is a real number, not an integer. y1 y D0 t' = ~ t( df Satterthwaite ), where s1 s + n n df 1 ( n 1)( n 1) 1 Satterthwaite = and ( n1 1)(1 c) + ( n 1) c c = s 1 s1 n n + s 1 1 n 5 1 E ( x1 x ) = µ 1 µ, 1 Var( x = + = + 1 x ) σ n1 n n1 n σ σ 1

003-005, The Trustees of Indiana University Comparing Group Means: 13 The SAS TTEST procedure and SPSS T-TEST command conduct F tests for equal variance. SAS reports the folded form F statistic, whereas SPSS computes Levene's weighted F statistic. In STATA, the.oneway command produces Bartlett s statistic for the equal variance test. The following is an example of Bartlett's test that does not reject the null hypothesis of equal variance.. oneway lung smoke Analysis of Variance Source SS df MS F Prob > F ------------------------------------------------------------------------ Between groups 313.03117 1 313.03117 8.85 0.0000 Within groups 455.68047 4 10.849534 ------------------------------------------------------------------------ Total 768.711555 43 17.877019 Bartlett's test for equal variances: chi(1) = 0.116 Prob>chi = 0.77 STATA, SAS, and SPSS all compute Satterthwaite s approximation of the degrees of freedom. In addition, the SAS TTEST procedure reports Cochran-Cox approximation and the STATA.ttest command provides Welch s degrees of freedom. 4. T-test in STATA With the.ttest command, you have to specify a grouping variable smoke in this example in the parenthesis of the by option.. ttest lung, by(smoke) level(95) Two-sample t test with equal variances Group Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- 0 16.98591.6747158 3.164698 15.5876 18.38906 1.3045.78753 3.418151 0.80493 3.83598 ---------+-------------------------------------------------------------------- combined 44 19.65318.6374133 4.81 18.3677 0.93865 ---------+-------------------------------------------------------------------- diff -5.334545.9931371-7.338777-3.330314 diff = mean(0) - mean(1) t = -5.3714 Ho: diff = 0 degrees of freedom = 4 Ha: diff < 0 Ha: diff!= 0 Ha: diff > 0 Pr(T < t) = 0.0000 Pr( T > t ) = 0.0000 Pr(T > t) = 1.0000 s 3.418 Let us first check the equal variance. The F statistic is 1.17 = L = ~ F(1,1). The ss 3.1647 degrees of freedom of the numerator and denominator are 1 (=-1). The p-value of.773, virtually the same as that of Bartlett s test above, does not reject the null hypothesis of equal variance. Thus, the t-test here is valid (t=-5.3714 and p<.0000).

003-005, The Trustees of Indiana University Comparing Group Means: 14 (16.9859.305) 0 t = = 5.3714 ~ t( + ), where 1 1 s pool + ( 1)3.1647 + ( 1)3.418 s pool = = 10.8497 + If only aggregate data of the two variables are available, use the.ttesti command and list the number of observations, mean, and standard deviation of the two variables.. ttesti 16.85591 3.164698.3045 3.418151, level(95) Suppose a data set is differently arranged (second type in Figure 1) so that one variable smk_lung has data for smokers and the other non_lung for non-smokers. You have to use the unpaired option to indicate that two variables are not paired. A grouping variable here is not necessary. Compare the following output with what is printed above.. ttest smk_lung=non_lung, unpaired Two-sample t test with equal variances Variable Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- smk_lung.3045.78753 3.418151 0.80493 3.83598 non_lung 16.98591.6747158 3.164698 15.5876 18.38906 ---------+-------------------------------------------------------------------- combined 44 19.65318.6374133 4.81 18.3677 0.93865 ---------+-------------------------------------------------------------------- diff 5.334545.9931371 3.330313 7.338777 diff = mean(smk_lung) - mean(non_lung) t = 5.3714 Ho: diff = 0 degrees of freedom = 4 Ha: diff < 0 Ha: diff!= 0 Ha: diff > 0 Pr(T < t) = 1.0000 Pr( T > t ) = 0.0000 Pr(T > t) = 0.0000 This unpaired option is very useful since it enables you to conduct a t-test without additional data manipulation. You may run the.ttest command with the unpaired option to compare two variables, say leukemia and kidney, as independent samples in STATA. In SAS and SPSS, however, you have to stack up two variables and generate a grouping variable before t- tests.. ttest leukemia=kidney, unpaired Two-sample t test with equal variances Variable Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- leukemia 44 6.89773.09611.638589 6.63574 7.0381 kidney 44.794545.07854.5190799.636731.9536 ---------+-------------------------------------------------------------------- combined 88 4.81159.4961.109994 4.365094 5.594 ---------+-------------------------------------------------------------------- diff 4.0357.14051 3.788673 4.81781

003-005, The Trustees of Indiana University Comparing Group Means: 15 diff = mean(leukemia) - mean(kidney) t = 3.5356 Ho: diff = 0 degrees of freedom = 86 Ha: diff < 0 Ha: diff!= 0 Ha: diff > 0 Pr(T < t) = 1.0000 Pr( T > t ) = 0.0000 Pr(T > t) = 0.0000 The F 1.5119 = (.653589^)/(.5190799^) and its p-value (=.1797) do not reject the null hypothesis of equal variance. The large t statistic 3.5356 rejects the null hypothesis that death rates from leukemia and kidney cancers have the same mean. 4.3 T-test in SAS The TTEST procedure by default examines the hypothesis of equal variances, and provides T statistics for either case. The procedure by default reports Satterthwaite s approximation for the degrees of freedom. Keep in mind that a variable to be tested is grouped by the variable that is specified in the CLASS statement. PROC TTEST H0=0 ALPHA=.05 DATA=masil.smoking; CLASS smoke; VAR lung; RUN; The TTEST Procedure Statistics Lower CL Upper CL Lower CL Upper CL Variable smoke N Mean Mean Mean Std Dev Std Dev Std Dev lung 0 15.583 16.986 18.389.4348 3.1647 4.56 lung 1 0.805.3 3.836.698 3.418 4.8848 lung Diff (1-) -7.339-5.335-3.33.7159 3.939 4.1865 Statistics Variable smoke Std Err Minimum Maximum lung 0 0.6747 1.01 5.45 lung 1 0.788 1.11 7.7 lung Diff (1-) 0.9931 T-Tests Variable Method Variances DF t Value Pr > t lung Pooled Equal 4-5.37 <.0001 lung Satterthwaite Unequal 41.8-5.37 <.0001 Equality of Variances Variable Method Num DF Den DF F Value Pr > F

003-005, The Trustees of Indiana University Comparing Group Means: 16 lung Folded F 1 1 1.17 0.773 The F test for equal variance does not reject the null hypothesis of equal variances. Thus, the t- test labeled as Pooled should be referred to in order to get the t -5.37 and its p-value.0001. If the equal variance assumption is violated, the statistics of Satterthwaite and Cochran should be read. If you have a summary data set with the values of variables (lung) and their frequency (count), specify the count variable in the FREQ statement. PROC TTEST DATA=masil.smoking; CLASS smoke; VAR lung; FREQ count; RUN; Now, let us compare the death rates from leukemia and kidney in the second data arrangement type of Figure 1. As mentioned before, you need to rearrange the data set to stack up two variables into one and generate a grouping variable (first type in Figure 1). DATA masil.smoking; SET masil.smoking; death = leukemia; leu_kid ='Leukemia'; OUTPUT; death = kidney; leu_kid ='Kidney'; OUTPUT; KEEP leu_kid death; RUN; PROC TTEST COCHRAN DATA=masil.smoking; CLASS leu_kid; VAR death; RUN; The TTEST Procedure Statistics Lower CL Upper CL Lower CL Upper CL Variable leu_kid N Mean Mean Mean Std Dev Std Dev Std Dev Std Err death Kidney 44.6367.7945.954 0.489 0.5191 0.6577 0.0783 death Leukemia 44 6.6357 6.898 7.038 0.573 0.6383 0.8087 0.096 death Diff (1-) -4.8-4.035-3.789 0.5063 0.5817 0.6838 0.14 T-Tests Variable Method Variances DF t Value Pr > t death Pooled Equal 86-3.54 <.0001 death Satterthwaite Unequal 8.6-3.54 <.0001 death Cochran Unequal 43-3.54 <.0001 Equality of Variances Variable Method Num DF Den DF F Value Pr > F

003-005, The Trustees of Indiana University Comparing Group Means: 17 death Folded F 43 43 1.51 0.1794 Compare this SAS output with that of STATA in the previous section. 4.4 T-test in SPSS In the T-TEST command, you need to use the /GROUP subcommand in order to specify a grouping variable. SPSS reports Levene's F.0000 that does not reject the null hypothesis of equal variance (p<.995). T-TEST GROUPS = smoke(0 1) /VARIABLES = lung /MISSING = ANALYSIS /CRITERIA = CI(.95).

003-005, The Trustees of Indiana University Comparing Group Means: 18 5. Independent Samples with Unequal Variances If the assumption of equal variances is violated, we have to compute the adjusted t statistic using individual sample standard deviations rather than a pooled standard deviation. It is also necessary to use the Satterthwaite, Cochran-Cox (SAS), or Welch (STATA) approximations of the degrees of freedom. In this chapter, you compare mean death rates from kidney cancer between the west (south) and east (north). 5.1 T-test in STATA As discussed earlier, let us check equality of variances using the.oneway command. The tabulate option produces a table of summary statistics for the groups.. oneway kidney west, tabulate Summary of kidney west Mean Std. Dev. Freq. ------------+------------------------------------ 0 3.006.300198 0 1.6183333.5983719 4 ------------+------------------------------------ Total.7945455.51907993 44 Analysis of Variance Source SS df MS F Prob > F ------------------------------------------------------------------------ Between groups 1.63947758 1 1.63947758 6.9 0.0118 Within groups 9.94661333 4.368417 ------------------------------------------------------------------------ Total 11.5860909 43.69443975 Bartlett's test for equal variances: chi(1) = 8.6506 Prob>chi = 0.003 Bartlett s chi-squared statistic rejects the null hypothesis of equal variance at the.01 level. It is appropriate to use the unequal option in the.ttest command, which calculates Satterthwaite s approximation for the degrees of freedom. Unlike the SAS TTEST procedure, the.ttest command cannot specify the mean difference D 0 other than zero. Thus, the null hypothesis is that the mean difference is zero.. ttest kidney, by(west) unequal level(95) Two-sample t test with unequal variances Group Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- 0 0 3.006.0671111.300198.865535 3.146465 1 4.618333.114.59837.365663.871004 ---------+-------------------------------------------------------------------- combined 44.794545.07854.5190799.636731.9536 ---------+-------------------------------------------------------------------- diff.3876667.139365.10477.6705611

003-005, The Trustees of Indiana University Comparing Group Means: 19 diff = mean(0) - mean(1) t =.7817 Ho: diff = 0 Satterthwaite's degrees of freedom = 35.1098 Ha: diff < 0 Ha: diff!= 0 Ha: diff > 0 Pr(T < t) = 0.9957 Pr( T > t ) = 0.0086 Pr(T > t) = 0.0043 See Satterthwaite s approximation of 35.110 in the middle of the output. If you want to get Welch s approximation, use the welch as well as unequal options; without the unequal option, the welch is ignored.. ttest kidney, by(west) unequal welch Two-sample t test with unequal variances Group Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- 0 0 3.006.0671111.300198.865535 3.146465 1 4.618333.114.59837.365663.871004 ---------+-------------------------------------------------------------------- combined 44.794545.07854.5190799.636731.9536 ---------+-------------------------------------------------------------------- diff.3876667.139365.105084.670509 diff = mean(0) - mean(1) t =.7817 Ho: diff = 0 Welch's degrees of freedom = 36.58 Ha: diff < 0 Ha: diff!= 0 Ha: diff > 0 Pr(T < t) = 0.9957 Pr( T > t ) = 0.0085 Pr(T > t) = 0.0043 Satterthwaite s approximation is slightly smaller than Welch s 36.58. Again, keep in mind that these approximations are not integers, but real numbers. The t statistic.7817 and its p- value.0086 reject the null hypothesis of equal population means. The north and east have larger death rates from kidney cancer per 100 thousand people than the south and west. For aggregate data, use the.ttesti command with the necessary options.. ttesti 0 3.006.300198 4.618333.59837, unequal welch As mentioned earlier, the unpaired option of the.ttest command directly compares two variables without data manipulation. The option treats the two variables as independent of each other. The following is an example of the unpaired and unequal options.. ttest bladder=kidney, unpaired unequal welch Two-sample t test with unequal variances Variable Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- bladder 44 4.11136.1454679.964949 3.8777 4.4145 kidney 44.794545.07854.5190799.636731.9536 ---------+-------------------------------------------------------------------- combined 88 3.457841.108668 1.019009 3.41933 3.673748 ---------+-------------------------------------------------------------------- diff 1.36591.1651806.9968919 1.6569 diff = mean(bladder) - mean(kidney) t = 8.031 Ho: diff = 0 Welch's degrees of freedom = 67.034

003-005, The Trustees of Indiana University Comparing Group Means: 0 Ha: diff < 0 Ha: diff!= 0 Ha: diff > 0 Pr(T < t) = 1.0000 Pr( T > t ) = 0.0000 Pr(T > t) = 0.0000 The F 3.4556 = (.964949^)/(.5190799^) rejects the null hypothesis of equal variance (p<0001). If the welch option is omitted, Satterthwaite's degree of freedom 65.9643 will be produced instead. For aggregate data, again, use the.ttesti command without the unpaired option.. ttesti 44 4.11136.964949 44.794545.5190799, unequal welch level(95) 5. T-test in SAS The TTEST procedure reports statistics for cases of both equal and unequal variance. You may add the COCHRAN option to compute Cochran-Cox approximations for the degree of freedom. PROC TTEST COCHRAN DATA=masil.smoking; CLASS west; VAR kidney; RUN; The TTEST Procedure Statistics Lower CL Upper CL Lower CL Upper CL Variable s_west N Mean Mean Mean Std Dev Std Dev Std Dev kidney 0 0.8655 3.006 3.1465 0.8 0.3001 0.4384 kidney 1 4.3657.6183.871 0.4651 0.5984 0.8394 kidney Diff (1-) 0.0903 0.3877 0.685 0.4013 0.4866 0.6185 Statistics Variable west Std Err Minimum Maximum kidney 0 0.0671.34 3.6 kidney 1 0.11 1.59 4.3 kidney Diff (1-) 0.1473 T-Tests Variable Method Variances DF t Value Pr > t kidney Pooled Equal 4.63 0.0118 kidney Satterthwaite Unequal 35.1.78 0.0086 kidney Cochran Unequal..78 0.0109 Equality of Variances Variable Method Num DF Den DF F Value Pr > F kidney Folded F 3 19 3.97 0.0034

003-005, The Trustees of Indiana University Comparing Group Means: 1 F 3.9749 = (.59837^)/(.300198^) and p <.0034 reject the null hypothesis of equal variances. Thus, individual sample standard deviations need to be used to compute the adjusted t, and either Satterthwaite s or the Cochran-Cox approximation should be used in computing the p-value. See the following computations. t ' = 3.006.6183 =.78187,.3001.5984 + 0 4 c = s df 1 Satterthwa s n1.3001 0 = n1 + s n.3001 0 +.5984 ( n 1)( n 1) ite = ( n 1)(1 c) + ( n 1) c 1 = 1.318, and 4 (0 1)(4 1) = (0 1)(1.318) + (4 1).318 1 = 35.1071 The t statistic.78 rejects the null hypothesis of no difference in mean death rates between the two regions (p<.0086). Now, let us compare death rates from bladder and kidney cancers using SAS. DATA masil.smoking3; SET masil.smoking; death = bladder; bla_kid ='Bladder'; OUTPUT; death = kidney; bla_kid ='Kidney'; OUTPUT; KEEP bla_kid death; RUN; PROC TTEST COCHRAN DATA=masil.smoking3; CLASS bla_kid; VAR death; RUN; The TTEST Procedure Statistics Lower CL Upper CL Lower CL Upper CL Variable bla_kid N Mean Mean Mean Std Dev Std Dev Std Dev Std Err death Bladder 44 3.878 4.111 4.4145 0.797 0.9649 1.6 0.1455 death Kidney 44.6367.7945.954 0.489 0.5191 0.6577 0.0783 death Diff (1-) 0.998 1.366 1.655 0.6743 0.7748 0.9107 0.165 T-Tests Variable Method Variances DF t Value Pr > t death Pooled Equal 86 8.03 <.0001 death Satterthwaite Unequal 66 8.03 <.0001 death Cochran Unequal 43 8.03 <.0001 Equality of Variances

003-005, The Trustees of Indiana University Comparing Group Means: Variable Method Num DF Den DF F Value Pr > F death Folded F 43 43 3.46 <.0001 Fortunately, the t-tests under equal and unequal variance in this case lead the same conclusion at the.01 level; that is, the means of the two death rates are not the same. 5.3 T-test in SPSS Like SAS, SPSS also reports t statistics for cases of both equal and unequal variance. Note that Levene's F 5.466 rejects the null hypothesis of equal variance at the.05 level (p<.04). T-TEST GROUPS = west(0 1) /VARIABLES = kidney /MISSING = ANALYSIS /CRITERIA = CI(.95).

003-005, The Trustees of Indiana University Comparing Group Means: 3 6. One-way ANOVA, GLM, and Regression The t-test is a special case of one-way ANOVA. Thus, one-way ANOVA produces equivalent results to those of the t-test. ANOVA examines mean differences using the F statistic, whereas the t-test reports the t statistic. The one-way ANOVA (t-test), GLM, and linear regression present essentially the same things in different ways. 6.1 One-way ANOVA Consider the following ANOVA procedure. The CLASS statement is used to specify categorical variables. The MODEL statement lists the variable to be compared and a grouping variable, separating them with an equal sign. PROC ANOVA DATA=masil.smoking; CLASS smoke; MODEL lung=smoke; RUN; The ANOVA Procedure Dependent Variable: lung Sum of Source DF Squares Mean Square F Value Pr > F Model 1 313.031173 313.031173 8.85 <.0001 Error 4 455.680473 10.8495340 Corrected Total 43 768.7115545 R-Square Coeff Var Root MSE lung Mean 0.40715 16.75995 3.93863 19.65318 Source DF Anova SS Mean Square F Value Pr > F smoke 1 313.031173 313.031173 8.85 <.0001 STATA.anova and.oneway commands also conduct one-way ANOVA.. anova lung smoke Number of obs = 44 R-squared = 0.407 Root MSE = 3.9386 Adj R-squared = 0.3931 Source Partial SS df MS F Prob > F -----------+---------------------------------------------------- Model 313.03117 1 313.03117 8.85 0.0000 smoke 313.03117 1 313.03117 8.85 0.0000 Residual 455.68047 4 10.849534 -----------+---------------------------------------------------- Total 768.711555 43 17.877019

003-005, The Trustees of Indiana University Comparing Group Means: 4 In SPSS, the ONEWAY command is used. ONEWAY lung BY smoke /MISSING ANALYSIS. 6. Generalized Linear Model (GLM) The SAS GLM and MIXED procedures and the SPSS UNIANOVA command also report the F statistic for one-way ANOVA. Note that STATA s.glm command does not perform one-way ANOVA. PROC GLM DATA=masil.smoking; CLASS smoke; MODEL lung=smoke /SS3; RUN; The GLM Procedure Dependent Variable: lung Sum of Source DF Squares Mean Square F Value Pr > F Model 1 313.031173 313.031173 8.85 <.0001 Error 4 455.680473 10.8495340 Corrected Total 43 768.7115545 R-Square Coeff Var Root MSE lung Mean 0.40715 16.75995 3.93863 19.65318 Source DF Type III SS Mean Square F Value Pr > F smoke 1 313.031173 313.031173 8.85 <.0001 The MIXED procedure has the similar usage as the GLM procedure. The output here is skipped. PROC MIXED; CLASS smoke; MODEL lung=smoke; RUN; In SPSS, the UNIANOVA command estimates univariate ANOVA models using the GLM method. UNIANOVA lung BY smoke /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /CRITERIA = ALPHA(.05) /DESIGN = smoke. 6.3 Regression

003-005, The Trustees of Indiana University Comparing Group Means: 5 The SAS REG procedure, STATA.regress command, and SPSS REGRESSION command estimate linear regression models. PROC REG DATA=masil.smoking; MODEL lung=smoke; RUN; The REG Procedure Model: MODEL1 Dependent Variable: lung Number of Observations Read 44 Number of Observations Used 44 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 1 313.03113 313.03113 8.85 <.0001 Error 4 455.68043 10.84953 Corrected Total 43 768.71155 Root MSE 3.9386 R-Square 0.407 Dependent Mean 19.65318 Adj R-Sq 0.3931 Coeff Var 16.75995 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept 1 16.98591 0.705 4.19 <.0001 smoke 1 5.33455 0.99314 5.37 <.0001 Look at the results above. The coefficient of the intercept 16.9859 is the mean of the first group (smoke=0). The coefficient of smoke is, in fact, mean difference between two groups with its sign reversed (5.33455=16.9859-.305). Finally, the standard error of the coefficient is the 1 1 1 1 denominator of the independent sample t-test,.99314= s pool + = 3.939 +, n 1 n where the pooled variance estimate 10.8497=3.939^ (see page 11 and 13). Thus, the t 5.37 is identical to the t statistic of the independent sample t-test with equal variance. The STATA.regress command is quite simple. Note that a dependent variable precedes a list of independent variables.. regress lung smoke Source SS df MS Number of obs = 44 -------------+------------------------------ F( 1, 4) = 8.85

003-005, The Trustees of Indiana University Comparing Group Means: 6 Model 313.03117 1 313.03117 Prob > F = 0.0000 Residual 455.68047 4 10.849534 R-squared = 0.407 -------------+------------------------------ Adj R-squared = 0.3931 Total 768.711555 43 17.877019 Root MSE = 3.939 lung Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- smoke 5.334545.9931371 5.37 0.000 3.330314 7.338777 _cons 16.98591.7054 4.19 0.000 15.5687 18.40311 The SPSS REGRESSION command looks complicated compared to the SAS REG procedure and STATA.regress command. REGRESSION /MISSING LISTWISE /STATISTICS COEFF OUTS R ANOVA /CRITERIA=PIN(.05) POUT(.10) /NOORIGIN /DEPENDENT lung /METHOD=ENTER smoke. Note that ANOVA, GLM, and regression report the same F (1, 4) 8.85, which is equivalent to t (4) -5.3714. As long as the degrees of freedom of the numerator is 1, F is always t^ (8.85=-5.3714^).

003-005, The Trustees of Indiana University Comparing Group Means: 7 7. Conclusion The t-test is a basic statistical method for examining the mean difference between two groups. One-way ANOVA can compare means of more than two groups. The number of observations in individual groups does not matter in the t-test or one-way ANOVA; both balanced and unbalanced data are fine. One-way ANOVA, GLM, and linear regression models all use the variance-covariance structure in their analysis, but present the results in different ways. Researchers must check four issues when performing t-tests. First, a variable to be tested should be interval or ratio so that its mean is substantively meaningful. Do not, for example, run a t-test to compare the mean of skin colors (white=0, yellow=1, black=) between two countries. If you have a latent variable measured by several Likert-scaled manifest variables, first run a factor analysis to get that latent variable. Second, examine the normality assumptions before conducting a t-test. It is awkward to compare means of variables that are not normally distributed. Figure illustrates a normal probability distribution on top and a Poisson distribution skewed to the right on the bottom. Although the two distributions have the same mean and variance of 1, they are not likely to be substantively interpretable. This is the rationale to conduct normality test such as Shapiro-Wilk W, Shapiro-Francia W, and Kolmogorov-Smirnov D statistics. If the normality assumption is violated, try to use nonparametric methods. Figure. Comparing Normal and Poisson Probability Distributions ( σ =1 and µ =1)

003-005, The Trustees of Indiana University Comparing Group Means: 8 Third, check the equal variance assumption. You should be careful when comparing means of normally distributed variables with different variances. You may conduct the folded form F test. If the equal variance assumption is violated, compute the adjusted t and approximations of the degree of freedom. Finally, consider the types of t-tests, data arrangement, and functionalities available in each statistical software (e.g., STATA, SAS, and SPSS) to determine the best strategy for data analysis (Table 3). The first data arrangement in Figure 1 is commonly used for independent sample t-tests, whereas the second arrangement is appropriate for a paired sample test. Keep in mind that the type II data sets in Figure 1 needs to be reshaped into type I in SAS and SPSS. Table 3. Comparison of T-test Functionalities of STATA, SAS and SPSS STATA 9.0 SAS 9.1 SPSS 13.0 Test for equal variance Bartlett s chi-squared (.ttest command) Folded form F (TTEST procedure) Levene s weighted F (T-TEST command) Approximation of the Satterthwaite s DF Satterthwaite s DF Satterthwaite s DF degrees of freedom (DF) Welch s DF Cochran-Cox DF Second Data Arrangement var1=var Reshaping the data set Reshaping the data set Aggregate Data.ttesti command FREQ option N/A SAS has several procedures (e.g., TTEST, MEANS, and UNIVARIATE) and useful options for t-tests. The STATA.ttest and.ttesti commands provide very flexible ways of handling different data arrangements and aggregate data. Table 4 summarizes usages of options in these two commands. Table 4. Summary of the Usages of the.ttest and.ttest Command Options Usage by(group var) unequal welch unpaired * Univariate sample var=c Paired (dependent) sample var1=var Equal variance (1 variable) Var O Equal variance ( variables) ** var1=var O Unequal variance (1 variable) Var O O O Unequal variance ( variables) var1=var O O O * The.ttesti command does not allow the unpaired option. ** The var1=var assumes second type of data arrangement in Figure 1.

003-005, The Trustees of Indiana University Comparing Group Means: 9 Appendix: Data Set Literature: Fraumeni, J. F. 1968. "Cigarette Smoking and Cancers of the Urinary Tract: Geographic Variations in the United States," Journal of the National Cancer Institute, 41(5): 105-111. Data Source: http://lib.stat.cmu.edu The data are per capita numbers of cigarettes smoked (sold) by 43 states and the District of Columbia in 1960 together with death rates per 100 thousand people from various forms of cancer. The variables used in this document are, cigar = number of cigarettes smoked (hds per capita) bladder = deaths per 100k people from bladder cancer lung = deaths per 100k people from lung cancer kidney = deaths per 100k people from kidney cancer leukemia = deaths per 100k people from leukemia smoke = 1 for those whose cigarette consumption is larger than the median and 0 otherwise. west = 1 for states in the South or West and 0 for those in the North, East or Midwest. The followings are summary statistics and normality tests of these variables.. sum cigar-leukemia Variable Obs Mean Std. Dev. Min Max -------------+----------------------------------------------------- cigar 44 4.91409 5.57386 14 4.4 bladder 44 4.11136.964949.86 6.54 lung 44 19.65318 4.81 1.01 7.7 kidney 44.794545.5190799 1.59 4.3 leukemia 44 6.89773.638589 4.9 8.8. sfrancia cigar-leukemia Shapiro-Francia W' test for normal data Variable Obs W' V' z Prob>z -------------+------------------------------------------------- cigar 44 0.93061 3.58.03 0.01381 bladder 44 0.9451.577 1.776 0.03789 lung 44 0.97809 1.09 0.055 0.4783 kidney 44 0.9773 1.065 0.10 0.4517 leukemia 44 0.9769 1.8 0.474 0.31759. tab west smoke smoke west 0 1 Total -----------+----------------------+---------- 0 7 13 0 1 15 9 4 -----------+----------------------+---------- Total 44

003-005, The Trustees of Indiana University Comparing Group Means: 30 References Fraumeni, J. F. 1968. "Cigarette Smoking and Cancers of the Urinary Tract: Geographic Variations in the United States," Journal of the National Cancer Institute, 41(5): 105-111. Ott, R. Lyman. 1993. An Introduction to Statistical Methods and Data Analysis. Belmont, CA: Duxbury Press. SAS Institute. 005. SAS/STAT User's Guide, Version 9.1. Cary, NC: SAS Institute. SPSS. 001. SPSS 11.0 Syntax Reference Guide. Chicago, IL: SPSS Inc. STATA Press. 005. STATA Reference Manual Release 9. College Station, TX: STATA Press. Walker, Glenn A. 00. Common Statistical Methods for Clinical Research with SAS Examples. Cary, NC: SAS Institute. Acknowledgements I am grateful to Jeremy Albright, Takuya Noguchi, and Kevin Wilhite at the UITS Center for Statistical and Mathematical Computing, Indiana University, who provided valuable comments and suggestions. Revision History 003. First draft 004. Second draft 005. Third draft (Added data arrangements and conclusion).

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 1 Regression Models for Event Count Data Using SAS, STATA, and LIMDEP Hun Myoung Park This document summarizes regression models for event count data and illustrates how to estimate individual models using SAS, STATA, and LIMDEP. Example models were tested in SAS 9.1, STATA 9.0, and LIMDEP 8.0. 1. Introduction. The Poisson Regression Model (PRM) 3. The Negative Binomial Regression Model (NBRM) 4. The Zero-Inflated Poisson Regression Model (ZIP) 5. The Zero-Inflated Negative Binomial Regression Model (ZINB) 6. Conclusion 7. Appendix 1. Introduction An event count is the realization of a nonnegative integer-valued random variable (Cameron and Trivedi 1998). Examples are the number of car accidents per month, thunder storms per year, and wild fires per year. The ordinary least squares (OLS) method for event count data results in biased, inefficient, and inconsistent estimates (Long 1997). Thus, researchers have developed various nonlinear models that are based on the Poisson distribution and negative binomial distribution. 1.1 Count Data Regression Models The left-hand side (LHS) of the equation has event count data. Independent variables are, as in the OLS, located at the right-hand side (RHS). These RHS variables may be interval, ratio, or binary (dummy). Table 1 below summarizes the categorical dependent variable regression models (CDVMs) according to the level of measurement of the dependent variable. Table 1. Ordinary Least Squares and CDVMs Model Dependent (LHS) Method Independent (RHS) Ordinary least Moment based OLS Interval or ratio squares method A linear function of interval/ratio or binary variables β + β X + β Binary response Binary (0 or 1) Maximum CDVMs Ordinal response Ordinal (1 st, nd, 3 rd ) likelihood Nominal response Nominal (A, B, C ) method... 0 1 1 X Event count data Count (0, 1,, 3 ) The Poisson regression model (PRM) and negative binomial regression model (NBRM) are basic models for count data analysis. Either the zero-inflated Poisson (ZIP) or the zero-inflated

003-005, The Trustees of Indiana University Regression Models for Event Count Data: negative binomial regression model (ZINB) is used when there are many zero counts. Other count models are developed to handle censored, truncated, or sample selected count data. This document, however, focuses on the PRM, NBRM, ZIP, and ZINB. 1. Poisson Models versus Negative Binomial Models µ y e µ The Poisson probability distribution, P( y µ ) =, has the same mean and variance y! (equidispersion), Var(y)=E(y)= µ. As the mean of a Poisson distribution increases, the probability of zeros decreases and the distribution approximates a normal distribution (Figure 1). The Poisson distribution also has the strong assumption that events are independent. Thus, this distribution does not fit well if µ differs across observations (heterogeneity) (Long 1997). The Poisson regression model (PRM) incorporates observed heterogeneity into the Poisson distribution function, Var ( yi xi ) = E( yi xi ) = µ i = exp( xiβ ). As µ increases, the conditional variance of y increases, the proportion of predicted zeros decreases, and the distribution around the expected value becomes approximately normal (Long 1997). The conditional mean of the errors is zero, but the variance of the errors is a function of independent variables, Var ( ε x) = exp( xβ ). The errors are heteroscedastic. Thus, the PRM rarely fits in practice due to overdispersion (Long 1997; Maddala 1983). Figure 1. Poisson Probability Distribution with Means of.5, 1,, and 5 i y i Γ( yi + vi ) vi µ i The negative binomial probability distribution is P( yi xi ) = yi v i v i i v,! Γ( ) + µ i + µ i where 1 / v = α determines the degree of dispersion and Γ is the Gamma probability distribution. As the dispersion parameter α increases, the variance of the negative binomial distribution also increases, Var ( y i x i ) = µ i ( 1+ µ i v i ). The negative binomial regression model (NBRM) incorporates observed and unobserved heterogeneity into the conditional mean, v

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 3 µ i = exp( xiβ + ε i ) (Long 1997). Thus, the conditional variance of y becomes larger than its conditional mean, E ( y i x i ) = µ i, which remains unchanged. Figure illustrates how the probabilities for small and larger counts increase in the negative binomial distribution as the conditional variance of y increases, given µ = 3. Figure. Negative Binomial Probability Distribution with Alpha of.01,.5, 1, and 5 The PRM and NBRM, however, have the same mean structure. If α = 0, the NBRM reduces to the PRM (Cameron and Trivedi 1998; Long 1997). 1.3 Overdispersion When Var ( yi xi ) > E( yi xi ), we are said to have overdispersion. Estimates of a PRM for overdispersed data are unbiased, but inefficient with standard errors biased downward (Cameron and Trivedi 1998; Long 1997). The likelihood ratio test is developed to examine the null hypothesis of no overdispersion, H 0 : α = 0. The likelihood ratio follows the Chi-squared distribution with one degree of freedom, LR = *(ln L NB ln L Poisson ) ~ χ (1). If the null hypothesis is rejected, the NBRM is preferred to the PRM. Zero-inflated models handle overdispersion by changing the mean structure to explicitly model the production of zero counts (Long 1997). These models assume two latent groups. One is the always-zero group and the other is the not-always-zero or sometime-zero group. Thus, zero counts come from the former group and some of the latter group with a certain probability. The likelihood ratio, LR = *(ln L ZINB ln L ZIP ) ~ χ (1), tests H 0 : α = 0 to compare the ZIP and NBRM. The PRM and ZIP as well as NBRM and ZINB cannot, however, be tested by this likelihood ratio, since they are not nested respectively. The Voung s statistic compares these non-nested models. If V is greater than 1.96, the ZIP or ZINB is favored. If V is less than -1.96, the PRM or NBRM is preferred (Long 1997).

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 4 1.4 Estimation in SAS, STATA, and LIMDEP The SAS GENMOD procedure estimates Poisson and negative binomial regression models. STATA has individual commands (e.g.,.poisson and.nbreg) for the corresponding count data models. LIMDEP has Poisson$ and Negbin$ commands to estimate various count data models including zero-inflated and zero-truncated models. Table summarizes the procedures and commands for count data regression models. Table. Comparison of the Procedures and Commands for Count Data Models Model SAS 9.1 STATA 9.0 LIMDEP 8.0 Poisson Regression (PRM) GENMOD.poisson Poisson$ Negative Binomial Regression (NBRM) GENMOD.nbreg Negbin$ Zero-Inflated Poisson (ZIP) -.zip Poisson; Zip; Rh$ Zero-inflated Negative Binomial (ZINB) -.zinb Negbin; Zip; Rh$ Zero-truncated Poisson (ZTP) -.ztp Poisson; Truncation$ Zero-truncated Negative Binomial (ZTNB) -.ztnb Negbin; Truncation$ The example here examines how waste quotas (emps) and the strictness of policy implementation (strict) affect the frequency of waste spill accidents of plants (accident). 1. 5 Long and Freese s SPost Module STATA users may take advantages of user-written modules such as SPost written by J. Scott Long and Jeremy Freese. The module allows researchers to conduct follow-up analyses of various CDVMs including event count data models. See.3 for examples of major SPost commands. In order to install SPost, execute the following commands consecutively. For more details, visit J. Scott Long s Web site at http://www.indiana.edu/~jslsoc/spost_install.htm.. net from http://www.indiana.edu/~jslsoc/stata/. net install spost9_ado, replace. net get spost9_do, replace

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 5. The Poisson Regression Model The SAS GENMOD procedure, STATA.poisson command, and LIMDEP Poisson$ command estimate the Poisson regression model (PRM)..1 PRM in SAS SAS has the GENMOD procedure for the PRM. The /DIST=POISSON option tells SAS to use the Poisson distribution. PROC GENMOD DATA = masil.accident; MODEL accident=emps strict /DIST=POISSON LINK=LOG; RUN; The GENMOD Procedure Model Information Data Set COUNT.WASTE Distribution Poisson Link Function Log Dependent Variable Accident Observations Used 778 Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance 775 87.079 3.6480 Scaled Deviance 775 87.079 3.6480 Pearson Chi-Square 775 4944.9473 6.3806 Scaled Pearson X 775 4944.9473 6.3806 Log Likelihood -667.91 Algorithm converged. Analysis Of Parameter Estimates Standard Wald 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 0.3901 0.0467 0.986 0.4816 69.84 <.0001 Emps 1 0.0054 0.0007 0.0040 0.0069 53.13 <.0001 Strict 1-0.704 0.0668-0.8350-0.5733 111.5 <.0001 Scale 0 1.0000 0.0000 1.0000 1.0000 NOTE: The scale parameter was held fixed. You will need to run a restricted model without regressors in order to conduct the likelihood ratio test for goodness-of-fit, LR = *(ln L ln L Re ) ~ χ ( J ), where J is the difference in Unrestricted stricted

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 6 the number of regressors between the unrestricted and restricted models. The chi-squared statistic is 14.818 = * [-667.91 - (-79.6400)] (p<.0000). PROC GENMOD DATA = masil.accident; MODEL accident= /DIST=POISSON LINK=LOG; RUN; The GENMOD Procedure Model Information Data Set Distribution Link Function Dependent Variable MASIL.ACCIDENT Poisson Log accident Number of Observations Read 778 Number of Observations Used 778 Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance 777 95.097 3.7993 Scaled Deviance 777 95.097 3.7993 Pearson Chi-Square 777 4919.9745 6.330 Scaled Pearson X 777 4919.9745 6.330 Log Likelihood -79.6400 Algorithm converged. Analysis Of Parameter Estimates Standard Wald 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 0.3168 0.0306 0.568 0.3768 107.0 <.0001 Scale 0 1.0000 0.0000 1.0000 1.0000 NOTE: The scale parameter was held fixed.. PRM in STATA STATA has the.poisson command for the PRM. This command provides likelihood ratio and Pseudo R statistics.. poisson accident emps strict Iteration 0: log likelihood = -181.511 Iteration 1: log likelihood = -181.5101 Iteration : log likelihood = -181.5101

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 7 Poisson regression Number of obs = 778 LR chi() = 14.8 Prob > chi = 0.0000 Log likelihood = -181.5101 Pseudo R = 0.0331 accident Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- emps.0054186.0007434 7.9 0.000.0039615.0068757 strict -.7041664.0667619-10.55 0.000 -.8350174 -.5733154 _cons.3900961.0466787 8.36 0.000.986076.4815846 Let us run a restricted model and then run the.display command in order to double check that the likelihood ratio for goodness-of-fit is 14.818.. poisson accident Iteration 0: log likelihood = -1883.91 Iteration 1: log likelihood = -1883.91 Poisson regression Number of obs = 778 LR chi(0) = 0.00 Prob > chi =. Log likelihood = -1883.91 Pseudo R = 0.0000 accident Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- _cons.3168165.0305995 10.35 0.000.56846.3767904. display * (-181.5101 - (-1883.91)) 14.818.3 Using the SPost Module in STATA The SPost module provides useful commands for follow-up analyses of various categorical dependent variable models. The.fitstat command calculates various goodness-of-fit statistics such as log likelihood, McFadden s R (or Pseudo R ), Akaike Information Criterion (AIC), and Bayesian Information Criterion (BIC).. quietly poisson accident emps strict. fitstat Measures of Fit for poisson of accident Log-Lik Intercept Only: -1883.91 Log-Lik Full Model: -181.510 D(775): 3643.00 LR(): 14.8 Prob > LR: 0.000 McFadden's R: 0.033 McFadden's Adj R: 0.03 Maximum Likelihood R: 0.148 Cragg & Uhler's R: 0.149 AIC: 4.690 AIC*n: 3649.00 BIC: -1515.943 BIC': -111.508

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 8 The.listcoef command lists unstandardized coefficients (parameter estimates), factor and percent changes, and standardized coefficients to help interpret regression results.. listcoef, help poisson (N=778): Factor Change in Expected Count Observed SD:.948675 ---------------------------------------------------------------------- accident b z P> z e^b e^bstdx SDofX -------------+-------------------------------------------------------- emps 0.0054 7.89 0.000 1.0054 1.97 38.1548 strict -0.70417-10.547 0.000 0.4945 0.7031 0.5003 ---------------------------------------------------------------------- b = raw coefficient z = z-score for test of b=0 P> z = p-value for z-test e^b = exp(b) = factor change in expected count for unit increase in X e^bstdx = exp(b*sd of X) = change in expected count for SD increase in X SDofX = standard deviation of X The.prtab command constructs a table of predicted values (events) for all combinations of categorical variables listed. The following example shows that the predicted number of accidents under the strict policy is.917 at the mean waste quota (emps=4.019).. prtab strict poisson: Predicted rates for accident ---------------------- strict Prediction ----------+----------- 0 1.8547 1 0.917 ---------------------- emps strict x= 4.01853.5077108 The.prvalue lists predicted values for a given set of values for the independent variables. For example, the predicted probability of a zero count is.3996 at the mean waste quota under the strict policy (strict=1). Note that the predicted rate of.917 is equivalent to.917 in the.prtab above.. prvalue, x(strict=1) maxcnt(5) poisson: Predictions for accident Predicted rate:.917 95% CI [.87, 1.0] Predicted probabilities: Pr(y=0 x): 0.3996 Pr(y=1 x): 0.3665 Pr(y= x): 0.1681 Pr(y=3 x): 0.0514 Pr(y=4 x): 0.0118 Pr(y=5 x): 0.00 emps strict x= 4.01853 1

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 9 The most useful command is the.prchange that calculates marginal effects (changes) and discrete changes. For instance, a standard deviation increase in waste quota form its mean will increase accidents by.3841 under the lenient policy (strict=0).. prchange, x(strict=0) poisson: Changes in Predicted Rate for accident min->max 0->1 -+1/ -+sd/ MargEfct emps.3070 0.0080 0.0101 0.3841 0.0101 strict -0.9375-0.9375-1.333-0.6568-1.3060 exp(xb): 1.8547 emps strict x= 4.019 0 sd(x)= 38.1548.5006 SPost also includes the.prgen command, which computes a series of predictions by holding all variables but one constant and allowing that variable to vary (Long and Freese 003). These SPost commands work with most categorical and count data models such as.logit,.probit,.poisson,.nbreg,.zip, and.zinb..4 PRM in LIMDEP The LIMDEP Poisson$ command estimates the PRM. LIMDEP reports log likelihoods of both the unrestricted and restricted models. Keep in mind that you must include the ONE for the intercept. POISSON; Lhs=ACCIDENT; Rhs=ONE,EMPS,STRICT$ +---------------------------------------------+ Poisson Regression Maximum Likelihood Estimates Model estimated: Aug 4, 005 at 04:56:45PM. Dependent variable ACCIDENT Weighting variable None Number of observations 778 Iterations completed 8 Log likelihood function -181.510 Restricted log likelihood -1883.91 Chi squared 14.818 Degrees of freedom Prob[ChiSqd > value] =.0000000 Chi- squared = 4944.94781 RsqP= -.0051 G - squared = 87.0794 RsqD=.043 Overdispersion tests: g=mu(i) : 4.70 Overdispersion tests: g=mu(i)^: 4.53 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 10 Constant.390096140.46678663E-01 8.357.0000 EMPS.5418599057E-0.7434193E-03 7.89.0000 4.01853 STRICT -.7041663804.6676196E-01-10.547.0000.5077108 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) SAS, STATA, and LIMDEP produce almost the same parameter estimates and standard errors (Table 3). The log likelihood in SAS is different from that of STATA and LIMDEP (-667.91 versus -181.5101). This difference seems to come from the generalized linear model that the GENMOD procedure uses. These log likelihoods are, however, equivalent in the sense that they result in the same likelihood ratio. Table 3. Summary of the Poisson Regression Model in SAS, STATA, and LIMDEP Model SAS 9.1 STATA 9.0 LIMDEP 8.0 Intercept.3901.3901.3901 (.0467) (.0467) (.0467) EMPS.0054.0054.0054 (.0007) (.0007) (.0007) STRICT -.704 -.704 -.704 (.0668) (.0668) (.0668) Log Likelihood (unrestricted) -667.91-181.5101-181.510 Log Likelihood (restricted) -79.6400-1883.91-1883.91 Likelihood Ratio for Goodness-of-fit 14.818 14.8 14.818

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 11 3. The Negative Binomial Regression Model The SAS GENMODE procedure, STATA.nbreg command, and LIMDEP Negbin$ command estimate the negative binomial regression model (NBRM). 3.1 NBRM in SAS The GENMOD procedure estimates the NBRM using the /DIST=NEGBIN option. Note that the dispersion parameter is equivalent to the alpha in STATA and LIMDEP. PROC GENMOD DATA = masil.accident; MODEL accident=emps strict /DIST=NEGBIN LINK=LOG; RUN; The GENMOD Procedure Model Information Data Set COUNT.WASTE Distribution Negative Binomial Link Function Log Dependent Variable Accident Observations Used 778 Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance 775 589.775 0.7610 Scaled Deviance 775 589.775 0.7610 Pearson Chi-Square 775 845.6033 1.0911 Scaled Pearson X 775 845.6033 1.0911 Log Likelihood 37.568 Algorithm converged. Analysis Of Parameter Estimates Standard Wald 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 0.3851 0.178 0.1345 0.6357 9.07 0.006 Emps 1 0.005 0.003 0.0008 0.0096 5.9 0.014 Strict 1-0.6703 0.1671-0.9978-0.347 16.09 <.0001 Dispersion 1 3.9554 0.3501 3.354 4.7048 NOTE: The negative binomial dispersion parameter was estimated by maximum likelihood. The restricted model produces a log likelihood of 8.867. Thus, the likelihood ratio for goodness-of-fit is 17.400 = * (37.568 8.867) (p<.00017). PROC GENMOD DATA = masil.accident; MODEL accident= /DIST=NEGBIN LINK=LOG; RUN;

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 1 The likelihood ratio for overdispersion is 1409.5838 = * (37.568 - (-667.91)). 3. NBRM in STATA STATA has the.nbreg command for the NBRM. The command reports three log likelihood statistics: for the PRM, restricted NBRM (constant-only model), and unrestricted NBRM (full model), which make it easy to conduct likelihood ratio tests.. nbreg accident emps strict Fitting comparison Poisson model: Iteration 0: log likelihood = -181.511 Iteration 1: log likelihood = -181.5101 Iteration : log likelihood = -181.5101 Fitting constant-only model: Iteration 0: log likelihood = -156.6761 Iteration 1: log likelihood = -115.6155 Iteration : log likelihood = -115.6643 Iteration 3: log likelihood = -115.4183 Iteration 4: log likelihood = -115.4183 Fitting full model: Iteration 0: log likelihood = -1117.1731 Iteration 1: log likelihood = -1116.701 Iteration : log likelihood = -1116.718 Iteration 3: log likelihood = -1116.718 Negative binomial regression Number of obs = 778 LR chi() = 17.40 Prob > chi = 0.000 Log likelihood = -1116.718 Pseudo R = 0.0077 accident Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- emps.0051981.00595.30 0.01.0007694.009667 strict -.670548.1671191-4.01 0.000 -.997801 -.347074 _cons.3851111.178468 3.01 0.003.134536.6356861 -------------+---------------------------------------------------------------- /lnalpha 1.37509.0885176 1.01599 1.54858 -------------+---------------------------------------------------------------- alpha 3.955434.350157 3.3543 4.704793 Likelihood ratio test of alpha=0: chibar(01) = 1409.58 Prob>=chibar = 0.000 The restricted model or constant-only model gives us a log likelihood -115.4183. Thus, the likelihood ratio for goodness-of-fit is 17.400 = * [-1116.718 - (-115.4183)] (p<.00017). The p-value is computed as follows (Note the.disp or.di is an abbreviation of the.display).. disp chitail(, 17.400).00016657

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 13 The likelihood ratio test for overdispersion results in a chi-squared of 1409.5838 (p<.0000) and rejects the null hypothesis of alpha=0. The statistically significant evidence of overdispersion indicates that the NBRM is preferred to the PRM.. di * (-1116.718 - (-181.5101)) 1409.5838 The p-value of the likelihood ratio for overdispersion is computed as,. di chitail(1, 1409.5838) 1.74e-308 Now, let us calculate marginal effects (or changes) at the means of independent variables. You should the read the discrete change labeled 0->1 of a binary variable strict, since its marginal change at the mean (.5077) is meaningless.. prchange nbreg: Changes in Predicted Rate for accident min->max 0->1 -+1/ -+sd/ MargEfct emps 1.536 0.0055 0.0068 0.585 0.0068 strict -0.8931-0.8931-0.8885-0.4383-0.871 exp(xb): 1.3011 emps strict x= 4.019.50771 sd(x)= 38.1548.5006 3.3 NBRM in LIMDEP LIMDEP has the Negbin$ command for the NBRM that reports the PRM as well. Note that the standard errors of parameter estimates are slightly different from those of SAS and STATA. The Marginal Effects$ and the Means$ subcommands compute marginal effects at the mean of independent variables. You may not omit the Means$ subcommand. NEGBIN; Lhs=ACCIDENT; Rhs=ONE,EMPS,STRICT; Marginal Effects; Means$ +---------------------------------------------+ Poisson Regression Maximum Likelihood Estimates Model estimated: Sep 08, 005 at 09:35:36AM. Dependent variable ACCIDENT Weighting variable None Number of observations 778 Iterations completed 8 Log likelihood function -181.510 Restricted log likelihood -1883.91 Chi squared 14.818 Degrees of freedom

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 14 Prob[ChiSqd > value] =.0000000 Chi- squared = 4944.94781 RsqP= -.0051 G - squared = 87.0794 RsqD=.043 Overdispersion tests: g=mu(i) : 4.70 Overdispersion tests: g=mu(i)^: 4.53 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Constant.390096140.46678663E-01 8.357.0000 EMPS.5418599057E-0.7434193E-03 7.89.0000 4.01853 STRICT -.7041663804.6676196E-01-10.547.0000.5077108 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) Normal exit from iterations. Exit status=0. +---------------------------------------------+ Negative Binomial Regression Maximum Likelihood Estimates Model estimated: Sep 08, 005 at 09:35:36AM. Dependent variable ACCIDENT Weighting variable None Number of observations 778 Iterations completed 8 Log likelihood function -1116.718 Restricted log likelihood -181.510 Chi squared 1409.584 Degrees of freedom 1 Prob[ChiSqd > value] =.0000000 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Constant.3851110699.185540.996.007 EMPS.519805734E-0.60075E-0.300.015 4.01853 STRICT -.670547660.1679839-4.006.0001.5077108 Dispersion parameter for count data model Alpha 3.95543401.35680876 11.086.0000 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +-------------------------------------------+ Partial derivatives of expected val. with respect to the vector of characteristics. They are computed at the means of the Xs. Observations used for means are All Obs. Conditional Mean at Sample Point 1.3011 Scale Factor for Marginal Effects 1.3011 +-------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Constant.501068939.19396434.583.0098 EMPS.676313170E-0.9746591E-0.74.030 4.01853 STRICT -.870595665.469308-3.881.0001.5077108 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.)

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 15 Read the coefficients (.0068 and -.871) to confirm that they are identical to the corresponding marginal effects calculated in STATA. SAS, STATA, and LIMDEP produce almost the same parameter estimates and goodness-of-fit statistics (Table 4). Note that SAS reports different log likelihoods, but the same likelihood ratio. Table 4. Summary of the Negative Binomial Regression Model in SAS, STATA, and LIMDEP Model SAS 9.1 STATA 9.0 LIMDEP 8.0 Intercept.3851.3851.3851 (.178) (.178) (.186) EMPS.005.005.005 (.003) (.003) (.003) STRICT -.6703 -.6703 -.6703 (.1671) (.1671) (.1673) Dispersion Parameter (Alpha) 3.9554 3.9554 3.9554 (.3501) (.3501) (.3568) Log Likelihood (unrestricted) 37.568-1116.718-1116.718 Log Likelihood (restricted) 8.867-115.4183-115.418 * Likelihood Ratio for Goodness-of-fit 17.400 17.40 17.400 Likelihood Ratio for Overdispersion 1409.5838 1409.5838 1409.5838 * LIMDEP mistakenly reports the log likelihood of the unrestricted Poisson regression model. The following plot compares the PRM and NBRM. Look at the predictions for zero counts of the two models. As the likelihood ratio test indicates, the NBRM seems to fit these data better than PRM. Figure 3. Comparison of the Poisson and Negative Binomial Regression Models

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 16 4. The Zero-Inflated Poisson Regression Model STATA and LIMDEP have commands for the zero-inflated Poisson regression model (ZIP). 4.1 ZIP in STATA (.zip) STATA has the.zip command to estimate the ZIP. The inflate() option specifies a list of variables that determines whether the observed count is zero. The vuong option computes the Vuong statistic to compare the ZIP and PRM.. zip accident emps strict, inflate(emps strict) vuong Fitting constant-only model: Iteration 0: log likelihood = -167.0779 Iteration 1: log likelihood = -1309.585 Iteration : log likelihood = -17.433 Iteration 3: log likelihood = -170.9543 Iteration 4: log likelihood = -170.953 Iteration 5: log likelihood = -170.953 Fitting full model: Iteration 0: log likelihood = -170.953 Iteration 1: log likelihood = -169.719 Iteration : log likelihood = -169.706 Iteration 3: log likelihood = -169.706 Zero-inflated Poisson regression Number of obs = 778 Nonzero obs = 80 Zero obs = 498 Inflation model = logit LR chi() =.46 Log likelihood = -169.71 Prob > chi = 0.918 Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- accident emps -.00077.0008633-0.3 0.748 -.001969.001415 strict -.093911.07903-1.7 0.05 -.35771.0504948 _cons 1.361978.0493 7.61 0.000 1.65308 1.458647 -------------+---------------------------------------------------------------- inflate emps -.0109897.00678-4.85 0.000 -.0154344 -.006545 strict 1.057031.1767509 5.98 0.000.7106059 1.403457 _cons.488656.111099 4.03 0.000.51849.7607 Vuong test of zip vs. standard Poisson: z = 8.40 Pr>z = 0.0000 The restricted model is estimated with the intercept only.. zip accident, inflate(emps strict) The Vuong statistic at the bottom compares the ZIP and PRM. Since the V 8.40 is greater than 1.96, we conclude that the ZIP is preferred to the PRM.

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 17 4. ZIP in LIMDEP The LIMDEP Poisson$ command needs to have the Zip and Rh subcommands. The Rh is equivalent to the inflate() option in STATA. The Alg=Newton$ subcommand is needed to use the Newton-Raphson algorithm because the default Broyden algorithm failed to converge. 1 POISSON; Lhs=ACCIDENT; Rhs=ONE,EMPS,STRICT; ZIP; Rh=ONE,EMPS,STRICT; Alg=Newton$ +---------------------------------------------+ Poisson Regression Maximum Likelihood Estimates Model estimated: Sep 06, 005 at 00:5:07PM. Dependent variable ACCIDENT Weighting variable None Number of observations 778 Iterations completed 8 Log likelihood function -181.510 Restricted log likelihood -1883.91 Chi squared 14.818 Degrees of freedom Prob[ChiSqd > value] =.0000000 Chi- squared = 4944.94781 RsqP= -.0051 G - squared = 87.0794 RsqD=.043 Overdispersion tests: g=mu(i) : 4.70 Overdispersion tests: g=mu(i)^: 4.53 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Constant.390096140.46678663E-01 8.357.0000 EMPS.5418599057E-0.7434193E-03 7.89.0000 4.01853 STRICT -.7041663804.6676196E-01-10.547.0000.5077108 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) Normal exit from iterations. Exit status=0. +----------------------------------------------------------------------+ Zero Altered Poisson Regression Model Logistic distribution used for splitting model. ZAP term in probability is F[tau x Z(i) ] Comparison of estimated models Pr[0 means] Number of zeros Log-likelihood Poisson.739 Act.= 498 Prd.= 1.6-181.51007 1 If you get a warning message of Error: 806: Line search does not improve fn. Exit iterations. Status=3 or Error: 805: Initial iterations cannot improve function. Status=3, you may change the optimization algorithm or increase the maximum number of iterations (e.g., Maxit=1000$).

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 18 Z.I.Poisson.6464 Act.= 498 Prd.= 50.9-159.88568 Note, the ZIP log-likelihood is not directly comparable. ZIP model with nonzero Q does not encompass the others. Vuong statistic for testing ZIP vs. unaltered model is 9.5740 Distributed as standard normal. A value greater than +1.96 favors the zero altered Z.I.Poisson model. A value less than -1.96 rejects the ZIP model. +----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Poisson/NB/Gamma regression model Constant 1.361977491.3944641E-01 56.880.0000 EMPS -.770010575E-03.37770090E-03 -.733.4633 4.01853 STRICT -.93915073E-01.333650E-01 -.77.0056.5077108 Zero inflation model Constant.4886559537.110013 4.00.0001 EMPS -.1098971050E-01.1549E-0-4.961.0000 4.01853 STRICT 1.057031399.17715551 5.967.0000.5077108 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) In order to estimate the restricted model, run the following command with the ONE only in the Lhs$ subcommand. The Rh$ subcommand remains unchanged. POISSON; Lhs=ACCIDENT; Rhs=ONE; ZIP; Alg=Newton; Rh=ONE,EMPS,STRICT$ Table 5 summarizes parameter estimates and goodness-of-fit statistics for the zero-inflated Poisson model. STATA and LIMDEP report the same parameter estimates, but they produce different standard errors and log likelihoods. In particular, LIMDEP returned a suspicious log likelihood for the restricted model, and thus ended up with the unlikely likelihood ratio of -.0304. In addition, the Vuong statistics in STATA and LIMDEP are different. Table 5. Summary of the Zero-Inflated Poisson Regression Model in STATA, and LIMDEP Model SAS 9.1 STATA 9.0 LIMDEP 8.0 Intercept 1.360 1.360 (.0493) (.039) EMPS -.0003 -.0003 (.0009) (.0004) STRICT -.094 -.094 (.079) (.0333) Intercept (Zero-inflated).4887.4887 (.111) (.11) EMPS (Zero-inflated) -.0110 -.0110 (.003) (.00) STRICT (Zero-inflated) 1.0570 1.0570 (.1768) (.177) Log Likelihood (unrestricted) -169.706-159.8857 Log Likelihood (restricted) -170.953-159.8705 Likelihood Ratio for Goodness-of-fit.46 -.0304 Vuong Statistic (ZINB versus NBRM) 8.40 9.5740

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 19 5. The Zero-Inflated NB Regression Model STATA and LIMDEP can estimate the zero-inflated negative binomial regression model (ZINB). 5.1 ZINB in STATA (.zinb) The STATA.zinb command estimates the ZINB. The vuong option computes the Vuong statistic to compare the ZINB and NBRM.. zinb accident emps strict, inflate(emps strict) vuong Fitting constant-only model: Iteration 0: log likelihood = -1190.5117 (not concave) Iteration 1: log likelihood = -1106.9874 Iteration : log likelihood = -1098.864 Iteration 3: log likelihood = -1095.3638 Iteration 4: log likelihood = -1094.037 Iteration 5: log likelihood = -1093.063 Iteration 6: log likelihood = -109.616 Iteration 7: log likelihood = -1091.798 Iteration 8: log likelihood = -1091.733 Iteration 9: log likelihood = -1091.739 Iteration 10: log likelihood = -1091.739 Fitting full model: Iteration 0: log likelihood = -1091.739 Iteration 1: log likelihood = -1089.5565 Iteration : log likelihood = -1089.5198 Iteration 3: log likelihood = -1089.5198 Zero-inflated negative binomial regression Number of obs = 778 Nonzero obs = 80 Zero obs = 498 Inflation model = logit LR chi() = 4.43 Log likelihood = -1089.5 Prob > chi = 0.1094 Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- accident emps -.0004407.000554-0.1 0.830 -.0044691.0035877 strict -.351317.1659173-1.96 0.050 -.650335.000060 _cons.7763065.1508037 5.15 0.000.4807367 1.071876 -------------+---------------------------------------------------------------- inflate emps -.087768.09551 -.19 0.09 -.395977 -.015763 strict 7.56388 3.055775.47 0.013 1.573179 13.5516 _cons.103115.3800045 0.7 0.786 -.6415835.8480065 -------------+---------------------------------------------------------------- /lnalpha.95514.1351387 6.85 0.000.6603845 1.190118 -------------+---------------------------------------------------------------- alpha.550.3408876 1.935536 3.8747 Vuong test of zinb vs. standard negative binomial: z = 4.13 Pr>z = 0.0000

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 0 The likelihood ratio, 360.404= *(-1089.5198 - (-169.71)), rejects the null hypothesis of no overdispersion, indicating that the ZINB can improve goodness-of-fit over the ZIP (p<.0000). The Vuong test, 4.13 > 1.96, suggests that the ZINB is preferred to the NBRM. 5. ZINB in LIMDEP The LIMDEP Negbin$ command needs to have the Zip and Rh subcommands for the ZINB. The following command produces the Poisson regression model, negative binomial model, and zero-inflated negative binomial model. You may omit the Alg=Newton$ subcommand. NEGBIN; Lhs=ACCIDENT; Rhs=ONE,EMPS,STRICT; Rh=ONE,EMPS,STRICT; ZIP; Alg=Newton$ +---------------------------------------------+ Poisson Regression Maximum Likelihood Estimates Model estimated: Sep 10, 005 at 00:0:00AM. Dependent variable ACCIDENT Weighting variable None Number of observations 778 Iterations completed 8 Log likelihood function -181.510 Restricted log likelihood -1883.91 Chi squared 14.818 Degrees of freedom Prob[ChiSqd > value] =.0000000 Chi- squared = 4944.94781 RsqP= -.0051 G - squared = 87.0794 RsqD=.043 Overdispersion tests: g=mu(i) : 4.70 Overdispersion tests: g=mu(i)^: 4.53 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Constant.390096140.46678663E-01 8.357.0000 EMPS.5418599057E-0.7434193E-03 7.89.0000 4.01853 STRICT -.7041663804.6676196E-01-10.547.0000.5077108 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) Normal exit from iterations. Exit status=0. +---------------------------------------------+ Negative Binomial Regression Maximum Likelihood Estimates Model estimated: Sep 10, 005 at 00:0:00AM. Dependent variable ACCIDENT Weighting variable None Number of observations 778 Iterations completed 1 Log likelihood function -1116.718 Restricted log likelihood -181.510 Chi squared 1409.584

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 1 Degrees of freedom 1 Prob[ChiSqd > value] =.0000000 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Constant.385111048.185540.996.007 EMPS.51980573E-0.60075E-0.300.015 4.01853 STRICT -.670547787.1679839-4.006.0001.5077108 Dispersion parameter for count data model Alpha 3.95543418.35680877 11.086.0000 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) Normal exit from iterations. Exit status=0. +----------------------------------------------------------------------+ Zero Altered Neg.Binomial Regression Model Logistic distribution used for splitting model. ZAP term in probability is F[tau x Z(i) ] Comparison of estimated models Pr[0 means] Number of zeros Log-likelihood Poisson.739 Act.= 498 Prd.= 1.6-181.51007 Neg. Bin..3470 Act.= 498 Prd.= 5.6-1116.7180 Z.I.Neg_Bin.6918 Act.= 498 Prd.= 489.5-1089.51977 Note, the ZIP log-likelihood is not directly comparable. ZIP model with nonzero Q does not encompass the others. Vuong statistic for testing ZIP vs. unaltered model is 4.170 Distributed as standard normal. A value greater than +1.96 favors the zero altered Z.I.Neg_Bin model. A value less than -1.96 rejects the ZIP model. +----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Poisson/NB/Gamma regression model Constant.7763063017.1517804 5.115.0000 EMPS -.440744013E-03.0666E-0 -.18.878 4.01853 STRICT -.351315411.16179883 -.009.0445.5077108 Dispersion parameter Alpha.550810.99400 8.430.0000 Zero inflation model Constant.103103951.37413759.76.787 EMPS -.087767804.68774937E-01-3.036.004 4.01853 STRICT 7.56389399.1639 3.404.0007.5077108 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) In order to estimate the restricted model, run the following command. You have to use the Alg=Newton$ subcommand to get the restricted model to converge. Negbin; Lhs=ACCIDENT; Rhs=ONE; Rh=ONE,EMPS,STRICT; ZIP; Alg=Newton$

003-005, The Trustees of Indiana University Regression Models for Event Count Data: Table 6 summarizes parameter estimates and goodness-of-fit statistics for the zero-inflated negative binomial regression model. STATA and LIMDEP reports the same results except standard errors and likelihood ratio for overdispersion. Table 6. Summary of the Zero-Inflated NBRM in STATA, and LIMDEP Model SAS 9.1 STATA 9.0 LIMDEP 8.0 Intercept.7763.7763 (.1508) (.1518) EMPS -.0004 -.0004 (.001) (.000) STRICT -.351 -.351 (.1659) (.1618) Intercept (Zero-inflated).103.103 (.3800) (.3741) EMPS (Zero-inflated) -.088 -.088 (.0955) (.0688) STRICT (Zero-inflated) 7.564 7.564 (3.0558) (.16) Dispersion Parameter (Alpha).55.55 (.3409) (.99) Log Likelihood (unrestricted) -1089.5198-1089.5198 Log Likelihood (restricted) -1091.739-1091.739 Likelihood Ratio for Goodness-of-fit 4.43 4.43 Likelihood Ratio for Overdispersion 360.404 340.7318 * Vuong Statistic (ZINB versus NBRM) 4.13 4.170 * The likelihood ratio for overdispersion is 340.7318 =*(-1089.5198 - (-159.8857)) Figure 4. Comparison of the Zero-Inflated PRM and the Zero-Inflated NBRM

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 3 6. Conclusion Like other econometric models, researchers must first examine the data generation process of a dependent variable to understand its behavior. Sophisticated researchers pay special attention to excess zeros, censored and/or truncated counts, sample selection, and other particular patterns of the data generation, and then decide which model best describes the data generation process. The Poisson regression model and negative binomial regression model have the same mean structure, but they describe the behavior of a dependent variable in different ways. Zero-inflated regression models integrate two different data generation processes to deal with overdispersion. Truncated or censored regression models are appropriate when data are (left and/or right) truncated or censored. Researchers need to spend more time and effort interpreting the results substantively. Like other categorical dependent variable models, count data models produce estimates that are difficult to interpret intuitively. Reporting parameter estimates and goodness-of-fit statistics are not sufficient. J. Scott Long (1997) and Long and Freese (003) provide good examples of meaningful count data model interpretations. Regarding statistical software, I would recommend STATA for general count data models and LIMDEP for special types of models. Although able to handle various models, LIMDEP does not seem stable and reliable. The SAS GENMODE procedure estimates the Poisson regression model and the negative binomial model, but it does not have easy ways of estimating other models. We encourage SAS Institute to develop an individual procedure, say the CLIM (Count and Limited Dependent Variable Model) procedure, to handle a variety of count data models.

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 4 Appendix: Data Set The data set used here is a part of the data provided for David H. Good s class of the School of Public and Environmental Affairs, Indiana University. Note that these data have been manipulated for the sake of data security. The variables in the data set include, 1. emps: the size of the waste quotas. strict: strictness of policy implementation (1=strict) 3. accident: the frequency of waste spill accidents of plant The followings summarize descriptive statistics of these variables. Note that there are many zero counts that indicate an overdispersion problem.. summarize accident emps strict Variable Obs Mean Std. Dev. Min Max -------------+-------------------------------------------------------- accident 778 1.37751.94867 0 31 emps 778 4.0185 38.1548 1 174 strict 778.507711.50061 0 1. tab accident strict strict accident 0 1 Total -----------+----------------------+---------- 0 14 84 498 1 41 9 70 38 3 70 3 8 13 41 4 16 13 9 5 10 3 13 6 1 7 19 7 4 3 7 8 4 6 9 3 5 10 0 11 3 1 4 1 0 13 1 0 1 14 1 0 1 15 3 0 3 16 1 0 1 17 0 18 1 1 1 0 1 1 31 1 0 1 -----------+----------------------+---------- Total 383 395 778

003-005, The Trustees of Indiana University Regression Models for Event Count Data: 5 References Allison, Paul D. 1991. Logistic Regression Using the SAS System: Theory and Application. Cary, NC: SAS Institute. Cameron, A. Colin, and Pravin K. Trivedi. 1998. Regression Analysis of Count Data. New York: Cambridge University Press. Greene, William H. 003. Econometric Analysis, 5 th ed. Upper Saddle River, NJ: Prentice Hall. Greene, William H. 00. LIMDEP Version 8.0 Econometric Modeling Guide. Plainview, New York: Econometric Software. Long, J. Scott, and Jeremy Freese. 003. Regression Models for Categorical Dependent Variables Using STATA, nd ed. College Station, TX: STATA Press. Long, J. Scott. 1997. Regression Models for Categorical and Limited Dependent Variables. Advanced Quantitative Techniques in the Social Sciences. Sage Publications. Maddala, G. S. 1983. Limited Dependent and Qualitative Variables in Econometrics. New York: Cambridge University Press. SAS Institute. 004. SAS/STAT 9.1 User's Guide. Cary, NC: SAS Institute. STATA Press. 005. STATA Base Reference Manual, Release 9. College Station, TX: STATA Press. Acknowledgements I am grateful to Jeremy Albright and Kevin Wilhite at the UITS Center for Statistical and Mathematical Computing, Indiana University, who provided valuable comments and suggestions. Revision History 003. First draft 004. Second draft 005. Third draft (Added LIMDEP examples)

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 1 Linear Regression Models for Panel Data Using SAS, STATA, LIMDEP, and SPSS Hun Myoung Park This document summarizes linear regression models for panel data and illustrates how to estimate each model using SAS 9.1, STATA 9.0, LIMDEP 8.0, and SPSS 13.0. This document does not address nonlinear models (i.e., logit and probit models), but focuses on linear regression models. 1. Introduction. Least Squares Dummy Variable Regression 3. Panel Data Models 4. The Fixed Group Effect Model 5. The Fixed Time Effect Model 6. The Fixed Group and Time Effect Model 7. Random Effect Models 8. The Poolability Test 9. Conclusion 1. Introduction Panel data are cross sectional and longitudinal (time series). Some examples are the cumulative General Social Survey (GSS) and Current Population Survey (CPS) data. Panel data may have group effects, time effects, or the both. These effects are analyzed by fixed effect and random effect models. 1.1 Data Arrangement A panel data set contains observations on n individuals (e.g., firms and states), each measured at T points in time. In other word, each individual (1 through n subject) includes T observations (1 through t time period). Thus, the total number of observations is nt. Figure 1 illustrates the data arrangement of a panel data set. Figure 1. Data Arrangement of Panel Data Group Time Variable1 Variable Variable3 1 1 1 1 T 1... T

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: n 1 n n T 1. Fixed Effect versus Random Effect Models Panel data models estimate fixed and/or random effects models using dummy variables. The core difference between fixed and random effect models lies in the role of dummies. If dummies are considered as a part of the intercept, it is a fixed effect model. In a random effect model, the dummies act as an error term (see Table 1). The fixed effect model examines group differences in intercepts, assuming the same slopes and constant variance across groups. Fixed effect models use least square dummy variable (LSDV), within effect, and between effect estimation methods. Thus, ordinary least squares (OLS) regressions with dummies, in fact, are fixed effect models. Table 1. Fixed Effect and Random Effect Models Fixed Effect Model Random Effect Model Functional form * ' ' y = ( α + μ ) + X β + v y = α + X β + ( μ + v ) it i it Intercepts Varying across group and/or time Constant Error variances Constant Varying across group and/or time Slopes Constant Constant Estimation LSDV, within effect, between effect GLS, FGLS Hypothesis test Incremental F test Breusch-Pagan LM test * v ~ IID(0, σ ) it v it The random effect model, by contrast, estimates variance components for groups and error, assuming the same intercept and slopes. The difference among groups (or time periods) lies in the variance of the error term. This model is estimated by generalized least squares (GLS) when the Ω matrix, a variance structure among groups, is known. The feasible generalized least squares (FGLS) method is used to estimate the variance structure when Ω is not known. A typical example is the groupwise heteroscedastic regression model (Greene 003). There are various estimation methods for FGLS including maximum likelihood methods and simulations (Baltagi and Cheng 1994). Fixed effects are tested by the (incremental) F test, while random effects are examined by the Lagrange multiplier (LM) test (Breusch and Pagan 1980). If the null hypothesis is not rejected, the pooled OLS regression is favored. The Hausman specification test (Hausman 1978) compares fixed effect and random effect models. Table 1 compares the fixed effect and random effect models. Group effect models create dummies using grouping variables (e.g., country, firm, and race). If one grouping variable is considered, it is called a one-way fixed or random group effects model. Two-way group effect models have two sets of dummy variables, one for a grouping variable and the other for a time variable. it it i it

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 3 1.3 Estimation and Software Issues LSDV regression, the within effect model, the between effect model (group or time mean model), GLS, and FGLS are fundamentally based on OLS in terms of estimation. Thus, any procedure and command for OLS is good for the panel data models. The REG procedure of SAS/STAT, STATA.regress (.cnsreg), LIMDEP regress$, and SPSS regression commands all fit LSDV1 dropping one dummy and have options to suppress the intercept (LSDV). SAS, STATA, and LIMDEP can estimate OLS with restrictions (LSDV3), but SPSS cannot. Note that the STATA.cnsreg command requires the.constraint command that defines a restriction (Table ). Table. Procedures and Commands in SAS, STATA, LIMDEP, and SPSS SAS 9.1 STATA 9.0 LIMDEP 8.0 SPSS 13.0 Regression (OLS) PROC REG.regress Regress$ Regression LSDV1 w/o a dummy w/o a dummy w/o a dummy w/o a dummy LSDV /NOINT Noconstant w/o One in Rhs /Origin LSDV3 RESTRICT.cnsreg Cls: N/A Fixed effect (within effect) Two-way fixed (within effect) Between effect Random effect Two-way random TSCSREG /FIXONE PANEL /FIXONE TSCSREG /FIXTWO PANEL /FIXTWO PANEL /BTWNG PANEL /BTWNT TSCSREG /RANONE PANEL /RANONE TSCSREG /RANTWO PANEL /RANTWO.xtreg w/ fe Regress;Panel;St r=;pds=;fixed$ N/A N/A Regress;Panel;St N/A r=;pds=;fixed$.xtreg w/ be Regress;Panel;St N/A r=;pds=;means$.xtreg w/ re Regress;Panel;St N/A r=;pds=;random$ N/A Problematic N/A SAS, STATA, and LIMDEP also provide the procedures (commands) that are designed to estimate panel data models conveniently. SAS/ETS has the TSCSREG and PANEL procedures to estimate one-way and two-way fixed and random effect models. 1 For the fixed effect model, these procedures estimate LSDV1, which drops one of the dummy variables. For the random effects model, they by default use the Fuller-Battese method (1974) to estimate variance components for group, time, and error. These procedures also support other estimation methods such as Parks (1967) autoregressive model and Da Silva moving average method. The TSCSREG procedure can handle balanced data only, whereas the PANEL procedure is able to deal with balanced and unbalanced data. The former provides one-way and two-way fixed and random effect models, while the latter supports the between effect model and pooled OLS regression as well. Despite advanced features of PANEL, output from the two procedures looks alike. The STATA.xtreg command estimates within effect (fixed effect) models with the fe option, between effect models with the be option, and random effect models with the re option. This command, however, does not fit the two-way fixed and random effect models. The LIMDEP 1 SAS recently announced the PROC PANEL, an experimental procedure, for panel data models.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 4 regress$ command with the panel; subcommand estimates panel data models, but this command is not sufficiently stable. SPSS has limited ability to analyze panel data. 1.4 Data Sets This document uses two data sets. The cross-sectional data set contains research and development (R&D) expenditure data of the top 50 information technology firms presented in OECD Information Technology Outlook 004. The panel data set has cost data for U.S. airlines (1970-1984) from Econometric Analysis (Greene 003). See the Appendix for the details.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 5. Least Squares Dummy Variable Regression A dummy variable is a binary variable that is coded either 1 or zero. It is commonly used to examine group and time effects in regression. Consider a simple model of regressing R&D expenditure in 00 on 000 net income and firm type. The dummy variable d1 is set to 1 for equipment and software firms and zero for telecommunication and electronics. The variable d is coded in the opposite way. Take a look at the data structure (Figure ). Figure. Dummy Variable Coding for Firm Type +-----------------------------------------------------------------+ firm rnd income type d1 d ----------------------------------------------------------------- Samsung,500 4,768 Electronics 0 1 AT&T 54 4,669 Telecom 0 1 IBM 4,750 8,093 IT Equipment 1 0 Siemens 5,490 6,58 Electronics 0 1 Verizon. 11,797 Telecom 0 1 Microsoft 3,77 9,41 Service & S/W 1 0.1 Model 1 without a Dummy Variable The ordinary least squares (OLS) regression without dummy variables, a pooled regression model, assumes a constant intercept and slope regardless of firm types. In the following regression equation, β 0 is the intercept; β 1 is the slope of net income in 000; and ε i is the error term. Model 1: R & D i = β 0 + β1income i + ε i The pooled model has the intercept of 1,48.697 and slope of.3. For a $ one million increase in net income, a firm is likely to increase R&D expenditure in 00 by $.3 million.. regress rnd income Source SS df MS Number of obs = 39 -------------+------------------------------ F( 1, 37) = 7.07 Model 1590406.5 1 1590406.5 Prob > F = 0.0115 Residual 836199.1 37 50305.38 R-squared = 0.1604 -------------+------------------------------ Adj R-squared = 0.1377 Total 99163705.6 38 609571. Root MSE = 1500.1 rnd Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- income.3053.0839066.66 0.01.0530414.393063 _cons 148.697 314.7957 4.71 0.000 844.8599 10.533 Pooled model: R&D = 1,48.697 +.3*income Despite moderate goodness of fit statistics such as F and t, this is a naïve model. R&D investment tends to vary across industries.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 6. Model with a Dummy Variable You may assume that equipment and software firms have more R&D expenditure than other types of companies. Let us take this group difference into account. We have to drop one of the two dummy variables in order to avoid perfect multicollinearity. That is, OLS does not work with both dummies in a model. The δ 1 in model is the coefficient that is valid in equipment and software companies only. Model : R & D i β income d + ε = 0 + β1 i + δ1 1i i Unlike Model 1, this model results in two different regression equations for two groups. The difference lies in the intercepts, but the slope remains unchanged.. regress rnd income d1 Source SS df MS Number of obs = 39 -------------+------------------------------ F(, 36) = 6.06 Model 4987948.9 1493974.4 Prob > F = 0.0054 Residual 74175756.7 36 060437.69 R-squared = 0.50 -------------+------------------------------ Adj R-squared = 0.104 Total 99163705.6 38 609571. Root MSE = 1435.4 rnd Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- income.180066.080348.71 0.010.0551004.380918 d1 1006.66 479.3717.10 0.043 34.41498 1978.837 _cons 1133.579 344.0583 3.9 0.00 435.796 1831.361 d1=1: R&D =,140.05 +.18*income = 1,113.579 +1,006.66*1 +.18*income d1=0: R&D = 1,133.579 +.18*income = 1,113.579 +1,006.66*0 +.18*income The slope.18 indicates a positive impact of two-year-lagged net income on a firm s R&D expenditure. Equipment and software firms on average spend $1,007 million more for R&D than telecommunication and electronics companies..3 Visualization of Model 1 and There is only a tiny difference in the slope (.3 versus.18) between Model 1 and Model. The intercept 1,483 of Model 1, however, is quite different from 1,134 for equipment and software companies and,140 for telecommunications and electronics in Model. This result appears to support Model. Figure 3 highlights differences between Model 1 and more clearly. The black line (pooled) in the middle is the regression line of Model 1; the red line at the top is one for equipment and software companies (d1=1) in Model ; finally the blue line at the bottom is for telecommunication and electronics firms (d=1 or d1=0). The dummy variable (firm types) and regressors (net income) may or may not be correlated.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 7 Figure 3. Regression Lines of Model 1 and Model This plot shows that Model 1 ignores the group difference, and thus reports the misleading intercept. The difference in the intercept between two groups of firms looks substantial. Moreover, the two models have the similar slopes. Consequently, Model considering fixed group effects seems better than the simple Model 1. Compare goodness of fit statistics (e.g., F, t, R, and SSE) of the two models. See Section 3.. and 4.7 for formal hypothesis testing..4 Alternatives to LSDV1 The least squares dummy variable (LSDV) regression is ordinary least squares (OLS) with dummy variables. The critical issue in LSDV is how to avoid the perfect multicollinearity or the so called dummy variable trap. LSDV has three approaches to avoid getting caught in the trap. They produce different parameter estimates of dummies, but their results are equivalent. The first approach, LSDV1, drops a dummy variable as in Model above. The second approach includes all dummies and, in turn, suppresses the intercept (LSDV). Finally, include the intercept and all dummies, and then impose a restriction that the sum of parameters of all dummies is zero (LSDV3). Take a look at the following functional forms to compare these three LSDVs. LSDV1: LSDV: LSDV3: R R R & Di = β 0 + β1incomei + δ1d1 i + ε i or R & Di = β 0 + β1incomei + δ d i + ε i & Di = β 1incomei + δ1d1 i + δ d i + ε i & Di = β 0 + β1incomei + δ1d1 i + δ d i + ε i, subject to δ 1 + δ = 0

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 8 The main differences among these approaches exist in the meanings of the dummy variable parameters. Each approach defines the coefficients of dummy variables in different ways (Table 3). The parameter estimates in LSDV are actual intercepts of groups, making it easy to interpret substantively. LSDV1 reports differences from the reference point (dropped dummy variable). LSDV3 computes how far parameter estimates are away from the average group effect. Accordingly, null hypotheses of t-tests in the three approaches are different. Keep in mind that the R of LSDV is not correct. Table 3 contrasts the three LSDVs. Table 3. Three Approaches of Least Squares Dummy Variable Models LSDV1: Drop one dummy LSDV: Suppress the intercept Dummy included a a α, d a d * * d d d 1 d LSDV3: Impose a restriction c c c α, d d Intercept? Yes No Yes All dummy? No (d-1) Yes (d) Yes (d) Restriction? No No d c i * = 0 Meaning of coefficient How far away from the reference point (dropped)? Fixed group effect How far away from the average group effect? Coefficients * a d i = α + * d dropped = d a α a i, * d 1, H 0 of T-test * * d d = 0 d * = 0 i dropped i d, d * * d 1 d * c d i = α + c 1 α = d * i d Source: David Good s Lecture (004) * This restriction reduces the number of parameters to be estimated, making the model identified. d d d c i * i 1 * d i, where = 0.5 Estimating Three LSDVs The SAS REG procedure, STATA.regress command, LIMDEP Regress$ command, and SPSS Regression command all fit OLS and LSDVs. Let us estimate three LSDVs using SAS and STATA..5.1 LSDV 1 without a Dummy LSDV 1 drops a dummy variable. The intercept is the actual parameter estimate of the dropped dummy variable. The coefficient of the dummy included means how far its parameter estimate is away from the reference point or baseline (i.e., the intercept). Here we include d instead of d1 to see how a different reference point changes the result. Check the sign of the dummy coefficient included and the intercept. Dropping other dummies does not make any significant difference. PROC REG DATA=masil.rnd00; MODEL rnd = income d; RUN;

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 9 The REG Procedure Model: MODEL1 Dependent Variable: rnd Number of Observations Read 50 Number of Observations Used 39 Number of Observations with Missing Values 11 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 4987949 1493974 6.06 0.0054 Error 36 74175757 060438 Corrected Total 38 99163706 Root MSE 1435.448 R-Square 0.50 Dependent Mean 03.56410 Adj R-Sq 0.104 Coeff Var 70.93536 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept 1 140.0468 434.48460 4.93 <.0001 income 1 0.1801 0.0803.71 0.0101 d 1-1006.6593 479.37174 -.10 0.048 d=0: R&D =,140.05 +.18*income =,140.05-1,006.66*0 +.18*income d=1: R&D = 1,133.579 +.18*income =,140.05-1,006.66*1 +.18*income.5. LSDV without the Intercept LSDV includes all dummy variables and suppresses the intercept. The STATA.regress command has the noconstant option to fit LSDV. The coefficients of dummies are actual parameter estimates; thus, you do not need to compute intercepts of groups. This LSDV, however, reports wrong R (.7135.50).. regress rnd income d1 d, noconstant Source SS df MS Number of obs = 39 -------------+------------------------------ F( 3, 36) = 9.88 Model 184685604 3 61561868.1 Prob > F = 0.0000 Residual 74175756.7 36 060437.69 R-squared = 0.7135 -------------+------------------------------ Adj R-squared = 0.6896 Total 58861361 39 6637470.79 Root MSE = 1435.4 rnd Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- income.180066.080348.71 0.010.0551004.380918 d1 140.05 434.4846 4.93 0.000 159.09 301.38 d 1133.579 344.0583 3.9 0.00 435.796 1831.361

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 10 d1=1: R&D =,140.05 +.18*income d=1: R&D = 1,133.579 +.18*income.5.3 LSDV 3 with a Restriction LSDV 3 includes the intercept and all dummies and then imposes a restriction on the model. The restriction is that the sum of all dummy parameters is zero. The STATA.constraint command defines a constraint, while the.cnsreg command fits a constrained OLS using the constraint()option. The number in the parenthesis indicates the constraint number defined in the.constraint command.. constraint 1 d1 + d = 0. cnsreg rnd income d1 d, constraint(1) Constrained linear regression Number of obs = 39 F(, 36) = 6.06 Prob > F = 0.0054 Root MSE = 1435.4 ( 1) d1 + d = 0 rnd Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- income.180066.080348.71 0.010.0551004.380918 d1 503.313 39.6859.10 0.043 17.0749 989.4184 d -503.313 39.6859 -.10 0.043-989.4184-17.0749 _cons 1636.89 310.0438 5.8 0.000 1008.094 65.69 d1=1: R&D =,140.05 +.18*income = 1,637 + 503 *1 + (-503)*0 +.18*income d=1: R&D = 1,133.579 +.18*income = 1,637 + 503 *0 + (-503)*1 +.18*income The intercept is the average of actual parameter estimates: 1,636 = (,140+1,133)/. In the SAS output below, the coefficient of RESTRICT is virtually zero and, in theory, should be zero. PROC REG DATA=masil.rnd00; MODEL rnd = income d1 d; RESTRICT d1 + d = 0; RUN; The REG Procedure Model: MODEL1 Dependent Variable: rnd NOTE: Restrictions have been applied to parameter estimates. Number of Observations Read 50 Number of Observations Used 39 Number of Observations with Missing Values 11 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 4987949 1493974 6.06 0.0054

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 11 Error 36 74175757 060438 Corrected Total 38 99163706 Root MSE 1435.448 R-Square 0.50 Dependent Mean 03.56410 Adj R-Sq 0.104 Coeff Var 70.93536 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept 1 1636.8917 310.04381 5.8 <.0001 income 1 0.1801 0.0803.71 0.0101 d1 1 503.3197 39.68587.10 0.048 d 1-503.3197 39.68587 -.10 0.048 RESTRICT -1 1.81899E-1 0.. * Probability computed using beta distribution. Table 4 compares how SAS, STATA, LIMDEP, and SPSS conducts LSDVs. SPSS is not able to fit the LSDV3. In LIMDEP, the b() of the Cls: indicates the parameter estimate of the second independent variable. In SPSS, pay attention to the /ORIGIN option for LSDV. Table 4. Estimating Three LSDVs Using SAS, STATA, LIMDEP, and SPSS LSDV 1 LSDV LSDV 3 SAS PROC REG; MODEL rnd = income d; RUN; PROC REG; MODEL rnd = income d1 d /NOINT; RUN; PROC REG; MODEL rnd = income d1 d; RESTRICT d1 + d = 0; RUN; STATA. regress ind income d. regress rnd income d1 d, noconstant. constraint 1 d1+ d = 0. cnsreg rnd income d1 d const(1) LIMDEP REGRESS; REGRESS; REGRESS; Lhs=rnd; Lhs=rnd; Lhs=rnd; Rhs=ONE,income, d$ Rhs=income, d1, d$ Rhs=ONE,income, d1, d; Cls: b()+b(3)=0$ SPSS REGRESSION /MISSING LISTWISE /STATISTICS COEFF R ANOVA /CRITERIA=PIN(.05) POUT(.10) /NOORIGIN /DEPENDENT rnd /METHOD=ENTER income d. REGRESSION /MISSING LISTWISE /STATISTICS COEFF R ANOVA /CRITERIA=PIN(.05) POUT(.10) /ORIGIN /DEPENDENT rnd /METHOD=ENTER income d1 d. N/A

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 1 3. Panel Data Models Panel data may have group effects, time effects, or both. These effects are either fixed effect or random effect. A fixed effect model assumes differences in intercepts across groups or time periods, whereas a random effect model explores differences in error variances. A one-way model includes only one set of dummy variables (e.g., firm), while a two way model considers two sets of dummy variables (e.g., firm and year). Model in Chapter, in fact, is a one-way fixed group effect panel data model. 3.1 Functional Forms and Notation The functional forms of one-way panel data models are as follows. ' Fixed group effect model: y it = ( α + μi ) + X it β + vit, where vit ~ IID(0, σ v ) ' Random group effect model: y = α + X β + ( μ + v ), where v ~ IID(0, σ ) it it The dummy variable is a part of the intercept in the fixed effect model and a part of error in the random effect model. vit ~ IID(0, σ v ) indicates that errors are independent identically distributed. The notations used in this document are, y i : dependent variable (DV) mean of group i. x : means of independent variables (IVs) at time t. t y and x for overall means of the DV and IVs, respectively. n: the number of groups or firms T : the number of time periods N=nT : total number of observations k : the number of regressors excluding dummy variables K=k+1 (including the intercept) i it it v 3. Fixed Effect Models There are several strategies for estimating fixed effect models. The least squares dummy variable model (LSDV) uses dummy variables, whereas the within effect does not. These strategies produce the identical slopes of non-dummy independent variables. The between effect model also does not use dummies, but produces different parameter estimates. There are pros and cons of these strategies (Table 5). 3..1 Estimations: LSDV, Within Effect, and Between Effect Model As discussed in Chapter, LSDV is widely used because it is relatively easy to estimate and interpret substantively. This LSDV, however, becomes problematic when there are many

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 13 groups or subjects in the panel data. If T is fixed and N, only coefficients of regressors are consistent. The coefficients of dummy variables, α + μi, are not consistent since the number of these parameters increases as N increases (Baltagi 001). This is so called the incidental parameter problem. Under this circumstance, LSDV is useless, calling for another strategy, the within effect model. The within effect model does not use dummy variables, but uses deviations from group means. Thus, this model is the OLS of ( yit yi ) = β '( xit xi ) + ( ε it ε i ) without an intercept. 3 You do not need to worry about the incidental parameter problem any more. The parameter estimates of regressors are identical to those of LSDV. The within effect model in turn has several disadvantages. Table 5. Three Strategies for Fixed Effect Models LSDV1 Within Effect Between Effect Functional form y iα + X β + ε y y = x x + ε ε y α + + ε i = i i i it i it i it i i = x i Dummy Yes No No Dummy coefficient Presented Need to be computed N/A Transformation No Deviation from the group means Group means Intercept (estimation) Yes No No R Correct Incorrect SSE Correct Correct MSE Correct Smaller Standard error of β Correct Incorrect (smaller) DF error nt-n-k nt-k (Larger) n-k Observations nt nt n Since this model does not report dummy coefficients, you need to compute them using the * formula d = g y g β ' xg Since no dummy is used, the within effect model has a larger degree of freedom for error, resulting in a small MSE (mean square error) and incorrect (larger) standard errors of parameter estimates. Thus, you have to adjust the standard error using the Within * df error nt k formula sek = sek = se LSDV k. Finally, R of the within effect model is not df error nt n k correct because an intercept is suppressed. The between group effect model, so called the group mean regression, uses the group means of the dependent and independent variables. Then, run OLS of yi = α + x i + εi The number of observations decreases to n. This model uses aggregated data to test effects between groups (or individuals), assuming no group and time effect. Table 5 contrasts LSDV, the within effect model, and the between group models. In two-way fixed effect model, LSDV and the between effect model are not valid. i 3 You need to follow three steps: 1) compute group means of the dependent and independent variables; ) transform variables to get deviations of individual values from the group means; 3) run OLS with the transformed variables without the intercept.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 14 3.. Testing Group Effects The null hypothesis is that all dummy parameters except one are zero: H 0 : μ 1 =... = μ n 1 = 0. This hypothesis is tested by the F test, which is based on loss of goodness-of-fit. The robust model in the following formula is LSDV and the efficient model is the pooled regression. 4 ( e' e Efficient ( e' e Robust e' e Robust ) ( n 1) ) ( nt n k) ( RRobust REfficient ) ( n 1) = (1 R ) ( nt n k) Robust ~ F( n 1, nt n k) If the null hypothesis is rejected, you may conclude that the fixed group effect model is better than the pooled OLS model. 3..3 Fixed Time Effect and Two-way Fixed Effect Models For the fixed time effects model, you need to switch n and T, and i and t in the formulas. Model: yit = α + τ t + β ' X it + ε it Within effect model: y y ) = β '( x x ) + ( ε ε ) Dummy coefficients: t Correct standard errors: ( it t it t it t * d t = y t β ' x se * k = se k df df Within error LSDV error = se k Tn k Tn T k Between effect model: y t = α + x t + ε t H 0 : τ 1 =... = τ T 1 = 0. ( e' eefficient e' erobust ) ( T 1) F-test: ~ F( T 1, Tn T k). ( e' e ) ( Tn T k) Robust The fixed group and time effect model uses slightly different formulas. The within effect model of this two-way fixed model has four approaches for LSDV (see 6.1 for details). Model: yit = α + μ i + τ t + β ' X it + ε it. * * Within effect Model: yit = yit yi y t + y and xit = xit xi x t + x. * * Dummy coefficients: d g = ( y g y ) b'( xg x ) and d t = ( y t y ) b'( x t x ) Correct standard errors: se * k = se df Within error k LSDV df error 1 =... = τ T 1 = = se k nt k nt n T k + 1 H 0 : μ 1 =... = μ n 1 = 0 and τ 0. ( e' eefficient e' erobust ) ( n + T ) F-test: ~ F[( n + T ),( nt n T k + 1)] ( e' e ) ( nt n T k + 1) Robust 4 When comparing fixed effect and random effect models, the fixed effect estimates are considered as the robust estimates and random effect estimates as the efficient estimates.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 15 3.3 Random Effect Models The one-way random group effect model is formulated as y = α + β ' X + μ + v, w it = μ i + vit where μi ~ IID(0, σ μ ) and vit ~ IID(0, σ v ). The μ i are assumed independent of v it and X it, which are also independent of each other for all i and t. Remember that this assumption is not necessary in the fixed effect model. The components of Cov w, w ) = E( w w ) are σ μ + if i=j and t=s and σ if i=j and t s. 5 ( it js it js σ v A random effect model is estimated by generalized least squares (GLS) when the variance structure is known and feasible generalized least squares (FGLS) when the variance is unknown. Compared to fixed effect models, random effect models are relatively difficult to estimate. This document assumes panel data are balanced. 3.3.1 Generalized Least Squares (GLS) When Ω is known (given), GLS based on the true variance components is BLUE and all the feasible GLS estimators considered are asymptotically efficient as either n or T approaches infinity (Baltagi 001). The Ω matrix looks like, σ μ + σ v Ω = σ μ T... σ μ T μ σ μ σ + σ... σ μ v............ σ μ σ μ... σ μ + σ v In GLS, you just need to compute θ using the Ω matrix: θ = 1 σ v Tσ μ + σ v. 6 Then transform variables as follows. * yit = yit θ yi * x x θ x for all X k it = it i * α = 1 θ * * * * * Finally, run OLS with the transformed variables: yit = α + x it β ε it. Since Ω is often unknown, FGLS is more frequently used rather than GLS. 3.3. Feasible Generalized Least Squares (FGLS) μ it ti i it 5 This implies that, ) ( js Corr w it w is 1 if i=j and t=s, and σ μ ( σ μ + σ v ) if i=j and t s. 6 If θ = 0, run pooled OLS. If θ = 1 andσ = 0, then run the within effect model. v

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 16 If Ω is unknown, first you have to estimate θ using ˆ σ μ and ˆ σ v : ˆ ˆ σ ˆ σ θ = 1 = 1. T ˆ σ ˆ σ T ˆ σ v μ + v v between The ˆ σ v is derived from the SSE (sum of squares due to error) of the within effect model or from the deviations of residuals from group means of residuals: ˆ σ v SSEwithin e' ewithin = = = nt n k nt n k n T i= 1 t= 1 ( v it v nt n k ) i, where v it are the residuals of the LSDV1. The ˆ σ μ comes from the between effect model (group mean regression): σ v SSE = ˆ σ between, where ˆ σ between between =. T n K ˆ ˆ σ μ Next, transform variables using θˆ and then run OLS: * y y θˆ y it = it i x x θˆ x * it = it i ˆ * α = θ 1 for all X k y * * * * * it = α + x it β ε it. The estimation of the two-way random effect model is skipped here, since it is complicated. 3.3.3 Testing Random Effects (LM test) The null hypothesis is that cross-sectional variance components are zero, H0 : σ u = 0. Breusch and Pagan (1980) developed the Lagrange multiplier (LM) test (Greene 003; Judge et al. 1988). In the following formula, e is the n X 1 vector of the group specific means of pooled regression residuals, and e' e is the SSE of the pooled OLS regression. The LM is distributed as chi-squared with one degree of freedom. nt e' DDe nt T e' e LM μ = 1 = 1 ~ χ (1) ( 1) '. T e e ( T 1) e' e Baltagi (001) presents the same LM test in a different way. LM μ = ( nt T 1) ( e ) it nt ( ) 1 Tei = eit ( T 1) eit 1 ~ χ (1). The two way random effect model has the null hypothesis of H0 : σ u = 0 and σ v = 0. The LM test combines two one-way random effect models for group and time, LM = LM + LM ~ χ (). μv μ v

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 17 3.4 Hausman Test: Fixed Effects versus Random Effects The Hausman specification test compares the fixed versus random effects under the null hypothesis that the individual effects are uncorrelated with the other regressors in the model (Hausman 1978). If correlated (H 0 is rejected), a random effect model produces biased estimators, violating one of the Gauss-Markov assumptions; so a fixed effect model is preferred. Hausman s essential result is that the covariance of an efficient estimator with its difference from an inefficient estimator is zero (Greene 003). m = ' ( ) ˆ 1 b b ( b b ) ~ χ ( k) Robust Efficient Robust Efficient ˆ = Var[ brobust befficient ] = Var( brobust ) Var( befficient ) is the difference between the estimated covariance matrix of the parameter estimates in the LSDV model (robust) and that of the random effects model (efficient). It is notable that an intercept and dummy variables SHOULD be excluded in computation., 3.5 Poolability Test What is poolability? It asks if slopes are the same across groups or over time. Thus, the null hypothesis of the poolability test is H 0 : β ik = β k. Remember that slopes remain constant in fixed and random effect models; only intercepts and error variances matter. The poolability test is undertaken under the assumption of μ ~ N(0, s ). This test uses the F ( e' e ' e e ) ( n 1) K i i statistic, Fobs = ~ F[ ( n 1) K, n( T K) ] ', where e eiei n( T K) I NT e' is the SSE of the pooled OLS and e ie ' i is the SSE of the OLS regression for group i. If the null hypothesis is rejected, the panel data are not poolable. Under this circumstance, you may go to the random coefficient model or hierarchical regression model. Similarly, the null hypothesis of the poolability test over time is H 0 : β tk = β k. The F-test is ' ( e' e etet ) ( T 1) K Fobs = = F[ ( T 1) K, T ( n K) ] ' etet T ( n K), where e te ' is SSE of the OLS t regression at time t.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 18 4. The Fixed Group Effect Model The one-way fixed group model examines group differences in the intercepts. The LSDV for this fixed model needs to create as many dummy variables as the number of groups or subjects. When many dummies are needed, the within effect model is useful since it transforms variables using group means to avoid dummies. The between effect model uses group means of variables. 4.1 The Pooled OLS Regression Model Let us first consider the pooled model without dummy variables.. regress cost output fuel load // pooled model Source SS df MS Number of obs = 90 -------------+------------------------------ F( 3, 86) = 419.34 Model 11.70545 3 37.5684839 Prob > F = 0.0000 Residual 1.33544153 86.0155839 R-squared = 0.9883 -------------+------------------------------ Adj R-squared = 0.9879 Total 114.040893 89 1.8135835 Root MSE =.1461 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- output.887385.013545 66.60 0.000.8563895.9090876 fuel.453977.00304.36 0.000.4136136.4943404 load -1.6751.34530-4.71 0.000 -.313948 -.941077 _cons 9.51693.9445 41.51 0.000 9.061 9.97645 cost = 9.517 +.883*output +.454*fuel -1.68*load. This model fits the data well (p<.0000 and R =.9883). We may, however, suspect fixed group effects that produce different intercepts across groups. As discussed in Chapter, there are three equivalent approaches of LSDV. They report the identical parameter estimates of regresors excluding dummies. Let us begin with LSDV1. 4. LSDV1 without a Dummy LSDV1 drops a dummy variable to identify the model. LSDV1 produces correct ANOVA information, goodness of fit, parameter estimates, and standard errors. As a consequence, this approach is commonly used in practice. LSDV produces six regression equations for six groups (airlines). Group1: cost = 9.706 +.919*output +.417*fuel -1.070*load Group: cost = 9.665 +.919*output +.417*fuel -1.070*load Group3: cost = 9.497 +.919*output +.417*fuel -1.070*load Group4: cost = 9.891 +.919*output +.417*fuel -1.070*load Group5: cost = 9.730 +.919*output +.417*fuel -1.070*load Group6: cost = 9.793 +.919*output +.417*fuel -1.070*load In SAS, the REG procedure fits the OLS regression model. Let us drop the last dummy g6, the reference point. PROC REG DATA=masil.airline; MODEL cost = g1-g5 output fuel load;

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 19 RUN; The REG Procedure Model: MODEL1 Dependent Variable: cost Number of Observations Read 90 Number of Observations Used 90 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 8 113.7487 14.1853 3935.79 <.0001 Error 81 0.96 0.00361 Corrected Total 89 114.04089 Root MSE 0.06011 R-Square 0.9974 Dependent Mean 13.36561 Adj R-Sq 0.997 Coeff Var 0.44970 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept 1 9.79300 0.6366 37.14 <.0001 g1 1-0.08706 0.0840-1.03 0.304 g 1-0.1830 0.07573-1.69 0.0941 g3 1-0.9598 0.0500-5.9 <.0001 g4 1 0.09749 0.03301.95 0.0041 g5 1-0.06301 0.0389 -.64 0.0100 output 1 0.9198 0.0989 30.76 <.0001 fuel 1 0.41749 0.0150 7.47 <.0001 load 1-1.07040 0.0169-5.31 <.0001 Note that the parameter estimate of g6 is presented in the intercept (9.793). Other dummy parameter estimates are computed with the reference point. The actual intercept of the group 1, for example, is computed as 9.706 = 9.793 + (-.087)*1 + (-.183)*0 + (-.960)*0 + (.0975)*0 + (-.0630)*0, where 9.793 is the reference point. STATA has the.regress command for OLS regression (LSDV).. regress cost g1-g5 output fuel load Source SS df MS Number of obs = 90 -------------+------------------------------ F( 8, 81) = 3935.79 Model 113.7487 8 14.185338 Prob > F = 0.0000 Residual.9687 81.0036168 R-squared = 0.9974 -------------+------------------------------ Adj R-squared = 0.997 Total 114.040893 89 1.8135835 Root MSE =.06011 cost Coef. Std. Err. t P> t [95% Conf. Interval]

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 0 -------------+---------------------------------------------------------------- g1 -.0870617.0841995-1.03 0.304 -.54594.080469 g -.18976.075781-1.69 0.094 -.78978.03776 g3 -.95988.050031-5.9 0.000 -.395513 -.196456 g4.097494.0330093.95 0.004.0318159.163171 g5 -.063007.038919 -.64 0.010 -.1105443 -.0154697 output.919846.098901 30.76 0.000.859816.9787565 fuel.4174918.0151991 7.47 0.000.387503.4477333 load -1.070396.0169-5.31 0.000-1.471696 -.6690963 _cons 9.793004.6366 37.14 0.000 9.68399 10.31761 Now, run the LIMDEP Regress$ command to fit the LSDV1. Do not forget to include ONE for the intercept in the Rhs;. --> REGRESS;Lhs=COST;Rhs=ONE,G1,G,G3,G4,G5,OUTPUT,FUEL,LOAD$ +-----------------------------------------------------------------------+ Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.36560933, S.D.= 1.131971444 Model size: Observations = 90, Parameters = 9, Deg.Fr.= 81 Residuals: Sum of squares=.9607777, Std.Dev.=.06010 Fit: R-squared=.997434, Adjusted R-squared =.99718 Model test: F[ 8, 81] = 3935.8, Prob value =.00000 Diagnostic: Log-L = 130.0865, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -5.58, Akaike Info. Crt.= -.691 Autocorrel: Durbin-Watson Statistic = 1.0645, Rho =.48677 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+ Constant 9.793017.6366104 37.14.0000 G1 -.870701949E-01.84199161E-01-1.034.304.16666667 G -.183060033.7577781E-01-1.694.0940.16666667 G3 -.959885994.500855E-01-5.917.0000.16666667 G4.974953376E-01.33009146E-01.954.0041.16666667 G5 -.63007704E-01.3891796E-01 -.637.0100.16666667 OUTPUT.91988143.9889967E-01 30.756.0000-1.174309 FUEL.4174910457.15199071E-01 7.468.0000 1.770359 LOAD -1.070395015.016894-5.307.0000.56046016 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) What if you drop a different dummy variable, say g1, instead of g6? Since the different reference point is applied, you will get different dummy coefficients. The other statistics such as goodness-of-fits, however, remain unchanged.. regress cost g-g6 output fuel load // LSDV1 dropping g1 Source SS df MS Number of obs = 90 -------------+------------------------------ F( 8, 81) = 3935.79 Model 113.7487 8 14.185338 Prob > F = 0.0000 Residual.9687 81.0036168 R-squared = 0.9974 -------------+------------------------------ Adj R-squared = 0.997 Total 114.040893 89 1.8135835 Root MSE =.06011 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- g -.041359.051839-1.64 0.105 -.0913441.00887 g3 -.08911.047986-4.88 0.000 -.940769 -.13765 g4.1845557.060757 3.04 0.003.0636769.3054345

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 1 g5.040547.0799041 0.30 0.764 -.134993.1830387 g6.0870617.0841995 1.03 0.304 -.080469.54594 output.919846.098901 30.76 0.000.859816.9787565 fuel.4174918.0151991 7.47 0.000.387503.4477333 load -1.070396.0169-5.31 0.000-1.471696 -.6690963 _cons 9.70594.19314 50.6 0.000 9.31686 10.090 When you have not created dummy variables, take advantage of the.xi prefix command. 7 Note that STATA by default drops the first dummy variable while the SAS TSCSREG and PANEL procedures in 4.5. drops the last dummy.. xi: regress cost i.airline output fuel load i.airline _Iairline_1-6 (naturally coded; _Iairline_1 omitted) Source SS df MS Number of obs = 90 -------------+------------------------------ F( 8, 81) = 3935.79 Model 113.7487 8 14.185338 Prob > F = 0.0000 Residual.9687 81.0036168 R-squared = 0.9974 -------------+------------------------------ Adj R-squared = 0.997 Total 114.040893 89 1.8135835 Root MSE =.06011 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- _Iairline_ -.041359.051839-1.64 0.105 -.0913441.00887 _Iairline_3 -.08911.047986-4.88 0.000 -.940769 -.13765 _Iairline_4.1845557.060757 3.04 0.003.0636769.3054345 _Iairline_5.040547.0799041 0.30 0.764 -.134993.1830387 _Iairline_6.0870617.0841995 1.03 0.304 -.080469.54594 output.919846.098901 30.76 0.000.859816.9787565 fuel.4174918.0151991 7.47 0.000.387503.4477333 load -1.070396.0169-5.31 0.000-1.471696 -.6690963 _cons 9.70594.19314 50.6 0.000 9.31686 10.090 4.3 LSDV without the Intercept LSDV reports actual parameter estimates of the dummies. Because LSDV suppresses the intercept, you will get incorrect F and R statistics. In the SAS REG procedure, you need to use the /NOINT option to suppress the intercept. Note that the F value of 497,985 and R of 1 are not likely. PROC REG DATA=masil.airline; MODEL cost = g1-g6 output fuel load /NOINT; RUN; The REG Procedure Model: MODEL1 Dependent Variable: cost Number of Observations Read 90 Number of Observations Used 90 7 The STATA.xi is used either as an ordinary command or a prefix command like.bysort. This command creates dummies from a categorical variable specified in the term i. and then run the command following the colon.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: NOTE: No intercept in model. R-Square is redefined. Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 9 16191 1799.03381 497985 <.0001 Error 81 0.96 0.00361 Uncorrected Total 90 1619 Root MSE 0.06011 R-Square 1.0000 Dependent Mean 13.36561 Adj R-Sq 1.0000 Coeff Var 0.44970 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t g1 1 9.70594 0.1931 50.6 <.0001 g 1 9.66471 0.19898 48.57 <.0001 g3 1 9.4970 0.496 4. <.0001 g4 1 9.89050 0.4176 40.91 <.0001 g5 1 9.73000 0.6094 37.9 <.0001 g6 1 9.79300 0.6366 37.14 <.0001 output 1 0.9198 0.0989 30.76 <.0001 fuel 1 0.41749 0.0150 7.47 <.0001 load 1-1.07040 0.0169-5.31 <.0001 STATA uses the noconstant option to suppress the intercept. Note that noc is its abbreviation.. regress cost g1-g6 output fuel load, noc Source SS df MS Number of obs = 90 -------------+------------------------------ F( 9, 81) =. Model 16191.3043 9 1799.03381 Prob > F = 0.0000 Residual.9687 81.0036168 R-squared = 1.0000 -------------+------------------------------ Adj R-squared = 1.0000 Total 16191.5969 90 179.906633 Root MSE =.06011 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- g1 9.70594.19314 50.6 0.000 9.31686 10.090 g 9.664706.19898 48.57 0.000 9.68794 10.0606 g3 9.49701.49584 4. 0.000 9.04944 9.944618 g4 9.890498.417635 40.91 0.000 9.409464 10.37153 g5 9.79997.60941 37.9 0.000 9.10804 10.4919 g6 9.793004.6366 37.14 0.000 9.68399 10.31761 output.919846.098901 30.76 0.000.859816.9787565 fuel.4174918.0151991 7.47 0.000.387503.4477333 load -1.070396.0169-5.31 0.000-1.471696 -.6690963 In LIMDEP, you need to drop ONE out of the Rhs; to suppress the intercept. Unlike SAS and STATA, LIMDEP reports correct R and F even in LSDV.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 3 --> REGRESS;Lhs=COST;Rhs=G1,G,G3,G4,G5,G6,OUTPUT,FUEL,LOAD$ +-----------------------------------------------------------------------+ Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.36560933, S.D.= 1.131971444 Model size: Observations = 90, Parameters = 9, Deg.Fr.= 81 Residuals: Sum of squares=.9607777, Std.Dev.=.06010 Fit: R-squared=.997434, Adjusted R-squared =.99718 Model test: F[ 8, 81] = 3935.8, Prob value =.00000 Diagnostic: Log-L = 130.0865, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -5.58, Akaike Info. Crt.= -.691 Model does not contain ONE. R-squared and F can be negative! Autocorrel: Durbin-Watson Statistic = 1.0645, Rho =.48677 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+ G1 9.70594953.193135 50.58.0000.16666667 G 9.66471569.19898117 48.571.0000.16666667 G3 9.49703673.495746 4.17.0000.16666667 G4 9.890513806.417645 40.910.0000.16666667 G5 9.730013568.6094094 37.88.0000.16666667 G6 9.793017.6366104 37.14.0000.16666667 OUTPUT.91988143.9889967E-01 30.756.0000-1.174309 FUEL.4174910457.15199071E-01 7.468.0000 1.770359 LOAD -1.070395015.016894-5.307.0000.56046016 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) 4.4 LSDV3 with Restrictions LSDV3 imposes a restriction that the sum of the dummy parameters is zero. The SAS REG procedure uses the RESTRICT statement to impose restrictions. PROC REG DATA=masil.airline; MODEL cost = g1-g6 output fuel load; RESTRICT g1 + g + g3 + g4 + g5 + g6 = 0; RUN; The REG Procedure Model: MODEL1 Dependent Variable: cost NOTE: Restrictions have been applied to parameter estimates. Number of Observations Read 90 Number of Observations Used 90 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 4 Model 8 113.7487 14.1853 3935.79 <.0001 Error 81 0.96 0.00361 Corrected Total 89 114.04089 Root MSE 0.06011 R-Square 0.9974 Dependent Mean 13.36561 Adj R-Sq 0.997 Coeff Var 0.44970 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept 1 9.71353 0.964 4.30 <.0001 g1 1-0.00759 0.0456-0.17 0.8683 g 1-0.0488 0.03798-1.9 0.03 g3 1-0.1651 0.01606-13.48 <.0001 g4 1 0.17697 0.0194 9.11 <.0001 g5 1 0.01647 0.03669 0.45 0.6547 g6 1 0.07948 0.04050 1.96 0.053 output 1 0.9198 0.0989 30.76 <.0001 fuel 1 0.41749 0.0150 7.47 <.0001 load 1-1.07040 0.0169-5.31 <.0001 RESTRICT -1 3.01674E-15 1.51088E-10 0.00 1.0000* * Probability computed using beta distribution. The dummy coefficients mean deviations from the averaged group effect (9.714). The actual intercept of group, for example, is 9.665 =9.714+ (-.049). Note that the 3.01674E-15 of RESTRICT below is virtually zero. In STATA, you have to use the.cnsreg command rather than.regress. The command, however, does not provide an ANOVA table and goodness-of-fit statistics.. constraint define 1 g1 + g + g3 + g4 + g5 + g6 = 0. cnsreg cost g1-g6 output fuel load, constraint(1) Constrained linear regression Number of obs = 90 F( 8, 81) = 3935.79 Prob > F = 0.0000 Root MSE =.06011 ( 1) g1 + g + g3 + g4 + g5 + g6 = 0 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- g1 -.0075859.0456178-0.17 0.868 -.0983509.083179 g -.048818.0379787-1.9 0.0 -.143875.067439 g3 -.165069.016064-13.48 0.000 -.484661 -.1845478 g4.1769698.019447 9.11 0.000.138308.156189 g5.0164689.0366904 0.45 0.655 -.0565335.089471 g6.0794759.0405008 1.96 0.053 -.001108.1600597 output.919846.098901 30.76 0.000.859816.9787565 fuel.4174918.0151991 7.47 0.000.387503.4477333 load -1.070396.0169-5.31 0.000-1.471696 -.6690963 _cons 9.71358.9641 4.30 0.000 9.56614 10.17044

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 5 LIMDEP has the Cls$ subcommand to impose restrictions. Again, do not forget to include ONE in the Rhs;. --> REGRESS;Lhs=COST;Rhs=ONE,G1,G,G3,G4,G5,G6,OUTPUT,FUEL,LOAD; Cls:b(1)+b()+b(3)+b(4)+b(5)+b(6)=0$ +-----------------------------------------------------------------------+ Linearly restricted regression Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.36560933, S.D.= 1.131971444 Model size: Observations = 90, Parameters = 9, Deg.Fr.= 81 Residuals: Sum of squares=.9607777, Std.Dev.=.06010 Fit: R-squared=.997434, Adjusted R-squared =.99718 (Note: Not using OLS. R-squared is not bounded in [0,1] Model test: F[ 8, 81] = 3935.8, Prob value =.00000 Diagnostic: Log-L = 130.0865, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -5.58, Akaike Info. Crt.= -.691 Note, when restrictions are imposed, R-squared can be less than zero. F[ 1, 80] for the restrictions =.0000, Prob = 1.0000 Autocorrel: Durbin-Watson Statistic = 1.0645, Rho =.48677 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+ Constant 1.105614.788696 43.469.0000 G1 -.416106889.89836871E-01-6.894.0000.16666667 G -.457340873.899154E-01-9.63.0000.16666667 G3 -.6503469.56175656E-01-46.79.0000.16666667 G4 -.3154336.41557714E-01-53.697.0000.16666667 G5 -.3904574.9995908E-01-79.746.0000.16666667 G6 -.39034870.33569388E-01-69.380.0000.16666667 OUTPUT.91988143.9889967E-01 30.756.0000-1.174309 FUEL.4174910457.15199071E-01 7.468.0000 1.770359 LOAD -1.070395015.016894-5.307.0000.56046016 LSDV3 in LIMDEP reports different dummy coefficients. But you may draw actual intercepts of groups in a manner similar to what you would do in SAS and STATA. The actual intercept of group 3, for example, is 9.497 = 1.1 + (-.65). 4.5 Within Group Effect Model The within effect model does not use the dummies and thus has larger degrees of freedom, smaller MSE, and smaller standard errors of parameters than those of LSDV. As a consequence, you need to adjust standard errors. This model does not report individual dummy coefficients either. The SAS TSCSREG procedure and LIMDEP Regress$ command report the adjusted (correct) MSE, SEE (Root MSE), R, and standard errors. 4.5.1 Estimating the Within Effect Model

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 6 First, let us manually estimate the within group effect model in STATA. You need to compute group means and transform dependent and independent variables using group means (log is skipped here).. egen gm_cost=mean(cost), by(airline) // compute group means. egen gm_output=mean(output), by(airline). egen gm_fuel=mean(fuel), by(airline). egen gm_load=mean(load), by(airline) You will get the following group means of variables. +------------------------------------------------------+ airline gm_cost gm_output gm_fuel gm_load ------------------------------------------------------ 1 14.67563.319696 1.7318.5971917 14.3747 -.03307 1.75171.5470946 3 13.3731 -.9166 1.7897.5845358 4 13.1358-1.635174 1.77803.5476773 5 1.36304 -.85681 1.791.5664859 6 1.7441 -.49898 1.7788.5197756 +------------------------------------------------------+. gen gw_cost = cost - gm_cost // compute deviations from the group means. gen gw_output = output - gm_output. gen gw_fuel = fuel - gm_fuel. gen gw_load = load - gm_load Now, we are ready to run the within effect model. Keep in mind that you have to suppress the intercept. Carefully check MSE, SEE, R, and standard errors.. regress gw_cost gw_output gw_fuel gw_load, noc // within effect Source SS df MS Number of obs = 90 -------------+------------------------------ F( 3, 87) = 3871.8 Model 39.0683861 3 13.07954 Prob > F = 0.0000 Residual.96861 87.003363481 R-squared = 0.996 -------------+------------------------------ Adj R-squared = 0.993 Total 39.361009 90.437344544 Root MSE =.058 gw_cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- gw_output.919846.08841 31.87 0.000.86196.976609 gw_fuel.4174918.0146657 8.47 0.000.38834.4466414 gw_load -1.070396.1946109-5.50 0.000-1.45706 -.6835858 * You may compute group intercepts using d = g y g β ' xg. For example, the intercept of airline 5 is computed as 9.730 = 1.363 {.919*(-.86) +.417*1.79 + (-1.073)*.566 }. In order to get the correct standard errors, you need to adjust them using the ratio of degrees of freedom of the within effect model and the LSDV. For example, the standard error of the logged output is computed as.099=.088*sqrt(87/81). 4.5. Using the SAS TSCSREG and PANEL Procedures The TSCSREG and PANEL procedures of SAS/ETS allows users to fit the within effect model conveniently. The procedures, in fact, report LSDV1, but you do not need to create dummy

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 7 variables and compute deviations from the group means. This procedures reports correct MSE, SEE, R, and standard errors, and conducts the F test for the fixed group effect as well. PROC SORT DATA=masil.airline; BY airline year; PROC TSCSREG DATA=masil.airline; ID airline year; MODEL cost = output fuel load /FIXONE; RUN; The TSCSREG Procedure Dependent Variable: cost Model Description Estimation Method FixOne Number of Cross Sections 6 Time Series Length 15 Fit Statistics SSE 0.96 DFE 81 MSE 0.0036 Root MSE 0.0601 R-Square 0.9974 F Test for No Fixed Effects Num DF Den DF F Value Pr > F 5 81 57.73 <.0001 Parameter Estimates Standard Variable DF Estimate Error t Value Pr > t Label CS1 1-0.08706 0.084-1.03 0.304 Cross Sectional Effect 1 CS 1-0.183 0.0757-1.69 0.0941 Cross Sectional Effect CS3 1-0.9598 0.0500-5.9 <.0001 Cross Sectional Effect 3 CS4 1 0.097494 0.0330.95 0.0041 Cross Sectional Effect 4 CS5 1-0.06301 0.039 -.64 0.0100 Cross Sectional Effect 5 Intercept 1 9.793004 0.637 37.14 <.0001 Intercept output 1 0.91985 0.099 30.76 <.0001 fuel 1 0.41749 0.015 7.47 <.0001 load 1-1.0704 0.017-5.31 <.0001

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 8 Note that a data set needs to be sorted in advance by variables to appear in the ID statement of the TSCSREG and PANEL procedures. The following PANEL procedure returns the same output. PROC PANEL DATA=masil.airline; ID airline year; MODEL cost = output fuel load /FIXONE; RUN; 4.5.3 Using STATA The STATA.xtreg command fits the within group effect model without creating dummy variables. The command reports correct standard errors and the F test for fixed group effects. This command, however, does not provide an analysis of variance (ANOVA) table and correct R and F statistics. The.xtreg command should follow the.tsset command that specifies grouping and time variables.. tsset airline year panel variable: airline, 1 to 6 time variable: year, 1 to 15 The fe of.xtreg indicates the within effect model and i(airline) specifies airline as the independent unit. Note that this command reports adjusted (correct) standard errors.. xtreg cost output fuel load, fe i(airline) // within group effect Fixed-effects (within) regression Number of obs = 90 Group variable (i): airline Number of groups = 6 R-sq: within = 0.996 Obs per group: min = 15 between = 0.9856 avg = 15.0 overall = 0.9873 max = 15 F(3,81) = 3604.80 corr(u_i, Xb) = -0.3475 Prob > F = 0.0000 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- output.919846.098901 30.76 0.000.859816.9787565 fuel.4174918.0151991 7.47 0.000.387503.4477333 load -1.070396.0169-5.31 0.000-1.471696 -.6690963 _cons 9.71358.9641 4.30 0.000 9.56614 10.17044 -------------+---------------------------------------------------------------- sigma_u.130775 sigma_e.06010514 rho.8843653 (fraction of variance due to u_i) F test that all u_i=0: F(5, 81) = 57.73 Prob > F = 0.0000 The last line of the output tests the null hypothesis that all dummy parameters in LSDV1 are zero (e.g., g1=0, g=0, g3=0, g4=0, and g5=0). Not the intercept of 9.714 is that of LSDV3. 4.5.4 Using LIMDEP

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 9 In LIMDEP, you have to specify the panel data model and stratification or time variables. The Panel$ and Fixed$ subcommands mean a fixed effect panel data model. The Str$ subcommand specifies a stratification variable. --> REGRESS;Lhs=COST;Rhs=ONE,OUTPUT,FUEL,LOAD;Panel;Str=AIRLINE;Fixed$ +-----------------------------------------------------------------------+ OLS Without Group Dummy Variables Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.36560933, S.D.= 1.131971444 Model size: Observations = 90, Parameters = 4, Deg.Fr.= 86 Residuals: Sum of squares= 1.3354495, Std.Dev.=.1461 Fit: R-squared=.98890, Adjusted R-squared =.98788 Model test: F[ 3, 86] = 419.33, Prob value =.00000 Diagnostic: Log-L = 61.7699, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -4.1, Akaike Info. Crt.= -1.84 Panel Data Analysis of COST [ONE way] Unconditional ANOVA (No regressors) Source Variation Deg. Free. Mean Square Between 74.6799 5. 14.9360 Residual 39.3611 84..468584 Total 114.041 89. 1.8136 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+ OUTPUT.887386341.135455E-01 66.599.0000-1.174309 FUEL.4539777119.030440E-01.359.0000 1.770359 LOAD -1.67507797.3453093-4.713.0000.56046016 Constant 9.5169131.945 41.514.0000 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +-----------------------------------------------------------------------+ Least Squares with Group Dummy Variables Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.36560933, S.D.= 1.131971444 Model size: Observations = 90, Parameters = 9, Deg.Fr.= 81 Residuals: Sum of squares=.9607777, Std.Dev.=.06010 Fit: R-squared=.997434, Adjusted R-squared =.99718 Model test: F[ 8, 81] = 3935.8, Prob value =.00000 Diagnostic: Log-L = 130.0865, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -5.58, Akaike Info. Crt.= -.691 Estd. Autocorrelation of e(i,t).573531 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+ OUTPUT.91988143.9889967E-01 30.756.0000-1.174309 FUEL.4174910457.15199071E-01 7.468.0000 1.770359 LOAD -1.070395015.016894-5.307.0000.56046016 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) LIMDEP reports both the pooled OLS regression and the within effect model. Like the SAS TSCSREG procedure, LIMDEP provides correct MSE, SEE, R, and standard errors.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 30 4.6 Between Group Effect Model: Group Mean Regression The between effect model uses aggregate information, group means of variables. In other words, the unit of analysis is not an individual observation, but groups or subjects. The number of observations jumps down to n from nt. This group mean regression produces different goodness-of-fits and parameter estimates from those of LSDV and the within effect model. Let us compute group means and run the OLS regression with them. The.collapse command computes aggregate information and saves into a new data set. Note that /// links two command lines.. collapse (mean) gm_cost=cost (mean) gm_output=output (mean) gm_fuel=fuel (mean) /// gm_load=load, by(airline). regress gm_cost gm_output gm_fuel gm_load Source SS df MS Number of obs = 6 -------------+------------------------------ F( 3, ) = 104.1 Model 4.9469814 3 1.64899375 Prob > F = 0.0095 Residual.03167596.015837963 R-squared = 0.9936 -------------+------------------------------ Adj R-squared = 0.9841 Total 4.97865717 5.995731433 Root MSE =.1585 gm_cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- gm_output.784568.1087646 7.19 0.019.3144803 1.50433 gm_fuel -5.53904 4.478718-1.3 0.343-4.7947 13.74647 gm_load -1.75107.743167-0.64 0.589-13.55397 10.0518 _cons 85.8081 56.48199 1.5 0.68-157.143 38.8305 The SAS PANEL procedure has the /BTWNG and /BTWNT option to estimate the between effect model. The TSCSREG procedure does not have this option. PROC PANEL DATA=masil.airline; ID airline year; MODEL cost = output fuel load /BTWNG; RUN; The PANEL Procedure Between Groups Estimates Dependent Variable: cost Model Description Estimation Method BtwGrps Number of Cross Sections 6 Time Series Length 15 Fit Statistics SSE 0.0317 DFE MSE 0.0158 Root MSE 0.158

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 31 R-Square 0.9936 Parameter Estimates Standard Variable DF Estimate Error t Value Pr > t Label Intercept 1 85.80901 56.4830 1.5 0.681 Intercept output 1 0.78455 0.1088 7.19 0.0188 fuel 1-5.5398 4.4788-1.3 0.347 load 1-1.7510.743-0.64 0.5886 The STATA.xtreg command has the be option to fit the between effect model. This command, however, does not report the ANOVA table.. xtreg cost output fuel load, be i(airline) Between regression (regression on group means) Number of obs = 90 Group variable (i): airline Number of groups = 6 R-sq: within = 0.8808 Obs per group: min = 15 between = 0.9936 avg = 15.0 overall = 0.1371 max = 15 F(3,) = 104.1 sd(u_i + avg(e_i.))=.158491 Prob > F = 0.0095 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- output.78455.1087663 7.19 0.019.3144715 1.50439 fuel -5.53978 4.47880-1.3 0.343-4.79471 13.74675 load -1.751016.74319-0.64 0.589-13.55401 10.05198 _cons 85.80901 56.4830 1.5 0.68-157.178 38.8358 LIMDEP has the Mean; subcommand to fit the between effect model. --> REGRESS;Lhs=COST;Rhs=ONE,OUTPUT,FUEL,LOAD;Panel;Str=AIRLINE;Means$ +-----------------------------------------------------------------------+ Group Means Regression Ordinary least squares regression Weighting variable = none Dep. var. = YBAR(i.) Mean= 13.36560933, S.D.=.9978636346 Model size: Observations = 6, Parameters = 4, Deg.Fr.= Residuals: Sum of squares=.31677706e-01, Std.Dev.=.1584 Fit: R-squared=.993638, Adjusted R-squared =.98410 Model test: F[ 3, ] = 104.13, Prob value =.00953 Diagnostic: Log-L = 7.185, Restricted(b=0) Log-L = -7.9538 LogAmemiyaPrCrt.= -3.635, Akaike Info. Crt.= -1.073 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ OUTPUT.78447689.1087616 7.194.0000.30561E-11 FUEL -5.54437466 4.4786519-1.34.174.1864891 LOAD -1.750947653.7430470 -.638.533.3541105 Constant 85.81483169 56.481148 1.519.187

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 3 4.7 Testing Fixed Group Effects (F-test) How do we know whether there are fixed group effects? The null hypothesis is that all dummy parameters except one are zero: H 0 : μ 1 =... = μ n 1 = 0. In order to conduct a F-test, let us take the SSE (e e) of 1.3354 from the pooled OLS regression and.96 from the LSDVs (LSDV1 through LSDV3) or the within effect model. Alternatively, you may draw R of.9974 from LSDV1 or LSDV3 and.9883 from the pooled OLS. Do not, however, use LSDV and the within effect model for R. (1.3354.96) (6 1) (.9974.9883) (6 1) The Fstatistic is computed as = ~ 57.7319[5,81]. (.96) (90 6 3) (1.9974) (90 6 3) The large F statistic rejects the null hypothesis in favor of the fixed group effect model (p<.0000). The SAS TSCSREG and PANEL procedures and STATA.xtreg command by default conduct the F test. Alternatively, you may conduct the same test with LSDV1. In SAS, add the TEST statement in the REG procedure and run the procedure again (other outputs are skipped). PROC REG DATA=masil.airline; MODEL cost = g1-g5 output fuel load; TEST g1 = g = g3 = g4 = g5 = 0; RUN; The REG Procedure Model: MODEL1 Test 1 Results for Dependent Variable cost Mean Source DF Square F Value Pr > F Numerator 5 0.0856 57.73 <.0001 Denominator 81 0.00361 In STATA, run the.test command, a follow-up command for the Wald test, right after estimating the model.. quietly regress cost g1-g5 output fuel load // LSDV1. test g1 g g3 g4 g5 ( 1) g1 = 0 ( ) g = 0 ( 3) g3 = 0 ( 4) g4 = 0 ( 5) g5 = 0 F( 5, 81) = 57.73 Prob > F = 0.0000

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 33 4.8 Summary Table 6 summarizes the estimation of panel data models in SAS, STATA, and LIMDEP. The SAS REG and TSCSREG procedures are generally preferred to STATA and LIMDEP commands. Table 6 Comparison of the Fixed Effect Model in SAS, STATA, LIMDEP * SAS 9.1 STATA 9.0 LIMDEP 8.0 OLS estimation PROC REG;. regress (cnsreg) Regress$ LSDV1 Correct Correct Correct (slightly different F) LSDV Incorrect F, (adjusted) R Incorrect F, (adjusted) R Correct (slightly different F) Correct R LSDV3 Correct. cnsreg command No R, ANOVA table but F Correct (slightly different F) Different dummy coefficients Panel Estimation PROC TSCSREG;. xtreg Regress; Panel$ PROC PANEL; Estimation type LSDV1 Within and between effect Within effect SSE (e e) Correct No Correct MSE or SEE Correct (adjusted) No Correct (adjusted) SEE Model test (F) No Incorrect Slightly different F (adjusted) R Correct Incorrect Correct Intercept Correct LSDV3 intercept No Coefficients Correct Correct Correct Standard errors Correct (adjusted) Correct (adjusted) Correct (adjusted) Effect test (F) Yes Yes No Between effect Yes (PROC PANEL;) Yes (the be option) N/A * Yes/No means whether the software reports the statistics. Correct/incorrect indicates whether the statistics are different from those of the least squares dummy variable (LSDV) 1 without a dummy variable.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 34 5. The Fixed Time Effect Model The fixed time effect model investigates how time affects the intercept using time dummy variables. The logic and method are the same as those of the fixed group effect model. 5.1 Least Squares Dummy Variable Models The least squares dummy variable (LSDV) model produces fifteen regression equations. This section does not present all outputs, but one or two for each LSDV approach. Time01: cost = 0.496 +.868*output -.484*fuel -1.954*load Time0: cost = 0.578 +.868*output -.484*fuel -1.954*load Time03: cost = 0.656 +.868*output -.484*fuel -1.954*load Time04: cost = 0.741 +.868*output -.484*fuel -1.954*load Time05: cost = 1.00 +.868*output -.484*fuel -1.954*load Time06: cost = 1.41 +.868*output -.484*fuel -1.954*load Time07: cost = 1.503 +.868*output -.484*fuel -1.954*load Time08: cost = 1.654 +.868*output -.484*fuel -1.954*load Time09: cost = 1.830 +.868*output -.484*fuel -1.954*load Time10: cost =.114 +.868*output -.484*fuel -1.954*load Time11: cost =.465 +.868*output -.484*fuel -1.954*load Time1: cost =.651 +.868*output -.484*fuel -1.954*load Time13: cost =.617 +.868*output -.484*fuel -1.954*load Time14: cost =.55 +.868*output -.484*fuel -1.954*load Time15: cost =.537 +.868*output -.484*fuel -1.954*load 5.1.1 LSDV1 without a Dummy Let us begin with the SAS REG procedure. The test statement examines fixed time effects. PROC REG DATA=masil.airline; MODEL cost = t1-t14 output fuel load; RUN; The REG Procedure Model: MODEL1 Dependent Variable: cost Number of Observations Read 90 Number of Observations Used 90 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 17 11.9570 6.6448 439.6 <.0001 Error 7 1.08819 0.01511 Corrected Total 89 114.04089 Root MSE 0.194 R-Square 0.9905

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 35 Dependent Mean 13.36561 Adj R-Sq 0.988 Coeff Var 0.91981 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept 1.53677 4.94053 4.56 <.0001 t1 1 -.04096 0.73469 -.78 0.0070 t 1-1.95873 0.775 -.71 0.0084 t3 1-1.88103 0.7036 -.61 0.0110 t4 1-1.79601 0.6988 -.57 0.01 t5 1-1.33693 0.50604 -.64 0.0101 t6 1-1.1514 0.4086 -.75 0.0075 t7 1-1.03341 0.3764 -.75 0.0076 t8 1-0.8874 0.3601 -.71 0.0085 t9 1-0.70719 0.9470 -.40 0.0190 t10 1-0.496 0.16679 -.54 0.0134 t11 1-0.07144 0.07176-1.00 0.38 t1 1 0.11457 0.09841 1.16 0.48 t13 1 0.07979 0.0844 0.95 0.3477 t14 1 0.01546 0.0764 0.1 0.830 output 1 0.86773 0.01541 56.3 <.0001 fuel 1-0.48448 0.36411-1.33 0.1875 load 1-1.95440 0.4438-4.4 <.0001 The following are the corresponding STATA and LIMDEP commands for LSDV1 (outputs are skipped).. regress cost t1-t14 output fuel load REGRESS;Lhs=COST;Rhs=ONE,T1,T,T3,T4,T5,T6,T7,T8,T9,T10,T11,T1,T13,T14,OUTPUT,FUEL,LOAD$ 5.1. LSDV without the Intercept Let us use LIMDEP to fit LSDV because it reports correct (although slightly different) F and R statistics. --> REGRESS;Lhs=COST;Rhs=T1,T,T3,T4,T5,T6,T7,T8,T9,T10,T11,T1,T13,T14,T15,OUTPUT,FUEL,LOAD$ +-----------------------------------------------------------------------+ Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.3656099, S.D.= 1.13197100 Model size: Observations = 90, Parameters = 18, Deg.Fr.= 7 Residuals: Sum of squares= 1.0881903, Std.Dev.=.194 Fit: R-squared=.990458, Adjusted R-squared =.9880 Model test: F[ 17, 7] = 439.6, Prob value =.00000 Diagnostic: Log-L = 70.9837, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -4.010, Akaike Info. Crt.= -1.177 Model does not contain ONE. R-squared and F can be negative! Autocorrel: Durbin-Watson Statistic =.93900, Rho = -.46950 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 36 T1 0.49580478 4.09583 4.869.0000.66666667E-01 T 0.57803885 4.156 4.875.0000.66666667E-01 T3 0.65573100 4.41771 4.890.0000.66666667E-01 T4 0.74075857 4.457497 4.885.0000.66666667E-01 T5 1.199830 4.440331 4.774.0000.66666667E-01 T6 1.411608 4.53861 4.718.0000.66666667E-01 T7 1.50335085 4.5713968 4.704.0000.66666667E-01 T8 1.654087 4.68858 4.684.0000.66666667E-01 T9 1.8957108 4.656906 4.688.0000.66666667E-01 T10.1138060 4.796483 4.614.0000.66666667E-01 T11.4653734 4.9499089 4.539.0000.66666667E-01 T1.65133704 5.008594 4.5.0000.66666667E-01 T13.61655508 4.9861391 4.536.0000.66666667E-01 T14.5583 4.9559418 4.551.0000.66666667E-01 T15.5367656 4.940531 4.56.0000.66666667E-01 OUTPUT.867767843.15408184E-01 56.316.0000-1.174309 FUEL -.4844835367.36410849-1.331.1875 1.770359 LOAD -1.95440438.4437771-4.418.0000.56046015 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) The following are the corresponding SAS REG procedure and STATA command for LSDV (outputs are skipped). PROC REG DATA=masil.airline; MODEL cost = t1-t15 output fuel load /NOINT; RUN;. regress cost t1-t15 output fuel load, noc 5.1.3 LSDV3 with a Restriction In SAS, you need to use the RESTRICT statement to impose a restriction. PROC REG DATA=masil.airline; MODEL cost = t1-t15 output fuel load; RESTRICT t1+t+t3+t4+t5+t6+t7+t8+t9+t10+t11+t1+t13+t14+t15=0; RUN; The REG Procedure Model: MODEL1 Dependent Variable: cost NOTE: Restrictions have been applied to parameter estimates. Number of Observations Read 90 Number of Observations Used 90 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 17 11.9570 6.6448 439.6 <.0001 Error 7 1.08819 0.01511 Corrected Total 89 114.04089

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 37 Root MSE 0.194 R-Square 0.9905 Dependent Mean 13.36561 Adj R-Sq 0.988 Coeff Var 0.91981 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept 1 1.66698 4.6405 4.69 <.0001 t1 1-1.17118 0.41783 -.80 0.0065 t 1-1.08894 0.40586 -.68 0.0090 t3 1-1.0115 0.4033 -.51 0.0144 t4 1-0.96 0.38177 -.43 0.0178 t5 1-0.46715 0.19076 -.45 0.0168 t6 1-0.5536 0.09856 -.59 0.0116 t7 1-0.16363 0.07190 -.8 0.058 t8 1-0.0196 0.0486-0.7 0.7907 t9 1 0.1659 0.0671.59 0.0115 t10 1 0.4468 0.17599.54 0.0133 t11 1 0.79834 0.3940.4 0.0179 t1 1 0.98435 0.38756.54 0.013 t13 1 0.94957 0.36537.60 0.0113 t14 1 0.8854 0.33549.64 0.010 t15 1 0.86978 0.309.7 0.0083 output 1 0.86773 0.01541 56.3 <.0001 fuel 1-0.48448 0.36411-1.33 0.1875 load 1-1.95440 0.4438-4.4 <.0001 RESTRICT -1-3.946E-15... * Probability computed using beta distribution. In STATA, define the restriction with the.constraint command and specify the restriction using the constraint() option of the.cnsreg command.. constraint define 3 t1+t+t3+t4+t5+t6+t7+t8+t9+t10+t11+t1+t13+t14+t15=0. cnsreg cost t1-t15 output fuel load, constraint(3) Constrained linear regression Number of obs = 90 F( 17, 7) = 439.6 Prob > F = 0.0000 Root MSE =.194 ( 1) t1 + t + t3 + t4 + t5 + t6 + t7 + t8 + t9 + t10 + t11 + t1 + t13 + t14 + t15 = 0 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- t1-1.171179.4178338 -.80 0.007 -.004115 -.3384 t -1.088945.4058579 -.68 0.009-1.898008 -.798816 t3-1.0115.403308 -.51 0.014-1.815078 -.07466 t4 -.9649.3817675 -.43 0.018-1.68765 -.165185 t5 -.4671515.1907596 -.45 0.017 -.847439 -.0868791 t6 -.55367.0985615 -.59 0.01 -.4518415 -.0588839 t7 -.163636.0718969 -.8 0.06 -.3069564 -.003088 t8 -.01955.048649-0.7 0.791 -.109887.0839768 t9.165876.067099.59 0.01.0375776.875976 t10.4468191.175994.54 0.013.0959814.7976568 t11.7983439.39407.4 0.018.1416916 1.454996 t1.9843536.3875583.54 0.013.11770 1.756937

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 38 t13.9495716.3653675.60 0.011.148 1.677918 t14.885448.335491.64 0.010.164554 1.554034 t15.869781.30933.7 0.008.31891 1.50875 output.867768.015408 56.3 0.000.8370111.898444 fuel -.4844835.3641085-1.33 0.188-1.1031.413535 load -1.954404.443777-4.4 0.000 -.83668-1.0754 _cons 1.66698 4.64053 4.69 0.000 1.4491 30.88486 The following are the corresponding LIMDEP command for LSDV3 (outputs are skipped). REGRESS;Lhs=COST;Rhs=ONE,T1,T,T3,T4,T5,T6,T7,T8,T9,T10,T11,T1,T13,T14,T15,OUTPUT,FUEL,LOAD; Cls:b(1)+b()+b(3)+b(4)+b(5)+b(6)+b(7)+b(8)+b(9)+b(10)+b(11)+b(1)+b(13)+b(14)+b(15)=0$ 5. Within Time Effect Model The within effect mode for the fixed time effects needs to compute deviations from the time means. Keep in mind that the intercept should be suppressed. 5..1 Estimating the Time Effect Model Let us manually estimate the fixed time effect model first.. egen tm_cost = mean(cost), by(year) // compute time means. egen tm_output = mean(output), by(year). egen tm_fuel = mean(fuel), by(year). egen tm_load = mean(load), by(year) +---------------------------------------------------+ year tm_cost tm_output tm_fuel tm_load --------------------------------------------------- 1 1.36897-1.79083 11.63606.4788587 1.45963-1.744389 11.66868.48683 3 1.60706-1.577767 11.67494.5358 4 1.7791-1.443695 11.73193.544486 5 1.94143-1.3981 1.6843.563566 6 13.045-1.39300 1.5386.5541809 7 13.15965-1.30416 1.6714.560745 8 13.9884-1.963 1.76768.5670587 9 13.4651-1.067003 1.86104.6179098 10 13.70187 -.903156 13.3183.633943 11 13.9134 -.905539 13.6646.580577 1 14.05984 -.8641667 13.8315.585643 13 14.1841 -.793916 13.75979.5803183 14 14.3517 -.648015 13.67403.580458 15 14.306 -.557684 13.6997.5797168 +---------------------------------------------------+. gen tw_cost = cost - tm_cost // transform variables. gen tw_output = output - tm_output. gen tw_fuel = fuel - tm_fuel. gen tw_load = load - tm_load. regress tw_cost tw_output tw_fuel tw_load, noc // within time effect Source SS df MS Number of obs = 90 -------------+------------------------------ F( 3, 87) = 015.95 Model 75.6459391 3 5.15313 Prob > F = 0.0000 Residual 1.0881903 87.01507934 R-squared = 0.9858 -------------+------------------------------ Adj R-squared = 0.9853 Total 76.734194 90.85601437 Root MSE =.11184

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 39 tw_cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- tw_output.867768.0140171 61.90 0.000.8398663.8955873 tw_fuel -.4844836.331359-1.46 0.147-1.14851.1738836 tw_load -1.954404.404388-4.86 0.000 -.75495-1.154514 * If you want to get intercepts of years, use d t = y t β ' x t. For example, the intercept of year 7 is 1.503=13.1597-{.8677*(-1.304) + (-.4845)*1.671 + (-1.9544)*.5607}. As discussed previously, the standard errors of the within effects model need to be adjusted. For instance, the correct standard error of fuel price is computed as.364 =.331*sqrt(87/7). 5.. Using the TSCSREG and PANEL procedures You need to sort the data set by variables (i.e., year and airline) to appear in the ID statement of the TSCSREG and PANEL procedures. PROC SORT DATA=masil.airline; BY year airline; PROC PANEL DATA=masil.airline; ID year airline; MODEL cost = output fuel load /FIXONE; RUN; The PANEL Procedure Fixed One Way Estimates Dependent Variable: cost Model Description Estimation Method FixOne Number of Cross Sections 15 Time Series Length 6 Fit Statistics SSE 1.088 DFE 7 MSE 0.0151 Root MSE 0.19 R-Square 0.9905 F Test for No Fixed Effects Num DF Den DF F Value Pr > F 14 7 1.17 0.3178 Parameter Estimates Standard Variable DF Estimate Error t Value Pr > t Label

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 40 CS1 1 -.04096 0.7347 -.78 0.0070 Cross Sectional Effect 1 CS 1-1.95873 0.78 -.71 0.0084 Cross Sectional Effect CS3 1-1.88103 0.704 -.61 0.0110 Cross Sectional Effect 3 CS4 1-1.79601 0.6988 -.57 0.01 Cross Sectional Effect 4 CS5 1-1.33693 0.5060 -.64 0.0101 Cross Sectional Effect 5 CS6 1-1.1514 0.4086 -.75 0.0075 Cross Sectional Effect 6 CS7 1-1.03341 0.3764 -.75 0.0076 Cross Sectional Effect 7 CS8 1-0.8874 0.360 -.71 0.0085 Cross Sectional Effect 8 CS9 1-0.70719 0.947 -.40 0.0190 Cross Sectional Effect 9 CS10 1-0.496 0.1668 -.54 0.0134 Cross Sectional Effect 10 CS11 1-0.07144 0.0718-1.00 0.38 Cross Sectional CS1 1 0.114571 0.0984 1.16 0.48 Cross Sectional Effect 1 CS13 1 0.079789 0.0844 0.95 0.3477 Cross Sectional Effect 13 CS14 1 0.015463 0.076 0.1 0.830 Cross Sectional Effect 14 Intercept 1.53677 4.9405 4.56 <.0001 Intercept output 1 0.86777 0.0154 56.3 <.0001 fuel 1-0.48448 0.3641-1.33 0.1875 load 1-1.9544 0.444-4.4 <.0001 The following TSCSREG procedure gives the same outputs. PROC TSCSREG DATA=masil.airline; ID year airline; MODEL cost = output fuel load /FIXONE; RUN; 5..3 Using STATA The STATA.xtreg command uses the fe option for the fixed effect model.. xtreg cost output fuel load, fe i(year) Fixed-effects (within) regression Number of obs = 90 Group variable (i): year Number of groups = 15 R-sq: within = 0.9858 Obs per group: min = 6 between = 0.481 avg = 6.0 overall = 0.565 max = 6 F(3,7) = 1668.37 corr(u_i, Xb) = -0.1503 Prob > F = 0.0000 cost Coef. Std. Err. t P> t [95% Conf. Interval]

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 41 -------------+---------------------------------------------------------------- output.867768.015408 56.3 0.000.8370111.898444 fuel -.4844835.3641085-1.33 0.188-1.1031.413535 load -1.954404.443777-4.4 0.000 -.83668-1.0754 _cons 1.66698 4.64053 4.69 0.000 1.4491 30.88486 -------------+---------------------------------------------------------------- sigma_u.807907 sigma_e.193801 rho.9770860 (fraction of variance due to u_i) F test that all u_i=0: F(14, 7) = 1.17 Prob > F = 0.3178 5..4 Using LIMDEP You need to pay attention to the Str=; subcommand for stratification. --> REGRESS;Lhs=COST;Rhs=ONE,OUTPUT,FUEL,LOAD;Panel;Str=YEAR;Fixed$ +-----------------------------------------------------------------------+ OLS Without Group Dummy Variables Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.36560933, S.D.= 1.131971444 Model size: Observations = 90, Parameters = 4, Deg.Fr.= 86 Residuals: Sum of squares= 1.3354495, Std.Dev.=.1461 Fit: R-squared=.98890, Adjusted R-squared =.98788 Model test: F[ 3, 86] = 419.33, Prob value =.00000 Diagnostic: Log-L = 61.7699, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -4.1, Akaike Info. Crt.= -1.84 Panel Data Analysis of COST [ONE way] Unconditional ANOVA (No regressors) Source Variation Deg. Free. Mean Square Between 37.3068 14..66477 Residual 76.7341 75. 1.031 Total 114.041 89. 1.8136 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+ OUTPUT.887386341.135455E-01 66.599.0000-1.174309 FUEL.4539777119.030440E-01.359.0000 1.770359 LOAD -1.67507797.3453093-4.713.0000.56046016 Constant 9.5169131.945 41.514.0000 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +-----------------------------------------------------------------------+ Least Squares with Group Dummy Variables Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.36560933, S.D.= 1.131971444 Model size: Observations = 90, Parameters = 18, Deg.Fr.= 7 Residuals: Sum of squares= 1.088193393, Std.Dev.=.194 Fit: R-squared=.990458, Adjusted R-squared =.9880 Model test: F[ 17, 7] = 439.6, Prob value =.00000 Diagnostic: Log-L = 70.9836, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -4.010, Akaike Info. Crt.= -1.177 Estd. Autocorrelation of e(i,t).573531 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 4 OUTPUT.867768093.15408179E-01 56.316.0000-1.174309 FUEL -.4844946699.36410984-1.331.1868 1.770359 LOAD -1.954414378.4437791-4.418.0000.56046016 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +------------------------------------------------------------------------+ Test Statistics for the Classical Model Model Log-Likelihood Sum of Squares R-squared (1) Constant term only -138.35814.114040981D+03.0000000 () Group effects only -10.5864.7673414157D+0.371354 (3) X - variables only 61.76991.13354495D+01.988897 (4) X and group effects 70.9836.1088193393D+01.9904579 Hypothesis Tests Likelihood Ratio Test F Tests Chi-squared d.f. Prob. F num. denom. Prob value () vs (1) 35.659 14.00117.605 14 75.00404 (3) vs (1) 400.56 3.00000 419.39 3 86.00000 (4) vs (1) 418.684 17.00000 439.617 17 7.00000 (4) vs () 383.05 3.00000 1668.364 3 7.00000 (4) vs (3) 18.47 14.18800 1.169 14 7.31776 +------------------------------------------------------------------------+ 5.3 Between Time Effect Model The between effect model regresses time means of dependent variables on those of independent variables. See also 3. and 4.6.. collapse (mean) tm_cost=cost (mean) tm_output=output (mean) tm_fuel=fuel /// (mean) tm_load=load, by(year). regress tm_cost tm_output tm_fuel tm_load // between time effect Source SS df MS Number of obs = 15 -------------+------------------------------ F( 3, 11) = 4074.33 Model 6.10479 3.07073493 Prob > F = 0.0000 Residual.005590631 11.00050839 R-squared = 0.9991 -------------+------------------------------ Adj R-squared = 0.9989 Total 6.177954 14.4441844 Root MSE =.054 tm_cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- tm_output 1.133337.051898.10 0.000 1.00449 1.465 tm_fuel.334486.0884 14.64 0.000.840035.3844937 tm_load -1.35077.47864-5.45 0.000-1.896189 -.805644 _cons 11.18505.3660016 30.56 0.000 10.37949 11.9906 The SAS PANEL procedure has the /BTWNT option to estimate the between effect model. PROC PANEL DATA=masil.airline; ID airline year; MODEL cost = output fuel load /BTWNT; RUN; The PANEL Procedure

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 43 Between Time Periods Estimates Dependent Variable: cost Model Description Estimation Method BtwTime Number of Cross Sections 6 Time Series Length 15 Fit Statistics SSE 0.0056 DFE 11 MSE 0.0005 Root MSE 0.05 R-Square 0.9991 Parameter Estimates Standard Variable DF Estimate Error t Value Pr > t Label Intercept 1 11.18504 0.3660 30.56 <.0001 Intercept output 1 1.133335 0.0513.10 <.0001 fuel 1 0.33449 0.08 14.64 <.0001 load 1-1.35073 0.478-5.45 0.000 You may use the be option in the STATA.xtreg command and the Means; subcommand in LIMDEP (outputs are skipped).. xtreg cost output fuel load, be i(year) // between time effect model --> REGRESS;Lhs=COST;Rhs=ONE,OUTPUT,FUEL,LOAD;Panel;Str=YEAR;Means$ 5.4 Testing Fixed Time Effects. The null hypothesis is that all time dummy parameters except one are zero: (1.3354 1.088) (15 1) H 0 : τ 1 =... = τ T 1 = 0. The F statistic is ~ 1.1683[14,7]. The p- (1.088) (6*15 15 3) value of.3180 does not reject the null hypothesis. The SAS TSCSREG and PANEL procedures and the STATA.xtreg command conduct the Wald test. You may get the same test using the TEST statement in LSDV1 and the STATA.test command (the output is skipped). PROC REG DATA=masil.airline; MODEL cost = t1-t14 output fuel load; TEST t1=t=t3=t4=t5=t6=t7=t8=t9=t10=t11=t1=t13=t14=0; RUN;. quietly regress cost t1-t14 output fuel load. test t1 t t3 t4 t5 t6 t7 t8 t9 t10 t11 t1 t13 t14

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 44 6. The Fixed Group and Time Effect Model The two-way fixed model considers both group and time effects. This model thus needs two sets of group and time dummy variables. LSDV and the between effect model are not valid in this model. 6.1 Least Squares Dummy Variable Models There are four approaches to avoid the perfect multicollinearity or the dummy variable trap. You may not suppress the intercept under any circumstances. Drop one cross-section and one time-series dummy variables. Drop one cross-section dummy and impose a restriction on the time-series dummies of τ t = 0 Drop one time-series dummy and impose a restriction on the cross-section dummies of μ g = 0 Include all dummy variables and impose two restrictions on the cross-section and timeseries dummies of μ = 0 τ = 0 g t 6. LSDV1 without Two Dummies Let us first run LSDV1 using STATA.. regress cost g1-g5 t1-t14 output fuel load Source SS df MS Number of obs = 90 -------------+------------------------------ F(, 67) = 1960.8 Model 113.864044 5.17563838 Prob > F = 0.0000 Residual.176848775 67.00639534 R-squared = 0.9984 -------------+------------------------------ Adj R-squared = 0.9979 Total 114.040893 89 1.8135835 Root MSE =.05138 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- g1.17485.086101.0 0.047.003861.346179 g.1114508.0779551 1.43 0.157 -.044148.670499 g3 -.143511.0518934 -.77 0.007 -.470907 -.0399313 g4.180087.031443 5.61 0.000.1160484.443691 g5 -.046694.04688 -.08 0.04 -.09154 -.0018463 t1 -.693138.3378385 -.05 0.044-1.367467 -.0188098 t -.6384366.33080-1.9 0.059-1.30171.043983 t3 -.5958031.394473-1.81 0.075-1.53383.0617764 t4 -.541537.3189139-1.70 0.094-1.178708.0944011 t5 -.473049.319459 -.04 0.045 -.9360088 -.0100769 t6 -.4704.18844 -.7 0.07 -.8033319 -.0510764 t7 -.3959783.173969 -.8 0.05 -.7418804 -.050076 t8 -.3398463.150106 -.6 0.07 -.6394596 -.04033 t9 -.718933.1348175 -.0 0.048 -.5409901 -.007964 t10 -.73857.0763495 -.98 0.004 -.37978 -.0749914 t11 -.111803.0319005-3.50 0.001 -.175477 -.048195 t1 -.033641.049008-0.78 0.436 -.119713.0519893 t13 -.0177346.036554-0.49 0.66 -.0901007.0546315 t14 -.0186451.030508-0.61 0.543 -.0795393.0449

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 45 output.817487.031851 5.66 0.000.7536739.880835 fuel.16861.163478 1.03 0.306 -.1576935.4949135 load -.88814.617373-3.37 0.001-1.40544 -.3603843 _cons 1.94004.1831 5.83 0.000 8.51434 17.36765 The following is the corresponding SAS REG procedure (outputs are skipped). PROC REG DATA=masil.airline; MODEL cost = g1-g5 t1-t14 output fuel load; RUN; The LIMDEP example is skipped here, since many dummy variables need to be listed in the Regress$ command. 6.3 LSDV1 + LSDV3: Dropping a Dummy and Imposing a Restriction In the second approach, you may drop either one group dummy or one time dummy. The following drops one time dummy, includes all group dummies, and imposes a restriction on group dummies. PROC REG DATA=masil.airline; MODEL cost = g1-g6 t1-t14 output fuel load; RESTRICT g1 + g + g3 + g4 + g5 + g6 = 0; RUN; The REG Procedure Model: MODEL1 Dependent Variable: cost NOTE: Restrictions have been applied to parameter estimates. Number of Observations Read 90 Number of Observations Used 90 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 113.86404 5.17564 1960.8 <.0001 Error 67 0.17685 0.0064 Corrected Total 89 114.04089 Root MSE 0.05138 R-Square 0.9984 Dependent Mean 13.36561 Adj R-Sq 0.9979 Coeff Var 0.38439 Parameter Estimates Parameter Standard

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 46 Variable DF Estimate Error t Value Pr > t Intercept 1 1.98600.540 5.84 <.0001 g1 1 0.1833 0.04601.79 0.0069 g 1 0.06549 0.03897 1.68 0.0975 g3 1-0.18947 0.01561-1.14 <.0001 g4 1 0.1345 0.0183 7.33 <.0001 g5 1-0.0965 0.03731 -.48 0.0155 g6 1-0.04596 0.04161-1.10 0.733 t1 1-0.69314 0.33784 -.05 0.0441 t 1-0.63844 0.3308-1.9 0.0588 t3 1-0.59580 0.3945-1.81 0.0750 t4 1-0.5415 0.31891-1.70 0.0938 t5 1-0.47304 0.3195 -.04 0.0454 t6 1-0.470 0.18844 -.7 0.066 t7 1-0.39598 0.17330 -.8 0.055 t8 1-0.33985 0.15011 -.6 0.068 t9 1-0.7189 0.1348 -.0 0.0477 t10 1-0.739 0.07635 -.98 0.0040 t11 1-0.11180 0.03190-3.50 0.0008 t1 1-0.03364 0.0490-0.78 0.4357 t13 1-0.01773 0.0366-0.49 0.663 t14 1-0.01865 0.03051-0.61 0.543 output 1 0.8175 0.03185 5.66 <.0001 fuel 1 0.16861 0.16348 1.03 0.3061 load 1-0.8881 0.6174-3.37 0.001 RESTRICT -1-1.9387E-16... * Probability computed using beta distribution. Alternatively, you may run the STATA.cnsreg command with the second constraint (output is skipped).. cnsreg cost g1-g6 t1-t14 output fuel load, constraint() The following drops one group dummy and imposes a restriction on time dummies.. cnsreg cost g1-g5 t1-t15 output fuel load, constraint(3) Constrained linear regression Number of obs = 90 F(, 67) = 1960.8 Prob > F = 0.0000 Root MSE =.05138 ( 1) t1 + t + t3 + t4 + t5 + t6 + t7 + t8 + t9 + t10 + t11 + t1 + t13 + t14 + t15 = 0 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- g1.17485.086101.0 0.047.003861.346179 g.1114508.0779551 1.43 0.157 -.044148.670499 g3 -.143511.0518934 -.77 0.007 -.470907 -.0399313 g4.180087.031443 5.61 0.000.1160484.443691 g5 -.046694.04688 -.08 0.04 -.09154 -.0018463 t1 -.374045.19187-1.95 0.055 -.757006.0089536 t -.31938.1860877-1.7 0.091 -.6907554.051097 t3 -.766893.1833501-1.51 0.136 -.646576.089789 t4 -.30399.179671-1.9 0.0 -.568837.1038 t5 -.153991.0864404-1.78 0.079 -.364649.0186066 t6 -.1080904.0448591 -.41 0.019 -.197696 -.0185513 t7 -.0768646.0319336 -.41 0.019 -.1406043 -.013148 t8 -.00736.004506-1.01 0.314 -.06155.000869 t9.04705.0908 1.6 0.109 -.010878.105688

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 47 t10.091781.081155 1.13 0.6 -.070531.53709 t11.073105.1491443 1.39 0.169 -.090389.5050039 t1.85477.1756365 1.63 0.109 -.0650993.6360447 t13.3013791.166094 1.8 0.074 -.030017.63775 t14.3004686.15361 1.96 0.055 -.0061606.6070978 t15.3191137.1474883.16 0.034.04759.6135015 output.817487.031851 5.66 0.000.7536739.880835 fuel.16861.163478 1.03 0.306 -.1576935.4949135 load -.88814.617373-3.37 0.001-1.40544 -.3603843 _cons 1.6093.07430 6.08 0.000 8.480603 16.7615 You may run the following SAS REG procedure to get the same result (output is skipped). PROC REG DATA=masil.airline; /* LSDV3 */ MODEL cost = g1-g5 t1-t15 output fuel load; RESTRICT t1+t+t3+t4+t5+t6+t7+t8+t9+t10+t11+t1+t13+t14+t15=0; RUN; 6.4 LSDV3 with Two Restrictions The third approach includes all group and time dummies and imposes two restrictions on group and time dummies.. cnsreg cost g1-g6 t1-t15 output fuel load, constraint( 3) Constrained linear regression Number of obs = 90 F(, 67) = 1960.8 Prob > F = 0.0000 Root MSE =.05138 ( 1) g1 + g + g3 + g4 + g5 + g6 = 0 ( ) t1 + t + t3 + t4 + t5 + t6 + t7 + t8 + t9 + t10 + t11 + t1 + t13 + t14 + t15 = 0 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- g1.18364.046016.79 0.007.0364849.01679 g.0654947.0389685 1.68 0.097 -.01867.143761 g3 -.1894671.0156096-1.14 0.000 -.064 -.158310 g4.13456.0183163 7.33 0.000.097693.170811 g5 -.096504.0373085 -.48 0.016 -.1671184 -.018184 g6 -.0459561.0416069-1.10 0.73 -.190038.0370916 t1 -.374045.19187-1.95 0.055 -.757006.0089536 t -.31938.1860877-1.7 0.091 -.6907554.051097 t3 -.766893.1833501-1.51 0.136 -.646576.089789 t4 -.30399.179671-1.9 0.0 -.568837.1038 t5 -.153991.0864404-1.78 0.079 -.364649.0186066 t6 -.1080904.0448591 -.41 0.019 -.197696 -.0185513 t7 -.0768646.0319336 -.41 0.019 -.1406043 -.013148 t8 -.00736.004506-1.01 0.314 -.06155.000869 t9.04705.0908 1.6 0.109 -.010878.105688 t10.091781.081155 1.13 0.6 -.070531.53709 t11.073105.1491443 1.39 0.169 -.090389.5050039 t1.85477.1756365 1.63 0.109 -.0650993.6360447 t13.3013791.166094 1.8 0.074 -.030017.63775 t14.3004686.15361 1.96 0.055 -.0061606.6070978 t15.3191137.1474883.16 0.034.04759.6135015 output.817487.031851 5.66 0.000.7536739.880835 fuel.16861.163478 1.03 0.306 -.1576935.4949135 load -.88814.617373-3.37 0.001-1.40544 -.3603843 _cons 1.66688.081068 6.09 0.000 8.513054 16.8071 The following SAS REG procedure gives you the same result (output is skipped). PROC REG DATA=masil.airline;

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 48 MODEL cost = g1-g6 t1-t15 output fuel load; RESTRICT g1 + g + g3 + g4 + g5 + g6 = 0; RESTRICT t1+t+t3+t4+t5+t6+t7+t8+t9+t10+t11+t1+t13+t14+t15=0; RUN; 6.5 Two-way Within Effect Model The two-way within group and time effect model requires a transformation of the data set as * * yit = yit yi y t + y and xit = xit xi x t + x. The following commands do this task.. gen w_cost = cost - gm_cost - tm_cost + m_cost. gen w_output = output - gm_output - tm_output + m_output. gen w_fuel = fuel - gm_fuel - tm_fuel + m_fuel. gen w_load = load - gm_load - tm_load + m_load. tabstat cost output fuel load, stat(mean) stats cost output fuel load ---------+---------------------------------------- mean 13.36561-1.174309 1.77036.560460 -------------------------------------------------- Now, run the OLS with the transformed variables. Do not forget to suppress the intercept.. regress w_cost w_output w_fuel w_load, noc // within effect Source SS df MS Number of obs = 90 -------------+------------------------------ F( 3, 87) = 307.86 Model 1.87739643 3.65798811 Prob > F = 0.0000 Residual.176848774 87.0003745 R-squared = 0.9139 -------------+------------------------------ Adj R-squared = 0.9109 Total.054451 90.084947 Root MSE =.04509 w_cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- w_output.817487.07951 9.4 0.000.761697.878048 w_fuel.16861.143461 1.18 0.43 -.1165364.4537565 w_load -.88814.96907-3.84 0.000-1.339349 -.4679 Note again that R, MSE, standard errors, and DF error are not correct. The dummy variable * * coefficients are computed as d g = ( y g y ) b'( xg x ) and d = ( y t y ) b'( x t x ) The standard errors also need to be adjusted; for instance, the standard error of the load factor is.617=.97*sqrt(87/67). t. 6.6 Using the TSCSREG and PANEL Procedures The SAS TSCSREG and PANEL procedures have the /FIXTWO option to fit the two-way fixed effect model. PROC TSCSREG DATA=masil.airline; ID airline year; MODEL cost = output fuel load /FIXTWO; RUN;

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 49 The TSCSREG Procedure Dependent Variable: cost Model Description Estimation Method FixTwo Number of Cross Sections 6 Time Series Length 15 Fit Statistics SSE 0.1768 DFE 67 MSE 0.006 Root MSE 0.0514 R-Square 0.9984 F Test for No Fixed Effects Num DF Den DF F Value Pr > F 19 67 3.10 <.0001 Parameter Estimates Standard Variable DF Estimate Error t Value Pr > t Label CS1 1 0.17483 0.0861.0 0.0470 Cross Sectional Effect 1 CS 1 0.111451 0.0780 1.43 0.1575 Cross Sectional Effect CS3 1-0.14351 0.0519 -.77 0.0073 Cross Sectional Effect 3 CS4 1 0.18009 0.031 5.61 <.0001 Cross Sectional Effect 4 CS5 1-0.04669 0.05 -.08 0.0415 Cross Sectional Effect 5 TS1 1-0.69314 0.3378 -.05 0.0441 Time Series Effect 1 TS 1-0.63844 0.331-1.9 0.0588 Time Series Effect TS3 1-0.5958 0.394-1.81 0.0750 Time Series Effect 3 TS4 1-0.5415 0.3189-1.70 0.0938 Time Series Effect 4 TS5 1-0.47304 0.319 -.04 0.0454 Time Series Effect 5 TS6 1-0.47 0.1884 -.7 0.066 Time Series Effect 6 TS7 1-0.39598 0.1733 -.8 0.055 Time Series Effect 7 TS8 1-0.33985 0.1501 -.6 0.068 Time Series Effect 8

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 50 TS9 1-0.7189 0.1348 -.0 0.0477 Time Series Effect 9 TS10 1-0.739 0.0763 -.98 0.0040 Time Series Effect 10 TS11 1-0.1118 0.0319-3.50 0.0008 Time Series Effect 11 TS1 1-0.03364 0.049-0.78 0.4357 Time Series Effect 1 TS13 1-0.01773 0.0363-0.49 0.663 Time Series Effect 13 TS14 1-0.01865 0.0305-0.61 0.543 Time Series Effect 14 Intercept 1 1.94004.18 5.83 <.0001 Intercept output 1 0.81749 0.0319 5.66 <.0001 fuel 1 0.16861 0.1635 1.03 0.3061 load 1-0.8881 0.617-3.37 0.001 The STATA.xtreg command does not fit the two-way fixed or random effect model. The following LIMDEP command fits the two-way fixed model. Note that this command has Str$ and Period$ specifications to specify stratification and time variables. This command presents the pooled model and one-way group effect model as well, but reports the incorrect intercept in the two-way fixed model, 1.667 (.081). REGRESS;Lhs=COST;Rhs=ONE,OUTPUT,FUEL,LOAD;Panel;Str=AIRLINE;Period=YEAR;Fixed$ 6.7 Testing Fixed Group and Time Effects The null hypothesis is that parameters of group and time dummies are zero: H 0 : μ 1 =... = μ n 1 = 0 and τ 1 =... = τ T 1 = 0. The F test compares the pooled regression and two-way group and time effect model. The F statistic of 3.1085 rejects the null hypothesis at the.01 significance level (p<.0000). (1.3354.1768) (6 + 15 ) (.1768) (6*15 6 15 3 + 1) ~ 3.1085[19,67] The SAS TSCSREG and PANEL procedures conduct the F-test for the group and time effects. You may also run the following SAS REG procedure and.regress command to perform the same test. PROC REG DATA=masil.airline; MODEL cost = g1-g5 t1-t14 output fuel load; TEST g1=g=g3=g4=g5=t1=t=t3=t4=t5=t6=t7=t8=t9=t10=t11=t1=t13=t14=0; RUN;. quietly regress cost g1-g5 t1-t14 output fuel load. test g1 g g3 g4 g5 t1 t t3 t4 t5 t6 t7 t8 t9 t10 t11 t1 t13 t14

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 51 7. Random Effect Models The random effects model examines how group and/or time affect error variances. This model is appropriate for n individuals who were drawn randomly from a large population. This chapter focuses on the feasible generalized least squares (FGLS) with variance component estimation methods from Baltagi and Chang (1994), Fuller and Battese (1974), and Wansbeek and Kapteyn (1989). 8 7.1 The One-way Random Group Effect Model When the omega matrix is not known, you have to estimateθ using the SSEs of the pooled model (.0317) and the fixed effect model (.96). ˆ ε The variance component of errorσ is.0036163 =.9687/(6*15-6-3) The variance component of group ˆ σ u is.0155971 =.03167596/(6-4) -.0036163/15 Thus, θˆ is.87668488 = 1.0036163 15*.03167596/(6-4) Now, transform the dependent and independent variables including the intercept.. gen rg_cost = cost -.87668488*gm_cost // transform variables. gen rg_output = output -.87668488*gm_output. gen rg_fuel = fuel -.87668488*gm_fuel. gen rg_load = load -.87668488*gm_load. gen rg_int = 1 -.87668488 // for the intercept Finally, run the OLS with the transformed variables. Do not forget to suppress the intercept. This is the groupwise heteroscedastic regression model (Greene 003).. regress rg_cost rg_int rg_output rg_fuel rg_load, noc Source SS df MS Number of obs = 90 -------------+------------------------------ F( 4, 86) =1964.7 Model 84.670313 4 71.1675783 Prob > F = 0.0000 Residual.311586777 86.0036310 R-squared = 0.9989 -------------+------------------------------ Adj R-squared = 0.9989 Total 84.9819 90 3.16646556 Root MSE =.06019 rg_cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- rg_int 9.67911.101638 45.81 0.000 9.10119 10.0457 8 Baltagi and Cheng (1994) introduce various ANOVA estimation methods, such as a modified Wallace and Hussain method, the Wansbeek and Kapteyn method, the Swamy and Arora method, and Henderson s method III. They also discuss maximum likelihood (ML) estimators, restricted ML estimators, minimum norm quadratic unbiased estimators (MINQUE), and minimum variance quadratic unbiased estimators (MIVQUE). Based on a Monte Carlo simulation, they argue that ANOVA estimators are Best Quadratic Unbiased estimators of the variance components for the balanced model, whereas ML, restricted ML, MINQUE, and MIVQUE are recommended for the unbalanced models.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 5 rg_output.9066808.05649 35.38 0.000.8557401.957615 rg_fuel.47784.014048 30.15 0.000.394898.4506587 rg_load -1.0645.000703-5.3 0.000-1.466 -.6667731 7. Estimations in SAS, STATA, and LIMDEP The SAS TSCSREG and PANEL procedures have the /RANONE option to fit the one-way random effect model. These procedures by default use the Fuller and Battese (1974) estimation method, which produces slightly different estimates from FGLS. PROC TSCSREG DATA=masil.airline; ID airline year; MODEL cost = output fuel load /RANONE; RUN; The TSCSREG Procedure Dependent Variable: cost Model Description Estimation Method RanOne Number of Cross Sections 6 Time Series Length 15 Fit Statistics SSE 0.3090 DFE 86 MSE 0.0036 Root MSE 0.0599 R-Square 0.993 Variance Component Estimates Variance Component for Cross Sections 0.018198 Variance Component for Error 0.003613 Hausman Test for Random Effects DF m Value Pr > m 3 0.9 0.809 Parameter Estimates Standard Variable DF Estimate Error t Value Pr > t Intercept 1 9.637 0.13 45.1 <.0001 output 1 0.90804 0.060 34.91 <.0001

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 53 fuel 1 0.4199 0.0141 9.95 <.0001 load 1-1.06469 0.1995-5.34 <.0001 The PANEL procedure has the /VCOMP=WK option for the Wansbeek and Kapteyn (1989) method, which is close to groupwise heteroscedastic regression. The BP option of the MODEL statement, not available in the TSCSREG procedure, conducts the Breusch-Pagen LM test for random effects. Note that two procedures estimate the same variance component for error (.0036) but a different variance component for groups (.018 versus.0160), PROC PANEL DATA=masil.airline; ID airline year; MODEL cost = output fuel load /RANONE BP VCOMP=WK; RUN; The PANEL Procedure Wansbeek and Kapteyn Variance Components (RanOne) Dependent Variable: cost Model Description Estimation Method RanOne Number of Cross Sections 6 Time Series Length 15 Fit Statistics SSE 0.3111 DFE 86 MSE 0.0036 Root MSE 0.0601 R-Square 0.993 Variance Component Estimates Variance Component for Cross Sections 0.016015 Variance Component for Error 0.003613 Hausman Test for Random Effects DF m Value Pr > m 1.63 0.449 Breusch Pagan Test for Random Effects (One Way) DF m Value Pr > m 1 334.85 <.0001 Parameter Estimates

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 54 Standard Variable DF Estimate Error t Value Pr > t Intercept 1 9.69513 0.107 45.71 <.0001 output 1 0.906918 0.057 35.30 <.0001 fuel 1 0.4676 0.0140 30.11 <.0001 load 1-1.0645 0.000-5.3 <.0001 The STATA.xtreg command has the re option to produce FGLS estimates. The.iis command specifies the panel identification variable, such as a grouping or cross-section variable that is used in the i() option.. iis airline. xtreg cost output fuel load, re i(airline) theta Random-effects GLS regression Number of obs = 90 Group variable (i): airline Number of groups = 6 R-sq: within = 0.995 Obs per group: min = 15 between = 0.9856 avg = 15.0 overall = 0.9876 max = 15 Random effects u_i ~ Gaussian Wald chi(3) = 11091.33 corr(u_i, X) = 0 (assumed) Prob > chi = 0.0000 theta =.87668503 cost Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- output.9066805.0565 35.38 0.000.8564565.9569045 fuel.47784.014048 30.15 0.000.395904.450665 load -1.064499.000703-5.3 0.000-1.45669 -.67368 _cons 9.67909.10164 45.81 0.000 9.15995 10.0398 -------------+---------------------------------------------------------------- sigma_u.1488859 sigma_e.06010514 rho.81193816 (fraction of variance due to u_i) The theta option reports the estimated theta (.8767). The sigma_u and sigma_e are square roots of the variance components for groups and errors (.0036=.0601^). In LIMDEP, you have to specify Panel$ and Het$ subcommands for the groupwise heteroscedastic model. Note that LIMDEP presents the pooled OLS regression and least square dummy variable model as well. --> REGRESS;Lhs=COST;Rhs=ONE,OUTPUT,FUEL,LOAD;Panel;Str=AIRLINE;Het=AIRLINE$ +-----------------------------------------------------------------------+ OLS Without Group Dummy Variables Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.36560933, S.D.= 1.131971444 Model size: Observations = 90, Parameters = 4, Deg.Fr.= 86 Residuals: Sum of squares= 1.3354495, Std.Dev.=.1461 Fit: R-squared=.98890, Adjusted R-squared =.98788 Model test: F[ 3, 86] = 419.33, Prob value =.00000 Diagnostic: Log-L = 61.7699, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -4.1, Akaike Info. Crt.= -1.84

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 55 Panel Data Analysis of COST [ONE way] Unconditional ANOVA (No regressors) Source Variation Deg. Free. Mean Square Between 74.6799 5. 14.9360 Residual 39.3611 84..468584 Total 114.041 89. 1.8136 +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+ OUTPUT.887386341.135455E-01 66.599.0000-1.174309 FUEL.4539777119.030440E-01.359.0000 1.770359 LOAD -1.67507797.3453093-4.713.0000.56046016 Constant 9.5169131.945 41.514.0000 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +-----------------------------------------------------------------------+ Least Squares with Group Dummy Variables Ordinary least squares regression Weighting variable = none Dep. var. = COST Mean= 13.36560933, S.D.= 1.131971444 Model size: Observations = 90, Parameters = 9, Deg.Fr.= 81 Residuals: Sum of squares=.9607777, Std.Dev.=.06010 Fit: R-squared=.997434, Adjusted R-squared =.99718 Model test: F[ 8, 81] = 3935.8, Prob value =.00000 Diagnostic: Log-L = 130.0865, Restricted(b=0) Log-L = -138.3581 LogAmemiyaPrCrt.= -5.58, Akaike Info. Crt.= -.691 Estd. Autocorrelation of e(i,t).573531 White/Hetero. corrected covariance matrix used. +-----------------------------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error t-ratio P[ T >t] Mean of X +---------+--------------+----------------+--------+---------+----------+ OUTPUT.91988143.19105357E-01 48.117.0000-1.174309 FUEL.4174910457.1353534E-01 30.851.0000 1.770359 LOAD -1.070395015.166097-4.941.0000.56046016 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +------------------------------------------------------------------------+ Test Statistics for the Classical Model Model Log-Likelihood Sum of Squares R-squared (1) Constant term only -138.35814.114040981D+03.0000000 () Group effects only -90.48804.3936109461D+0.6548513 (3) X - variables only 61.76991.13354495D+01.988897 (4) X and group effects 130.08647.9607777D+00.9974341 Hypothesis Tests Likelihood Ratio Test F Tests Chi-squared d.f. Prob. F num. denom. Prob value () vs (1) 95.740 5.00000 31.875 5 84.00000 (3) vs (1) 400.56 3.00000 419.39 3 86.00000 (4) vs (1) 536.889 8.00000 3935.818 8 81.00000 (4) vs () 441.149 3.00000 3604.83 3 81.00000 (4) vs (3) 136.633 5.00000 57.733 5 81.00000 +------------------------------------------------------------------------+ Error: 45: REGR;PANEL. Could not invert VC matrix for Hausman test.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 56 +--------------------------------------------------+ Random Effects Model: v(i,t) = e(i,t) + u(i) Estimates: Var[e] =.36160D-0 Var[u] =.119159D-01 Corr[v(i,t),v(i,s)] =.767356 Lagrange Multiplier Test vs. Model (3) = 334.85 ( 1 df, prob value =.000000) (High values of LM favor FEM/REM over CR model.) Fixed vs. Random Effects (Hausman) =.00 ( 3 df, prob value = 1.000000) (High (low) values of H favor FEM (REM).) Reestimated using GLS coefficients: Estimates: Var[e] =.36491D-0 Var[u] =.39309D-01 Var[e] above is an average. Groupwise heteroscedasticity model was estimated. Sum of Squares.147779D+01 +--------------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ OUTPUT.904138041.4615477E-01 36.730.0000-1.174309 FUEL.438986905.13746498E-01 30.837.0000 1.770359 LOAD -1.064558659.1993313-5.341.0000.56046016 Constant 9.610634379.077404 47.396.0000 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) Like SAS TSCSREG and PANEL procedures, LIMDEP estimates a slightly different variance component for groups (.0119), thus producing different parameter estimates. In addition, the Hausman test is not successful in this example. 7.3 The One-way Random Time Effect Model Let us computeθˆ using the SSEs of the between effect model (.0056) and the fixed effect model (1.088). The variance component for error ˆ σ ε is.01511375 = 1.088190/(15*6-15-3) The variance component for time ˆ σ v is -.000107 =.005590631/(15-4)-.01511375/6 Theθˆ is -1.663 = 1.01511375 6*.005590631/(15-4). gen rt_cost = cost - (-1.663)*tm_cost // transform variables. gen rt_output = output - (-1.663)*tm_output. gen rt_fuel = fuel - (-1.663)*tm_fuel. gen rt_load = load - (-1.663)*tm_load. gen rt_int = 1 - (-1.663) // for the intercept. regress rt_cost rt_int rt_output rt_fuel rt_load, noc

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 57 Source SS df MS Number of obs = 90 -------------+------------------------------ F( 4, 86) =. Model 79944.1804 4 19986.0451 Prob > F = 0.0000 Residual 1.7971995 86.00845581 R-squared = 1.0000 -------------+------------------------------ Adj R-squared = 1.0000 Total 79945.973 90 888.88591 Root MSE =.14438 rt_cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- rt_int 9.516098.148981 63.90 0.000 9.0038 9.81157 rt_output.8883838.0143338 61.98 0.000.8598891.9168785 rt_fuel.439731.019051 34.04 0.000.4136186.464977 rt_load -1.79176.48869-5.15 0.000-1.77754 -.785598 However, the negative value of the variance component for time is not likely. This section presents examples of procedures and commands for the one-way time random effect model without outputs. In SAS, use the TSCSREG or PANEL procedure with the /RANONE option. PROC SORT DATA=masil.airline; BY year airline; PROC TSCSREG DATA=masil.airline; ID year airline; MODEL cost = output fuel load /RANONE; RUN; PROC PANEL DATA=masil.airline; ID year airline; MODEL cost = output fuel load /RANONE BP; RUN; In STATA, you have to switch the grouping and time variables using the.tsset command.. tsset year airline panel variable: year, 1 to 15 time variable: airline, 1 to 6. xtreg cost output fuel load, re i(year) theta In LIMDEP, you need to use the Period$ and Random$ subcommands. REGRESS;Lhs=COST;Rhs=ONE,OUTPUT,FUEL,LOAD;Panel;Pds=15;Het=YEAR$ 7.4 The Two-way Random Effect Model in SAS The random group and time effect model is formulated as yit = α + β ' X ti + ui + γ t + ε it. Let us first estimate the two way FGLS using the SAS PANEL procedure with the /RANTWO option. The BP option conducts the Breusch-Pagan LM test for the two-way random effect model. PROC PANEL DATA=masil.airline; ID airline year; MODEL cost = output fuel load /RANTWO BP;

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 58 RUN; The PANEL Procedure Fuller and Battese Variance Components (RanTwo) Dependent Variable: cost Model Description Estimation Method RanTwo Number of Cross Sections 6 Time Series Length 15 Fit Statistics SSE 0.3 DFE 86 MSE 0.007 Root MSE 0.050 R-Square 0.989 Variance Component Estimates Variance Component for Cross Sections 0.017439 Variance Component for Time Series 0.001081 Variance Component for Error 0.0064 Hausman Test for Random Effects DF m Value Pr > m 3 6.93 0.0741 Breusch Pagan Test for Random Effects (Two Way) DF m Value Pr > m 336.40 <.0001 Parameter Estimates Standard Variable DF Estimate Error t Value Pr > t Intercept 1 9.36677 0.440 38.38 <.0001 output 1 0.866448 0.055 33.98 <.0001 fuel 1 0.436163 0.017 5.41 <.0001 load 1-0.98053 0.35-4.39 <.0001 Similarly, you may run the TSCSREG procedure with the /RANTWO option.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 59 PROC TSCSREG DATA=masil.airline; ID airline year; MODEL cost = output fuel load /RANTWO; RUN; 7.5 Testing Random Effect Models The Breusch-Pagan Lagrange multiplier (LM) test is designed to test random effects. The null hypothesis of the one-way random group effect model is that variances of groups are zero: H 0 : σ u = 0. If the null hypothesis is not rejected, the pooled regression model is appropriate. The e e of the pooled OLS is 1.33544153 and e' e is.0665147. 6*15 15 *.0665 LM is 334.8496= 1 ~ χ (1) (15 1) 1.3354 with p <.0000. With the large chi-squared, we reject the null hypothesis in favor of the random group effect model. The SAS PANEL procedure with the /BP option and the LIMDEP Panel$ and Het$ subcommands report the LM statistic. In STATA, run the.xttest0 command right after estimating the one-way random effect model.. quietly xtreg cost output fuel load, re i(airline). xttest0 Breusch and Pagan Lagrangian multiplier test for random effects: cost[airline,t] = Xb + u[airline] + e[airline,t] Estimated results: Var sd = sqrt(var) ---------+----------------------------- cost 1.81358 1.131971 e.003616.0601051 u.015597.148886 Test: Var(u) = 0 chi(1) = 334.85 Prob > chi = 0.0000 The null hypothesis of the one-way random time effect is that variance components for time are zero, H 0 : σ v = 0. The following LM test uses Baltagi s formula. The small chi-squared of 1.547 does not reject the null hypothesis at the.01 level. LM is 1.547 = Tn ( n 1) ( ne ) t e it 1 = 15*6.7817 1 (6 1) 1.3354 ~ χ (1) with p<.135. quietly xtreg cost output fuel load, re i(year). xttest0 Breusch and Pagan Lagrangian multiplier test for random effects:

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 60 cost[year,t] = Xb + u[year] + e[year,t] Estimated results: Var sd = sqrt(var) ---------+----------------------------- cost 1.81358 1.131971 e.0151138.1938 u 0 0 Test: Var(u) = 0 chi(1) = 1.55 Prob > chi = 0.135 The two way random effects model has the null hypothesis that variance components for groups and time are all zero. The LM statistic with two degrees of freedom is 336.3968 = 334.8496 + 1.547 (p<.0001). 7.6 Fixed Effects versus Random Effects How do we compare a fixed effect model and its counterpart random effect model? The Hausman specification test examines if the individual effects are uncorrelated with the other regressors in the model. Since computation is complicated, let us conduct the test in STATA.. tsset airline year panel variable: airline, 1 to 6 time variable: year, 1 to 15. quietly xtreg cost output fuel load, fe. estimates store fixed_group. quietly xtreg cost output fuel load, re. hausman fixed_group. ---- Coefficients ---- (b) (B) (b-b) sqrt(diag(v_b-v_b)) fix_group. Difference S.E. -------------+---------------------------------------------------------------- output.919846.9066805.016041.0153877 fuel.4174918.47784 -.005867.0058583 load -1.070396-1.064499 -.0058974.055088 b = consistent under Ho and Ha; obtained from xtreg B = inconsistent under Ha, efficient under Ho; obtained from xtreg Test: Ho: difference in coefficients not systematic chi(3) = (b-b)'[(v_b-v_b)^(-1)](b-b) =.1 Prob>chi = 0.5469 (V_b-V_B is not positive definite) The Hausman statistic.1 is different from the PANEL procedure s 1.63 and Greene (003) s 4.16. It is because SAS, STATA, and LIMDEP use different estimation methods to produce slightly different parameter estimates. These tests, however, do not reject the null hypothesis in favor of the random effect model.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 61 7.7 Summary Table 7 summarizes random effect estimations in SAS, STATA, and LIMDEP. The SAS PANEL procedure is highly recommended. Table 7 Comparison of the Random Effect Model in SAS, STATA, LIMDEP * SAS 9.1 STATA 9.0 LIMDEP 8.0 Procedure/Command PROC TSCSREG PROC PANEL. xtreg Regress; Panel$ One-way /RANONE /RANONE WK re Str=;Pds=;Het;Random$ Two-way /RANTWO /RANTWO No Problematic SSE (e e) Slightly different Correct No No MSE or SEE Slightly different Correct No No Model test (F) No No Wald test No (adjusted) R Slightly different Slightly different Incorrect No Intercept Slightly different Correct Correct Slightly different Coefficients Slightly different Correct Correct Slightly different Standard errors Slightly different Correct Correct Slightly different Variance for group Slightly different Correct Correct (sigma) Slightly different Variance for error Correct Correct Correct (sigma) Correct Theta No No theta No Breusch-Pagan (LM) No BP option. xttest0 Yes Hausman Test (H) Incorrect Yes. hausman Yes (unstable) * Yes/No means whether the software reports the statistics. Correct/incorrect indicates whether the statistics are different from those of the groupwise heteroscedastic regression.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 6 8. The Poolability Test In order to conduct the poolability test, you need to run group by group OLS regressions and/or time by time OLS regressions. If the null hypothesis is rejected, the panel data are not poolable. In this case, you may consider the random coefficient model and hierarchical regression model. 8.1 Group by Group OLS Regression In SAS, use the BY statement in the REG procedure. Do not forget to sort the data set in advance. PROC SORT DATA=masil.airline; BY airline; PROC REG DATA=masil.airline; MODEL cost = output fuel load; BY airline; RUN; In STATA, the if qualifier makes it easy to run group by group regressions.. forvalues i= 1(1)6 { // run group by group regression display "OLS regression for group " `i' regress cost output fuel load if airline==`i' } OLS regression for group 1 Source SS df MS Number of obs = 15 -------------+------------------------------ F( 3, 11) = 1843.46 Model 3.4184348 3 1.13941449 Prob > F = 0.0000 Residual.006798918 11.000618083 R-squared = 0.9980 -------------+------------------------------ Adj R-squared = 0.9975 Total 3.45044 14.44645886 Root MSE =.0486 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- output 1.18318.0968946 1.1 0.000.9699164 1.396444 fuel.3865867.0181946 1.5 0.000.3465406.46639 load -.46169.4013571-6.13 0.000-3.34501-1.57848 _cons 10.846.97551 36.49 0.000 10.19174 11.5005 OLS regression for group Source SS df MS Number of obs = 15 -------------+------------------------------ F( 3, 11) = 319.50 Model 6.476084 3.1587408 Prob > F = 0.0000 Residual.007587838 11.000689803 R-squared = 0.9988 -------------+------------------------------ Adj R-squared = 0.9985 Total 6.48380868 14.46319191 Root MSE =.066 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- output 1.459104.079856 18.40 0.000 1.84597 1.63361 fuel.3088958.07443 11.34 0.000.489315.36886 load -.74785.3765-11.47 0.000-3.47854 -.01716 _cons 11.9743.430951 7.71 0.000 11.0139 1.9346

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 63 OLS regression for group 3 Source SS df MS Number of obs = 15 -------------+------------------------------ F( 3, 11) = 608.10 Model 3.7986673 3 1.648891 Prob > F = 0.0000 Residual.0869767 11.0007907 R-squared = 0.9940 -------------+------------------------------ Adj R-squared = 0.994 Total 3.8157365 14.755607 Root MSE =.0456 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- output.768305.1554418 4.68 0.001.3847054 1.068956 fuel.451517.0381103 11.85 0.000.367634.535399 load -.7513069.6105989-1.3 0.44 -.0956.5961 _cons 8.699815.8985786 9.68 0.000 6.7057 10.67757 OLS regression for group 4 Source SS df MS Number of obs = 15 -------------+------------------------------ F( 3, 11) = 777.86 Model 7.375558 3.45750853 Prob > F = 0.0000 Residual.03475343 11.003159304 R-squared = 0.9953 -------------+------------------------------ Adj R-squared = 0.9940 Total 7.407779 14.590918 Root MSE =.0561 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- output.9353749.075966 1.3 0.000.768616 1.10488 fuel.463763.044347 10.46 0.000.366119.5613333 load -.7756708.470786-1.65 0.18-1.811856.605148 _cons 9.164608.60341 15. 0.000 7.83890 10.49031 OLS regression for group 5 Source SS df MS Number of obs = 15 -------------+------------------------------ F( 3, 11) = 1999.89 Model 7.08313716 3.3610457 Prob > F = 0.0000 Residual.01986435 11.001180585 R-squared = 0.998 -------------+------------------------------ Adj R-squared = 0.9977 Total 7.0961359 14.506865971 Root MSE =.03436 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- output 1.07699.077155 13.96 0.000.9065471 1.46051 fuel.9054.043413 6.73 0.000.1964845.387639 load -1.06847.3336308-3.6 0.004-1.941163 -.475305 _cons 11.77079.7430078 15.84 0.000 10.13544 13.40614 OLS regression for group 6 Source SS df MS Number of obs = 15 -------------+------------------------------ F( 3, 11) = 60.49 Model 11.1173565 3 3.70578551 Prob > F = 0.0000 Residual.01566333 11.00143938 R-squared = 0.9986 -------------+------------------------------ Adj R-squared = 0.998 Total 11.1330199 14.79515705 Root MSE =.03774 cost Coef. Std. Err. t P> t [95% Conf. Interval] -------------+---------------------------------------------------------------- output.9673393.03178 30.07 0.000.896575 1.038151 fuel.30358.030835 9.81 0.000.344839.3701678 load.105038.4767508 0. 0.830 -.944886 1.154354

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 64 _cons 10.77381.409591 6.30 0.000 9.87309 11.6753 8. Poolability Test across Groups The null hypothesis of the poolability test across groups is H 0 : β ik = β k. The e e is 1.3354, the SSE of the pooled OLS regression. The e ie ' i is.1007 =.0068 +.0076 +.09 +.0348 +.0130 +.0157. (1.3354.1007 (6 1)4.1007 6(15 4) Thus, the F statistic is ~ 40.481[ 0,66] The large 40.481 rejects the null hypothesis of poolability (p<.0000). We conclude that the panel data are not poolable with respect to group. 8.3 Poolability Test over Time The null hypothesis of the poolability test over time is computed from the 15 time by time regression. H : β = β 0 tk k. The sum of e ' t te is. di.044807673 +.03093978 +.016506613 +.01170358 +.01410454 + ///.00046986 +.063648817 +.08543085 +.04939439 +.07711957 + ///.09913538 +.08740016 +.14334897 +.066075346 +.0375616.7505079 The F statistic is.4175[ 84,30] (1.3354.7505) (15 1)4 =.7505 15(6 4) The small F statistic does not reject the null hypothesis in favor of poolable panel data with respect to time (p<.9991).

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 65 9. Conclusion Panel data models investigate group and time effects using fixed effect and random effect models. The fixed effect models ask how group and/or time affect the intercept, while the random effect models analyze error variance structures affected by group and/or time. Slopes are assumed unchanged in both fixed effect and random effect models. Fixed effect models are estimated by least squares dummy variable (LSDV) regression, the within effect model, and the between effect model. LSDV has three approaches to avoid perfect multicollinearity. LSDV1 drops a dummy, LSDV suppresses the intercept, and LSDV3 includes all dummies and imposes restrictions instead. LSDV1 is commonly used since it produces correct statistics. LSDV provides actual parameter estimates of group intercepts, but reports incorrect R and F statistic. Note that the dummy parameters of three LSDV approaches have different meanings and thus different t-tests. The within effect model does not use dummy variables but deviations from the group means. Thus, this model is useful when there are many groups and/or time periods in the panel data set (no incidental parameter problem at all). The dummy parameter estimates need to be computed afterward. Because of its larger degrees of freedom, the within effect model produces incorrect MSE and standard errors of parameters. As a result, you need to adjust the standard errors to conduct the correct t-tests. Random effect models are estimated by the generalized least squares (GLS) and the feasible generalization least squares (FGLS). When the variance structure is known, GLS is used. If unknown, FGLS estimates theta. Parameter estimates may vary depending on estimation methods. Fixed effects are tested by the F-test and random effects by the Breusch-Pagan Lagrange multiplier test. The Hausman specification test compares a fixed effect model and a random effect model. If the null hypothesis of uncorrelation is rejected, the fixed effect model is preferred. Poolabiltiy is tested by running group by group or time by time regressions. Among the four statistical packages addressed in this document, I would recommend SAS and STATA. In particular, the SAS PANEL procedure, although experimental now, provides various ways of analyzing panel data. STATA is very handy to manipulate panel data, but it does not fit two-way effect models. LIMDEP is able to estimate various panel data models, but it is not stable enough. SPSS is not recommended for panel data models.

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 66 APPENDIX: Data sets Data set 1: Data of the top 50 information technology firms presented in OECD Information Technology Outlook 004 (http://thesius.sourceoecd.org/). firm = IT company name type = type of IT firm rnd = 00 R&D investment in current USD millions income = 000 net income in current USD millions d1 = 1 for equipment and software firms and 0 for telecommunication and electronics. tab type d1 d1 Type of Firm 0 1 Total ----------------+----------------------+---------- Telecom 18 0 18 Electronics 17 0 17 IT Equipment 0 6 6 Comm. Equipment 0 5 5 Service & S/W 0 4 4 ----------------+----------------------+---------- Total 35 15 50. sum rnd income Variable Obs Mean Std. Dev. Min Max -------------+-------------------------------------------------------- rnd 39 03.564 1615.417 0 5490 income 50 509.78 3104.585-73 11797 Data set : Cost data for U.S. airlines (1970-1984) presented in Greene (003). URL: http://pages.stern.nyu.edu/~wgreene/text/tables/tablelist5.htm airline = airline (six airlines) year = year (fifteen years) output0 = output in revenue passenger miles, index number cost0 = total cost in $1,000 fuel0 = fuel price load = load factor, the average capacity utilization of the fleet. tsset panel variable: airline, 1 to 6 time variable: year, 1 to 15. sum output0 cost0 fuel0 load Variable Obs Mean Std. Dev. Min Max -------------+-------------------------------------------------------- output0 90.5449946.5335865.03768 1.93646 cost0 90 1154 119075 68978 474830 fuel0 90 471683 3950.9 103795 1015610 load 90.560460.057934.43066.67687

005 The Trustees of Indiana University (1/10/005) Linear Regression Model for Panel Data: 67 References Baltagi, Badi H. 001. Econometric Analysis of Panel Data. Wiley, John & Sons. Baltagi, Badi H., and Young-Jae Chang. 1994. "Incomplete Panels: A Comparative Study of Alternative Estimators for the Unbalanced One-way Error Component Regression Model." Journal of Econometrics, 6(): 67-89. Breusch, T. S., and A. R. Pagan. 1980. "The Lagrange Multiplier Test and its Applications to Model Specification in Econometrics." Review of Economic Studies, 47(1):39-53. Fox, John. 1997. Applied Regression Analysis, Linear Models, and Related Methods. Newbury Park, CA: Sage. Freund, Rudolf J., and Ramon C. Littell. 000. SAS System for Regression, 3 rd ed. Cary, NC: SAS Institute. Fuller, Wayne A. and George E. Battese. 1973. "Transformations for Estimation of Linear Models with Nested-Error Structure." Journal of the American Statistical Association, 68(343) (September): 66-63. Fuller, Wayne A. and George E. Battese. 1974. "Estimation of Linear Models with Crossed- Error Structure." Journal of Econometrics, : 67-78. Greene, William H. 00. LIMDEP Version 8.0 Econometric Modeling Guide, 4th ed. Plainview, New York: Econometric Software. Greene, William H. 003. Econometric Analysis, 5th ed. Upper Saddle River, NJ: Prentice Hall. Hausman, J. A. 1978. "Specification Tests in Econometrics." Econometrica, 46(6):151-171. SAS Institute. 004. SAS/ETS 9.1 User s Guide. Cary, NC: SAS Institute. SAS Institute. 004. SAS/STAT 9.1 User s Guide. Cary, NC: SAS Institute. http://www.sas.com/ STATA Press. 005. STATA Base Reference Manual, Release 9. College Station, TX: STATA Press. STATA Press. 005. STATA Longitudinal/Panel Data Reference Manual, Release 9. College Station, TX: STATA Press. STATA Press. 005. STATA Time-Series Reference Manual, Release 9. College Station, TX: STATA Press. Wooldridge, Jeffrey M. 00. Econometric Analysis of Cross Section and Panel Data. Cambridge, MA: MIT Press. Acknowledgements I have to thank Dr. Heejoon Kang in the Kelley School of Business and Dr. David H. Good in the School of Public and Environmental Affairs, Indiana University at Bloomington, for their insightful lectures. I am also grateful to Jeremy Albright and Kevin Wilhite at the UITS Center for Statistical and Mathematical Computing for comments and suggestions. Revision History 005.11 First draft

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 1 Categorical Dependent Variable Models Using SAS, STATA, LIMDEP, and SPSS Hun Myoung Park This document summarizes regression models for categorical dependent variables and illustrates how to estimate individual models using SAS 9.1, STATA 9.0, LIMDEP 8.0, and SPSS 13.0. 1. Introduction. The Binary Logit Model 3. The Binary Probit Model 4. Bivariate Logit/Probit Models 5. Ordered Logit/Probit Models 6. The Multinomial Logit Model 7. The Conditional Logit Model 8. The Nested Logit Model 9. Conclusion 10. Appendix 1. Introduction The categorical variable here refers to a variable that is binary, ordinal, or nominal. Event count data are discrete (categorical) but often considered continuous. When the dependent variable is categorical, the ordinary least squares (OLS) method can no longer produce the best linear unbiased estimator (BLUE); that is, OLS is biased and inefficient. Consequently, researchers have developed various categorical dependent variable models (CDVMs). The nonlinearity of CDVMs makes it difficult to interpret outputs, since the effect of a change in a variable depends on the values of all other variables in the model (Long 1997). 1.1 Categorical Dependent Variable Models In CDVMs, the left-hand side (LHS) variable or dependent variable is neither interval nor ratio, but rather categorical. The level of measurement and data generation process (DGP) of a dependent variable determines the proper type of CDVM. Thus, binary responses are modeled with the binary logit and probit regressions, ordinal responses are formulated into the ordered logit/probit regression models, and nominal responses are analyzed by multinomial logit, conditional logit, or nested logit models. Independent variables on the right-hand side (RHS) may be interval, ratio, or binary (dummy). The CDVMs adopt the maximum likelihood (ML) estimation method, whereas OLS uses the moment based method. The ML method requires assumptions about probability distribution functions, such as the logistic function and the complementary log-log function. Logit models use the standard logistic probability distribution, while probit models assume the standard

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: normal distribution. This document focuses on logit and probit models only. Table 1 summarizes CDVMs in comparison with OLS. OLS CDVMs Table 1. Ordinary Least Squares and CDVMs Model Dependent (LHS) Estimation Independent (RHS) Interval or ratio Moment based method Ordinary least squares Binary response Binary (0 or 1) Ordinal response Ordinal (1 st, nd, 3 rd ) Nominal response Nominal (A, B, C ) Event count data Count (0, 1,, 3 ) Maximum likelihood method A linear function of interval/ratio or binary variables β + β X + β 0 1 1 X... 1. Logit Models versus Probit Models How do logit models differ from probit models? The core difference lies in the distribution of errors. In the logit model, errors are assumed to follow the standard logistic distribution with ε π e mean 0 and variance, λ( ε ) =. The errors of the probit model are assumed to follow ε 3 (1 + e ) 1 the standard normal distribution, φ( ε ) = e. π Figure 1. Comparison of the Standard Normal and Standard Logistic Probability Distributions ε PDF of the Standard Normal Distribution CDF of the Standard Normal Distribution PDF of the Standard Logistic Distribution CDF of the Standard Logistic Distribution The probability density function (PDF) of the standard normal probability distribution has a higher peak and thinner tails than the standard logistic probability distribution (Figure 1). The

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 3 standard logistic distribution looks as if someone has weighed down the peak of the standard normal distribution and strained its tails. As a result, the cumulative density function (CDF) of the standard normal distribution is steeper in the middle than the CDF of the standard logistic distribution and quickly approaches zero on the left and one on the right. The two models, of course, produce different parameter estimates. In binary response models, the estimates of a logit model are roughly π 3 times larger than those of the corresponding probit model. These estimators, however, are almost the same in terms of the standardized impacts of independent variables and predictions (Long 1997). In general, logit models reach convergence in estimation fairly well. Some (multinomial) probit models may take a long time to reach convergence, although the probit works well for bivariate models. 1.3 Estimation in SAS, STATA, LIMDEP, and SPSS SAS provides several procedures for CDVMs, such as LOGISTIC, PROBIT, GENMOD, QLIM, MDC, and CATMOD. Since these procedures support various models, a CDVM can be estimated by multiple procedures. For example, you may run a binary logit model using the LOGISTIC, PROBIT, GENMODE, and QLIM. The LOGISTIC and PROBIT procedures of SAS/STAT have been commonly used, but the QLIM and MDC procedures of SAS/ETS are noted for their advanced features. Table. Procedures and Commands for CDVMs Model SAS 9.1 Stata 9.0 LIMDEP 8.0 SPSS13.0 OLS (Ordinary least squares) REG.regress Regress$ Regression Binary Bivariate Ordinal Nominal Binary logit QLIM, GENMOD, LOGISTIC, PROBIT, CATMOD.logit, logistic Logit$ Logistic regression Binary probit QLIM, GENMOD, LOGISTIC, PROBIT.probit Probit$ Probit Bivariate logit QLIM - - - Bivariate probit QLIM.biprobit Bivariateprobit$ - Ordered logit QLIM, PROBIT, LOGISTIC.ologit Ordered$, Logit$ Plum Generalized logit -.gologit * - - Ordered probit QLIM, PROBIT, LOGISTIC.oprobit Ordered$ Plum Multinomial logit CATMOD.mlogit Mlogit$, Logit$ Nomreg Conditional logit MDC, PHREG.clogit Clogit$, Logit$ Coxreg Nested logit MDC.nlogit Nlogit$ ** - Multinomial probit MDC.mprobit - - * User-written commands written by Fu (1998) and Williams (005) ** The Nlogit$ command is supported by NLOGIT 3.0, which is sold separately. The QLIM (Qualitative and LImited dependent variable Model) procedure analyzes various categorical and limited dependent variable regression models such as censored, truncated, and sample-selection models. This QLIM procedure also handles Box-Cox regression and bivariate

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 4 probit and logit models. The MDC (Multinomial Discrete Choice) Procedure can estimate multinomial probit, conditional logit, and nested (multinomial) logit models. Unlike SAS, STATA has individualized commands for corresponding CDVMs. For example, the.logit and.probit commands respectively fit the binary logit and probit models. The LIMDEP Logit$ and Probit$ commands support a variety of CDVMs that are addressed in Greene s Econometric Analysis (003). SPSS supports some related commands for CDVMs but has limited ability to analyze categorical data. Because of its limitation, SPSS outputs are skipped here. Table summarizes the procedures and commands for CDVMs. 1. 4 Long and Freese s SPost Module STATA users may take advantages of user-written modules such as J. Scott Long and Jeremy Freese s SPost. The module allows researchers to conduct follow-up analyses of various CDVMs including event count data models. See section. for major SPost commands. In order to install SPost, execute the following commands consecutively. For more details, visit J. Scott Long s Web site at http://www.indiana.edu/~jslsoc/spost_install.htm.. net from http://www.indiana.edu/~jslsoc/stata/. net install spost9_ado, replace. net get spost9_do, replace If you want to use Vincent Kang Fu s gologit (000) and Richard Williams gologit (005) for the generalized ordered logit model, type in the following.. net search gologit. net install gologit from(http://www.stata.com/users/jhardin). net install gologit from(http://fmwww.bc.edu/repec/bocode/g)

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 5. The Binary Logit Regression Model exp( xβ ) The binary logit model is represented as Pr ob( y = 1 x) = = Λ( xβ ), where Λ 1+ exp( xβ ) indicates a link function, the cumulative standard logistic probability distribution function. This chapter examines how car ownership (owncar) is affected by monthly income (income), age, and gender (male). See the appendix for details about the data set..1 Binary Logit in STATA (.logit) STATA provides two equivalent commands for the binary logit model, which present the same result in different ways. The.logit command produces coefficients with respect to logit (log of odds), while the.logistic reports estimates as odd ratios.. logistic owncar income age male Logistic regression Number of obs = 437 LR chi(3) = 18.4 Prob > chi = 0.0004 Log likelihood = -73.84758 Pseudo R = 0.03 owncar Odds Ratio Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- income.989886.5677504-0.0 0.986.316431 3.046443 age 1.7966.088997 3.55 0.000 1.116561 1.466505 male 1.513669.3111388.0 0.044 1.01179.64633. logit In order to get the coefficients (log of odds), simply run the.logit without any argument right after the.logistic command. Or run an independent.logit command with all arguments.. logit owncar income age male Iteration 0: log likelihood = -8.9651 Iteration 1: log likelihood = -73.93537 Iteration : log likelihood = -73.84761 Iteration 3: log likelihood = -73.84758 Logistic regression Number of obs = 437 LR chi(3) = 18.4 Prob > chi = 0.0004 Log likelihood = -73.84758 Pseudo R = 0.03 owncar Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- income -.010169.5735533-0.0 0.986-1.134313 1.113975 age.465678.069549 3.55 0.000.110539.388817 male.4145366.05557.0 0.044.0116606.817416 _cons -4.68741 1.474519-3.18 0.001-7.57745-1.79738

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 6 Note that a coefficient of the.logit is the logarithmic transformed corresponding estimator of the.logistic. For example,.465678= log(1.7966). STATA has post-estimation commands that conduct follow-up analyses. The.predict command computes predictions, residuals, or standard errors of the prediction and stores them into a new variable.. predict r, residual The.test and.lrtest commands respectively conduct the Wald test and likelihood ratio test.. test income age ( 1) income = 0 ( ) age = 0 chi( ) = 1.57 Prob > chi = 0.0019. Using the SPost Module in STATA The SPost module provides useful follow-up analysis commands (ado files) for various categorical dependent variable models (Long and Freese 003). The.fitstat command calculates various goodness-of-fit statistics such as log likelihood, McFadden s R (or Pseudo R ), Akaike Information Criterion (AIC), and (Bayesian Information Criterion (BIC).. fitstat Measures of Fit for logistic of owncar Log-Lik Intercept Only: -8.965 Log-Lik Full Model: -73.848 D(433): 547.695 LR(3): 18.35 Prob > LR: 0.000 McFadden's R: 0.03 McFadden's Adj R: 0.018 Maximum Likelihood R: 0.041 Cragg & Uhler's R: 0.056 McKelvey and Zavoina's R: 0.059 Efron's R: 0.040 Variance of y*: 3.495 Variance of error: 3.90 Count R: 0.638 Adj Count R: -0.033 AIC: 1.7 AIC*n: 555.695 BIC: -084.916 BIC': 0.005 The likelihood ratio for goodness of fit is computed as,. di *(-73.848 - (-8.965)) 18.34 The.listcoef command lists unstandardized coefficients (parameter estimates), factor and percent changes, and standardized coefficients to help interpret results. The help option tells how to read the outputs.. listcoef, help

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 7 logistic (N=437): Factor Change in Odds Odds of: 1 vs 0 ---------------------------------------------------------------------- owncar b z P> z e^b e^bstdx SDofX -------------+-------------------------------------------------------- income -0.01017-0.018 0.986 0.9899 0.998 0.179 age 0.4657 3.545 0.000 1.796 1.4876 1.6108 male 0.41454.017 0.044 1.5137 1.79 0.4953 ---------------------------------------------------------------------- b = raw coefficient z = z-score for test of b=0 P> z = p-value for z-test e^b = exp(b) = factor change in odds for unit increase in X e^bstdx = exp(b*sd of X) = change in odds for SD increase in X SDofX = standard deviation of X The.prtab command constructs a table of predicted values (events) for all combinations of categorical variables listed. The following example shows that 60 percent of female and 70 percent of male students are likely to own cars, given the mean values of income and age.. prtab male logistic: Predicted probabilities of positive outcome for owncar ---------------------- male Prediction ----------+----------- 0 0.6017 1 0.6958 ---------------------- income age male x=.6168398 0.691076.570838 The.prvalue lists predicted probabilities of positive and negative outcomes for a given set of values for the independent variables. Note both the.prtab and.prvalue commands report the identical predicted probability that male students own cars,.6017, holding other variables at their means.. prvalue, x(male=0) rest(mean) logistic: Predictions for owncar Pr(y=1 x): 0.6017 95% ci: (0.586,0.6706) Pr(y=0 x): 0.3983 95% ci: (0.394,0.4714) income age male x=.6168398 0.691076 0 The most useful command is the.prchange, which calculates marginal effects (changes) and discrete changes at the given set of values of independent variables. The help option tells how to read the outputs. For instance, the predicted probability that a male students owns a car is.094 (0->1) higher than that of female students, holding other variables at their mean.. prchange, help

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 8 logit: Changes in Predicted Probabilities for owncar min->max 0->1 -+1/ -+sd/ MargEfct income -0.0019-0.003-0.003-0.0004-0.003 age 0.4404 0.003 0.0555 0.0893 0.0556 male 0.0940 0.0940 0.093 0.046 0.0934 0 1 Pr(y x) 0.3430 0.6570 income age male x=.61684 0.6911.5708 sd(x)=.17918 1.61081.495344 Pr(y x): probability of observing each y for specified x values Avg Chg : average of absolute value of the change across categories Min->Max: change in predicted probability as x changes from its minimum to its maximum 0->1: change in predicted probability as x changes from 0 to 1 -+1/: change in predicted probability as x changes from 1/ unit below base value to 1/ unit above -+sd/: change in predicted probability as x changes from 1/ standard dev below base to 1/ standard dev above MargEfct: the partial derivative of the predicted probability/rate with respect to a given independent variable The SPost module also includes the.prgen, which computes a series of predictions by holding all variables but one interval variable constant and allowing that variable to vary (Long and Freese 003).. prgen income, from(.1) to(1.5) x(male=1) rest(median) generate(ppcar) logistic: Predicted values as income varies from.1 to 1.5. income age male x=.580000 1 1 The above command computes predicted probabilities that male students own cars when income changes from $100 through $1,500, holding age at its median of 1 and stores them into a new variable ppcar..3 Using the SAS LOGISTIC and PROBIT Procedures SAS has several procedures for the binary logit model such as the LOGISTIC, PROBIT, GENMOD, and QLIM. The LOGISTIC procedure is commonly used for the binary logit model, but the PROBIT procedure also estimates the binary logit. Let us first consider the LOGISTIC procedure. PROC LOGISTIC DESCENDING DATA = masil.students; MODEL owncar = income age male; RUN; The LOGISTIC Procedure

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 9 Model Information Data Set MASIL.STUDENTS Response Variable owncar Number of Response Levels Model binary logit Optimization Technique Fisher's scoring Number of Observations Read 437 Number of Observations Used 437 Response Profile Ordered Total Value owncar Frequency 1 1 84 0 153 Probability modeled is owncar=1. Model Convergence Status Convergence criterion (GCONV=1E-8) satisfied. Model Fit Statistics Intercept Intercept and Criterion Only Covariates AIC 567.930 555.695 SC 57.010 57.015 - Log L 565.930 547.695 Testing Global Null Hypothesis: BETA=0 Test Chi-Square DF Pr > ChiSq Likelihood Ratio 18.351 3 0.0004 Score 17.4697 3 0.0006 Wald 16.7977 3 0.0008 Analysis of Maximum Likelihood Estimates Standard Wald Parameter DF Estimate Error Chi-Square Pr > ChiSq Intercept 1-4.687 1.4745 10.0855 0.0015 income 1-0.010 0.5736 0.0003 0.9859

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 10 age 1 0.466 0.0695 1.5686 0.0004 male 1 0.4145 0.056 4.0670 0.0437 Odds Ratio Estimates Point 95% Wald Effect Estimate Confidence Limits income 0.990 0.3 3.046 age 1.80 1.117 1.467 male 1.514 1.01.65 Association of Predicted Probabilities and Observed Responses Percent Concordant 58.9 Somers' D 0.46 Percent Discordant 34.3 Gamma 0.64 Percent Tied 6.8 Tau-a 0.11 Pairs 4345 c 0.63 The SAS LOGISTIC, PROBIT, and GENMOD procedures by default uses a smaller value in the dependent variable as success. Thus, the magnitudes of the coefficients remain the same, but the signs are opposite to those of the QLIM procedure, STATA, and LIMDEP. The DESCENDING option forces SAS to use a larger value as success. Alternatively, you may explicitly specify the category of successful event using the EVENT option as follows. PROC LOGISTIC DESCENDING DATA = masil.students; MODEL owncar(event= 1 ) = income age male; RUN; The SAS LOGISTIC procedure computes odds changes when independent variables increase by the units specified in the UNITS statement. The SD below indicates a standard deviation increase in income and age (e.g., - means a two unit decrease in independent variables). PROC LOGISTIC DESCENDING DATA = masil.students; MODEL owncar = income age male; UNITS income=sd age=sd; RUN; The UNITS statement adds the Adjusted Odds Ratios to the end of the outputs above. Note that the odds changes of the two variables are identical to those under the e^bstdx of the previous SPost.listcoef output. Adjusted Odds Ratios Effect Unit Estimate income 0.179 0.998 age 1.6108 1.488 Now, let us use the PROBIT procedure to estimate the same binary logit model. The PROBIT requires the CLASS statement to list categorical variables. The /DIST=LOGISTIC option indicates the probability distribution to be used in maximum likelihood estimation.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 11 PROC PROBIT DATA = masil.students; CLASS owncar; MODEL owncar = income age male /DIST=LOGISTIC; RUN; Probit Procedure Model Information Data Set MASIL.STUDENTS Dependent Variable owncar Number of Observations 437 Name of Distribution Logistic Log Likelihood -73.847577 Number of Observations Read 437 Number of Observations Used 437 Class Level Information Name Levels Values owncar 0 1 Response Profile Ordered Total Value owncar Frequency 1 0 153 1 84 PROC PROBIT is modeling the probabilities of levels of owncar having LOWER Ordered Values in the response profile table. Algorithm converged. Type III Analysis of Effects Wald Effect DF Chi-Square Pr > ChiSq income 1 0.0003 0.9859 age 1 1.5686 0.0004 male 1 4.0670 0.0437 Analysis of Parameter Estimates Standard 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 1 Intercept 1 4.687 1.4745 1.797 7.577 10.09 0.0015 income 1 0.010 0.5736-1.1140 1.1343 0.00 0.9859 age 1-0.466 0.0695-0.389-0.1103 1.57 0.0004 male 1-0.4145 0.056-0.8174-0.0117 4.07 0.0437 Unlike LOGISTIC, PROBIT does not have the DESCENDING option. Thus, you have to switch the signs of coefficients when comparing with those of STATA and LIMDEP. The PROBIT procedure also does not have the UNITS statement to compute changes in odds..4 Using the SAS GENMOD and QLIM Procedures The GENMOD provides flexible methods to estimate generalized linear model. The DISTRIBUTION (DIST) and the LINK=LOGIT options respectively specify a probability distribution and a link function. PROC GENMOD DATA = masil.students DESC; MODEL owncar = income age male /DIST=BINOMIAL LINK=LOGIT; RUN; The GENMOD Procedure Model Information Data Set Distribution Link Function Dependent Variable MASIL.STUDENTS Binomial Logit owncar Number of Observations Read 437 Number of Observations Used 437 Number of Events 84 Number of Trials 437 Response Profile Ordered Total Value owncar Frequency 1 1 84 0 153 PROC GENMOD is modeling the probability that owncar='1'. Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance 433 547.695 1.649 Scaled Deviance 433 547.695 1.649 Pearson Chi-Square 433 436.435 1.0079 Scaled Pearson X 433 436.435 1.0079

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 13 Log Likelihood -73.8476 Algorithm converged. Analysis Of Parameter Estimates Standard Wald 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1-4.687 1.4745-7.577-1.797 10.09 0.0015 income 1-0.010 0.5736-1.1343 1.1140 0.00 0.9859 age 1 0.466 0.0695 0.1103 0.389 1.57 0.0004 male 1 0.4145 0.056 0.0117 0.8174 4.07 0.0437 Scale 0 1.0000 0.0000 1.0000 1.0000 NOTE: The scale parameter was held fixed. If you have categorical (string) independent variables, list the variables in the CLASS statement without creating dummy variables. PROC GENMOD DATA = masil.students DESC; CLASS male; MODEL owncar = income age male /DIST=BINOMIAL LINK=LOGIT; RUN; Users may also provide their own link functions using the FWDLINK and INVLINK statements instead of the LINK=LOGIT option. PROC GENMOD DATA = masil.students DESC; FWDLINK link=log(_mean_/(1-_mean_)); INVLINK invlink=1/(1+exp(-1*_xbeta_)); MODEL owncar = income age male /DIST=BINOMIAL; RUN; All three GENMOD examples discussed so far produce the identical result. The QLIM procedure estimates not only logit and probit models, but also censored, truncated, and sample-selected models. You may provide characteristics of the dependent variable either in the ENDOGENOUS statement or the option of the MODEL statement. PROC QLIM DATA=masil.students; MODEL owncar = income age male; ENDOGENOUS owncar ~ DISCRETE (DIST=LOGIT); RUN; Or, PROC QLIM DATA=masil.students; MODEL owncar = income age male /DISCRETE (DIST=LOGIT); RUN; The QLIM Procedure

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 14 Discrete Response Profile of owncar Index Value Frequency Percent 1 0 153 35.01 1 84 64.99 Model Fit Summary Number of Endogenous Variables 1 Endogenous Variable owncar Number of Observations 437 Log Likelihood -73.84758 Maximum Absolute Gradient 9.6319E-6 Number of Iterations 8 AIC 555.69515 Schwarz Criterion 57.01489 Goodness-of-Fit Measures Measure Value Formula Likelihood Ratio (R) 18.35 * (LogL - LogL0) Upper Bound of R (U) 565.93 - * LogL0 Aldrich-Nelson 0.0401 R / (R+N) Cragg-Uhler 1 0.0409 1 - exp(-r/n) Cragg-Uhler 0.0563 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.0415 1 - (1-R/U)^(U/N) Adjusted Estrella 0.034 1 - ((LogL-K)/LogL0)^(-/N*LogL0) McFadden's LRI 0.03 R / U Veall-Zimmermann 0.071 (R * (U+N)) / (U * (R+N)) McKelvey-Zavoina 0.1699 N = # of observations, K = # of regressors Algorithm converged. Parameter Estimates Standard Approx Parameter Estimate Error t Value Pr > t Intercept -4.68741 1.474519-3.18 0.0015 income -0.010169 0.573553-0.0 0.9859 age 0.46568 0.069549 3.55 0.0004 male 0.414537 0.05553.0 0.0437 Finally, the CATMOD procedure fits the logit model to the functions of categorical response variables. This procedure, however, produces slightly different estimators compared to those of other procedures discussed so far. This procedure is, therefore, less recommended for the binary logit model. The DIRECT statement specifies interval or ratio variables used in the MODEL. The /NOPROFILE suppresses the display of the population profiles and the response profiles.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 15 PROC CATMOD DATA = masil.students; DIRECT income age; MODEL owncar = income age male /NOPROFILE; RUN;.5 Binary Logit in LIMDEP (Logit$) The Logit$ command in LIMDEP estimates various logit models. The dependent variable is specified in the Lhs$ (left-hand side) subcommand and a list of independent variables in the Rhs$ (right-hand side). You have to explicitly specify the ONE for the intercept. The Marginal Effects$ and the Means$ subcommands compute marginal effects at the mean values of independent variables. LOGIT; Lhs=owncar; Rhs=ONE,income,age,male; Marginal Effects; Means$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ Multinomial Logit Model Maximum Likelihood Estimates Model estimated: Sep 17, 005 at 05:31:8PM. Dependent variable OWNCAR Weighting variable None Number of observations 437 Iterations completed 5 Log likelihood function -73.8476 Restricted log likelihood -8.9651 Chi squared 18.3509 Degrees of freedom 3 Prob[ChiSqd > value] =.393373E-03 Hosmer-Lemeshow chi-squared = 8.44648 P-value=.39111 with deg.fr. = 8 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Characteristics in numerator of Prob[Y = 1] Constant -4.68741385 1.4745190-3.176.0015 INCOME -.101689609E-01.57355331 -.018.9859.6168398 AGE.465677833.6954911E-01 3.545.0004 0.691076 MALE.4145365774.055576.017.0437.570838 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +--------------------------------------------------------------------+ Information Statistics for Discrete Choice Model. M=Model MC=Constants Only M0=No Model Criterion F (log L) -73.84758-8.9651-30.9053 LR Statistic vs. MC 18.3509.00000.00000 Degrees of Freedom 3.00000.00000.00000 Prob. Value for LR.00039.00000.00000 Entropy for probs. 73.84758 8.9651 30.9053 Normalized Entropy.90407.93417 1.00000

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 16 Entropy Ratio Stat. 58.11548 39.88039.00000 Bayes Info Criterion 565.93495 584.17004 64.05044 BIC - BIC(no model) 58.11548 39.88039.00000 Pseudo R-squared.03.00000.00000 Pct. Correct Prec. 63.84439.00000 50.00000 Means: y=0 y=1 y= y=3 yu=4 y=5, y=6 y>=7 Outcome.3501.6499.0000.0000.0000.0000.0000.0000 Pred.Pr.3501.6499.0000.0000.0000.0000.0000.0000 Notes: Entropy computed as Sum(i)Sum(j)Pfit(i,j)*logPfit(i,j). Normalized entropy is computed against M0. Entropy ratio statistic is computed against M0. BIC = *criterion - log(n)*degrees of freedom. If the model has only constants or if it has no constants, the statistics reported here are not useable. +--------------------------------------------------------------------+ +-------------------------------------------+ Partial derivatives of probabilities with respect to the vector of characteristics. They are computed at the means of the Xs. +-------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Characteristics in numerator of Prob[Y = 1] Constant -1.055883.3318304-3.180.0015 INCOME -.9163775E-0.195338 -.018.9859.6168398 AGE.5556544593E-01.155340E-01 3.577.0003 0.691076 Marginal effect for dummy variable is P 1 - P 0. MALE.940341103E-01.4676710E-01.01.044.570838 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +----------------------------------------+ Fit Measures for Binomial Choice Model Logit model for variable OWNCAR +----------------------------------------+ Proportions P0=.350114 P1=.649886 N = 437 N0= 153 N1= 84 LogL = -73.84758 LogL0 = -8.9651 Estrella = 1-(L/L0)^(-L0/n) =.04153 +----------------------------------------+ Efron McFadden Ben./Lerman.03963.03.56318 Cramer Veall/Zim. Rsqrd_ML.04010.07099.04087 +----------------------------------------+ Information Akaike I.C. Schwarz I.C. Criteria 1.7161 57.01489 +----------------------------------------+ Frequencies of actual & predicted outcomes Predicted outcome has maximum probability. Threshold value for predicting Y=1 =.5000 Predicted ------ ---------- + ----- Actual 0 1 Total ------ ---------- + ----- 0 1 13 153

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 17 1 6 58 84 ------ ---------- + ----- Total 47 390 437 Note that the marginal effects above are identical to those of the SPost.prchange command in section.. LIMDEP computes discrete changes for binary variables like male..6 Binary Logit in SPSS SPSS has the Logistic regression command for the binary logit model. LOGISTIC REGRESSION VAR=owncar /METHOD=ENTER income age male /CRITERIA PIN(.05) POUT(.10) ITERATE(0) CUT(.5). Table 3 summarizes parameter estimates and goodness-of-fit statistics across procedures and commands for the binary logit model. Estimates and their standard errors produced are almost identical except some rounding errors. As shown in Table 3, the QLIM and LOGISTIC are recommended for categorical dependent variables. Note that the PROBIT procedure returns the opposite signs of estimates. Table 3. Parameter Estimates and Goodness-of-fit Statistics of the Binary Logit Model LOGISTIC PROBIT GENMOD QLIM STATA LIMDEP Intercept -4.687 (1.4745) 4.687 (1.4745) -4.687 (1.4745) -4.687 (1.4745) -4.687 (1.4745) -4.687 (1.4745) income -.010 (.5736).010 (.5736) -.010 (.5736) -.010 (.5736) -.010 (.5736) -.010 (.5736) age.466 (.0695) -.466 (.0695).466 (.0695).466 (.0695).466 (.0695).466 (.0695) male.4145 (.056) -.4145 (.056).4145 (.056).4145 (.056).4145 (.056).4145 (.056) Log likelihood 547.695 * -73.8476-73.8476-73.8476-73.8476-73.8476 Likelihood test 18.351 18.35 18.4 18.351 Pseudo R.03.03.03 AIC 555.695 ** 555.695 ** 1.716 Schwarz 57.015 57.0149 57.0150 BIC 565.9350 * The LOGISTIC procedure reports (-*log likelihood). ** AIC*N

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 18 3. The Binary Probit Regression Model The probit model is represented as Pr ob( y = 1 x) = Φ( xβ ), where Φ indicates the cumulative standard normal probability distribution function. 3.1 Binary Probit in STATA (.probit) STATA has the.probit command to estimate the binary probit regression model.. probit owncar income age male Iteration 0: log likelihood = -8.9651 Iteration 1: log likelihood = -73.8483 Iteration : log likelihood = -73.81741 Iteration 3: log likelihood = -73.81741 Probit regression Number of obs = 437 LR chi(3) = 18.30 Prob > chi = 0.0004 Log likelihood = -73.81741 Pseudo R = 0.033 owncar Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- income.0005613.347684 0.00 0.999 -.6808873.680098 age.1487005.0409837 3.63 0.000.068374.9071 male.57911.156085.05 0.040.011731.5040993 _cons -.83671.8730955-3.3 0.001-4.534907-1.11435 In order to get standardized estimates and factor changes, run the SPost.listcoef command.. listcoef probit (N=437): Unstandardized and Standardized Estimates Observed SD:.477558 Latent SD: 1.0371456 - owncar b z P> z bstdx bstdy bstdxy SDofX -------------+----------------------------------------------------------------- income 0.00056 0.00 0.999 0.0001 0.0005 0.0001 0.179 age 0.14870 3.68 0.000 0.395 0.1434 0.309 1.6108 male 0.5791.053 0.040 0.178 0.487 0.13 0.4953 - You may compute the marginal effects and discrete change using the SPost.prchange.. prchange, x(income=1 age=1 male=0) probit: Changes in Predicted Probabilities for owncar min->max 0->1 -+1/ -+sd/ MargEfct income 0.000 0.000 0.000 0.0000 0.000 age 0.4900 0.0014 0.0567 0.091 0.0567

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 19 male 0.0937 0.0937 0.0981 0.0487 0.0984 0 1 Pr(y x) 0.38 0.6178 income age male x= 1 1 0 sd(x)=.17918 1.61081.495344 3. Using the PROBIT and LOGISTIC Procedures The PROBIT and LOGISTIC procedures estimate the binary probit model. Keep in mind that the coefficients of PROBIT has opposite signs. PROC PROBIT DATA = masil.students; CLASS owncar; MODEL owncar = income age male; RUN; Probit Procedure Model Information Data Set MASIL.STUDENTS Dependent Variable owncar Number of Observations 437 Name of Distribution Normal Log Likelihood -73.8174115 Number of Observations Read 437 Number of Observations Used 437 Class Level Information Name Levels Values owncar 0 1 Response Profile Ordered Total Value owncar Frequency 1 0 153 1 84 PROC PROBIT is modeling the probabilities of levels of owncar having LOWER Ordered Values in the response profile table. Algorithm converged.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 0 Type III Analysis of Effects Wald Effect DF Chi-Square Pr > ChiSq income 1 0.0000 0.9987 age 1 13.1644 0.0003 male 1 4.160 0.0400 Analysis of Parameter Estimates Standard 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1.837 0.8731 1.114 4.5349 10.46 0.001 income 1-0.0006 0.3477-0.680 0.6809 0.00 0.9987 age 1-0.1487 0.0410-0.90-0.0684 13.16 0.0003 male 1-0.579 0.156-0.5041-0.0117 4. 0.0400 The LOGISTIC procedure requires a normal probability distribution as a link function (/LINK=PROBIT or /LINK=NORMIT). PROC LOGISTIC DATA = masil.students DESC; MODEL owncar = income age male /LINK=PROBIT; RUN; The LOGISTIC Procedure Model Information Data Set MASIL.STUDENTS Response Variable owncar Number of Response Levels Model binary probit Optimization Technique Fisher's scoring Number of Observations Read 437 Number of Observations Used 437 Response Profile Ordered Total Value owncar Frequency 1 1 84 0 153 Probability modeled is owncar=1. Model Convergence Status

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 1 Convergence criterion (GCONV=1E-8) satisfied. Model Fit Statistics Intercept Intercept and Criterion Only Covariates AIC 567.930 555.635 SC 57.010 571.955 - Log L 565.930 547.635 Testing Global Null Hypothesis: BETA=0 Test Chi-Square DF Pr > ChiSq Likelihood Ratio 18.954 3 0.0004 Score 17.4697 3 0.0006 Wald 17.4690 3 0.0006 Analysis of Maximum Likelihood Estimates Standard Wald Parameter DF Estimate Error Chi-Square Pr > ChiSq Intercept 1 -.837 0.8796 10.3048 0.0013 income 1 0.000548 0.3496 0.0000 0.9987 age 1 0.1487 0.0413 1.960 0.0003 male 1 0.579 0.157 4.096 0.040 Association of Predicted Probabilities and Observed Responses Percent Concordant 57.8 Somers' D 0.49 Percent Discordant 3.9 Gamma 0.74 Percent Tied 9.3 Tau-a 0.113 Pairs 4345 c 0.64 3.3 Using the GENMODE and QLIM Procedures The GENMOD procedure also estimates the binary probit model using the /DIST=BINOMIAL and /LINK=PROBIT options in the MODEL statement. PROC GENMOD DATA = masil.students DESC; MODEL owncar = income age male /DIST=BINOMIAL LINK=PROBIT; RUN; The GENMOD Procedure Model Information Data Set MASIL.STUDENTS

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: Distribution Link Function Dependent Variable Binomial Probit owncar Number of Observations Read 437 Number of Observations Used 437 Number of Events 84 Number of Trials 437 Response Profile Ordered Total Value owncar Frequency 1 1 84 0 153 PROC GENMOD is modeling the probability that owncar='1'. Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance 433 547.6348 1.647 Scaled Deviance 433 547.6348 1.647 Pearson Chi-Square 433 437.070 1.0093 Scaled Pearson X 433 437.070 1.0093 Log Likelihood -73.8174 Algorithm converged. Analysis Of Parameter Estimates Standard Wald 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 -.837 0.8731-4.5349-1.114 10.46 0.001 income 1 0.0006 0.3477-0.6809 0.680 0.00 0.9987 age 1 0.1487 0.0410 0.0684 0.90 13.16 0.0003 male 1 0.579 0.156 0.0117 0.5041 4. 0.0400 Scale 0 1.0000 0.0000 1.0000 1.0000 NOTE: The scale parameter was held fixed. The QLIM procedure provides various goodness-of-fit statistics. The DIST=NORMAL option indicates the normal probability distribution used in estimation. PROC QLIM DATA=masil.students; MODEL owncar = income age male /DISCRETE (DIST=NORMAL); RUN; The QLIM Procedure

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 3 Discrete Response Profile of owncar Index Value Frequency Percent 1 0 153 35.01 1 84 64.99 Model Fit Summary Number of Endogenous Variables 1 Endogenous Variable owncar Number of Observations 437 Log Likelihood -73.81741 Maximum Absolute Gradient 3.8848E-8 Number of Iterations 10 AIC 555.6348 Schwarz Criterion 571.95456 Goodness-of-Fit Measures Measure Value Formula Likelihood Ratio (R) 18.95 * (LogL - LogL0) Upper Bound of R (U) 565.93 - * LogL0 Aldrich-Nelson 0.040 R / (R+N) Cragg-Uhler 1 0.041 1 - exp(-r/n) Cragg-Uhler 0.0565 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.0417 1 - (1-R/U)^(U/N) Adjusted Estrella 0.035 1 - ((LogL-K)/LogL0)^(-/N*LogL0) McFadden's LRI 0.033 R / U Veall-Zimmermann 0.071 (R * (U+N)) / (U * (R+N)) McKelvey-Zavoina 0.070 N = # of observations, K = # of regressors Algorithm converged. Parameter Estimates Standard Approx Parameter Estimate Error t Value Pr > t Intercept -.83671 0.873096-3.3 0.001 income 0.000561 0.347684 0.00 0.9987 age 0.148701 0.040984 3.63 0.0003 male 0.57911 0.15608.05 0.0400 3.4 Binary Probit in LIMDEP (Probit$) The LIMDEP Probit$ command estimates various probit models. Do not forget to include the ONE for the intercept.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 4 PROBIT; Lhs=owncar; Rhs=ONE,income,age,male$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ Binomial Probit Model Maximum Likelihood Estimates Model estimated: Sep 17, 005 at 10:8:56PM. Dependent variable OWNCAR Weighting variable None Number of observations 437 Iterations completed 4 Log likelihood function -73.8174 Restricted log likelihood -8.9651 Chi squared 18.954 Degrees of freedom 3 Prob[ChiSqd > value] =.3854E-03 Hosmer-Lemeshow chi-squared = 8.1837 P-value=.41573 with deg.fr. = 8 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Index function for probability Constant -.8367089.87309548-3.34.001 INCOME.561515407E-03.3476843.00.9987.6168398 AGE.148700534.40983697E-01 3.68.0003 0.691076 MALE.579111914.1560848.053.0400.570838 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +----------------------------------------+ Fit Measures for Binomial Choice Model Probit model for variable OWNCAR +----------------------------------------+ Proportions P0=.350114 P1=.649886 N = 437 N0= 153 N1= 84 LogL = -73.81741 LogL0 = -8.9651 Estrella = 1-(L/L0)^(-L0/n) =.04166 +----------------------------------------+ Efron McFadden Ben./Lerman.03984.0333.5637 Cramer Veall/Zim. Rsqrd_ML.04016.0711.04100 +----------------------------------------+ Information Akaike I.C. Schwarz I.C. Criteria 1.7148 571.95456 +----------------------------------------+ Frequencies of actual & predicted outcomes Predicted outcome has maximum probability. Threshold value for predicting Y=1 =.5000 Predicted ------ ---------- + ----- Actual 0 1 Total ------ ---------- + -----

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 5 0 5 148 153 1 8 76 84 ------ ---------- + ----- Total 13 44 437 3.5 Binary Probit in SPSS SPSS has the Probit command to fit the binary probit model. This command requires a variable (e.g., n in the following example) with constant 1. COMPUTE n=1. PROBIT owncar OF n WITH income age male /LOG NONE /MODEL PROBIT /PRINT FREQ /CRITERIA ITERATE(0) STEPLIMIT(.1). Table 4 summarizes parameter estimates and goodness-of-fit statistics produced. Note that the LOGISTIC procedure reports slightly different estimates and standard errors. I would recommend the SAS QLIM procedure, STATA, and LIMDEP for the binary probit model. Table 4.Parameter Estimates and Goodness-of-fit Statistics of the Binary Probit Model LOGISTIC PROBIT GENMOD QLIM STATA LIMDEP Intercept -.837 (.8796).837 (.8731) -.837 (.8731) -.837 (.8731) -.837 (.8731) -.837 (.8731) income.0005 (.3496) -.0006 (.3477).0006 (.3477).0006 (.3477).0006 (.3477).0006 (.3477) age.1487 (.0413) -.1487 (.0410).1487 (.0410).1487 (.0410).1487 (.0410).1487 (.0410) male.579 (.157) -.579 (.156).579 (.156).579 (.156).579 (.156).579 (.156) Log likelihood 547.653 * -73.8174-73.8174-73.8174-73.8174-73.8174 Likelihood test 18.954 18.95 18.30 18.954 Pseudo R.033.033.033 AIC 555.635 ** 555.6348 ** 1.715 Schwarz 571.955 571.9546 571.9546 BIC * The LOGISTIC procedure reports (-*log likelihood). ** AIC*N

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 6 4. Bivariate Probit/Logit Regression Models Bivariate regression models have two equations for the two dependent variables. This chapter explains the bivariate regression model with two binary dependent variables. Like the seemingly unrelated regression model (SUR), biviriate probit/logit models assume that the independent, identically distributed errors are correlated (Greene 003). The bivariate probit model, although consuming relatively much time, is more likely to converge than the bivariate logit model. SAS supports both the bivariate probit and logit models, while STATA and LIMDEP estimate the bivariate probit model. Here we consider a model for car ownership (owncar) and housing type (offcamp). 4.1 Bivariate Probit in STATA (.biprobit) STATA has the.biprobit command to estimate the bivariate probit model. The two dependent variables precede a set of independent variables.. biprobit owncar offcamp income age male Fitting comparison equation 1: Iteration 0: log likelihood = -8.9651 Iteration 1: log likelihood = -73.8483 Iteration : log likelihood = -73.81741 Iteration 3: log likelihood = -73.81741 Fitting comparison equation : Iteration 0: log likelihood = -54.97403 Iteration 1: log likelihood = -45.919608 Iteration : log likelihood = -43.685448 Iteration 3: log likelihood = -43.365 Iteration 4: log likelihood = -43.309675 Iteration 5: log likelihood = -43.309654 Comparison: log likelihood = -317.1707 Fitting full model: Iteration 0: log likelihood = -317.1707 Iteration 1: log likelihood = -307.15684 Iteration : log likelihood = -306.49535 Iteration 3: log likelihood = -306.46018 Iteration 4: log likelihood = -306.45493 Iteration 5: log likelihood = -306.45408 Iteration 6: log likelihood = -306.45395 Iteration 7: log likelihood = -306.4539 Bivariate probit regression Number of obs = 437 Wald chi(6) = 30.13 Log likelihood = -306.4539 Prob > chi = 0.0000 Coef. Std. Err. z P> z [95% Conf. Interval] -------------+----------------------------------------------------------------

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 7 owncar income -.0017168.347905-0.00 0.996 -.683598.6801645 age.149475.040938 3.65 0.000.0690383.94568 male.59464.155633.07 0.039.013368.50556 _cons -.83465.8719679-3.5 0.001-4.543651-1.15599 -------------+---------------------------------------------------------------- offcamp income.7519064.854937 0.91 0.36 -.8660316.369844 age.5895658.1491 3.95 0.000.97098.880336 male.3939644.834889 1.39 0.165 -.1616637.949595 _cons -10.34593.947501-3.51 0.000-16.193-4.568938 -------------+---------------------------------------------------------------- /athrho.3875 7.0167 0.09 0.930-50.9678 55.7018 -------------+---------------------------------------------------------------- rho.983658.907811-1 1 Likelihood-ratio test of rho=0: chi(1) = 1.3463 Prob > chi = 0.0000 4. Bivariate Probit in SAS The SAS QLIM procedure is able to estimate both the bivariate logit and probit models. You need to provide two equations that may or may not have different sets of independent variables. PROC QLIM DATA=masil.students; MODEL owncar = income age male; MODEL offcamp = income age male; ENDOGENOUS owncar offcamp ~ DISCRETE(DIST=NORMAL); RUN; Or, simply, PROC QLIM DATA=masil.students; MODEL owncar offcamp = income age male /DISCRETE; RUN; The QLIM Procedure Discrete Response Profile of owncar Index Value Frequency Percent 1 0 153 35.01 1 84 64.99 Discrete Response Profile of offcamp Index Value Frequency Percent 1 0 1.75 1 45 97.5 Model Fit Summary Number of Endogenous Variables

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 8 Endogenous Variable owncar offcamp Number of Observations 437 Log Likelihood -306.4539 Maximum Absolute Gradient.16967E-6 Number of Iterations 7 AIC 68.90784 Schwarz Criterion 661.54730 Algorithm converged. Parameter Estimates Standard Approx Parameter Estimate Error t Value Pr > t owncar.intercept -.834511 0.871964-3.5 0.001 owncar.income -0.00173 0.347904-0.00 0.9960 owncar.age 0.14943 0.04094 3.65 0.0003 owncar.male 0.5946 0.15563.07 0.0388 offcamp.intercept -10.34500.947054-3.51 0.0004 offcamp.income 0.751837 0.85398 0.91 0.364 offcamp.age 0.589515 0.149197 3.95 <.0001 offcamp.male 0.393859 0.83458 1.39 0.1647 _Rho 0.999990 0.. 4.3 Bivariate Probit in LIMDEP (Bivariateprobit$) LIMDEP has the Bivariateprobit$ command to estimate the bivariate probit model. The Lhs$ subcommand lists the two binary dependent variables, whereas Rh1$ and Rh$ respectively indicate independent variables for the two dependent variables. In this model, you may not switch the order of dependent variables (Lhs=owncar,offcamp;) to avoid convergence problems. BIVARIATEPROBIT; Lhs=offcamp,owncar; Rh1=ONE,income,age,male; Rh= ONE,income,age,male$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ FIML Estimates of Bivariate Probit Model Maximum Likelihood Estimates Model estimated: Sep 17, 005 at 10:36:5PM. Dependent variable OFFOWN Weighting variable None Number of observations 437 Iterations completed 35 Log likelihood function -306.4539 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Index equation for OFFCAMP

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 9 Constant -10.3450835 3.659558 -.87.0047 INCOME.7518407011.8574898.88.3780.6168398 AGE.5895189160.1857787 3.174.0015 0.691076 MALE.3938599470.9308051 1.344.1790.570838 Index equation for OWNCAR Constant -.834513147.8485468-3.34.0008 INCOME -.17310966E-0.34451 -.005.9960.6168398 AGE.14946338.3973976E-01 3.755.000 0.691076 MALE.594618946.1565094.065.0389.570838 Disturbance correlation RHO(1,).9941311591.73338053E+09.000 1.0000 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) Joint Frequency Table: Columns=OWNCAR Rows =OFFCAMP (N) = Count of Fitted Values 0 1 TOTAL 0 1 0 1 ( 0) ( 0) ( 0) 1 141 84 45 ( 0) ( 437) ( 437) TOTAL 153 84 437 ( 0) ( 437) ( 437) SAS, STATA, and LIMDEP produce almost the same parameter estimates and standard errors with slight differences after the decimal point. 4.4 Bivariate Logit in SAS The QLIM procedure also estimates the bivariate logit model using the DIST=LOGIT option. Unfortunately, this model does not fit in SAS. PROC QLIM DATA=masil.students; MODEL owncar = income age male; MODEL offcamp = income age male; ENDOGENOUS offcamp owncar ~ DISCRETE(DIST=LOGIT); RUN; Or, PROC QLIM DATA=masil.students; MODEL owncar offcamp = income age male /DISCRETE(DIST=LOGIT); RUN;

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 30 5. Ordered Logit/Probit Regression Models Suppose we have an ordinal dependent variable such as the degree of illegal parking (0=none, 1=sometimes, and =often). The ordered logit and probit models have the parallel regression assumption, which is violated from time to time. 5.1 Ordered Logit/Probit in STATA (.ologit and.oprobit) STATA has the.ologit and.oprobit commands to estimate the ordered logit and probit models, respectively.. ologit parking income age male Iteration 0: log likelihood = -103.78713 Iteration 1: log likelihood = -9.739147 Iteration : log likelihood = -90.036393 Iteration 3: log likelihood = -89.861679 Iteration 4: log likelihood = -89.860105 Iteration 5: log likelihood = -89.860105 Ordered logistic regression Number of obs = 437 LR chi(3) = 7.85 Prob > chi = 0.0000 Log likelihood = -89.860105 Pseudo R = 0.134 parking Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- income -.5140709 1.8319-0.40 0.689-3.0908.00094 age -.736588.1894339-3.89 0.000-1.10754 -.364975 male -1.709.4705859 -.61 0.009 -.14943 -.3047605 -------------+---------------------------------------------------------------- /cut1-1.74479 3.787616-0.16839-5.3103 /cut -10.8395 3.801685-18.841-3.381786 STATA estimates τ m, /cut1 and /cut, assuming β 0 = 0 (Long and Freese 003). This parameterization is different from that of SAS and LIMDEP, which assume τ = 1 0.. oprobit parking income age male Iteration 0: log likelihood = -103.78713 Iteration 1: log likelihood = -90.990455 Iteration : log likelihood = -89.49688 Iteration 3: log likelihood = -89.430915 Iteration 4: log likelihood = -89.430754 Ordered probit regression Number of obs = 437 LR chi(3) = 8.71 Prob > chi = 0.0000 Log likelihood = -89.430754 Pseudo R = 0.1383 parking Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- income -.1869839.6116037-0.31 0.760-1.385705 1.011737

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 31 age -.3594853.094817-3.89 0.000 -.540746 -.17846 male -.5867871.0553 -.66 0.008-1.019009 -.1545655 -------------+---------------------------------------------------------------- /cut1-6.000986 1.869046-9.66448 -.33774 /cut -5.118676 1.86909-8.769911-1.46744 5. The Parallel Assumption and the Generalized Ordered Logit Model The.brant command of SPost is valid only in the.ologit command. This command tests the parallel regression assumption of the ordinal regression model. The outputs here are skipped.. quietly ologit parking income male. brant The parallel regression assumption is often violated. If this is the case, you may use the multinomial regression model or estimate the generalized ordered logit model (GOLM) using either the.gologit command written by Fu (1998) or the.gologit command by Williams (005). Note that Fu s module does not impose the restriction of ( τ j xβ j ) ( τ j 1 xβ j 1 ) (Long s class note 003).. gologit parking income age male, autofit Testing parallel lines assumption using the.05 level of significance... Step 1: male meets the pl assumption (P Value = 0.9901) Step : income meets the pl assumption (P Value = 0.8958) Step 3: age meets the pl assumption (P Value = 0.7964) Step 4: All explanatory variables meet the pl assumption Wald test of parallel lines assumption for the final model: ( 1) [0]male - [1]male = 0 ( ) [0]income - [1]income = 0 ( 3) [0]age - [1]age = 0 chi( 3) = 0.04 Prob > chi = 0.998 An insignificant test statistic indicates that the final model does not violate the proportional odds/ parallel lines assumption If you re-estimate this exact same model with gologit, instead of autofit you can save time by using the parameter pl(male income age) Generalized Ordered Logit Estimates Number of obs = 437 Wald chi(3) = 1.74 Prob > chi = 0.0001 Log likelihood = -89.860105 Pseudo R = 0.134 ( 1) [0]male - [1]male = 0

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 3 ( ) [0]income - [1]income = 0 ( 3) [0]age - [1]age = 0 parking Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- 0 income -.5140709 1.8319-0.40 0.689-3.0908.00094 age -.736588.1894339-3.89 0.000-1.107543 -.364975 male -1.709.4705859 -.61 0.009 -.14943 -.3047605 _cons 1.74479 3.787616 3.36 0.001 5.310 0.16839 -------------+---------------------------------------------------------------- 1 income -.5140709 1.8319-0.40 0.689-3.0908.00094 age -.736588.1894339-3.89 0.000-1.107543 -.364975 male -1.709.4705859 -.61 0.009 -.14943 -.3047605 _cons 10.8395 3.801686.85 0.004 3.381785 18.841 5.3 Ordered Logit in SAS The QLIM, LOGISTIC, and PROBIT procedures estimate ordered logit and probit models. As shown in Tables 3 and 4, the QLIM procedure is most recommended. Note that the DIST=LOGISTIC indicates the logit model to be estimated. PROC QLIM DATA=masil.students; MODEL parking = income age male /DISCRETE (DIST=LOGISTIC); RUN; The QLIM Procedure Discrete Response Profile of parking Index Value Frequency Percent 1 0 413 94.51 1 0 4.58 3 4 0.9 Model Fit Summary Number of Endogenous Variables 1 Endogenous Variable parking Number of Observations 437 Log Likelihood -89.86011 Maximum Absolute Gradient 8.14046E-7 Number of Iterations 3 AIC 189.701 Schwarz Criterion 10.11988 Goodness-of-Fit Measures Measure Value Formula Likelihood Ratio (R) 7.854 * (LogL - LogL0)

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 33 Upper Bound of R (U) 07.57 - * LogL0 Aldrich-Nelson 0.0599 R / (R+N) Cragg-Uhler 1 0.0618 1 - exp(-r/n) Cragg-Uhler 0.1633 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.066 1 - (1-R/U)^(U/N) Adjusted Estrella 0.0418 1 - ((LogL-K)/LogL0)^(-/N*LogL0) McFadden's LRI 0.134 R / U Veall-Zimmermann 0.1861 (R * (U+N)) / (U * (R+N)) McKelvey-Zavoina 0.646 N = # of observations, K = # of regressors Algorithm converged. Parameter Estimates Standard Approx Parameter Estimate Error t Value Pr > t Intercept 1.744794 3.787615 3.36 0.0008 income -0.514071 1.8319-0.40 0.6887 age -0.73659 0.189434-3.89 0.0001 male -1.709 0.470586 -.61 0.0091 _Limit 1.91184 0.468050 4.08 <.0001 The SAS QLIM procedure estimates the intercept and τ, assuming τ = 1 0. The estimated intercept of SAS is equivalent to (0-/cut1) in STATA. The _Limit of SAS is the difference between cut points of STATA, 1.91184=-10.8395-(-1.74479). The SAS LOGISTIC and PROBIT procedures are also used to estimate the ordered logit and probit models. These procedures recognize binary or ordinal response models by examining the dependent variable. PROC LOGISTIC DATA = masil.students DESC; MODEL parking = income age male /LINK=LOGIT; RUN; Like the STATA.ologit command, The LOGISTIC procedure fits the model, assuming the intercept is zero. The parameter estimates and standard errors are slightly different from those of the QLIM procedure and the.ologit command. Other parts of the output are skipped. Analysis of Maximum Likelihood Estimates Standard Wald Parameter DF Estimate Error Chi-Square Pr > ChiSq Intercept 1 10.834 3.811 8.0784 0.0045 Intercept 1 1 1.7444 3.801 11.354 0.0008 income 1-0.514 1.908 0.1587 0.6904 age 1-0.736 0.1900 15.01 0.0001 male 1-1.71 0.4709 6.790 0.009 PROC PROBIT DATA = masil.students;

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 34 RUN; CLASS parking; MODEL parking = income age male /DIST=LOGISTIC; The PROBIT procedure returns almost the same results as the QLIM procedure except for the signs of the estimates. Other parts of the output are skipped. Analysis of Parameter Estimates Standard 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1-1.7448 3.7876-0.1684-5.31 11.3 0.0008 Intercept 1 1.9118 0.4680 0.9945.89 16.68 <.0001 income 1 0.5141 1.83 -.0009 3.091 0.16 0.6887 age 1 0.7363 0.1894 0.3650 1.1075 15.11 0.0001 male 1 1.71 0.4706 0.3048.1494 6.80 0.0091 5.4 Ordered Probit in SAS The QLIM procedure by default estimates a probit model. The DIST=NORMAL, the default option, may be omitted. PROC QLIM DATA=masil.students; MODEL parking = income age male /DISCRETE (DIST=NORMAL); RUN; The QLIM Procedure Discrete Response Profile of parking Index Value Frequency Percent 1 0 413 94.51 1 0 4.58 3 4 0.9 Model Fit Summary Number of Endogenous Variables 1 Endogenous Variable parking Number of Observations 437 Log Likelihood -89.43075 Maximum Absolute Gradient 4.69307E-6 Number of Iterations 17 AIC 188.86151 Schwarz Criterion 09.6117 Goodness-of-Fit Measures Measure Value Formula

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 35 Likelihood Ratio (R) 8.713 * (LogL - LogL0) Upper Bound of R (U) 07.57 - * LogL0 Aldrich-Nelson 0.0617 R / (R+N) Cragg-Uhler 1 0.0636 1 - exp(-r/n) Cragg-Uhler 0.168 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.0683 1 - (1-R/U)^(U/N) Adjusted Estrella 0.0439 1 - ((LogL-K)/LogL0)^(-/N*LogL0) McFadden's LRI 0.1383 R / U Veall-Zimmermann 0.1915 (R * (U+N)) / (U * (R+N)) McKelvey-Zavoina 0.3011 N = # of observations, K = # of regressors Algorithm converged. Parameter Estimates Standard Approx Parameter Estimate Error t Value Pr > t Intercept 6.000986 1.869053 3.1 0.0013 income -0.186984 0.611605-0.31 0.7598 age -0.359485 0.0948-3.89 0.0001 male -0.586787 0.056 -.66 0.0078 _Limit 0.88310 0.196555 4.49 <.0001 The QLIM procedure and.oprobit command produce almost the same result except for the τ estimate. The _Limit of SAS is the difference of the cut points of STATA,.8831=-5.118676- (-6.000986). The PROBIT and LOGISTIC procedures also estimate the ordered probit model. Keep in mind that the signs of the coefficients are reversed in the PROBIT procedure. PROC LOGISTIC DATA = masil.students DESC; MODEL parking = income age male /LINK=PROBIT; RUN; Analysis of Maximum Likelihood Estimates Standard Wald Parameter DF Estimate Error Chi-Square Pr > ChiSq Intercept 1 5.1181 1.8373 7.7601 0.0053 Intercept 1 1 6.0004 1.8441 10.587 0.0011 income 1-0.1869 0.6160 0.091 0.7615 age 1-0.3595 0.0908 15.6767 <.0001 male 1-0.5868 0.03 7.0941 0.0077 PROC PROBIT DATA = masil.students; CLASS parking; MODEL parking = income age male /DIST=NORMAL; RUN;

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 36 Analysis of Parameter Estimates Standard 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1-6.0010 1.8691-9.6643 -.3377 10.31 0.0013 Intercept 1 0.883 0.1966 0.4971 1.675 0.15 <.0001 income 1 0.1870 0.6116-1.0117 1.3857 0.09 0.7598 age 1 0.3595 0.095 0.178 0.5407 15.11 0.0001 male 1 0.5868 0.05 0.1546 1.0190 7.08 0.0078 5.5 Ordered Logit/Probit in LIMDEP (Ordered$) The LIMDEP Ordered$ command estimates ordered logit and probit models. The Logit$ subcommand runs the ordered logit model. ORDERED; Lhs=parking; Rhs=ONE,income,age,male; Logit$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ Ordered Probability Model Maximum Likelihood Estimates Model estimated: Sep 18, 005 at 05:53:44PM. Dependent variable PARKING Weighting variable None Number of observations 437 Iterations completed 13 Log likelihood function -89.86011 Restricted log likelihood -103.7871 Chi squared 7.85404 Degrees of freedom 3 Prob[ChiSqd > value] =.3896741E-05 Underlying probabilities based on Logistic Cell frequencies for outcomes Y Count Freq Y Count Freq Y Count Freq 0 413.945 1 0.045 4.009 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Index function for probability Constant 1.7447944 3.7876161 3.365.0008 INCOME -.5140708643 1.83193 -.401.6887.6168398 AGE -.73658881.18943391-3.887.0001 0.691076 MALE -1.7091964.47058590 -.608.0091.570838 Threshold parameters for index Mu(1) 1.91184193.46804996 4.085.0000 +---------------------------------------------------------------------------+ Cross tabulation of predictions. Row is actual, column is predicted.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 37 Model = Logistic. Prediction is number of the most probable cell. +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ Actual Row Sum 0 1 3 4 5 6 7 8 9 +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ 0 413 413 0 0 1 0 0 0 0 4 4 0 0 +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ Col Sum 437 437 0 0 0 0 0 0 0 0 0 +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ LIMDEP and SAS QLIM produce the same results for the ordered logit model. Note that _Limit in SAS is equivalent to Mu(1), the threshold parameter, in LIMDEP. The ordered probit model is estimated by the Ordered$ command without the Logit$ subcommand. The command by default fits the ordered logit model. The output is comparable to that of the QLIM procedure. ORDERED; Lhs=parking; Rhs=ONE,income,age,male$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ Ordered Probability Model Maximum Likelihood Estimates Model estimated: Sep 18, 005 at 05:55:4PM. Dependent variable PARKING Weighting variable None Number of observations 437 Iterations completed 11 Log likelihood function -89.43075 Restricted log likelihood -103.7871 Chi squared 8.7175 Degrees of freedom 3 Prob[ChiSqd > value] =.57557E-05 Underlying probabilities based on Normal Cell frequencies for outcomes Y Count Freq Y Count Freq Y Count Freq 0 413.945 1 0.045 4.009 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Index function for probability Constant 6.000985035 1.8690536 3.11.0013 INCOME -.1869836008.61160494 -.306.7598.6168398 AGE -.35948594.948090E-01-3.887.0001 0.691076 MALE -.586787057.05578 -.661.0078.570838 Threshold parameters for index Mu(1).883095981.19655461 4.489.0000 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +---------------------------------------------------------------------------+ Cross tabulation of predictions. Row is actual, column is predicted.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 38 Model = Probit. Prediction is number of the most probable cell. +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ Actual Row Sum 0 1 3 4 5 6 7 8 9 +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ 0 413 413 0 0 1 0 0 0 0 4 4 0 0 +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ Col Sum 874 437 0 0 0 0 0 0 0 0 0 +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ 5.6 Ordered Logit/Probit in SPSS The Plum command estimates the ordered logit and probit models in SPSS. The Threshold points in SPSS are equivalent to the cut points in STATA. PLUM parking WITH income age male /CRITERIA = CIN(95) DELTA(0) LCONVERGE(0) MXITER(100) MXSTEP(5) PCONVERGE(1.0E-6) SINGULAR(1.0E-8) /LINK = LOGIT /PRINT = FIT PARAMETER SUMMARY. PLUM parking WITH income age male /CRITERIA = CIN(95) DELTA(0) LCONVERGE(0) MXITER(100) MXSTEP(5) PCONVERGE(1.0E-6) SINGULAR(1.0E-8) /LINK = PROBIT /PRINT = FIT PARAMETER SUMMARY.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 39 6. The Multinomial Logit Regression Model Suppose we have a nominal dependent variable such as the mode of transportation (walk, bike, bus, and car). The multinomial logit and conditional logit models are commonly used; the multinomial probit model is not often used mainly due to the practical difficulty in estimation. However, STATA does have the.mprobit command to fit the model. In the multinomial logit model, the independent variables contain characteristics of individuals, while they are the attributes of the choices in the conditional logit model. In other words, the conditional logit estimates how alternative-specific, not individual-specific, variables affect the likelihood of observing a given outcome (Long 003). Therefore, data need to be appropriately arranged in advance. 6.1 Multinomial Logit/Probit in STATA (.mlogit and.mprobit) STATA has the.mlogit command for the multinomial logit model. The base() option indicates the value of the dependent variable to be used as the base category for the estimation. You may omit the default option, base(0).. mlogit transmode income age male, base(0) Iteration 0: log likelihood = -444.84113 Iteration 1: log likelihood = -411.18604 Iteration : log likelihood = -406.36474 Iteration 3: log likelihood = -406.351 Iteration 4: log likelihood = -406.3509 Multinomial logistic regression Number of obs = 437 LR chi(9) = 77.03 Prob > chi = 0.0000 Log likelihood = -406.3509 Pseudo R = 0.0866 transmode Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- 1 income 4.01801 1.34443.99 0.003 1.38986 6.653057 age.1915917.13998 1.38 0.169 -.081417.4646006 male.58886.4039971 0.64 0.53 -.5335311 1.050108 _cons -6.903473.97678 -.3 0.00-1.73785-1.069091 -------------+---------------------------------------------------------------- income 8.951041 1.338539 6.69 0.000 6.3755 11.57453 age.1374997.1451938 0.95 0.344 -.1470749.4074 male.1573179.4191014 0.38 0.707 -.6641057.9787415 _cons -9.091051 3.08813 -.94 0.003-15.14366-3.03844 -------------+---------------------------------------------------------------- 3 income 4.10485 1.0304 4.08 0.000.187755 6.3315 age.345736.0995071 3.47 0.001.150693.540754 male.540549.769887 1.95 0.051 -.00639 1.083143 _cons -8.388756.13579-3.93 0.000-1.57483-4.0681 (transmode==0 is the base outcome)

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 40 Let us see if the base outcome changes. As shown in the following, the parameter estimates and standard errors are changed, whereas the goodness-of-fit remains unchanged. The two.mlogit commands with different bases fit the same model but present the result in different manner. The SAS CATMOD procedure in the next section uses the largest value as the base outcome.. mlogit transmode income age male, base(3) Iteration 0: log likelihood = -444.84113 Iteration 1: log likelihood = -411.18604 Iteration : log likelihood = -406.36474 Iteration 3: log likelihood = -406.351 Iteration 4: log likelihood = -406.3509 Multinomial logistic regression Number of obs = 437 LR chi(9) = 77.03 Prob > chi = 0.0000 Log likelihood = -406.3509 Pseudo R = 0.0866 transmode Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- 0 income -4.10485 1.0304-4.08 0.000-6.3315 -.187755 age -.345736.0995071-3.47 0.001 -.540754 -.150693 male -.540549.769887-1.95 0.051-1.083143.00639 _cons 8.388756.13579 3.93 0.000 4.0681 1.57483 -------------+---------------------------------------------------------------- 1 income -.194639 1.00606-0.19 0.848 -.164305 1.779377 age -.15413.1131506-1.36 0.173 -.3759031.067639 male -.819663.3443963-0.8 0.413 -.9569706.3930379 _cons 1.48583.43091 0.61 0.541-3.7916 6.49783 -------------+---------------------------------------------------------------- income 4.740556.944716 5.0 0.000.888953 6.59158 age -.0839.1164954-1.79 0.074 -.4365507.00108 male -.38937.349047-1.10 0.73-1.067013.3011389 _cons -.70953.460119-0.9 0.775-5.5404 4.119449 (transmode==3 is the base outcome) The SPost.mlogtest command conducts a variety of statistical tests for the multinomial logit model. This command supports not only Wald and likelihood ratio tests, but also Hausman and Small-Hsiao tests for the independence of irrelevant alternatives (IIA) assumption. The.mlogtest command works with the.mlogit command only.. mlogtest, hausman smhsiao base **** Hausman tests of IIA assumption Ho: Odds(Outcome-J vs Outcome-K) are independent of other alternatives. Omitted chi df P>chi evidence ---------+------------------------------------ 0 0.60 8 1.000 for Ho 1-3.307 8 1.000 for Ho -0.319 8 1.000 for Ho 3.315 8 0.970 for Ho ----------------------------------------------

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 41 **** Small-Hsiao tests of IIA assumption Ho: Odds(Outcome-J vs Outcome-K) are independent of other alternatives. Omitted lnl(full) lnl(omit) chi df P>chi evidence ---------+--------------------------------------------------------- 0-10.685-116.139 9.09 4 0.059 for Ho 1-131.938-18.574 6.78 4 0.151 for Ho -155.078-150.308 9.540 4 0.049 against Ho 3-71.735-67.571 8.37 4 0.080 for Ho ------------------------------------------------------------------- The STATA.mprobit command fits the multinomial probit model. The model took longer time to converge than the multinomial logit model.. mprobit transmode income age male Iteration 0: log likelihood = -45.053 Iteration 1: log likelihood = -407.9597 Iteration : log likelihood = -406.3865 Iteration 3: log likelihood = -406.38431 Iteration 4: log likelihood = -406.38431 Multinomial probit regression Number of obs = 437 Wald chi(9) = 64.47 Log likelihood = -406.38431 Prob > chi = 0.0000 transmode Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- _outcome_ income.651949.838566 3.18 0.001 1.01958 4.84318 age.1501467.0903614 1.66 0.097 -.069584.37519 male.1967795.6047 0.75 0.453 -.31683.71038 _cons -5.07538 1.953866 -.60 0.009-8.904835-1.458 -------------+---------------------------------------------------------------- _outcome_3 income 5.757611.8443105 6.8 0.000 4.10793 7.4149 age.11865.09466 1.9 0.196 -.068959.306609 male.166947.77189 0.60 0.549 -.3770444.7096339 _cons -6.547874.031953-3. 0.001-10.53043 -.565319 -------------+---------------------------------------------------------------- _outcome_4 income.7516.693663 3.97 0.000 1.39067 4.111177 age.760071.074178 3.7 0.000.130608.413933 male.4371.086763.03 0.043.01489.835 _cons -6.375609 1.598767-3.99 0.000-9.509134-3.4083 (transmode=0 is the base outcome) 6. Multinomial Logit in SAS SAS has the CATMOD procedure for the multinomial logit model. In the CATMOD procedure, the RESPONSE statement is used to specify the functions of response probabilities. PROC CATMOD DATA = masil.students; DIRECT income age male; RESPONSE LOGITS;

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 4 MODEL transmode = income age male /NOPROFILE; RUN; The CATMOD Procedure Data Summary Response transmode Response Levels 4 Weight Variable None Populations 414 Data Set STUDENTS Total Frequency 437 Frequency Missing 0 Observations 437 Maximum Likelihood Analysis Maximum likelihood computations converged. Maximum Likelihood Analysis of Variance Source DF Chi-Square Pr > ChiSq ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Intercept 3 15.91 0.001 income 3 45.7 <.0001 age 3 14.66 0.001 male 3 4.73 0.197 Likelihood Ratio 1E3 778.33 1.0000 Analysis of Maximum Likelihood Estimates Function Standard Chi- Parameter Number Estimate Error Square Pr > ChiSq ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Intercept 1 8.3888.1358 15.43 <.0001 1.4853.4309 0.37 0.541 3-0.703.4601 0.08 0.7753 income 1-4.105 1.030 16.65 <.0001-0.195 1.0061 0.04 0.8483 3 4.7406 0.9447 5.18 <.0001 age 1-0.3457 0.0995 1.07 0.0005-0.1541 0.113 1.86 0.1731 3-0.08 0.1165 3.19 0.0739 male 1-0.5403 0.770 3.80 0.0511-0.80 0.3444 0.67 0.419 3-0.389 0.3490 1.0 0.76 As mentioned before, the CATMOD procedure uses the largest value of the dependent variable as a base outcome. Accordingly, you need to compare the above with the STATA output of the base(3) option. The two outputs are the same except for the likelihood ratio. 6.3 Multinomial Logit in LIMDEP (Mlogit$)

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 43 In LIMDEP, you may use either the Mlogit$ or simply the Logit$ commands to fit the multinomial logit model. Both commands produce the identical result. Like STATA, LIMDEP by default uses the smallest value as the base outcome. MLOGIT; Lhs=transmod; Rhs=ONE,income,age,male$ Or, use the old style command. LOGIT; Lhs=transmod; Rhs=ONE,income,age,male$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ Multinomial Logit Model Maximum Likelihood Estimates Model estimated: Sep 19, 005 at 09:19:3AM. Dependent variable TRANSMOD Weighting variable None Number of observations 437 Iterations completed 6 Log likelihood function -406.351 Restricted log likelihood -444.8411 Chi squared 77.0309 Degrees of freedom 9 Prob[ChiSqd > value] =.0000000 +---------------------------------------------+ +---------+--------------+----------------+--------+---------+----------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] Mean of X +---------+--------------+----------------+--------+---------+----------+ Characteristics in numerator of Prob[Y = 1] Constant -6.90347671.9767801 -.319.004 INCOME 4.01801309 1.3444306.989.008.6168398 AGE.1915916653.139981 1.375.1690 0.691076 MALE.5888630.40399708.639.56.570838 Characteristics in numerator of Prob[Y = ] Constant -9.091051495 3.088131 -.944.003 INCOME 8.951040805 1.3385396 6.687.0000.6168398 AGE.137499675.14519378.947.3436 0.691076 MALE.1573178944.41910143.375.7074.570838 Characteristics in numerator of Prob[Y = 3] Constant -8.388756169.1357918-3.98.0001 INCOME 4.10485161 1.0304 4.080.0000.6168398 AGE.345736198.99507140E-01 3.474.0005 0.691076 MALE.540549359.7698869 1.950.0511.570838 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) +--------------------------------------------------------------------+ Information Statistics for Discrete Choice Model. M=Model MC=Constants Only M0=No Model Criterion F (log L) -406.3509-444.84113-605.81064 LR Statistic vs. MC 77.0309.00000.00000

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 44 Degrees of Freedom 9.00000.00000.00000 Prob. Value for LR.00000.00000.00000 Entropy for probs. 406.3509 444.84113 605.81064 Normalized Entropy.67071.7349 1.00000 Entropy Ratio Stat. 398.97109 31.93900.00000 Bayes Info Criterion 867.36958 944.40167 166.34067 BIC - BIC(no model) 398.97109 31.93900.00000 Pseudo R-squared.08658.00000.00000 Pct. Correct Prec. 64.98856.00000 5.00000 Means: y=0 y=1 y= y=3 yu=4 y=5, y=6 y>=7 Outcome.1648.089.0961.6499.0000.0000.0000.0000 Pred.Pr.1648.089.0961.6499.0000.0000.0000.0000 Notes: Entropy computed as Sum(i)Sum(j)Pfit(i,j)*logPfit(i,j). Normalized entropy is computed against M0. Entropy ratio statistic is computed against M0. BIC = *criterion - log(n)*degrees of freedom. If the model has only constants or if it has no constants, the statistics reported here are not useable. +--------------------------------------------------------------------+ Frequencies of actual & predicted outcomes Predicted outcome has maximum probability. Predicted ------ -------------------- + ----- Actual 0 1 3 Total ------ -------------------- + ----- 0 5 0 0 67 7 1 0 0 1 38 39 0 0 1 41 4 3 6 0 0 78 84 ------ -------------------- + ----- Total 11 0 44 437 Note that the variable name TRANSMOD was truncated because LIMDEP allows up to eight characters for a variable name. LIMDEP and STATA produce the same result of the multinomial logit model. 6.4 Multinomial Logit in SPSS SPSS has the Nomreg command to estimate the multinomial logit model. Like SAS, SPSS by default uses the largest value as the base outcome. NOMREG transmode WITH income age male /CRITERIA CIN(95) DELTA(0) MXITER(100) MXSTEP(5) CHKSEP(0) LCONVERGE(0) PCONVERGE(0.000001) SINGULAR(0.00000001) /MODEL /INTERCEPT INCLUDE /PRINT PARAMETER SUMMARY LRT.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 45 7. The Conditional Logit Regression Model Imagine a choice of the travel modes among air flight, train, bus, and car. The data set and model here are adopted from Greene (003). The model examines how the generalized cost measure (cost), terminal waiting time (time), and household income (income) affect the choice. These independent variables are not characteristics of subjects (individuals), but attributes of the alternatives. Thus, the data arrangement of the conditional logit model is different from that of the multinomial logit model (Figure ). Figure. Data Arrangement for the Conditional Logit Model ++ subject mode choice air train bus cost time income air_inc 1 1 0 1 0 0 70 69 35 35 1 0 0 1 0 71 34 35 0 1 3 0 0 0 1 70 35 35 0 1 4 1 0 0 0 30 0 35 0 1 0 1 0 0 68 64 30 30 0 0 1 0 84 44 30 0 3 0 0 0 1 85 53 30 0 4 1 0 0 0 50 0 30 0 3 1 0 1 0 0 19 69 40 40 3 0 0 1 0 195 34 40 0 The example data set has four observations per subject, each of which contains attributes of using air flight, train, bus, and car. The dependent variable choice is coded 1 only if a subject chooses that travel mode. The four dummy variables, air, train, bus, and car, are flagging the corresponding modes of transportation. See the appendix for details about the data set. 7.1 Conditional Logit in STATA (.clogit) STATA has the.clogit command to estimate the condition logit model. The group() option specifies the variable (e.g., identification number) that identifies unique individuals.. clogit choice air train bus cost time air_inc, group(subject) Iteration 0: log likelihood = -05.8187 Iteration 1: log likelihood = -199.3679 Iteration : log likelihood = -199.1851 Iteration 3: log likelihood = -199.1837 Iteration 4: log likelihood = -199.1837 Conditional (fixed-effects) logistic regression Number of obs = 840 LR chi(6) = 183.99 Prob > chi = 0.0000 Log likelihood = -199.1837 Pseudo R = 0.3160 choice Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- air 5.07443.7790551 6.68 0.000 3.68053 6.734363

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 46 train 3.869043.443169 8.73 0.000 3.00053 4.737555 bus 3.163194.450659 7.03 0.000.80689 4.045699 cost -.0155015.004408-3.5 0.000 -.04141 -.00686 time -.096148.0104398-9.1 0.000 -.1165865 -.0756631 air_inc.01387.01064 1.9 0.195 -.006869.033401 -------- Let us run the.listcoef command to compute factor changes in odds. For a one unit increase in the waiting time for a given travel mode, for example, we can expect a decrease in the odds of using that travel by 9 percent (or a factor of.9084), holding other variables constant.. listcoef clogit (N=840): Factor Change in Odds Odds of: 1 vs 0 -------------------------------------------------- choice b z P> z e^b -------------+------------------------------------ air 5.0744 6.684 0.000 18.665 train 3.86904 8.731 0.000 47.8965 bus 3.16319 7.05 0.000 3.6460 cost -0.01550-3.517 0.000 0.9846 time -0.0961-9.07 0.000 0.9084 air_inc 0.0139 1.95 0.195 1.0134 -------------------------------------------------- 7. Conditional Logit in SAS SAS has the MDC procedure to fit the conditional logit model. The TYPE=CLOGIT indicates the conditional logit model; the ID statement specifies the identification variable; and the NCHOICE=4 tells that there are four choices of the travel mode. PROC MDC DATA=masil.travel; MODEL choice = air train bus cost time air_inc /TYPE=CLOGIT NCHOICE=4; ID subject; RUN; The MDC Procedure Conditional Logit Estimates Algorithm converged. Model Fit Summary Dependent Variable choice Number of Observations 10 Number of Cases 840 Log Likelihood -199.1837 Maximum Absolute Gradient.7315E-8 Number of Iterations 5 Optimization Method Newton-Raphson AIC 410.5674

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 47 Schwarz Criterion 430.33938 Discrete Response Profile Index CHOICE Frequency Percent 0 1 58 7.6 1 63 30.00 3 30 14.9 3 4 59 8.10 Goodness-of-Fit Measures Measure Value Formula Likelihood Ratio (R) 183.99 * (LogL - LogL0) Upper Bound of R (U) 58.4 - * LogL0 Aldrich-Nelson 0.467 R / (R+N) Cragg-Uhler 1 0.5836 1 - exp(-r/n) Cragg-Uhler 0.65 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.6511 1 - (1-R/U)^(U/N) Adjusted Estrella 0.61 1 - ((LogL-K)/LogL0)^(-/N*LogL0) McFadden's LRI 0.316 R / U Veall-Zimmermann 0.6354 (R * (U+N)) / (U * (R+N)) N = # of observations, K = # of regressors Conditional Logit Estimates Parameter Estimates Standard Approx Parameter DF Estimate Error t Value Pr > t air 1 5.074 0.7791 6.68 <.0001 train 1 3.8690 0.4431 8.73 <.0001 bus 1 3.163 0.4503 7.03 <.0001 cost 1-0.0155 0.004408-3.5 0.0004 time 1-0.0961 0.0104-9.1 <.0001 air_inc 1 0.0133 0.0103 1.9 0.1954 Alternatively, you may use the PHREG procedure that estimates the Cox proportional hazards model for survival data and the conditional logit model. In order to make the data set consistent with the survival analysis data, you need to create a failure time variable, failure=1 choice. The identification variable is specified in the STRATA statement. The NOSUMMARY option suppresses the display of the event and censored observation frequencies. PROC PHREG DATA=masil.travel NOSUMMARY; STRATA subject; MODEL failure*choice(0)=air train bus cost time air_inc; RUN;

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 48 The PHREG Procedure Model Information Data Set MASIL.TRAVEL Dependent Variable failure Censoring Variable choice Censoring Value(s) 0 Ties Handling BRESLOW Number of Observations Read 840 Number of Observations Used 840 Convergence Status Convergence criterion (GCONV=1E-8) satisfied. Model Fit Statistics Without With Criterion Covariates Covariates - LOG L 58.44 398.57 AIC 58.44 410.57 SBC 58.44 430.339 Testing Global Null Hypothesis: BETA=0 Test Chi-Square DF Pr > ChiSq Likelihood Ratio 183.9869 6 <.0001 Score 173.4374 6 <.0001 Wald 103.7695 6 <.0001 Analysis of Maximum Likelihood Estimates Parameter Standard Hazard Variable DF Estimate Error Chi-Square Pr > ChiSq Ratio air 1 5.0743 0.77905 44.6799 <.0001 18.65 train 1 3.86904 0.44313 76.343 <.0001 47.896 bus 1 3.16319 0.4507 49.3530 <.0001 3.646 cost 1-0.01550 0.00441 1.3671 0.0004 0.985 time 1-0.0961 0.01044 84.7778 <.0001 0.908 air_inc 1 0.0139 0.0106 1.6763 0.1954 1.013 While the MDC procedure reports t statistics, the PHREG procedure computes chi-squared (e.g., 1.3671=-3.5^). The PHREG presents the hazard ratio at the last column of the output, which is equivalent to the factor changes under the e^b column of the SPost.listcoef command.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 49 7.3 Conditional Logit in LIMDEP (Clogit$) LIMDEP fits the conditional logit model using either the Clogit$ or the Logit$ command. The Clogit$ command has the Choices$ subcommand to list the choices available. CLOGIT; Lhs=choice; Rhs=air,train,bus,cost,time,air_inc; Choices=air,train,bus,car$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ Discrete choice (multinomial logit) model Maximum Likelihood Estimates Model estimated: Sep 19, 005 at 09:0:39PM. Dependent variable Choice Weighting variable None Number of observations 10 Iterations completed 6 Log likelihood function -199.184 Log-L for Choice model = -199.1837 R=1-LogL/LogL* Log-L fncn R-sqrd RsqAdj Constants only -83.7588.985.9150 Response data are given as ind. choice. Number of obs.= 10, skipped 0 bad obs. +---------------------------------------------+ +---------+--------------+----------------+--------+---------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] +---------+--------------+----------------+--------+---------+ AIR 5.0744399.77905514 6.684.0000 TRAIN 3.8690470.4431685 8.731.0000 BUS 3.1631941.4506593 7.05.0000 COST -.15501553E-01.44079931E-0-3.517.0004 TIME -.961479610E-01.10439847E-01-9.07.0000 AIR_INC.1387065E-01.106407E-01 1.95.1954 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) The Clogit$ command has the Ias$ subcommand to conduct the Hausman test for the IIA assumption (e.g., Ias=air,bus$). Unfortunately, the subcommand does not work in this model because the Hessian is not positive definite. The Logit$ command takes the panel data analysis approach. The Pds$ subcommand specifies the number of time periods. The two commands produce the same result. LOGIT; Lhs=choice; Rhs=air,train,bus,cost,time,air_inc; Pds=4$ +--------------------------------------------------+ Panel Data Binomial Logit Model Number of individuals = 10

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 50 Number of periods = 4 Conditioning event is the sum of CHOICE Distribution of sums over the 4 periods: Sum 0 1 3 4 5 6 Number 0 10 0 0 0 5 10 Pct..00100.00.00.00.00.00.00 +--------------------------------------------------+ Normal exit from iterations. Exit status=0. +---------------------------------------------+ Logit Model for Panel Data Maximum Likelihood Estimates Model estimated: Sep 19, 005 at 09:1:58PM. Dependent variable CHOICE Weighting variable None Number of observations 840 Iterations completed 6 Log likelihood function -199.184 Hosmer-Lemeshow chi-squared = 51.448 P-value=.00000 with deg.fr. = 8 Fixed Effects Logit Model for Panel Data +---------------------------------------------+ +---------+--------------+----------------+--------+---------+ Variable Coefficient Standard Error b/st.er. P[ Z >z] +---------+--------------+----------------+--------+---------+ AIR 5.0744399.77905514 6.684.0000 TRAIN 3.8690470.4431685 8.731.0000 BUS 3.1631941.4506593 7.05.0000 COST -.15501553E-01.44079931E-0-3.517.0004 TIME -.961479610E-01.10439847E-01-9.07.0000 AIR_INC.1387065E-01.106407E-01 1.95.1954 (Note: E+nn or E-nn means multiply by 10 to + or -nn power.) 7.4 Conditional Logit in SPSS Like the SAS PHREG procedure, the SPSS Coxreg command, which was designed for survival analysis data, provides a backdoor way of estimating the conditional logit model. COXREG failure WITH air train bus cost time air_inc /STATUS=choice(1) /STRATA=subject.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 51 8. The Nested Logit Regression Model Now, consider a nested structure of choices. When the IIA assumption is violated, one of the alternatives is the nested (multinomial) logit model. This chapter replicates the nested logit model discussed in Greene (003). P ( choice, branch) = P( choice branch) * P( branch) P( choice branch) = Pchild ( α 1air + αtrain + α3bus + β1 cost + βtime) P branch) = P ( γ air _ inc + τ IV + τ IV ) ( parent income fly fly ground ground 8.1 Nested Logit in STATA (.nlogit) The STATA.nlogit command estimates the nested multinomial logit model. First you need to create a variable based on the specification of the tree using the.nlogitgen command. From the top, the parent-level has fly and ground branches; the fly branch of the child-level has air flight (1); the ground branch has train (), bus (3), and car (4).. nlogitgen tree = mode(fly: 1, ground: 3 4) new variable tree is generated with groups label list lb_tree lb_tree: 1 fly ground The.nlogittree command.displays the tree-structure defined by the.nlogitgen command.. nlogittree mode tree tree structure specified for the nested logit model top --> bottom tree mode -------------------------- fly 1 ground 3 4 The.nlogit command consists of three parts. The dependent or choice variable follows the command. Utility functions of the parent and child-levels are then specified. The group()option specifies an identification or grouping variable.. nlogit choice (mode=air train bus cost time) (tree=air_inc), /// group(subject) notree nolog Nested logit regression Levels = Number of obs = 840 Dependent variable = choice LR chi(8) = 194.9313

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 5 Log likelihood = -193.65615 Prob > chi = 0.0000 Coef. Std. Err. z P> z [95% Conf. Interval] -------------+---------------------------------------------------------------- mode air 6.0455 1.198907 5.04 0.000 3.69441 8.3907 train 5.064679.660317 7.65 0.000 3.76711 6.3637 bus 4.09630.615158 6.66 0.000.890614 5.30199 cost -.0315888.0081566-3.87 0.000 -.0475754 -.01560 time -.116183.014193-7.97 0.000 -.1403111 -.084954 -------------+---------------------------------------------------------------- tree air_inc.0153337.0093814 1.63 0.10 -.0030534.033709 -------------+---------------------------------------------------------------- (incl. value parameters) tree /fly.5859993.1406199 4.17 0.000.3103894.861609 /ground.3889488.13663 3.15 0.00.1465753.63134 LR test of homoskedasticity (iv = 1): chi()= 10.94 Prob > chi = 0.004 The notree option does not show the tree-structure and the nolog suppresses an iteration log of the log likelihood. Note that the /// joins the next command line with the current line. 8. Nested Logit in SAS The SAD MDC procedure fits the conditional logit model as well as the nested multinomial logit model. For the nested logit model, you have to use the UTILITY statement to specify utility functions of the parent (level ) and child level (level 1), and the NEST statement to construct the decision-tree structure. Note that 3 4 @ reads that there are three nodes at the child level under the branch at the parent-level. PROC MDC DATA=masil.travel; MODEL choice = air train bus cost time air_inc /TYPE=NLOGIT CHOICE=(mode); ID subject; UTILITY U(1,) = air train bus cost time, U(, 1 ) = air_inc; NEST LEVEL(1) = (1 @ 1, 3 4 @ ), LEVEL() = (1 @ 1); RUN; The MDC Procedure Nested Logit Estimates Algorithm converged. Model Fit Summary Dependent Variable choice Number of Observations 10 Number of Cases 840

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 53 Log Likelihood -193.65615 Maximum Absolute Gradient 0.0000147 Number of Iterations 15 Optimization Method Newton-Raphson AIC 403.3130 Schwarz Criterion 430.08916 Discrete Response Profile Index mode Frequency Percent 0 1 58 7.6 1 63 30.00 3 30 14.9 3 4 59 8.10 Goodness-of-Fit Measures Measure Value Formula Likelihood Ratio (R) 194.93 * (LogL - LogL0) Upper Bound of R (U) 58.4 - * LogL0 Aldrich-Nelson 0.4814 R / (R+N) Cragg-Uhler 1 0.6048 1 - exp(-r/n) Cragg-Uhler 0.6451 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.6771 1 - (1-R/U)^(U/N) Adjusted Estrella 0.6485 1 - ((LogL-K)/LogL0)^(-/N*LogL0) McFadden's LRI 0.3348 R / U Veall-Zimmermann 0.655 (R * (U+N)) / (U * (R+N)) N = # of observations, K = # of regressors Nested Logit Estimates Parameter Estimates Standard Approx Parameter DF Estimate Error t Value Pr > t air_l1 1 6.043 1.1989 5.04 <.0001 train_l1 1 5.0646 0.660 7.65 <.0001 bus_l1 1 4.0963 0.615 6.66 <.0001 cost_l1 1-0.0316 0.008156-3.87 0.0001 time_l1 1-0.116 0.0141-7.97 <.0001 air_inc_lg1 1 0.0153 0.009381 1.63 0.10 INC_LG1C1 1 0.5860 0.1406 4.17 <.0001 INC_LG1C 1 0.3890 0.137 3.15 0.0017 The /fly and /ground in the STATA output above are equivalent to the INC_LG1C1 and INC_LG1C in the SAS output. SAS and STATA produce the same result.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 54 9. Conclusion The appropriate type of categorical dependent variable model (CDVM) is determined largely by the level of measurement of the dependent variable. The level of measurement should be, however, considered in conjunction with your theory and research questions (Long 1997). You must also examine the data generation process (DGP) of a dependent variable to understand its behavior. Sophisticated researchers pay special attention to censoring, truncation, sample selection, and other particular patterns of the DGP. If your dependent variable is a binary variable, you may use the binary logit or probit regression model. For ordinal responses, try to fit the ordered logit/probit regression models. If you have a nominal response variable, investigate the DGP carefully and then choose one of the multinomial logit, conditional logit, and nested logit models. In order to use the conditional logit and nested logit, the data set requires a different setup. You should check the key assumptions of the CDVMs when fitting the models. Examples are the parallel regression assumption in the ordered logit model and the independence of irrelevant alternatives (IIA) assumption in the multinomial logit model. You may conduct the Brant test and Hausman test for these assumptions. Since CDVMs are nonlinear, they produce estimates that are difficult to interpret intuitively. Consequently, researchers need to spend more time and effort interpreting the results substantively. Reporting parameter estimates and goodness-of-fit statistics is not sufficient. J. Scott Long (1997) and Long and Freese (003) provide good examples of meaningful interpretations using predicted probabilities, factor changes in odds, and marginal/discrete changes of predicted probabilities. Regarding statistical software for CDVMs, I would recommend the SAS QLIM and MDC procedures of SAS/ETS (see Table 3 and 4). SAS has other procedures such as LOGISTIC, GENMODE, and PROBIT for CDVMs, but the QLIM procedure seems best for binary and ordinal response models, and the MDC procedure is good for nominal dependent variable models. I also strongly recommend STATA with SPost, since it has various useful commands for CDVMs such as.prchange,.listcoef, and.prtab. I encourage SAS Institute to develop additional statements similar to those SPost commands. LIMDEP supports various CDVMs addressed in Greene (003) but does not seem stable and reliable. Thus, I recommend LIMDEP for CDVMs that SAS and STATA do not support. SPSS is not currently recommended for CDVMs.

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 55 Appendix: Data Sets The first data set students is a subset of data provided for David H. Good s class in the School of Public and Environmental Affairs (SPEA). The data were manipulated for the sake of data security. owncar: 1 if a student owns a car parking: Illegal parking (0=none, 1=sometimes, and =often) offcamp: 1 if a student lives off-campus transmode: the mode of transportation (0=walk, 1=bike, =bus, 3=car) age: students age income: monthly income male: 1 for male and 0 for female. tab male owncar owncar male 0 1 Total -----------+----------------------+---------- 0 76 111 187 1 77 173 50 -----------+----------------------+---------- Total 153 84 437. tab male offcamp offcamp male 0 1 Total -----------+----------------------+---------- 0 7 180 187 1 5 45 50 -----------+----------------------+---------- Total 1 45 437. tab male parking parking male 0 1 Total -----------+---------------------------------+---------- 0 170 13 4 187 1 43 7 0 50 -----------+---------------------------------+---------- Total 413 0 4 437. tab male transmode transmode male 0 1 3 Total -----------+--------------------------------------------+---------- 0 38 18 0 111 187 1 34 1 173 50 -----------+--------------------------------------------+---------- Total 7 39 4 84 437

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 56. sum income age Variable Obs Mean Std. Dev. Min Max -------------+-------------------------------------------------------- income 437.6168398.17918.4 1.7 age 437 0.69108 1.61081 18 9 The second data set travel on travel mode choice is adopted from Greene (003). You may get the data from http://pages.stern.nyu.edu/~wgreene/text/tables/tablelist5.htm subject: identification number mode: 1=Air, =Train, 3=Bus, 4=Car choice: 1 if the travel mode is chosen time: terminal waiting time, 0 for car cost: generalized cost measure income: household income air_inc: interaction of air flight and household income, air*income air: 1 for the air flight mode, 0 for others train: 1 for the train mode, 0 for others bus: 1 for the bus mode, 0 for others car: 1 for the car mode, 0 for others failure: failure time variable, 1-choice. tab choice mode mode choice 1 3 4 Total -----------+--------------------------------------------+---------- 0 15 147 180 151 630 1 58 63 30 59 10 -----------+--------------------------------------------+---------- Total 10 10 10 10 840. sum time income air_inc Variable Obs Mean Std. Dev. Min Max -------------+-------------------------------------------------------- time 840 34.5899 4.94861 0 99 income 840 34.5476 19.67604 7 air_inc 840 8.636905 17.9106 0 7

003-005, The Trustees of Indiana University Categorical Dependent Variable Models: 57 References Allison, Paul D. 1991. Logistic Regression Using the SAS System: Theory and Application. Cary, NC: SAS Institute. Greene, William H. 00. LIMDEP Version 8.0 Econometric Modeling Guide. Plainview, New York: Econometric Software. Greene, William H. 003. Econometric Analysis, 5 th ed. Upper Saddle River, NJ: Prentice Hall. Long, J. Scott, and Jeremy Freese. 003. Regression Models for Categorical Dependent Variables Using STATA, nd ed. College Station, TX: STATA Press. Long, J. Scott. 1997. Regression Models for Categorical and Limited Dependent Variables. Advanced Quantitative Techniques in the Social Sciences. Sage Publications. Maddala, G. S. 1983. Limited Dependent and Qualitative Variables in Econometrics. New York: Cambridge University Press. SAS Institute. 004. SAS/STAT 9.1 User's Guide. Cary, NC: SAS Institute. SPSS Inc. 001. SPSS 11.0 Syntax Reference Guide. Chicago, IL: SPSS Inc. STATA Press. 004. STATA Base Reference Manual, Release 8. College Station, TX: STATA Press. Stokes, Maura E., Charles S. Davis, and Gary G. Koch. 000. Categorical Data Analysis Using the SAS System, nd ed. Cary, NC: SAS Institute. Acknowledgements I am grateful to Jeremy Albright and Kevin Wilhite at the UITS Center for Statistical and Mathematical Computing for comments and suggestions. I also thank J. Scott Long in Sociology and David H. Good in the School of Public and Environmental Affairs, Indiana University, for their insightful lectures and data set. Revision History 003. First draft 004. Second draft 005. Third draft (Added bivariate logit/probit models and the nested logit model with LIMDEP examples).