On the Use of Instrumental Variables in Accounting Research

Transcription

1 On the Use of Instrmental Variables in Acconting Research David F. Larcker The Wharton School University of Pennsylvania 133A Steinberg-Dietrich Hall Philadelphia, PA, Tjomme O. Rstics The Wharton School University of Pennsylvania 1300 Steinberg-Dietrich Hall Philadelphia, PA, Revised November 9, 004 Abstract The se of instrmental variables is becoming an increasingly common method for dealing with the econometric problems cased by endogeneity in acconting research. While instrmental variables estimation is the standard textbook soltion to mitigating the inconsistency in parameter estimates cased by endogeneity, the appropriateness of instrmental variable methods in real settings is not obvios. We provide conditions nder which instrmental variables methods are likely to be preferred over reglar OLS. Or srvey of the se of instrmental variables methods in acconting research raises dobt abot whether these conditions are generally met. We illstrate these concerns with two examples from contemporary acconting research: the effect of disclosre on cost of capital and the effect of insider power on CEO compensation levels. We conclde with some recommendations on the preferred approach for applying instrmental variables estimation. We appreciate helpfl comments from Ted Goodman, Jeffrey Ng, Scott Richardson and Rodrigo Verdi. We grateflly acknowledge financial spport from the Wharton school. Tjomme Rstics is also gratefl for financial spport from the Deloitte & Toche Fondation.

2 1. Introdction The econometric problems cased by the endogeneity of predictor variables in a regression model are one of the most difficlt isses for empirical acconting research. Endogeneity exists when some of the right-hand-side (RHS) variables in an eqation are correlated with the tre (bt nobserved) error term in the eqation. This problem commonly occrs when the RHS variables are choice variables and some of the determinants of these choice variables also affect the dependent variable. If these determinants of the RHS variables are not inclded in the regression eqation being estimated, the reslting ordinary least sqares (OLS) parameter estimates will be inconsistent de to the well-known correlated omitted variables problem. In order to mitigate the econometric problems cased by endogeneity, it has become commonplace in acconting research to implement some type of instrmental variables (IV) estimation procedre. In particlar, the researcher first selects a set of variables that are assmed to be exogenos and then two-stage least sqares (SLS) estimation is sed to estimate the coefficients in the regression model. 1 This standard textbook soltion to endogeneity works if the researcher can find instrmental variables that are correlated with the endogenos regressor bt ncorrelated with the error in the strctral eqation. However, as Maddala (1977, p. 154) points ot Where do yo get sch a variable? Eqally importantly for applied research, what happens to the statistical properties of IV estimates when the instrmental variables do not precisely conform to the textbook definition of these variables? 1 Althogh or discssion focses on SLS estimation, the same concerns are inherent in treatment effects models (Heckman-type models), 3SLS, fll information maximm likelihood (FIML), and other estimation methods that rely on instrmental variables. 1

3 The prpose of this paper is to synthesize the extensive literatre in statistics and econometrics that examines the properties of IV estimators and provide acconting researchers with a framework to appropriately evalate and interpret their application of instrmental variables. Three aspects of IV estimation are considered in or analysis. First, we examine both asymptotic and finite sample properties of OLS and IV estimators. Second, the importance of problems with weak instrments (i.e., instrments that explain only a modest proportion of the variation in the endogenos variable) is examined. Finally, we analyze the sitation where the selected instrmental variables are not completely exogenos (i.e., the instrments that are somewhat correlated with the error term in the strctral model or semi-endogenos ). Or analytical reslts and nmerical simlations indicate that the IV approach typically sed in acconting research is in many cases nlikely to prodce estimates with desirable econometric properties. It can easily be the case that IV estimates are more biased than simple OLS estimates that make no explicit correction for endogeneity. In addition, we examine the reslts prodced by OLS and IV estimators for two typical acconting research stdies where there is sbstantial reason to sspect that the primary regressor is endogenos. In both stdies, we conclde that OLS estimation is preferred to IV estimation. The primary implication of or analysis is that acconting researchers need to be mch more carefl in applying IV estimation. In particlar, acconting researchers shold generally refrain from making broad claims that endogeneity problems have been effectively mitigated by the se of IV methods. The remainder of the paper is composed of five sections. Section provides a smmary evalation of the instrmental variables applications in acconting. The

4 asymptotic and finite sample properties of IV estimators are developed in Sections 3 and 4, respectively. Section 5 compares the OLS and IV reslts for two contemporary topics in acconting research whether the cost of capital is an increasing fnction of corporate disclosre and whether chief exective officer (CEO) compensation is a fnction of the power of corporate insiders. A smmary of or analysis and recommendations regarding the se of IV methods in acconting research is provided in Section 6.. Instrmental Variable Applications in Acconting Research In order to ascertain the se of IV estimation, we condcted an electronic search for the term SLS in Acconting, Organizations and Society, Contemporary Acconting Research, Jornal of Acconting and Economics, Jornal of Acconting Research, Jornal of Financial Economics, and The Acconting Review. This search prodced the 35 articles identified in Table 1. Most of these articles have been pblished after 000, indicating that IV estimation is a commonly sed methodology in contemporary acconting research. Acconting researchers generally se instrmental variables in order to mitigate endogeneity of the predictor variables or to identify a simltaneos system of endogenos variables (Table ). Most of the stdies have reasonably large sample sizes with the mean (median) nmber of observations being,393 (654). The typical stdy has less than three endogenos regressors and sed between five and seven instrments (Table 3). The mean (median) R from the first-stage regression (only reported by 6 of the 35 articles) is 31 (6) percent. However, one problem with these R measres is that they Instrmental variables are also sed to mitigate measrement error in the independent variables. This has a long history in acconting research and dates back to at least Beaver, Kettler and Scholes (1970). We do not consider these applications in or discssion becase sophisticated latent variable models exist to address measrement error isses when there are mltiple indicators for the same constrct. 3

5 are generally not the partial explanatory power for the instrments that are niqe to the first stage regression. 3 Ths, the strength of the instrmental variables in the first-stage regression is likely sbstantially overstated. It is also instrctive to examine the stated reasons and econometric claims made by these researchers sing IV methods. Rather than discssing individal papers, it is perhaps more sefl to paraphrase the typical discssion in pblished papers. Acconting researchers clearly nderstand that endogeneity is a serios econometric problem that confonds their interpretation of coefficient estimates: ignoring the simltaneos endogeneity of a and b engenders problems of estimation bias, which prevents meaningfl interpretation of variable coefficients. Or econometric approach enables s to relax the somewhat implasible assmptions of prior research regarding the exogenos natre of either a or b. we investigate whether previosly docmented associations between a and b are the reslt of biased estimation indced by sing endogenos variables in single-eqation models. However, one serios concern across these stdies is that there is little attempt to develop a model that explicitly identifies and jstifies the endogenos (choice) variables and the exogenos and instrmental variables (i.e., those variables that are assmed to be either pre-determined or are otside the model being examined). 4 Most acconting 3 The typical analysis in applied research involves an endogenos y that is a fnction of an endogenos x variable and a set of exogenos control variables (z 1 ). In addition, there are mltiple instrments, exogenos variables (z ) that are not inclded in the eqation describing y. In this case the proper measre of the strength of the instrment is the partial R. We can easily compte the partial R as the following: ( R y,z - R y, z 1 )/(1- R y, z 1 ), where z is the combined set of z 1 and z. 4 The ideal approach is perhaps as follows: develop an economic theory of the decision making process regarding the relation of interest, translate the theory into a set of strctral eqation models that describe the decision setting, precisely identify the endogenos and exogenos variables, develop the redced form eqations where the endogenos variables are only fnctions of exogenos variables, and estimate the 4

6 research estimates some type of convenient representation that is assmed to be the redced form, bt there is almost never any discssion of the nderlying strctral eqation model. Moreover, there is almost no discssion regarding the choice of specific variables for instrments. For example, it is not clear why these instrmental variables are assmed to be exogenos (i.e., ncorrelated with the error term in the strctral model) and whether the instrmental variables exhibit a lower correlation with the strctral eqation error term than the endogenos regressor variable. Despite these isses, acconting researchers typically make rather bold claims abot the ability of IV methods in addressing endogeneity: Or findings are robst to controlling for endogeneity among or variables. We address potential endogeneity problems by estimating a -stage least sqares (SLS) model. The reslts of the SLS sggest that the relation is not driven by the potential endogeneity of or variables. Althogh IV methods may mitigate the econometric problems indced by endogeneity, we believe that the typical instrments sed in acconting stdies are not well jstified and are likely to be inadeqate. 5 In order to get a feel for the difficlty in obtaining a good instrment, consider the following example. Sppose we are interested in estimating the effect of disclosre qality on firms cost of capital. To get a meaningfl variation in disclosre qality we se an international dataset. In this case we are obviosly worried that there are redced form model parameters. Assming that the model is identified, the strctral eqation parameters of interest can then be derived from the estimates obtained from the redced form eqations. 5 Some papers attach varios caveats to the econometric approach, sch as Only a and b are treated as endogenos, other firm-specific variables are assmed to be exogenos or pre-determined variables. This may not be appropriate. Ths, the sccess of the IV method appears to be ambigos even to the researcher. 5

7 nidentified factors that affect both cost of capital and disclosre qality. We might consider sing a contry legal origin (English common law, French code law etc.) as an instrment for disclosre qality (several international acconting papers se legal origin as either instrment or key independent variable, e.g., Ball, Kothari, and Robin, 000; Bshman, Piotroski and Smith, 005; Lez, Nanda, and Wysocki, 003). Unlike many other instrments that are typically sed in acconting research, legal origin seems clearly predetermined. That is, the company does not determine the legal origin, so we are not worried abot reverse casality. Althogh, even here, there are some cases where the company can adopt a different legal origin throgh a cross listing. However, this can be controlled for in the regression analysis. However, the fact that legal origin is predetermined does not necessarily make it a good instrment. What we shold worry abot is that legal origin affects other instittions (sch as property rights and investor protection laws) that cold also affect a firm s cost of capital. This is graphically displayed in Figre 1. The relation we are ltimately interested in is the relation between disclosre and cost of capital, (a). We can se legal origin as an instrment throgh relation (b). However, this only gives the correct inference if either (c) or (d) (or both) are eqal to zero. This condition is nlikely to be satisfied. Ths there is some reason to be skeptical abot this instrment. In order to provide insight into the choice of estimation methods, it is necessary to identify the sitations where the se of instrmental variables prodces better estimates than ordinary least sqares (OLS). In the remainder of the paper, we focs on two aspects of this econometric qestion. First, we examine the bias of the estimates if the instrments are strictly exogenos, bt they have weak explanatory power for explaining 6

8 the endogenos variable. Second, we examine the bias of the estimates when the instrments are not completely exogenos. The impact of these two fndamental isses on the bias of instrmental variables estimators is examined in Section 3 (asymptotic reslts) and Section 4 (finite sample reslts). 3. Asymptotic Properties of Instrmental Variable Estimators 3.1. Basic Strctre The typical model in describing the impact of the predictor variable (x) on the otcome variable (y) is the following: y = β x + (1) It is well known that a consistent estimate of β can be obtained sing OLS as long the correlation between x and is eqal to zero. This can be seen from the probability limit (plim) of the OLS estimator: plimb OLS cov( x, ) = β + = β + corr( x, ) () var( x) x If x and are ncorrelated, the second term will go to zero and the OLS estimator is a consistent estimator of the tre coefficient. However, this assmption will not be satisfied when there are determinants of x that are also correlated with y and these other determinants are not inclded in eqation (1). When x and are correlated, the second term in eqation () will not go to zero and the OLS estimate of β will be inconsistent. If x and are correlated, the typical textbook prescription is to se instrmental variables (e.g., Wooldridge, 00; Greene, 00). That is, it is necessary to incorporate a variable (z) that is correlated with x bt not with. If sch a variable exists, eqation (1) 7

9 can be estimated sing instrmental variables estimation (IV). The reslting estimator will be consistent, as can be seen from the probability limit of the IV estimator: plimb IV cov( z, ) corr( z, ) = β + = β + (3) cov( z, x) corr( z, x) x If z is correlated with x bt not with, the second term goes to zero as the sample size increases and the IV estimator is a consistent estimator of the tre coefficient. This holds even for small corr (z,x), as long as corr (z,x) 0. However, as we discss in Section 4, the size of corr (z,x) can case serios problems in finite samples. Bartels (1991) also derives the asymptotic mean sqare error of the instrmental variable parameter estimate as: [ n ] [ 1/ R ] [ 1+ nr ] (4) / x xz z where R ij is the sqared poplation correlation between variables i and j, and n is the sample size. The first term in eqation (4) is the asymptotic mean sqared error of OLS and the second term (which is greater than one) is related to the loss in efficiency cased by sing an IV estimate as opposed to an OLS estimate. The third term is related to the bias in the IV estimator cased by the se of inappropriate instrments. This analysis demonstrates that even if corr (z,) is eqal to zero, the actal asymptotic standard error for the IV estimator will be greater than the OLS standard error by the sqare root of [ / R ] 1 xz. Althogh asymptotic bias in the IV estimate is not an isse (assming that corr (z,) = 0), the associated standard error for the IV estimate is sbstantially larger than the standard error for the OLS estimate. For example, if the R = 0. 5(abot the median first stage R, see Table 3), the estimated IV standard error will larger than the OLS xz 8

10 standard error by a factor ( = 1/ 0. 5 ). Ths, ignoring the impact of bias in the estimate, the power associated with IV estimation may be sbstantially less than that for OLS. 3.. Semi-Endogenos Instrments Finding a trly exogenos variable that is also correlated with the x is a danting task for applied researchers. 6 As discssed by Bartels (1991), it is sefl to nderstand whether a semi-endogenos variable (i.e., an instrment that is somewhat correlated with the error term in the strctral eqation) will prodce IV estimates that are preferred to OLS estimates. We know from eqation (3) that the reslting IV estimator will not be consistent, bt the IV estimate may still have an asymptotic bias that is smaller than the bias in the OLS estimate. It is possible to identify the circmstances where the bias in the IV estimator is smaller by comparing the bias terms in eqations () and (3). 7 The IV estimator has smaller bias if the following holds: x R R z xz < Rx (5) x Rearranging and simplifying eqation (5) yields the following condition for the speriority of the IV estimator over the OLS estimator: R z < R R (6) xz x As can be seen from eqation (6), the relative endogeneity of x and z, and the correlation between x and z are the critical determinants of whether IV estimators are 6 Determining whether the corr (z,) is eqal to zero is especially problematic becase is not observable. This means that it is not possible to directly estimate the correlation between z and. We can estimate the second stage residal, û. This can be sed in a test of overidentifying restrictions, see section We compare the sqared bias terms to avoid problems with sign flips. 9

11 preferred to OLS estimators. For example, if the R = then the correlation between z and can be no more than 10% of the correlation between x and for IV estimation to be preferred over OLS. If the instrment selected by the researcher is moderately to highly correlated with the x variable (which can be tested) and a compelling theoretical or practical argment can be made regarding why the instrment is considerably more exogenos than the x variable, the IV estimator will be preferred to the OLS estimator. However, if the correlation between the instrment and the x variable is low and the researcher has some concern abot whether the instrment is trly exogenos, it can easily be the case that the OLS estimator is preferred to the instrmental variable estimator Testing the Appropriateness of Instrments Several tests have been developed in connection with the se of instrmental variables estimation (e.g., Chapter 6 in Wooldridge, 00). The most common is the Hasman test (Hasman, 1978) which provides a formal test on whether the IV estimator is significantly different from the OLS estimator. Under the assmption of the appropriateness of the instrments, this test can be sed to determine the existence of an endogeneity problem and ths the appropriateness of sing OLS. This test statistic can also easily be compted by inclding both the observed x and the predicted x variable from the first stage regression into an OLS version of the second stage regression. If the coefficient on the predicted x is significant, the Hasman test rejects the nll of no endogeneity problem. Variations of this test are applied in the majority of the papers investigated in or srvey (see Table 4). xz 10

12 Ideally we wold like to have data on the strctral error terms, so that we can test whether the instrments and the second stage error are ncorrelated. 8 Unfortnately, the nobservability of this strctral eqation error term renders this test impossible. However, it is possible to correlate the instrments with the estimated error term in the second stage eqation. In case of over-identified models (the nmber of instrments exceeds the nmber of endogenos regressors), we can se this test to determine the appropriateness of the instrments nder the assmption that at least one of the instrments is valid (see also Hasman, 1978). 9 This test shold be performed before the Hasman test, as the latter is not valid if the over-identifying restrictions test rejects the appropriateness of the instrments (e.g., Godfrey and Htton, 1994). The over-identifying restriction test statistic can be obtained by a regression of the second stage residals on all exogenos variables. If the instrments are valid, the coefficients on the instrments shold be close to zero. The formal test is based on the R from this model being close to zero. In particlar, nr is distribted χ with K-L degrees of freedom, where K is the nmber of exogenos variables niqe to the first stage and L is the nmber of endogenos explanatory variables. It is important to note that the test reqires that at least one of the instrments is valid (i.e., exogenos). If this does not hold, we can have a sitation where the instrments have similar bias, so that the test will not reject (even in large samples), even thogh the coefficient can be severely biased. 8 In or srvey of the acconting literatre, we fond several instances where the athors correlated the instrment(s) with the dependent variable (with or withot controls) instead of the error term. Upon finding an insignificant relation, they conclded that they had a valid instrment. This is a completely inappropriate procedre. 9 Obviosly, this will be zero by constrction for jst-identified models (the nmber of instrments eqals the nmber of endogenos regressors). 11

13 In or srvey, we investigate whether athors perform a test of over-identifying restrictions. Since the latter can only be tested when the model is over-identified, we split the sample in jst identified and over-identified models and report the reslts separately (Table 4). In contrast to the Hasman test, we find that very few papers tilize this test even thogh most se over-identified models. Finally, in very large samples both tests will always reject, and therefore it is sefl to spplement the formal test with a sensitivity analysis that examines whether the se of different instrmental variables yields very different reslts. 4. Finite Sample Properties of Instrmental Variable Estimators The analysis in Section 3 focsed strictly on an asymptotic (very large sample) reslts. While this asymptotic analysis is straightforward to compte and provides important limiting reslts, it does not prodce insight into the properties of OLS and IV estimators applied in finite samples. Richardson (1968) and Sawa (1969) provide the exact finite sample properties of some class of IV estimators. They show that the finite sample bias of the IV estimator is in the same direction as the bias in the OLS estimator. Moreover, Nelson and Startz (1990a, 1990b) find the asymptotic distribtion of the IV estimator is a very poor approximation to the finite sample distribtion when the instrment is only weakly correlated with the regressor. In addition to the above, an extensive literatre has evolved arond the problems with weak instrments (e.g., Bond, Jaeger, and Baker (1995), Staiger and Stock (1997), Hahn and Hasman (003)) 1

14 4.1 Basic Strctre We examine the finite sample properties of IV estimators sing the following model: x = λ + ε z = γε + θ + δ y = βx + (7) The actal endogenos variable (y) is assmed to be a linear fnction of the predictor variable (x) and the random strctral eqation error (). The estimate of primary interest is the strctral eqation parameter (β). The predictor variable (x) is assmed to be a fnction of a random variable (ε) pls a fnction of the random strctral eqation error () via the λ parameter. If λ.is eqal to zero, x is strictly exogenos. The instrmental variable (z) is composed of three components. First, z is assmed to be a fnction of random variable (ε) via the γ parameter. Since ε is the exogenos part of x, the parameter γ partially determines the strength of the instrmental variable. Second, the instrmental variable also allows z to be semi-endogenos via the relation to the strctral eqation error (). If the parameter θ is eqal to zero, z will be strictly exogenos in large samples. Third, z is a fnction of random error (δ). We assme that ε, δ, and have a normal distribtion with a poplation mean of zero and variances of,, respectively. Finally, we assme that the poplation covariance ε, δ matrix of ε, δ, and is diagonal. Althogh this strctre is simple, it is sfficiently complex to illstrate the finite sample isses with IV estimation. For this model, the IV estimator can be characterized as follows: 13

15 1 z m z biv = β + n = β +, (8) 1 γm m θm λm xz εε + δε + ε + z n where the m ij denotes either sample second moments or sample cross prodcts. When we set β = 0 and θ = 0, eqation (8) simplifies to 10 : b m z IV = (9) γmεε + mδε + λm z This expression will asymptotically approach zero becase this is expression for the bias in the IV estimator. However, in finite samples, the estimator is not wellbehaved becase, even if θ = 0, m z is not eqal to zero in finite samples. Moreover, similar to the analysis in Nelson and Startz (1990a), there is a discontinity in b IV at the point where m z = -(γ m εε +m δε )λ -1. The bias in b IV when m z is either a large negative or positive nmber is eqal to λ -1. However, when m z approaches -(γ m εε +m δε )λ -1 from the left (right), b IV approaches positive (negative) infinity. Ths, the moments for b IV do not exist. Obviosly, the OLS estimator is also biased in finite samples and the bias (recall β = 0) is eqal to: m = (10) mεε + λ m x bols Althogh b IV and b OLS are biased in finite samples, it is mathematically difficlt to compte the exact finite sample distribtion for the estimators in order to compare their statistical properties. As a reslt, we develop or finite sample reslts sing the type of nmerical simlations that is common in applied econometric research. 10 We do not make the assmption that δ and ε are ncorrelated in finite samples as in Nelson and Startz (1990a and 1990b). Making this assmption wold imply that is the only random variable in the system casing a singlar covariance matrix (see the comment by Maddala and Jeong (199) regarding the impact of this on the reslts in Nelson and Startz (1990a)). 14

16 4. Simlation Reslts The specific approach sed for the simlation is discssed in Appendix. In particlar, or reslts are based on 144 independent simlations where we vary the sample size (n = 100, 00, 500, and 1,000), endogeneity in the regressor x (corr(x,) = 0.1, 0.4, and 0.7), endogeneity in the instrmental variable z (corr(z,) = 0.0, 0.1, 0.4, and 0.7), and the strength of the instrmental variable (corr(x,z) = 0.1, 0.4, and 0.7). These parameter vales were selected to roghly correspond to the statistics reported in typical acconting stdies with regard to sample size and the explanatory power for the firststage regressions. For each simlation we generate 1000 independent samples with the indicated poplation moments. The percentiles for the distribtion of the OLS and SLS parameter estimates when the tre β = 0 are presented in Table 5A, 5B, and 5C for corr(x,) = 0.1, 0.4, and 0.7, respectively. The shaded cells in these tables represent cases where SLS is preferred to OLS based on median bias, that is, in those cells the median of SLS estimates is closer to zero than the median of OLS estimates. 11 However, even in those cases it is not clear that SLS shold be the preferred estimator, since the dispersion is (mch) higher than the dispersion of OLS estimates, especially when the instrments are weak. In sitations where there is low endogeneity in the x variable (Table 5A), OLS estimates generally dominate SLS estimates in terms of median bias. As might be expected, asymptotically, SLS is preferred only when the instrment is perfect (i.e., corr(z,) = 0.0). However, even in these cases, the variability of the SLS estimates 11 This is consistent with the asymptotic reslts, that is, in those cells the ineqality in eqation (6) holds for the poplation moments. 15

17 tends to be mch larger than that for OLS, althogh this otcome is somewhat mitigated for larger sample sizes and strong instrmental variables. The reslts in Table 5A indicate that the se of SLS in sitation where there is minimal endogeneity will prodce estimates for β with ndesirable properties relative to OLS. This is especially tre when the selected instrments are weak and semi-endogenos. Ths, the indiscriminate application of SLS is not desirable in acconting research. As endogeneity in the x variable increases (Tables 5B and 5C), the median bias for OLS estimates is generally smaller than the median bias for SLS estimates when the instrments are weak. Even in sitations where the instrments have a strong association with the x variable, OLS estimates are preferred to SLS estimates when the z exhibits moderate to strong endogeneity. The SLS estimates exhibit lower median bias when the instrments are strong, z has modest endogeneity, and the endogeneity in x is large. However, it is also important to highlight that the variability of the SLS estimates is generally larger than the variability of OLS estimates (althogh the variability for SLS estimates is a decreasing fnction of sample size). The reslts in Table 5 have important implication for the econometric methods applied in acconting research. For example, consider the case where the researcher sspects moderate to high endogeneity in the x variable (e.g., corr(x,) = 0.4 or 0.7) and the explanatory power for the first-stage regression is similar to that reported in or srvey of acconting applications (i.e., corr(z,x) = 0.4 or 16 percent explanatory power). OLS estimates will be preferred to the SLS estimates nless the researcher can provide a compelling argment that the endogeneity in the instrmental variable z is very small (i.e., corr(z,) = 0.1 or one percent explanatory power). For most empirical acconting 16

18 research, it is difficlt to believe that the variables selected as instrments exhibit a level of endogeneity that is this low. Althogh the simlation reslts are specific to the model in eqation (7) and selected parameter vales, the analysis presented in Table 5 raises serios concern abot the desirability of IV estimation methods to correct for endogenos regressors Testing the appropriateness of instrments Similar to the asymptotic case, it is important to test the appropriateness of the instrmental variables model by performing a Hasman test and a test of over-identifying restrictions. In addition to the isses discssed in Section 3.3, there is also the problem that the small sample distribtion of these test statistics can differ qite dramatically from the assmed asymptotic distribtion (e.g., Hahn and Hasman, 003). Absent knowledge abot the exact finite sample properties of these tests, simlation can be sed to estimate the approximate size of the test statistics for a specific stdy. For example, Abernethy, Bowens and Van Lent (004) perform sch a simlation analysis and report that the empirical distribtion of the Hasman test in their sample is different from the asymptotic distribtion. Finally, it is sefl to spplement the formal tests with a sensitivity analysis where the researcher identifies the impact of sing different instrments on the IV estimates. 1 Althogh not reported in tables, we also examined the rejection percentages based on the t-statistics for the OLS and IV estimators. If β = 0, we wold desire that the rejection freqency is very low at conventional levels of statistical significance. Unsrprisingly, we find that the rejection freqency goes to 1 as the bias and the sample size increase. Since the IV standard errors are (mch) larger than the OLS standard errors, we are less likely to reject the nll sing IV. In this case that is a good thing, becase the nll happens to be the tre vale. However, in more general settings the tre vale of β is likely not zero. The higher dispersion of the IV estimator then inhibits the detection a non-zero parameter. This is consistent with the findings of some of the papers in table 1, where the athors find little difference between the IV estimate and the OLS estimate, bt the IV estimate is not significant whereas the OLS estimate is. In those cases the Hasman test wold not reject and one shold adopt the OLS estimate. Unfortnately, this is not always done. 17

19 5. Applications of OLS and IV Estimation Methods In order to illstrate the theoretical isses discssed above, we apply OLS and IV estimation methods in two contemporary acconting research settings. The first example examines the association between the cost of capital and corporate disclosre and the second example assesses the association between chief exective officer (CEO) compensation and the power of corporate insiders. In each analysis, we se instrmental variables that are commonly selected by prior researchers. We also apply the overidentifying restriction and Hasman (1978) tests for assessing the extent of endogeneity in the regressors Cost of Capital and Corporate Disclosre The effect of volntary disclosre on cost of capital has received considerable interest from acconting researchers, bt this topic remains controversial from both a theoretical and econometric perspective. We rely on several papers that have attempted to address the potential endogeneity of the disclosre choice. Lez and Verrecchia (000), Hail (00), and Brown and Hillegeist (003) find the expected statistically positive relation between their disclosre proxy and selected measres for the cost of capital after sing IV estimation. In contrast, Cohen (003) finds that the relation between reporting qality and cost of capital is no longer significant after taking into accont the endogeneity of the choice of reporting qality. The prpose of or analysis is not to resolve this isse bt rather to demonstrate common pitfalls in the application of instrmental variables estimation. 13 A variety of alternative tests have been sggested (e.g., a graphical diagnostic developed by De Lna and Johansson, 004). We plan to evalate these tests in ftre revisions. 18

20 Or measre of cost of capital is based on several measres of implied cost of capital. Rather than arbitrarily picking one metric, we se the average of the for implied cost of capital measre investigated by Gay, Kothari and Sh (003). These are the Gebhardt, Lee and Swaminathan measre, the Clas and Thomas measre, the Gordon growth model, and the Gode and Mohanran measre. Or disclosre measre is based on the prior work of Francis, Olsson, LaFond and Schipper (005). Specifically, a regression model is estimated by indstry/year of crrent accrals on sales growth, PPE, and past, crrent and ftre cash flow and save the residal for each firm-year. For each firm we then calclate the standard deviation of these residals over the past five years and call this measre AccralsQality (AQ). Higher scores on this measre mean poorer information qality. For instrmental variables, we simply pick a set of variables previosly sed for this prpose in other papers. Unfortnately, in most acconting stdies on volntary disclosre, the instrmental variables are selected in an arbitrary manner and this violates one of the most important principles in instrmental variables estimation (i.e., the carefl selection and jstification of instrments). Or instrmental variables are the natral log of the nmber of owners, one-year sales growth, capital intensity (PPE/assets), litigation risk (dmmy, eqal to 1 if the firm is in a high litigation indstry), operating margin, length of operating cycle (in days), and the presence of a Big-six aditor. In addition we se the following control variables: log of market vale of eqity, book-to-market eqity, nmber of analysts, leverage (total debt/market vale of eqity), and retrn on assets. The cost of capital measres are compted as of Jly 1 for each year from We restrict the sample to firms with December fiscal year end and all of or 19

21 independent variables are measred sing data of six months before the comptation of the cost of capital measre. In order to redce the inflence of otliers, we trncate all variables at the 1 st and 99 th percentile. The regressions are estimated in a pooled timeseries cross section setting after sbtracting the year specific mean from each variable in order to remove temporal effects. For ease of interpretation, we also divide the disclosre measre (AccralsQality) by its year specific standard deviation. This allows an easy interpretation of the coefficient of Accrals Qality, since the coefficient is the effect on cost of capital (in percentage points) of a one standard deviation change in AQ. The descriptive statistics are displayed in Table 6. In or analysis, we first estimate an OLS regression of cost of capital on AQ and the control variables. The reslts are displayed in Table 7. We find that a one standard deviation increase in AQ is associated with a statistically significant 0.44% increase in cost of capital. This is consistent with prior literatre that finds that better disclosre is associated with lower cost of capital (recall that higher AQ means lower qality disclosre). We then estimate a SLS regression sing the previosly discssed instrments. Consistent with econometric literatre (e.g., Chapter 5 from Wooldridge, 00) we inclde all exogenos variables in the first stage, not jst the assmed instrments. The R of this first-stage model is 7%. However, this overstates the tre explanatory power of the instrments as the control variables also contribte to this R. After removing the contribtion of the control variables, the partial R is approximately 16%. At this point, it is necessary to qalitatively evalate whether the instrmental variables estimation is likely to improve over the OLS estimate. From eqation (6), we 0

22 know that the sqared correlation of the instrments with the strctral error term has to be less than 16% (the partial R ) of the comparable sqared correlation between AQ and the strctral error for SLS to provide better estimates than OLS. Ths the selected instrments mst be sbstantially more exogenos than AQ for the SLS estimates to dominate the OLS estimates. Withot a rigoros jstification of the instrments there is no way to evalate this critical ineqality. Althogh somewhat sbjective, we believe that variables sch as operating margin, capital intensity, and growth rates are endogenos since they are cased by the same variables as the determinants of volntary disclosre. Ths, or intition is that OLS estimates are likely to dominate SLS estimates. In the second stage reslts, we find that the effect of AQ on cost of capital has increased relative to the OLS estimate. A one standard deviation increase in AQ now leads to a statistically significant 1.1% increase in cost of capital. At this point, it wold be sefl to determine whether it is conceivable that increases in disclosre can case this level of change in the cost of capital. Unrealistically high or low estimates wold case sspicion abot the qality of the instrments. In this case the estimate does not seem nreasonable. After presenting the SLS reslts, acconting researchers typically then perform a Hasman test (done by the majority of the papers we investigated). In this case the Hasman test strongly rejects the exogeneity of AQ, and this leads the researcher to conclde that the SLS estimate is preferable to the OLS estimate. However, the validity of this conclsion critically depends on the appropriateness of the instrments (i.e., that the instrmental variables are actally exogenos). 1

23 If mltiple instrments are available for the endogenos variable, as we have in this case, it is necessary to compte a test of over-identifying restrictions. If this test rejects the appropriateness of the instrments, it is not appropriate to proceed to the Hasman test (e.g., Godfrey and Htton, 1994). Eqivalently, it is possible to examine the sensitivity of second stage estimates for the instrmented AQ to the se of different (sets of) instrments. The intition of this test is that if the instrments are valid, then each shold give s the tre coefficient. Ths the estimates prodced by different instrments shold be similar. This test is implemented in the last colmns in Table 7 (i.e., nconstrained second stage). The model is an OLS regression of cost of capital on all the independent variables. However, for ease of comparison, we replaced each independent variable by the prodct of its original vale and its first stage coefficient. This facilitates the interpretation, becase the coefficient on each instrment is eqal to the second stage coefficient on AQ in a model where that instrment is the only instrment and the rest of the so-called instrments are treated as control variables (i.e. they are inclded explicitly in the second stage). If the instrments are valid, the reslting coefficients for the instrments shold be close to each other and therefore close to the SLS estimate (which is the weighted average of these estimates). The reslts in Table 7 illstrate that the coefficients on the assmed exogenos variables vary considerably. For example, if nmber of owners wold have been sed as the sole instrment, we wold have fond a negative and statistically insignificant coefficient on AQ. However, if operating margin or operating cycle wold have been sed as instrments, we wold have obtained implasibly high estimates of 10%-0% increases in cost of capital for a one standard deviation increase in AQ. Not srprisingly,

24 we find that a formal test rejects the eqivalence of these coefficients (χ = 148.1, p < ). Note that the fact that sales growth and capital intensity have mid range coefficients that are reasonably close to the SLS coefficient does not make them any better or worse than the other instrments. The sensitivity analysis and the formal tests indicate that or set of instrments is dbios, and nlikely to prodce better estimates than OLS. A cationary note on the se of the over-identifying restrictions test is that research in econometrics (e.g., Hahn and Hasman, 003) has shown that the size of the test in finite samples can differ significantly from the asymptotic size, leading to false rejections. In addition, it is necessary to consider the power of this test. In very large samples the test may be so powerfl that economically small deviations lead to rejections (even thogh SLS is mch better than OLS), whereas in small samples the test may lack power to reject even economically important deviations (even thogh SLS might well be worse than OLS). Ths, it is important to spplement the formal test with some sensitivity analysis sch as or nconstrained second stage to assess the similarities (or dissimilarities) in the coefficient estimates obtained when sing different sets of variables as instrments. A final note of cation is that neither this test nor the sensitivity analysis will pick p problems when all instrments exhibit similar problems. That is, if the instrments all lead to a bias in the same direction with comparable magnitde, this test will not reject (even in large samples), bt the coefficient on the primary endogenos variable of interest can be severely biased. Therefore the test shold be sed as a check on instrments jstified by economic theory and shold not be sed to select instrments, nless one has a proper instrment to benchmark the coefficients against. Overall, the test 3

25 on over-identifying restrictions is a sefl tool in evalating the desirability of SLS verss OLS methods. However, it cannot replace the carefl selection and jstification of the variables sed as instrments. 5. CEO Compensation and Insider Power Or second stdy investigates the impact of insider power on CEO compensation. Prior literatre sch as Core, Holthasen, and Larcker (1999) has docmented an association between certain board characteristics and excess CEO compensation levels. A potential concern is that there are many variables that affect both insider power and CEO pay levels and which have not been properly controlled in the regression. Recent research into the determinants of corporate governance has feled this concern (e.g., Gillan, Hartzell, and Starks, 003; Doidge, Karolyi and Stlz, 004b; Black, Jang, and Kim, 004). 14 Or sample is primarily developed from data provided by Eqilar, Inc. Similar to Larcker, Richardson and Tna (004), we obtain CEO compensation and board of director data for,106 companies with fiscal year ends between Jne, 00, and May, 003. The dependent variable for or analysis is total compensation for the CEO, measred as the natral logarithm of total remneration (i.e., the sm of base salary, annal bons and the expected vales for stock options, performance plans, and restricted stock). The primary independent variable is insider power. We measre this constrct by adding the standardized scores of board size, percentage of otside directors older than 14 Another potential concern is that governance and compensation levels are jointly determined which wold make single eqation methods inappropriate. The simltaneos eqation or nonrecrsive strctre is sbstantially more difficlt to solve becase instrments are reqired for both the CEO compensation and insider power endogenos variables in order to identify the system. In or example, we will model only the impact of insider power on CEO compensation. 4

26 70, percent of otside directors on at least 4 boards, and whether an insider is chairman, from this we sbtract the standardized score of the average percentage share ownership by otside directors. In order to increase interpretability of this measre, we standardize this measre to have a zero mean and a standard deviation of one. Based on prior research and a variety of instittional conjectres (e.g., Levitt, 004), we expect that corporate insiders will have more power where the chairman is also the CEO, the board is large, and the otside member are old and/or bsy with other commitments. Holding aside isses related to endogeneity, we expect insider power to have a positive association with the level of CEO compensation. Or control variables are the standard economic variables that have been sed in many prior stdies of exective compensation: firm size (natral logarithm of market vale), book-to-market ratio, retrn on assets, stock retrn, standard deviation of retrn on assets, and standard deviation of stock retrns. We expect that the level of compensation is increasing in firm size, extent of investment opportnities, stock market and operating performance, and risk or volatility in the performance measres. As instrments for insider power, we se the following (somewhat predetermined) CEO characteristics: natral logarithm of CEO age, natral logarithm of CEO tenre and whether the CEO is the fonder of the company (measred as an indicator variable). We expect that insider power will have a positive association with variables related to either longevity of the CEO or the role of firm fonder. Since different stock exchanges impose different governance reqirements on the firm, we incorporate an indicator variable for whether the firm is listed on NYSE/AMEX. Since instittional shareholders can limit insider power, we inclde the percentage holdings by blockholders and the percentage 5

27 holdings by activist instittions (e.g., pblic pension fnds) as instrments (these data were obtained from Spectrm filings). Finally, the external aditor also has the potential to mitigate insider power and we measre this factor sing an indicator variable for Big For aditor and the ratio of non-adit fees to adit fees. We expect that insider power will be a decreasing fnction of the se of higher qality aditors and less fees from nonadit services. The final sample with complete data consists of 1,483 firms. The descriptive statistics for these variables are displayed in Table 8. The OLS regression of CEO compensation on insider power and control variables is presented in Table 9. Most of the economic control variables have the expected signs, althogh firm size is the primary determinant for CEO compensation. We also find that a one standard deviation increase in insider power is associated with an 8% increase in total CEO compensation. Whether this estimate is interpretable depends on the extent of endogeneity in the insider power variable. The first stage regression for IV estimation regresses insider power on all exogenos variables, and firm size also appears to be the most important determinant of insider power. The statistical significance and signs for the coefficient estimates are mixed (e.g., CEO age and tenre have the expected positive relation, bt fonder has an nexpected negative relation with insider power). Althogh the adjsted R for the entire model is 9%, the partial R for determining the strength of the instrment is only abot 5%. Given this reslt, the hrdle for instrmental variable estimation to be preferred is very high. That is, we know from eqation (6) that the sqared correlation between or set of instrments and the strctral error has to be less than 5% of the corresponding sqared correlation between insider power and the strctral error. Unless the researcher is highly confident in the choice of instrments 6

28 (which is clearly open to qestion here), OLS is likely to be the preferred estimator. It is important to note that the OLS estimator may have considerable bias, bt it is still likely to exhibit better statistical properties than the SLS estimator. The second stage reslts indicate that there is a negative relation between insider power and CEO compensation (i.e., a one standard deviation increase in insider power is now associated with a 4% decrease in CEO remneration). However, this estimate is not statistically significant at conventional levels. Not srprisingly, we find that the Hasman test cannot reject the eqivalence of the OLS and the SLS estimates (χ = 1.5, p= 0.99). The appropriateness of the instrment can be assessed by examining the nconstrained second stage. As in the disclosre stdy, we regress the dependent variable (total compensation) on all the exogenos variables, where the independent variables have been transformed into the prodct of the original vale and the first stage coefficient. The coefficient on each of the instrments can be interpreted as the SLS estimate if that variable wold have been the niqe exclded variable, and all other variables treated as control variables. Similar to the reslts from the disclosre stdy, the coefficients exhibit sbstantial variability in both size and sign. As wold be expected, the formal test of over-identifying restrictions rejects the nll of no relation between the instrments and the error term (χ = 41.1, p < ). Ths, the reslts from the compensation analysis provide even stronger evidence than the disclosre analysis that an IV estimator does not represent an improvement over the OLS estimator in typical acconting research stdies. 7