Causality and potential outcomes Average causal effects

Transcription

1 treatment effects The term treatment effect refers to the causal effect of a bnary (0 1) varable on an outcome varable of scentfc or polcy nterest. Economcs examples nclude the effects of government programmes and polces, such as those that subsdze tranng for dsadvantaged workers, and the effects of ndvdual choces lke college attendance. The prncpal econometrc problem n the estmaton of treatment effects s selecton bas, whch arses from the fact that treated ndvduals dffer from the non-treated for reasons other than treatment status per se. Treatment effects can be estmated usng socal experments, regresson models, matchng estmators, and nstrumental varables. A treatment effect s the average causal effect of a bnary (0 1) varable on an outcome varable of scentfc or polcy nterest. The term treatment effect orgnates n a medcal lterature concerned wth the causal effects of bnary, yes-or-no treatments, such as an expermental drug or a new surgcal procedure. But the term s now used much more generally. The causal effect of a subsdzed tranng programme s probably the mostly wdely analysed treatment effect n economcs (see, for example, Ashenfelter, 1978, for one of the frst examples, or Heckman and Robb, 1985 for an early survey). Gven a data-set descrbng the labour market crcumstances of tranees and a non-tranee comparson group, we can compare the earnngs of those who dd partcpate n the programme and those who dd not. Any emprcal study of treatment effects would typcally start wth such smple comparsons. We mght also use regresson methods or matchng to control for demographc or background characterstcs. In practce, smple comparsons or even regresson-adjusted comparsons may provde msleadng estmates of causal effects. For example, partcpants n subsdzed tranng programmes are often observed to earn less than ostensbly comparable controls, even after adjustng for observed dfferences (see, for example, Ashenfelter and Card, 1985). Ths may reflect some sort of omtted varables bas, that s, a bas arsng from unobserved and uncontrolled dfferences n earnngs potental between the two groups beng compared. In 1

2 general, omtted varables bas (also known as selecton bas) s the most serous econometrc concern that arses n the estmaton of treatment effects. The lnk between omtted varables bas, causalty, and treatment effects can be seen most clearly usng the potental-outcomes framework. Causalty and potental outcomes The noton of a causal effect can be made more precse usng a conceptual framework that postulates a set of potental outcomes that could be observed n alternatve states of the world. Orgnally ntroduced by statstcans n the 1920s as a way to dscuss treatment effects n randomzed experments, the potental outcomes framework has become the conceptual workhouse for non-expermental as well as expermental studes n many felds (see Holland, 1986, for a survey and Rubn, 1974; 1977, for nfluental early contrbutons). Potental outcomes models are essentally the same as the econometrc swtchng regressons model (Quandt, 1958), though the latter s usually ted to a lnear regresson framework. Heckman (1976; 1979) developed smple two-step estmators for ths model. Average causal effects Except n the realm of scence fcton, where parallel unverses are sometmes magned to be observable, t s mpossble to measure causal effects at the ndvdual level. Researchers therefore focus on average causal effects. To make the dea of an average causal effect concrete, suppose agan that we are nterested n the effects of a tranng programme on the post-tranng earnngs of tranees. Let Y 1 denote the potental earnngs of ndvdual f he were to receve tranng and let Y 0 denote the potental earnngs of ndvdual f not. Denote tranng status by a dummy varable, D. For each ndvdual, we observe Y = Y 0 + D (Y 1 Y 0 ), that s, we observe Y 1 for tranees and Y 0 for everyone else. Let E[ ] denote the mathematcal expectaton operator,.e., the populaton average of a random varable. For contnuous random varables, E[Y ] = yf(y)dy, where f(y) s the densty of Y. By the law of large numbers, sample averages converge to populaton averages so we can thnk of E[ ] as gvng the sample average n very large samples. The two most wdely studed 2

3 average causal effects n the treatment effects context are the average treatment effect (ATE), E[Y 1 Y 0 ], and the average treatment effect on the treated (ATET), E[Y 1 Y 0 D = 1]. Note that the ATET can be rewrtten [ E Y - Y D =1]= [ E Y D =1]- [ E Y D =1] Ths expresson hghlghts the counter-factual nature of a causal effect. The frst term s the average earnngs n the populaton of tranees, a potentally observable quantty. The second term s the average earnngs of tranees had they not been traned. Ths cannot be observed, though we may have a control group or econometrc modellng strategy that provdes a consstent estmate. Selecton bas and socal experments As noted above, smply comparng those who are and are not treated may provde a msleadng estmate of a treatment effect. Snce the omtted varables problem s unrelated to samplng varance or statstcal nference, but rather concerned wth populaton quanttes, t too can be effcently descrbed by usng mathematcal expectaton notaton to denote populaton averages. The contrast n average outcomes by observed treatment status s EY [ D = 1] EY [ D = 0] = EY [ D = 1] EY [ D = 0] 0 1 = EY [ Y0 D = 1] + { EY [ 0 D = 1] EY [ 0 D = 0] } 1 Thus, the nave contrast can be wrtten as the sum of two components, ATET, plus selecton bas due to the fact that the average earnngs of non-tranees, E[Y 0 D = 0], need not be a good standn for the earnngs of tranees had they not been traned, E[Y 0 D = 1]. The problem of selecton bas motvates the use of random assgnment to estmate treatment effects n socal experments. Random assgnment ensures that the potental earnngs of tranees had they not been traned an unobservable quantty are well-represented by the randomly selected control group. Formally, when D s randomly assgned, E[Y D = 1] E[Y D = 0] = E[Y 1 Y 0 D = 1] = E[Y 1 Y 0 ]. Replacng E[Y D = 1] and E[Y D = 0] wth the correspondng sample analog provdes a consstent estmate of ATE. 3

4 Regresson and matchng Although t s ncreasngly common for randomzed trals to be used to estmate treatment effects, most economc research stll uses observatonal data. In the absence of an experment, researchers rely on a varety of statstcal control strateges and/or natural experments to reduce omtted varables bas. The most commonly used statstcal technques n ths context are regresson, matchng, and nstrumental varables. Regresson estmates of causal effects can be motvated most easly by postulatng a constant-effects model, where Y 1 Y 0 = α (a constant). The constant-effects assumpton s not strctly necessary for regresson to estmate an average causal effect, but t smplfes thngs to postpone a dscusson of ths pont. More mportantly, the only source of omtted-varables bas s assumed to come from a vector of observed covarates, X, that may be correlated wth D. The key assumpton that facltates causal nference (sometmes called an dentfyng assumpton), s that EY [ X, D] = X β, (1) 0 where β s a vector of regresson coeffcents. Ths assumpton has two parts. Frst, Y 0 (and hence Y 1, gven the constant-effects assumpton) s mean-ndependent of D condtonal on X. Second, the condtonal mean functon for Y 0 gven X s lnear. Gven eq. (1), t s straghtforward to show that E{ Y( D E[ D X])}/ E{ D( D E[ D X])} = α. (2) Ths s the coeffcent on D from the populaton regresson of Y on D and X (that s, the regresson coeffcent n an nfnte sample). Agan, the law of large numbers ensures that sample regresson coeffcents estmate ths populaton regresson coeffcent consstently. Matchng s smlar to regresson n that t s motvated by the assumpton that the only source of omtted varables or selecton bas s the set of observed covarates, X. Unlke regresson, however, treatment effects are constructed by matchng ndvduals wth the same covarates nstead of through a lnear model for the effect of covarates. The key dentfyng assumpton s also weaker, n that the effect of covarates on Y 0 need not be lnear. Instead of (1), the condtonal ndependence assumpton becomes 4

5 Ths mples EY [ X, D] = EY [ X],for j= 0,1. (3) j j X, D EY [ Y D = 1] = E{ EY [ X, D = 1] [ Y = 1] D = 1} X, D = EEY1 X D = Y0 = D { [, 1] [ 0] = 1} (4a) and, lkewse, EY [ 1 Y ] = E{ EY [ 1 X, D = 1] [ Y X, D = 0]} 0 0 (4b) In other words, we can construct ATET or ATE by averagng X-specfc treatment-control contrasts, and then reweghtng these X-specfc contrasts usng the dstrbuton of X for the treated (for ATET) or usng the margnal dstrbuton of X (for ATE). Snce these expressons nvolve observable quanttes, t s straghtforward to construct consstent estmators from ther sample analogs. The condtonal ndependence assumpton that motvates the use of regresson and matchng s most plausble when researchers have extensve knowledge of the process determnng treatment status. An example n ths sprt s the Angrst (1998) study of the effect of voluntary mltary servce on the cvlan earnngs of solders after dscharge, dscussed further below. Regresson and matchng detals In practce, regresson estmates can be understood as a type of weghted matchng estmator. If, for example, E[D X ] s a lnear functon of X (as t mght be f the covarates are all dscrete), then t s possble to show that eq. (2) s equvalent to a matchng estmator that weghts cell-bycell treatment-control contrasts by the condtonal varance of treatment n each cell (Angrst, 1998). Ths equvalence hghlghts the fact that the most mportant econometrc ssue n a study that reles on condtonal ndependence assumptons to dentfy causal effects s the valdty of these condtonal ndependence assumptons, not whether regresson or matchng s used to mplement them. A computatonal dffculty that sometmes arses n matchng models s how to fnd good matches for each possble value of the covarates when the covarates take on many values. For 5

6 example, begnnng wth Ashenfelter (1978), many studes of the effect of tranng programmes have shown that tranees typcally experence a perod of declnng earnngs before they go nto tranng. Because lagged earnngs s both contnuous and multdmensonal (snce more than one perod s earnngs seem to matter), t may be hard to match tranees and controls wth exactly the same pattern of lagged earnngs. A possble soluton n ths case s to match tranees and controls on the propensty score, the condtonal probablty of treatment gven covarates. Propenstyscore matchng reles on the fact that, f condtonng on X elmnates selecton bas, then so does condtonng on P[D = 1 X ], as frst noted by Rosenbaum and Rubn (1983). Use of the propensty score reduces the dmensonalty of the matchng problem snce the propensty score s a scalar, though n practce t must stll be estmated. See Deheja and Wahba (1999) for an llustraton. Regresson and matchng example Between 1989 and 1992, the sze of the mltary declned sharply because of ncreasng enlstment standards. Polcymakers would lke to know whether the people many of them black men who would have served under the old rules but were unable to enlst under the new rules were hurt by the lost opportunty for servce. The Angrst (1998) study was meant to answer ths queston. The condtonal ndependence assumptons seems plausble n ths context because solders are selected on the bass of a few well-documented crtera related to age, schoolng, and test scores and because the control group also appled to enter the mltary. Nave comparsons clearly overestmate the beneft of mltary servce. Ths can be seen n Table 1, whch reports dfferences-n-means, matchng, and regresson estmates of the effect voluntary mltary servce on the Socal Securty-taxable earnngs of men who appled to jon the mltary between 1979 and The matchng estmates were constructed from the sample analog of (4a), that s, from covarate-value-specfc dfferences n earnngs, weghted to form a sngle estmate usng the dstrbuton of covarates among veterans. The covarates n ths case were the age, schoolng, and test-score varables used to select solders from the pool of applcants. Although whte veterans earn $1,233 more than non-veterans, ths dfference becomes negatve once the adjustment for dfferences n covarates s made. Smlarly, whle 6

7 non-whte veterans earn $2,449 more than non-veterans, controllng for covarates reduces ths to $840. Table 1 Matchng and regresson estmates of the effects of voluntary mltary servce n the Unted States Average earnngs n Dfferences n means Matchng estmates Regresson estmates Regresson mnus Race matchng (1) (2) (3) (4) (5) Whtes 14,537 1,233.4 (60.3) (70.5) 88.8 (62.5) (28.5) Non-whtes 11,664 2,449.1 (47.4) (62.7) 1,074.4 (50.7) (32.5) Notes: Fgures are n nomnal US dollars. The table shows estmates of the effect of voluntary mltary servce on the Socal Securty-taxable earnngs of men who appled to enter the armed forces durng The matchng and regresson estmates control for applcants year of brth, educaton at the tme of applcaton, and Armed Forces Qualfcaton Test (AFQT) score. There are 128,968 whtes and 175,262 non-whtes n the sample. Standard errors are reported n parentheses. Source: Adapted from Angrst (1998, Tables II and V). Table 1 also shows regresson estmates of the effect of voluntary servce, wth the same covarates used n the matchng estmates controlled for. These are estmates of α r n the equaton d α D + e Y = β +, X X X r where β X s a regresson-effect for X = X and α r s the regresson parameter. Ths corresponds to a saturated model for dscrete X. The regresson estmates are larger than (and sgnfcantly dfferent from) the matchng estmates. But the regresson and matchng estmates are not very dfferent economcally, both pontng to a small earnngs loss for Whte veterans and a modest gan for non-whtes. Instrumental varables estmates of treatment effects The condtonal ndependence assumpton requred for regresson or matchng to dentfy a treatment effect s often mplausble. Many of the necessary control varables are typcally unmeasured or smply unknown. Instrumental varables (IV) methods solve the problem of 7

8 mssng or unknown controls, much as a randomzed tral also obvates the need for regresson or matchng. To see how ths s possble, begn agan wth a constant effects model wthout covarates, so Y 1 Y 0 = α. Also, let Y 0 = β + ε, where β E[Y 0 ]. The potental outcomes model can now be wrtten Y αd = β + + ε, (5) where α s the treatment effect of nterest. Because D s lkely to be correlated wth ε, regresson estmates of eq. (5) do not estmate α consstently. Now suppose that n addton to Y and D there s a thrd varable, Z, that s correlated wth D, but unrelated to Y for any other reason. In a constant-effects world, ths s equvalent to sayng Y 0 and Z are ndependent. It therefore follows that ε E[ Z ], (6) a condtonal ndependence restrcton on the relaton between Z and Y 0, nstead of between D and Y 0 as requred for regresson or matchng strateges. The varable Z s sad to be an IV or just an nstrument for the causal effect of D on Y. Suppose that Z s also a 0 1 varable. Takng expectatons of (5) wth Z swtched off and on, we mmedately obtan a smple formula for the treatment effect of nterest: { EY [ Z = 1] EY [ Z = 0]}/{ ED [ Z = 1] ED [ Z = 0]} = α. (7) The sample analog of ths equaton s sometmes called the Wald estmator, snce t frst appear n a paper by Wald (1940) on errors-n-varables problems. There are other more complcated IV estmators nvolvng contnuous, mult-valued, or multple nstruments. For example, wth a mult-valued nstrument, we mght use the sample analog of Cov(Z, Y )/ Cov(D, Y ). Ths smplfes to the Wald estmator when Z s 0 1. The Wald estmator captures the man dea behnd most IV estmaton strateges snce more complcated estmators can usually be wrtten as a lnear combnaton of Wald estmators (Angrst, 1991). IV example To see how IV works n practce, t helps to use an example, n ths case the effect of Vetnamera mltary servce on the earnngs of veterans later n lfe (Angrst, 1990). In the 1960s and 8

9 early 1970s, young men were at rsk of beng drafted for mltary servce. Concerns about farness also led to the nsttuton of a draft lottery n 1970 that was used to determne prorty for conscrpton n cohorts of 19-year-olds. A natural nstrumental varable for the Vetnam veteran treatment effect s draft-elgblty status, snce ths was determned by a lottery over brthdays. In partcular, n each year from 1970 to 1972, random sequence numbers (RSNs) were randomly assgned to each brth date n cohorts of 19-year-olds. Men wth lottery numbers below an elgblty celng were elgble for the draft, whle men wth numbers above the celng could not be drafted. In practce, many draft-elgble men were stll exempted from servce for health or other reasons, whle many men who were draft-exempt nevertheless volunteered for servce. So veteran status was not completely determned by randomzed draft elgblty; elgblty and veteran status are merely correlated. Table 2 IV estmates of the effects of mltary servce on US whte men born 1950 Earnngs year Earnngs Veteran status Wald estmate of Mean Elgblty Mean Elgblty veteran effect effect effect (1) (2) (3) (4) (5) , (210.5) (.040) 2,741 (1,324) , (39.7) 1,470 (250) , (34.5) Notes: Fgures are n nomnal US dollars. There are about 13,500 observatons wth earnngs n each cohort. Standard errors are shown n parentheses. Sources: Adapted from Angrst (1990, Tables 2 and 3), and unpublshed author tabulatons. Earnngs data are from Socal Securty admnstratve records. Veteran status data are from the Survey of Program Partcpaton. For whte men who were at rsk of beng drafted n the draft lotteres, draftelgblty s clearly assocated wth lower earnngs n years after the lottery. Ths can be seen n Table 2, whch reports the effect of randomzed draft-elgblty status on average Socal Securty-taxable earnngs n column (3). Column (1) shows average annual earnngs for purposes of comparson. For men born n 1950, there are sgnfcant negatve effects of elgblty status on earnngs n 1970, when these men were beng drafted, and n 1981, ten years 9

10 later. In contrast, there s no evdence of an assocaton between elgblty status and earnngs n 1969, the year the lottery drawng for men born n 1950 was held but before anyone born n 1950 was actually drafted. Because elgblty status was randomly assgned, the clam that the estmates n column (3) represent the effect of draft elgblty on earnngs seems uncontroversal. The only nformaton requred to go from draft-elgblty effects to veteran-status effects s the denomnator of the Wald estmator, whch s the effect of draft-elgblty on the probablty of servng n the mltary. Ths nformaton s reported n column (4) of Table 2, whch shows that draft-elgble men were 0.16 more lkely to have served n the Vetnam era. For earnngs n 1981, long after most Vetnam-era servcemen were dscharged from the mltary, the Wald estmates of the effect of mltary servce amount to about 15 percent of earnngs. Effects were even larger n 1970, when affected solders were stll n the army. IV wth heterogeneous treatment effects The constant-effects assumpton s clearly unrealstc. We d lke to allow for the fact that some men may have benefted from mltary servce whle others were undoubtedly hurt by t. In general, however, IV methods fal to capture ether ATE or ATET n a model wth heterogeneous treatment effects. Intutvely, ths s because only a subset of the populaton s affected by any partcular nstrumental varable. In the draft lottery example, many men wth hgh lottery numbers volunteered for servce anyway (ndeed, most Vetnam veterans were volunteers), whle many draft-elgble men nevertheless avoded servce. The draft lottery nstrument s not nformatve about the effects of mltary servce on men who were unaffected by ther draft-elgblty status. On the other hand, there s a sub-populaton who served solely because they were draft-elgble, but would not have served otherwse. Angrst, Imbens and Rubn (1996) call the populaton of men whose treatment status can be manpulated by an nstrumental varable the set of complers. Ths term comes from an analogy to a medcal tral wth mperfect complance. The set of complers are those who take ther medcne, that s, they serve n the mltary when draft-elgble but they do not serve otherwse. Under reasonably general assumptons, IV methods can be reled on to capture the effect of 10

11 treatment on complers. The average effect for ths group s called a local average treatment effect (LATE), and was frst dscussed by Imbens and Angrst (1994). A formal descrpton of LATE requres one more bt of notaton. Defne potental treatment assgnments D 0 and D 1 to be ndvdual s treatment status when Z equals 0 or 1. One of D 0 or D 1 s counterfactual snce observed treatment status s D = D0 + Z( D1 D0). The key dentfyng assumptons n ths setup are (a) condtonal ndependence, that s, that the jont dstrbuton of {Y 1, Y 0, D 1, D 0 } s ndependent of Z ; and (b) monotoncty, whch requres that ether D 1 D 0 for all or vce versa. Monotoncty requres that, whle the nstrument mght have no effect on some ndvduals, all of those who are affected should be affected n the same way (for example, draft elgblty can only make mltary servce more lkely, not less). Assume wthout loss of generalty that monotoncty holds wth D 1 D 0. Gven these two assumptons, the Wald estmator consstently estmates LATE, wrtten formally as E[Y 1 Y 0 D 1 > D 0 ]. In the draft lottery example, ths s the effect of mltary servce on those veterans who served because they were draft elgble but would not have served otherwse. In general, LATE complers are a subset of the treated. An mportant specal case where LATE = ATET s when D 0 equals zero for everyone. Ths happens n a socal experment wth mperfect complance n the treated group and no one treated n the control group. IV Detals Typcally, covarates play a role n IV models, ether because the IV dentfcaton assumptons are more plausble condtonal on covarates or because of statstcal effcency gans. Lnear IV models wth covarates can be estmated most easly by two-stage least squares (2SLS), whch can also be used to estmate models wth mult-valued, contnuous, or multple nstruments. See Angrst and Imbens (1995) or Angrst and Krueger (2001) for detals and addtonal references. Joshua D. Angrst See also nstrumental varables wth weak nstruments; matchng estmators; regresson- 11

12 dscontnuty analyss; Rubn causal model; selecton bas and self-selecton; two-stage least squares and the k-class estmator Bblography Angrst, J Lfetme earnngs and the Vetnam era draft lottery: evdence from socal securty admnstratve records. Amercan Economc Revew 80, Angrst, J Grouped-data estmaton and testng n smple labor-supply models. Journal of Econometrcs 47, Angrst, J Estmatng the labor market mpact of voluntary mltary servce usng Socal Securty data on mltary applcants. Econometrca 66, Angrst, J. and Imbens, G Two-stage least squares estmates of average causal effects n models wth varable treatment ntensty. Journal of the Amercan Statstcal Assocaton 90, Angrst, J., Imbens, G. and Rubn, D Identfcaton of causal effects usng nstrumental varables. Journal of the Amercan Statstcal Assocaton 91, Angrst, J. and Krueger, A Instrumental varables and the search for dentfcaton: from supply and demand to natural experments. Journal of Economc Perspectves 15(4), Ashenfelter, O Estmatng the effect of tranng programs on earnngs. Revew of Economcs and Statstcs 6, Ashenfelter, O. and Card, D Usng the longtudnal structure of earnngs to estmate the effect of tranng programs. Revew of Economcs and Statstcs 67, Deheja, R. and Wahba, S Causal effects n nonexpermental studes: reevaluatng the evaluaton of tranng programs. Journal of the Amercan Statstcal Assocaton 94, Heckman, J The common structure of statstcal models of truncaton, sample selecton, and lmted dependent varables and a smple estmator for such models. Annals of Economc and Socal Measurement 5, Heckman, J Sample selecton bas as a specfcaton error. Econometrca 47,

13 Heckman, James J. and Robb, R Alternatve methods for evaluatng the mpact of nterventons. In J. Heckman & B. Snger (Eds.), Longtudnal Analyss of Labor Market Data (pp ). New York: Cambrdge Unversty Press. Holland, P Statstcs and causal nference. Journal of the Amercan Statstcal Assocaton 81, Imbens, G. and Angrst, J Identfcaton and estmaton of local average treatment effects. Econometrca 62, Quandt, R The estmaton of the parameters of a lnear regresson system obeyng two separate regmes. Journal of the Amercan Statstcal Assocaton 53, Rosenbaum, P. and Rubn, D The central role of the propensty score n observatonal studes for causal effects. Bometrka 70, Rubn, D Estmatng causal effects of treatments n randomzed and non-randomzed studes. Journal of Educatonal Psychology 66, Rubn, D Assgnment to a treatment group on the bass of a covarate. Journal of Educatonal Statstcs 2, Wald, A The fttng of straght lnes f both varables are subject to error. Annals of Mathematcal Statstcs 11,