The use of negative controls to detect confounding and other sources of error in experimental and observational science

Transcription

1 The use of negatve controls to detect confoundng and other sources of error n expermental and observatonal scence Marc Lpstch Erc Tchetgen Tchetgen Ted Cohen eppendx. Use of negatve control outcomes to detect uncontrolled confoundng We brefly provde an analytcal bass for the use of an deal U-comparable negatve control outcome to detect the presence of uncontrolled confoundng. For the sake of smplcty we lmt our treatment to mean regresson wth dentty lnk functon where confoundng adjustment s done by condtonng on potental confounders n the model and we provde detals only for the case of dchotomous ( U. Nonetheless wth addtonal techncal arguments our results can be generalzed to other types of regresson models and to settngs wth contnuous ( U. The fact that has no effect on the mean of N wthn levels of L and U mples that E(N UL=E(N UL. Thus margnalzng over U gven and L we obtan the followng expresson EN ( L = EE [ ( N U L L ] = EEN [ ( UL L ] In the case of dchotomous and U ths result mmedately leads to the followng smple and ntutve expresson for the confounded condtonal effect of on Nfor each level of L Δ ( L = E( N = L E( N = 0 L = [ EN ( U= L EN ( U= 0 L] [ EU ( = L EU ( = 0 L] Ths formula confrms that n the absence of uncontrolled confoundng Δ ( L = 0 for all values of L. Ths s because under the assumpton of no uncontrolled confoundng any random varable U not n (LY ether s ndependent of N gven L makng EN ( U= L EN ( U= 0 L = 0 or s ndependent of gven L makng EU ( = L EU ( = 0 L = 0. However n the presence of uncontrolled confoundng we can generally expect that Δ( L 0for some value of L. The above observaton leads to the followng general strategy to detect the presence of uncontrolled confoundng by testng the null hypothess: Δ ( L = 0 for all values of L. In practce as L may contan 2 or more contnuous components or multple categorcal varables ths s acheved by fttng a regresson model EN ( Lθ wth unknown parameter θ for the mean of N gven covarates

2 and L; and subsequently testng for a sgnfcant effect of n ths model. For nstance n the smple case where one correctly specfes the model EN ( L θ = θ0 + θ + θ2' L+ θ3' Lwhere L = ( L ' L 2 '' and θ = [ θ 0 θ θ 2 ' θ 3 ']' a test for the presence of uncontrolled confoundng thus nvolves a standard statstcal test of the null hypothess of zero values for the coeffcents for the regresson of N on.e. a test that θ and θ 2 are both zero. We note that none of the above arguments requre any addtonal restrcton on the relatonshp between U and L whch can generally be dependent. Furthermore the usual modelng caveats equally apply to ths stuaton n partcular the valdty of the proposed test s relant on the analyst s ablty to approprately account for measured confounders L; a msspecfed model EN ( Lθ wll often lead to ncorrect conclusons about the presence of uncontrolled confoundng. eppendx2.use of Negatve control exposures to detect uncontrolled confoundng. In the case of an deal U-comparable negatve control exposure knowledge that B does not affect the mean of Y wthn levels of L and U mples that EY ( BUL = EY ( UL. Thus upon margnalzng over U gven B and L we obtan EY ( BL = EEY [ ( UL BL ]. When B and U are bnary ths expresson provdes a smple and ntutve formula for the confounded condtonal effect of B on Y: δ ( L = EY ( B= L EY ( B= 0 L = [ EY ( U = L EY ( U = 0 L] [ EU ( B= L EU ( B= 0 L ] Thus smlar to our result for negatve control outcomes n the presence of uncontrolled confoundng we expect that δ( L 0 for some jont level of and L. test for uncontrolled confoundng s thus a test of the null hypothess δ ( L = 0 for all values of and L. Ths s easly operatonalzed by regressng Y on L and B n a parametrc mean regresson model and subsequently performng a standard statstcal test for the effect of B on Y gven and L n the model. eppendx3. remark on the use of negatve control varables to nfer the drecton and magntude of confoundng bas. In the event that the null hypothess of no uncontrolled confoundng s correctly rejected (.e. Δ( L 0for some value of L or δ( L 0for some value of and L an nterestng queston arses: can the magntude and/or the drecton of confoundng bas of an estmate of the condtonal effect of on Y be nferred from an estmate of Δ( L the regresson of the negatve control outcome on the exposure of nterest orδ( L the regresson of the outcome of nterest on the negatve control exposure? s we now argue ths s generally mpossble unless addtonal assumptons are made beyond those stated so far.

3 To llustrate ths pont consder the case where a researcher dentfes an deal U-comparable negatve control outcome N; furthermore suppose that the data were generated by a process descrbed by the model EY ( LU ; = + + ' L+ U for the mean of Y gven L and U where = ( 0 2' 3 so that encodes the unknown condtonal causal effect of on Y. Now because U s not observed suppose the analyst fts the reduced model that specfes 3 = 0 to data Y and L. Then by standard lnear regresson theory the least-squares estmate of the effect of n ths reduced model can be shown to converge n probablty (wth ncreasng sample sze to the quantty 3π + where π denotes the asymptotc value of the least-squares estmate of the effect of n a (possbly ncorrect lnear regresson of U on and L. Therefore the product quantfes the asymptotc bas of the ordnary leastsquares estmate of the effect of on Y due to uncontrolled confoundng. Next suppose that n order to detect the presence of confoundng bas the analyst uses the regresson model gven n secton of ths ppendx E( N L θ = θ 0 + θ + θ 3 ' L of the negatve control outcome on and L where for smplcty we set θ 2 =0. When ths latter model s correctas prevously establshed the confounded effect of on N wthn levels of L equals Δ ( L = [ E( N U = L E( N U = 0 L] [ E( U = L E( U = 0 L] = θ. Therefore we see that although the formulae for the confoundng bas θ and π 3 n estmatng the effect of on N and the effect of on Y respectvelyare both a product of two terms there s a pror no reason why these two expressons should be equal unless one s wllng to make addtonal assumptons. For nstance the equalty θ = 3 π would ndeed hold f the followng two assumpons were met: the mean functon EU ( L s n the lnear span of and L and 2 wthn levels of and L the effect of U on N s equal to the effect of U on Y.e. EN ( U= L EN ( U= 0 L = EN ( U = L EN ( U = 0 L = EY ( U = L EY ( U = 0 L Ths s because the frst assumpton mples that π = EU ( = L EU ( = 0 L when the mean of U depends lnearly on and L whereas the second assumpton states that the magntude of the assocaton between U and N s equal to that between U and Y.e 3 = EN ( U= L EN ( U= 0 L. By combnng and 2 we obtan θ = 3 π. Clearly both of these assumptons are emprcally untestable as they

4 drectly nvolve the uncontrolled confounder and would generally be unrealstc unless they are based on very frm scentfc understandng. In summary the regresson results for a negatve control outcome or a negatve control exposure cannot be used n a smple way to correct the equvalent regressons for the outcome of nterest or the exposure of nterest respectvely even n the smple lnear settng consdered here. Nonetheless upon rejectng the hypothess of no uncontrolled confoundng the results of negatve controls can be addtonally nformatve n some smple settngs wthout the need for potentally unrealstc assumptons. To llustrate suppose that there s no L so that a sample of ndependent and dentcally dstrbuted data on YU s generatedwhere and U can ether be dscrete or contnuous. However data on U are not observed and the followng models hold: EY ( U ; = U; EU ( ; ρ = ρ0 + ρu where ρ ( ρ ρ ; = 0 EB ( U; η = η0 + ηu where η = ( η 0 η and ρ η and 3 are bounded away from zero so that U confounds both the null assocaton between and Y and the assocaton of B wth Y. In ths smple settng when U s unobserved the asymptotc bas of the least-squares estmate of the margnal effect of on Y can be re-expressed as ρ 2 3 var( U 3 = ( λ where ρ va r( ρ E[var( U] E[var( U] λ = =. When and U are contnuous var( var[ EU ( ] + E[var( U ] the fracton λ can be nterpreted as the proporton of the varance of due to the effect ofu on the mean of; f λ = 0 there s no uncontrolled confoundng and thus there s no correspondng bas whereas as the assocaton between U and the mean of explans an ncreasng proporton of the varance of λ tends to and the worst-case confoundng bas corresponds to the lmtng value 3. s λ drectly nvolves the varance of U there s generally no hope of ρ estmatng t from the observed data. However as we show next approprate use of the negatve control exposure B permts dentfcaton of the sgn of the bas of wth no further assumpton requred. Specfcally consder the statstc ( Y YB T =. We wll show below that T converges n probablty to + 3 ( B ρ so that ˆ 3 3 T converges to λ. Now because 0 < λ < the sgn of λ ρ ρ agrees wth the sgn of 3 ( λ ; thus n large samples we can expect the sgn of ρ

5 T ˆ to generally agree wth the sgn of the bas of the least-squares estmator of the effect of on Y. Formally ths s true wth probablty tendng to one. Furthermore t s nterestng to note that for an assumed value of λ one obtans the followng bas corrected least-squares estmator % T ( λ = T. In λ prncple ths opens up the possblty of performnga smple senstvty analyss by varyng λ to assess the potental mpact of the magntude of uncontrolled confoundng on the corrected least-squares estmator % ( λ. The above dscusson dd not allow for the presence of observed confounders L. Nonetheless the results generalze qute naturally f confoundng adjustment s done by stratfcaton. However no smlar results are currently avalable for stuatons where confoundng adjustment s performed ether by condtonng n the model or by nverse-probablty weghtng. We fnally provde techncal arguments supportng our results. Frst to derve the large sample bas of we note that ( Y = = + + Y ( U U ( òy òy 3 where ( ( ( ò = Y ( + + U and the overbar denotes the sample average. The y 0 3 thrd term n the above expresson for converges to zero n probablty wth ncreasng sample sze whereas the second term whch consttutes the bas of least-squares ( U ρ U ( U = ρ U U ( U + U ( U ( ( ( converges n probablty to ρ 2 3 var( U 3 = ( λ where we use the fact that ρ va r( ρ ( U U( ρu E([ U E( U][ E( U] converges n probablty to = 0. ( E([ E( ] It remans to show that T converges to + 3. Ths follows from ρ ( Y ( = = + ( + YB U UB òy òy B T 3 ( B ( B ( where the thrd term converges to zero n probablty and the second term can be wrtten:

6 ( U UB ( U U( η U + ò b 3 = 3 ( B [ ρ( U U + ( ò3 ò3( η U + ò b ( U U U = η 3 + [ ηρ( U U U + η( ò ò U + ρ( U U ò + ( ò ò ò ] 3 3 b 3 3 b ( U U òb 3. [ ηρ ( U UU + η( ò3 ò3 U + ρ( U U ò b + ( ò3 ò3 ò b The second term n ths last equaton s easly shown to converge to zero n probablty by an applcaton of the law of large numbers whereas the frst term converges n probablty to the desred quantty.