AN EMPIRICAL INVESTIGATION OF META-ANALYSIS USING RANDOMIZED CONTROLLED CLINICAL TRIALS IN A PARTICULAR CENTRE

Transcription

1 Appled Quanttatve Methods n Medcne A EMPIRICAL IVESTIGATIO OF META-AALYSIS USIG RADOMIZED COTROLLED CLIICAL TRIALS I A PARTICULAR CETRE C. POURAJA Department of Statstcs, Tuberculoss Research centre (ICMR), Chenna, Inda E-mal: cponnuraja@gmal.com Abstract: Meta-analyss s the combnaton of results from varous ndependent studes. In a meta-analyss, combnng survval data from dfferent clncal trals, an mportant ssue s the possble heterogenety between trals. Such nter-tral varaton can not only be explaned by heterogenety of treatment effects across trals but also by heterogenety of ther baselne rsk. In addton, one mght examne the relatonshp between magntude of the treatment effect and the underlyng rsk of the patents n the dfferent trals. However, the need for medcal research and clncal practce to be based on the totalty of relevant and sound evdence has been ncreasngly recognzed. In ths paper, we revew the advances of meta-analyss usng clncal trals TB data. Ths paper examnes sxteen reportng results of randomzed clncal trals conducted n a partcular centre at consecutve perods. Every study pools that the results from the relevant trals n order to evaluate the effcacy of a certan treatment between cases and control. There s a need for emprcal effort comparng random effects model wth the fxed effects model n the calculaton of a pooled relatve rsk n the meta-analyss n systematc revews of randomzed controlled clncal trals. We revew heterogenety and random effects analyses and assessng bas wthn and across studes. We compare the two approaches wth regards to statstcal sgnfcance, summary relatve rsk, and confdence ntervals. Key words: fxed effects model; random effects model; heterogenety of treatment effects. Introducton Meta-analyss provdes an objectve way of combnng nformaton from ndependent studes lookng at the same clncal questons and has been appled most often to treatment effects n randomzed clncal trals. We understand meta-analyss as beng the use of statstcal technques to combne the results of studes addressng the same queston nto a summary measure. Standard meta-analyss methods for provdng an overall estmate of the treatment effects rely on certan assumpton (Whtehead and Whtehead, 99). Metaanalyss s the term gven to retrospectve nvestgatons n whch data from all known studes of a partcular clncal ssue are assembled and evaluated collectvely and quanttatvely. It dffers n mportant ways from tradtonal narratve revews, n that there s a commtment to scentfc prncples n assemblng and analyzng the data, va protocol-drven lbrary searches and data abstracton, n addton to the formalsm of statstcal analyss. There s a 66

2 Appled Quanttatve Methods n Medcne need for more emprcal work on methodology, propertes and lmtatons of underlyng statstcal methodology (Engels, et al, 000). Heterogenety, by whch we mean varaton among the results of ndvdual trals beyond that expected from chance alone, s an mportant ssue n meta-analyss. Heterogenety may ndcate that trals evaluated dfferent nterventons or dfferent populatons. It s clear that when there are substantal dfferences among tral results, and n the face of heterogenety, a sngle estmate may be msleadng and should be avoded and exploraton of heterogenety s also a crtcal mportant component of meta-analyss of randomzed trals (Thompson and Pocock, 99; Thompson 994; Lau et al. 995). Most of the arguments presented aganst random effects model could be consdered as explanatons of the lmtatons of usng covarates to explan the heterogenety n tral results. There s lmted emprcal experence comparng results from random effects and fxed effects models, partcularly when the results are heterogeneous (Thompson and Pocock, 99). The random effects model ncorporates the heterogenety of treatment effects across studes n the analyss of the overall treatment effcacy (DerSmonan and Lard, 986). We present an emprcal nvestgaton from meta-analyss of randomzed clncal trals ncluded n systematc revews as well as reports conducted n the area of tuberculoss nfected patents; we compare the two approaches wth regards to statstcal sgnfcance, summary relatve rsk, and confdence ntervals. The results of any ndvdual tral must be absorbed and debated by the scentfc communty before wholesale recommendatons regardng treatment practce are observed. Randomzed trals and metaanalyses have dstnct but complementary goals. Meta-analyss can be used productvely n plannng new clncal trals, and n supplyng updated nformaton to study montors n the course of a tral. Ths process of debate necessarly nvolves the weghng of evdence from dfferent sources, and meta-analyss can and does play an mportant role n ths process (Begg, 996).. Defnton of models The two models have been used here, they are fxed effects model (FEM) and random effects model (REM). Fxed effects model assumes that there s a common effect and a random component, whch means samplng error, s responsble for dfference among tral results, that s, t assumes heterogenety of nterventon effects. Ths approach provdes nferences only about the set of trals under revew, gvng weght to each tral based on the wthn study samplng varance. The ndvdual study sample sze and the number of events are the leadng factors n the weght assgned to each tral n the pooled estmate of the relatve rsk. The FEM formulatons are nverse varance method, Mantel-Haenszel method and Peto s method. However, the Peto s modfed estmate can gve based answers n a few crcumstances, such as when there s severe mbalance n treatment allocaton wthn ndvdual studes or n the presence of very large treatment effects. The REM provdes nference based on the assumpton that the observed trals are a sample from a hypothetcal populaton of trals. Also to account for the varaton among trals results a random term s added to compute the weghts n the REM, representng among trals varaton, as often estmated from a functon of the ch-squared test for heterogenety. Ths term adds a common varance component to the weght of each tral n the meta-analyss, whch tends equalze the weghts assgned to small and large trals (Vllar, et al., 00). The dsproportonate overall nfluence of small trals s more evdent when there s heterogenety 67

3 Appled Quanttatve Methods n Medcne of tral results because the among trals varance becomes larger and domnates the wthntral random effects. When heterogenety s present, t may be napproprate to combne the separate tral estmates nto a sngle number, partcularly usng fxed effects methods that assume a common treatment effect. Random effects methods, whch provde an attractve approach to summarzng heterogeneous results, model heterogenety as varaton of ndvdual tral treatment effects around a populaton average effect. The key dstncton between these two types of models concerns the belef regardng behavor of tral effects as tral sample szes get very large. If one beleves that the ndvdual tral effects would converge to a common value for all trals, a fxed effects model s approprate, whereas f one beleves that ndvdual trals would stll demonstrate separate effects, then a random effects model s preferable (Thompson and Pocock, 99). The random effects model antcpates better than the fxed effects model by Fless (993) and also the atonal Research Councl (99) make known the benefts of usng random effects model. 3. A meta-analyss of sxteen randomzed clncal trals For the present analyss we examne sxteen clncal trals at same centre each reportng results from several ndependent trals over a perod between 956 and 995. All the sxteen trals have been categorzed nto two groups based on ther duraton segment. Each revew pools the results from the relevant trals n order to evaluate the effcacy of a certan treatment for a specfed condton. These revews lack of consstent assessment of homogenety of treatment effect before poolng. We dscuss both fxed effects and random effects approach to combnng evdence from a seres of experments comparng two treatments. Ths approach ncorporates the heterogenety of effects n the analyss of the overall treatment effcacy. The model can be extended to nclude relevant covarates whch would reduce the heterogenety and allow for more specfc therapeutc recommendatons. Most often to explore heterogenety s stratfcaton. Studes are categorzed accordng to the characterstcs of the study or the characterstcs of the subjects n the study and a summary estmate of effect s estmated n each of the categores (Pettt, 00). 4. Statstcal methods Results of the outcome were abstracted and are expressed as summary relatve rsk and 95 per cent confdence nterval (CI) for both random and fxed effects models. The summary relatve rsk for the FEM was calculated usng the Mantel-Haenszel method whle the DerSmonan and Lard method was used for the REM. Mantel-Haenszel Method Ths s for calculatng a summary estmate of effect across strata. Snce studes are dentfed for a meta-analyss as strata, the Mantel-Haenszel method s an approprate for analyzng data for a meta-analyss based on fxed effect. It s used when the measure of effect s a rato mesure. Klenbaum, Kupper, and Morgenstern (98) gve formulas that would allow n Mantel-Haenszel to be appled. otatons for applcatons of Mantel- Haenszel 68

4 Appled Quanttatve Methods n Medcne Treated Control Total Recurrent a b g on Recurrent c d h Total e f n Summary odds rato OR mh Sum( W OR ) SumW ( a d ) OR ( b c ) W varance Varance n b c ) ( 95% confdence nterval ln ORmh e ±.96 varanveormh where varance OR mh s calculated as Robns, Greenland, and Breslow(986). The Varance mh where F SumF SumG SumH + + ( SumR) SumR SumS ( SumS) a + d a d n G H [ a + d ( b + c )] + [ b c ( a + d )] n [ b c ( b + c )] a d R n b c S n n Formula for calculate a statstc for a test of homogenety of effects; [ ( lnor lnor ) ] Q SumW, where, Q s referred to the ch-square dstrbuton wth one degree of freedom. mh DerSmonan & Lard Method The DerSmonan and Lard (986) method s based on the random-effects model. Formulas for applyng the DerSmonan-Lard method summarzng studes n the case where effects are measured as odds ratos are gven by Fless and Gross (99) lnor dl ( lnor ) Sum( W ) Sum W ; where OR dl s the DerSmonan-Lard summary estmate of the odds rato, W s the DerSmonan-Lard weghtng factor for the th study, and OR s the odds rato from the th study 69

5 Appled Quanttatve Methods n Medcne W D + W where W s gven n MH and [ Q ( S ) ] ( SumW ) SumW D ; and D 0 f Q < S [ Sum( W ) ] where S s the number of studes and ( lnor lnor ) mh ; Q SumW from ths formula 95% CI ln OR dl e ±.96 varanve ; where Varance SumW The fxed effects let Y denote the generc measure of the effect of an expermental nterventon; let W denotes the recprocal of the varance of effect sze. Under the assumpton of the fxed set of studes, an estmator of the assumed common underlyng effect sze s Y W Y W and the standard error of the estmator s SE (Y ) W let ψ s the populaton effect sze for an approxmate 00(-α)% confdence nterval, then Y zα W ψ Y + z α W Under the assumpton of random effects, the studes are random samples from a largest populaton, the mean populaton szeψ, about whch the study-specfc effect sze vary. An approxmaton 00(- α) % confdence nterval for ψ s where W Y z α W ψ ( D + W ) Y + z α W Y W W Y D denotes the study varaton n effect sze and ths s calculated as D 0 f Q D [ Q ( ) ]/ U f Q > as Der Smonan and Lard, (986) 70

6 Appled Quanttatve Methods n Medcne U ( ) W SW W where W and S W are the mean and varance of the W S The nconsstency of studes are beng measured based on the classcal measure of heterogenety s Cochran s Q, whch s calculated as the weghted sum of squared dfferences between ndvdual study effects and the pooled effect across studes, wth the weghts beng those used n the poolng method. Q s dstrbuted as a ch-square statstc wth k- (number of studes mnus one) degrees of freedom. Q has low power as a comprehensve test of heterogenety (Gavaghan et al. 000) n partcular when the number of trals s small n meta-analyss. If the number of studes are large where Q has more power as a test of heterogenety (Hggns et al. 003). Q s ncluded n each meta-analyss functon because t forms part of the DerSmonan-Lard random effects poolng method (DerSmonan and Lard 986). An addtonal test, due to (Breslow and Day 980), s provded wth the odds rato meta-analyss. We transformed the summary relatve rsks and the correspondng upper and lower lmts of the 95 per cent CI for the two models to the natural logarthmc scale. I-squared statstc descrbes the percentage of varaton across studes that are due to heterogenety rather than chance (Hggns and Thompson, 00; Hggns et al., 003). (Q - df) I 00% Q We calculated the mean and standard devaton and range of the summary relatve rsk obtaned usng the two methods. To assess the dfferences between the summary relatve rsks and between the wdths of the Confdence Intervals obtaned usng the two methods we calculated the mean of the pared dfferences. To nvestgate the average relatve rsk as a functon of the dfference we plotted the dfferences between the logs of the relatve rsks (log RR random-log RR fxed) aganst the mean of these two values. Graphs were plotted separately by heterogenety status. The statstcal evaluaton of bas was conducted usng the Begg and the Egger test. The complete analyss performed by STATA verson 9., the meta command uses nverse-varance weghng to calculate fxed and random effects summary estmates, and, optonally to produce a forest plot. The advantage n usng Meta command s that we requre varables contanng the effect estmate and ts correspondng standard error for each study. When one arm of a study contans no events- or, equally, all events - we have what s termed a zero cell n the x table. Zero cells create problems n the computaton of rato measures of treatment effect, and the standard error of ether dfference or rato measures. If no relapses any of the tral of any one group, the estmated odds rato s zero and the standard error cannot be estmated. A common way to deal wth ths problem s to add 0.5 to each cell of the x for the tral (Cox and Snell, 989). Because our ncluson crtera selected meta-analyses that had few trals wth arms wth zero events, ths correcton for zero cells had a mnmal mpact on conclusons. If there are no events n ether the nterventon or control arms of the tral, however, then any measure of effect summarzed as a rato s undefned, and unless the absolute rsk dfference scale s used nstead, the tral has to be dscarded from the meta-analyss. 7

7 Appled Quanttatve Methods n Medcne 5. Results The followng table gves data from 6 randomzed controlled clncal trals of tuberculoss patents conssts of both long term and short term treatments. The effects of treatment are beng compared based on fxed and random effects method usng metaanalyss. Table shows the trals conssts both expermental as well as control groups for treatng the patents.. Table. Summary trals data Study name Study year Treated Group Control Group Total Cured Relapse Total Cured Relapse STO STO STO5A STO5B STO STO STO STO STO STOA STO STO STO STO STO STO The table shows the magntude of the change n the pooled estmate gven by the random and fxed effects models to the trals between long-term treatment trals, short-term treatment trals and ther combnaton n the calculaton of the meta-analyss (exponental form) of tuberculoss care for nfected ndvduals. Table. The magntude of the change n the pooled estmate Pooled estmate n o. of Trals Moment -based Test of Heterogenety Trals the meta-analyss n metaanalyss estmate of REM FEM Q statstc P value studes Varance Long Term (9df) P< Short Term (5df) P> Combned (5df) P< The tests of the heterogenety are statstcally sgnfcant n long-term trals and combned trals of long-term and short-term. Even though t s arguably suffcent, not possble to examne the null hypothess that all studes are evaluatng almost same effect 7

8 Appled Quanttatve Methods n Medcne Study ID RR (95% CI) % Weght STO STO3 STO5A STO5B STO7 STO8 STO9 STO0 STO STOA STO STO3 STO4 STO6 STO7 STO8 Overall (I-squared 53.7%, p 0.006) 0.78 (0.6,.36) 0.50 (0.0,.) 0.6 (0.,.85).40 (0.54, 0.73) 0.49 (0.7, 0.90).9 (0.6,.9).6 (0.8, 9.43).79 (.7,.54).6 (0.40, 6.54) 0.5 (0.05, 5.6).03 (0.60,.77) 0.8 (0.57,.5) 0.47 (0.,.76) 0.5 (0.9, 0.89) 0.73 (0.40,.35).55 (0.69, 3.45) 0.94 (0.80,.) Fgure. Forest Plot In a forest plot the contrbuton of each study to the meta-analyss (ts weght) s represented by the area of a box whose centre represents the sze of the treatment effects estmated from that study. The summary treatment effect s shown by a mddle of a damond whose left and rght extremes represent the correspondng confdence nterval. Both the output and the graph show that there s a clear effect of treatments curng tuberculoss among patents. The meta-analyss domnated by the large study3, study0 and study6 trals whch contrbute around 50% of the weght n ths analyss. Moreover the I-squared s constructed the nconsstency s 53.7 % ( P0.006). Table 3. The summary of treatment effect Study Weghts 95% CI Est Fxed Random Lower Upper STO STO STO5A STO5B STO STO STO STO STO STOA STO STO STO STO STO STO

9 Appled Quanttatve Methods n Medcne ote that remarkable dfferences between the fxed and random effects summary estmates n the long term and the combnaton of long term and short term trals, whch arses because the studes are weghted much more equally n the random effects analyss. Ths shows the accountablty of heterogenety s comparable more n random effects than n the fxed effects method. Fgure based on random effects, shows the overall performances both fxed and random effects analyses. It s clear that the smaller studes such as study and study 3 are gven relatvely more weght n the random effects than wth the fxed effect model. STO STO3 STO5A STO5B STO7 STO8 STO9 STO0 STO STOA STO STO3 STO4 STO6 STO7 STO8 Combned. 0 Odds Rato logor Begg's funnel plot wth pseudo 95% confdence lmts s.e. of: logor Fgurea. Forest Plot Fgurea. Funnel Plot The method of assessng the effect of bas s usng funnel plot as gven below. In whch the effect szes form a study s plotted aganst the study s sample sze. There s evdence of bas usng the Eggar test based on weghted regresson method (p0.004) but not usng the Begg such as rank correlaton method. It s assumng that there s no heterogenety but here there are three studes are sgnfcantly dfferng due to heterogenety. 6. Dscussons The two approaches, the assumptons of a fxed and random set communcate the bass of estmaton for each approach for a general measure of effect sze. The fxed effect model s condtonal on the stronger assumpton that there s no true heterogenety between studes also they are all estmatng the same true effect and only dffer because of samplng varaton, where as the random effects method attempts to ncorporate statstcal heterogenety nto overall estmate of an average effect. The random effects model predcts better than the fxed effects model also to conclude that the modelng would be mproved by an ncrease n use of random effects model than the fxed effects model. There s revews focused meta-analyss usng revewed artcles or publshed materals over a perod or even n the several felds. But here we llustrated the meta-analyss appled for clncal trals n a partcular centre and embossed the less heterogenety among all the ndependent trals. References. Begg. C. B. The role of meta-analyss n montorng clncal trals, Statstcs n Medcne, 5, 996, pp Breslow,. E. and Day,. E. Combnaton of results from a seres of tables; control of confoundng, In Statstcal Methods n Cancer Research, Volume : The Analyss of 74

10 Appled Quanttatve Methods n Medcne Case-control Data. IARC Scentfc Publcatons o. 3, Internatonal Agency for Health Research on Cancer: Lyon, Cox, D. R. and Snell, E. J. Analyss of bnary data, Chapman and Hall, ew York, DerSmonan, R. and Lard,. Meta-Analyss n clncal trals, Controlled Clncal Trals, 7, 986, pp Egger, M., Zellwegar-Zahner, T., Schneder, M., Junker, C., Lengeler, C. and Antes, G. Language bas n randomzed controlled trals publshed n Englsh and German, Lancet, 350, 997, pp Engels, E., Schmd, C. H., Terrn,., Olkn, I. and Lau, J. Heterogenety and statstcal sgnfcance n meta-analyss: an emprcal study of 5 meta-analyses, Statstcs n Medcne,9, 000, pp Evertt, B. S. and Pckles, A. Statstcal Aspects of the desgn and analyss of clncal trals, Revsed edton, Imperal College Press, London, Fless, J. L. and Gross, A. J. Meta-Analyss n Epdemology, wth specal reference to studes of the assocaton to envronmental tobacco smoke and lung cancer: a crtque, Journal of Clncal Epdemology, 44, 99, pp Fless, J. L. The statstcal bass of meta-analyss, Statstcal Methods n Medcal Research,, 993, pp Gavaghan, D. J., Moore, A. R. and McQay, H. J. An evaluaton of homogenety tests n metaanalyss n pan usng smulatons of patent data, Pan, 85, 000, pp Hggns, J. P. T. and Thompson, S. G. Quantfyng heterogenety n a meta-analyss, Statstcs n Medcne,, 00, pp Klenbaum, D. G., Kupper, L. L. and Morgenstern, H. Epdemologc Research: Prncples and Quanttatve Methods, Belmont, Calf, Lfetme Learnng, Lau, J., Schmd, C. H. and Chalmers, T. C. Cumulatve meta-analyss of clncal trals buld evdence for exemplary meta-analyss, Journal of Clncal Epdemology, 48, 995, pp Mantel,. and Haenszel, W. Statstcal aspects of the analyss of data from retrospectve studes n dsease, Journal of the atonal Cancer Insttute,, 959, pp Pettt, D. B. Approaches to heterogenety n meta-analyss, Statstcs n Medcne, 0, 00, pp Robns, J., Greenland, S. and Breslow,. E. A general estmator for the varance of the Mantel- Haenszel odds rato, Amercan Journal of Epdemology, 4, 986, pp Thompson, S. G. and Pocock, S. Can meta-analyss be trusted?, Lancet, 338, 99, pp Thompson, S. G. Why sources of heterogenety n meta-analyss should be nvestgated, Brtsh Medcal Journal, 309, 994, pp Vllar, J., Mackey, M. E., Carrol, G. and Donnar, A. Meta-analyses n systematc revews of randomzed controlled trals n pernatal medcne: comparson of fxed and random effects models, Statstcs n Medcne, 0, 00, pp Whtehead, A. and Whtehead, J. A general approach to the meta-analyss of randomzed clncal trals, Statstcs n Medcne, 0, 99, pp Correspondng Author cponnuraja@gmal.com Ph: , Fax: Postal Address C.Ponnuraja Department of Statstcs Tuberculoss Research Centre (ICMR) Chetpet, Chenna 60003, Inda 75