Successfully Integrating an HIV Risk Reduction Intervention into a Community-Based Substance Abuse Treatment Program

Transcription

1 The American Journal of Drug and Alcohol Abuse, 33: , 2007 Copyright Q Informa Healthcare ISSN: print/ online DOI: / Successfully Integrating an HIV Risk Reduction Intervention into a Community-Based Substance Abuse Treatment Program Michael M. Copenhaver, Ph.D., 1 I.-Ching Lee, Ph.D., 1 and Arthur Margolin, Ph.D. 2 1 Center for Health, Intervention and Prevention (CHIP), Department of Psychology, University of Connecticut, Connecticut, USA 2 Yale University School of Medicine, Department of Psychitry, New Haven, Connecticut, USA Abstract: A number of evidence-based HIV risk reduction interventions are now widely available to clinicians. However, difficulties with intervention transportability have historically impeded the integration of evidence-based interventions into clinical settings. In this article, we discuss one successful approach to transportability through appropriate intervention selection, adaptation, placement, and evaluation. Our intervention adaptation process required substantially reducing the content and duration of an evidence-based intervention. Findings from 226 assessed participants suggested significant drug- and sex-related risk reduction outcomes, particularly among participants at greatest risk. Results point to a need to examine the relative costs=benefits of comprehensive, evidence-based interventions vs. shortened, adapted versions thereof. Keywords: Behavioral intervention, HIV risk reduction, injection drug use, intervention transportability, substance abuse treatment 1. INTRODUCTION Current estimates suggest that approximately 950,000 people in the United States have contracted HIV infection with deaths due to AIDS Address correspondence to Michael M. Copenhaver, Ph.D., Center for Health, Intervention, and Prevention (CHIP), Department of Psychology, University of Connecticut, 2006 Hillside Road, Storrs, CT michael.copenhaver@ uconn.edu 109

2 110 M. M. Copehhaver et al. totaling 450,000 individuals even though a wide variety of HIV prevention approaches exist (1, 2). Given the absence of an effective vaccine to prevent new HIV infections, behavioral interventions remain a primary tool for reducing the risk and transmission of HIV (3). Experts believe that wide deployment of more effective behavioral interventions would enhance our ability to reduce the rate of new HIV infections (4). A number of behavioral interventions have shown evidence of effectiveness within target populations and are now widely available in a box as complete intervention packages (see Thus, there is great potential to integrate effective HIV risk reduction interventions into substance abuse treatment programs where large numbers of high-risk individuals present for treatment. Regrettably, analogous efforts to disseminate evidence-based interventions for treating drug dependence have shown that broad availability of effective interventions does not dramatically change implementation by clinicians in real-world clinical settings (5). Thus, even though an intervention may have been developed with the intent for it to be ultimately implemented in clinical settings, this goal can be impeded by a host of factors that typically distinguish clinical settings from the research settings in which interventions are tested. Particularly salient are the differences between these 2 settings with respect to the quantity and quality of resources available to competently deliver, monitor, and evaluate interventions that are often lengthy and complex (6, 7). Such barriers can disrupt intervention deployment at one or more organizational levels, ranging from the intervention participants to the organizational leaders (8, 9). In the present study, we sought to design and deploy an optimized risk reduction intervention by taking into account both the organizational characteristics of the target community-based organization (CBO) and published empirical outcome data. Structured interviews (instrument available upon request) were conducted with treatment providers at several comparable CBOs to: (1) specify what factors would influence the extent to which a new intervention would be accepted by treatment providers and patients, (2) identify factors that would impact whether the intervention would be sustainable over time, and (3) identify possible barriers to the intervention being delivered as intended (9). Interview data were then used in conjunction with information gleaned from published empirical research to design the resulting intervention, the Community-Friendly Health Recovery Program (CHRP). The

3 Integrating an HIV Risk Reduction Intervention 111 primary objective of this article is to present outcomes associated with the CHRP intervention after it was fully integrated within the target CBO. 2. MATERIALS AND METHODS The CHRP intervention is a substantially abbreviated form of the comprehensive evidence-based Holistic Health Recovery Program (HHRP) (10, 11) that resulted from the process of adapting HHRP for optimal use within a drug treatment CBO. CHRP is a manual-guided behavioral intervention comprising four 50-minute group sessions (Active Health Care Participation; Reducing the Harm of Injection Drug Use; Harm Reduction with Latex; Negotiating Harm Reduction with Partners) that target sex- and drug-related HIV risks among Injection Drug Users (IDUs) in treatment. The sessions were provided by 2 trained bachelor s level facilitators who delivered intervention content using cognitive remediation strategies designed to accommodate the mild to moderate cognitive difficulties that are common among IDUs in treatment (12). The intervention was inserted in a modular manner into 37 normally scheduled methadone treatment group meetings at the APT Foundation, Inc. in New Haven, CT. The APT Foundation clinics serve approximately 1385 patients, 39% of whom are female and 65% of whom can be classified as economically disadvantaged (i.e., receiving some type of entitlement such as Title 19, SAGA, or SSI=DI). Overall, these clinics serve 28% HIVpositive, 15% status unknown, and 57% HIV-negative individuals Participants Participants were 412 patients enrolled in methadone maintenance treatment at the APT Foundation, Inc. in New Haven, Connecticut. Two hundred twenty-six of these patients completed a post-test measure and were included in the outcome analyses. Patients were not compensated for their participation and their status in the treatment program was not affected by whether they participated in the intervention. Thus, patients completed pre- and post-test measures at a lower rate than in trials of similar interventions in which participant incentives were offered (13, 14). However, no differences were found in terms of demographic characteristics or outcome measures between patients who completed post-test measures and those who did not. The characteristics of the full sample of participants were as follows: IDUs (73%), male (51%), never married (51%), unemployed (56%), English speaking (90%), and age range 19 to 58 years (mean age was 38.6

4 112 M. M. Copehhaver et al. years). Participants were 68% Caucasian, 18% Black, 13% Hispanic, and 1% American Indian. The study protocol was approved by the Investigational Review Board (IRB) at the University of Connecticut and by the research review board at the APT Foundation, Inc., and all participants provided signed informed consent prior to their participation Measures Several items from the Risk Assessment Battery (RAB) (15) were used to assess participants sex- and drug-related HIV risk behavior. The scale required participants to categorize and quantify their drug use and sexual activity during the prior week. Drug-use related items included how they used drugs, whether they used new syringes or cleaned syringes, and if so, how they cleaned them, and whether they shared syringes, rinse water, cooker, or cotton. Sexual activity related items assessed whether they had used a condom and, if not, whether it was due to abstinence from sexual activity. Consistent with the Information Motivation Behavior Skills (IMB) (16) model of health behavior change on which our intervention was based, participants also completed a brief assessment that covered the following domains: drug- and sex-related HIV-risk reduction knowledge (Information component), personal and social motivation to reduce HIV risk behavior (Motivation component), and self-efficacy about reducing HIV risk behavior (Behavioral skills component). Four items were used to assess knowledge about safer sex and safer injection drug use (e.g., If an HIV þ person only has sex with another HIV þ person, they don t need to use condom; If an HIV þ person shared needles with another HIV þ person, they don t need to clean the needles. ). Six items examined participants motivation to use condoms and clean needles, intentions to engage in safer sexual activity and safer injection drug use practices, and their perception of significant others beliefs about the importance of using condoms and using clean needles. Four items assessed participants behavioral skills in the form of their self-efficacy about abstaining from sexual activity and using a condom and about abstaining from injecting drugs and using clean needles. This brief assessment has been used in a randomized controlled trial of an evidence-based intervention (OPTIONS) (17) in order to expeditiously inform intervention clinicians about HIV-related information, motivation, and behavioral skills deficits among participants. Participants completed pre-test measures immediately before participating in their first intervention session; they completed post-test measures immediately after participating in their 4th (final) session. Pre- and post-test measures each took approximately 15 minutes to complete.

5 Integrating an HIV Risk Reduction Intervention 113 Based on the brief IMB-based assessment and 2 quantitative items from the RAB, confirmatory factor analyses were used to test the IMB model on drug- and sex-related risk reduction outcome measures. The model indices for sex-related outcomes were acceptable, chi-square (df ¼ 11) ¼ 7.02, p ¼.54. CFI ¼ 1, RMSEA ¼.00, as were those for drug-related outcomes, chi-square (df ¼ 11) ¼ 12.13, p ¼.44. CFI ¼ 1, RMSEA ¼ RESULTS Data were analyzed using a repeated measures analysis of variance with Intervention (pre- vs. post-intervention) as a within-subjects variable and with Gender (male vs. female) and HIV Risk Group (high, moderate, and low based on composite HIV risk scores at pre-test) as between-subjects variables on all quantitative variables pertaining to sex-related and drugrelated outcomes. A positive Intervention main effect was found when comparing scores across all quantitative variables at pre-test and posttest (Table 1). A Gender main effect was also found indicating that female participants tended to score higher at pre- and post-test across all HIV risk reduction outcomes. A nonsignificant Gender by Intervention interaction, however, indicated that the relative impact of the intervention was comparable regardless of gender. Analyses focused next on identifying significant outcomes within the drug- and sex-related HIV risk reduction domains, respectively. Separate repeated measures analyses of variance were conducted on drug-related HIV risk reduction outcomes (knowledge, personal motivation, social motivation, self-efficacy, and behavior) and sex-related HIV risk reduction outcomes (knowledge, personal motivation, social motivation, self-efficacy, and behavior). Since a significant Gender main effect was found when examining the overall intervention effect, we again included Gender as a between-subjects variable and Intervention as a withinsubjects variable. In addition, because we were mostly interested in the impact of the intervention on participants at relatively greater risk for contracting or transmitting HIV, we examined high, moderate, and low HIV risk groups based on participants composite drug-related and sex-related HIV risk scores at pre-test. Drug-Related Outcomes. A repeated measures analysis of variance of drug-related outcomes was conducted to examine the effects of the Intervention, Gender, and Drug-Related HIV Risk Group. We found significant main effects for Intervention and for Drug-Related HIV Risk Group. We also found a significant Intervention by Drug-Related HIV Risk Group interaction.

6 114 M. M. Copehhaver et al. Table 1. Main effects of intervention, gender, and HIV risk group on risk reduction 1 Intervention 2 Gender 2 Risk groups Pre-test Post-test Male Female High Moderate Low Sample Size Drug related outcomes 3,4 Knowledge (accuracy) 83.5% a 91% b 84.4% 90.2% 80.4% c 88.6% d 93.4% d Personal motivation c 4.71 d 4.90 e Social motivation c 4.61 d 4.92 e Self-efficacy a 4.48 b 3.94 c 4.38 d 4.66 e Reported clean c, d 5.97 c 5.84 d needle use 6 Sample Size Sex-related outcomes 7,8,9 Knowledge (accuracy) 72.2% a 81.2% b 76.1% 77.3% 73.0% 75.4% 81.6% Personal motivation a 3.93 b c 3.87 d 4.47 e Social motivation a 4.76 b c 4.78 d 4.93 d Self-efficacy a 3.74 b 2.58 c 3.47 d 4.33 e Reported condom use Notes: 1 Different letters indicate significant Dunn s post hoc test result at a 0 ¼ Overall Intervention effect: F (1, 223) ¼ 9.81, p <.01; Overall Gender effect: F (1, 223) ¼ 18.85, p < Intervention effect: F (5, 211) ¼ 3.36, p <.01; on Knowledge: F (1, 215) ¼ 14.07, p < Risk Group: F (10, 424) ¼ 10.76, p <.001; on Knowledge: F (2, 215) ¼ 6.31, p <.01; on Personal Motivation: F (2, 215) ¼ 28.18, p <.001; on Social Motivation: F (2, 215) ¼ 16.40, p <.001; on Self-efficacy: F (2, 215) ¼ 19.79, p <.001; and on Reported Injection Drug Use: F (2, 215) ¼ 3.57, p < Ona1to5scale: a higher score indicates greater motivation. 6 Ona1to6scale: a higher score indicates greater HIV risk reduction. 7 Intervention effect: F (5, 211) ¼ 10.60, p <.001; on Knowledge: F (1, 215) ¼ 20.72, p <.001; on Personal Motivation: F (1, 215) ¼ 7.66, p <.01; and on Social Motivation: F (1, 215) ¼ 23.07, p < Gender effect: F (5, 211) ¼ 4.10, p <.01; on Self-Efficacy: F (1, 215) ¼ 8.35, p < Risk Group: F (10, 424) ¼ 21.17, p <.001; on Personal Motivation: F (2, 215) ¼ 67.36, p <.001; on Social Motivation: F (2, 215) ¼ 28.36, p <.001; and on Self-Efficacy: F (2, 215) ¼ 71.12, p <.001. Participants showed significant improvements in terms of overall drug-related risk reduction outcomes (Table 1). Univariate analyses revealed this Intervention effect was most strongly driven by the significant gains in participants knowledge about drug-related HIV risk.

7 Integrating an HIV Risk Reduction Intervention 115 Analyses examining outcomes as a function of HIV risk level revealed that participants in the low drug-related HIV risk group maintained more favorable risk reduction scores from pre-test to post-test in terms of overall drug-related outcomes vs. those in both the moderate and high drugrelated risk groups. This pattern of between Drug-Related HIV Risk Group differences was consistent across all drug-related risk reduction outcomes including knowledge, personal motivation, social motivation, self-efficacy, and reported injection drug use behavior. Interestingly, the Drug-Related HIV Risk Group effect was superceded by an Intervention by Drug-Related HIV Risk Group interaction (Table 2). The interaction was significant with regard to improvement in Table 2. Drug- and sex-related risk reduction by HIV risk group High Moderate Low Pre-test Post-test Pre-test Post-test Pre-test Post-test Sample size Drug-related outcomes 1 Knowledge (accuracy) 75.3% 85.4% 84.3% 92.9% 91.7% 95.1% Personal motivation Social motivation Self-efficacy Reported clean needle use Sex-related outcomes 5 Knowledge (accuracy) 67.9% 82.9% 79.6% 83.6% 68.7% 77.3% Personal motivation Social motivation Self-efficacy Reported condom use Notes: 1 Intervention by Drug-related HIV Risk Group: F (10, 424) ¼ 9.05, p <.001; on Personal Motivation: F (2, 215) ¼ 19.05, p <.001; on Social Motivation: F (2, 215) ¼ 6.27, p <.01; on Self-efficacy: F (2, 215) ¼ 20.17, p <.001; and on Reported IDU behavior: F (2, 215) ¼ 4.63, p < Ona1to5scale: a higher score indicates greater motivation. 3 Because the interaction between intervention and group based on pre-test performance is significant, post hoc tests based on Dunn s tests were conducted comparing those groups across pre-test and post-test. 4 Ona1to6scale: a higher score indicates greater HIV risk reduction. 5 Intervention by Sex-related HIV Risk Group: F (10, 424) ¼ 9.40, p <.001; on Personal Motivation: F(2, 215) ¼ 25.36, p <.001; Social Motivation: F (2, 215) ¼ 12.10, p <.001; Self-Efficacy: F (2, 215) ¼ 12.51, p <.001. Indicates significant Dunn s post hoc test results.

8 116 M. M. Copehhaver et al. personal motivation, social motivation, self-efficacy, and reported injection drug use behavior. Post hoc analyses based on Dunn s tests (18) revealed that the interaction was primarily driven by greater relative improvement in the high drug-related HIV risk group vs. the other risk groups. Sex-Related Outcomes. A repeated measures analysis of variance of sexrelated HIV risk reduction outcomes (knowledge, personal motivation, social motivation, self-efficacy, and reported sexual behavior) was conducted to examine the effects of the Intervention, Gender, and Sex-Related HIV Risk Group. Significant main effects were found for Intervention and for Sex-Related HIV Risk Group; we also found a significant Intervention by Sex-Related HIV Risk Group interaction (Table 1). Significant overall sex-related risk reduction improvements were found. Univariate analyses indicated that improvements occurred when examining outcomes specific to participants sex-related HIV risk reduction knowledge, personal motivation, and social motivation. A Gender effect was also found, suggesting that male and female participants responded differently on the sex-related HIV risk reduction outcome measures. Compared to male participants, female participants reported relatively higher self-efficacy, indicating greater confidence in their ability to abstain from sex and=or to use condoms. Analyses examining outcomes as a function of sex-related HIV risk level revealed that those in the low sex-related HIV risk group maintained more favorable scores in terms of sex-related HIV risk reduction from pre-test to post-test compared with the moderate and the high risk groups. The same pattern of significant Sex-Related HIV Risk Group effects were found with regard to personal motivation, social motivation, and self-efficacy for safer sex. Of particular interest, the Sex-Related HIV Risk Group effect was superceded by an Intervention by Sex-Related HIV Risk Group interaction (Table 2). The interaction was significant with regard to personal motivation to engage in safer sex, social motivation to engage in safer sex, and self-efficacy for safer sex. Post hoc tests using Dunn s tests (18) indicated that members of the high Sex-Related HIV Risk Group improved relatively more than the other 2 risk groups. Although members of the low Sex-Related HIV Risk Group reported lower personal motivation and lower self-efficacy in the post-test than in the pre-test, this unexpected finding may have been due to a ceiling effect. Members of the low Sex-Related HIV Risk Group still demonstrated higher personal motivation and self-efficacy to engage in safer sex than the other 2 risk groups.

9 Integrating an HIV Risk Reduction Intervention DISCUSSION Analyses supported the conclusion that our intervention led to the enhancement of HIV risk reduction knowledge, motivation, behavioral skills, and behavior. The overall Intervention effect showed that, when all participants were considered, the intervention was sufficiently potent to produce significant positive changes by the end of the four-week intervention period. Moreover, when examining specific drug- and sex-related HIV risk reduction outcomes among participants at greatest risk for contracting or transmitting HIV, the intervention exerted the strongest positive influence. The intervention also helped participants at less risk maintain their relatively low HIV risk levels. These findings are consistent with reviews of the empirical research literature that highlight the importance of providing HIV risk reduction interventions that (1) focus equally on drug- and sex-related risks, (2) provide interpersonal skills training (e.g., negotiating safer syringe use), (3) include more than one intervention facilitator, and (4) are delivered concurrently with drug treatment (19, 20). The findings are also consistent with the large body of literature evaluating behavioral interventions that target difficult-to-change behaviors (e.g., substance use, smoking) in that, even when effective, outcomes tend to be relatively modest in terms of effect size (21). The present study contributes to the literature devoted to systematically addressing the array of issues that arise in the process of transporting research-based interventions to clinical settings (6 9, 22). We have attempted to extend this literature by including the interrelated processes of intervention adaptation, deployment, and evaluation that we found useful and that may inform similar efforts by others. The pervasive challenge faced by efforts of this kind involves adapting an evidence-based intervention in such a manner that it retains potency among the target population while accommodating itself to the resource constraints of CBOs. In the present study, the adaptation process required reducing the original evidence-based HHRP by approximately 85%; yet, we nevertheless found evidence suggesting significant reductions in HIV risk. Limitations of the current study should also be noted. These limitations arise principally from the fact that this study was conducted and evaluated as a fully integrated clinical intervention within a CBO, as opposed to being conducted with a selected set of research participants

10 118 M. M. Copehhaver et al. within a purely research-based context. This meant, for example, that while participation in the intervention was encouraged, patients were not offered inducements, such as cash payments or vouchers with monetary value, to participate. Although offered in a real-world setting as a part of regular treatment groups, there was no onus on patients to attend and nonparticipation did not negatively affect their status in the treatment program (as it might have for nonattendance at groups that were a component of the patient s treatment plan). At least in part for these reasons, completion of pre- and post-test measures was substantially lower than would have been expected had we conducted a more conventional controlled study in which participant incentives were offered. In addition, because of the clinical context in which this study was implemented, we were obliged to use a very brief assessment battery. Although our statistical analyses were able to account for the completion rate, we recognize that the intervention was unable to reach a number of patients who declined to participate. This otherwise undesirable outcome was informative in a practical sense, however, insofar as it demonstrates the level of patient participation that may be expected in the absence of some form of incentives, even when HIV risk reduction is well integrated into treatment. The above limitations notwithstanding, this study clearly points to the potential benefits of systematically adapting and implementing evidence-based HIV risk reduction approaches by taking into consideration the real world needs and constraints of CBOs. There is an urgent need for a multiplicity of bridges from research to clinical domains in order to ensure that evidence-based interventions available in the literature are actually implemented in clinical settings, even if this means considerably reducing the duration and content areas of the original intervention in order to negotiate acceptability within the clinical context of CBOs. ACKNOWLEDGMENT Funding to support the conduct of this research was provided by the Connecticut Department of Public Health AIDS Division (DPH Log # ) to Michael M. Copenhaver. Funding to support the data analysis and preparation of this manuscript was provided by National Institute on Drug Abuse, grant K23-DA to Michael M. Copenhaver. We wish to thank the patients and staff of the APT Foundation, Inc. for their contribution to the current study, including June Rosner and Gina Lombardi for facilitating the many aspects of the intervention.

11 Integrating an HIV Risk Reduction Intervention 119 REFERENCES 1. Centers for Disease Control and Prevention. Special data request. Available at (Accessed August 18, 2004, from 2. Joint United Nations Programme on HIV=AIDS. Q&A II: Basic facts about the AIDS epidemic and its impact, UNAIDS questions and answers. Available at (Accessed August 16, 2004, from unaids.org) 3. Semaan S, Des Jarlais DC, Sogolow E, Johnson WD, Hedges LV, Ramirez G, Flores SA, Norman L, Sweat MD, Needle R. A meta-analysis of the effect of HIV prevention interventions on the sex behaviors of drug users in the United States. J AIDS 2002; 30:S73 S Copenhaver M, Fisher JD. Experts outline ways to improve the decade-long U.S. trend of 40,000 new HIV infections per year. AIDS Beh 2006; published online 1= Institute of Medicine. In Bridging the Gap Between Practice and Research: Forging Partnerships with Community-Based Drug and Alcohol Treatment. Washington, DC: National Academy Press, Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. We don t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. J Consult Clin Psychol 2005; 73: Morgenstern J, Morgan TJ, McCrady BS, Keller DS, Carroll KM. Manualguided cognitive behavioral therapy training: A promising method for disseminating empirically supported substance abuse treatments to the practice community. Psychol Addict Behav 2001; 5: Knudsen HK, Roman PM. Modeling the use of innovations in private treatment organizations: The role of absorptive capacity. J Subst Abuse Treat 2004; 16: Schoenwald SK, Hoagwood K. Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services 2001; 52: Margolin A, Avants SK, Warburton LA, Hawkins KA, Shi J. A randomized clinical trial of a manual-guided risk reduction intervention for HIV-positive injection drug users. Health Psychol 2003; 22: Avants SK, Margolin A, Usubiaga MH, Doebrick C. Targeting HIV-Related outcomes with intravenous drug users maintained on methadone: A randomized clinical trial of a harm reduction group therapy. J Subst Abuse Treat 2004; 26: Copenhaver M, Avants SK, Margolin A, Warburton LA. Intervening effectively with drug abusers infected with HIV: Taking into account the potential for cognitive impairment. J Psychoactive Drugs 2003; 35: Stein MD, Anderson B, Charuvastra A, Maksad J, Friedmann PD. A brief intervention for harzardous drinkers in a needle exchange program. J Subst Abuse Treat 2002; 22:23 31.

12 120 M. M. Copehhaver et al. 14. Sterk CE, Theall KP, Elifson KW. Effectiveness of a risk reduction intervention among African-American women who use crack cocaine. AIDS Educ Prev 2003; 15: Metzger D, Woody GE, Navaline H, McLellan AT, Meyers K, Boney T, Mulvaney F, Williams J, Dyanick ST, Jonson A, Davis B, Green P, Abrams M, Oglesby P, Davis R, Zanis D, Abellanas L, Incmicoski R, O Brien CP. The Risk Assessment Battery (RAB): Validity and reliability. Presented at The Sixth Annual Meeting of National Cooperative Vaccine Development Group for AIDS; October 30 November 4, 1993; Alexandria, VA. 16. Fisher JD, Fisher WA. Changing AIDS-risk behavior. Psychol Bull 1992; 111: Fisher JD, Cornman DH, Osborn CY, Amico KR, Fisher WA, Friedland GA. Clinician-initiated HIV risk reduction intervention for HIV-positive persons: Formative research, acceptability, and fidelity of the OPTIONS project. J AIDS 2004; 37:S78 S Howell DC. Statistical Methods for Psychology. 3rd ed. Belmont, CA: Duxbury Press, Copenhaver M, Lee IC, Harman J, Johnson BT, Carey M. Behavioral HIV risk reduction among injection drug users: Meta-analytic evidence of efficacy. J Subst Abuse Treat 2006; 31: Metzger DS, Navaline H, Woody G. Drug abuse treatment as AIDS prevention. Pub Health Rep 1998; 113: Pigott TD. Missing predictors in models of effect size. Eval Health Prof 2001; 24: Hoagwood K, Hibbs E, Brent D, Jensen P. Introduction to the special section: Efficacy and effectiveness in studies of child and adolescent psychotherapy. J Consult Clin Psychol 1995; 63:

13