Internal Validity of Project MATCH Treatments Discriminability and Integrity

Transcription

1 Page 1 of 19 Journal of Consulting and Clinical Psychology April 1998 Vol. 66, No. 2, by the American Psychological Association For personal use only--not for distribution. Internal Validity of Project MATCH Treatments Discriminability and Integrity Kathleen M. Carroll Yale University School of Medicine Gerard J. Connors Research Institute on Addiction Ned L. Cooney Yale University School of Medicine Carlo C. DiClemente University of Maryland Baltimore County Dennis M. Donovan University of Washington Ronald R. Kadden University of Connecticut School of Medicine Richard L. Longabaugh Brown University Bruce J. Rounsaville Yale University School of Medicine Philip W. Wirtz George Washington University Allen Zweben ABSTRACT Project MATCH (Matching Alcoholism Treatments to Client Heterogeneity) is a multisite collaborative project designed to evaluate patient treatment interactions in alcoholism treatment. To evaluate whether major threats to the internal validity of the independent (treatment) variable in Project MATCH could be ruled out, we investigated several aspects of treatment integrity and discriminability. In this study, 1,726 alcohol-dependent participants at 10 sites were randomized to 3 treatments: cognitive behavioral treatment (CBT), motivational enhancement therapy (MET), and 12-step facilitation (TSF). Participants received treatment either as outpatients or as aftercare following a more intensive inpatient or day hospital treatment. For both the outpatient and aftercare arms of the study, treatments were discriminable in that therapists implemented each of the treatments according to manual guidelines and rarely used techniques associated with comparison approaches. Participants received a high level of exposure to their study treatments, and the intended contrast in treatment dose between MET and the 2 more intensive treatments (CBT and TSF) was obtained. Alcoholics Anonymous involvement was significantly higher for participants assigned to TSF versus MET or CBT, whereas the treatments did not differ in utilization of other nonstudy treatments. Nonspecific aspects of treatment such as therapist skillfulness and level of the therapeutic alliance were comparable across treatment conditions. This article was presented on August 13, 1996, at the 104th Annual Convention of the American

2 Page 2 of 19 Psychological Association, Toronto, Ontario, Canada. Project MATCH is a cooperative agreement supported by the National Institute on Alcohol Abuse and Alcoholism. We gratefully acknowledge the collaboration of the Project MATCH Research Group. Kathryn Nuro, Charla Nich, Roseann Bisighini, Joann Corvino, Heidi Behr, and Tami Frankforter made important contributions to the data collection and analyses, as did the tape raters who participated in this project. Correspondence may be addressed to Kathleen M. Carroll, Substance Abuse Center, Room S208, 34 Park Street, New Haven, Connecticut, Electronic mail may be sent to kathleen.carroll@yale.edu Received: September 9, 1996 Revised: March 15, 1997 Accepted: May 28, 1997 Comparing the effectiveness of different therapies requires implementation of well-defined, differentiable treatments and substantial control over threats to internal validity, such as diffusion or confounding of treatments, variations in treatment delivery, and differential attrition ( Kazdin, 1995 ). Critical requirements in such studies include the demonstration of both treatment integrity (i.e., the treatments were delivered adequately and as intended; Yeaton & Sechrest, 1981 ) and treatment discriminability (i.e., the treatments compared were substantively different from one another in both their defining characteristics and their implementation). It is also important that the treatments not differ substantially on variables indicating the quality of their delivery (e.g., therapist skill or competence). Protection of the independent variable, and hence internal validity, in comparative therapy research has been strengthened by what Waskow (1984) and others described as a "technology model" of research design and implementation. This model seeks to control extraneous variability and threats to internal validity in clinical trials through specification and standardization of treatments in manuals; thorough description of the nature of treatment delivery, including qualifications, training, and supervision of therapists; and close monitoring of treatment implementation. This model is exemplified by the methods used for the National Institute of Mental Health (NIMH) Treatment of Depression Collaborative Research Program ( Elkin, Parloff, Hadley, & Autry, 1985 ), which demonstrated that manual-guided therapies for depression could be discriminated from one another with high accuracy ( DeRubeis, Hollon, Evans, & Bemis, 1982 ; Hill, O'Grady, & Elkin, 1992 ). Matching research, which seeks to identify interaction effects between patients and treatment, rather than treatment main effects, places complicated demands on the evaluation of treatment integrity and discriminability (see Donovan & Mattson, 1994 ; Finney & Moos, 1986 ; Lindstrom, 1992 ). For example, in main effects research, it is generally sufficient to demonstrate that the treatments compared are differentiable. However, because matching research is predicated on interactions between specific patient characteristics and particular treatment components ( DiClemente, Carroll, Connors, & Kadden, 1994 ), it is necessary that overlap of "active ingredients" across treatments compared be minimized, because significant overlap could obscure matching effects. Thus, in matching research, treatments must be discriminable with respect to active ingredients hypothesized to differentiate the treatments compared. Second, in contrast to main effects research in which sample homogeneity is desirable, matching research requires variability in patient characteristics and, therefore, greater heterogeneity ( Donovan et al., 1994 ). However, patient heterogeneity may present challenges to treatment integrity, because therapists may be more likely to deviate from manual guidelines as they seek to treat a more varied patient sample. Third, adequate power to detect patient treatment interactions in matching research requires large sample sizes and hence greater challenges in implementing treatments consistently and robustly with large numbers of patients, therapists, and, in some cases, multiple sites

3 Page 3 of 19 and settings ( Carroll, Kadden, Donovan, Zweben, & Rounsaville, 1994 ). Project MATCH (Matching Alcoholism Treatments to Client Heterogeneity) is a multisite collaborative project supported by the National Institute on Alcohol Abuse and Alcoholism, and is designed to evaluate a series of a priori hypotheses regarding patient treatment interactions in alcoholism treatment ( Donovan & Mattson, 1994 ; Project MATCH Research Group, 1993 ). Project MATCH involved two independent but parallel matching studies: one with participants recruited from outpatient settings ( N = 952), the other with participants receiving aftercare treatment following inpatient or day hospital treatment ( N = 774). Participants were randomly assigned to one of three manual-guided treatment conditions: 12-step facilitation (TSF; Nowinski, Baker, & Carroll, 1992 ), cognitive behavioral coping skills training (CBT; Kadden et al., 1992 ), or motivational enhancement therapy (MET; Miller, Zweben, DiClemente, & Zychtarik, 1992 ). Treatment was provided in 10 sites (5 outpatient, 5 aftercare) affiliated with nine clinical research units to provide geographic as well as client heterogeneity. Participant assessment was conducted at baseline and at 3-month intervals until 15 months after randomization to treatment. All participants were required to meet Diagnostic and Statistical Manual of Mental Disorders (third edition, revised; DSM-III-R ) criteria for current alcohol abuse or dependence as well as other inclusion and exclusion criteria (see Project MATCH Research Group, 1993 ). In this report, we address the integrity and discriminability of Project MATCH treatments: that is, whether the three treatments were implemented as intended and whether common threats to internal validity were controlled such that alternative explanations of the findings could be ruled out. The following research questions are addressed: (a) Did the therapists implement study treatments as described in the respective manuals? Were the treatments discriminable? (b) Did participants receive an adequate dose of their study treatments? Were intended contrasts in treatment dose between MET versus CBT and TSF obtained? (c) Were the Project MATCH treatments confounded by exposure to other forms of treatment? Were extratreatment exposures consistent with treatment assignment? (d) Did the treatments differ with respect to nonspecific aspects of treatment that might be related to outcome, such as the therapeutic alliance and therapist skill? This report evaluates these treatment discriminability and integrity issues in the context of a large multisite clinical trial and may serve as an example for evaluating these issues in future clinical trials involving the comparison of treatment interventions. Treatments Method The three Project MATCH treatments were selected based on the following criteria: (a) evidence of clinical effectiveness with alcohol-dependent populations, (b) evidence and theoretical rationale for predicting interactions with particular patient characteristics, (c) acceptability for use by practitioners in the treatment community, (d) active therapeutic ingredients that were specifiable and distinct from the other Project MATCH treatments, and (e) feasibility of implementation in a clinical trial ( Donovan et al., 1994 ). Treatment specification and the development of the treatment manuals for Project MATCH differed from previous studies in the emphasis placed on minimizing overlap among them and anticipating application of study treatments to a broad range of alcoholics in diverse clinical settings. Considerable effort was made to identify the "active ingredients" of each treatment type (i.e., the mechanisms of action through which each treatment was hypothesized to interact with patient characteristics) and to reduce overlap among the three treatments with respect to these active ingredients. For example, emphasis on the development of specific skills for coping with cravings or high-risk situations should be present principally in the CBT condition, whereas therapist prescriptions for attending Alcoholics

4 Page 4 of 19 Anonymous (AA) meetings and invoking a higher power should be unique to the TSF condition. Furthermore, even in cases in which Project MATCH treatments held certain concepts in common (e.g., "high-risk situations" in CBT and "people, places, and things" in TSF), efforts were made to heighten distinctions between them through the use of language unique to each approach. The general format of each of the Project MATCH treatment manuals consists of an overview of the treatment approach and its goals, an introductory set of general instructions to the therapists regarding use of the manual, a description of the structure of the treatment sessions, and guidelines for dealing with potential problem situations (e.g., outside crises, missed sessions, resistance) in a manner consistent with each treatment type followed by detailed agendas for each session. Because Project MATCH was conducted as both outpatient treatment and as aftercare following a more intensive inpatient treatment, guidelines were provided for focusing on issues of achieving abstinence for participants at outpatient sites and on using the Project MATCH treatment to help maintain sobriety among participants at aftercare sites. Treatment parameters. Treatments were delivered in individual sessions over a 12-week period, with weekly sessions for CBT and TSF, whereas MET consisted of four sessions occurring during the 1st, 2nd, 6th, and 12th weeks. All three treatments had a goal of abstinence. Up to two additional emergency sessions were allowed in each condition to help therapists manage participants in crisis. Spouses or significant others could participate in up to two of the scheduled sessions; protocols for these sessions were specified in each treatment manual. CBT. This treatment ( Kadden et al., 1992 ) is based on the principles of social learning theory and views drinking behavior as functionally related to major problems in the individual's life. It posits that addressing this broad spectrum of problems will prove more effective than focusing on drinking alone. Emphasis is placed on overcoming skill deficits and increasing the individual's ability to cope with highrisk situations that commonly precipitate relapse. The treatment manual relies on a prior one by Monti, Abrams, Kadden, and Cooney (1989) for specific instructions for many of the treatment sessions. Coping skills training focused on interpersonal difficulties, such as social pressure to drink or inadequate communication skills, and intrapersonal problems, such as anger or depression. Material discussed during sessions was supplemented by extrasession tasks to encourage implementation and practice of coping skills. MET. This ( Miller et al., 1992 ) is based on principles of motivational psychology and is designed to produce rapid, internally motivated change. This treatment strategy does not attempt to train or guide the client, step by step, through recovery but instead uses motivational strategies to mobilize the participant's own resources. MET consists of four carefully planned and individualized treatment sessions. The first two sessions focus on structured feedback from the initial assessment, motivation for change, and developing plans for making changes. The final two sessions (held at Weeks 6 and 12) provide opportunities for the therapist to reinforce progress and check or revise the change plan. TSF. This treatment ( Nowinski, Baker, & Carroll, 1992 ) is grounded in the concept of alcoholism as a spiritual and medical disease. The content of this intervention is consistent with the 12 steps of AA, with primary emphasis given to Steps 1 through 5. In addition to abstinence from alcohol, a major goal of the

5 Page 5 of 19 treatment is to foster the participant's commitment to and participation in AA. During the course of the 12 sessions, participants are actively encouraged to attend AA meetings and to maintain journals of their AA attendance and participation. Therapy sessions are highly structured, following a similar format for each week that includes inquiry about drinking and other target symptoms, review and reinforcement of AA participation, introduction and explication of a theme for the current sessions, and setting goals for AA participation for the next week. Therapists Because of the importance of minimizing overlap across the treatments in a matching study, as well as the low likelihood that therapists could implement all three treatments equally well, therapists were nested within the conditions (i.e., each therapist performed only one type of Project MATCH treatment). To promote consistency and quality of treatment delivery across sites and treatment conditions, the following selection criteria were required of Project MATCH therapist candidates: (a) completion of a master's degree in counseling, psychology, social work, or a closely related field or certification as an alcoholism counselor; (b) at least 2 years of clinical experience after completion of degree or certification, (c) submission of a taped clinical work sample for review, (d) commitment to and experience with the MATCH treatment that the therapist would be conducting, and (e) experience treating alcoholics. By setting uniform training and experience standards across conditions, while also using therapists representative of the usual practitioners of the study treatments, these selection criteria were intended to strike a balance between comparability of therapists across treatment conditions and generalizability of findings to the broader field of alcohol treatment ( Carroll et al., 1994 ). For example, therapists were selected for the CBT treatment who were experienced in cognitive and behavioral techniques; thus, the CBT therapists were predominantly doctoral- or master's-level psychologists. Because the TSF treatment required familiarity and commitment to the principles of 12-step recovery, therapists were sought who were experienced in or who had gone through 12-step recovery themselves and had been abstinent for several years, and were master's-level or certified alcoholism counselors. Finally, the MET intervention was more recently developed; thus, potential therapists had little previous experience in MET per se. Therapists selected for MET typically had experience in systems theory, family therapy, and motivational counseling. Eighty therapists (28 in CBT, 24 in MET, and 28 in TSF) were certified and treated at least one Project MATCH participant. Therapists were predominantly female (62.7%), master's-level clinicians (58%; 23% had doctoral degrees and 20% had bachelor's degrees), with substantial posttraining experience (mean years of experience = 6.3 ± 4.5 SD ). Therapist Training and Monitoring Training and supervision of therapists were centralized at the Project MATCH Coordinating Center using methods developed in previous large-scale collaborative studies ( Elkin, Parloff, Hadley, & Autry, 1985 ). All therapists attended a training seminar that included the background and rationale for Project MATCH, detailed review of the treatment manual, review of taped examples of treatment sessions, practice exercises, and extensive discussion of unique issues related to treating participants in matching studies, particularly consideration of challenges related to treating a heterogeneous patient population while conforming to manual guidelines. Each therapist was then assigned a minimum of two training cases (four for MET), which were conducted following the Project MATCH manuals. All training case sessions were videotaped and sent to the Coordinating Center, where supervisors reviewed them and provided weekly individual supervision to each therapist via telephone ( Witte & Wilber, 1997 ). Therapists were certified by the Coordinating Center after successful completion of training cases. Therapists completed a mean of 26 supervised training sessions before they were certified.

6 Page 6 of 19 After certification, all treatment sessions continued to be videotaped and sent to the Coordinating Center, but intensity of supervision was reduced. Supervisors reviewed only a portion of sessions (one quarter of all TSF and CBT sessions, one half of all MET sessions) from each participant and provided individual supervision on a monthly, rather than weekly, basis. Additional on-site group supervision was provided weekly by the project coordinator at each clinical research unit. To identify therapists whose performance drifted during the course of the study, ongoing evaluations by the Coordinating Center supervisors of each therapist's performance were provided to investigators at each site on a monthly basis. Therapists whose performance drifted during the course of the study were "redlined" by the Coordinating Center, and the frequency of supervision and monitoring of sessions was increased until performance returned to acceptable levels. Process Assessments Process assessment in Project MATCH encompassed multiple domains (treatment dose, treatment discriminability and integrity, therapeutic alliance, and extrasession activities) and perspectives (patient, therapist, supervisors, and independent observers; summarized in DiClemente et al., 1994 ). Treatment discriminability. The methods used for the assessment of treatment discriminability in the NIMH Collaborative Study by DeRubeis and colleagues (1982) were adapted for Project MATCH as follows: First, Likert-type items intended to tap the unique active ingredients of CBT, MET, and TSF were generated from the manuals. Additional items, not described here, were generated to assess interventions that were thought to mediate matching effects of some of the matching hypotheses (e.g., therapy structure). Second, a rating manual was developed that specified the definition and intent of each item and included guidelines for high versus low ratings on each item, sample therapist participant dialogue, and differentiation of each item from other similar items ( Nuro et al., 1995 ). Third, samples of Project MATCH session tapes were rated using the resulting MATCH Tape Rating Scale (MTRS) to assess item reliability. Items that had low reliabilities were reworded or dropped. The MTRS and rating manual were finalized in six iterations, which involved ratings of approximately 50 randomly selected treatment sessions each. The MTRS was then used to rate a portion of all Project MATCH sessions. Selectivity was necessary because of the large number of participants, the need to include some session process data from all participants (to evaluate process variables thought to underly the causal chains of the matching hypotheses), and the differences in the number and timing of sessions between CBT and TSF versus MET. The second session from each participant was rated to provide comparable representation of all three treatments and maximize the number of participants included. Of the 1,507 participants who completed at least two sessions, 1,422 ratings were completed (94%); the discrepancy was due primarily to equipment failures or inaudible tapes. To assess variability in therapist behavior across time, a smaller sample (294) of sessions that occurred midway through treatment (modally Week 6; sixth session for TSF and CBT, third session for MET) was rated as well. This sample was selected to provide a balanced sample with respect to treatment arm, site, treatment, and therapist. Treatment dose, exposure to nonstudy treatments. The number and duration of therapy sessions were assessed based on session reports and checklists completed by the therapists after each session. Exposure to nonstudy treatments, including detoxification, inpatient and outpatient treatment, and self-help involvement during the 90-day treatment period, was assessed at the end of treatment using Form 90 ( Miller & DelBoca, 1994 ), a structured interview.

7 Page 7 of 19 Therapist skillfulness. Therapist skillfulness, or competence, was assessed using a uniform treatment-independent measure that assessed general therapist skill, therapist empathy, and therapist nonverbal behavior. Therapist competence measures geared to each type of treatment (e.g., CBT, MET, and TSF; see Waltz, Addis, Koerner, & Jacobson, 1993 ), would have required that observers be trained to assess competence in the three types of treatment (and hence raters could not be unaware of treatment type). Instead, we devised uniform measures so independent raters, who were unaware of participants' treatment assignment and who were not familiar with the content of the treatment manuals, could rate both the MTRS and the skillfulness measure for the large number of sessions to be rated. Therapeutic alliance. The Working Alliance Inventory (WAI; Horvath & Greenberg, 1986 ) was selected for use in Project MATCH for several reasons. First, the orientation of the WAI system is eclectic and thus better suited to the three treatments being studied in Project MATCH than other assessment approaches that reflect a more psychodynamic formulation of the alliance. Second, the psychometric properties of the WAI are comparatively well established ( Tichenor & Hill, 1989 ; Tracey & Kokotovic, 1989 ). Third, there are parallel forms of the WAI for rating by the patient and the therapist. Participants completed the WAI-C (client version) after the second session. Raters and Rater Training Raters for the MTRS and Skillfulness scales were 19 experienced clinicians, most of whom also had some background in alcohol treatment. Most were women (89%) and White (95%). Forty-seven percent were master's- or doctoral-level psychologists, 42% were master's-level social workers, and 11% were psychiatric nurses. Raters averaged 9.8 (± 6.6 SD ) years of clinical experience. In terms of their theoretical orientation, 33% reported a primarily psychodynamic orientation, 22% eclectic, 17% interpersonal, 11% cognitive behavioral, 11% strategic or family therapy, and 6% 12-step orientation. Rater training largely paralleled procedures used in therapist training ( Carroll & Nuro, 1997 ). First, rater trainees attended a didactic seminar that included detailed review of the rating manual as well as several group practice ratings on taped examples of items. Second, rater trainees rated at least 10 practice tapes, which were evaluated with respect to consensus ratings of those tapes provided by supervisors at the Coordinating Center. Raters were certified after their ratings achieved acceptable reliability with respect to the consensus ratings. Rater recalibration sessions were held quarterly to prevent rater drift. Data Analyses Treatment discriminability, dose of treatment, therapist skillfulness, and working alliance data were analyzed using analysis of variance (ANOVA) models to evaluate treatment effects. Because of the large sample size, however, even small, clinically insignificant differences between treatment groups were likely to be statistically significant. Therefore, to evaluate the magnitude of these effects more clearly in the context of other sources of variance (site, therapist, and rater effects), multiple-groups profile analysis ( Harris, 1985 ) was used to evaluate treatment, session, and site effects. This approach uses theta to estimate the amount of variance for each effect and was used to assess discriminability of study treatments from the NIMH Collaborative Study ( Hill et al., 1992 ). The model for the analyses included all effects and their interactions, for example, Treatment (TSF, CB, MET), Site, Treatment Site, and Therapist (within site and treatment), with rater as a covariate. For the subsample that included

8 Page 8 of 19 sessions from both Treatment Weeks 2 and 6, additional analyses were used to evaluate treatment week effects, in which the model included Treatment, Site, Treatment Site, Therapist (within treatment and site), Week (2 vs. 6), and Treatment by Week. Results Treatment Integrity and Discriminability Psychometric analyses of MTRS. Table 1 summarizes psychometric analysis of the MTRS and presents estimates of interrater reliability, internal consistency, and treatment subscale loadings for the individual items composing the CBT, and TSF scales of the MTRS. Intraclass correlation coefficients were calculated to provide an estimate of item reliabilities from a reliability sample in which 12 randomly selected tapes were each rated by 13 raters ( n = 156). Using a random-effects model ( Shrout & Fleiss, 1979 ; model [2,1]) as an estimate of reliabilities for independent samples, estimates for the CBT, MET, and TSF treatment subscales scores were.46,.69, and.88, respectively. Estimates from Shrout and Fleiss model (3,1), which estimates reliability for this set of raters (fixed effects), were.51,.79, and.91, respectively. Internal consistency of the treatment subscales, determined by coefficient alpha, were.70 (CBT),.82 (TSF), and.80 (MET) (see Table 1 ). Confirmatory rather than exploratory factor analysis was used to evaluate factor structure, because the scale was developed specifically to characterize the three types of treatment evaluated in this trial; therefore, an underlying three-factor structure, corresponding to the three treatment subscales, was hypothesized a priori ( Floyd & Widaman, 1995 ). Two items that were shared across the CBT and TSF scales were dropped a priori from this and subsequent discriminability analyses (assignment of homework, review of homework assignments). The adjusted goodness-of-fit index for the three-factor solution for the Week 2 data was 0.91, χ 2 (116, N = 1,412 = 1,184.6) and for the Week 6 data 0.89, χ 2 (116, N = 291) = 307.4, suggesting the factor structure we prespecified was supported and consistent across treatment sessions. Phi coefficients, an index of the level of the relationship between scales, suggested low and often negative correlations between pairs of the treatment subscales. The magnitude and direction of the coefficients were roughly comparable for both the early and late sessions (Week 2: CBT MET =.14, MET TSF =.05, CBT TSF =.07; Week 6: CBT MET =.08, MET TSF =.07, CBT TSF =.39). Discriminability analyses. As shown in Table 2, mean raw scale scores were significantly different by treatment for both the outpatient and aftercare arms of the study. Subsequent univariate contrasts were all significant in the expected direction. For example, for the outpatient study, the mean CBT scale score was higher for participants assigned to CBT treatment than those assigned to MET or TSF, F (2, 802) = , p <.001. For the MET subscale, participants assigned to MET had a higher mean score than those assigned to CBT or TSF, F (2, 802) = , p <.001. Finally, for the TSF scale, participants assigned to TSF had higher scores than those assigned to CBT or MET, F (2, 802) = 1,414.19, p <.001. To evaluate the treatment differences in the context of other sources of variance in subscale scores (e.g., site, raters, therapists) and the large sample size, multiple-groups profile analysis ( Harris, 1985 ; Stevens, 1992 ) was used to evaluate treatment discriminability in the context of the magnitude of other effects. This analytic method is similar to a multifactor repeated measures ANOVA, but it allows more than a single dependent measure to be included in the analysis ( Hill et al., 1992 ; Stevens, 1992 ) and

9 Page 9 of 19 thus is a comparatively efficient model for simultaneously evaluating multiple sources of variance (in this case, treatments, sites, therapists, raters, sessions) on multiple scales treated simultaneously (in this case, the CBT, TSF, and MET subscale scores). The statistic for this analysis is Roy's theta (comparable to canonical R 2 ), which indicates the percentage of variance accounted for by each factor coefficient of determination. The statistics do not sum to 1 because each effect represents a different set of predictor variables in the equation. Also, because of the need for a balanced model for these analyses, the sample size does not total 1,422. Treatment and site effects. Tables 3 and 4 present the results of the multiple-groups profile analysis for the Week 2 data for both the outpatient and aftercare arms of the study. All effects were significant at p <.05. These analyses suggest the Project MATCH treatments were highly discriminable, because the treatment main effect accounted for the majority of the variance in the three treatment subscale scores for both the outpatient ( θ =.84) and aftercare ( θ =.85) arms of the study. Theta values for the other effects (rater, therapist, site, and Site Treatment) evaluated suggest that these account for comparatively small proportions of variance in the treatment scores. For example, for the outpatient study, the therapist effect accounted for 18% of variance, whereas site effects and Treatment Site effects accounted for only 1% and 4% of the variance. On the aftercare side, therapist effects accounted for 10% of variance, and site (6%) and Treatment Site (8%) effects were slightly higher than for the outpatient study. The univariate results, which illuminate the source and degree of variance by treatment, are also presented in Tables 3 and 4. For both the inpatient and outpatient studies, each of the Treatment subscale scores (CBT, MET, TSF) was significantly different by treatment. For the outpatient study, the Treatment subscale scores were not significantly different by site or Treatment Site, with the exception of CBT, suggesting that implementation of CBT differed by site. For the aftercare study, there was slightly more variation by site and Treatment Site; each of these effects was significant with the exception of the site effect for TSF. Figures 1 and 2 present standardized Treatment subscale scores by treatment and site for each of the study arms. Although there was some variation by site, the figures illustrate that the three treatments were quite distinct and implemented with some consistency across the Project MATCH sites for both arms of the study. Session effects. Multiple-groups profile analyses were also used to evaluate changes in implementation of the Project MATCH treatments over time. The model for these analyses includes all effects from the previous model but adds effects for treatment session, and therefore includes only the subsample of 294 participants for whom both an early (Week 2) and a later (Week 6) session was rated. For the outpatient study, as in the model described previously, the treatment effect is significant and accounts for most of the variance in the three treatment subscale scores ( θ =.82) and the session effect is also significant ( θ =.18), suggesting some reduction in scores between Weeks 2 and 6 for all treatments. For the aftercare study, the site effect was also statistically significant and accounted for substantial variance in scores ( θ =.16). The aftercare univariate statistics suggest significant session effects for TSF and MET but not CBT, indicating treatment subscale scores decreased between Weeks 2 and 6 for TSF and MET, but scores for CBT did not change significantly.

10 Page 10 of 19 Outpatient versus inpatient study. Although Project MATCH was conducted as two independent but parallel studies, the issue of whether there were differences in treatment by study arm is of interest, particularly to the extent that matching results differed by arm ( Project MATCH Research Group, 1997 ). Another multiple-groups profile analysis, using only the Week 2 data, evaluated the three treatment subscale scores simultaneously by treatment, arm, and Treatment Arm, with raters and therapists as covariates. Again, this analysis suggested the majority of variance was due to treatment ( θ =.82), with comparatively little variance associated with study arm ( θ =.04), site ( θ =.01), Treatment Arm ( θ =.01), and covariate ( θ =.11) effects. Treatment Dose Mean number of sessions attended by treatment and site is presented in Table 5. There were sharp differences in number of sessions attended by treatment condition. These were consistent with the intended dose contrasts; participants assigned to TSF and CBT attended more than twice as many sessions as those assigned to MET in both arms of the study. There was also higher attendance in CBT than TSF on the outpatient side. For the outpatient side of the study, the mean number of sessions completed, across sites, was 8.3 for CBT, 7.5 for TSF, and 3.3 for MET. For the aftercare study, the mean number of sessions was 8.0 for CBT, 7.3 for TSF, and 3.1 for MET. Table 5 also presents the proportion of available sessions attended by treatment for both arms of the study, which addresses the issue of adequacy of participants' exposure to study treatment. Although session attendance was comparatively high across conditions, differences were statistically significant for both arms of the study, because fewer sessions were offered in MET than the other two treatments. For the outpatient study, participants in CBT, MET, and TSF attended 69%, 83%, and 63% of sessions offered, respectively. For the aftercare study, rates were 67% (CBT), 78% (MET), and 61% (TSF). Significant Other Involvement Although each of the treatments allowed up to two significant other sessions, involvement of significant others in treatment was seen as a key component of MET ( Miller et al., 1992 ) and thus more actively encouraged in MET. Table 6 presents involvement of significant others in Treatment Treatment condition and study arm. For both arms of the study, one quarter to one third of participants had a significant other (usually a spouse) participate in at least one of the treatment sessions, with significantly higher rates in MET compared with CBT and TSF for the outpatient study but no significant differences for the aftercare study. Significant other sessions as a percentage of all sessions is also presented in Table 6. As expected, for both arms of the study, participation was significantly higher in MET than CBT or TSF, comprising 17% of outpatient sessions and 13% of aftercare sessions in MET versus 3 to 4% of CBT and TSF sessions. Exposures to Nonstudy Interventions Substantial exposure to nonstudy treatment during the treatment phase of the trial, or differential levels of such exposure across treatment conditions, could pose another threat to internal validity. Data for the multiple types of possible nonstudy treatment were combined because the level of participation in any one category was very low, and median values for any single category of participation or combination category (inpatient treatment or outpatient treatment) was 0 for both arms of the study. The data presented in Table 7 suggest that the total level of participation in non-match treatment, including inpatient treatment-detoxification or professional outpatient psychotherapy or pharmacotherapy for a

11 Page 11 of 19 substance use or psychiatric disorder, was very low overall and did not differ significantly across the three treatments for both arms of the study. Although exposure to professional treatment outside the study was infrequent and comparable across treatment conditions, participation in AA and other self-help groups, which by design was intended to be significantly higher in TSF than CBT or MET, did distinguish the treatments in the direction expected. For the outpatient study, participants assigned to TSF reported attending AA meetings an average of 21% of treatment days versus 4% for CBT and 7% for MET, F (2, 918) = 80.1, p <.001. For the aftercare arm of the study, percentage of days of AA attendance was 41% for TSF versus 28.2% for CBT and 29% for MET, F (2, 737) = 11.4, p <.001. The higher overall rate of AA attendance for aftercare participants very likely reflects substantial encouragement to attend AA meetings as part of the inpatient for day hospital treatment, which preceded their involvement with Project MATCH treatments. Nonspecific Aspects of Treatment Therapist skillfulness. Unadjusted means for the four variables assessing Therapist Skillfulness (general skillfulness, boundaries, empathy, and nonverbal behavior) Study Arm and treatment type are given in Table 8. Overall, therapists in each condition were scored by the independent observers as having "good" levels of skillfulness across conditions, with significantly higher ratings for MET than CBT or TSF for some variables on the outpatient side and more consistent effects favoring MET and CBT on the aftercare side. Again, to put the magnitude of these effects in context, multiple-groups profile analysis was used. On the outpatient side of the study, although all of the effects were significant, each of the effects accounts for comparatively little variance in skillfulness ratings. For example, Roy's theta values for the therapist (.11), treatment (.02), site (.04), Treatment Site (.03), and covariate (rater;.02) effects were all comparatively small. On the aftercare side, all of the effects were significant, again with more of the explained variance in this model associated with the therapist (.13) and site (.13) effects than treatment (.07), Treatment Site (.06), or rater (.03). Thus, relatively little variation in therapist skillfulness appeared to be due to treatment differences; therapist effects accounted for higher proportions of variance in these ratings. Working alliance. The multiple-groups profile analysis for the Task, Bond, and Goal scales (again considered simultaneously) of the client version of the WAI-C suggested little variability across Project MATCH sites and treatments from the participants' perspective. As shown in Table 9, although there were some significant effects, the effects were not consistent across subscales or treatment conditions. Multiplegroups profile analyses suggested that, for the outpatient study, only the treatment and therapist effects were significant, and theta values for the treatment, site, Treatment Site, and therapist effects were 03,.00,.03, and 04, respectively, which suggests that treatment type accounted for relatively little of the variance in WAI scores. The univariate tests suggest significant differences by treatment and therapist only for the Goal subscale and not the Task and Bond subscales. A significant Treatment Site interaction was seen for the Bond subscale as well. For the aftercare side, only the Treatment Site effect was significant. Thetas for the treatment, site, Treatment Site, and therapist effects were.02,.05, and.05, respectively. Discussion We evaluated several dimensions of treatment discriminability, treatment exposure, and nonspecific aspects of treatment in a large-scale, multisite study of psychosocial treatments for alcohol abuse and dependence. For both the outpatient and aftercare arms of the study, treatments were discriminable in

12 Page 12 of 19 that study therapists made use of techniques associated with their particular treatment manual and little use of techniques associated with comparison approaches. On average, participants received adequate exposure to the study treatments and intended contrasts in treatment dose between MET and the two more intensive treatments (CBT and TSF) were obtained. During the active phase of the study, participants infrequently reported involvement in professional treatment outside of the study. As expected, involvement in AA was significantly higher for participants in TSF versus the other two treatments for both arms of the study. Nonspecific aspects of treatment such as therapist skillfulness and level of the therapeutic alliance were largely comparable across treatment conditions. These findings suggest that, overall, the manual-guided treatments in Project MATCH were implemented successfully and at a high level of integrity, and several major threats to internal validity were ruled out. First, support for the critical issue of treatment discriminability (i.e., whether the treatments as practiced differed from one another as specified in the respective treatment manuals) comes from several sources of data. Based on independent evaluators ratings of the treatment sessions, correlations between the three treatment subscale scores were low or negative. Factor analysis of the MTRS suggests that CBT is a treatment characterized by emphasizing skill training, identifying and analyzing past and future high-risk situations, using practice exercises, and making distinctions between limited episodes of alcohol use and full relapse; MET is characterized by emphasizing setting goals for treatment, increasing commitment to change drinking, addressing patient ambivalence, and providing feedback around negative consequences of drinking; and TSF is characterized by emphasizing 12-step principles of recovery and AA involvement, confronting patient denial, and emphasizing the disease concept of alcoholism. Treatment integrity was also supported in that mean treatment subscale scores were significantly different in the expected directions; most of the variance was accounted for by treatment condition. Other sources of variance, such as site, Treatment Site, and week (time) accounted for smaller proportions of variance in treatment subscale scores relative to treatment modality for both arms of the study, suggesting that treatment implementation was comparatively uniform across this very large national study. However, therapist effects were second to treatment condition in terms of variance accounted for (18% for the outpatient study, 10% for the aftercare study). This is consistent with findings from previous large multisite psychotherapy efficacy studies ( Hill et al., 1992 ) and suggests that, although the intensive efforts made in Project MATCH to train and supervise therapists closely throughout the study may have reduced variability in the treatment variable, the effect of the individual therapist implementing the treatment is still considerable. Therapist characteristics and therapist effects on outcome will be the subject of future reports ( Project MATCH Research Group, in press ). Similarly, analyses of the session videotapes also suggested some site variation in treatment implementation, particularly for the aftercare study, suggesting that treatment delivery was not completely uniform across sites. Like the therapist effects, however, the magnitude of the site effects was comparatively small with respect to the treatment effects. Beyond the differences in treatment subscale scores based on the independent observers' ratings of a sample of videotaped treatment sessions, a number of other process variables support the integrity of Project MATCH treatments. For example, a primary goal of TSF was to encourage participant participation in AA or other self-help groups. AA participation during treatment was high for participants assigned to TSF and low in the other two types of treatment, particularly for the outpatient arm of the study. The higher levels of AA participation for CBT and MET on the aftercare side of the study most likely reflect an emphasis on AA attendance in the inpatient programs from which participants were recruited and the unfeasibility of barring participants from attending AA if they desired. Similarly, the MET manual stressed the importance of significant other participation in treatment. Although a maximum of two significant other sessions was set for all treatments, significant other participation was higher in MET than TSF or CBT for both arms of the study. The impact of these

13 Page 13 of 19 and other treatment process variables (e.g., homework completion in CBT) on treatment participation and outcome is the subject of future reports. A second major potential threat to internal validity is inadequate or differential treatment compliance, which is a common problem in alcohol treatment research ( Miller, 1985 ). In Project MATCH, however, participants attended a high proportion of the treatment sessions offered, and intended contrasts in treatment dose between MET and the two more intensive forms of treatment (TSF and CBT) were obtained. This is significant not only from the perspective of treatment integrity and potential compliance bias ( Feinstein, 1979 ), but also because several of the primary matching hypotheses were based on the existence of sharp differences in intensity between MET and the other two forms of treatment. Another potential threat to internal validity would be substantial confounding of study treatments by exposure to nonstudy treatments. This would be particularly problematic if, for example, participants in the less intensive MET differentially sought out treatment outside of the study. However, participants' exposures during the 3 months of Project MATCH treatment to nonstudy treatments (e.g., pharmacotherapy or psychotherapy related to either alcohol or psychiatric problems) was low overall and not significantly different by study treatment. It should be noted that although self-help involvement was significantly higher among participants assigned to TSF for both the outpatient and aftercare studies, there was some participation on self-help by participants in CBT and MET as well, most markedly on the aftercare side. Thus, although the desired contrast in AA attendance between TSF and the other two treatments was achieved, some overlap did occur. However, attempting to prevent participants from attending self-help groups if they wanted to attend would have been unfeasible and perhaps unethical; similarly, limiting the sample pool to only those who stated they would not attend AA unless assigned to that condition would have severely limited the generalizability of the sample. A third major threat to internal validity would be posed by differences in the treatments along nonspecific dimensions, which were assumed to be uniform across the treatments. In general, nonspecific process variables such as therapist skillfulness (from the independent raters' perspective) and working alliance (from the patients' perspective) were comparable across treatments. Although some of the ANOVAs indicated significant treatment effects, the magnitude of the difference in scores by treatment is comparatively small, and statistical significance is most likely due to the large sample size; moreover, the multiple-groups profile analyses suggested the proportion of variance in these variables accounted for by treatment condition was small. As the largest patient treatment matching study conducted to date, Project MATCH has a number of important strengths. Broadly, these include a very large sample size and consequent power to detect matching effects; diversity with respect to geographic area and treatment setting so that results are likely to be generalizable; careful consideration given to critical issues in matching research such as sample characteristics; selection of matching, process, and outcome variables; specification of matching hypotheses a priori; and analysis of patient treatment interactions over time (see Donovan & Mattson, 1994 ). With regard to the treatments evaluated and their implementation, methodological strengths included careful consideration of treatments selected, state-of-the-art treatment manuals, selection of a large cohort of therapists representative of the types of practitioners who apply these treatments in clinical settings, extensive efforts to reduce variability in treatment implementation through therapist training and supervision throughout the study, and an extensive process battery to evaluate treatment implementation at multiple time points and from multiple perspectives. The treatment process assessments represent the largest and most detailed study evaluating the process of alcoholism treatment undertaken to date. Nevertheless, the data presented here have some limitations. Regarding the ratings of therapy sessions, although the number of sessions included here (1,700 sessions) is quite large from the perspective of traditional psychotherapy process research and includes at least 1 session from more than 90% of all

14 Page 14 of 19 patients assigned to treatment, it represents only a small proportion of all sessions in Project MATCH (more than 11,000 sessions). Thus, the bulk of the evidence supporting treatment discriminability is based on a single, early session for most patients. The inclusion of a small subsample of sessions that occurred at the midpoint of treatment (Week 6) partially addressed this issue and provided some support for the consistency of treatment delivery across time. However, detailed evaluation of delivery of the active ingredients of Project MATCH treatments across time based on session ratings was not possible because of the large size of the sample and thus the cost of a larger process sample. We did, however, collect detailed therapist self-reports on delivery of manual-specified ingredients for all sessions; future reports will address the correspondence of rater versus therapist reports of treatment delivery and may provide the opportunity to address more complex questions regarding the delivery of specific interventions across time and their relation to outcomes in Project MATCH. Another limitation was the mixed level of reliability of some of the MTRS scales, which limits the strength of several of the findings presented here. However, the large group of raters required for this study precluded a completely balanced design for the reliability analysis, so it is possible that the interrater reliability indexes were artificially low. Finally, although the skillfulness ratings indicated that the treatments were comparable with respect to "generic" competence factors, we did not, as noted previously, specifically assess the therapists' competence in delivering each of the MATCH treatments, because it would have required a second assessment of this very large sample of tapes by raters who were not unaware of the contents of the treatment manuals or the participants' treatment assignment. Our initial findings regarding treatment integrity and discriminability address several important questions raised by the very limited support for the a priori matching hypotheses ( Project MATCH Research Group, 1997 ). The data presented here enable us to rule out some alternate explanations of those findings. In particular, it does not appear that few matching effects were found because there was little difference in the treatments as implemented. Rather, results from these analyses suggest that therapists consistently delivered the study treatments in accordance with the respective treatment manuals and largely refrained from using key "active ingredients" associated with the comparison approaches. Process variables associated with hypothesized active ingredients of each approach, such as AA attendance, also supported treatment discriminability. Similarly, compliance bias is unlikely because participants received high levels of their intended treatment "dose." The high and consistent levels of treatment compliance also suggest that the study treatments were well accepted by most study participants. That, together with the care given to therapist training and supervision as well as other design features of the study, may have worked to produce the good outcomes seen overall. Although these data do suggest that treatments differed from each other on several intended dimensions of treatment process, these differences existed in the context of important nonspecific dimensions, including therapist skill and therapeutic alliance, that in general did not distinguish the treatments. In other words, although Project MATCH treatments differed in many ways, they were also similar on many dimensions, only some of which were directly assessed. Empirical studies of psychotherapy process have, in general, demonstrated stronger relationships between nonspecific aspects of therapy and outcome than between specific "active ingredients" and outcome ( Orlinsky, Grawe & Parks, 1994 ; Stiles & Shapiro, 1992 ), suggesting that delivery of an appropriate, high-quality treatment by a competent therapist may be more important than the specific nature of the treatment delivered. In the next phase of Project MATCH, detailed causal chain analyses ( Longabaugh, Wirtz, DiClemente, & Litt, 1994 ) will be undertaken to evaluate other aspects of the relationship between treatment and patient characteristics that were predicted to underlie each of the matching hypotheses, and thus better explain why individual matching hypotheses were or were not supported. These and other specific questions regarding the relationships among patient, therapist, and process variables, as well as the relationship between treatment process variables and outcome, will be the subject of future reports.

15 Page 15 of 19 References Carroll, K. M., Kadden, R., Donovan, D., Zweben, A. & Rounsaville, B. J. (1994). Implementing treatment and protecting the validity of the independent variable in treatment matching studies.( Journal of Studies on Alcohol (Suppl. 12), ) Carroll, K. M. & Nuro, K. F. (1997). The use and development of treatment manuals.(in K. M. Carroll (Ed.), Improving compliance with alcoholism treatment (NIAAA Project MATCH Monograph Series, Vol. 6, pp , NIH Publication ). Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism.) DeRubeis, R. J., Hollon, S. D., Evans, M. D. & Bemis, K. M. (1982). Can psychotherapies for depression be discriminated?(a systematic investigation of cognitive therapy and interpersonal therapy. Journal of Consulting and Clinical Psychology, 50, ) DiClemente, C., Carroll, K. M., Connors, G. & Kadden, R. (1994). Process assessment in treatment matching research.( Journal of Studies on Alcohol (Suppl. 12), ) Donovan, D., Kadden, R., DiClemente, C. C., Carroll, K. M., Longabaugh, R., Zweben, A. & Rychtarick, R. (1994). Issues in the selection and development of therapies in alcoholism treatment matching.( Journal of Studies on Alcohol (Suppl. 12), ) Donovan, D. M. & Mattson, M. E. (Eds.) (1994). Alcoholism treatment matching research: Methodological and clinical approaches.( Journal of Studies on Alcohol (Suppl. 12).) Elkin, I., Parloff, M. B., Hadley, S. W. & Autry, J. H. (1985). NIMH treatment of depression collaborative research program: Background and research plan.( Archives of General Psychiatry, 42, ) Feinstein, A. R. (1979). "Compliance bias" and the interpretation of therapeutic trials.(in R. B. Haynes, D. W. Taylor, & D. L. Sackett (Eds.), Compliance in healthcare (pp ). Baltimore, MD: Johns Hopkins University Press.) Finney, J. W. & Moos, R. H. (1986). Matching patients with treatments: Conceptual and methodological issues.( Journal of Studies on Alcohol, 47, ) Floyd, F. J. & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments.( Psychological Assessment, 7, ) Harris, R. J. (1985). A primer of multivariate statistics (2nd ed.).(new York: Academic Press) Hill, C. E., O'Grady, K. E. & Elkin, I. (1992). Applying the Collaborative Study Psychotherapy Rating Scale to rate therapist adherence in cognitive-behavior therapy, interpersonal therapy, and clinical management.( Journal of Consulting and Clinical Psychology, 60, ) Horvath, A. O. & Greenberg, L. (1986). The development of the Working Alliance Inventory.(In L. S. Greenberg & W. M. Pinsof (Eds.), The psychotherapeutic process: A research handbook (pp ). New York: Guilford Press.) Kadden, R., Carroll, K. M., Donovan, D., Cooney, N., Monti, P., Abrams, D., Litt, M. & Hester, R. (1992). Cognitive-behavioral coping skills therapy manual: A clinical research guide for therapists treating individuals with alcohol abuse and dependence (NIAAA Project MATCH Monograph Series Vol. 3, DHHS Publication No. [ADM] ).(Rockville, MD: National Institute on Alcohol Abuse and Alcoholism) Kazdin, A. E. (1995). Methods of psychotherapy research.(in B. Bongar & L. E. Beutler (Eds.), Comprehensive textbook of psychotherapy: Theory and practice (pp ). New York: Oxford University Press.) Lindstrom, L. (1992). Managing alcoholism: Matching clients to treatment. (New York: Oxford University Press) Longabaugh, R., Wirtz, P. W., DiClemente, C. C. & Litt, M. (1994). Issues in the development of client treatment matching hypotheses.( Journal of Studies on Alcohol (Suppl. 12), ) Miller, W. R. (1985). Motivation for treatment: A review with special emphasis on alcoholism. ( Psychological Bulletin, 98, )

16 Page 16 of 19 Miller, W. R. & DelBoca, F. K. (1994). Measurement of drinking behavior using Form 90 Family of Instruments.( Journal of Studies on Alcohol (Suppl. 12), ) Miller, W. R., Zweben, A., DiClemente, C. C. & Rychtarik, R. G. (1992). Motivational enhancement therapy manual: A clinical research guide for therapists treating individuals with alcohol abuse and dependence (NIAAA Project MATCH Monograph Series Vol. 2, DHHS Publication No. [ADM] ).(Rockville, MD: National Institute on Alcohol Abuse and Alcoholism) Monti, P. M., Abrams, D. B., Kadden, R. M. & Cooney, N. L. (1989). Treating alcohol dependence: A coping skills training guide in the treatment of alcoholism. (New York: Guilford Press) Nowinski, J., Baker, S. & Carroll, K. M. (1992). Twelve-step facilitation therapy manual: A clinical research guide for therapists treating individuals with alcohol abuse and dependence (NIAAA Project MATCH Monograph Series Vol. 1, DHHS Publication No. [ADM] ).(Rockville, MD: National Institute on Alcohol Abuse and Alcoholism) Nuro, K., Carroll, K. M., Behr, H., Bisighini, R. & McLean, R. (1995). Raters manual for the MATCH Tape Rating Scale. (Unpublished manuscript) Orlinsky, D. E., Grawe, K. & Parks, B. K. (1994). Process and outcome in psychotherapy-noch einmal. (In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change, 4th ed. (pp ). New York: Wiley.) Project MATCH Research Group (1993). Project MATCH: Rationale and methods for a multisite clinical trial matching alcoholism patients to treatment.( Alcoholism: Clinical and Experimental Research, 17, ) Project MATCH Research Group (1997). Matching alcohol treatments to client heterogeneity: Posttreatment drinking outcomes.( Journal of Studies on Alcohol, 58, 7 29.) Project MATCH Research Group in press Psychotherapy Research. Shrout, P. E. & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. ( Psychology Bulletin, 86, ) Stevens, J. (1992). Applied multivariate statistics for the social sciences (2nd ed.).(hillsdale, NJ: Erlbaum) Stiles, W. B. & Shapiro, D. A. (1994). Disabuse of the drug metaphor: Psychotherapy process-outcome correlation.( Journal of Consulting and Clinical Psychology, 62, ) Tichenor, V. & Hill, C. E. (1989). A comparison of six measures of working alliance.( Psychotherapy, 26, ) Tracey, T. J. & Kokotovic, A. M. (1989). Factor structure of the Working Alliance Inventory. ( Psychological Assessment, 1, ) Waltz, J., Addis, M. E., Koerner, K. & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence.( Journal of Consulting and Clinical Psychology, 61, ) Waskow, I. E. (1984). Specification of the technique variable in the NIMH Treatment of Depression Collaborative Research Program.(In J. B. W. Williams & R. L. Spitzer (Eds.), Psychotherapy research: Where are we and where should we go (pp ). New York: Guilford Press.) Witte, G. & Wilber, C. H. (1997). A case study in clinical supervision: Experience from Project MATCH.(In K. M. Carroll (Ed.), Improving compliance with alcoholism treatment (NIAAA Project MATCH Monograph Series, Vol. 6, pp Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism.) Yeaton, W. H. & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: Strength, integrity, and effectiveness.( Journal of Consulting and Clinical Psychology, 49, ) Figure 1. Treatment discriminability, outpatient study: treatment subscale standardized scores by treatment and site ( n = 635). c (and CBT) = cognitive behavioral treatment; t = 12-step facilitation; m (and MET) = motivational enhancement therapy.

17 Page 17 of 19 Figure 2. Treatment discriminability, aftercare study: treatment subscale standardized scores by treatment and site ( n = 528). c (and CBT) = cognitive behavioral treatment; t = 12-step facilitation; m (and MET) = motivational enhancement therapy. Table 1.

18 Page 18 of 19 Table 2. Table 3. Table 4. Table 5. Table 6. Table 7.

19 Page 19 of 19 Table 8. Table 9.