An increased openness to diverse theoretical perspectives

Special Article Psychotherapy Research New Findings and Implications for Training and Practice Per Høglend, M.D., Ph.D. The last decade has seen progress in psychotherapy research, despite the methodological complexity in this field. However, empirical research has influenced training and clinical practice to only a limited extent. This article is a brief evaluation of trends and some findings in modern psychotherapy research that may influence professional psychotherapy training and practice. (The Journal of Psychotherapy Practice and Research 1999; 8:257 263) An increased openness to diverse theoretical perspectives and a somewhat greater interest in empirical evaluation have characterized the field of professional psychotherapy during recent years. Several aspects of the macro-theories of personality and therapeutic change such as the behavioral, psychoanalytic, humanistic, and others have not held up to scrutiny by empirical research, nor have they been translatable into operations that will bear scrutiny. Renewed versions of hermeneutic, social constructionist, and qualitative approaches appear promising but so far have had limited value. More pragmatic clinical theories and mini-theories to guide questions of technique are derived from empirical research, and the trend toward mini-theories centered on specific problem domains and empirical evaluations will continue. Today there are fewer classical analysts, strict behaviorists, or biomedical reductionists than in the past. 1 The empirical analysis and merging of behavioral and cognitive psychotherapy is one salient trend. Behaviorists have not only accepted the importance of cognition, but are now including affect and altered versions of the idea of unconscious processes in their treatment. Psychodynamic therapy has become more short-term and focused, especially in North America. Research evidence has undermined the notion that transference interpretation in particular is the key to ef- Received December 11, 1998; revised March 30, 1999; accepted April 22, 1999. From the Department of Psychiatry, University of Oslo, P.O. Box 85, Vinderen, 0319 Oslo, Norway. Send correspondence to Dr. Høglend at the above address. Copyright 1999 American Psychiatric Association J Psychother Pract Res, 8:4, Fall 1999 257

Psychotherapy Research: New Findings ficacy in the psychodynamic approach, at least within brief time limits. 2 4 The psychodynamic approach seems to move closer to the cognitive-behavioral orientation, but it retains an important focus on the depths of the therapeutic relationship and work on defenses as clinical tools. The nondirective approach of Carl Rogers and others can be enhanced with more directive and expressive techniques. 5 COMMON FACTORS VERSUS SPECIFIC TECHNIQUES Psychotherapy is generally effective beyond the effects of placebo controls. 6 8 On the average, 63 out of 100 patients achieve successful outcomes with systematic psychotherapy versus 38 out of 100 with placebo or minimal treatment. A good deal of change can be stimulated in a much shorter time than previously thought. McNeilly and Howard 6 indicate that about half the outpatients who undergo psychotherapy show significant improvement by the eighth session and that most cases of mild and moderate severity show good symptom change within about 26 weekly sessions. The demonstrated effect sizes are not due to effects of small sample sizes, nonrandomized designs, or bias in published versus unpublished studies. 8 However, it is difficult to conceptualize theoretically the overall equal outcomes of various techniques, schools, and formats. Mainstream theoretical approaches assume that common factors exist in all or almost all therapies and account for a considerable amount of observed change. The common factors may be the therapeutic relationship, faith in the wisdom of the sanctioned healer, and reconditioning via learning. Yet there is no good theory or precise description of such factors. The overall equal average outcomes of different therapies may also be due to insensitive outcome measures and/or small sample sizes (low power) in most studies. When neurotic symptoms are not too severe, common factors may be responsible for most of the outcome variance. However, recent empirical evidence demonstrates that, at least with more severe cases, important specific technique differences emerge in the treatment of depression, 9,10 phobias and panic, 11,12 substance abuse, 13 schizophrenia, 14 and health-related dysfunctions such as drinking, smoking, headache, compulsions, and insomnia. Active and specifically targeted interventions have on the average added something unique over and above the effect of common or unspecific factors. Furthermore, several patient characteristics seem to interact with variation in therapeutic techniques in producing differential outcomes. For example, less directive treatments appear to be superior to more directive treatments among patients with an internal locus of perceived control, whereas the opposite relationship seems to hold among patients with an external locus of perceived control. 15,16 More motivated and interpersonally better adjusted patients may do better with an exploratory dynamic technique, whereas more dependent patients do better (or equally well) with supportive technique. 17,18 Patients with a life history of high-quality interpersonal relationships tend to do better with a brief, focused dynamic treatment, whereas patients with a life history of low-quality interpersonal relationships tend to do better with a less focused psychodynamic therapy. 4,19,20 Perfectionistic patients may need long-term therapy, whereas less perfectionistic patients do relatively well with brief treatments. 21 Patients preoccupied with interpersonal relatedness may benefit mainly from the real relationship offered by a more active therapist, whereas patients preoccupied with self-definition, autonomy, and control may benefit mainly from interpretations and insight. 21 Depressed patients with an elevated level of avoidant traits may do better with cognitive therapy, whereas patients with an elevated level of obsessive traits do better with interpersonal therapy. 22 Patients low on a trait measure of antisocial personality disorder do better in self-directed verbal psychotherapies, whereas patients with more antisocial personality traits do better in more directive treatments. 23 Most of these findings are derived from post hoc exploratory analyses. Less robust findings from studies of patient-treatment matching have emerged so far. If replicated and extended, however, such findings could prove of great importance. THE PATIENT AND THE THERAPIST Psychotherapy research has demonstrated, as experienced clinicians have believed for a long time, that the patients personal characteristics are most important for the therapeutic alliance and for outcome. Suitable patients achieve successful outcome three times as often as less suitable patients. Suitable patients are generally characterized by less problematic personality traits, a good ability to establish stable mutual relationships, and a good ability to verbalize and cooperate. 24 258 J Psychother Pract Res, 8:4, Fall 1999

Høglend The evidence that therapist differences produce differences in outcome has increased substantially. 25 Skillful therapists use the techniques prescribed by the therapy method or manual, but in a flexible and competent way. They adjust their interventions according to the patient s maturity and responses. They sustain an optimistic but realistic attitude toward the patient, and they very seldom intervene in a critical, unfriendly, or unclear way. They do not avoid certain difficult issues in the therapeutic process, and they focus on important cognitive and affective material. However, it is much less clear what it is that creates a skillful therapist. The research in this area is limited and inconclusive. It is not consistently demonstrated that variables such as age, personality, profession, formal psychotherapy training, years of clinical experience, personal psychotherapy, or amount of supervision have any significant relationship to the therapist s skillfulness or the outcome of psychotherapy. Medical doctors and clinical psychologists with many years of training, personal therapy, and supervision tend to have lower attrition rates. But they do not consistently achieve better outcomes than, for example, relatively inexperienced social workers or psychiatric nurses. This area urgently needs more and better quality research documentation. PROCESS RESEARCH Significant progress is being made in studying more clearly defined process units and their relationships to outcome. More than 100 empirical findings support the position that the quality of the therapeutic alliance (i.e., the quality of the cooperation and mutual understanding between the patient and the therapist) is significantly associated with favorable outcome. The findings generally favor a cooperative attitude from the therapist versus a more directive attitude. 26 Thus, there is almost conclusive evidence that suitable patients who are open and have the ability to establish a good therapeutic alliance with a skillful therapist who conducts a clearly defined therapeutic method that is not too brief will achieve a favorable outcome. A long waiting period before treatment is clearly undesirable. OUTCOME MEASUREMENT Precise measurement of clinical phenomena continues to be relatively problematic in process and outcome research. Hundreds of measures are used once or only a few times, without adequately demonstrated reliability and validity. However, significant progress has been made in some areas for example, with the new alliance scales and measures of depression. Methodological problems have led to a bewildering array of suggestions about the design and statistical analysis of quantitative change. 27 Individual growth curve modeling, based on many repeated assessments of each patient s status over time, is currently recommended. 28 Outcome assessment to some degree must involve different perspectives (patient, therapist, independent evaluator), different dimensions (symptoms, personality, social functioning), and also mode-specific and general outcome measures. It is clear that different measures and measurement methods are consistently different in their tendencies to reflect change. Meta-analysis combining different measures cannot overcome the differences between measures. A possibly important innovation has been the renewed emphasis on clinical significance as opposed to statistical significance of results. Clinical significance can be determined by establishing a normative reference criterion based on the absence of symptoms or the status of a normal group of nonpatients. It is interesting that the more stringent categories of clinical significance are comparable to results identified earlier by simple rating scales that used categories such as cured, much improved, moderately improved, and unchanged. Under both these sets of criteria, about one-fourth to onethird of outpatient cases were considered cured at the end of psychotherapy. 1 There is a considerable interest in the long-term effects of therapy, but there is limited empirical knowledge. It had been concluded earlier that differences between treated groups and control groups are smaller at follow-up than at the end of therapy. Untreated control patients tended to catch up with treated subjects with the passage of time. 29 More recent reviews, however, tend to find that differences between treated and control patients are virtually the same at post-treatment and follow-ups. 30,31 Some studies even suggest a trend toward the continuing increase of differences during long-term follow-up periods. 32 35 For recurrent disorders, no brief psychological treatments have clearly demonstrated ability to protect against relapses after therapy. Cognitive therapy for depression may, however, have a small relapse-prevention effect. Some studies outside behavioral psychology have also begun to explore long-term J Psychother Pract Res, 8:4, Fall 1999 259

Psychotherapy Research: New Findings strategies to enhance the maintenance of treatment effects. 36 MEDICATION AND PSYCHOTHERAPY Discussions of the relative efficacy of pharmacotherapy and psychotherapy, in comparison and in combination with one another, tend to be less influenced by ideology in recent years. More clinicians in psychiatry and psychology believe that combined treatment provides improvement over the individual treatments. It generally has been difficult to demonstrate significant benefits for combined treatments in comparison with single modalities. However, exceptions exist. Supportive or educational psychotherapeutic interventions for both patient and family, when added to long-term neuroleptic treatment, significantly reduce the relapse rate over and above either treatment alone for schizophrenic patients. 37 Furthermore, exposure plus antidepressive medication does better in agoraphobia than either treatment alone. 38 Three earlier controlled studies, which evaluated the effect of psychotherapy and pharmacotherapy in long-term treatment of depression, supported the hypothesis that symptom relief, most readily produced by pharmacotherapy, rendered the patient more accessible to psychotherapy, which produced the strongest effects in social functioning and interpersonal relationships. 37 A recent mega-analysis demonstrated that combined therapy for severe depressions (Hamilton Rating Scale for Depression score 20 during 4 weeks) is superior to psychotherapy alone. 39 A large-scale maintenance study, 36 however, reported no trend toward value of combined treatment over expert pharmacotherapy alone. In the treatment of several anxiety disorders, both psychological and pharmacological interventions are on the average equally effective. However, exposure has more lasting benefits with obsessive-compulsive patients than medication alone, 11 and cognitive therapy seems superior for bulimia, compared with medication. The usefulness of combined treatments remains to be more adequately tested empirically, on larger patient samples and on inpatients. It is interesting to note that no psychotherapy treatment manual that is designed to facilitate the effects of medication or increase compliance with medication is yet published. Benzodiazepines may inhibit long-term effects of exposure therapy for agoraphobia, 40 but there are no other empirical studies to indicate that pharmacotherapy inhibits effects of psychotherapy or vice versa. As to whether nonresponders to medication do better with psychotherapy or with a combination of the two treatments, there is no systematic empirical evidence available to guide clinical decision making on this question. THERAPIES FOR SPECIFIC DISORDERS Although the American Psychological Association Task Force produced a list of empirically validated therapies (efficacious treatments) for mental disorders in 1995, such an attempt carries severe limitations. 41 There are promising psychological treatments for most mild to moderately severe adult disorders. Most of the treatments are brief cognitive or behavioral interventions. The only exceptions are supportive-expressive psychodynamic therapy for opiate dependence and interpersonal therapy (IPT) for depression. 42 However, psychotherapy interventions that are not included on the list are missing, by and large, because they have not been subjected to controlled clinical trials with such patient populations. In particular, the psychodynamic practice community has tended to place less value on clinical trial methodology. EFFECTIVENESS IN GENERAL SETTINGS The so-called effectiveness or clinical utility of the treatments that is, what happens when the treatments are exported from controlled conditions (mostly university settings) into the field is not extensively studied. IPT for depression effectively treats major depression among primary care patients when provided by mental health professionals. Approximately 70% of patients participating in full psychotherapy or pharmacotherapy protocols, but only 20% of primary physicians usualcare patients, were judged as recovered at 8-month follow-up. 43 The effects of multisystemic therapy (MST) for violent and chronic juvenile offenders have been very impressive in controlled settings, but the effectiveness of MST could not be maintained in community mental health settings. 44 The Consumer Reports study of about 3,000 patients treated by mental health professionals and family doctors may give some indications about the effectiveness of psychotherapy as it is practiced in the community. 45 Mental health professionals did significantly better than family doctors, but only when the treatment duration was more than 6 months. Long-term treatments 260 J Psychother Pract Res, 8:4, Fall 1999

Høglend ( 6 months) were associated with better outcomes. The overall average effects for treatment by mental health professionals converged roughly with meta-analyses of efficacy. 7,8 The Consumer Reports study was retrospective, the patient sample mostly self-selected, with no control groups, and outcome measures were global. It cannot answer questions about efficacy, but it strongly indicates that psychotherapy helps those who actually seek it. Cost-benefit analyses of psychotherapies have so far been too sparse and primitive to guide clinical decisions. METHODOLOGICAL PLURALISM Methodological pluralism seems to be supported by more and more people studying clinical phenomena. Journal editors and granting agencies still seem to focus on experimental and multivariate quantitative methodologies, mainly because methodological procedures and quality criteria have been difficult to establish in phenomenological and qualitative research. Designs using no-treatment control groups, placebo control treatments, or comparison with an alternative therapy cannot identify the efficiency of a specific technique or component of the treatment package. Therefore, so-called dismantling and constructive design strategies, in which single components are added, subtracted, or varied, tend to be used more often in psychotherapy research. Use of treatment manuals tends to reduce the effects of therapist differences and facilitates replication and comparison of findings across studies. However, although therapists can be trained to manual adherence, there seems to be no straightforward way of improving alliance and outcome by manual training. 4,46,47 Treatment manuals may underemphasize clinical judgment and flexibility. RESEARCH-INFORMED PSYCHOTHERAPY The question of how psychotherapy works that is, what are the specific mechanisms of change is still subject to much speculation. For example, four mechanisms of change have been described: mastery/coping, clarification of meaning/motives, problem actuation/experience of problem, and personal resource activation. 48 All four mechanisms can be activated by concrete therapeutic procedures, depending on the needs of the individual patient and his or her situation. Therapists allegiance to therapy schools and theoretical models may be blinders to integration of the many different therapeutic procedures for activation of basic change procedures (interpretation, exposure, Socratic dialogue, gestalt, support, education, family). Integration may be a challenge for the next decade of psychotherapy research. Clinical trials (for example, dismantling designs), large-scale observational studies in the field, detailed process research, and intensive studies with good qualitative methodology may all contribute. IMPLICATIONS FOR TRAINING AND CLINICAL PRACTICE Research findings are not readily absorbed into therapist communities. Practitioners may reject psychotherapy research as oversimplistic. Or they may limit their interest to only those studies that apply directly to their current practice. There is also an inherent tension between clinical judgment, creativity, and innovation on the one hand, and firm clinical guidelines based on efficacy research on the other hand. Clinical judgment can never be fully manualized. Expert practitioners may be those who are able to use manuals and technical recommendations flexibly in accordance with patient characteristics and the clinical situation. The aim of clinical guidelines should be to achieve practice that is firmly rooted in research findings but at the same time to nurture development of clinical judgment. There is no incompatibility between jazz improvisation and extensive scale practice. There should be no incompatibility between learning to use manualized techniques and development of clinical flexibility and creativity. Current research and clinical experience both suggest that training in more than one treatment modality enhances therapist expertise. Training should therefore include both exploratory psychotherapy and a more structured psychotherapy modality. Psychotherapy curricula should also include some knowledge about specifically targeted techniques for different disorders. Nonspecific (or poorly structured) psychotherapy is less effective for many psychiatric disorders. 49 Preliminary evidence indicates that manualized training in specific treatment modalities should use audiotaped or videotaped sessions and structure the learning experiences to provide specific directive feedback, with a focus on therapists thought processes. 50 J Psychother Pract Res, 8:4, Fall 1999 261

Psychotherapy Research: New Findings Knowledge based on current research is important in training psychotherapists for clinical practice. Trainees need a good deal of research-based knowledge in order to be able to educate and inform patients about psychiatric disorders and alternative available treatments; at the same time, some knowledge about patients orientation, abilities, and willingness to accept the recommended treatment techniques may be an important aspect of therapist expertise. Trainees should achieve competence in recognizing when more specialized care or long-term treatment is required. Some knowledge of long-term maintenance treatments and procedures for relapse prevention may be central in the treatment of patients with recurrent disorders, high comorbidity, or poor social functioning. Despite the limited evidence for the superiority of combined psychotherapy and pharmacotherapy treatment, combined treatment is reasonable to try with nonresponders. Therapists also need to learn how to monitor treatment delivery and patient progress. Use of a brief measure of symptoms and a measure of the therapeutic alliance may give important feedback about treatment progress. Evidence-based psychotherapy practice implies continuous exchange of information among researchers, clinicians, service providers, and third-party payers. Furthermore, consumers of psychotherapy are exposed to unbalanced and potentially misleading information about mental health treatment through the media. Providing users with research-based information about effective treatments for various disorders may influence clinical practice in favorable directions. REFERENCES 1. Bergin AE, Garfield SL: Overview, trends and future issues, in Handbook of Psychotherapy and Behavior Change, 4th edition, edited by Bergin AE, Garfield SL. New York, Wiley, 1994, pp 821 830 2. Piper WE, Azim HF, Joyce AS, et al: Transference interpretations, therapeutic alliance, and outcome in short-term individual psychotherapy. Arch Gen Psychiatry 1991; 48:946 956 3. Høglend P, Heyerdahl O, Amlo S, et al: Interpretations of the patient-therapy relationship in brief dynamic psychotherapy: effects on long-term mode-specific changes. J Psychother Pract Res 1993; 2:296 306 4. Henry WP, Strupp HH, Schacht TE, et al: Psychodynamic approaches, in Handbook of Psychotherapy and Behavior Change, 4th edition, edited by Bergin AE, Garfield SL. New York, Wiley, 1994, pp 467 508 5. Greenberg LS, Elliot RK, Lietaer G: Research on experiential psychotherapies, in Handbook of Psychotherapy and Behavior Change, 4th edition, edited by Bergin AE, Garfield SL. New York, Wiley, 1994, pp 509 539 6. McNeilly CL, Howard KJ: The effects of psychotherapy: a reevaluation based on dosage. Psychother Res 1991; 1:74 78 7. Lambert MJ, Bergin AE: The effectiveness of psychotherapy, in Handbook of Psychotherapy and Behavior Change, 4th edition, edited by Bergin AE, Garfield SL. New York, Wiley, 1994, pp 143 189 8. Lipsey MW, Wilson DB: The efficacy of psychological, educational, and behavioral treatment: confirmation from meta-analysis. Am Psychol 1993; 48:1181 1209 9. Elkin J, Shea MT, Watkins T, et al: NIMH Treatment of Depression Collaborative Research Program: general effectiveness of treatments. Arch Gen Psychiatry 1989; 46:971 983 10. Shapiro DA, Barkham M, Rees A, et al: Effects of treatment duration and severity of depression on the effectiveness of cognitive-behavioral and psychodynamic interpersonal psychotherapy. J Consult Clin Psychol 1994; 62:522 534 11. Marks JM: Fears, Phobias and Rituals. Oxford, UK, Oxford University Press, 1987 12. Barlow DH, Craske M, Cerny J, et al: Behavioral treatment of panic disorder. Behavior Therapy 1989; 20:261 282 13. Carrol KM, Rousaville BJ, Gordon LT, et al: Psychotherapy and pharmacotherapy for ambulatory cocaine abusers. Arch Gen Psychiatry 1994; 51:177 187 14. Goldstein NJ: Schizophrenia: the interaction of family and neuroleptic therapy, in Combining Pharmacotherapy and Psychotherapy in Clinical Practice, edited by Beitman BD, Klerman GL. New York, Spectrum, 1984, pp 167 186 15. Shoham-Salomon V, Avner R, Neeman R: You re changed if you do and changed if you don t: mechanisms underlying paradoxical interventions. J Consult Clin Psychol 1989; 57:590 598 16. Beutler LE: Have all won and must all have prizes? Revisiting Luborsky et al. s verdict. J Consult Clin Psychol 1991; 59:226 232 17. Horowitz MJ, Marmar C, Weiss DS, et al: Brief psychotherapy of bereavement reactions: the relationship of process to outcome. Arch Gen Psychiatry 1984; 41:438 448 18. Piper WE, Joyce AS, McCallum M, et al: Interpretive and supportive forms of psychotherapy and patient personality variables. J Consult Clin Psychol 1998; 66:558 567 19. Høglend P, Sørlie T, Heyerdahl O, et al: Brief dynamic psychotherapy: patient suitability, treatment length and outcome. J Psychother Pract Res 1993; 2:230 241 20. Høglend P, Piper WE: Focal adherence in brief dynamic psychotherapy: a comparison of findings from two independent studies. Psychotherapy Research 1995; 32:618 628 21. Blatt SJ: Contributions of psychoanalysis to the understanding and treatment of depression. J Am Psychoanal Assoc 1998; 46:723 752 22. Barber JP, Muenz LR: The role of avoidance and obsessiveness in matching patients to cognitive and interpersonal psychotherapy: empirical findings from the Treatment for Depression Collaborative Research Program. J Consult Clin Psychol 1996; 64:951 958 23. Beutler LE, Engle D, Mohr D, et al: Predictors of differential response to cognitive, experiential, and self-directed psychotherapeutic procedures. J Consult Clin Psychol 1991; 59:333 340 262 J Psychother Pract Res, 8:4, Fall 1999

Høglend 24. Garfield SL: Research on client variables in psychotherapy, in Handbook of Psychotherapy and Behavior Change, 4th edition, edited by Bergin AE, Garfield SL. New York, Wiley, 1994, pp 190 228 25. Beutler LE, Machado PPP, Neufeldt SA: Therapist variables, in Handbook of Psychotherapy and Behavior Change, 4th edition, edited by Bergin AE, Garfield SL. New York, Wiley, 1994, pp 229 269 26. Orlinsky DE, Grawe K, Parks BK: Process and outcome in psychotherapy: noch einmal, in Handbook of Psychotherapy and Behavior Change, 4th edition, edited by Bergin AE, Garfield SL. New York, Wiley, 1994, pp 270 376 27. Rogosa DR, Willett JB: Demonstrating the reliability of the difference score in the measurement of change. Journal of Educational Measurement 1983; 20:335 343 28. Gibbons DR, Hedecker D, Elkin I, et al: Some conceptual and statistical issues in analysis of longitudinal psychiatric data. Arch Gen Psychiatry 1993; 50:739 750 29. Smith ML, Glass GV, Miller TI: The Benefits of Psychotherapy. Baltimore, The Johns Hopkins University Press, 1980 30. Blanchard EB: Long-term effects of behavioral treatment of chronic headache. Behavior Therapy 1987; 18:375 385 31. Robinson LA, Berman JS, Niemeyer RA: Psychotherapy of the treatment of depression: a comprehensive review of controlled outcome research. Psychol Bull 1990; 100:30 49 32. Snyder DK, Wills RM, Grady-Fletcher A: Long-term effectiveness of behavioral versus insight-oriented marital therapy: a 4- year follow-up study. J Consult Clin Psychol 1991; 59:138 141 33. Fairburn CG, Norman PA, Welch SL, et al: A prospective study of outcome in bulimia nervosa and the long-term effects of three psychological treatments. Arch Gen Psychiatry 1995; 52:304 312 34. Borduin CM, Mann BJ, Cone LT, et al: Multisystemic treatment of serious juvenile offenders: long-term prevention of criminality and violence. J Consult Clin Psychol 1995; 63:569 578 35. Høglend P: Long-term effects of transference interpretations: comparing results from a quasi-experimental and a naturalistic long-term follow-up study of brief dynamic psychotherapy. Acta Psychiatr Scand 1996; 93:205 211 36. Frank E, Kupfer DJ, Perel JM, et al: Three-year outcomes for maintenance therapies in recurrent depression. Arch Gen Psychiatry 1990; 47:1093 1099 37. Klerman GL, Weissman MM, Markowitz JC, et al: Medication and psychotherapy, in Handbook of Psychotherapy and Behavior Change, edited by Bergin AE, Garfield SL. New York, Wiley, 1994, pp 734 782 38. Wolfe BE, Maser JD: Treatment of Panic Disorder: A Consensus Development Conference. Washington, DC, American Psychiatric Press, 1994, pp 177 197 39. Thase ME, Greenhouse JB, Frank E, et al: Treatment of major depression with psychotherapy or psychotherapy-pharmacotherapy combinations. Arch Gen Psychiatry 1997; 54:1009 1015 40. Marks IM, Swinson RP, Basoglu M, et al: Alprazolam and exposure alone, and combined in panic disorder with agoraphobia: a controlled study in London and Toronto. Br J Psychiatry 1993; 162:776 787 41. Chambless DL, Hollon SD: Defining empirically supported therapies. J Consult Clin Psychol 1998; 66:7 18 42. De Rubeis RJ, Crits-Cristoph P: Empirically supported individual and group psychological treatments for adult mental disorders. J Consult Clin Psychol 1998; 66:37 52 43. Schulberg HC, Block MR, Madonia MJ, et al: Treating major depression in primary care practice: eight-month clinical outcomes. Arch Gen Psychiatry 1996; 53:913 919 44. Henggeler SW, Melton GB, Brondino MJ, et al: Multisystemic therapy with violent and chronic juvenile offenders and their families: the role of treatment fidelity in successful dissemination. J Consult Clin Psychol 1997; 65:821 833 45. Seligman MEP: The effectiveness of psychotherapy: the Consumer Reports study. Am Psychol 1995; 59:965 974 46. Svartberg M, Stiles T: Predicting patient change from therapist competence and patient-therapist complementarity in shortterm anxiety-provoking psychotherapy. J Consult Clin Psychol 1992; 60:304 307 47. Henry WP, Strupp HH, Butler SF, et al: Effects of training in time-limited dynamic psychotherapy: changes in therapist behavior. J Consult Clin Psychol 1993; 61:434 440 48. Grawe K: Research-informed psychotherapy. Psychother Res 1997; 7:1 19 49. Roth A, Fonagy P: What Works for Whom? New York, Guilford, 1996 50. Henry WP, Schacht TE, Strupp HH, et al: Effects of training in time-limited dynamic psychotherapy: mediators of therapists responses to training. J Consult Clin Psychol 1993; 61:441 447 J Psychother Pract Res, 8:4, Fall 1999 263