Treatment Programs for Youth With Emotional and Behavioral Disorders: An Outcome Study of Two Alternate Approaches

Transcription

1 Mental Health Services Research, Vol. 4, No. 2, June 2002 ( C 2002) Treatment Programs for Youth With Emotional and Behavioral Disorders: An Outcome Study of Two Alternate Approaches Linda A. Wilmshurst 1 Youth with severe emotional and behavioral disorders (EBD) were randomly assigned for 3 months of intensive treatment to a 5-day residential program (5DR Program) or a community-based alternative, family preservation program (FP Program). Programs differed not only in method of service delivery (residential unit vs. home-based), but also in treatment philosophy (solution focused brief therapy vs. cognitive behavioral). Results confirmed high rates of comorbidity in this population for externalizing and internalizing disorders. A significant Treatment Program interaction was evident for internalizing disorders. At 1-year follow-up, significantly higher percentages of youth from the FP Program revealed a reduction of clinical symptoms for ADHD, as well as, general anxiety and depression, whereas significant proportion of youth from the 5DR Program demonstrated clinical deterioration and increased symptoms of anxiety and depression. Results have implications for future treatment of youth with EBD and suggest that greater emphasis be placed on research linking treatment to specific symptom clusters, especially highly comorbid clusters in this hard to serve population. KEY WORD: treatment programs for EBD youth. It has been estimated that one fifth of all children and adolescents in North America experience a diagnosable psychological disorder (U.S. Department of Health and Human Services [USDHSS], 1999). However, diagnosis of mental disorders in childhood is often a complicated process that requires consideration of contextual issues (family, peers, school, home, and community), developmental level and high tendencies toward comorbidy among disorders (Angold, Costello, & Erkanli, 1999). As a result, linking treatment outcomes to specific diagnostic categories has presented considerable challenge and many investigators have questioned the usefulness of categorical labels for youth who demonstrate emotional and behavioral disorders (EBD), suggesting instead that emphasis be placed on determining subgroups of troubled youth and potentially matching these 1 Department of Psychology & Philosophy, Texas Woman s University, PO Box Denton, Texas 76204; lwilmshurst@ twu.edu. subgroups to specific treatment interventions and outcomes (Kershaw & Sonuga-Barke, 1998; Rosenblatt & Furlong, 1998; Walrath, Nicherson, Crowel, & Leaf, 1998). Youth with EBD have largely been serviced by a continuum of care, progressing from the least intrusive alternative (e.g., outpatient therapy) to the most restrictive environment (e.g., residential treatment centers [RTC]). However, research concerning treatment effectiveness for youth has come primarily from analogue-type studies, whereas research concerning the efficacy of community-based outpatient treatment has been minimal (Weisz, Weiss, Han, Granger, & Morton, 1995). Studies of RTC placements have been sparse and lacking in research controls (Blotcky, Dimperio, & Gottett, 1984; USDHSS, 1999). Research concerning the efficacy of home-based treatments, such as family preservation, has also suffered historically, due to the use of a single variable analysis (out-of-home placement) and inadequate controls (Littell & Schuerman, 1995). More recent /02/ /0 C 2002 Plenum Publishing Corporation

2 86 Wilmshurst controlled studies of family preservation have produced disappointing longer term results on a number of child outcome measures (Meezan & McCroskey, 1996). However, results from studies of multisystemic therapy (MST) suggest that providing services within the community can be a successful alternative to outof-home placement for juvenile offenders (Henggeler & Borduin, 1990; Schoenwald, Borduin, & Henggeler, 1998) and an alternative to hospitalization for children and adolescents with serious behavioral and emotional problems (Schoenwald, Ward, Henggeler, & Rowland, 2000). The success of the MST approach has been attributed to several factors: the focus on multiple determinants of antisocial behavior; a model of service delivery consistent with the ecological validity inherent in a family preservation approach (treatment delivered within the natural environment); and treatment methods based on problem solving through cognitive behavioral techniques (Schoenwald et al., 1998). More recently, treatment fidelity, in therapist adherence to the MST protocol, has also been isolated as a strong mediator for treatment success (Huey, Henggeler, Brondino, & Pickerel, 2000). Although research comparing community-based treatment alternatives is sparse, according to Chamberlain and Reid, juveniles who were assigned to specialized foster care programs (SFC) were more successful in their subsequent community placements than peers assigned to RTCs or family/relatives homes (Chamberlain & Reid, 1991). Furthermore, juveniles who had been randomly assigned to SFC programs demonstrated significantly fewer arrests 1 year posttreatment than peers in group home settings (Chamberlain & Reid, 1998). Although the major trends in child welfare have pointed to an increased emphasis on family-based treatment as the alternative to out-of-home placement, a large number of children continue to be placed in RTC programs. The concept that RTCs appear to serve as an alternative to psychiatric inpatient care for many youth (Frank & Dewa, 1992) has reawakened an interest in the need for outcome data regarding RTC programs. However, a recent survey of outcome research practices in 93 RTCs for youth with EBD (Nansel et al., 1998) revealed minimal use (14%) of any type of comparison group in the research practices. Recent longitudinal findings of children s RTCs across six states suggest disappointing rates of longer term success, despite initial gains (Brown & Greenbaum, 1994). Although Asarnow, Aoki, and Elson (1996) examined outcomes and service utilization among inpatient populations, only one controlled study comparing a RTC with a community alternative has been reported. Rubenstein, Armentrout, Levin, and Herald (1978) found no difference in outcomes for youth with comparable backgrounds who were assigned to either an RTC placement or therapeutic foster care. This study was designed to address the need for a controlled research program to evaluate communitybased treatment alternatives for high-risk youth with EBD and to provide a comparison of a family-based program with a residential alternative. The study evaluated short-term (after 3 months of intensive treatment) and longer term (1 year postdischarge) treatment outcomes for youth with EBD who were randomly assigned to one of two treatment approaches: family preservation program (FP Program) or a 5-Day residential program (5DR Program). Treatment outcome goals were as follows: (a) reduce the prevalence of externalizing and internalizing disorders; (b) increase normative functioning; and (c) increase prosocial behavior. METHOD Participants Participants were 82 youth, who were randomly assigned to one of two treatment alternatives: the FP Program (48 youth: 41 males, 9 females, M age = 10.35, SD = 2.28), or the 5DR Program (34 youth: 29 males, 5 females, M age = 11.12, SD = 1.76). There were four additional families; however, these families withdrew from participation in the programs after assignment but prior to participation (two from each program), and have been excluded from the analysis. All youth were assigned to the same children s mental health agency by the Children s Services Network (CSN), a single-point access mechanism for at-risk children and youth requiring intensive services. All youth presented to CSN were rated by their case managers on the CSN Assessment of Risks and Needs, a rating scale designed by the central body to provide a rating of the youth s risk level (Low, Moderate, High, Very High) in a number of areas: prior and current offenses, family factors, education, peer relations, substance abuse, leisure/recreation, personality/ behavior, and attitude/orientation. All children who rated within the high to very high range in this assessment of risk/need were referred to the local mental health service provider for residential treatment. The local mental health service provider is a fully accredited member of Children s Mental Health

3 Treatment Programs for Youth With Behavioral Disorders 87 Ontario. The agency is a nonprofit registered charitable organization operated by a volunteer Board of Directors and offers a wide spectrum of outpatient and inpatient services. Referrals from CSN directed to the mental health agency were submitted to the Program Director responsible for Intensive Services placements. The Program Director subsequently assigned each youth, by alternate draw, to either the FP Program or the 5DR Program, based on the availability of service at the time. A description of the randomization process within this overflow design is available in the Procedures Section. At intake, the primary worker assigned to the case informed parents and youth that questionnaires would be provided for completion at three time intervals: intake, discharge and 1 year after discharge from the program. Participants were informed that confidentiality of participant information would be maintained and that information would be used exclusively to assist with program evaluation. Questionnaires were collected by volunteers who received inservice training by the Program Supervisors, regarding proper administration of data questionnaires, should additional assistance be required. Volunteers were instructed to offer financial incentives at 1-year follow-up for completion of questionnaires by parents ($20.00) and youth ($10.00). Volunteers were available by telephone or in person to assist parents to complete the forms in their home or at the agency (at the parent s request). Volunteers also collected protocols if transportation was a problem. Attrition rates varied by instrument and respondent. Parent rates for data collection across all three time periods were 84% for the Social Skills Rating System (SSRS) and 79% for the Standardized Client Information System (SCIS). Completed SCIS Parent protocols were available for 38 youth from the FP Program and 27 youth from the 5DR Program. Teacher responses were available at the following rates: posttreatment data (SCIS 54%; SSRS 58%) and 1-year follow-up (SCIS 61%; SSRS 48%). Missing teacher data was often linked to posttreatment dates, which corresponded to school breaks, or change in teachers or schools. Many teachers were reluctant to complete forms at 1 year follow-up, if they had not filled out intake forms. Youth completed only 50% of the SCIS forms at posttreatment and 1 year follow-up, although response rates for the SSRS revealed 78% return at posttreatment and 68% return for 1 year follow-up. Because of the high rates of attrition for teacher and youth data, the majority of results presented relate to parent data. Demographics Demographics for youth in the original study population (n = 82) and study sample (n = 65) are presented in Table 1. Youth categorized as the Table 1. Demographic and Pretreatment Parent Data (SCIS) for Study Youth Before and After Attrition Study sample a Study population FP Program 5DR Program Demographics (n = 82) (n = 38) (n = 27) Housing: Foster care 7% 5% 7% Extended family 3% 4% 5% Biological mother 91% 90% 88% Single mother 49% 48% 48% Biological parents 22% 19% 18% Partnered families 20% 22% 19% Income: Under $15,000 23% 20% 23% Parent education: 29% 26% 29% Below secondary level Pretreatment SCIS b parent ratings Comorbid externalizing 84% 89% 92% Comorbid internalizing 37% 39% 37% Comorbid externalizing & 68% 71% 74% internalizing symptoms a Study sample numbers after attrition, based on completed parent SCIS reports. b SCIS percentages for comorbid externalizing (conduct, oppositional, ADHD) and internalizing symptoms (general Anxiety, separation anxiety, depression) represent ratings in two or more symptom categories in excess of T score of 70.

4 88 Wilmshurst study sample represent 79% of the total population for whom all parent protocols of the SCIS were completed. There were minimal differences between the final study population and the original population across all demographic variables measured (see Table 1). The children were of predominant Caucasian backgrounds (95%), with only minor representation from other racial groups (5%). The most commonly reported annual income before taxes was $10,000 $14,999, and single parent mothers comprised over 85% of the respondents in this income category. Notably the profile of the study population and study sample were consistent with projections for families at high risk for developing conduct problems, namely: low income, low education, and single parenthood (Rae-Grant, Thomas, Offord, & Boyle, 1989). Prevalence of Clinical Symptoms Analysis of parent responses to the SCIS at intake revealed that over 80% of youth were rated as having clinical symptom levels above the 98th percentile in more than one category of externalizing behaviors (conduct, oppositional, attention deficit hyperactivity) and more than one third for comorbid internalizing behaviors (general anxiety, separation anxiety, depression). Within the study sample, parents noted over 70% of youth to have co-occurring symptoms within the clinical range for both internalizing and externalizing behaviors. Therapists/Program Clinicians Educational requirements for all therapists and clinicians in the FP Program and the 5DR Program included receipt of a college diploma in child and youth work or equivalent specialization (2 years post secondary school in an accredited program), accreditation by the child and youth Counselor Association, or a university Baccalaureate degree in social science. In addition, all therapists were required to have at least 2 years experience working within a residential setting of a children s mental health center. All clinicians were provided additional in-service and training in crisis management and other areas specific to their individual programs (e.g., cognitive behavioral methods, strategies for skill building, solution focused brief therapy). Therapists were directly administratively responsible to their Program Supervisor, who in turn was responsible to the Program Director for Intensive Services. Program Directors were provided with intensive professional development in treatment methods specific to their program (e.g., cognitive behavioral or solution focused therapy) and service delivery method (family preservation vs. residential placements). Additional clinical input on a case by case basis was available to both programs from the agency psychologist, psychiatrist, and family therapists (MA in social work or family therapy). Family Preservation Program (FP Program) Model of Service Delivery In-home service was the core component of the FP Program, which provided intensive in-home support for up to 12 hr a week over a 12-week period. The average family contact time during the 12-week period of intensive contact was hr (SD = 15.02). During this intensive phase, workers were available on call 24 hr a day, 7 days per week. After the completion of the intensive phase, on-going support was available in the form of parent support groups offered either at the agency or through a community-based program (i.e., Boys & Girls Club). The average family contact time between discharge from the intensive period to 1-year follow-up was an additional hr of contact (SD = 17.86). Treatment Approach Treatment was based on the philosophy that solutions can be found within the families through a working partnership with the family preservation worker. The program contained many elements found in the MST approach, such as flexible intervention approach; family preservation model of service delivery (in-home, time limited, and present oriented); emphasis on positive family strengths as mediators for change and use of problem-solving and cognitive behavioral and behavioral methods as strategies to enhance change. However, the FP Program was not affiliated with the MST training program, through supervision or adherence to the prescribed MST manual and differed from the MST program in several ways. Therapists in the FP Program had either a diploma in child and youth care, or bachelor s level education, compared to MST requirements of master s level therapists. PhD level professionals (psychologists) were available for purposes of individual case consultation

5 Treatment Programs for Youth With Behavioral Disorders 89 and training; however, direct supervision of FP Program therapists was conducted through formal and informal supervision meetings conducted by the Program Supervisor who was a highly experienced, bachelor s-level professional. The FP Program offered a number of treatment options to meet the needs of the family, and the focus was on developing resources within the family s own milieu. Clinicians were trained in methods of crisis intervention and family counseling/support based on a number of cognitive behavioral and behavioral skill building techniques (Barkley, 1990; D Zurilla, 1986; Feindler, 1990; Kendall & Braswell, 1993; Scott, 1989) to assist with child management and family functioning, that is, advocacy, parenting skills, problem solving, anger management. Therapists received training in cognitive behavioral methods at in-service sessions presented by the agency psychologist and regionally through attendance at workshops and conferences on cognitive behavioral methods. Training materials and resources were maintained in a professional binder that contained procedures for several cognitive behavioral strategies, such as teaching specific problem-solving strategies (define the problem; generate possible alternatives; select an alternative and monitor the results); guidelines for fostering generalization across situations; and strategies for managing and training anger control. Linkages within the community at-large and agency support programs were also available. Each child was assigned to a primary therapist who assumed responsibility for case management for the entire 3-month intensive treatment phase. Therapists offered treatment in the home and treatment was targeted to specific problems and solutions based on cognitive behavioral methods. For example, if a parent was having difficulty managing a child at breakfast, the therapist would schedule a visit at that time to observe interactions and assist the parents in defining specific problem behaviors, suggesting alternative behaviors/perceptions (reframing) and or contingency management alternatives. Intensive treatment was provided for 3 months for all families in the program. 5-Day Residential Program (5DR Program) Model of Service Delivery The residential program was housed in a residential treatment unit, licensed for nine beds to meet the needs of children and youth with EBD (6 14 years of age). Programs were highly individualized and flexible, allowing the child access to Day Treatment Schools situated on or off campus, or regular community-based schooling. Active involvement and support of the parent or guardian was considered essential and support was available through multiagency or community-based programs. The 5DR Program differed from more traditional RTCs because the program operated Monday through Friday and youth were discharged to their homes on weekends. The program offered a 3-month intensive residential placement phase (5 days a week) and a 3-month reintegration phase, after discharge, when continued child and parent contact was encouraged to assist with family reunification. Treatment Approach Services provided were based on Solution focused principles derived from Brief Therapy models of treatment (Miller, Hubble, & Duncan, 1996). The underlying premise of treatment was that the client is more vested in the treatment process if the client determines the direction and purpose of the intervention. Rather than focusing on the problem, the emphasis was on assisting the client to identify areas of change required to provide a preferred lifestyle. Clinicians worked with parents and youth through therapy sessions and homework assignments designed to provide responses to a series of solution focused questions, such as coping questions, scaling questions, future oriented questions, and competence and resource questions (Miller et al., 1996). Family sessions were optimally provided in 1-hr sessions approximately every 2 weeks while the child was in the intensive phase. All family sessions were conducted at the agency and additional family contact to discuss more immediate concerns was initiated by telephone. All youth in this study assigned to the residential program remained in residence, Monday through Friday, and went home on weekends during the intensive 3-month period. During the period of residential involvement, the average family contact time was hr. (SD = 16.96), with an additional average family contact time of hr. (SD = 16.76) for the period between discharge and 1 year postdischarge. Each youth in the 5DR Program was assigned to a primary therapist who was responsible for case management for the entire 3-month treatment phase.

6 90 Wilmshurst Comparative Treatment Summary The FP Program and the 5DR Program offered 3 months of intensive treatment, for youth provided by therapists (child care workers) with similar educational backgrounds. Youth were assigned to a primary therapist who was responsible for case management under the direction of the Program Supervisor for the FP Program or 5DR Program. Supervisors were accountable to the Director of Intensive Services. Program Supervisors conducted weekly meetings with staff to insure administrative adherence to program procedures, as well as integrity to treatment approach (cognitive behavioral vs. brief solution focused therapy). Agency support services (psychologist, psychiatrist, family therapists) were available on a client by client basis for all youth in both programs. Treatment programs differed in service delivery (in home vs. residential placement) and treatment methods (cognitive behavioral vs. brief solution focused therapy). Treatment Integrity Clinicians in each program met weekly during the intensive program phase with their respective Program Supervisor who monitored case progress and provided feedback regarding the development of treatment goals and adherence to prescribed treatment methods inherent in each of the programs. Supervisors evaluated program integrity by reporting structures (weekly case review formats, specific to treatment goals and face-to-face meetings) that were constructed to evaluate the program s adherence to cognitive behavioral (FP Program) or solution focused brief therapy (5DR Program) protocols. Case notes regarding treatment goals and strategies were written after each family/and or child session and made available for viewing by team members and supervisory personnel via access to computerized files. The structure of case note reporting was unique to each program and reflected the treatment methods employed (scaling question format for brief solution focused therapy cases in the 5DR Program; problem-solving orientation for FP Program clients). Each program also maintained a procedural manual summarizing the program vision based on therapeutic approach, administrative case reporting forms, and prescribed treatment suggestions (based on brief solution focused therapy [5DR Program] or problem solving orientation [FP Program] for commonly occurring problems. Procedure Randomization Process The randomization process was conducted within an overflow design that adhered to the following restrictions and guidelines. Referrals directed to the mental health agency were submitted to the Program Director for Intensive Services. The Program Director assigned each youth, by alternate draw, to either the FP Program or the 5DR Program, based on availability of service at the time. Within this overflow design, two preexisting conditions could override a purely random assignment: a no-wait list criteria and the fact that the 5DR Program was only licensed for nine beds. Therefore, if a child would have been assigned to the 5DR Program and the 5DR Program was at capacity, then the youth was automatically reassigned to the FP Program. Measures The data were collected during three assessment periods: prior to admission to the program (hereafter referred to as pretreatment); at the end of the 3-month intensive treatment program (hereafter referred to as posttreatment; and 1 year following the completion of treatment. Several measures were used. The Standardized Client Information System (SCIS) The SCIS was developed by the Ontario Association of Children s Mental Health (OACMHC) in consultation with the Chedoke-McMaster Child Epidemiology Unit. The SCIS scales were originally used to estimate prevalence of emotional and behavioral disorders for the Ontario Child Health Study (OCHS). A description of the methodological aspects of the survey (Boyle et al., 1987) validation procedures and report of the psychometric properties for the revised Ontario Child Health Study Scales (Boyle, Offord, Racine, et al., 1993) are well documented. The majority of scale items for the SCIS were chosen from the Child Behavior Checklist (Achenbach & Edelbrock, 1981) and are representative of diagnostic categories based on DSM criteria. Parallel forms are available for parents, teachers, and youth, 11 years of age or older. Parents and youth respond to 60 items, whereas teachers are provided with 46-item questionnaires. Respondents provide information concerning

7 Treatment Programs for Youth With Behavioral Disorders 91 severity of symptoms for externalizing behaviors of conduct, oppositional and attention deficit hyperactivity and internalizing behaviors of general anxiety, separation anxiety, and depression. In addition to subscale scores above, scores were also available for total externalizing and total internalizing symptoms. The Social Skills Rating System (SSRS) Parents, teachers, and children completed parallel forms (Gresham & Elliott, 1990) providing measures of Social Competence measured by ratings for social skills in areas of cooperation, assertion, selfcontrol, and responsibility. Teachers also provide ratings for overall Academic Competence. Across all forms and levels, the reported median coefficent alpha reliability for the SSRS was.90 for the Social Competence Scale and.95 for Academic Competence Scale. The number of items vary according to respondent and level of assessment (elementary or secondary school) and are as follows: parents (elementary 55 items, secondary 52 items); teachers (elementary 57 items, secondary 51 items); youth (elementary 34 items, secondary 39 items). Scores are presented as standard scores with a mean of 100 and standard deviation of 15. RESULTS Data Analysis Initial analysis was conducted to report descriptive statistics regarding participant demographics (see Table 1). There were no significant differences between the population sample and study sample or between youth assigned to treatment programs. Research Questions and Overall Evaluation Design Beyond descriptive statistics, two separate sets of analyses were conducted to answer four research questions. The first question was whether parent ratings on the dependent measures (SCIS and SSRS) would change over time (pretreatment, posttreatment, and 1-year follow-up). The second question was whether parent ratings on the dependent measures would be influenced by program (FP Program and 5DR Program). The third research question was whether parent ratings on the dependent measures would reveal an interaction between program and time. In order to answer the first three questions, four separate mixed ANOVAs were conducted to evaluate the main effect for time (three levels), the main effect for program (two programs), and the interaction main effects (Program Time). The dependent measures were parent ratings on four scales: Total Externalizing and Total Internalizing Scales from the SCIS and Social Competence and Behavior Problems from the SSRS. A separate set of analyses was conducted to address the fourth research question: whether program participants demonstrated changes on outcome measures that were clinically as well as statistically significant. This set of analyses was conducted using the Reliable Change Index (RCI; Jacobson & Traux, 1991). The first set of analyses addressed issues of differential treatment effects relative to types and magnitude of behavioral change, whereas the second set of analyses addressed issues of qualitative differences, that is, the measurement of changes in behavior that might signify a shift from clinical (dysfunctional) to normal levels of functioning. Analysis of Pretreatment Data Pretreatment data including demographic information and prevalence rates for severity of symptoms noted on the SCIS subscales are presented in Table 1. Initial analyses of pretreatment data was conducted using a chi-square analysis of demographic data (ages, income, & ethnicity) collected prior to treatment for youth who participated in the two programs. There were no significant differences on any demographic variables between youth assigned to the FP Program or the 5DR Program. Analysis of Teacher and Youth Data Consistent with previous research, reports from teachers (Boyle et al., 1993; Webster-Stratton, 1998), and youth self-reports did not indicate levels of severity of problems noted by parents, nor the degree of change over the three rating periods. High rates of attrition in teacher and youth data also suggested the need for cautious interpretation. Consistent with previous research concerning seriously disturbed youth, self reports on the SCIS amd SSRS indicated that youth rated themselves consistently within the normative ranges for all outcomes (Matson & Ollendick,

8 92 Wilmshurst 1988; Versi, 1995) with little variability across time of rating. SCIS Results Two separate mixed ANOVAs were conducted to evaluate the outcomes of Total Externalizing and Total Internalizing. The design was a 2 (treatment groups) 3 (time) ANOVA with repeated measures on the factor of time and total externalizing and total internalizing as the dependent measures. The means and standard deviations for the outcome measures are presented in Table 2. Results of the mixed ANOVA for the variable of total externalizing revealed a significant main effect for time, F(2, 62) = 28.67, p =.001. The main effects for treatment groups, F(1, 62) = 0.07, p =.78, and Treatment Group Time, F(2, 62) = 0.67, p =.51, were not significant. Paired-samples t tests were computed to assess differences in ratings between time periods for the above main effect, using Holm s sequential Bonferroni approach to control for Type 1 error. Difference in mean ratings for total externalizing noted significant declines in ratings between pretreatment and posttreatment, t(64) = 6.72, p =.001, and pretreatment and 1-year follow-up, t(64) = 7.26, p =.001. Results of the mixed ANOVA analysis for the outcome of total internalizing revealed a significant Time Program interaction, F(2, 62) = 3.92, p =.025. Analysis of the main effects for time, F(2, 62) = 3.06, p =.054, and treatment program, F(1, 62) = 0.29, p =.59, were not significant. Paired-samples t tests were conducted to follow up the significant interaction of Treatment Program Time, controlling for Type 1 error rate across these tests using Holm s sequential Bonferroni approach. Results revealed that only youth who had been involved in the FP Program noted a significant reduction in total internalizing scores when pretreatment levels were compared to levels obtained at 1-year follow up, t(37) = 4.07, p =.001. Youth who had attended the 5DR Program did not show improved scores and in fact symptom levels for total internalizing were noted to increase between pretreatment and 1-year follow-up (see Table 2). Teacher trends for total internalizing scores supported parent ratings: teachers noted an increase in internalizing symptoms from pretreatment (n = 9, M = 59.42, SD = 13.13) to 1- year follow-up (M = 66, SD = 8.2) for youth from the 5DR Program and symptom reduction from pretreatment (n = 11, M = 68.09, SD = 12.87) to 1- year follow-up (M = 62.18, SD = 6.3) for youth for the FP Program. SSRS Results Two separate mixed ANOVAs were conducted to evaluate the outcomes of Social Competence and Behavior Problems. The design was a 2 (treatment groups) 3 (time) ANOVA with repeated measures on the factor of time, and social competence and behavior problems as the dependent measures. Results of the mixed ANOVA for the variable of social competence revealed a significant main effect Table 2. Parent Ratings on Outcomes for the Standardized Client Information System (SCIS) and Social Skills Rating System (SSRS) Across Three Time Periods Pretreatment Posttreatment 1-year follow-up Outcome measure M SD M SD M SD Total Externalizing (SCIS) a FP Program (n = 38) DR Program (n = 27) Total Internalizing (SCIS) FP Program (n = 38) DR Program (n = 27) Social Competence (SSRS) b FP Program (n = 39) DR Program (n = 30) Behavior Problems (SSRS) FP Program (n = 39) DR Program (n = 30) Note. Means represent pairwise comparisons across rating periods. a SCIS: M = 50, SD = 10, greater scores suggest greater pathology. b SSRS: M = 100, SD = 15.

9 Treatment Programs for Youth With Behavioral Disorders 93 for time, F(2, 66) = 11.61, p =.001. The main effects for treatment groups, F(1, 66) = 0.07, p =.11, and Treatment Group Time, F(2, 66) = 0.14, p =.86, were not significant. Paired-samples t tests were computed to assess differences in ratings between time periods for the above main effect, using Holm s sequential Bonferroni approach to control for Type 1 error. Parent social competence ratings revealed significant increases in ratings between pretreatment and posttreatment, t(68) = 4.29, p =.001, and pretreatment and 1-year follow-up, t(68) = 4.54, p =.001. Results of the mixed ANOVA for the variable of behavior problems revealed a significant main effect for time, F(2, 66) = 24.89, p =.001. The main effects for treatment groups, F(1, 66) = 0.06, p =.81, and Treatment Group Time, F(2, 66) = 0.32, p =.72, were not significant. Paired-samples t tests were computed to assess differences in ratings between time periods for the above main effect, using Holm s sequential Bonferroni approach to control for Type 1 error. Ratings for behavior problems indicated significant decrease between pretreatment and posttreatment, t(68) = 5.84, p =.001, and pretreatment and 1-year follow-up, t(68) = 6.83, p =.001. Means and standard deviations for parent ratings of social competence and behavior problems are available in Table 2. Analysis of Therapeutic Change: Clinical Significance Although the above analysis provided important information concerning statistically significant short-term and longer-term treatment effects involving group means, the inherent limitation in such an approach is the lack of information provided concerning whether program participants demonstrated changes that were clinically significant (Jacobson & Truax, 1991). In order to determine whether changes were clinically significant, outcomes from the SCIS subscales were evaluated in relation to two criteria suggested by Jacobson, Roberts, Berns, and McGlinchey (1999). First, the RCI was calculated (Jacobson & Traux, 1991) to determine whether the degree of change was statistically reliable (e.g., exceeded a margin of measurement error). All change scores meeting the first criterion were further evaluated to determine whether symptom reduction would result in movement from a dysfunctional/clinical to normal range of functioning. A cutoff point of 2 SDs was selected (T score in excess of 70) to represent the clinical cutoff. The use of both criteria allowed for the partitioning of all change scores noting significant clinical positive change (RCI) into categories of recovered (movement from clinical to normal range) or improved, but not recovered (clinically significant change, but not within normal limits). It was also possible to determine the percentage of participants who demonstrated significant symptom increases or clinically significant deterioration (RCD). Given the above analysis, it was possible to determine percentages of youth evidencing clinically significant symptom reduction or deterioration on the subscales of the SCIS. Results of the RCI analysis revealed significant between program differences. The percentage of youth from the FP Program who demonstrated significant symptom reduction for attention deficit hyperactivity disorder at posttreatment (29%) and 1- year follow-up (63%) was significantly greater than youth from the 5DR Program at posttreatment (11%) X 2 (1, N = 65) = , p =.01, and 1-year followup (22%), X 2 (1, N = 65) = 34.39, p = 01. Significant between program differences were also noted in the long-term reduction of Internalizing symptoms for general anxiety X 2 (1, N = 65) = 16.61, p =.001, and depression X 2 (1, N = 65) = 7.46, p =.01. Youth from the FP Program noted a 24% reduction in symptoms of general anxiety, compared with a 3% reduction for 5DR Youth, whereas scores at the clinical level for depression were reduced by 26% for the FP youth compared with 11% for the 5DR Youth. Comparisons of the percentage of youth who noted significant symptom increases also revealed significant between group findings. A significantly greater percentage of youth who had attended the 5DR Program noted symptom increases between pretreatment and 1-year follow-up for all internalizing disorders, relative to youth who had attended the FP Program. A significantly greater percentage of youth from the 5DR Program noted symptom increases in areas of general anxiety, X 2 (1, N = 65) = 10.51, p =.01, separation anxiety, X 2 (1, N = 65) = 6.66, p =.01, and depression, X 2 (1, N = 65) = 11.64, p =.001. DISCUSSION This study compared treatment outcomes (at discharge and 1 year follow-up) for youth with severe EBD who had been randomly assigned to a FP Program or a 5DR Program. Both treatment programs involved 3 months of intensive treatment; however, programs differed in method of service delivery (home-based vs. residential-component) and

10 94 Wilmshurst treatment philosophy (cognitive behavioral vs. brief solution-focused therapy). The study sample was representative of youth with severe EBD, evident in comorbidity of externalizing and internalizing symptoms (McConaughy & Skiba, 1993; Zoccolillio, 1992) and at-risk family characteristics: low income, low education, and single parenthood (Rae-Grant et al., 1989). Although parent ratings (SCIS & SSRS) revealed a significant reduction in overall externalizing symptoms, behavior problems, and increase in prosocial behaviors for all children who had participated in treatment, these results must be interpreted with caution. Initial parent ratings were often at the extreme ranges of symptom endorsement (total externalizing scores in excess of the 98th percentile; social competence scores at the 4th percentile), and therefore, it was not possible to rule out whether subsequent ratings may have adjusted in response to a regression toward the mean. Given that regression effects should operate equally for both treatment groups, emphasis was directed toward evaluation of between group differences. Differential treatment effects were noted for several outcomes. Youth from the FP Program noted an incremental decrease in total internalizing symptoms across the rating periods, whereas youth who had been involved with the 5DR Program noted the reverse. More specifically, youth from the FP Program noted significant longer-term reduction of symptoms of ADHD, general anxiety, and depression, whereas a significantly greater percentage of youth from the 5DR Program revealed clinical deterioration in increased symptom presentation for all internalizing symptom clusters (general anxiety, separation anxiety, and depression). One possible explanation for the success of the FP Program can be found in previous research support for the use of cognitive or behavioral methods in the treatment of ADHD (e.g., parent training in contingency management: Barkley, 1990, 1997) as well as in the reduction of symptoms of anxiety (Kendall, 1994) and depression (Stark, Reynolds & Kaslow, 1987). Another possible comparison is the success of the MST approach, which also used behavioral methods and a family preservation model of service delivery (Schoenwald et al., 1998). In this study, there was a clear delineation between the focus of service delivery: The FP Program focused on delivery of service within the home, whereas the 5DR Program involved removal of the child from the home. In addition, there was almost twice the family contact time provided by the FP Program compared to the 5DR Program. All of these variables may have contributed to successful symptom reduction for children who had participated in the FP Program. However, symptom increases for youth who had attended the 5DR Program are more difficult to substantiate. Perhaps, one possible explanation for deterioration in youth who had attended the 5DR Program was that, unlike the FP Program, the residential intervention involved a peer group component, because youth were living with troubled peers for 3 months. There is research support to suggest that a risk of iatrogenic effects due to peer aggregation for high risk youth (Dishion, McCord, & Poulin, 1999). However, present results noted deterioration for internalizing symptoms only. One can also speculate that for some youth, removal from home may have served to exacerbate existing levels of anxiety, resulting in excessive worry about their future, concern about past behavior, and a heightened awareness of the potential of removal again, at some point in the future. Prior to discussing the implications of the results, it is necessary to caution the reader of study limitations in the following areas. The relatively small sample size (SCIS: n = 65), although common in treatment samples for severe clinical populations (Rosenblatt & Furlong, 1998), may have limited the statistical power in analyses. Whether additional effects were suppressed due to small sample size is unclear. In addition, it is also not possible to determine the degree to which smaller numbers of youth in the 5DR Program may have contributed to decreased ability to detect additional significant findings. A very small number of female participants, although consistent with ratios cited in prevalence data (Wolraich, 1996), also negated the interpretation of the impact of gender on treatment outcomes. Despite attempts to use a multirater approach, ultimately information was obtained exclusively from parent ratings, due to high rates of attrition for teacher and youth reports. Initial parent ratings were in the extreme, possibly overestimated to obtain treatment; however, given initial levels, a regression toward the mean for subsequent ratings could not be ruled out. The study would have benefitted from the inclusion of diagnostic interviews to provide further objective analysis of symptom presentation and possible diagnostic information. A third limitation is that despite numerous attempts to obtain information regarding the relative cost of providing treatment by either method (FP Program compared to 5DR Program), it was not possible to obtain this information.

11 Treatment Programs for Youth With Behavioral Disorders 95 In summary, the study addressed the need for a controlled research program to evaluate communitybased treatment alternatives for high risk youth with EBD. A major goal of the study was to provide research support for the efficacy of real world treatment in producing longer lasting behavioral change. The study demonstrated that there was a significant difference between programs, and the FP Program was superior to the 5DR Program in the reduction of symptoms for total internalizing, ADHD, generalized anxiety, and depression. Furthermore, these gains were maintained, 1 year posttreatment. Because the two programs differed in method of service delivery (in home vs. residential) and treatment methods (cognitive behavioral vs. brief solution focused therapy), it is possible that treatment method alone (cognitive behavioral) may have been instrumental in producing the differential outcome effects. However, evidence that a significant percentage of youth from the 5DR Program demonstrated increased symptoms for internalizing (anxiety and depression) 1 year posttreatment suggests the need for more intensive research efforts into possible iatrogenic effects of removal out of the home for some children. Implications of these results become even more concerning when one considers that the residential alternative investigated in the current study was a relatively short-term (3 months) and modified version (5 day vs. full-time) of more traditional residential placements. Given the increasing evidence for comorbidity of externalizing and internalizing disorders for youth with EBD, it is very important for future research to recognize the complex nature of this population by including outcome measures for internalizing as well as externalizing disorders. This study highlights the importance of evaluating community-based treatment alternatives and demonstrates the importance of linking treatment effectiveness to symptom presentations. The need for increased research emphasis in these areas cannot be overemphasized (USDHHS, 1999). REFERENCES Achenbach, T. M., & Edelbrock, C. (1981). Behavioral problems and competencies reported by parents of normal and disturbed children aged 4 through 16. Monographs of the Society for Research in Child Development, 46(4, Serial No. 188). Angold, A. E., Costello, E. J., & Erkanli, A. (1999). Comorbidity. Journal of Child Psychology and Psychiatry, 40, Asarnow, J. R., Aoki, W., & Elson, S. (1996). Children in residential treatment: A follow-up study. Journal of Clinical Child Psychology, 25, Barkley, R. A. (1990). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment. New York: Guilford. Barkley, R. A. (1977). Defiant children: A clinician s manual for assessment and parent training (2nd ed.). New York: Guilford. Blotcky, M. J., Dimperio, T. L., & Gossett, J. T. (1984). Follow-up of children treated in psychiatric hospitals. American Journal of Psychiatry, 141, Boyle, M. H., Offord, D. R., Hoffman, H. G., Catlin, G. P., Byles, J. A., Cadman, D. T., et al. (1987). Ontario child health study: I. Methodology. Archives of General Psychiatry, 44, Boyle, M. H., Offord, D. R., Racine, Y., Fleming, J. E., Szatmari, P., & Sanford, M. (1993). Evaluation of the Revised Ontario Health Study Scales. Journal of Child Psychiatry, 34(2), Brown, E. C., & Greenbaum, R. (1994, March). Reinstitutionalization after posttreatment from residential mental health facilities: An example of competing-risks survival analysis. Paper presented at the 7th annual Florida Mental Health Institute Research Conference. Chamberlain, P., & Reid, J. (1991). Using a specialized foster care community treatment model for children and adolescents leaving the state mental hospital. Journal of Community Psychology, 19, Chamberlain, P., & Reid, J. (1998). Comparison of two community alternatives to incarceration for chronic juvenile offenders. Journal of Consulting and Clinical Psychology, 66, Dishion, T. J., McCord, J., & Poulin, F. (1999). When interventions harm: Peer groups and problem behavior. American Psychologist, 54, D Zurilla, T. J. (1986). Problem-solving therapy: A social competence approach to clinical intervention. New York: Springer. Feindler, E. (1990). Cognitive strategies in anger control interventions. In P. C. Kendall (Ed.), Child and adolescent behavior therapy: Cognitive behavioral procedures. New York: Guilford. Frank, R. G., & Dewa, C. S. (1992). Insurance, system structure and the use of mental health services by children and adolescents. Clinical Psychology Review, 12, Gresham, F. M., & Elliott, S. N. (1990). Social skills rating system manual. Circle Pines, MN: American Guidance Service. Henggeler, S. W., & Borduin, C. M. (1990). Family therapy and beyond: A multisystemic approach to treating the behavior problems of children and adolescents. Pacific Grove, CA: Brooks/Cole. Huey, S., Jr., Henggeler, S. W., Brondino, M. J., & Pickrel, S. G. (2000). Mechanisms of change in multisystemic therapy: Reducing delinquent behavior through therapist adherence and improved family and peer functioning. Journal of Consulting and Clinical Psychology, 68, Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, Jacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchey, J. B. (1999). Methods for determining the clinical significance of treatment effects: Description, application, and alternatives. Journal of Consulting and Clinical Psychology, 67, Kendall, P. C. (1994). Treating anxiety disorders in children: Results of a randomized clinical trial. Journal of Consulting and Clinical Psychology, 62, Kendall, P. C., & Braswell, L. (1993). Cognitive behavioral therapy for impulsive children (2nd ed.). New York: Guilford. Kershaw, P., & Songua-Barke, E. (1998). Emotional and behavioral difficulties: Is this a useful category? The implications of clustering and comorbidity, the relevance of a taxonomic approach. Educational and Child Psychology, 15, Littell, J. H., & Schuerman, J. R. (1995). A synthesis of research on family preservation and family reunification programs. Part of the National Evaluation of Family Preservation Services for the Office of the Assistant Secretary for Planning

12 96 Wilmshurst and Evaluation, Department of Health and Human Services. Westat, Inc., in association with James Bell Associates, and the Chaplin Hall Center for Children at the University of Chicago. Available from Matson, J. L., & Ollendick, T. H. (1988). Enhancing children s social skills: Assessment and training. Elmsford, NY: Pergamon. McConaughy, S. H., & Skiba, R. J. (1993). Comorbidity of externalizing and internalizing problems. School Psychology Review, Meezan, W., & McCroskey, J. (1996). Improving family functioning through family preservation services: Results of the Los Angeles experiment. Family Preservation Journal, 46, Miller, S. D., Hubble, M. A., & Duncan, B. L. (1996). Handbook of solution focused brief therapy. San Francisco: Jossey-Bass. Nansel, T. R., Raines, S., Jackson, D. L., Teal, C. R., Force, R. C., Klingsporn, M., et al. (1998). A survey of residential treatment centers outcome research practices. Residential Treatment for Children and Youth, 15, Rae-Grant, N., Thomas, H. B., Offord, D. R., & Boyle, M. (1989). Risk, protective factors and the prevalence for behavioral and emotional disorders in children and adolescents. American Academy of Child and Adolescent Psychiatry, Rosenblatt, J. A., & Furlong, M. J. (1998). Outcomes in a system of care for youths with emotional and behavioral disorders: An examination of differential change across clinical profile. Journal of Child and Family Studies, 7, Rubenstein, J. S., Armentrout, J. A., Levin, S., & Herald, D. (1978). The parent-therapist program: Alternate care for emotionally disturbed children. American Journal of Orthopsychiatry, 48, Schoenwald, S. K., Borduin, C. M., & Henggeler, S. W. (1998). Multisystemic therapy: Changing the natural and service ecologies of adolescents and families. In M. E. Epstein, K. Kutash, & A. Duchnowski (Eds.), Outcomes for children and youth with behavioral disorders and their families. Austin, TX: ProEd. Schoenwald, S. K., Ward, D. M., Henggeler, S. W., & Rowland, M. D. (2000). Multisystemic therapy versus hospitalization for crisis stabilization of youth: Placement outcomes 4 months postreferral. Mental Health Services Research, 2, Scott, M. (1989). A cognitive behavioral approach to client s problems. New York: Routledge. Stark, K. D., Reynolds, W. M., & Kaslow, N. J. (1987). A comparison of the relative efficacy of self-control therapy and a behavioral problem-solving therapy for depression in children. Journal of Abnormal Child Psychology, 15, U.S. Department of Health and Human Services. (1999). Mental health: A report of the Surgeon General (Chap. 3). Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institute of Mental Health. Versi, M. (1995). Differential effects of cognitive behavior modifications on seriously emotionally disturbed adolescents exhibiting internalizing or externalizing problems. Journal of Child and Family Studies, 4, Walrath, C., Nicherson, K., Crowel, R., & Leaf, P. (1998). Serving children with serious emotional disturbance in a system of care. Do mental health and non-mental health agency referrals look the same? Journal of Emotional and Behavioral Disorders, 6, Webster-Stratton, C. (1998). Preventing conduct problems in head start children: Strengthening parenting competencies. Journal of Consulting and Clinical Psychology, 66, Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995). Effects of psychotherapy with children and adolescents revisited: A meta-analysis of treatment outcome studies. Psychological Bulletin, 117, Wolraich, M. (Ed.). (1996). The classification of child and adolescent mental diagnoses in primary care: Diagnostic and statistical manual for primary care, child and adolescent version. Elk Grove Village, IL: American Academy of Pediatrics. Zoccolillo, M. (1992). Co-occurrence of conduct disorder and its adult outcomes with depressive and anxiety disorders: A review. Journal of the American Academy of Child and Adolescent Psychiatry, 32,