School psychologists, clinical psychologists, neuropsychologists,

Diagnostic Group Differences in Parent and Teacher Ratings on the BRIEF and Conners Scales Journal of Attention Disorders Volume 11 Number 3 November 2007 398-406 2007 Sage Publications 10.1177/1087054707299399 http://jad.sagepub.com hosted at http://online.sagepub.com Jeremy R. Sullivan University of Texas at San Antonio Cynthia A. Riccio Texas A&M University, College Station Objective: Behavioral rating scales are common instruments used in evaluations of ADHD and executive function. It is important to explore how different diagnostic groups perform on these measures, as this information can be used to provide criterion-related validity evidence for the measures. Method: Data from 92 children and adolescents were used to examine differences among participants in a No Diagnosis group, ADHD group, and Other Clinical group in terms of parent and teacher ratings on the Behavior Rating Inventory of Executive Function (BRIEF) and Conners Rating Scales Revised Short Form. Results: Participants in the ADHD and Other Clinical groups generally received similar scores on the scales, and both groups were generally rated higher in ADHD characteristics and executive dysfunction than were participants in the No Diagnosis group. Conclusion: Although the measures were successful at distinguishing clinical from nonclinical participants, their ability to distinguish among different clinical groups deserves further investigation. (J. of Att. Dis. 2007; 11(3) 398-406) Keywords: ADHD; executive function; assessment School psychologists, clinical psychologists, neuropsychologists, and physicians often are called on for assistance with the diagnosis of attention-deficit/hyperactivity disorder (ADHD) in children and adolescents, as well as for making recommendations for interventions and modifications to be implemented in educational settings. Thus, measures of ADHD characteristics are important tools for professionals as they conduct evaluations and design interventions for individuals with ADHD, and it is useful to know how different diagnostic groups (e.g., individuals with ADHD, individuals with other clinical diagnoses, individuals with no diagnosis) perform on these measures (Forbes, 2001). Given the role that executive dysfunction plays in the manifestation of ADHD (Barkley, 1997; Gioia, Isquith, Kenworthy, & Barton, 2002; Mahone et al., 2002), measures of executive function also are potentially useful in the assessment and diagnosis of ADHD. Executive function represents a complex construct that includes multiple abilities such as working memory, planning, emotional self-regulation, attention, organization, motivation, motor control, and goal-directed behaviors (Barkley, 1997; Powell & Voeller, 2004). Naglieri (2005) and Naglieri and Das (2005) recently summarized research using the Cognitive Assessment System (CAS; Naglieri & Das, 1997), which supports the notion that children with ADHD demonstrate deficits in executive function. The CAS is based on the Planning, Attention, Simultaneous, and Successive (PASS) theory of intelligence and purports to assess constructs related to executive function. Based on the summarized research, children and adolescents who meet diagnostic criteria for ADHD tend to score lower on the Planning scale of the CAS than children and adolescents who do Authors Note: A preliminary version of this article was pre sented at the annual meeting of the American Psychological Association, August 2006, New Orleans, LA. Address correspondence to Jeremy R. Sullivan, University of Texas at San Antonio, Department of Counseling, Educational Psychology, and Adult & Higher Education, 501 West Durango Blvd., San Antonio, TX 78207-4415; phone: 210-458-2408; fax: 210-458-2605; e-mail: jeremy.sullivan@utsa.edu. 398

Sullivan, Riccio / BRIEF and Conners Scales 399 not meet the criteria. This pattern makes sense when we consider that the process of planning includes the generation, evaluation, and execution of a plan, as well as self-monitoring and impulse control (Naglieri & Das, 2005, p. 124), and these are among the executive deficits implicated in ADHD. Incorporating behavioral observations and normative behavioral ratings into comprehensive evaluations of ADHD has been advocated by numerous experts in the field (e.g., Barkley, 1998; DuPaul & Stoner, 2003). As a result, parent- and teacher-completed behavior rating scales such as the Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000) and Conners Rating Scales Revised Short Form (Conners, 1997) are increasingly being used as measures of observable characteristics of ADHD and executive dysfunction. However, little is known with regard to diagnostic group differences in parent and teacher ratings on these scales. The purpose of this study was to examine differences among diagnostic groups in terms of the scores obtained on these measures using data from a clinical sample of children and adolescents. It is hoped that our results may be used to inform school psychologists and other professionals as they select and interpret the measures they use in assessing ADHD and executive dysfunction. Participants Method Participants in this study included 92 children and adolescents who were recruited for a Memory, Attention, and Planning Study at a large university in the southwest. Participants for the larger study were recruited with announcements distributed to local physicians, schools, bulletin boards, a counseling center, and the newspaper. The announcement indicated that the investigation focused on memory, attention, and planning/problem solving, with no direct mention of ADHD. Criteria for inclusion in the larger study were as follows: (a) Full Scale IQ greater than or equal to 80 on the Wechsler Intelligence Scale for Children Third Edition (WISC-III; Wechsler, 1991), (b) ability to speak and read English, (c) no history of severe head injury, and (d) no previous diagnosis of schizophrenia. For purposes of this study, following a comprehensive psychological evaluation, participants were divided into three groups: those who met Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994) criteria for ADHD (n = 41), those who met DSM-IV criteria for some other clinical disorder (n = 25), and those who received no diagnosis (n = 26). Of the participants in the Other Clinical group, the diagnoses included learning disabilities, adjustment disorders, mood disorders, substance use disorders, and conduct and oppositional defiant disorders. Within the ADHD group, 14 participants were classified as ADHD Predominantly Inattentive Type, and 27 were classified as ADHD Combined Type. None of the participants were classified as ADHD Predominantly Hyperactive- /Impulsive Type. Our sample of children and adolescents ranged in age from 9 to 15 years (M = 11.32, SD = 1.99); 62 (67%) were male and 30 (33%) were female. With regard to ethnicity, for the total sample, 74 (80%) of the participants were White, 10 (11%) were African American, 7 (8%) were Hispanic, and 1 (1%) was Asian. Detailed demographic information for the three groups is provided in Table 1. The sample used in this study included participants used in several other investigations. For example, Sullivan and Riccio (2006) used data from these participants to examine the relationship between the Frontal Lobe/Executive Control Scale of the Behavior Assessment System for Children (BASC; Reynolds & Kamphaus, 1992) and the parent forms of the BRIEF and Conners rating scales. In addition, Jarratt, Riccio, and Siekierski (2005) used data from these participants to examine relationships between scales on the BASC and BRIEF. Our study is presented separately because it addresses research questions that are distinct from those examined in the previous studies. Instruments BRIEF. The BRIEF Parent Form and BRIEF Teacher Form are behavioral rating scales designed to measure executive function in children and adolescents aged 5 to 18. Both forms include 86 items, and both provide scores on eight clinical scales and three broad indexes. From the manual (Gioia et al., 2000), the eight clinical scales on the BRIEF include Inhibit (impulse control and ability to inhibit behaviors), Shift (flexibility and ability to transition appropriately), Emotional Control (ability to control emotional responses), Initiate (ability to initiate tasks or activities appropriately), Working Memory (task persistence and ability to retain information while completing a task), Plan/Organize (ability to set goals and develop a plan of action based on anticipation of future circumstances), Organization of Materials (ability to keep materials and workspaces organized), and Monitor (ability to assess performance and monitor behaviors). These scales combine to form the broad indexes of Behavioral Regulation (sum of raw scores for Inhibit, Shift, and Emotional Control) and Metacognition (sum of raw scores for Initiate, Working Memory, Plan/Organize, Organization of Materials, and Monitor), in addition to an overall score called Global Executive Composite

400 Journal of Attention Disorders Table 1 Demographic Information for the Sample by Group Other No Diagnosis ADHD Clinical (n = 26) (n = 41) (n = 25) Gender Male 11 35 16 Female 15 6 9 Ethnicity White 19 34 21 African American 3 4 3 Hispanic 3 3 1 Asian 1 0 0 History of grade 1 10 6 retention History of special 1 10 5 education History of medication 2 30 7 Mean age in years 11.2 (2.2) 11.4 (2.0) 11.3 (1.9) Mean parent 15.8 (2.4) 14.8 (2.3) 14.6 (2.4) education level Mean Full Scale IQ 109.4 (14.2) 100.3 (10.0) 98.2 (11.7) Note: ADHD = Attention-Deficit/Hyperactivity Disorder. Numbers in parentheses represent standard deviations. (sum of raw scores for all eight clinical scales). Thus, the individual scales and broad indexes of the BRIEF attempt to cover the multiple components of executive function (e.g., cognitive, behavioral, emotional) and the multiple potential sources of deficit associated with executive dysfunction. On the BRIEF, raw scores on each of the scales and indexes are converted to T-scores with a mean of 50 and a standard deviation of 10, and higher obtained T-scores are indicative of a higher degree of dysfunction. The BRIEF manual provides evidence for the reliability, validity, and diagnostic utility of the scales (Gioia et al., 2000), and the BRIEF has received favorable reviews in the literature (e.g., Baron, 2000; Pizzitola, 2002). The BRIEF Parent Form normative data were based on 1,419 children and adolescents within the following categories (percentages are rounded and therefore approximate): 57% female, 43% male, 81% White, 12% African American, 3% Hispanic, 4% Asian, and 1% Native American/Eskimo. Socioeconomic status (SES) categories were represented as follows: 3% upper class, 22% upper middle, 36% middle, 32% lower middle, and 6% lower. The BRIEF Teacher Form normative data were based on 720 children and adolescents within the following categories: 56% female, 44% male, 72% White, 14% African American, 4% Hispanic, 6% Asian, and 1% Native American/Eskimo. Representation of SES categories was similar to the representation observed for the parent form. A limitation of the normative data is that all data were collected in Maryland. Internal consistency coefficients (using Cronbach s alpha) for scales on the parent and teacher forms ranged from.80 to.98 using data from the normative sample. Strauss, Sherman, and Spreen (2006) noted that testretest reliability coefficients were above.80 for scales on the teacher form and ranged from.70 to.90 on the parent form scales. The BRIEF has been used as a measure of executive function in research studies involving numerous disorders, such as spina bifida with hydrocephalus (Burmeister et al., 2005), traumatic brain injury (Mangeot, Armstrong, Colvin, Yeates, & Taylor, 2002), and bipolar disorder (Shear, DelBello, Rosenberg, & Strakowski, 2002). Previous research also has employed clinical samples to explore the relations of the BRIEF scales with other rating scales and generally has found that the BRIEF scales were correlated with similar scales on other measures. For example, Mahone et al. (2002) found that the five BRIEF Parent Form scales they analyzed were highly and significantly correlated with scores on similar scales, such as the Attention Problems scale on the Child Behavior Checklist Parent Report Form (Achenbach, 1991), items assessing inattention and hyperactivity-impulsivity on the ADHD Rating Scale IV Home Version (DuPaul, Power, Anastopoulos, & Reid, 1998), and the ADHD scale on the parent form of the Diagnostic Interview for Children and Adolescents Fourth Edition (Reich, Welner, & Herjanic, 1997). Similarly, Jarratt et al. (2005) found that many scales on the parent and teacher forms of the BRIEF were significantly correlated with similar scales on the parent and teacher forms of the BASC, leading these authors to conclude that the instruments were measuring somewhat similar constructs. Finally, Gioia et al. (2002) used the BRIEF Parent Form to explore profiles of executive function among different diagnostic groups, including children with traumatic brain injury, reading disabilities, autistic spectrum disorders, and ADHD. The authors found that children in the autistic spectrum disorders and ADHD groups showed the highest elevations in scale scores, and children in the reading disabilities and severe traumatic brain injury groups scored significantly higher than children in the moderate traumatic brain injury and control groups. In addition, the specific profiles of BRIEF scale elevations differed for each of the diagnostic groups, and these profile differences generally were consistent with the authors hypotheses and expectations. Conners Rating Scales Revised Short Form. The Conners Parent Rating Scales (CPRS) Short Form is a

Sullivan, Riccio / BRIEF and Conners Scales 401 27-item rating scale completed by parents to assess characteristics of ADHD and oppositional behaviors in children and adolescents aged 3 to 17. The Conners Teacher Rating Scales (CTRS) Short Form is the teachercompleted version and includes 28 items. Both of the short forms are abbreviated versions of the Conners Rating Scales Revised (Conners, 1997) and therefore contain fewer scales than the long forms. From the manual (Conners, 1997), the four scales on the CPRS Short Form and CTRS Short Form include Oppositional (tendency to break rules, have difficulty with authority, and become easily annoyed or angered), Cognitive Problems/ Inattention (difficulties related to inattention, disorganization, poor task completion, and academic struggles), Hyperactivity (restlessness and impulsivity), and ADHD Index (used to identify children and adolescents who are at risk for a diagnosis of ADHD). As with the BRIEF, scores on the Conners scales are converted to T-scores, and higher obtained T-scores represent a higher degree of pathology or dysfunction. Evidence for the short forms internal consistency, test-retest reliability, and factorial validity is provided in the manual (Conners, 1997), and the psychometric characteristics of the short forms appear to be similar to those of the long forms (Collett, Ohan, & Myers, 2003). The normative sample for the CPRS Short Form included 2,426 children and adolescents (male = 1,220; female = 1,206). The ethnicity categories of the parents or guardians providing ratings were as follows (percentages are rounded and therefore approximate): 84% White, 4% African American, 4% Hispanic, 2% Asian, 1% Native American, and 5% Other or No Response. The normative sample for the CTRS Short Form included 1,897 children and adolescents (male = 945; female = 952). The ethnicity categories of the students being rated by their teachers were as follows: 81% White, 7% African American, 6% Hispanic, 1% Asian, 1% Native American, and 3% Other or No Response. Total internal consistency coefficients (using Cronbach s alpha) based on the normative data ranged from.86 to.94 for scales on the CPRS Short Form and from.88 to.95 for scales on the CTRS Short Form. Furthermore, test-retest reliability coefficients (with a 6- to 8-week interval) ranged from.62 to.85 for scales on the CPRS Short Form (N = 49) and from.72 to.92 for scales on the CTRS Short Form (N = 50). Scale scores from the short forms were highly correlated with parallel scale scores from the long forms within the normative sample (all coefficients were above.95), suggesting a high degree of equivalence across parallel scales on the short and long forms (Conners, 1997). Preliminary research suggests that parent and teacher ratings on the Conners scales are able to discriminate children with ADHD from children without ADHD to a reasonable degree of accuracy (Conners, 1997; Conners, Sitarenios, Parker, & Epstein, 1998a, 1998b), but research with the short forms has been limited. Procedure All participants received a comprehensive psychological evaluation that included measures of cognitive ability, achievement, language, memory, executive function, attention, behavior, and emotional functioning. All measures were administered consistent with standardized procedures and in random order. Participation was voluntary with consent obtained from parents and assent obtained from participants. Following completion of the evaluations, participants received a comprehensive report of the results, along with recommendations when appropriate. The comprehensive evaluations for this research were conducted at a university counseling and assessment clinic. At least two individuals (advanced doctoral students and at least one licensed psychologist) independently reviewed the results of the cognitive, achievement, emotional, and behavioral measures and provided diagnostic recommendations according to DSM-IV criteria. Because the BRIEF was considered one of the experimental measures of executive function, diagnosticians were blind to participants BRIEF scores when reaching diagnostic decisions. Interdiagnostician agreement using the licensed psychologist and one doctoral student (i.e., case manager for the particular participant) was determined to be 90% (Cohen s Kappa = 0.84). The interdiagnostician agreement for children and adolescents diagnosed with a subtype of ADHD was found to be 97% (Cohen s Kappa = 0.93). For the purposes of this study, following the process of evaluation and diagnosis, participants were classified into one of the three diagnostic groups (i.e., No Diagnosis, ADHD, Other Clinical) to allow for group comparisons on the behavioral ratings provided by participants parents and teachers. Results A multivariate analysis of variance (MANOVA) was conducted with T-scores on each of the four instruments (BRIEF Parent Form, BRIEF Teacher Form, PRS Short Form, CTRS Short Form) to detect statistically significant diagnostic group differences among the scales included on the instruments. Due to the large number of comparisons being made, alpha was set at p.001 to control for Type I errors. When the overall MANOVA was statistically significant, follow-up univariate analyses of

402 Journal of Attention Disorders Table 2 Group Comparisons of Mean Scores and Standard Deviations for the Behavior Rating Inventory of Executive Function (BRIEF) Parent Form Scales No Diagnosis (n = 26) ADHD (n = 41) Other Clinical (n = 25) ANOVA Results M SD M SD M SD F(2, 89) Partial η 2 Inhibit 49.3 a,b 10.6 64.9 a 14.1 64.1 b 17.6 10.65*.19 Shift 47.7 a,b 10.2 62.9 a 13.5 61.1 b 12.8 12.73*.22 Emotional Control 46.5 a,b 9.7 62.2 a 13.1 65.0 b 15.9 15.77*.26 Initiate 50.0 a,b 8.9 62.8 a 10.5 61.6 b 12.3 12.76*.22 Working Memory 52.9 a,b 10.2 71.5 a 9.1 64.8 b 11.1 27.97*.39 Plan/Organize 52.0 a,b 10.1 69.1 a 10.1 65.9 b 12.3 21.01*.32 Organization of Materials 53.8 a 9.8 63.3 a 7.6 55.9 12.0 9.27*.17 Monitor 49.7 a,b 9.3 65.9 a 10.3 63.2 b 12.8 18.94*.30 Behavioral Regulation Index 47.5 a,b 9.5 66.0 a 13.2 65.5 b 16.1 17.97*.29 Metacognition Index 51.9 a,b 9.4 69.9 a 9.1 64.4 b 11.7 26.23*.37 Global Executive Composite 50.4 a,b 9.0 70.0 a 10.2 66.0 b 13.6 26.78*.38 Note: ADHD = Attention-Deficit/Hyperactivity Disorder; ANOVA = analysis of variance. Means in a row sharing subscripts are significantly different at the p.001 level, based on post hoc analyses. F statistics and partial η 2 effect size values are based on the follow-up univariate ANOVA results conducted individually with each of the 11 scales. * Group main effect is statistically significant, p <.001. variance (ANOVAs) were conducted on scores for each of the individual scales on the four instruments. When these ANOVAs were statistically significant, post hoc analyses were conducted to determine which of the three diagnostic groups were statistically different from one another. BRIEF Analyses Beginning with the BRIEF Parent Form, mean T- scores and standard deviations for the three groups on each of the scales are presented in Table 2. The MANOVA revealed a statistically significant multivariate main effect for the group variable, Wilks s Λ =.39, F(22, 158) = 4.29, p <.001, multivariate partial η 2 =.37. Based on the significant MANOVA, univariate ANOVAs were conducted with each of the 8 scales and three broad indexes on the BRIEF Parent Form as follow-up tests. The ANOVAs revealed a significant main effect for the group variable at the p <.001 level across all 11 scales. F statistics and partial η 2 effect size values for each scale based on the follow-up ANOVAs are provided in Table 2. For each scale, Bonferroni post hoc analyses were conducted with the mean scores to determine which of the three groups were statistically different from one another; results of these analyses also are presented in Table 2 using subscripts. On every scale except for Organization of Materials, both the ADHD and Other Clinical groups scored significantly higher than the No Diagnosis group, indicating that both groups are likely to exhibit a much higher degree of difficulty related to the constructs measured by the BRIEF, based on parent reports. The ADHD group scored higher than the Other Clinical group on all but the Emotional Control scale, but most of the group differences in scores between the two clinical groups were small, and none were statistically significant based on post hoc analyses. Mean T-scores and standard deviations for the three groups on each of the BRIEF Teacher Form scales are presented in Table 3. Unlike with the BRIEF Parent Form, the Teacher Form MANOVA was not significant at the p.001 level, Wilks s Λ =.50, F(22, 88) = 1.67, p =.05, multivariate partial η 2 =.29. Because the MANOVA did not reveal a multivariate main effect for the group variable at the predetermined alpha level, follow-up univariate ANOVAs were not conducted on the individual scales of the BRIEF Teacher Form. Observation of Table 3 indicates that on many scales on the BRIEF Teacher Form, there were differences between the No Diagnosis and other (i.e., ADHD and/or Other Clinical) groups that were greater than 10 points in magnitude, but the lack of a significant multivariate main effect for the group variable at the p.001 level prevented us from conducting univariate ANOVAs on these individual scales. Given their magnitude, however, these between-group differences in teacher ratings may be considered practically or clinically significant. In fact, the No Diagnosis and ADHD group means differed by more than 10 points on all 11 scales, and the No Diagnosis and Other Clinical group means differed by more than 10 points on 8 of the 11 scales. In contrast, the ADHD and Other Clinical

Sullivan, Riccio / BRIEF and Conners Scales 403 Table 3 Group Comparisons of Mean Scores and Standard Deviations for the Behavior Rating Inventory of Executive Function (BRIEF) Teacher Form Scales No Diagnosis (n = 16) ADHD (n = 23) Other Clinical (n = 18) M SD M SD M SD Inhibit 54.9 11.9 67.4 16.4 66.6 20.7 Shift 51.6 8.7 63.4 14.5 60.0 15.0 Emotional Control 52.5 7.9 65.0 17.0 61.6 15.9 Initiate 51.1 9.9 66.5 13.0 69.6 15.4 Working Memory 57.5 14.7 74.2 13.7 72.5 19.7 Plan/Organize 54.4 13.9 68.4 9.9 67.5 14.5 Organization of Materials 63.9 24.1 74.2 21.7 68.9 22.5 Monitor 54.9 12.3 68.7 12.0 67.4 15.8 Behavioral Regulation Index 53.3 9.2 66.8 15.4 64.1 16.6 Metacognition Index 57.6 13.4 74.4 16.6 71.6 18.0 Global Executive Composite 55.8 12.2 72.3 13.0 70.7 17.5 Note: ADHD = Attention-Deficit/Hyperactivity Disorder; ANOVA = analysis of variance; MANOVA = multivariate analysis of variance. Because teacher ratings were not obtained for all participants, n = 57 for these analyses. Follow-up univariate ANOVA results (i.e., F statistics, partial η 2 effect size values) for each of the 11 scales are not provided because the MANOVA did not reveal a multivariate main effect for the group variable at the p.001 level. Table 4 Group Comparisons of Mean Scores and Standard Deviations for the Conners Parent Rating Scale (CPRS) Short Form Scales No Diagnosis (n = 26) ADHD (n = 41) Other Clinical (n = 25) ANOVA Results M SD M SD M SD F(2, 89) Partial η 2 Oppositional 49.2 a,b 6.2 62.2 a 14.7 63.7 b 17.9 9.05*.17 Cognitive Problems/Inattention 51.9 a,b 10.1 71.2 a 9.5 67.7 b 12.5 27.94*.39 Hyperactivity 52.0 a 9.6 68.3 a 15.0 63.8 17.5 10.22*.19 ADHD Index 52.7 a,b 9.8 69.8 a 12.1 66.7 b 14.5 16.18*.27 Note: ADHD = Attention-Deficit/Hyperactivity Disorder; ANOVA = analysis of variance. Means in a row sharing subscripts are significantly different at the p.001 level, based on post hoc analyses. F statistics and partial η 2 effect size values are based on the follow-up univariate ANOVA results conducted individually with each of the four scales. * Group main effect is statistically significant, p <.001. group means did not differ by 10 points for any of the scales; these two groups scored quite similarly on most scales. The fact that the sample size for the teacher form analyses (n = 57) was smaller than the sample size for the parent form analyses (n = 92) may have contributed to more difficulty obtaining a statistically significant main effect for the group variable with the teacher form scores, even though between-group differences were of a similar magnitude to those observed with the parent form scores. A correlation analysis revealed that parent ratings on the BRIEF scales were significantly correlated with teacher ratings on the same scales. These correlation coefficients ranged from.31 for Organization of Materials to.59 for the Behavioral Regulation Index (median Pearson r coefficient =.48), and all correlations were statistically significant (p.05). Thus, there was a moderate degree of consistency between the parents and teachers providing ratings of participants behaviors using the BRIEF scales. Conners Analyses Mean T-scores and standard deviations for the three groups on each of the CPRS Short Form scales are presented in Table 4. The MANOVA revealed a statistically significant multivariate main effect for the group variable, Wilks s Λ =.57, F(8, 172) = 7.02, p <.001, multivariate partial η 2 =.25. Based on the significant MANOVA results, univariate ANOVAs were conducted with each of the four scales on the CPRS Short Form as follow-up tests. The ANOVAs revealed a significant main effect for the group variable at the p <.001 level across all four scales. F statistics and partial η 2 effect size

404 Journal of Attention Disorders values for each scale based on the follow-up ANOVAs are provided in Table 4. For each scale, Bonferroni post hoc analyses were conducted with the mean scores to determine which of the three groups were statistically different from one another; results of these analyses also are presented in Table 4 using subscripts. On all four of the CPRS Short Form scales, participants in the ADHD group received significantly higher parent ratings than participants in the No Diagnosis group. Participants in the Other Clinical group received significantly higher parent ratings than participants in the No Diagnosis group on all but the Hyperactivity scale. Finally, although participants in the ADHD group scored higher than participants in the Other Clinical group on all but the Oppositional scale, these differences were not statistically significant. Mean T-scores and standard deviations for the three groups on each of the CTRS Short Form scales are presented in Table 5. The CTRS Short Form MANOVA did not reveal a multivariate main effect for the group variable that was significant at the p.001 level, Wilks s Λ =.70, F(8, 120) = 2.89, p =.006, multivariate partial η 2 =.16. Because the MANOVA did not reveal a multivariate main effect for the group variable at the predetermined alpha level, follow-up univariate ANOVAs were not conducted on the individual scales of the CTRS Short Form. Observation of Table 5 indicates that on the Cognitive Problems/Inattention and ADHD Index scales, there were differences between the No Diagnosis and other (i.e., ADHD and Other Clinical) groups that were greater than 10 points in magnitude, but the lack of a significant multivariate main effect for the group variable at the p.001 level prevented us from conducting univariate ANOVAs on these individual scales. Given their magnitude, however, these between-group differences in teacher ratings may be considered practically or clinically significant. In contrast, between-group differences on the Oppositional and Hyperactivity scales were rather small across all three groups, and the ADHD and Other Clinical group means were similar across scales. As with the BRIEF Teacher Form analyses, the fact that the sample size for the CTRS Short Form analyses (n = 66) was smaller than the sample size for the CPRS Short Form analyses (n = 92) may have resulted in difficulty obtaining a statistically significant main effect for the group variable with the CTRS Short Form scores. At the same time, however, it must be noted that the between-group differences for the CTRS Short Form scores were not of the same magnitude as those observed with the CPRS Short Form scores. This discrepancy can be seen by comparing the group mean scores presented in Table 4 with those presented in Table 5. Table 5 Group Comparisons of Mean Scores and Standard Deviations for the Conners Teacher Rating Scale (CTRS) Short Form Scales Other No Diagnosis ADHD Clinical (n = 18) (n = 26) (n = 22) M SD M SD M SD Oppositional 48.3 5.5 56.0 14.7 52.7 11.2 Cognitive Problems/ 49.1 7.6 61.2 12.5 61.8 11.1 Inattention Hyperactivity 52.8 11.2 57.6 14.0 55.4 12.0 ADHD Index 51.1 11.6 63.7 13.5 62.4 13.4 Note: ADHD = Attention-Deficit/Hyperactivity Disorder; ANOVA = analysis of variance; MANOVA = multivariate analysis of variance. Because teacher ratings were not obtained for all participants, n = 66 for these analyses. Follow-up univariate ANOVA results (i.e., F statistics, partial η 2 effect size values) for each of the four scales are not provided because the MANOVA did not reveal a multivariate main effect for the group variable at the p.001 level. A correlation analysis revealed that parent ratings on the Conners scales were significantly correlated with teacher ratings on the same scales. These correlation coefficients ranged from.51 for Oppositional to.58 for the ADHD Index (median Pearson r coefficient =.54), and all correlations were statistically significant (p.001). Thus, there was a moderate degree of consistency between the parents and teachers providing ratings of participants behaviors using the Conners scales; agreement between parents and teachers was somewhat higher with the Conners scales as compared with the BRIEF scales. Discussion The purpose of this study was to investigate diagnostic group differences in terms of parent and teacher ratings on the BRIEF and Conners scales, which are frequently used measures of characteristics of ADHD and executive function. Children and adolescents in the ADHD and Other Clinical groups received significantly higher mean T- scores than participants in the No Diagnosis group on most of the scales included on the BRIEF Parent Form and CPRS Short Form. A similar pattern was found for scales on the BRIEF Teacher Form and CTRS Short Form, but univariate statistical analyses were not conducted with these scales because the multivariate group main effect was not significant at the predetermined level. At the same time, scores for the ADHD and Other Clinical groups were typically similar across scales, suggesting that, to some extent, the characteristics of ADHD

Sullivan, Riccio / BRIEF and Conners Scales 405 and executive dysfunction measured by the instruments under study may not be specific to children and adolescents with ADHD. This is consistent with research suggesting that executive dysfunction may be involved in psychiatric disorders other than just ADHD (e.g., Gioia et al., 2002; Shear et al., 2002) and also is consistent with the observation made by Anderson, Anderson, Northam, Jacobs, and Mikiewicz (2002) that executive dysfunction may be implicated in multiple disorders as a result of the wide range of behavioral, emotional, social, and cognitive symptoms that appear to be related to executive dysfunction. Conversely, these findings may suggest that although the BRIEF and Conners scales are able to distinguish children with ADHD or other clinical diagnoses from children without a diagnosis, the scales are less successful at discriminating children with ADHD from those with other clinical diagnoses. It is unfortunate that the heterogeneity of the Other Clinical group with regard to the diversity of diagnoses represented prevents any definitive conclusions about how children and adolescents with specific diagnoses other than ADHD may perform on the scales under study. Upon examination of the standard deviations for mean T-scores in Tables 2 through 5, we see a general tendency for the standard deviations for the Other Clinical group to be somewhat higher than the standard deviations for the No Diagnosis and ADHD groups. The higher degree of variance among scores in the Other Clinical group is likely explained by the diagnostic heterogeneity of the participants within this group. Further research using larger and more welldefined samples of children with other clinical diagnoses (e.g., oppositional defiant and conduct disorders, mood disorders, anxiety disorders) will be necessary to provide more reliable information about the concurrent validity and diagnostic utility of these instruments. Results of the correlational analyses indicate that scores on the identical scales of the BRIEF Parent Form and BRIEF Teacher Form were moderately and significantly correlated with one another, suggesting that there was significant agreement among parent and teacher ratings on the BRIEF. The same can be said for scores on the Conners scales, as parent ratings on the CPRS Short Form and teacher ratings on the CTRS Short Form were significantly related. The moderate magnitude of these correlations seems desirable in light of the purpose of assessment using multiple informants: If consistency between raters is very low, the behaviors of concern may be related primarily to a specific setting or context; if consistency between raters is extremely high, we gain only limited additive information by gathering ratings from multiple sources. Thus, moderately correlated ratings suggest a degree of behavioral stability across settings and contexts, while also allowing for (a) some differences in perspective among raters and (b) the possibility that the child s behaviors may be more conspicuous or disruptive in some situations than in others (e.g., based on the unique characteristics of a particular classroom, social situation, or academic subject). There are several limitations with our study that deserve consideration. First, none of the participants in our study were diagnosed as ADHD Predominantly Hyperactive/Impulsive Type. Thus, comparisons among the three ADHD subtypes on the scales were not possible. Second, the extent to which our results are generalizable is limited to populations similar to our sample (e.g., children and adolescents of average or better cognitive ability, with the ability to speak and read English, and with no history of severe head injury or schizophrenia). Third, information from the teacher rating scales was not obtained for all of the participants, resulting in a smaller number of participants in each of the three groups for the BRIEF Teacher Form and CTRS Short Form analyses. This reduced number of participants likely contributed to the lack of a statistically significant main effect for the group variable during the multivariate analyses with the teacher scales. Finally, our decision to use the short form of the Conners scales may be seen as a limitation. Using the long form of the Conners scales would have provided more thorough information with regard to group differences, as the long form includes additional scales (e.g., Anxious-Shy, Social Problems, Psychosomatic) that may better differentiate the ADHD group from the Other Clinical group. Given the importance of considering executive function deficits in the assessment and intervention for children and adolescents with ADHD, it is advantageous to obtain knowledge with regard to the performance of different diagnostic groups on the measures we use. Our data suggest that the BRIEF and Conners scales were able to distinguish clinical from nonclinical participants and that there was a moderate level of agreement between parents and teachers in describing children s behavior with these instruments. At the same time, the scales were less successful at discriminating children with ADHD from those with other clinical diagnoses. Thus, clinicians attempting to conduct comprehensive psychological evaluations where potential comorbidity and other clinical diagnoses are explored should consider the use of more omnibus rating scales (e.g., Behavior Assessment System for Children Second Edition; Reynolds & Kamphaus, 2004) in addition to the BRIEF and Conners scales. This approach would seem especially appropriate in situations where the reason for referral is unclear or includes multiple behavioral or emotional concerns, as more omnibus measures contain

406 Journal of Attention Disorders scales that assess a range of behaviors, thereby making them useful in differential diagnosis. Combining the information gained from the BRIEF and Conners scales with information gained from omnibus measures will result in a more complete diagnostic picture of the child and will likely contribute to the development of appropriate interventions. References Achenbach, T. M. (1991). Manual for the Child Behavior Checklist. Burlington: University of Vermont, Department of Psychiatry. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Anderson, V. A., Anderson, P., Northam, E., Jacobs, R., & Mikiewicz, O. (2002). Relationships between cognitive and behavioral measures of executive function in children with brain disease. Child Neuropsychology, 8, 231-240. Barkley, R. A. (1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121, 65-94. Barkley, R. A. (1998). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed.). New York: Guilford. Baron, I. S. (2000). Test review: Behavior Rating Inventory of Executive Function. Child Neuropsychology, 6, 235-238. Burmeister, R., Hannay, H. J., Copeland, K., Fletcher, J. M., Boudousquie, A., & Dennis, M. (2005). Attention problems and executive functions in children with spina bifida and hydrocephalus. Child Neuropsychology, 11, 265-283. Collett, B. R., Ohan, J. L., & Myers, K. M. (2003). Ten-year review of rating scales. V: Scales assessing attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 1015-1037. Conners, C. K. (1997). Conners Rating Scales Revised. North Tonawanda, NY: Multi-Health Systems. Conners, C. K., Sitarenios, G., Parker, J.D.A., & Epstein, J. N. (1998a). The Revised Conners Parent Rating Scale (CPRS-R): Factor structure, reliability, and criterion validity. Journal of Abnormal Child Psychology, 26, 257-268. Conners, C. K., Sitarenios, G., Parker, J.D.A., & Epstein, J. N. (1998b). Revision and restandardization of the Conners Teacher Rating Scale (CTRS-R): Factor structure, reliability, and criterion validity. Journal of Abnormal Child Psychology, 26, 279-291. DuPaul, G. J., Power, T. J., Anastopoulos, A. D., & Reid, R. (1998). ADHD Rating Scale IV: Checklists, norms, and clinical interpretation. New York: Guilford. DuPaul, G. J., & Stoner, G. (2003). ADHD in the schools: Assessment and intervention strategies (2nd ed.). New York: Guilford. Forbes, G. B. (2001). A comparison of the Conners Parent & Teacher Rating Scales, the ADD-H Comprehensive Teacher s Rating Scale, and the Child Behavior Checklist in the clinical diagnosis of ADHD. Journal of Attention Disorders, 5, 25-40. Gioia, G. A., Isquith, P. K., Guy, S. C., & Kenworthy, L. (2000). Behavior Rating Inventory of Executive Function. Odessa, FL: Psychological Assessment Resources. Gioia, G. A., Isquith, P. K., Kenworthy, L., & Barton, R. M. (2002). Profiles of everyday executive function in acquired and developmental disorders. Child Neuropsychology, 8, 121-137. Jarratt, K. P., Riccio, C. A., & Siekierski, B. M. (2005). Assessment of attention deficit hyperactivity disorder (ADHD) using the BASC and BRIEF. Applied Neuropsychology, 12, 83-93. Mahone, E. M., Cirino, P. T., Cutting, L. E., Cerrone, P. M., Hagelthorn, K. M., Hiemenz, J. R., et al. (2002). Validity of the Behavior Rating Inventory of Executive Function in children with ADHD and/or Tourette syndrome. Archives of Clinical Neuropsychology, 17, 643-662. Mangeot, S., Armstrong, K., Colvin, A. N., Yeates, K. O., & Taylor, H. G. (2002). Long-term executive function deficits in children with traumatic brain injuries: Assessment using the Behavior Rating Inventory of Executive Function (BRIEF). Child Neuropsychology, 8, 271-284. Naglieri, J. A. (2005). The Cognitive Assessment System. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., pp. 441-460). New York: Guilford. Naglieri, J. A., & Das, J. P. (1997). Cognitive Assessment System. Itasca, IL: Riverside. Naglieri, J. A., & Das, J. P. (2005). Planning, Attention, Simultaneous, Successive (PASS) Theory: A revision of the concept of intelligence. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., pp. 120-135). New York: Guilford. Pizzitola, K. M. (2002). Test review: Behavior Rating Inventory of Executive Function. Journal of Psychoeducational Assessment, 20, 400-405. Powell, K. B., & Voeller, K.K.S. (2004). Prefrontal executive function syndromes in children. Journal of Child Neurology, 19, 785-797. Reich, W., Welner, Z., & Herjanic, B. (1997). Diagnostic Interview for Children and Adolescents IV. North Tonawanda, NY: Multi- Health Systems. Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children. Circle Pines, MN: American Guidance Service. Reynolds, C. R., & Kamphaus, R. W. (2004). Behavior Assessment System for Children (2nd ed.). Circle Pines, MN: American Guidance Service. Shear, P. K., DelBello, M. P., Rosenberg, H. L., & Strakowski, S. M. (2002). Parental reports of executive dysfunction in adolescents with bipolar disorder. Child Neuropsychology, 8, 285-295. Strauss, E., Sherman, E.M.S., & Spreen, O. (2006). A compendium of neuropsychological tests: Administration, norms and commentary. New York: Oxford University Press. Sullivan, J. R., & Riccio, C. A. (2006). An empirical analysis of the BASC Frontal Lobe/Executive Control Scale with a clinical sample. Archives of Clinical Neuropsychology, 21, 495-501. Wechsler, D. (1991). Wechsler Intelligence Scale for Children (3rd ed.). San Antonio, TX: Psychological Corporation. Jeremy R. Sullivan is an assistant professor of educational psychology at the University of Texas at San Antonio. His research interests focus on issues related to psychological assessment and professional issues in school psychology (e.g., training, supervision, and ethics). Cynthia A. Riccio is a professor, the director of training for the school psychology program, and a member of the neuroscience faculty at Texas A&M University. Her research interests include ADHD, pediatric neuropsychology, and learning/language disorders. She has more than 50 refereed journal articles published or in press and has coauthored a book on the use of continuous performance tests in clinical practice.