Clinical Usefulness of the Oppositional Defiant Disorder Rating Scale (ODDRS)

Clinical Usefulness of the Oppositional Defiant Disorder Rating Scale (ODDRS) Journal of Emotional and Behavioral Disorders 18(4) 247 255 Hammill Institute on Disabilities 2010 Reprints and permission: http://www. sagepub.com/journalspermissions.nav DOI: 10.1177/1063426609349734 http://jebd.sagepub.com Elizabeth M. O Laughlin, 1 Jessica L. Hackenberg, 2 and Maria M. Riccardi 1 Abstract The present study examined the reliability, validity, and clinical utility of the Oppositional Defiant Disorder Rating Scale (ODDRS) in a population of children referred for ADHD evaluation. The diagnostic benefit of using a rating scale specific to Oppositional Defiant Disorder (ODD), in addition to a broad range behavior scale, was also investigated. Parents and teachers of 177 clinic-referred children provided ratings on the ODDRS. The average correlation between parent and teacher ratings was minimal (r =.13). Children diagnosed with ODD through structured parent interviews were found to have elevated scores on the parent version of the ODDRS, as compared to children without a research diagnosis of ODD. However teacher ratings did not discriminate between the ODD and no-odd groups. There was minimal evidence of incremental validity as classification rates for the ODDRS were almost identical to classification rates based on the Aggression subscale of the Behavior Assessment System for Children (BASC-2). Keywords assessment, externalizing behavior, conduct problems Oppositional Defiant Disorder (ODD) is one of the most prevalent externalizing disorders in childhood, affecting approximately 2% to 16% of children and adolescents (American Psychiatric Association [APA], 1994). Until recently there has not been an assessment tool specific only to ODD symptoms that directly corresponds to the diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; APA, 1994). In lieu of an ODD-specific measure, practitioners often administer a broad-based behavioral measure to assess for this disorder. However, broad-based measures, such as the Behavior Assessment System for Children: Second Edition (BASC-2; Reynolds & Kamphaus, 2004) and the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001) do not include items consistent with DSM-IV diagnostic criteria for ODD. Rather, select aspects of ODD may be reflected in the Aggression or Conduct Disorder subscales of such measures. Thus, the development of a scale specific to the diagnostic criteria of ODD offers many potential benefits to practitioners who are in need of a time- and cost-effective method of assessing specifically for disruptive behavior. In contrast to the criteria for Attention Deficit/ Hyperactivity Disorder (ADHD), the DSM-IV criteria for ODD do not require that the behavior be present in two or more settings (APA, 1994). However, identifying whether the oppositional behavior is occurring at school, in addition to the home setting, is useful in determining the pervasiveness and severity of the problematic behavior, thus enhancing treatment planning. For example, if the oppositional behavior is occurring only in the home setting, this may be indicative that a parent training intervention may be most effective in modifying the behavior of the child. In contrast, if the practitioner finds that disruptive behavior is occurring in both the home and school setting, and that the oppositional behavior is pervasive toward all adults, it may be more beneficial and productive to use a more extensive ecological approach to intervention. To tailor interventions to a child s specific oppositional behavior pattern, the practitioner must first obtain assessment data from both settings, thus a brief symptom rating scale specific to the behavior of greatest concern may be most time and cost effective. The Oppositional Defiant Disorder Rating Scale (ODDRS) was developed as a measure of oppositionality to address the need for a brief symptom rating scale specific to ODD behavior (Hommersen, Murray, Ohan, & Johnson, 2006). Although the ODDRS has shown promise as a brief screening tool in a preliminary reliability and validity study, further study is needed to examine the sensitivity, specificity, and clinical utility of the ODDRS as a diagnostic tool. 1 Indiana State University 2 Akron, OH Corresponding Author: Elizabeth M. O Laughlin Indiana State University Root Hall Terre Haute, IN 47809 Email: lizo@indstate.edu

248 Journal of Emotional and Behavioral Disorders 18(4) The present study builds on the preliminary reliability and validity information provided by Hommersen et al. (2006) by examining the diagnostic utility of the ODDRS in a clinical population. In addition to examining interrater reliability across settings (parent and teacher), the present study considers convergent validity in comparing the ODDRS with the BASC-2. Characteristics of ODD Oppositional Defiant Disorder was first introduced in the DSM-III as a disorder characterized by a recurrent pattern of negative, hostile, and defiant behavior toward authority figures that is beyond developmental norms and is not representative of the more severe antisocial symptoms of conduct disorder (APA, 1980; Waldman & Lilienfeld, 1991). Since that time many attempts have been made to identify the clinical flavor or the defining features of the ODD that make it unique from the other disruptive behavior disorders. Based on the DSM-IV field trials, Frick et al. (1994) suggested that spiteful and vindictive and angry and resentful criterion were the two most consistent predictive symptoms of ODD and indicated that they could be viewed as defining the core features of the disorder. In comparison, Searight, Nahlik, and Campbell (1995) state that the defining feature critical in determining a diagnosis of ODD is a pattern of negativistic, angry behavior present for at least 6 months as well as overt defiance and hostility toward parents beyond that which is developmentally appropriate. Conceptualization of the Disruptive Behavior Disorder Spectrum Complicating the conceptualization and differential diagnosis of disruptive behavior disorders is the difference in the number of symptoms and level of impairment necessary to make a diagnosis among the disorders. The DSM-IV requires that at least six inattentive or hyperactive symptoms be present for a diagnosis of ADHD, whereas only four oppositional symptoms are required for a diagnosis of ODD (APA, 1994). Furthermore, to diagnose ADHD, the diagnostic criteria state that there must be at least two areas of significant impairment in functioning. In contrast, the diagnostic criteria for ODD do not specify the number of settings in which impairment must be present. The different levels of impairment required for each disorder complicate the conceptualization of disruptive behavior disorders overall, as ADHD seems to be more representative of a generalized way of behaving that exists in multiple settings, whereas the behavior related to ODD may be specific to certain situations or may be a way of relating to specific individuals. These differences in the number of symptoms and level of impairment required for a diagnosis between ADHD and ODD make it much easier for a child to meet criteria for an ODD diagnosis. Given that ODD requires impairment in only one setting, it seems feasible that in some cases a child could meet ODD criteria related to parent-child conflict as opposed to a more general pattern of behavior across settings. In addition to a difference in the level of symptomatology and impairment required to diagnosis ODD and ADHD, the high rates of comorbidity have also complicated conceptualization of the disorders as distinctly unique behavioral constructs. Some researchers have suggested that ODD and ADHD may be conceptually linked in some way as they are highly correlated and are frequently comorbid within the same children (Pillow, Pelham, Hoza., Molina, & Stultz, 1998). Eiraldi, Power, and Nezu (1997) found that children with ADHD-Combined Type were much more likely to have a diagnosis of ODD than were children with ADHD-Inattentive Type or controls, which may be representative of the overlap of impulsivity in the constructs of hyperactivity and oppositionality. The clinical relationship between ODD and ADHD is so common that some researchers argue that pure ODD is relatively rare in comparison (van Lier, Verhulst, van der Ende, & Crijnen, 2003). Diagnostic Utility of Parent and Teacher Ratings There has been a great deal of consideration paid to the relative usefulness of parent versus teacher ratings in diagnosing child psychopathology. In particular, there has been significant discussion of the source-specific nature of ODD (Drabick, Gadow, & Loney, 2007). Nolan, Gadow, and Sprafkin (2001) found that teacher reports of ADHD and ODD symptoms had relatively high convergence with population prevalence rates of ADHD and ODD as well as with specific ADHD subtypes. Abikoff, Courtney, Pelham, and Koplewicz (1993) found that teachers were more likely to endorse hyperactive and impulsive behaviors when the child was also perceived as oppositional. However, they also found that teachers ratings of oppositional and conduct problem behaviors were generally accurate and not inflated when presented with co-occurring hyperactive/ impulsive behaviors. More recently Smith (2007) found that for younger children with externalizing behavior in an outpatient setting, parents and teachers appeared to be equivalent in their efficacy as raters of behavior. Drabick et al. (2007) found that looking at mother- or teacher-defined ODD (i.e., source-specific information) resulted in better internal validity and differentiation than combining informants. Thus, there appears to be good support for considering both mother and teacher ratings in assessment of ODD in children.

O Laughlin et al. 249 Assessment of ODD The most common assessment tools employed when considering a disruptive behavior disorder diagnosis are broadbased behavioral or symptom rating scales, many of which list the DSM-IV ADHD and ODD symptoms and ask for a frequency rating. The inherent weakness in such measures is the large symptom overlap between ADHD and ODD created by the DSM-IV symptom criteria as well as the cross-loading of hyperactivity/impulsivity characteristics on both ADHD and ODD scales. Results of studies comparing different broad-range rating scales (e.g., Eiraldi, Power, Karustis, & Goldstein, 2000) have suggested that multi-axial rating scales are useful for discriminating children with ADHD from those who do not have the disorder, but they are not as useful in discriminating between externalizing disorders. For example, Eiraldi et al. (2000) found that both the Devereux Scales of Mental Disorders (DSMD) and the CBCL were better at ruling out than ruling in ODD, and the DSMD Conduct and CBCL Aggressive Behavior subscales were the best predictors of ODD. In contrast to broad-based behavioral rating scales, such as the CBCL and the BASC-2, there have been several measures developed specifically to assess for the three specific disruptive behavior disorders. The Disruptive Behavior Disorders Rating Scale (DBDRS; Pelham, Gnagy, Greenslade, & Milich, 1992) and Vanderbilt ADHD Diagnostic Parent Rating Scale (Wolraich et al., 2003) each have ADHD type-specific, ODD, and CD scales. Although these measures may represent a more clinically useful tool when attempting to differentiate between specific disruptive disorders, a measure that combines these three disorders also presents specific problems. The previously mentioned symptom overlap between ADHD and ODD can make some of the symptoms seem repetitive to parents or teachers completing the rating forms. In addition, a halo effect may occur in which parents or teachers who endorse hyperactive/impulsive behaviors are more likely to also endorse oppositional behaviors. There is currently little information regarding the psychometrics of ODD items when administered separately versus combined with ADHD items such as on the DBDRS. If attempting to avoid the ODD dimension of symptoms from influencing a rater s perceptions of the ADHD dimension of symptoms, having separate assessment tools that are based directly on the DSM-IV diagnostic criteria would seem to offer several advantages. For example, the ADHD Rating Scale IV (DuPaul, Power, Anastopoulos, & Reid, 1998) is frequently used as a narrow-range measure of ADHD. No such rating scale specific to the assessment of parent- and teacher-reported DSM-IV ODD symptoms existed until recently. Hommersen et al. (2006) introduced the ODDRS as a brief measure that corresponded directly to the symptoms of ODD found in the DSM-IV-TR. The ODDRS was constructed as a parent measure that utilizes a 4-point rating scale to indicate the extent to which each of the eight ODD symptoms is representative of his or her child. An initial examination of the reliability and validity of the ODDRS indicated that it has the potential to be useful as a screening tool and to provide valuable information regarding the child behaviors that form the DSM-IV criteria for ODD (Hommersen et al., 2006). Present Study The present study examined the psychometric properties of the Oppositional Defiant Disorder Rating Scale in a sample of children referred for an ADHD evaluation. Data were gathered through archival methods and represented data collection over the course of approximately 36 months. More specifically, the study explored the interrater reliability of the ODDRS across the home and school settings, the convergent validity of the ODDRS with the BASC-2 Aggression scale, and incremental validity in the differential diagnosis of ODD beyond the use of the Aggression subscale on the BASC-2. Methods Participants Participants included 177 children (124 male and 53 female) between the ages of 5 and 13 (M = 7.91 years, SD = 1.73). Children were assigned a research diagnosis of ODD or no ODD based on parent endorsement of four or more ODD symptoms in a structured interview as well as report of significant impairment in functioning related to oppositional behavior (e.g., home, school, or peer relationships). Based on the described research criteria, 64 children met criteria for the ODD group and 113 children were classified in the no-odd group. A research diagnosis of ADHD was also determined based on the structured parent interview. Based on the research classifications, 69 children met criteria for ADHD but not ODD, 7 children met criteria for ODD only, 57 children met criteria for both ADHD and ODD, and 44 did not meet criteria for ODD or ADHD. Thus the majority of children in the ODD group (57/64) also met criteria for an ADHD diagnosis. Approximately half of the children in the no-odd group (57%) had an ADHD diagnosis, 22% were given a learning disability diagnosis, 11% were given no diagnosis, and the remaining children were diagnosed with adjustment disorder, mood/anxiety disorder, or pervasive developmental disorder. Given that the research diagnosis was based on parent interview data only, several of the children in the ODD group (7%) were given a clinical diagnosis other than ODD (i.e., other data indicated that

250 Journal of Emotional and Behavioral Disorders 18(4) ODD symptoms were secondary to another disorder). Participants were predominantly Caucasian children (87.6%) and came from low-income households (i.e., 57% reported a family income of less than $30,000, and 43% qualified for Medicaid). There were no significant differences between the ODD and no-odd groups in terms of child age, ethnicity, or family income ( p-values >.12), although family income tended to be slightly higher for children in the no- ODD group. Procedures The measures included as part of this study were a part of the standard assessment battery and procedures currently used at an ADHD Evaluation Clinic at a Midwestern University. Doctoral-level psychology graduate clinicians were trained and supervised by a licensed psychologist in the administration and interpretation of the assessment measures. The evaluation was conducted either by one graduate clinician and one licensed psychologist or by two graduate clinicians under the supervision of a licensed psychologist. Parents of children referred to the ADHD Evaluation clinic provided informed consent for assessment data to be used for research purposes. All measures and procedures were approved by the university Institutional Review Board. Although the clinical diagnosis took into consideration all measures used in the assessment battery, the research diagnosis of ODD was based only on information collected during a structured interview (Disruptive Behavior Disorders Parent Structured Interview, Pelham, 2006). Measures Oppositional Defiant Disorder Rating Scale (Hommersen et al., 2006). The Oppositional Defiant Disorder Rating Scale is a normative parent rating scale designed to measure the symptoms of Oppositional Defiant Disorder. The eight items of the scale reflect the eight ODD criteria in the DSM-IV-TR (APA, 2000); raters rate each symptom using a 4-point response scale ranging from 0 (not at all) to 3 (very much). The ODDRS has demonstrated good internal consistency, with a coefficient alpha of.92. It has also demonstrated moderately high interrater reliability between parents (r =.70) and moderate 1-year test-retest reliability (r =.54). Considering validity, Hommersen et al. (2006) also report that the scale is significantly correlated with the Aggression and Delinquency subscales of the Child Behavior Checklist (Achenbach & Rescorla, 2001). For the purposes of this study, the ODDRS was slightly modified (i.e., my child was changed to this child, etc.) to create a teacher version. Behavioral Assessment System for Children: Second Edition (Reynolds & Kamphaus, 2004). The BASC-2 is a wide range behavior rating scale of child psychopathology. Both Parent (PRS-C) and teacher rating forms (TRS-C) were used in the present study. The parent form contains 160 items, and the teacher form contains 139 items. Ratings are made on a 4-point scale ranging from 1 (never) to 4 (almost always). The general gender-specific normative scores were used based on responses from more than 13,000 teachers and parents of children between the ages of 2 and 18 years. The measure is divided into two broad dimensions (Internalizing and Externalizing behavior) and eight specific dimensions, including Aggression, Attention Problems, Conduct Problems, Depression, School Problems, Somatization, Atypicality, and Hyperactivity. Interpretation is based on t-scores, with the suggested clinically significant range including score of 70 and above. One-week test-retest reliability estimates of the TRS-C and PRS-C for the general norm sample range from.84 to.90 and.77 to.91, respectively (Reynolds & Kamphaus, 2004). Internal consistency coefficients for the TRS-C and the PRS-C for the general norm sample range from.87 to.97 and.89 to.94, respectively (Reynolds & Kamphaus, 2004). The BASC-2 does not have a subscale specific to the symptoms or criteria of Oppositional Defiant Disorder; however some behavioral symptoms of the Aggression subscale such as argues when denied own way, loses temper too easily, defies teachers or caregivers, and annoys others on purpose, overlap with ODD criteria and are consistent with the conceptualization of the disorder. Disruptive Behavior Disorders Structured Parent Interview (Pelham, 2004). The Disruptive Behavior Disorders Structured Parent Interview is a clinician administered semistructured interview for use with parents, consisting of DSM-IV symptoms with situational probes that specifically focus on ADHD, ODD, and CD. The interview contains questions to assess for the symptoms of ADHD and ODD as specified by the criteria of the DSM-IV-TR (APA, 2000) and uses a 4-point response scale to rate symptom severity (0 = not a problem; 3 = severe problem). In addition, the Disruptive Behavior Disorders Structured Parent Interview also assesses for the degree of impairment across settings and has been shown to have excellent psychometric properties (Pelham et al., 1992). For the present study, only the ODD and ADHD sections of the Disruptive Behavior Disorders Structured Parent Interview were administered. Research diagnoses for both the ADHD and ODD groups were based on number of symptoms endorsed as a moderate or severe problem (4 or more for ODD, and 6 or more Inattentive or Hyperactive/Impulsive symptoms for ADHD) and parent report of impairment due to ODD or ADHD symptoms (rating of moderate or severe impairment in peer relationships, family relations, or school/academic performance).

O Laughlin et al. 251 Results Reliability Internal consistency. Cronbach s alpha was a =.90 for parent ratings (n = 173) and a =.92 for teacher ratings (n = 163). This estimate of internal consistency is very similar to the alpha of.92 for parent ratings reported by Hommersen et al. (2006). Interrater reliability. Overall, average interrater reliability between parents and teachers on the eight ODDRS items was minimal, r =.11. Five of the eight individual items were significantly correlated for parents and teachers, including Item 1 (loses temper often), Item 3 (noncomplicance), Item 4 (annoys others on purpose), Item 6 (easily annoyed by others), and Item 7 (angry/resentful). The strongest correlation between parent and teacher ratings was for item 4 (annoys others on purpose, r =.21) whereas the lowest correlation (.07) was for Item 5 (blames others for mistakes). The correlation between parent and teacher summary scores on the ODDRS was significant (r =.19, p <.01). In addition, there was a significant positive correlation between the number of items (or symptoms) identified as pretty much or very much true of the child by parents and teachers on the ODDRS (r =.22, p <.01). Parents and teachers tended to agree on the absence of ODD symptoms (51%); however parent and teacher agreement on the presence of four or more ODD symptoms was only 12%. For 30% of cases, parents reported four or more ODD symptoms, whereas teachers did not, and in 11% of cases, teachers reported four or more ODD symptoms, whereas parents did not. Validity Convergent validity. Correlational analyses were used to examine the relationship between the ODDRS and the Aggression subscale of the BASC-2. Considering parent ratings, the parent ODDRS summary score was strongly correlated with parent ratings on the BASC-2 Aggression subscale (r =.85, p <.01). For teacher ratings, the teacher version ODDRS summary score was also strongly correlated with teacher ratings on the BASC-2 Aggression subscale (r =.82, p <.01). The correlation between the ODDRS parent summary score and the teacher BASC-2 Aggression score was r =.20, (p <.05) and the correlation between the ODDRS teacher summary score and the parent BASC-2 Aggression score was r =.18, (p <.05). To further examine convergent validity, the number of ODD symptoms identified by parents in the structured interview as moderate or severe was compared with the number of items on the parent and teacher ODDRS identified as pretty much or very much representative of the child. Both parent and teacher responses on the ODDRS were significantly correlated with number of ODD symptoms endorsed by parents during the clinical interview. As would be expected, parent responses on the ODDRS were more Table 1. Oppositional Defiant Disorder Rating Scale (ODDRS) Means and Standard Deviations for Individual Items Grouped by the Diagnosis of Oppositional Defiant Disorder (ODD) ODD (n = 64) No ODD (n = 113) ODDRS Parent Scale M SD M SD Item 1 2.22 0.77 1.40 0.82 Item 2 2.22 0.88 1.25 0.88 Item 3 2.02 0.92 1.10 1.00 Item 4 1.77 0.99 0.95 1.10 Item 5 2.39 0.85 1.13 1.05 Item 6 1.78 0.97 1.20 1.00 Item 7 1.45 1.05 0.65 0.87 Item 8 1.09 0.99 0.33 0.81 ODD (n = 64) No ODD (n = 113) ODDRS Teacher Scale M SD M SD Item 1 0.75 0.92 0.80 1.02 Item 2 1.11 1.01 0.97 1.00 Item 3 1.25 1.18 1.09 1.04 Item 4 1.22 1.13 0.85 0.97 Item 5 1.42 1.08 1.12 1.05 Item 6 0.97 0.99 0.80 0.91 Item 7 0.65 0.91 0.43 0.82 Item 8 0.59 0.90 0.32 0.73 Note: Range for ODDRS items is 0 to 3. Item1: loses his/her temper, Item 2: argues with adults, Item 3: refuses to comply with adults requests, Item 4: deliberately annoys people, Item 5: blames others for his/her mistakes, Item 6: touchy or easily annoyed, Item 7: angry and resentful, Item 8: spiteful or vindictive. strongly correlated with the number of ODD symptoms endorsed in the clinical interview (r =.61, p <.01), as compared with teacher responses on the ODDRS (r =.26, p <.01). Group differences on ODDRS. Table 1 presents parent and teacher mean ratings, grouped by ODD and no ODD, on individual items on the ODDRS. For children in the ODD group, parents and teachers had the highest mean rating for Item 5 (blames others). However there was little similarity in parent versus teacher ratings overall on the individual items. A multivariate analysis of variance (MANOVA), grouping by ODD diagnosis and entering the parent and teacher ODDRS as dependent variables, was significant, F(2,174) = 39.73, p <.001. Parent ODDRS scores were significantly higher for children in the ODD group as compared with the no-odd group, F(1, 175) = 79.53, p <.001. The difference between the ODD and no-odd group for teacher ODDRS total was not significant (p =.07; see Table 2 for group means). Discriminant Analyses A series of discriminant analyses were conducted to assess the clinical utility of the ODDRS. Specifically, we evaluated

252 Journal of Emotional and Behavioral Disorders 18(4) Table 2. Oppositional Defiant Disorder Rating Scale (ODDRS) Summary Score Mean and Standard Deviation By Diagnostic Group Diagnostic Group Parent ODDRS M (SD) Teacher ODDRS M (SD) ODD (n = 64) 14.94b (5.30) 7.89 (6.52) No ODD (n = 113) 7.83a (4.97) 6.18 (5.70) Note: Subscripts with different values are significantly different. Range for ODDRS Summary Score is 0 to 24. ODD = Oppositional Defiant Disorder. whether or not the ODDRS significantly predicted an ODD diagnosis over and above the BASC-2 Aggression subscale. One assumption of the discriminant analysis is that predictor variables will not be highly correlated. Multicollinearity is indicated by very small tolerance values (i.e., <.10) in the SPSS (version 14.0, Chicago, Illinois) discriminant analysis output. Despite the high correlations between the BASC-2 Aggression subscale and parent and teacher ODDRS, the lowest tolerance value in the discriminant analysis was.39, suggesting that the level of multicollinearity was acceptable. For the discriminant analysis, the diagnosis of ODD was established by the results of the Disruptive Behavior Disorders Structured Parent Interview so that the variable used to establish diagnosis was independent of the predictor variables entered into the discriminant analysis. In the first step of the sequential discriminant analysis, parent and teacher BASC-2 Aggression subscale scores were entered in a stepwise manner. As seen in Table 3, the initial analysis was significant (Wilks s λ =.696, p <.001; canonical correlation =.551). Only the parent BASC-2 Aggression subscale entered as a significant predictor, not the teacher BASC-2 Aggression subscale. Group means for the function indicate that those in the ODD group had a function mean of.87, and those with a no-odd diagnosis had a function mean of.46. Considering classification results, 77.4% of cases were correctly classified based on the parent BASC-2 Aggression subscale. Predicting ODD diagnosis based on the parent BASC-2 Aggression subscale only resulted in 20.3% false positives (i.e., child classified as ODD based on Aggression subscale, but not ODD based on research diagnosis) and 26.6% false negative cases (i.e., child not classified as ODD based on parent Aggression subscale, but classified as ODD in research diagnosis). In the second step of the sequential discriminant analyses, parent and teacher ODDRS scores were entered in addition to the parent BASC-2 Aggression subscale in a stepwise manner. (See Table 3.) This second discriminant analysis was also significant (Wilks s λ =.667, p <.001; canonical correlation =.577). The parent BASC-2 Aggression subscale and the parent ODDRS score entered as significant predictors, whereas the teacher ODDRS was not a significant predictor. Group means for the function indicate that those in the ODD group had a function mean of.95 and those in the no-odd group had a function mean of.52. With the addition of the ODDRS parent score, the correct classification rate was almost identical to the previous model (76.3% vs. 76.8%). Adding the parent ODDRS score to the analysis resulted in a slight increase in false positive cases (from 20.3% to 22.1%) and slight decrease (26.6% to 25%) in false negative cases. Finally, a third discriminant analysis was conducted, entering the ODDRS parent score only. The model was significant (Wilks s λ =.689, p <.001, canonical correlation =.558). The rate of correct classification was 76.3%, with 22.1% false positives and 26.6% false negatives. Comparing the results of the three discriminant analyses, it appears that the strongest discriminatory power, (i.e., lowest Wilks s λ value, highest canonical correlation) was found for the combination of parent BASC-2 Aggression subscale and ODDRS predictors. However, there was almost no difference in rate of correct classification between the parent BASC-2 aggression or parent ODDRS alone and when the two predictors were combined, yielding no support for the incremental validity of the ODDRS if used in addition to the BASC-2. Discussion Results of the present study provide additional psychometric support for the ODDRS in terms of ability to distinguish between diagnostic groups, interrater reliability, as well as convergent validity. Parent scores on the ODDRS were significantly higher for children in the ODD group as compared with the no-odd group. Group differences were significant for parent, but not teacher, ratings on the ODDRS, likely reflecting the fact that the research diagnosis was based on the parent interview. A common rater effect is likely, such that the association between the parent ODDRS and diagnostic group was elevated because of shared rater variance. Previous research has found that informant is an important consideration in diagnosis of disruptive behavior disorders and that it is not uncommon for different informants to endorse different disorders (e.g., Drabick et al., 2007). Given that participants in the present study were children referred to an ADHD Evaluation clinic, it is possible that teachers may have attributed ODD behaviors to ADHD and thus reported fewer ODD symptoms as compared with parents. Considering the pattern of correlations between raters and measures, it appears that correlations within rater (i.e., parent ratings on the ODDRS and BASC-2 Aggression subscale) were much higher than correlations between raters (i.e., parent and teacher ratings on the ODDRS), suggesting the possibility of rater bias and/or differences in child

O Laughlin et al. 253 Table 3. Results of Stepwise Discriminant Analyses Predicting Oppositional Defiant Disorder (ODD) Research Diagnosis Analysis 1 Variables Entered Eigenvalue Canonical Correlation Wilks s λ Chi-Square df p Analysis 1* BASC-2 Parent Aggression 0.437 0.551 0.696 63.234 1 0.0001 Analysis 2** BASC-2 Parent AggressionODDRS 0.499 0.577 0.667 70.479 2 0.0001 Parent Analysis 3 ODDRS Parent 0.452 0.558 0.689 65.067 1 0.0001 Note: BASC-2 = Behavioral Assessment System for Children 2; ODDRS = Oppositional Defiant Disorder Rating Scale. *BASC-2 Teacher Aggression did not enter as a significant predictor. **ODDRS Teacher total did not enter as a significant predictor. behavior in different settings. The low agreement between parents and teachers in identifying children with four or more ODD symptoms underscores that disruptive (or specifically oppositional) behavior may frequently occur only in one setting, may be influenced by the presence or absence of peers, or may be qualitatively experienced differently by parents and teachers. Previous research has suggested that low parent-teacher agreement in regards to disruptive behavior may reflect the situation-specific nature of the behavioral criteria of the disorders themselves (Owens & Hoza, 2003). Correlations between parent and teacher ratings on the BASC-2 Aggression subscale and the ODDRS provide strong support for convergent validity of the ODDRS. Although the BASC-2 Aggression subscale is conceptually representative of both passive and active verbal and physical aggressive behaviors, and the ODDRS (like the DSM- IV criteria) is more behaviorally oriented, it appears that parents and teachers tend to endorse items on both scales very similarly. The present results also showed support for convergent validity of the ODDRS with a structured parent interview, with both parent and teacher responses on the ODDRS significantly correlated with the number of ODD symptoms endorsed in the clinical interview. Results of discriminant analyses revealed the parent ODDRS performed very similarly to the BASC-2 parent Aggression subscale in correctly identifying participants with an ODD research diagnosis. These results suggest that the BASC-2 Aggression subscale alone is fairly effective in identifying children diagnosed with ODD and that the addition of the ODDRS as a narrow range measure does not result in significantly improved classification. In terms of clinical practice, the present results suggest that when an assessment battery includes a broad-range behavior rating scale, such as the BASC-2, inclusion of a measure such as the ODDRS may be redundant. Biederman, Ball, Monuteaux, Kaiser, and Faraone (2008) recently found support for the CBCL Aggression subscale as a timeand cost-effective screening measure to identify ODD criteria in sample of ADHD children. However, given that the ODDRS was similar in effectiveness to the BASC-2 Aggression subscale in identifying ODD, there is support for use of the ODDRS as an alternate to the BASC-2 in screening for ODD. Therefore, the ODDRS may find its niche as a screening measure for clinicians who are looking for a quick and inexpensive method of ruling out ODD or providing support for a possible ODD diagnosis. However, given that neither the ODDRS nor the BASC-2 solicits information regarding impairment, and the ODDRS does not provide information regarding symptoms of other disorders that may account for reported oppositional behavior, use of a clinical interview in addition to either measure is necessary to obtain sufficient information for a diagnosis of ODD. Overall, this study allowed the authors to examine the clinical utility of using an ODD-specific brief symptom rating scale in addition to, or in lieu of, a broad-based behavior rating scale in the diagnosis of Oppositional Defiant Disorder. Results of the study suggest that the ODDRS does show promise as a screening tool with clinical populations, particularly in practice settings in which more broad-based behavior rating scales are either unavailable or impractical. Using broad-based behavior rating scales on a large-scale basis in a clinical setting can be difficult given that practitioners are often not reimbursed for time spent scoring assessment measures. This is becoming a growing problem in the era of managed care companies, which generally require preauthorization for psychological testing and do not reimburse for administrative time. Furthermore, broadbased behavior rating scale forms can be costly, and computer-scoring programs for behaviors rating scales can be expensive for small mental health centers or private practices. In contrast, the ODDRS appears to be a quick, brief, and relatively low-cost assessment measure that would allow for practitioners to assess specifically for ODD symptoms. The ODDRS may also be useful as a research measure, providing a quick method of screening for ODD symptoms based on DSM-IV criteria. Future research could examine the usefulness of the ODDRS as compared with another brief rating scale such as the Disruptive Behavior

254 Journal of Emotional and Behavioral Disorders 18(4) Disorder Rating Scale (Pelham et al., 1992) in differentiating ODD from ADHD and no ADHD or ODD in a clinical population. Limitations As a majority of the children referred to the ADHD Evaluation clinic were male and Caucasian, one limitation of the present study includes a lack of ethnic and gender diversity, which has the potential to limit the generalizability of the results. However, previous research on the ODDRS did not find any significant gender or ethnicity differences (Hommersen et al., 2006), and an examination of means for males and females in the study did not suggest any significant gender differences for subjects in the present sample. As our sample was composed of children referred for assessment for ADHD, most participants presented with ADHD characteristics, thus results may not generalize to children meeting ODD criteria who do not present with other disruptive behavior symptoms. However, our sample is reflective of children who present in many clinical settings, including mental health agencies, primary care settings, and psychological practices, as ADHD symptoms often co-occur with ODD behaviors. In addition, much of the previous research on assessment of ODD has involved samples of children with ADHD characteristics (e.g., Biederman et al., 2008; Drabick et al., 2007), and it has been suggested that pure ODD is relatively rare (van Lier et al., 2003). Finally, the ODD research diagnosis was not equivalent to a clinical diagnosis as other diagnoses (e.g., Pervasive Developmental Disorder) may have accounted for the ODD symptoms reported. Thus, the current results may differ from findings based on clinically diagnosed ODD. Summary of Contributions of the Present Study Overall, the present study expanded on Hommersen et al. s (2006) initial examination of the reliability and validity of the ODDRS by expanding the parent version of the ODDRS to a version for teachers to be used in the school setting. Furthermore, this study was able to investigate the clinical utility of the ODDRS in a true clinical setting, in the context of a university-affiliated ADHD assessment clinic. Results suggest that the ODDRS is useful as a screening measure for ODD, particularly in settings in which use of broadbased behavior ratings such as the BASC-2 or the CBCL is not possible. In addition the newly developed teacher ODDRS offers an opportunity for practitioners to quickly and efficiently determine whether a child is exhibiting oppositional behavior in more than one setting, which can help to inform treatment decisions. Declaration of Conflicting Interests The authors declared no potential conflicts of interest with respect to the authorship or publication of this article. Funding The authors received no financial support for the research and/or authorship of this article. References Abikoff, H., Courtney, M., Pelham, W. E., & Koplewicz, H. S. (1993). Teachers ratings of disruptive behaviors: The influence of halo effects. Journal of Abnormal Child Psychology, 21(5), 519-533. Achenbach, T. M., & Rescorla, L. A. (2001). Manual for ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families. American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: American Psychiatric Association. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., rev.). Washington, DC: American Psychiatric Association. Biederman, J., Ball, S., Monuteaux, M., Kaiser, R., & Faraone, S. (2008, July). CBCL Clinical Scales discriminate ADHD youth with structured-interview derived diagnosis of oppositional defiant disorder (ODD). Journal of Attention Disorders, 12(1), 76-82. Drabick, D., Gadow, K. D., & Loney, J. (2007). Source-specific Oppositional Defiant Disorder: Comorbidity and risk factors in referred elementary schoolboys. Journal of the American Academy of Child and Adolescent Psychiatry, 46(1), 92-101. DuPaul, G. J., Power, T. J., Anastopoulos, A. D., & Reid, R. (1998). ADHD Rating Scale-IV: Checklists, norms, and clinical interpretation. New York: Guilford Press. Eiraldi, R. B., Power, T. J., & Nezu, C. M. (1997). Patterns of comorbidity associated with subtypes of Attention-Deficit/ Hyperactivity Disorder among 6- to 12-year-old children. Journal of the American Academy of Child and Adolescent Psychiatry, 36(4), 503-514. Eiraldi, R. B., Power, T. J., Karustis, J. L., & Goldstein, S. G. (2000). Assessing ADHD and comorbid disorders in children: the child behavior checklist and the Devereux Scales of mental disorders. Journal of Clinical Child Psychology, 29(1), 3-16. Frick, P. J., Lahey, B. B., Applegate, B., Kerdyck, L., Ollendick, T., & Hynd, G. W., et al. (1994). DSM-IV Field Trials for the Disruptive Behavior Disorders: Symptom Utility Estimates. Journal of the American Academy of Child and Adolescent Psychiatry, 33(4), 529-539.

O Laughlin et al. 255 Hommersen, P., Murray, C., Ohan, J. L., & Johnson, C. (2006). Oppositional Defiant Disorder Rating Scale: Preliminary evidence of reliability and validity. Journal of Behavioral and Emotional Disorders, 14, 118-125. Nolan, E. E., Gadow, K. D., & Sprafkin, J. (2001). Teacher Reports of DSM-IV ADHD, ODD, and CD symptoms in schoolchildren. Journal of the American Academy of Child and Adolescent Psychiatry, 40(2), 241-249. Owens, J., & Hoza, B. (2003). Diagnostic utility of the DSM-IV- TR symptoms in the prediction of DSM-IV-TR ADHD subtypes and ODD. Journal of Attention Disorders, 7(1), 11-27. Pelham, W. E. (2006). Structured Disruptive Behavior Disorders parent interview. Retrieved January 17, 2010 from http://ccf. buffalo.edu/pdf/dbd_interiew.pdf Pelham, W., Gnagy, E., Greensledge, K., & Milich, R. (1992). Teacher ratings of the DSM-III-R symptoms for the disruptive behavior disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 31(2), 210-218. Pillow, E. R., Pelham, W. E., Hoza, B., Molina, B., & Stultz, C. (1998). Confirmatory factor analyses examining attention deficit hyperactivity disorder symptoms and other childhood disruptive behaviors. Journal of Abnormal Child Psychology, 26(4), 293-309. Reynolds, C. R., & Kamphaus, R. W. (2004). Behavior Assessment System for Children: Second edition manual. Circle Pines, MN: AGS Publishing. Searight, H. R., Nahlik, J. E., & Campbell, D. C. (1995). Attention-Deficit/Hyperactivity. Disorder: Assessment, diagnosis, and management. Journal of Family Practice, 40(3), 270-279. Smith, S. R. (2007). Making sense of multiple informants in child and adolescent psychopathology: a guide for clinicians. Journal of Psychoeducational Assessment, 25, 139-149. van Lier, P., Verhulst, F. C., van der Ende, J., & Crijnen, A. (2003). Classes of disruptive behavior in a sample of young elementary school children. Journal of Child Psychology and Psychiatry, 44(3), 377-387. Waldman, I. D., & Lilienfeld, S. O. (1991). Diagnostic efficacy of symptoms for Oppositional Defiant Disorder and Attention- Deficit Hyperactivity Disorder. Journal of Consulting and Clinical Psychology, 59(5), 732-738. Wolraich, M., Lambert, W., Doffing, M., Bickman, L., Simmons, T., & Worley, K. (2003). Psychometric properties of the Vanderbilt ADHD Diagnostic Parent Rating Scale in a referred population. Journal of Pediatric Psychology, 28(8), 559-568. About the Authors Elizabeth M. O Laughlin, PhD, is an associate professor in psychology at Indiana State University. Her research interests include assessment and treatment of ADHD in children and parenting interventions. Jessica L. Hackenberg, PsyD, is currently working as a pediatric psychologist at Akron Children s Hospital. Her research interests include the health-related quality of life in children with Cerebral Palsy and pediatric procedural anxiety related to electroencephalogram (EEG). Maria M. Riccardi, MS, is working on her doctorate in clinical psychology at Indiana State University.