Psychometric Properties of the Children's Atypical Development Scale

Transcription

1 Journal of Abnormal Child Psychology, Vol. 22, No. 2, 1994 Psychometric Properties of the Children's Atypical Development Scale Mark A. Stein, 1,3 Emily Szumowski, 1 Ron Sandoval, 1 David Nadelman, 2 Tara O'Brien, 1 Matt Krasowski, 1 and Warren Phillips I The Children's Atypical Development Scale (CADS) is a 53-item rating scale designed to measure unusual behaviors in children. Principal-factor analysis on a clinic-referred and pediatric sample of 474 children resulted in a four-factor solution: Communication Deficits, Lability, Social Relatedness Deficits, and Preoccupation. The CADS is internally consistent and has adequate temporal stability. CADS factor scores were differentially associated with parent and teacher rating scales, IQ, and Continuous Performance Test errors. The scale shows promise as a clinical and research tool for assessing atypical behaviors associated with pervasive developmental disorder and other neurobehavioral disorders. Odd or unusual childhood behaviors are associated with a variety of childhood disorders, including pervasive developmental disorder (PDD), Asperger's syndrome and multiple complex developmental disorder (MCDD) (Towbin, Dykens, Pearson, & Cohen, 1993). Although a number of rating scales and questionnaires are available which assess behaviors associated with autism (e.g., Childhood Autism Rating Scale, Abberent Behavior Scale), these scales may be less applicable for higher-functioning children (Rutter & Schopler, 1988). On the other hand, scales frequently used with nondevelopmentally delayed children to assess learning or behavior problems (e.g., Conners Parent or Teacher Rating Scale, ACTeRS) do not adequately sample odd or unusual behaviors. To address these limitations, Dinklage and Guevremont recently developed the Children's Atypical De- Manuscript received in final form August 10, This research was supported in part by a grant from the Smart Family Foundation. The authors are grateful to James P. O'Donnell, Catherine Lord, and Frank A. Zelko for their comments on an earlier draft. 1Departments of Psychiatry and Pediatrics, University of Chicago, Chicago, Illinois North Suburban Clinic, Hoffman Estates, Illinois Address all correspondence to Dr. Mark A. Stein, Section of Child and Adolescent Psychiatry, University of Chicago, 5840 S. Maryland Ave., Chicago, Illinois /94/ / Plenum Publishing Corporation

2 168 Stein et al. velopment Scale (CADS; see Barkley, 1990, pp ). The CADS appears to be unique in that it utilizes a relatively brief questionnaire format designed to be completed by parents and to sample unusual behaviors observed in higher-functioning children. The purpose of the present series of studies is to explore the psychometric properties of the CADS. Instrument Children's Atypical Development Scale. The CADS is a 53-item parent rating scale designed to measure a variety of unusual behaviors. The item content was generated by four experienced clinicians who compiled an extensive list of unusual behaviors based on Cohen, Paul, and Volkmar (1986) and the medical records of 40 children diagnosed with PDD (pervasive developmental disorder) (personal communication, Barkley, July 1993). The CADS was then completed by parents of 23 children clinically diagnosed with PDD using DSM-IIIR criteria (American Psychiatric Association, 1987) and 20 children with attention deficit hyperactivity disorder (ADHD). Both groups had normal intelligence and were referred to the same academic medical center. Two control groups of children with learning disabilities (LD) (n = 12) obtained from an educational clinic and normals (n -- 12) were also added. Parents were asked to rate each item on a 3- point scale. CADS items were most characteristic of the PDD group, endorsed to a lesser extent by parents of ADHD children, and rarely reported for the LD and normal control groups (Barkley, 1990). STUDY 1: FACTOR STRUCTURE OF THE CADS Overview The purpose of Study 1 was to explore the factor structure of the CADS and to examine the relationship of CADS factor scores with major demographic variables using a sample of children with predominantly disruptive behavior disorders and pediatric controls. Subjects and Setting The base sample of 474 children was composed of 381 children conseeutively referred to a child psychiatry clinic specializing in attention and learning problems and 93 pediatric outpatients not referred nor currently in treatment for psychiatric or learning problems. There were 364 boys and

3 Psychometric Properties of the CADS girls from ages 4 to 16 (M = 8.7, SD = 3.1). DSM-III-R diagnoses based upon multidisciplinary team consensus after semistructured interviews with the parents and children, psychological testing, and physical exams were obtained for 335 of the 381 psychiatric-referred children. Of the psychiatric-referred children, 59% (n = 220) met criteria for ADHD, which was the most common diagnosis, 36% (n = 135) had oppositional defiant or conduct disorder, 8% (n = 30) had an anxiety disorder, and 8% (n = 30) were diagnosed with PDD. There was a high degree of comorbidity, with the majority of children meeting criteria for two or more Axis I disorders. Although there was a wide range in socioeconomic status (SES), the vast majority of the families were from Hollingshead (1975) classes I, II, and III (M = 2.2, SD = 1.0). Statistical Analysis To examine the factor structure of the CADS, exploratory factor-analytic procedures, following the recommendations of Cattell (1988) and Comrey (1988) were employed. Principal-factor analysis (PFA) with squared multiple correlations as communality estimates were conducted using oblique rotation criteria (direct quartimin). The number of factors to extract was determined on the basis of examination of the screen plot and eigenvalues greater than 1. Factor loadings for each item were required to be above.40. Results Factor Analysis. PFA resulted in a four-factor solution which accounted for 73% of the variance (see Table I). The first factor, Communication Deficits, accounted for 54% of the variance and contained eight items reflecting deficits in expressive and receptive interpersonal communication. The second factor, Lability, accounted for 9% of the variance and included five items. This factor is defined by extreme mood lability and explosiveness, with an absence of concern for, or awareness of, social appropriateness. The third factor, Social Relatedness Deficits, accounted for 6% of the variance and contained seven items reflecting lack of interest in social interactions and withdrawal. A fourth, minor factor, Preoccupation, accounted for 4% of the variance and is defined by two items reflecting fantasizing and rigidity. These four factors correspond closely with the DSM-III-R criteria for pervasive developmental disorder.

4 170 Stein et al. Table I. CADS Factor Loadings (N = 474) a Item number and content Factors 1 II III IV I, Communication Deficits 23. Desen'bes the details of an event but misses the meaning "Misses the point" or main idea in conversation Rambling speech.7..~ Confuses causes of events.6"/ Confuses sequence of events Makes irrelevant comments Speaks without concern about being understood.5.._ Seems to be extremely naive II. 38. Lability Gets angry for little apparent reason Extreme reactions to inconvenience Mood changes quickly without apparent reason.05.6._ Preoccupied with violent stories Overreacts to pain Excessively preoccupied with violence III. Social Relatedness Deficits 19. Lacks interest in peers _ Dislikes being held or touched Does not respond to initiations of other children _ Makes poor eye contact with others _ Overly suspicious Poor judge of other people's reactions Lacks compassion when others are hurt _.! -.14 IV. 44. Preoccupation Spends time fantasizing _ Obsessive interest in narrow or atypical topic _..0 Eigenvalue % Total variance aunderlined values indicate significant factor Ioadings % 9% 6% 4% Relationships with Demographic Variables No gender differences were found on the four CADS factors, or for the total CADS score of all 53 items. Pearson correlation coefficients between age and CADS scores were computed for the total sample and for boys and girls separately. There were significant correlations between age and Social Relatedness Deficits for girls (r =.32, p <.001), and for the total sample (r =.13, p <.01). All other correlations with age were nonsignificant, raging from -.04 to.06 for boys and.05 to.10 for girls. Since the percentage of variance accounted for by age on each score was very small (0 to 2%), separate age norms were deemed unnecessary.

5 Psychometric Properties of the CADS 171 A significant correlation was found between SES and Factor I (r =.21, p <.001), Factor II (r =.14, p <.05), and total CADS (r =.19, p <.01), indicating that parents from lower SES families were more likely to report more Communication Deficits, Lability, and higher total CADS scores. STUDY Ih RELIABILITY Overview The purpose of Study II was to examine the internal consistency and temporal stability of the CADS factors. Method The internal consistency of each CADS factor was assessed by Cronbach's alpha (Nunnally, 1978) using the sample employed in Study I. Data on test-retest reliability were based on 32 clinic-referred children seen during the past year whose mothers completed the CADS prior to the first appointment and again before the feedback session. There is typically a 2-week interval between the first appointment and feedback. However, due to the length of the waiting list, the time between administrations ranged from 2 weeks to 2 months, with an average interval of 30 days. Results Cronbach's alpha and 30-day test-retest reliabilities are reported in Table II. The first three CADS major factors were internally consistent with alphas ranging from.80 to.87. Scale IV, Preoccupied, was not internally consistent. However, estimates of internal consistency corrected for Table II. Reliability of CADS Factors and Total Score Number of Cronbach's CADS factor/score items a Test-retest Communication Deficits Lability Social Relatedness Preoccupation CADS total

6 172 Stein et al. scale length using the Spearman-Brown formula resulted in a corrected Cronbach'a alpha of.82 for the Preoccupied factor. Test-retest reliability for the total CADS score was high (r =.87, p <.001), while individual factor scores were moderately stable, with Pearson correlations ranging from.61 to.80. STUDY III: CONCURRENT VALIDITY Overview Standardized parent and teacher rating scales and individually administered measures of intelligence, attention, and impulsivity served as criterion variables for the concurrent validation of CADS factor scores. We hypothesized that CADS factor scores would be differentially associated with criterion measures reflecting socialization deficits, IQ, and impulsivity and internalizing symptoms such as anxiety and thought disturbance. For example, we expected that the Communication Deficits and Social Relatedness Deficits factors would relate most strongly to ratings of poor social skills, social withdrawal, and anxiety, while the Lability factor would relate most strongly to impulsive, delinquent, aggressive, and oppositional behaviors as well as anxiety and withdrawal. Method Subjects were 219 clinic-referred children and 82 pediatric outpatients from Study I foi" whom complete data on the validity measures were available. (For pediatric patients, younger children, and those evaluated during the summer, teacher ratings were unavailable.) In addition to the CADS, parents completed the Child Behavior Checklist (CBCL; Achenbach & Edelbrock, 1983) and the Conners Parent Rating Scale (CPRS; Goyette, Conners, & Ulrich, 1978). The child's primary teacher completed the ADD- H Comprehensive Teacher Rating Scale (ACTeRS; Ullman, Sleator, & Sprague, 1985) and the Teacher Report Form (TRF; Achenbach & Edelbrock, 1986). As a measure of IQ, children were administered the WISC-R (Wechsler, 1974) or the WPPSI-R (Wr 1989). Each child was also administered a computerized version of the Continuous Performance Test (CPT; Conners, 1985) which involved the visual presentation of 50 "X" targets interspersed with other letters presented in random order. Commission and omission errors derived from the CPT are indices of impulsivity and distractibility, respectively.

7 Psychometric Properties of the CADS 173 Results CADS scores correlated significantly with the parent report measures, including virtually all the subscales on Conners Parent Rating Scale, as well as on both the Internalizing and Externalizing scales of the CBCL. Correlations ranged from.38 to.50 for Internalizing and.21 to.63 for Externalizing subscales (see Table III). Correlations between CADS scores and measures not based on parental report showed a more variable pattern (see Table IV). High scores on the CADS Communication Deficits factor were associated with higher teacher ratings of social skills deficits, withdrawal, anxiety/depression, and lower IQ. A different pattern was found for Lability, which correlated with teacher reports of both externalizing and internalizing problems, including anxiety/depression, delinquent behavior, and thought problems, and with impulsivity on the CPT. High scores on the Social Relatedness Deficits factor were significantly associated with social skill deficits, withdrawal, anxiety/depression, and thought problems, but not with measures of IQ or attention, while high scores on Preoccupation were associated with internalizing problems such as thought problems, withdrawal, and somatic complaints. Finally, total CADS was associated with anxiety, social skill deficits, withdrawal, thought problems, delinquency, and lower IQ. Table III. Correlations of CADS Factor Scores with Parent Rating Scales = CADS factor Variable I II III IV Total CPRS Conduct Problems CPRS Learning Problem CPRS Psychosomatic.16 b.18 b CPRS Impulsive/Hyperactive CPRS Anxiety.11, n.s..24 b CPRS Hyperactivity Index CBCL Withdrawal.17 b.32 CBCL Somatic Complaints.15 b.19 b CBCL Anxious/Depressed CBCL Social Problems CBCL Thought Problems CBCL Attention Problems CBCL Delinquent Behavior CBCL Aggressive Behavior , n.s b b.13, n.s ~' b.19 b b.05, n.s..16 b b b.60 =Factor I = Communication Deficits; II = Lability; IIl ffi Social Relatedness Deficits; IV = Preoccupation; CADS = total Child Atypical Development Scale score; CPRS = Conners Parent Rating Scale; CBCL = Child Behavior Checklist. Unless otherwise noted, all correlations are significant at p <.001, two tailed. bp <.05, two-tailed.

8 174 Stein et ai. Table IV. Correlations (N = 219) of CADS Factor Scores with Teacher Ratings and Cognitive Measures* CADS factor Variable I II III IV CADS ACTeRS Attention ACTeRS Hyperactivity b ACTeRS Social Skills -.20 b -,18 b -.29 c c ACTeRS Oppositional c ACTeRS Withdrawn '.20 c.22 c.19 b TRF Somatic Complaints b.14 TRF Anxious/Depressed.16 b.29 c.16 b c TRF Social Problems.22 c.22 c.19 b c TRF Thought Problems c.19 b.25 c.32 c TRF Attention Problems TRF Delinquent Behavior c.15 b c TRF Aggressive Behavior c Full-Scale IQ -.30 c b CPT Commission Errors b CPT Omission Errors *Factor I = Communication Deficits; II = Lability; III = Social Relatedness Deficits; IV = Preoccupation; CADS = total Child Atypical Development Scale score; ACTeRs = ADD-H Comprehensive Teacher Rating Scale; TRF = Child Behavior Checklist--Teacher Report Form; CPT = Continuous Performance Test. bp <.05 Cp <.ol DISCUSSION Three studies were conducted to explore the factor structure of the CADS and to evaluate the reliability and concurrent validity of the scale. The four CADS factors found reflect the major characteristics contained in the DSM-III-R criteria for pervasive development disorders: qualitative impairment in reciprocal social interaction (Social Relatedness Deficits), impairments in communication and imaginative activity (Communication Deficits), and markedly restricted repertoire of activities and interests (Preoccupations). Lability corresponds with the "associated feature" of abnormalities of mood regulation. There is a strong correspondence between the statistically derived CADS factors and the clinically derived DSM-III-R criteria for PDD. The total CADS score and Communications Deficits and Social Relatedness Deficits factors appear to demonstrate satisfactory levels of internal consistency and test-retest reliability. The least temporally stable factor, Lability, by definition reflects dramatic changes in mood and behavior. Consequently, modest test-retest reliability may be a reflection of variability in the underlying construct rather than inconsistencies in accu-

9 Psychometric Properties of the CADS 175 racy of reporting. The fourth factor, Preoccupations, contains only two items. In future revisions of the CADS, this factor would benefit from the inclusion of additional items to increase its reliability. Reflecting primarily source variance, CADS scores correlated highly with all parent report measures. Among the teacher ratings, total CADS scores were most highly correlated with scores on factors reflecting anxiety/depression, thought problems, and social problems. The CADS factor scores demonstrated differential associations with criterion measures which largely corresponded to anticipated patterns. Both IQ and SES were inversely associated with total CADS and Communication Deficits, but not with the other factors. In general, parents of children from lower SES families were likely to report more communication problems and unusual behaviors. Our sample was characterized by a preponderance of boys with disruptive behavior disorders and average IQ. Since the obtained factor structure may be dependent on the characteristics of our sample, caution should be used in extrapolating to samples that differ significantly in demographic characteristics or presenting problems. In summary, the CADS appears to be a useful rating scale for assessing unusual behaviors. Potential uses include: screening for unusual behaviors to alert clinicians to consider a wider range of diagnoses; description and quantification of unusual behaviors; and as a pre- and postintervention measure. At present, clinicians should consider potential limitations or bias in parent reports, and should utilize information obtained from other sources (e.g., teacher ratings, psychometric testing, mental status examination) to confirm diagnostic hypotheses. Future research with the scale should be directed toward determining the ability of the CADS for distinguishing PDD and other "autistic-like" disorders from autism, ad from other psychiatric and learning disorders whose symptoms often overlap. REFERENCES Achenbach, T. M., & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington: University of Vermont. Achenbach, T. M., & Edelbrock, C. (1986). Manual for the Teacher's Report Form and Teacher Version of the Child Behavior Profile. Burlington: University of Vermont. American Psychiatric Association (1987). Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev.). Washington, DC: Author. Barkley, R. A. (1990). Attention deficit hyperactivity disorder: a handbook for diagnosis and treatment: New York: Guilford Press.

10 176 Stein et al. Cattell, R. B. (1988). The meaning and strategic use of factor analysis. In J. R. Nesselroade & R. B. Cattell (Eds.), Handbook of multivariate experimental psychology (3rd ed.). New York: Plenum Press. Cohen, D. J., Paul, R., & Volkmar, F. R. (1986). Issues in the classification of pervasive and other developmental disorders: Toward DSM-IV. Journal of the American Academy of Child Psychiatry, 25, Comrey, A. L. (1988). Factor-analytic methods of scale development in personality and clinical psychology. Journal of Consulting and Clinical Psychology, 56, Conners, C. K. (1985). The computerized continuous performance test. Psychopharmacology Bulletin, 21, Goyette, C. H., Conners, C. K., & Ulrich, R. F. (1978). Normative data for revised Conners parent and teacher rating scales. Journal of Abnormal Child Psychology, 6, Hollingshead, A. B. (1975). Four factor index of Social Status. New Haven, CT: Yale University Press. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-HilL Rutter, M., & Schopler, E. (1988). Autism and pervasive developmental disorders. In M. Ratter, A. Tuma, & I. Lann (Eds.), Assessment and diagnosis in child psychopathology. New York: Guilford Press. Towbin, K. F., Dykens, E. M., Pearson, G., & Cohen, D. J. (1993). Conceptualizing "borderline syndromes of childhood" and "childhood schizophrenia" as a developmental disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 32, Ullmann, R. K., Sleator, E. K., & Sprague, R. L. (1985). A new rating scale for diagnosis and monitoring of ADD children. Psychopharmacology Bulletin, 20, Wechsler, D. (1974). Manual for the Wechsler Intelligence Scale for Children-Revised. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1989). Manual for the Wechsler Preschool and Primary Scales of Intelligence. San Antonio, TX: The Psychological Corporation.