Three Assessments for School- Aged Children- McAndrew 1. Conners 3 rd Edition

Transcription

1 Three Assessments for School- Aged Children- McAndrew 1 Conners 3 rd Edition Purpose of Instrument The Conners 3 rd Edition is an instrument to assess children and teens ages Teachers, parents, and the child report the frequency of a number of behaviors. Its main focus is on assessing attention deficit hyperactivity disorders (ADHD), but it includes scales for commonly co-morbid childhood conditions such as oppositional defiant disorder (ODD), and conduct disorder (CD). It has scales for DMS-IV symptoms, which can be useful in supporting diagnoses. The Conners 3 is also useful in the school setting in making decisions about appropriate special education eligibility, and interventions. It can be used to monitor and evaluate interventions relating to ADHD, and has multiple forms for different situations. Selection and Use Conners assessment was first developed in the 1960s, so it has been in use for quite some time. During this time it has gone through 2 revisions. Over the past 2 decades, it has been studied and/or used in more than 50 research studies; at least 6 have been done in different cultures. As described in the testing manual, this 3 rd edition went through a 3-phase process: initial planning, pilot study, and normative study (Conners, 2008). The test package includes a full form, a short form and 10 question ADHD index (3AI) each with a report for parents and teachers, and a self-report. There is another 10-question index for teachers and parents that assesses global functioning for psychopathologies (3AG). The ADHD Index can be administered to a group as a way to screen for ADHD, and be used for follow-up assessments as needed. The parent and self-report forms are available in Spanish. The self-report forms require a 3 rd -5 th grade reading level; parent and teacher forms are at about a 5 th grade reading level.

2 Three Assessments for School- Aged Children- McAndrew 2 It is important to use the age and gender norms because there are different norms for a 6- year old boy and a 16-year old girl. Boys were found to have higher ratings over all, and there were more hyperactive symptoms for younger children. Quality of Test Internal consistency was.90 or above for parents and teacher scales, and.85 or above for self-report scales (Arffa, 2010). There was good agreement between 2 parents or 2 teachers with an average reliability of.74 to.94. The test-retest reliability was highest for parents (.72-98), then teachers ( ), and least reliable were self-reports ( ) though the content scales were more reliable overall than the symptom scales (Conners, 2008). Conners 3 is convergent with earlier Conners versions, the Behavior Assessment System for Children- 2 nd Ed., the Achenbach System of Empirically Based Assessment, and the Behavioral Rating Inventory of Executive Functioning. Through factor analyses that created scale structure validity, construct validity was established. The test distinguishes between youth with ADHD and those in the general population; it also discriminates between ADHD subtypes, other disruptive disorders and learning disorders (Arffa, 2010). The test also has response bias scales in case a reporter is positive or negatively biasing their responses or responding randomly. Quality of Test Manual The manual is well written and user friendly, according to Dunn (2010). It includes how to administer and score the test, case studies, a description of how the test was made, copies of the test, and copious amounts of data. Administration and Scoring The test takes about 20 minutes for around 100 questions. The short form is just 10 minutes for about 40 questions. It can be taken on-line, scores can be entered into the Conners 3

3 Three Assessments for School- Aged Children- McAndrew 3 software program that can generate reports, or the assessment can be hand scored by a paraprofessional though they need to be overseen by a qualified assessor. When scoring by hand numerical answers are transferred from the QuikScore form to a grid, and summed by scale. This is transferred to a T-score using gender and age category norms. Interpretation and Adequacy of Test Norms The manual describes a 6-step process for interpretation. First, assess validity checking for response bias, make sure student was within the age range, and not new to North America where the test was normed. Second, review the content and DSM-IV symptom scale scores, the key part of the assessment. Third, examine scores overall, and look at impairment items since impairment is a key feature of clinical or educational labels. This step also involves looking at the 3AI index (whether the student is more like a student with ADHD or a student in the general population), and 3GI index (indicates psychological difficulty). The fourth step involves looking at the assessment on an item level, especially scales that are elevated. Looking at questions that were rated with a 3, the highest score, can help zero in on issues, and possible interventions. Several items might flag concern of depression or anxiety or, severe conduct, and need to be assessed. This is a good time to look at any comments left by raters. The next step is to consolidate results between raters and with other sources of information. Finally, results are reported (Conners, 2008). The normative study included 1,000 youth, and 1,200 parents and teachers in more than 100 different sites in North America (Dunn, 2010). The normative sample closely mirrored the ethnic and racial demographics of the 2000 census, and represented different parent education levels as well (Arffa, 2010). The manual itself has 6 pages which break down the normative sample by age, gender, and race/ethnicity so that if there was a question about the sample size of

4 Three Assessments for School- Aged Children- McAndrew 4 Asian boys aged 8 one could find that the normative sample included 1 such boy, though there were 50 boys aged 8, and 4 Asians who were 8 (Conners, 2008). Other important statistics while interpreting results is the standard error of measurement (SEM), and what constitutes reliable change both of which are included in the manual. SEM is reported for each content and symptom scale by gender and age band. Reliable change is quantified both for raw score and T- score in each scale. Assist with Counseling Goals The Conners 3 is helpful in assessing ADHD, and often co-occurring childhood conditions. Its many scales can be helpful in shaping interventions, and the assessment can be used to evaluate how interventions are working. It is useful because it gets information from 3 sources to get a better view of the child or adolescent. Behavior Assessment System for Children 2 nd Edition Purpose of Instrument Through parental, & teacher rating scales (PRS & TRS) and self-reports of personality (SRP) from children & adolescents aged 2-25 the Behavior Assessment System for Children (BASC-2) is designed to help classify emotional and behavioral disorders and aid in developing treatment plans, especially in educational settings, but also in research, forensics and clinical situations, though there is limited validity for the latter uses (Stein, 2007). This instrument also includes a structured developmental history, and student observation system; however, no psychometric data are provided for these two parts of the instrument, and some question their effectiveness (Stein, 2007). The BASC-2 does evaluate both positive or adaptive behaviors as well as negative behaviors. The broad age range is broken down into pre-school (ages 2-5),

5 Three Assessments for School- Aged Children- McAndrew 5 children (ages 6-11), adolescent (ages in high school), and college (ages 18-25). As the test manual states, college instruments are new with the BASC-2 and only available in self-report measurements (Reynolds, 2007). Selection and Use Generally this assessment is strong, especially within the age range from Younger ages are not as representative in regards to mother s education, and the college level tests are new and have limited assessment options. Another sub-group that would warrant caution when using this test, are those diagnosed with a mood disorder (Stein, 2007). Preschoolers tended to have the highest interrater reliabilities, so for non-preschoolers multiple elevated scores would be important. When using the parent report it would be important to know the parents because it does include a lot of information, some overly technical (Watson, 2007). Quality of Test The BASC-2 and the original BASC were constructed after reviewing literature and other behavior rating scales. Clinicians, educators and students were involved in the development process. Pilot groups, standardization testing and statistical analyses including factor analysis were used in producing the final scales (Watson, 2007). Internal consistency coefficients are generally high for all three measures, (Stein, 2007). Children had the highest median reliabilities on the PRS & TRS (.84 &.86) while college age had the highest on the SRP (.84). It is recommended to use composite scores for decision-making purposes because reliability coefficients are lower for individual scales than for composite scores (Stein, 2007). Concurrent validity is especially strong for externalizing behavior on the TRS & PRS ( ), and weaker for internalizing behavior ( ) as measured against the Achenbach System of Empirically Based Assessment (ASEBA). Both internalizing and externalizing rating

6 Three Assessments for School- Aged Children- McAndrew 6 scales showed high concurrency with the original Conners rating scales, except on the anxiety scale ( ) apparently due to different content (Stein, 2007). For the SRP, moderate to strong concurrency was shown with the ASEBA and Conners. When comparing to the Children s Depression Inventory, the children s SRP was not very reliable (.29) and adolescents moderately so (.69). Overall, the reliability of the SRP was not as strong as the TRS & PRS. The clinical scale and composite score profiles verify the construct validity though the variance for one group, those with mood disorders, is notable, likely due in part to a low N value (Stein, 2007). Another problem for the instrument is its low correlation between children s TRS and PRS composites (.48) and scales (.38) (Watson, 2007). The test does include validity checks to catch overly positive or negative responses, random responses, or responses based on misunderstandings of the test (Reynolds, 2004). Quality of Test Manual The manual is thorough, though it can be cumbersome due to its length (590 pages), and many tables. Watson recommends a users manual separate from the technical manual to make it easier to find what is needed for test administration and scoring (2007). Administration and Scoring The administration of the PRS, TRS or SRP takes between minutes, and involves scoring behaviors as Almost Always/Often/Sometimes/Never. Once finished with the rating scale, the form is separated to reveal the Item Scoring Page, Summary Table, and several indices. Checking validity indices is one of the first steps, then summing item scores, and computing raw scores. After choosing which norm set to use, the T scores are found in the manual s appendices along with a value that can be added (and subtracted) to find the 90% confidence interval. This is done for each individual scale and then scales are added to find composite scores. There is a

7 Three Assessments for School- Aged Children- McAndrew 7 table to check if the difference between 2 composites is statistically significant at the.05 level. Finally, information can be graphed, and the list of critical items examined (Reynolds, 2007). Interpretation and Adequacy of Test Norms Both the general and clinical normative samples were broken down by age and gender. The general group included over 13,000 subjects aged 2-18 tested between , and closely reflects US census demographics in terms of race, ethnicity, and geographic locations. The clinical subjects were identified by their parents as having a diagnosis, but this group was less representative of the US in general. This clinical group included 1,779 children aged 4-18, and had more males at every age range. There were 2 groups for subjects above 18: a clinical norm group for ages (in high school, mostly in special education), and 706 college students (ages 18-25) for whom there is little information. (Stein, 2007) The manual includes an interpretative guide for each scale and composite that identifies high and low scores. It also gives some background information about each of the scales and composites so that if a subject scores high in the learning problems area it describes that this area is reported on the TRS, and that 25-30% of children with ADHD also have one or more learning disabilities (Reynolds, 2007). For final interpretation it is recommended to look first at the composite scores, then scale scores, and finally at item level (Reynolds, 2007). Assist with Counseling Goals The BASC-2 is only a one-point-in-time measure which is not adequate for diagnosing on it s own (Watson, 2007), but would be good in corroborating what teachers, parents and students are noticing. It could be one piece in determining what problems a youth might be struggling with, and how best to treat those problems. Due to the very large scope of problems that the BASC-2 reports on, this test would be helpful in focusing further testing or interventions.

8 Three Assessments for School- Aged Children- McAndrew 8 Woodcock-Johnson(r) III Purpose of Instrument This cognitive assessment was standardized on people from age 2 to age 90, and might be used in schools, clinics, or research settings. It s most applicable to school-aged subjects (Sandoval, 2003), and its uses include diagnosis, educational planning, vocational guidance, assessing growth, and program evaluation (Woodcock, 2001). It contains 2 instruments that share the same normative sample, the Woodcock-Johnson III Tests of Cognitive Ability (WS III COG), and the Woodcock-Johnson III Test of Achievement (WS III ACH) which describes more academic strengths and weaknesses. Each of these 2 instruments are broken down into the standard or extended batteries containing about 10 subtests each. The instrument can be administered as a whole, or as individual subtests depending on the situation (Cizek, 2003) though there is greater validity in score interpretation based on clusters of tests (Woodcock, 2001). For instance the WS III ACH Standard Battery has 12 subtests that produce 10 cluster scores along with a Total Achievement Score making the assessment flexible to what is needed. Selection and Use The WJ III must be considered the premier battery for measuring both the cognitive abilities and school achievement of school-aged children and young adults, (Sandoval, 2003). While the sampling techniques for subjects out of school are not as strong, the school-aged normative sample leaves little room for hesitation in using this instrument. Yet a few remain. In discussions of validity, the test describes that children with ADHD followed expected patterns, but does not mention gifted children or children with mental retardation leaving one to wonder where they fit in the norming sample (Sandoval, 2003). A large source of validity comes from an internal structure that was analyzed to link with the Cattell-Horn and Carroll (CHC)

9 Three Assessments for School- Aged Children- McAndrew 9 conceptualization of cognitive abilities, but analysis reports did not include children younger than six (Sandoval, 2003). If student had taken another standardized cognitive test, one would think before administering WS III COG because scores are typically lower than on other measures. The score range is larger going as high as 300 (Sandoval, 2003). It is good that the norming groups were the same for WJ III COG and WJ III ACH. There is increased confidence when looking at discrepancies between cognitive abilities and achievement because the same norming group was used for the WJ III COG and WJ III ACH (Cizek, 2003). Also, bias reviews were conducted and items that were potentially biased were removed from the test or revised (Cizek, 2003). Quality of Test Internal consistency reliability estimates appear to be uniformly high, with single test ranging from s, and clusters being in the.90s (Cizek, 2003). These were calculated using Rasch approaches for speeded tests, and traditional split-half reliabilities for non-speeded tests. Stability reliability was also measured on select tests with time intervals up to 10 years. The median reliabilities mostly ranged from s. The WJ III ACH has 2 forms (A and B) which have been shown to be equivalent (Sandoval, 2003). The internal structure was supported by confirmatory factor analyses that showed that correlations were higher between tests that would be expected to have more similar scores than amongst tests with differing skill sets. Validity was partially based on factor analyses of tests that showed consistency with the CHC model of cognitive ability. Also, items are similar to other cognitive or achievement tests, or based on sound experimentation (Sandoval, 2003). The growth of cluster scores at various ages matches what one would expect with steep growth from age 5 to 25, but not afterwards.

10 Three Assessments for School- Aged Children- McAndrew 10 Quality of Test Manual The test manual includes an examiner s manual for the WJ III COG and WJ III ACH, and a technical manual. This is a lot of manual to sort through, but if all one needs is the user manual for the cognitive test they don t have to carry around the rest. The manual is easy to understand. In fact the scoring guide for the writing tests is included in the WJ III ACH examiner s manual, and the interrater correlations were reported in the upper.90s (Cizek, 2003). The Standards for Educational and Psychological Testing are included throughout the manual as professional reminders such as, In testing individuals with disabilities for diagnostic and intervention purposes, the test should not be used as the sole indicator of the test taker s functioning. Instead, multiple sources of information should be used, (Woodcock, 2003). Administration and Scoring The test is administered using easel binders that include directions for the examiner, and form a barrier for the examiner to write behind. There are four binders (one for each standard and extended battery of cognitive and achievement), audio cassettes and a test booklet needed for some tests. A complete battery will take an experienced tester about minutes, but the most difficult part is finding the floor and ceiling levels (Sandoval, 2003). Most tests are untimed, but a few do measure speed and fluency. Similar to the Wechsler Adult Intelligence Scale-III, the WJ III alternates types of tests- timed/untimed, and content. There are testing accommodations for answers given in another language and for students with handicaps, but the practicality and validity has not been established, (Sandoval, 2003). Scoring cannot be done by hand but the software program, Compuscorer, is used which can print in Spanish and adjust confidence bands based on the type of score and norm chosen (Cizek, 2003). Interpretation and Adequacy of Test Norms

11 Three Assessments for School- Aged Children- McAndrew 11 The 2 instruments (COG and ACH) were normed on the same group of 8,818 individuals matched to US demographics as regards gender, race, Hispanic origin, geographic location, community size, and type of school or university. If the sample size was not representative weighting was used to be a more accurate representation, however, no weighting was extreme (Sandoval, 2003). For school-age subjects a three-stage sampling procedure was used (community, schools within community, and students within schools) while at the same time including public, private, and home-schooled students. Also, included were students with disabilities and English language learners. Weaknesses appeared in the adult sampling with college-aged subjects being recruited from bulletin boards, and adult sampling is not described in enough detail to know if sampling would be considered representative (Cizek, 2003). Assist with Counseling Goals This test is designed to measure cognitive ability, achievement, and discrepancies between these or other smaller clusters that make up cognitive ability and achievement. It would be useful in communicating these abilities and discrepancies to parents and teachers, setting realistic counseling goals, and planning interventions. The WJ III ACH includes scores for basic interpersonal communication skills (BICS), and cognitive-academic language proficiency (CALP) which are helpful in assessing student struggles in light of English not being their primary language (Cizek, 2003).

12 Three Assessments for School- Aged Children- McAndrew 12 References Arffa, Sharon. (2010) [Review of Conners 3 rd Edition]. In K.F. Geisinger, R. A. Spies, & J. F. Carlson (Eds.), The eighteenth mental measurements yearbook [Electronic version]. Cizek, Gregory J. (2003) [Review of Woodcock Johnson(r) III]. In B. Plake, J. C. Impara & R. Spies (Eds.), The fifteenth mental measurements yearbook [Electronic version]. Conners, C. Keith. (2008). Conners 3 rd edition. Toronto, ON: Multi-Health Systems Inc. Dunn, Thomas M. (2010) [Review of Conners 3 rd Edition]. In K.F. Geisinger, R. A. Spies, & J. F. Carlson (Eds.), The eighteenth mental measurements yearbook [Electronic version]. Reynolds, Cecil R., & Kamphaus, Randy W. (2004). BASC-2: Behavior Assessment System for Children, Second Edition. Circle Pines, MN: AGS Publishing. Sandoval, Jonathan. (2003) [Review of Woodcock Johnson(r) III]. In B. Plake, J. C. Impara & R. Spies (Eds.), The fifteenth mental measurements yearbook [Electronic version]. Stein, Stephanie. (2007) [Review of Behavior Assessment System for Children 2 nd Edition]. In K. F. Geisinger, R. A. Spies, J. F. Carlson, & B. S. Plake (Eds.), The seventeenth mental measurements yearbook [Electronic version]. Watson, T. Steuart, & Wickstrom, Katherine. (2007) [Review of the Behavior Assessment System for Children]. In K. F. Geisinger, R. A. Spies, J. F. Carlson, & B. S. Plake (Eds.), The seventeenth mental measurements yearbook [Electronic version]. Woodcock, R.W., McGrew, K.S., & Mather, N. (2001). Woodcock-Johnson III. Rolling Meadows, IL: Riverside Publishing.