Test Framework. Affiliation Disclosure. Overview. Broad Overview. Reduced Testing Time. NAP Convention Presentation

Transcription

1 Affiliation Disclosure NAP Convention Presentation Dr. Drozdick is an employee of Pearson Not incented by kit sales Final Structure, Administration, and Scoring of the WISC V Lisa Drozdick, PhD WISC-V Research Director 1 2 Overview 1. WISC-V Test Framework 2. WISC-V Scores 3. Subtest Administration and Scoring 4. Completion of Record Form 5. Qinteractive 6. Final Q & A 3 Test Framework 1 4 Broad Overview A choice of traditional paper and pencil or digital format. 5-Factor structure 7-Subtest FSIQ Addition of Complementary subtests to measure additional cognitive processes Supports a processing strengths and weakness analysis approach to identification of SLD Statistically linked to the KTEA-3 and the WIAT-III, with combination analyses Basic training included with the kit Briefer instructions, using developmentally appropriate language Reduced Testing Time 5 primary index scores: 65 minutes mean (10 minutes shorter than WISC-V mean) FSIQ: 48 minutes mean (27 minutes shorter than WISC-V mean) Shorter discontinue rules, fewer items, selecting subtests with briefer administration time to contribute to these scores 5 1

2 WISC-V Traditional Format Paper/Pencil Digital Format on Q-interactive WISC-V Test Framework Scoring Options Handscore Q-global Scoring & Reporting Score Report Combination Reports Narrative Reports Automatic Scoring & Reporting via Q- interactive Similar score report output as those available on Q-global, plus: Automatic subtest scoring Immediate scaled scores Substitution and Proration = No More Core and Supplemental WISC-V Scores Only one substitution OR proration on FSIQ No substitution or prorations on any index score Less necessary with the expanded composite score options 2 10 Full Scale (FSIQ) Headline placed here Text here Primary Index Scores Headline placed here Text here VCI VSI FRI WMI PSI Perceptual Reasoning Index (PRI) Replaced 2

3 Ancillary Index Scores Complementary Index Scores Headline placed here Text here Headline placed here Text here QRI AWMI NVI GAI CPI On Record Form Analysis Pages and in Administration and Scoring Manual Supplement: Optional carry-along NSI STI SRI On Record Form Analysis Pages and in Administration and Scoring Manual Supplement: Optional carry-along Note: SRI = NSI + STI... SRI NSL + NSQ + IST + DST + RST Descriptive Classifications Composite Score Range WISC V Descriptive Classification Traditional Descriptive Classification ( Old ) 130 and above Extremely High Very Superior Very High Superior High Average High Average Average Average Low Average Low Average Very Low Borderline 69 and below Extremely Low Extremely Low Raw Process Scores Simple raw scores; not age referenced, convert to base rates 6 Longest Span and Sequence Scores (example: LDSf, LDSb, LDSs) 10 Error Scores (example: rotation errors on BD, CD, and SS, number of errors on Naming Speed Literacy) Interpretation on Naming Speed subtests, based only on time Process observations (e.g., Don t Know, No Response) Not on Record Form Appendix D in Technical and Interpretive Manual) 18 3

4 Process Observations Contrast Scores Provide information about performance on a task of interest in comparison to other children who scored at the same level on a related task 6 in total (example: DSF vs. DSB) Not on Record Form Appendix C in Technical and Interpretive Manual Subtest Administration and Scoring 3 22 Dropped WISC IV Subtests Word Reasoning Redundant measure of verbal comprehension (high correlation with Information) Picture Completion Construct not as representative of visual spatial ability as others (secondary verbal loading) And we needed the space for new subtests New Subtests to Extend Content Coverage and Clinical Utility Primary Subtests Visual Puzzles: Visual Spatial ability Figure Weights: Quantitative fluid reasoning Picture Span: Visual working memory Complementary Subtests Naming Speed Literacy Naming Speed Quantity Immediate Symbol Translation Delayed Symbol Translation Recognition Symbol Translation

5 Changes to Retained Verbal Comprehension Subtests Subtests: Similarities, Vocabulary, Information, Comprehension Modifications Revised scoring rules with data-based queries Reviewed vocabulary level (no more advantages ) New, contemporary item content Updated art with increased international portability * Why do people use passwords? Verbal Comprehension Subtest Admin and Scoring No significant changes to administration, just sample responses and queries and SHORTER DISCONTINUES (3 consecutive) What does 3 consecutive mean? A. numbered? or B. administered? Most frequent administration error on Wechsler intelligence protocols among experienced and novice examiners: Second most frequent administration error on Wechsler intelligence protocols: Third most frequent administration error on Verbal Comprehension subtests: *all items and item images are fake to protect the innocent Block Design Administration Block Design Scoring New diamond and X-shaped designs Process scores BDn BDp BDde BDre But first...memorizing the WISC-V (and WPPSI-IV) discontinue rules in 10 seconds or less Shortened discontinue Instructions slightly briefer Frequent admin errors Presenting blocks wrong (Moon, Blakey, Gorsuch, & Fantuzzo, 1991) Forgetting to start stopwatch Rotations How often are they corrected? When? Do I give credit if child rotates it to the correct orientation before the time limit would have expired? 27 changing time limit reminder Optional partial score column Recording new process scores 28 Visual Puzzles Administration Child views completed puzzle and selects three response options that combine to reconstruct the puzzle Item time limit of 30 seconds Measures ability to analyze and synthesize abstract information Visual Puzzles Scoring Which three pieces go together to make this puzzle? Most common admin error?: Common error alert! circle responses for later analysis

6 Matrix Reasoning Administration and Scoring Two item types retained and taught 2x2 matrix serial order New items Shortened and simplified discontinue Instructions slightly briefer Most frequent admin error?: Most frequent scoring error?: Figure Weights Administration Child views scale with missing weight(s) and selects the response option that balances the scale Item time limit of 20 or 30 seconds Measures quantitative fluid reasoning Which one of these weighs the same as this? Most frequent admin error?: Figure Weights Scoring Picture Concepts Administration and Scoring Common error alert! circle response changing time limit reminder after item Common error alert! circle responses for later analysis Images not reused New items Picture Concepts lore: Alternate responses penalize GT and creative children Most frequent admin Copyright 2014 Pearson errors?: Education, Inc. its affiliates. All rights reserved. 34 Arithmetic Administration and Scoring New items One repetition on difficult items, none on others Cannot repeat: 1-20 (child is told explicitly) Can repeat: (child is told explicitly) Questions longer, second half Pause timing if repeating Process observation ideas... minutes, plays for A band sets up for 25 minutes, plays for 40 minutes, and packs up for 20 minutes. The next band sets up for 20 minutes, plays for 45 minutes, and packs up for 15 minutes. The last band sets up for minutes, and packs up for 10 minutes. If the first band starts setting up at 6:30, what time is it when the last band finishes packing up? 35 Digit Span Administration Added trial to Forward ceiling Added some trials for gradient Added new Sequencing task Most frequent admin error: reading wrong speed or different tone/inflection across digits 1 digit per second Don t chunk Consistent tone, drop tone last digit read Second most frequent admin error? Why can t I do that? Third most frequent admin error? Why can t I do that? What if child interrupts me while I m reading numbers? Digits may occur more than once per trial: one item on Forward, multiple trials on Sequencing What if child fails the Sequencing qualifying item? 36 6

7 Digit Span Scoring Picture Span Administration Child views one or more pictures, then selects them in sequential order from a larger picture array Two points for correct pictures in the correct order and one point for correct pictures in the incorrect order Simple visual span task with proactive interference Research indicates proactive interference increases processing demands of working memory tasks (Blalock & McCabe, 2011; Carroll, et al., 2010) Admin Tips Remember to: Stimulus Page Response Page 37 Point to the pictures in the order I showed you. 38 Picture Span Scoring Most common scoring error? Letter-Number Sequencing Administration B D C Teaching Items Longest Span Scores Record Verbatim Responses changing exposure time reminder Distinguishing between 1 and 2 points change in max point value 39 Teaching modified for floor Teaching change: teach to order letters before numbers first then when multiple letters/numbers introduced, teach to sequence Most frequent admin error? 1 number or letter per second Don t chunk Consistent tone, drop tone last character read Second most frequent admin error? Why can t I do that? What if child interrupts me while I m reading numbers? No rhyming characters on a single trial for improved error interpretation Pop Quiz: Which letters and numbers do not appear on the new Letter-Number Sequencing? What if child fails either qualifying item? 40 Letter-Number Sequencing Scoring For first two items, incorrect to say letters before numbers now 1 B 2 C F 3 A 5 E 2 DON T dc yet! Coding Administration and Scoring Item difficulty consistent across rows Changed symbols for digital Roation errors score No major changes to administration (instructions a little briefer) Most frequent administration error on Coding? Most frequent scoring error on Coding? What is a rotation error? Brain teaser: Why is the Coding rotation error max raw score lower than the Coding max total raw score? Why isn t the rotation error score a scaled score? Most frequent question about Coding norms

8 Use of Coding Scoring Template Symbol Search Administration and Scoring New symbols Set and rotation error scores No major changes to administration (instructions a little briefer) #1 and #2 administration error on Symbol Search? What is a set error and a rotation error? How do I find them easily? rotation error set error Why aren t the set error score and the rotation error scores scaled scores? Remember to score: correct minus incorrect Most frequent question about Symbol Search norms? 44 Symbol Search Scoring Use of Symbol Search Scoring Key Cancellation Administration and Scoring New art Designed by quadrant (target to distracter ratio) No major changes to administration (instructions a little briefer) Most frequent administration error on Cancellation? Most frequent scoring error on Cancellation? Remember to score: correct minus incorrect Process scores: CAr (random) and CAs (structured) Use of Cancellation Scoring Template

9 Naming Speed Subtests Child names elements as quickly as possible Child takes two or three tasks, depending on age Each task has a sample item and a 2-page test item Current rapid naming tasks are relatively less sensitive to math disability if comorbid reading disability excluded (Korkman, Kirk, & Kemp, 2007; Pauly, Linkersdörfer, Lindberg, Woerner, Hasselhorn, Lonnemann, 2011; Willburger, Fussenegger, Moll, Wood, & Landerl, 2008) Quantity naming added to improve sensitivity to math disability (Pauly et al., 2011; Willburger et al., 2008) Naming Speed Literacy Administration 1. Color-Object Naming (age 6) 2. Size-Color-Object Naming (ages 6 8) 3. Letter-Number Naming (ages 7 16) 49 Name them as fast as you can without making mistakes. 50 Naming Speed Quantity Administration Naming Speed Literacy Scoring 1: 1 4 (age 6) (ages 7 16) Doggie Doggie SC SC Name how many squares are in each box as fast as you can without making mistakes Only mark errors (NScoe) Synonym = not an error SC = self correction (not an error) rcd completion times in seconds 52 Naming Speed Literacy Scoring cont d Naming Speed Literacy Scoring cont d Stop rules (age 6: Items 1-2; age 7-8: items 2-3; age 9+: item 3) No errors = no slash marks

10 Naming Speed Quantity Scoring 2 From NSL Record Form Page No errors = no slash marks Symbol Translation Subtests Child learns symbols-words pairs and translates symbols into learned meanings Measure visual-verbal associative memory, which is related to reading, written expression, and math skills Immediate, Delayed, and Recognition Symbol Translation subtests Immediate ST teaches visual-verbal pairs in a stepwise manner, with repetition of associations introduced in the previous step, then recalls the learned associations by translating symbol strings Delayed ST administered 20 to 30 minutes after completion of Immediate subtest, recalls the learned associations from Immediate Immediate ST: learning and recall task Delayed ST: recall Recognition ST: recognize meaning from four read aloud while viewing the symbol 57 Immediate Symbol Translation Delayed Symbol Translation Administration Teaching: This is a man. This is a boat. Tell me what each one means. Recall: Man...Boat Tell me what each one means. A man is on a boat. No teaching on Delayed Symbol Translation, just recall Repeat visual-verbal pairs from previous item through Item 10, then no more repeats 58 Recognition Symbol Translation Child views a symbol and selects the associated meaning from the response options, using the learned pairs from Immediate Symbol Translation What does this one mean? A. Mom B. Us C. People D. Man These choices are read aloud only, not viewed by the child. Don t read the letters, just the choices 59 Completing the Record Form

11 Summary Page (Derive Subtest Scaled Scores, FSIQ, Primary Index Scores) Primary Analysis Page (Mean and Pairwise Comparisons for Primary Index Scores) Ancillary and Complementary Analysis Page (Derive Complementary Subtest Scores & Ancillary & Complementary Index Scores, Do Pairwise Comparisons) Ancillary Complementary Process Analysis Page (Derive Process Scores, Do Pairwise Comparisons) WISC-V Qi Administration and Scoring 5 63 What s New on WISC V Qi? Accessibility to manual content Manual content available via button Prompts available via button Revised popup wording for accuracy and clarity Redesigned some subtests for single-page administration 64 Manual Content via Button Verbatim Prompts via Button Figure Weights

12 Information: WISC V on Qi Information: WISC V on Qi Moved special queries with asterisks to top of column Revised Popup Wording Single Page Administration Block Design: WISC V on Qi Visual Puzzles: WISC V Digital Dimension Error Replaced copy button with simple tap to correct design Added process score buttons 69 One-page administration 70 Matrix Reasoning: WISC V on Qi Figure Weights: WISC V on Qi More Process Observation Buttons One-page administration

13 Picture Concepts: WISC V on Qi Arithmetic: WISC V on Qi One-page administration Improved visibility of child s responses Corrective Feedback Selection-Based Fade One page item Digit Span: WISC V on Qi Letter-Number Seq.: WISC V on Qi Picture Span: WISC V on Qi Picture Span: WISC V on Qi Automated stimulus exposure time

14 Coding: WISC V on Qi Coding: WISC V on Qi Rotation Errors 75) 60) Symbol Search: WISC V on Qi Cancellation: WISC V on Qi 40) Set Errors Rotation Errors 40) 40) Naming Speed Literacy: WISC V on Qi Naming Speed Literacy: WISC V on Qi Default=Correct One touch=incorrect Touch again=child self-corrects 83 Default=Correct One touch=incorrect Touch again=child self-corrects 84 14

15 Naming Speed Quantity: WISC V on Qi Immediate Symbol Translation: WISC V on Qi Immediate Symbol Translation Default=Incorrect One touch=correct Delayed Symbol Translation: WISC V on Qi Recognition Symbol Translation: WISC V on Qi Recognition Symbol Translation Item 1 Delayed Symbol Translation No learning or review of symbol-word associations in Delayed Final Q & A 6 89 References Benson, N., Hulac, D., & Kranzler, J. H. (2010). Independent examination of the Wechsler Adult Intelligence Scale Fourth Edition (WAIS IV): What does the WAIS IV measure? Psychological Assessment, 22, doi: /a Blalock, L. D., & McCabe, D. P. (2011). Proactive interference and practice effects in visuospatial working memory span task performance. Memory, 19, Carroll, L. M., Jalbert, A., Penney, A. M., Neath, I., Surprenant, A. M., & Tehan, G. (2010). Evidence for proactive interference in the focus of attention of working memory. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 64 (3), Keith, T. Z., Fine, J. G., Taub, G. E., Reynolds, M. R., & Kranzler, J. H. (2006). Higher order, multisample, confirmatory factor analysis of the Wechsler Intelligence Scale for Children Fourth Edition: What does it measure? School Psychology Review, 35, Kolen, M. J., & Brennan, R. L. (2014). Test equating, scaling, and linking: Methods and practices. New York: Springer. Korkman, M., Kirk, M., & Kemp, S. (2007). NEPSY Second Edition. Bloomington, MN: Pearson

16 References (continued) Pauly, H., Linkersdörfer, J., Lindberg, S., Woerner, W., Hasselhorn, M. (2011). Domain-specific Rapid Automatized Naming deficits in children at risk for learning disabilities. Journal of Neurolinguistics, 24 (5), Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, Ward, L. C., Bergman, M. A., & Hebert, K. R. (2012). WAIS-IV subtest covariance structure: Conceptual and statistical considerations. Psychological Assessment, 24, Weiss, L. G., Keith, T. Z., Zhu, J., & Chen, H. (2013a). WAIS IV and clinical validation of the four- and five-factor interpretive approaches. Journal of Psychoeducational Assessment, 31, Weiss, L. G., Keith, T. Z., Zhu, J., & Chen, H. (2013b). WISC IV and clinical validation of the four- and five-factor interpretive approaches. Journal of Psychoeducational Assessment, 31, Willburger, E., Fussenegger,B., Moll,K., Wood,G., & Landerl,K. (2008). Naming speed in dyslexia and dyscalculia. Learning and Individual Differences, 18 (2), Copyright