Intelligence testing: A matter of life and death? Chad W. Buckendahl Buros Center for Testing University of Nebraska, Lincoln

Intelligence testing: A matter of life and death? Chad W. Buckendahl Buros Center for Testing University of Nebraska, Lincoln

Buros Center for Testing Oscar K. Buros, 1935; UNL, 1979 Improve the science and practice of testing Buros Institute of Mental Measurements (BIMM): commercially available tests Mental Measurements Yearbook (MMY) Test in Print (TIP) Buros Institute for Assessment Consultation and Outreach (BIACO): proprietary tests Psychometric services for educational licensure, certification, admissions, and employment testing

Case Background Defendant (Erick Vela) participated in a bank robbery in Norfolk, NE in September, 2002 Five people were killed during the robbery One of four defendants convicted Motion asserting mental retardation

Legal Background Atkins v. Virginia (2002) Prohibited execution of persons with mental retardation Additional Nebraska statute An intelligence quotient of 70 or below on a reliably administered intelligence quotient test shall be presumptive evidence of mental retardation.

Diagnosing Mental Retardation Relies on multiple criteria American Psychiatric Association and American Association of Mental Retardation: Sub-average intellectual functioning, Significant limitations in adaptive skills such as communications, self-care, and self-direction, and Manifest before age of 18

Defining primary criteria Sub-average intelligence: IQ 70 or below (2 std. dev. below mean) Considers measurement error Adaptive functioning: Coping with common life demands Standards of personal independence Socio-cultural background Community setting

Intelligence tests Tests administered by three different defense experts: Wechsler Abbreviated Scale of Intelligence (WASI): July 2003 Wechsler Adult Intelligence Scales (3 rd ed.) (WAIS-III): November 2003 Stanford-Binet (5 th ed.) (SB5): July 2004

Adaptive Behavior Tests July 2003: none November/December 2003: none 2004-05: Vineland Adaptive Behavior Scales Defense: defendant s sister State: two friends of the defendant

Malingering tests Administered when there are concerns about motivation/effort in performance July, 2003: none November, 2003: Test of Memory Malingering (TOMM) Victoria Symptom Validity Test (VSVT) Validity Indicator Profile (VIP) 21-item test July, 2004: none

Professional Standards Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999) Validity Reliability Administration Scoring Score Interpretation

Validity Argument Ultimately, the validity of an intended interpretation of test scores relies on all the available evidence relevant to the technical quality of a testing system. This includes...appropriate test administration and scoring... (Standards, p. 17)

Administration and Scoring The usefulness and interpretability of test scores require that a test be administered and scored according to the developer s instructions... Without such standardization, the accuracy and comparability of score interpretations would be reduced. (Standards, p. 61)

Scores WASI: 87 (Full Scale IQ) 82 (Verbal IQ), 94 (Performance IQ) WAIS-III: 75 (FSIQ) 75 (VIQ), 78 (PIQ) SB5: 66 (FSIQ) 56 (VIQ), 79 (Nonverbal IQ)

Interpreting scale scores Full scale score relative to average (mean) score for population (i.e. 100) Full scale score relative to the statutory cut score (i.e. 70) Expected range of full scale and subscale scores given defendant s observed scores

0.2 Comparison of subject's observed WASI scores with the general population Density 0.18 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 Mental Retardation Overall WASI distribution Full Scale IQ Verbal IQ Performance IQ 0 50 60 70 80 90 100 110 120 130 140 150 IQ Score

0.20 Comparison of subject's observed WASI FSIQ score with a score of 70 Density 0.15 0.10 Mental Retardation Given an observed WASI FSIQ score of 87, one would have an approximately 0.0000002% chance of having a true score of 70 or lower. 0.05 0.00 50 55 60 65 70 75 80 85 90 95 100 FSIQ

WASI caution [The WASI] should not be used alone to make diagnosis or educational placement decisions. It is not meant to replace more comprehensive measures of intelligence, such as the WISC-III and the WAIS-III." (Technical Manual, p.8)

0.2 Comparison of subject's observed WAIS-III scores with the general population Density 0.18 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 Mental Retardation Overall WAIS-III Full Scale IQ Verbal IQ Performance IQ 0 50 60 70 80 90 100 110 120 130 140 150 IQ Score

Comparison of subject's observed WAIS-III FSIQ score with a score of 70 Density 0.20 0.15 0.10 0.05 Mental Retardation Given an observed WAIS-III FSIQ score of 75, one would have an approximately 1.7% chance of having a true score of 70 or lower. 0.00 50 55 60 65 70 75 80 85 90 95 100 FSIQ

Interpreting WAIS-III Scores...a low score on the WAIS-III does not necessarily reflect a low level of intellectual functioning... other factors such as the following may be implicated: cultural and linguistic discrepancy from the test s standardization sample, disabling distractibility or anxiety, severe psychopathology, deafness, poor motivation or inadequate persistence, and extremely oppositional behavior or very poor rapport with the examiner. Before diagnosing low intellectual functioning or mental retardation, the examiner must rule out these factors. (emphasis added) Administration and Scoring Manual (pp.7-8)

0.2 Comparison of subject's observed SB5 scores with the general population D ensity 0.18 0.16 0.14 0.12 0.1 0.08 0.06 0.04 Mental Retardation Overall SB5 Full Scale IQ Verbal IQ Non-Verbal IQ 0.02 0 50 60 70 80 90 100 110 120 130 140 150 IQ Score

SB5 VIQ/NVIQ Differences The defendant had a Full scale IQ (FSIQ) score of 66 Verbal IQ (VIQ) score of 56 Nonverbal IQ (NVIQ) score of 79 23 point difference between VIQ and NVIQ Only 1.9% of the standardization sample had a difference of 23 or more between VIQ and NVIQ

Interpreting SB5 Scores when significant differences exist between the NVIQ and the VIQ, examiners should be cautious about evaluating FSIQ as a summary of an individual s general ability level...when the context of the assessment and examinee s background is influenced by such factors as communication disorders, learning disabilities, autism, or non-english background, the NVIQ may be a better indicator of global cognitive potential. (emphasis added) Examiner s Manual (p. 134)

Density 0.20 0.15 0.10 0.05 Comparison of subject's observed SB5 NVIQ score with a score of 70 Mental Retardation Given an observed SB5 NVIQ score of 79, one would have an approximately 0.1% chance of having a true score of 70 or lower. 0.00 50 55 60 65 70 75 80 85 90 95 100 NVIQ

SB5, 07/2004 150 140 130 120 110 100 90 80 70 60 50 Full Scale Scores Over Time WAIS-III, 11-12/2003 WASI, 07/2003 Full Scale IQ

SB5, 07/2004 150 140 130 120 110 100 90 80 70 60 50 Verbal Scale Scores Over Time WAIS-III, 11-12/2003 WASI, 07/2003 Verbal IQ

SB5, 07/2004 150 140 130 120 110 100 90 80 70 60 50 Nonverbal Scale Scores Over Time WASI, 07/2003 WAIS-III, 11-12/2003 Performance/Nonverbal IQ

Summary of Intelligence Tests Chance of true score being 70 Test WASI FSIQ or lower (%) <.00001 Verbal IQ 0.06 Performance IQ <0.00001 WAIS-III FSIQ 1.7 Verbal IQ 2.7 Performance IQ 1.9 SB5 FSIQ 95.9 Verbal IQ 97.0 Nonverbal IQ 0.1

Adaptive Functioning Information Vineland Adaptive Behavior Scales School records Course grades, remedial programs Observations Family members, teachers, inmates Employment record, corrections officials

Defense approach Strict interpretation of the statute (i.e. reliably administered intelligence test) Suggested that accuracy (validity) could be obtained without precision (reliability) Consider the standard error in determining the range of possible true IQ scores Suggested that observed scores across tests were different due to the Flynn effect (i.e. intergenerational influence on norm sample)

Judge s Ruling Presumption of MR rejected Validity of presumptive score: 3 rd intelligence test administered by defense Probability of 66 on SB5 was remote No malingering tests administered on it Examiner did not follow interpretation directions in test publisher s manual

Judge s Ruling (cont.) Sub-average intelligence Considered standard error WAIS-III score met the criterion Adaptive functioning Preponderance of evidence did not support limitations in this area Age of onset NE s statute does not specify age Defense motion denied

Future Research Questions Measurement Application of standard setting methodology for cut score Validity of score interpretations given intended uses Instrument development (e.g., content, norm sample) Legal Implications for future cases/legislation Court s application of current validity theory Psychological Criterion definition of mental retardation (rather than norm) Interpretation of intelligence as unique from achievement Stability of intelligence as a construct