David Lohman Belin-Blank Center & Iowa Testing Programs University of Iowa

Transcription

1 David Lohman Belin-Blank Center & Iowa Testing Programs University of Iowa

2 Gifted or talented : It matters Academic aptitudes Overview Adapting instruction to individual differences Ability profiles: CogAT ABC classification system Commonality of large ability differences t the extremes

3 Giftedness perspective Encourages qualitative distinctions between the gifted and not gifted Assumes that, if well measured, giftedness is fixed One-time identification early in primary school & permanent separation into special classes or schools Use of identification methods (IQ or other composite scores) that exclude children with uneven ability profiles

4 Some parents covet the label and will go to great lengths to obtain it Program administrators scramble to achieve better racial & ethnic balance often using inferior tests and ignoring important measures of readiness for greater challenge

5 Talent identification & development perspective Discourages categorical distinctions since all children have talents that can be developed Encourages direct links between identification practices & development activities Encourages ongoing identification Expects that children will excel in one or two domains rather than in all domains Encourages going beyond national norms for school-based talent identification Facilitates efforts to encourage talent development among poor and ELL children

6 Aptitude: What it is Aptitude is the degree of readiness to learn & perform well in a particular situation. Examples of academic aptitudes To use previously acquired knowledge and skill appropriately To make good inferences and generalizations To manage one s emotions; to persist

7 Aptitude: What it is NOT Not fixed Not just abilities also achievements Not just cognitive constructs (affective, conative) Not just positive attributes Not independent of context or circumstance What does the context demand, evoke, afford

8 Aptitude Is a more inclusive concept than talent, and a much more inclusive concept than ability. Inherently developmental and multidimensional. Emphasizes symbiotic configurations of ability, personality, and motivational traits That resonate with the demands and affordances of the educational or training context.

9 Aptitude Talent Abilities g+

10 Gifted is usually conflated with cognitive ability Developed, actually (gifted assumed fixed) Abilities g+ Includes Achievements

11 Emphasizes Potential Talent Multidimensional Beyond cognitive Abilities g+ Development of Expertise

12 Includes Motivation & Affect Aptitude Demands & Affordances of Environment Talent Abilities g+ Includes Conation Aptitude Complexes

13 Aptitude is a more inclusive concept than talent, and a much more inclusive concept than ability. Inherently developmental and multidimensional. Emphasizes symbiotic configurations of affective and conative traits or propensities. That resonate with the demands and affordances of the educational or training context.

14 Using an aptitude perspective What is the desired end state? What is the developmental sequence(s) What is required at each developmental level to develop this knowledge & skill? What is made likely or useful? What will impede learning?

15 What does the situation/context/environment make likely or useful? Examples Affordances of the Environment Resonance or attunement of the person and the situation: Do the characteristics that the person brings to the situation match the affordances of the situation?

16 Academic accomplishment 1 Learning Context 1 Learning Context 2 a b c d e Person characteristics

17 Background Sources Cronbach et al. (2002). Remaking the concept of aptitude: Extending the legacy of Richard Snow, Erlbaum. Lohman, D. (2005). An aptitude perspective on talent : Implications for identification of academically gifted minority students. Journal for the Education of the Gifted, 2005, 28, ) Gagne, F. DMGT Dawes Theory of work adjustment

18 Inferring Aptitude 1. Retrospectively from (comparatively) rapid acquisition of knowledge/skills in a domain 2. Comparatively high performance on tasks that share critical cognitive, affective, or conative processes

19 Implications for Gifted ID More than g : V, Q, & S Knowledge and skills increasingly important with age/experience Interests e.g. Renzulli learning Willingness to persist in working to achieve excellence/competence Defining the situation is part of defining the aptitude

20 Guiding efforts to adapt instruction

21 All, Some, 0ne Some guidance on how teachers can do this applies to all children, some of it applies better to some children than to other children, and some applies only to some children and not at all to others Achievement tests inform decisions about WHAT to teach Measures of ability and affect inform decisions about HOW to teach

22 Myths about adapting instruction All students are pretty much alike

23 Reading Vocab Across Grades 400 V O C A B U L A R Y th %-tile 80th %-tile 50th %-tile 20th %-tile 1st %-tile K Grade

24 Reading Vocab Across Grades 400 V O C A B U L A R Y th %-tile 80th %-tile 50th %-tile 20th %-tile 1st %-tile K Grade

25 Myths about adapting All students are pretty much alike Every student is unique instruction Adaptations should be based on learning styles If the method is right, the outcome will be good

26 Examples of correlations Predictor and criterion r N Aspirin and reduced risk of death by heart attack a.02 22,071 General batting skill as a Major League baseball player and hit success on a given instance at bat a.06 Calcium intake and bone mass in premenopausal women a.08 2,493 Effect of nonsteroidal anti-inflammatory drugs (e.g., ibuprofen) on pain reduction a.14 8,488 Weight and height for U.S. adults a.44 16,948

27 Myths about adapting instruction All students are pretty much alike Every student is unique Adaptations should be based on learning styles If the method is right, the outcome will be good Individualization requires separate learning tasks

28 Conceptualizing transactions between persons and situations Propensities as tendencies to act or think or feel in certain ways Environmental cues that elicit these propensities are called affordances Affordances are what a situation offers, or makes likely, or makes useful. The propensities that assist a person in attaining a goal in a situation function as aptitudes

29 Important Characteristics of Students Cognition (knowing) Domain knowledge & skill (Gc - Crystallized) Reasoning abilities in the symbol systems used to communicate knowledge (Gf Fluid) Verbal Quantitative Spatial

30 Important Characteristics of Students Cognition (knowing) Domain knowledge & skill Reasoning abilities in the symbol systems used to communicate knowledge (Verbal, Quant., Spatial) Affection (feeling) anxiety, interests, working alone/with others Conation (willing) persistence, impulsivity

31 Important Characteristics of Classrooms Structure Novelty/Complexity/Abstractness Dominant symbol system Opportunities for working alone or with others

32 How do you do it? Remediate weaknesses? How can teachers regulate/focus the cognitive demands of activities? Group kids by learning style? See Short Guide for Teachers on the CogAT Form 6 website under: Free online resources provide up-to-date support for teachers

33 General Principles of Instructional Adaptation Build on Strength Focus on working memory Scaffold wisely Emphasize strategies When grouping, aim for diversity

34 Primary uses of CogAT To guide efforts to adapt instruction to the needs and abilities of students To provide an alternative measure of cognitive development To identify students whose predicted levels of achievement differ markedly from their observed levels of achievement

35 Cognitive Abilities Test (Form 7) Verbal Reasoning Verbal/Picture Analogies Sentence Completion* Verbal/Picture Classification Quant. Reasoning Number Analogies Number Puzzles Number Series Nonverbal Reasoning Figure Matrices Paper Folding Figure Classification *English/Spanish prompts General Reasoning

36 How best to summarize CogAT scores? Pattern of scores? Overall level of scores? Link this to general principles

37 Main points on profiles Profiles can be measured reliably (but not quickly) Most children show significantly uneven patterns of ability And these have important implications for HOW they learn best Extreme discrepancies among abilities are much more common among the most able children than among average ability children Therefore scores that average across domains exclude many of the most talented children

38 CogAT ABC Profile system Level = Median (middle) age stanine 6 A 5 B (V+) 3 C (Q+ V-) 8 E (N-)

39 CogAT ABC Profile system Measuring the pattern A profiles: Confidence bands overlap for all three scores. Scores are at roughly the same level B profiles: One score is above or Below the other two scores, which do not differ C profiles: Two scores Contrast E profiles: Extreme B or C profiles (>=24 SAS points)

40 A Profile SAS PR V Q N

41 B Profiles SAS PR V Q N N- SAS PR V Q N N+

42 Extreme B Profile Weakness SAS PR V Q N SAS Max SAS Min = 31 E (N-)

43 Profiles provide readable summary of the pattern and level of each student s scores No need to estimate whether scores differ Get the user quickly from the score report to suggestions for teaching (..index) Part VII of the IGTC On the web at Suggesting how (not what) to teach

44 Profile Frequencies for CogAT6 Percent Percent Total A: All three scores (V, Q, N) at approximately the same level A B: One score above or Below the other two scores B (V+) 8.3 E (V+) 1.8 B (V-) 7.9 E (V-) 1.6 B (Q+) 5.9 E (Q+) 0.8 B (Q-) 5.5 E (Q-) 0.6 B (N+) 6.9 E (N+) 1.2 B (N-) 7.1 E (N-) C: Two scores Contrast C (V+Q-) 2.4 E (V+Q-) 0.6 C (V-Q+) 2.3 E (V-Q+) 0.6 C (V+N-) 2.9 E (V+N-) 0.9 C (V-N+) 2.6 E (V-N+) 1.0 C (Q+N-) 2.0 E (Q+N-) 0.5 C (Q-N+) 1.9 E (Q-N+)

45 Percent Profile Frequency by Median Stanine Low Median Stanine High Flat Lohman, D. F., Gambrell, J., & Lakin, J. (2008). The commonality of extreme discrepancies in the ability profiles of academically gifted students. Psychological Science, 50,

46 Percent Profile Frequency by Median Stanine Median Stanine Flat Weakness Lohman, D. F., Gambrell, J., & Lakin, J. (2008). The commonality of extreme discrepancies in the ability profiles of academically gifted students. Psychological Science, 50,

47 Percent Profile Frequency by Median Stanine Median Stanine Flat Strength Weakness Lohman, D. F., Gambrell, J., & Lakin, J. (2008). The commonality of extreme discrepancies in the ability profiles of academically gifted students. Psychological Science, 50,

48 Percent Profile Frequency by Median Stanine Median Stanine Flat Strength Weakness Lohman, D. F., Gambrell, J., & Lakin, J. (2008). The commonality of extreme discrepancies in the ability profiles of academically gifted students. Psychological Science, 50,

49 Conclusions Extreme discrepancies in abilities are much more common among the most (and least) able students than among average ability students. Therefore, procedures for identifying academically talented students that rely on a composite score that averages across ability domains will exclude many children who reason well in particular symbol systems

50 Screen on g but with a generous cut Placement requires distinguishing at least verbal from quant/spatial And evidence on current levels of knowledge and skill in domains in which education/training can be offered And evidence of interest And flexibility in programming

51

52 What are the programming options?

53 The higher the stakes, the costlier the errors For everyone School-based talent development programs should try to avoid the need for high-stakes decisions. Competition for places corrupts the indicators Attempts to diversify programs are primarily limited by the diversity of educational services offered the tests used to identify talent the normative perspectives used to interpret test scores

54 What educational options are available? Special schools? Acceleration whole grade or single subject? Special self-contained classes within schools? Pull-out programs during school? Achievement requirements? Enrichment activities? During school? After school? Outside of school? Options greatest for targeted, school-based enrichment

55 Suppose the goals were: NOT to identify gifted ELL/U-R minority students but to identify the most academically talented children in these groups NOT to admit to the gifted program but to encourage the development of their talents

56 Measurement issues: ELL and low SES

57 Obvious Construct irrelevant sources of difficulty English language for ELL students Reading for children who cannot read Not so obvious Unfamiliar test formats Unfamiliar content SES, gender, and regional differences Only one kind of content or format

58 Reducing construct-irrelevant sources of difficulty Providing informative directions in student s language(s) E.g. Both English & Spanish (familiar teacher) Repeat practice items, provide additional examples More than perfunctory practice items/tests The necessity of good practice tests Better level the playing field Can help teach more broadly useful thinking skills

59 Construct over- or underrepresentation Over-representation (i.e., more than reasoning abilities) Working memory, processing speed (dropped in WISC-IV GIA) Under-representation General ability (g) only Profiles of reasoning abilities important for the most and least able Especially for placement decisions Only figural/spatial reasoning (e.g., Raven, NNAT) Observed scores g

60 Assessment goals for all students Measure the multiple personal characteristics that indicate aptitude or talent for domains in which training can be offered When possible, objectively Distinguishing potential (or talents) from current accomplishments Emphasize talent identification and development not giftedness Especially for ELL children

61 Inferences of ability for ELL children Non-normative Experiences Nonverbal tests (common norms) Common assessments (special norms)

62 Option 1 Same norms as for other children Figural Reasoning (NV) tests Help but do not eliminate differences Construct under-representation Quantitative reasoning an important ally Small differences and directly linked to instruction Picture-Verbal and Picture-Quant best for young children If flat ( A ) profile, use VQN Composite Importance of informative practice Measure or discover interests Current level of tool skills and achievement in particular domains

63 WISC-IV Spanish Target population: Bilingual Spanish-speaking children in the U.S. with no more than 5 years of U.S. schooling Translated & adapted the verbal tests Calibrated the Spanish Verbal Scale with the normative (English) Verbal Scale Index scores plus 2 PRs that estimated opportunity to learn by % education in U.S. Plus Parental educational level

64 Language versus cultural loading Language load reduced but what about cultural loading? Reasoning Indexes Verbal Comprehension (Vocab, Similarities, Comprehension) Perceptual Reasoning (Block Design, Picture Concepts, Matrix Reasoning) Which Index is most affected by % education received in U.S. schools?

65 Scale Score (M = 100, SD = 15) WISC-IV Spanish Scores by Percent Education in the U.S Full Scale IQ Verbal Comp Perceptual Reasoning WISC-IV Score Minimal (<30%) Most All (100%)

66 Scale Score (M = 100, SD = 15) WISC-IV Spanish Scores by Percent Education in the U.S Full Scale IQ Verbal Comp Perceptual Reasoning WISC-IV Score Minimal (<30%) Most All (100%)

67 Scale Score (M = 100, SD = 15) WISC-IV Spanish Scores by Percent Education in the U.S Full Scale IQ Verbal Comp Perceptual Reasoning WISC-IV Score Minimal (<30%) Most All (100%)

68 Comprehensive nonverbal tests McCallum, Bracken, & Wasserman (2001). Essentials of nonverbal assessment.

69 Figural Nonverbal Picture Quantitative Picture Verbal

70 SAS differences between non-ell and ELL students (CogAT Form 7 Levels 5-8) Figures Pictures Pictures Nonverbal Quant Verbal

71 Inferences of ability for ELL children Non-normative Experiences Nonverbal tests (common norms) Common assessments (special norms)

72 Option 2 Comparing with OTL group A broader estimate of talent using commonly administered ability and achievement test scores, especially language based Often simple ranks will do Cumulate more precise norms (PRs) across years Lohman (in press) Nontraditional uses of traditional assessments. In C. M. Callahan & H. Hertberg-Davis (Eds.) Fundamentals of gifted education (draft on my website)

73 Measuring more than nonverbal abilities Predicting success in any domain that requires more than the sorts of abilities that are measured by nonverbal tests (especially figural reasoning tests) Single best predictor of success in schooling is Verbal Reasoning Applies to ALL children especially those not fluent in the language of instruction Can use same measures used with non-ell children, but must use different norms

74 Many state Codes define a student with high abilities as one who:..performs at, or shows the potential for performing at, an outstanding level of accomplishment in at least one domain when compared to other students of the same age, experience, or environment;

75 Using appropriate (multiple) norm or comparison groups If schools do not need national norms to identify promising runners or swimmers, then why are they mandatory for identifying promising scientists? Importance of multiple perspectives the nation, the district, the school, one or more OTL groups

76 Multiple perspectives Age versus grade norms National or international State District School OTL within any of the above

77 National norms may not be as dependable as you think Stanford-Binet V Over-representation of students with extreme scores in the norming sample. Consequence? Cattell-Culture-Fair Convenience samples of UK and US students in 1960 s Raven Progressive Matrices User data from the 1970 s published in 1984 and then again in 2000 (scores about 10 points too high) NNAT Mean = 100 but SD = 15 only at Level E (e.g., 23 at level A)

78 Getting Local Norms Ask for them!

79

80

81 Use Ranks instead of Percentile Ranks

82

83 Using Excel to get ranks

84 Get the data into an excel spreadsheet. SAS scores Potential grouping variable (e.g. ELL status)

85 To get local ranks, sort (rank order) the data by SAS scores.

86 To get separate ranks for each ELL group sort by ELL and then SAS.

87 Ranks Percentile Ranks Percentile Ranks Percentage of examinees who scored below a given score Percentile Ranks for normally distributed scores Retains information on gaps between scores Only need Mean and SD

88 Using Excel to get Local (or OTL) Percentile Ranks

89 Assumptions Many ways to get Percentile Ranks Method used here assumes scores are distributed normally This is reasonable if using SAS scores Allows you to see similarity between OTL, local,and national norms Easy to use, and to cumulate across years

90 Get the mean and standard deviation for all SAS scores Use Average function for the mean Use STDEV function for SD Mean = 93.8 SD = 13.4

91 To get local Percentile Ranks (LPRs) use the excel function NORMDIST=(X, M, SD, true) Insert local mean (M) and SD =NORMDIST (X, 93.8, 13.4, true) where X = CogAT score

92 Using different means and SD s for ELL/non-ELL groups gives different LPRs for ELL and non-ell students

93 Lohman, D. F. (in press). Nontraditional uses of traditional measures. In C. M. Callahan & H. Hertberg-Davis (Eds.) Fundamentals of gifted education. (on my webpage)

94 Integrating Teacher Ratings (and other important but less reliable information)

95 Methods for combining measures Convert to points and sum, either overall or by content areas (math versus literacy/verbal) Conversion scheme arbitrary Often distorts information in original scores Less reliable and valid measures both allow and deny admission Average scale scores or NCE scores (not PRs) Assumes SD s are similar Adding ratings difficult unless all children are rated

96 Lohman-Renzulli matrix Uses teacher ratings to provide services to students with strong (but not stellar) scores but not to remove them from students with high test scores Distinguishes between Verbal & Quant domains Allows for inclusion of achievement test scores Averages scale scores appropriately Allows for ratings on only nominated students

97 Lohman-Renzulli Matrix Teacher Rating on Learning Ability, Motivation, or Creativity Low teacher ratings High teacher ratings CogAT Verbal OR Quantitative -Nonverbal ( >95 th PR ) ( 80 th 95 th PR) II IV I III

98 Lohman-Renzulli Matrix Teacher Rating on Learning Ability, Motivation, or Creativity Low teacher ratings High teacher ratings CogAT Verbal OR Quantitative -Nonverbal ( >95 th PR ) ( 80 th 95 th PR) II IV I III

99 Lohman-Renzulli Matrix Teacher Rating on Learning Ability, Motivation, or Creativity Low teacher ratings High teacher ratings CogAT Verbal OR Quantitative -Nonverbal ( >95 th PR ) ( 80 th 95 th PR) II IV I III

100 Lohman-Renzulli Matrix Teacher Rating on Learning Ability, Motivation, or Creativity Low teacher ratings High teacher ratings CogAT Verbal OR Quantitative -Nonverbal ( >95 th PR ) ( 80 th 95 th PR) II IV I III

101 Using Riverside s data manager

102

103

104

105

106 Lohman-Renzulli Matrix Teacher Rating on Learning Ability, Motivation, or Creativity Low teacher ratings High teacher ratings CogAT Verbal OR Quantitative -Nonverbal ( >95 th PR ) ( 80 th 95 th PR) II IV I III

107

108 Form 7 of CogAT?

109 Main ideas A talent development perspective is helpful All children must bring the same cognitive, affective, & conative resources to the learning context Inferences of ability or talent ALWAYS assume similar OTL Using existing measures in this way is not difficult and, in fact, quite helpful Different levels of instructional challenge and support are essential

110 My little brother