The teacher says: In two minutes I will erase the blackboard, and I'm only here, I have five more lines to copy. And some children erased it, and I haven t finished yet and I can t remember the last word, or I still have nine more lines, and I m only on the first one, and then a boy comes along and erases everything and then I don t know what to write Development, Reliability and Validity of a Handwriting Proficiency Screening Questionnaire (HPSQ) Sara Rosenblum Ph.D. Head, Department of Occupational Therapy, Laboratory of Complex Human Activity and Participation (CHAP) Faculty of Social Welfare & Health Studies, University of Haifa, Haifa, Israel 2 1 Proficient handwriting: Produces legible text with minimum effort. In this case, handwriting is automatic and does not interfere with the content as generated by the creative thinking process (Scardamalia, Bereiter, & Goleman, 1982) Poor handwriting- handwriting deficiency- dysgraphia A disturbance or difficulty in the production of written language that has to do with the mechanics of writing (Hamstra-Bletz & Blote,1993) Writing is main occupation in school Elementary school children typically spend up to 50% of the school day engaged in writing tasks, some of which are performed under time constraints (McHale & Cermak, 1992; Tseng & Chow, 2000). Writing skills are required for: Recording in class, knowledge organization, homework, exams Personal notes The child s self-perception as a writer 4 3 The Current Situation Lack of tools for evaluating handwriting performance in the school environment (O'Hare, 2004: Stefansson & Karlsdottier,2003). Most require substantial amounts of time to administer and score (see Rosenblum, Weiss, & Parush, 2003 for more details) Teachers/clinicians observe and suspect that the child has handwriting deficiencies No quick standardized screening tools exist to confirm these concerns (Daniel & Froude, 1998). 6 Who has poor handwriting? Approximately 11-12% of female students and 21-32% of male students in regular schools (Karlsdottir & Stefansson, 2002; Smits- Engelsman, Van Galen, & Michels, 1995). Children with: Developmental Coordination Disorder/Dyspraxia - clumsiness (Smits- Engelsman et al., 2001; 2003) Learning Disabilities (Waber & Bernstein, 1994) Dysgraphia A specific LD ADHD (Tucha & Lange, 2001) Asperger s syndrome (Henderson & Green, 2001) Cerebral palsy /head trauma or other physical/motor handicaps 5 1
The consequences of handwriting difficulties go beyond the task performance itself: Lower marks for poor writing products The conceptual framework adopted by the ICF. From the International Classification of Functioning, Disability and Health (WHO, 2001, p. 18). Copyright 2001 by the World Health Organization. Reprinted with permission. Communication and sharing of data between educators and occupational therapists regarding children's handwriting difficulties (Stefansson & Karlsdottir, 2003). 8 (Graham, Harris, and Fink (2000) May influence perceptions about children s competence as writers (Klein & Tub, 2005). It is important to identify handwriting difficulties as early as possible, both as a preventive and a corrective aid (Berninger & Amtmann, 2003). 7 HPSQ Content Validity Information: clinicians observations, the literature: Main characteristics related to non-proficient handwriting: 1. Legibility 2. Time and speed of performance 3. Physical and emotional well-being. Educators and clinicians report: Non-proficient handwriting characteristics: Slow and tedious Typically accompanied by difficulties in letter production and rapid muscle fatigue (Tseng & Chow, 2000; Benbow, 1995; Feder et al., 2000; Mojet, 1991). Poor handwriting product - incorrect letter formation, poor alignment, reversals, uneven size of letters and irregular spacing between letters and words (Graham, Berninger, & Weintraub, 1998). Establishing the tool s content validity 10 9 Expert Validity Questionnaire (HPSQ) (Rosenblum, 2008) The expert panel (more than 10 years experience with school age children: three expert consultants, three pediatric occupational therapists, four experienced teachers) Rating each item s: Relevance for the concept being evaluated (yes or no), Clearness of wording : Clearness - 100% agreement for the 10 questionnaire items Relevance- 100% agreement for items 1-6 and 8-10 80% agreement - item 7 (hand pain while writing). 12 11 2
Examination of HPSQ Reliability and Validity Participants 230 children in 2nd 8th grades recruited from four regular public schools 54% were boys and 46% were girls. Most participants were right hand dominant (91%). Instruments (concurrent and construct validity) 1. Computerized Penmanship Evaluation Tool (ComPET- Rosenblum, Parush, & Weiss, 2003a). 2. The Hebrew Handwriting Evaluation (HHE) (Erez & Parush, 1999). Expert Validity Qualitative evaluation process: Three teachers observe four children (previously observed as non-proficient handwriters) and fill in the HPSQ and rate the questionnaire s practicality : Items were clearly written HPSQ completion required five to 10 minutes per child. Stated that it is very practical HPSQ gave an accurate portrayal of the individual children s handwriting deficiencies that had been previously observed 14 13 Questionnaire results The HPSQ total score was summed for each participant The mean questionnaire score of the 230 children in the study sample was 5.87 (SD=6.51), and the median score was 4.00. Internal Reliability For the 10-item scale: ά=.90 Instruments ComPET Hebrew Handwriting Evaluation The reliability for each of the three factors: Legibility items (1,2,10) ά=.82, Performance time items (3,4,9) ά=.85 Physical and emotional well-being items (5,6,7,8) ά=.81 16 Temporal spatial and pressure measures Global legibility; Number of letters erased or overwritten; Spatial arrangement; Number of letters written in the first minute 15 Concurrent Validity Moderate significant correlations (r=.52-.65, p<.001) between the HPSQ score and scores on the Hebrew Handwriting Evaluation (HHE, Erez & Parush, 1999) of the handwriting product as well as measures of the handwriting process (Computerized Penmanship Evaluation Tool- ComPET, Rosenblum et al., 2003a), Test-retest and and Inter-rater Reliability The intra-class correlation coefficient for the overall test retest scores was.84 Inter-rater reliability: the intra-class correlation coefficient for the questionnaire's overall score was.92 The ICC values for the individual questionnaire items ranged from.64 to.91 18 17 3
Construct Validity - ComPET Handwriting Process Measures Children with poor handwriting displayed significantly inferior performance compared to children with proficient handwriting : More letter segments (U=1859, n=33, p=.0001) More time to form each segment (U=1449, n=33 p=.0001) Slower velocity (U=2316, n=33, p=.031) More in air time while paragraph copying (U=1998, n=33, p=.001) Higher pressure implemented on the writing surface (U=1721, n=33, p=.0001) Construct Validity The participants (230 children) were divided into two groups based on their questionnaire total scores (Mean=5.87 SD=6.51) Group 1 - children who achieved a score of 0-13 (Mean + one SD) - Proficient - 85% of the entire sample Group 2 included those who achieved a score of 14 and up - Poor - 15% of the entire sample 20 19 Principal Components Factor Analysis Two factors: 1. Includes items 3-9 (performance time and wellbeing) and accounted for 54% of the variance. 2. Includes items 1, 2 and 10 (legibility) and accounted for 13% of the variance. Thus, the two factors together explain 67% of the variance. Construct Validity Hebrew Handwriting Evaluation (HHE) of the Writing Product. Children with poor handwriting displayed significantly inferior performance compared to children with proficient handwriting for three out of five of the HHE outcome measures : Global legibility (U=1584, n=34, p=.0001) Number of letters written in the first minute (U=1377, n=34, p=.0001) Spatial arrangement (U=1584, n=34, p=.0001). 22 21 The HPSQ item content stayed the same The wording was changed from third person to second person in the HPSQ-C, to address the children themselves. For example, item no. 3 was changed from 'Does the child not have enough time to copy tasks from the blackboard?' to: 'Does it ever happen that you do not manage to finish copying from the blackboard on time?' The possible answers stayed the same: Never, Rarely, Sometimes, Often or Always (scored from 0 to 4). Reporting about the child's performance ability by adults appears to be not enough Legal, social, ethical and philosophical viewpoints state that children have the right to express their opinion (Sturgess, Rodger & Ozanne, 2002; Taylor 2000). Children wishe to express their feelings about their handwriting performance. When children are engaged in a specific activity, in many cases, they will be the best judges of the quality of their performance (Primeau & Ferguson 1999) 24 23 4
Summary and Conclusions The HPSQ is a highly reliable and valid screening tool (high internal consistency (.90, test-retest reliability.84, inter-rater reliabliity (.92)- p<.01) The tool items successfully reflect the overall constellation of handwriting problems in children. The FA findings are interesting: Children who do not succeed in fulfilling time requirements in class experience a greater lack of physical and emotional well-being than children whose handwriting product is illegible! Good internal consistency (α=.77) Concurrent validity: Significant moderate correlation was found between the final scores of the Child questionnaire (HPSQ-C) and the Teacher questionnaire (HPSQ) (r=.51 P<.001). Construct validity: The questionnaire (HPSQ-C) significantly distinguishes between children with and without handwriting deficiencies, based on the scores of the Hebrew Handwriting Evaluation of the handwriting product (HHE) as well as measures of the handwriting process (ComPET). 26 25 Mean final score Preliminary results- England Wendy cumines Unpublished thesis HPSQ scores Mean legibility (1,2,10) Mean performance time Mean Well being Israel N=39 Grade 5 year 5 Ages 10-11 HD: 13% 5.48 (6.86) 1.43 (2.20) 1.51 (2.15) 2.53 (3.56) England N=46 Grade 5 year 6 Ages 10-11 HD: 17% 11.74 (7.93) 3.34 (2.67) 5.06 (2.92) 3.43 (3.17) Summary and Conclusions The HPSQ supplies the following information: Locates and identifies the child s handwriting deficits using a standardized tool Clarifies that there is a problem teacher, parents, the child Creates a common language Enables directing the child to occupational therapists Assists the occupational therapist in targeting the criteria and elements of handwriting that teachers expect to be focused upon in therapy (Hammer Schmidt, & Sudsawad, 2004). 28 27 The HPSQ is being in use today in: Brazil Belgium England Indonesia Malaysia Florida, USA Thailand Hong Kong 29 5