Published online: 19 Dec 2006.

Transcription

1 This article was downloaded by: [ ] On: 07 July 2015, At: 23:49 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: Registered office: 5 Howick Place, London, SW1P 1WG Journal of Vocational Education & Training Publication details, including instructions for authors and subscription information: Less pain, more gain: rapid skill development using old way new way Paul Baxter a, Harry Lyndon b, Shelley Dole c & Diana Battistutta d a Personal Best Systems, Mt Ommaney, Queensland, Australia b Department of Education, Training and Employment, South Australia, Australia c Royal Melbourne Institute of Technology, Australia d Queensland University of Technology, Australia Published online: 19 Dec To cite this article: Paul Baxter, Harry Lyndon, Shelley Dole & Diana Battistutta (2004) Less pain, more gain: rapid skill development using old way new way, Journal of Vocational Education & Training, 56:1, 21-50, DOI: / To link to this article: PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content ) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at

2 Journal of Vocational Education and Training, Volume 56, Number 1, 2004 Less Pain, More Gain: rapid skill development using Old Way New Way PAUL BAXTER Personal Best Systems, Mt Ommaney, Queensland, Australia HARRY LYNDON Department of Education, Training and Employment, South Australia, Australia SHELLEY DOLE Royal Melbourne Institute of Technology, Australia DIANA BATTISTUTTA Queensland University of Technology, Australia ABSTRACT This study addresses an issue of global concern in skills training, namely, the rapid and permanent eradication of persistent habit errors, and bad or unsafe working practices. This article offers an alternative human factors explanation for the profound difficulties and low transfer of training experienced during error pattern retraining, and the correction of habitual performance faults. It describes Old Way New Way, a metacognitive strategy for achieving rapid and permanent error and technique correction and habit unlearning, and presents the results of an experimental trial of this behaviour change methodology. Vocational education students, representing a broad range of skill types, were recruited and randomised to one of two error correction modes, or to a control group. One Old Way New Way correction session with students yielded 80% or higher performance improvement that was maintained over three post-test periods. Students and teachers reacted positively to the Old Way New Way learning method. The high level of transfer of learning obtained is consistent with results in other settings. Implications for education, training, coaching and other performance enhancement settings are discussed. This experiment addressed a human factors issue of global concern in skills training, namely the rapid and permanent eradication of persistent 21

3 Paul Baxter et al operator errors and bad or unsafe work habits. Teachers of manual skills typically report that incorrect, inefficient or unsafe manipulation of equipment and tools acts as a barrier to rapid, efficient and safe acquisition of manual skills by students. Student errors often result from either misconceptions about the way equipment and tools function, or from bad habits acquired from earlier self-taught experiences prior to commencement of their vocational studies. Left uncorrected, these operator faults soon develop into established habits and are then much harder to eradicate. Given that most if not all students come to the skill learning situation with preconceived notions of how things work, and how to manipulate equipment and tools, the potential for incomplete or misinformation and for poor, incorrect or even unsafe work practices is high. Students who exhibit persistent misconceptions and bad habits present a real challenge to the teacher who is attempting to help them achieve competence. Conventional teaching methods generally focus on improving the effectiveness of the initial teaching phase ( getting it right the first time ) and on subsequent re-teaching when progress is unsatisfactory ( remediation ). However, in terms of unlearning theory, simply persevering with getting the student to practise the right way, while ignoring what the student already knows (especially their incorrect technique), will activate habit interference with learning of the correct skill and greatly slow down or block skill acquisition. The present study involved an experimental comparison of the relative effectiveness of conventional skill correction, namely re-teaching, with Old Way New Way (Lyndon, 1989, 2000), a metacognitive approach to skill correction and development, for correcting common error patterns in operator performance. Error Patterns The failure to achieve rapid and permanent habit retraining and unlearning of erroneous knowledge and behaviour in education, in the therapeutic sciences, in sport, and in skilled performance in workplaces is widely documented. Despite quality training and re-training, people just keep lapsing back into their old ways. Error patterns, also known as learned errors, over-learned errors, habit capture errors, habit intrusion errors or simply habit errors, have been demonstrated in a wide spectrum of human performance where automated skill, knowledge, or behavioural routines are involved. Error patterns are evident in skilled performance when an operator repeatedly and consistently reverts to a previously learned action or procedure that is inappropriate. For example, one of the students in this study habitually gripped the tenon saw incorrectly when cutting wood. Not only did the 22

4 RAPID SKILL DEVELOPMENT student use an incorrect grip, but he also used the same incorrect grip each time. This consistently incorrect performance is the signature of a habit pattern or habit error. Error patterns have been observed and studied in the performance of both physical and conceptual skills, including: the learning of mathematics (Drucker et al, 1987), science (Rowell et al, 1990) and spelling (DeMasters et al, 1986); in athletic and sports performance (Hanin et al, 2002); in artistic performance (Khan et al, 1995); in driving a motor vehicle (Lourens, 1992) and in flying an aircraft (Degani & Wiener, 1993); in working with computers (Prumper et al, 1992); in medical practice (Cohen et al, 2002); in speech therapy (Lyndon & Malcolm, 1984); in overuse and sports injuries (Purdam, 1989; Khan et al, 1995); in postural problems (Gieck et al, 1989); in foreign language learning (Chung-yu, 1976); in management training and organisational change (Newstrom, 1983; West, 1994). The wide prevalence of habitual error patterns (Reason, 1990) has serious implications for corrective attempts. The significance of consistent and persistent errors, and misconceptions as obstacles to learning new ideas and learning new ways of doing things, however, is all too often underestimated (Ausubel, 1968; Houston, 1991). Habit pattern errors are among the most common of all error forms and the most difficult to eradicate. The extreme practical difficulty in eliminating habit errors in workplace settings like aviation and medical practice has led to the belief that error eradication attempts should be abandoned in favour of error management, i.e. controlling or minimising the consequences of errors (Reason, 1990). Crew resource management (CRM) training, an example of error management involving teams, has been widely adopted in aviation and, more recently, in medical practice (Helmreich, 2000). One of the aims of the current article is to show that, contrary to common experience, persistent habit errors can be eradicated using a specific teaching method. How Do Habitual Error Patterns Arise? Many habit errors develop when, for some reason, e.g. misinterpreted instructions, the performer learns to do things incorrectly and this learned error progresses, through practice, to the autonomous stage of performance (Pyke, 1980). At this point, the habit pattern is no longer under conscious control. 23

5 Paul Baxter et al Everyday situations present many opportunities for errors in understanding and in skilled performance to develop. Errors do not only occur when someone learns to do things incorrectly. Skilled performers are often faced with the need to change aspects of their performance in the face of new situations, e.g. new equipment, new procedures, when new techniques are required and so on. What was perfectly correct, acceptable and even best practice, can suddenly become wrong and unacceptable. The better someone has learned the original routine or action in the first place, the harder it is to change over to the new way (Lawler, 1996; Zapf et al, 1992). An example is the situation faced by sports coaches when they take over from someone else and the techniques taught by the new coach conflict with those taught by the old. If the old system or techniques have been well learned, the athletes will revert to the old system when placed under the stress of actual game performance (Tutko & Richards, 1971). Lack of consistency between systems or between different parts of a system can also be a major cause of error for expert performers (Zapf et al, 1992). Self-imposed change also creates problems, e.g. a golf player changes her club grip and her game deteriorates. This makes players put off changing until they absolutely have to do so. During the period of adaptation to the new way, performance often drops and errors increase (Maschette, 1985). Rule changes requiring a change in action sequences can also give rise to interference from old habits (Crampton & Adams, 1995). Whether the need for change arises out of an erroneous performance or out of the desire to improve further something that is already satisfactory, the key element in all these situations is that the learner is faced with having to change what he or she already knows. As we shall see later, having to change one s own established ways in the face of new and conflicting knowledge, is the root cause of the problem. Conventional Explanations of Habit Errors and Associated Teaching Methods Conventional explanations of why persistent errors arise and why they are so hard to eradicate are often based on assumed intellectual or perceptual deficits (e.g. Kephart, 1960). Under this deficit model, errors are seen as a sign that learning did not take place. Students had learnt little or nothing from the original instruction. Although the individual received instruction and appeared to pay attention, the information or learning did not take. The underlying assumption is that the student does it the wrong way because he or she still does not know the right way. 24

6 RAPID SKILL DEVELOPMENT Conventional deficit-based explanations of learning failure imply only one solution, i.e. once you assume that a consistent and persistent error implies a lack of knowledge or skill, then the obvious solution is to give the person the missing knowledge or skill, i.e. re-teach it. Re-teaching and retraining invariably follow a predictable pattern, i.e. tell them what they are doing wrong, show them the right way, model it for them and ask them to copy it and then get them to practise it (Ashlock, 2002; Maschette, 1985). Although different learning modes, e.g. tactile, aural, may be employed from those used in the original learning situation, the general approach tends to follow this model. Re-teaching and retraining is usually very time-consuming, expensive of resources and largely unsuccessful (Read, 1987; Baxter & Dole, 1990; Dole, 1991; Connell & Peck, 1993), yet we persist with it. Correction methodologies that do produce worthwhile results are often complicated, time- and resource-intensive, and difficult for all but highly trained practitioners to implement successfully (West, 1971). Even when learning gains are made during conventional retraining, these improvements often fail to transfer to situations outside the original setting where the retraining took place. Learners may appear to make satisfactory progress while under the teacher s or trainer s close supervision, but they revert to their old incorrect way of doing things when left to their own devices or when they leave the instructional setting. This happens because the cues for correct performance, e.g. the presence of the teacher close at hand, are withdrawn. Consequently, short-term gains are not permanent and soon fade over time. Reversion to old incorrect habits in the face of stressful performance situations, i.e. poor transfer of learning, is commonly experienced in the world of sport (Maschette, 1985; Young, 1985), ballet (Khan et al, 1995), language learning (Mukattash, 1986), education (Read, 1987; Lyndon, 1989), management training and organisational development (Newstrom, 1983), and in almost every other field of human performance. Clearly, something is wrong with the theory underpinning conventional methods of error correction and habit reversal. Teaching Methods for Correcting Habit Errors Persistent errors, by definition, are resistant to correction by conventional means and have become the target for special treatment (Dawson & Lyndon, 1997). These special methods, described by Dole et al (1997), include error pattern analysis and intervention (Gable et al, 1991; Ashlock, 2001), cognitive conflict and conflict teaching (Bell et al, 1986), using errors as springboards for enquiry (Borassi, 1994), belief-based teaching (Rauff, 1994) and teaching by analogy (Tirosh, 1990). These methods can be difficult to implement, often lack a sound theoretical framework, are not effective with many students, and can 25

7 Paul Baxter et al have undesirable side effects, e.g. a loss of student self-esteem and selfconfidence when confronted by one s own error (Bell et al, 1986; Rowell & Dawson, 1983). A New Explanation of Habit Errors Lyndon (1989) has proposed that the observed lack of learning transfer and associated regression to erroneous ways is due to the welldocumented mental mechanism of proactive inhibition. Mental mechanisms that affect learning and memory have been widely studied by psychologists. One of these mechanisms, proactive inhibition (PI), is an interference effect on learning and memory produced by, conflicting associations that are learned prior to the learning of the task to be recalled (Underwood, 1966, p. 564). In effect, if what the person has learned previously is in conflict with the new material he or she is trying to learn, PI is involuntarily activated and interferes with the recall of the new material. This effect on the recall of new learning and the associated problems with transfer of learning to new settings have been well documented in many experimental manipulations of the proactive inhibitory mechanism (e.g. Postman & Gray, 1977; Miller et al, 1986). However, the implications of such interference for error correction and habit reversal, and for ways to accelerate learning were not sufficiently explored. Lyndon (1989), in a novel interpretation and synthesis of well researched and accepted psychological learning principles, has extended our understanding of the influence of PI in meaningful learning situations and in habit change, and produced an explanation of why habitual errors in knowledge and skill are so difficult to eradicate: Given that repetition of a behaviour is a sign that learning has occurred, consistent, habitual errors indicate the presence, rather than the absence, of learning. In this case, what the person knows is how to do it wrong. When new information or ideas disagree or conflict with what a person already knows, this conflict generates proactive inhibition (Underwood, 1957, 1966), that causes accelerated forgetting (Underwood, 1966) of the new knowledge or skill. PI does not prevent learning from occurring, it merely prevents the association of conflicting ideas (Underwood, 1966). It does not matter whether what the person already knows is correct or incorrect. PI protects all prior knowledge and skills because without an individual s cognitive intervention there is no differentiation between what is right and what is wrong in a given context. At the autonomous level it is the prior association that is retained. 26

8 RAPID SKILL DEVELOPMENT PI therefore exerts a protective effect on prior learning, inhibiting change and protecting erroneous (as well as correct) knowledge and skills. Performance becomes cue-dependent and the individual reverts to prior behaviour patterns when the trainer s or instructor s presence is withdrawn, thus inhibiting transfer of learning to other settings (Postman & Gray, 1977), ensuring that the erroneous knowledge and behaviour continue to resist correction. By a process of psychological interference, then, old learning disables new learning. According to Lyndon, PI and accelerated forgetting are the reasons why old habits die hard. A key point in the PI explanation of resistance to change is the notion that PI is automatically activated whenever new incoming information is different from what is already learned and stored. In practical terms, conventional, i.e. currently available, methods of teaching, training, coaching, instructing and other behaviour change methods, inadvertently activate PI, and actually make it harder for an individual to understand and adopt the new information or skill he or she is trying to learn. Furthermore, this process operates at the unconscious level and the teacher, as well as the learner, are unaware of the mental interference being generated. The only outward signs of mental conflict are the typical symptoms associated with the adaptation period to new knowledge, namely mental confusion, slowed performance, increased error rate and reversion to prior behaviour patterns when asked to perform independently or under stress. According to Lyndon, the adaptation period that we all go through when learning something new is actually symptomatic of interference from PI, and indicates an abnormal, rather than a natural, learning situation. The literature on behaviour changes, particularly in the fields of education, psychology and the enhancement of skilled performance in sport and work, emphasises the challenge posed by habit errors with their associated transfer of learning problem, yet offers few practical solutions for dealing with these profound and universal learning obstacles (Solomon, 1994; Bliss, 1995). Old Way New Way (Lyndon, 1989, 2000) offers a new perspective on this age-old problem. Old Way New Way Learning Lyndon (1989, 2000) has developed an innovative teaching method to deal with the interference effects of proactive inhibition. Old Way New Way is a novel synthesis and interpretation of existing and newly emerging concepts and principles, including automaticity in behaviour (Bargh & Chartrand, 1999), learned errors (Reason, 1990), the influence of 27

9 Paul Baxter et al prior learning (Ausubel, 1968), metacognition (Flavell, 1987), and proactive inhibition and accelerated forgetting (Underwood, 1957, 1966). Experimental and quasi-experimental studies and field trials in sport (Hanin et al, 2002), in mathematics and science education (Lyndon, 1989, 2000; Baxter & Dole, 1990; Dole, 1991, 1993, 1999; Dawson & Lyndon, 1997; Baxter et al, 1999; Henderson et al, 1999; Lyndon & Dawson, 1995; Rowell et al, 1990), in speech therapy (Lyndon & Malcolm, 1984) and in workplace training (Weaver et al, 2000) have given consistently positive results with Old Way New Way. Typically, after one successful correction session with Old Way New Way, the learner has an 80% or higher probability of performing in the new way, a 20% or lower probability of performing in the old way. and a 90% probability of self-detecting an old way when it occurs and then selfcorrecting it. Spontaneous recovery (Underwood, 1966) of the old way can be expected at 2-3 weeks after the original learning trial, but is easily handled. The purpose of the present study with vocational education teachers and students was to compare the relative effectiveness for skill correction and work habit change of Old Way New Way, compared with re-teaching or retraining, the most widely used approach to skill correction (Lyndon, 2000). An Experimental Trial of the Old Way New Way Method The aim of the current experiment was to determine whether Old Way New Way would be superior to re-teaching with a larger sample under rigorous experimental conditions in terms of: reduction in student error rates; reduction in time to achieve learning criterion (accelerated learning); persistence of any learning improvements. Students and teachers were also observed and interviewed to collect qualitative data on their reactions to the Old Way New Way teaching method. Method Subjects Experienced vocational education teachers from eight different trade areas identified the kinds of errors in skilled performance that students typically make when learning their respective trades. These teachers also identified students in their own classes who required skill correction. On the basis of unobtrusive baseline observation of these students conducted by independent trained observers during one session of 28

10 RAPID SKILL DEVELOPMENT normal classroom instruction, 35 vocational education students were selected for study from a sample pool of 130. Three of the 35 recruited students were excluded as ineligible, since they were taught Old Way New Way in a group, rather than individually (Old Way New Way can be used with groups, but that was not the focus of this experiment). Selected students ranged in age from 16 to 19 years. Thirty-two subjects were observed at baseline and of these, post-test data were available on 26 at the first session, 19 at the second, and 17 at the third and final post-test session. Measures Employed Measures for comparison included: procedural improvement (error rates); time to criterion (acceleration of learning); persistence of any learning improvements. Observation conducted during the experiment and interviews with teachers and students afterwards yielded qualitative data on the following: incidence of self-detection and self-correction of errors by students; student reactions; student understanding of concepts involved in performance of the skill; transfer to other areas of learning; teachers reactions. Procedure Selection of teachers. Thirteen teachers originally volunteered and nine completed the experiment. One teacher withdrew during the training period, another teacher only had night classes and an independent observer could not be found for those classes and two other teachers could not specify target errors for their current students to the required level of detail. The selected teachers ranged in age from 25 to 50 years and had been teaching from 5 to 20 years. Training of teachers and observers in the Old Way New Way methodology. Teachers and observers undertook four days of intensive training in the Old Way New Way methodology, and in classroom observation and recording of the selected target behaviours. This training involved role playing, simulations and demonstrations of errors using actual tools of the trade that teachers brought to training classes, as well as instruction in the theoretical underpinning of Old Way New Way and other error 29

11 Paul Baxter et al correction methodologies. The theoretical background of Old Way New Way was presented to teachers through discussion of the concepts of attention, remembering and forgetting from a psychological perspective situation. In particular, persistent errors and the influence of proactive inhibition as a factor inhibiting learning were related to vocational education and training. Full details of the training workshop content and procedures are contained in the Conceptual Mediation Program Workshop Handbook (Lyndon, 2003). The training period was also used to make a final selection of target behaviours/skill errors to be corrected. The data record forms used by observers to record target behaviours were also tested and refined during the training. The intensive training program was split into three parts: two consecutive training days that dealt with the theoretical concepts involved and included practical illustrations; one day used for site visits to the classrooms and workshops where observation would be conducted when the experiment started, to assess the suitability of the environment for observation purposes and to determine suitable strategies for observation; another day of training with role-playing, simulations, final selection of target behaviours, and testing and refinement of observation record forms. Qualitative baseline data on teachers and observers personal conceptions of learning, memory and error correction were also collected during the training period for comparison with similar data collected later in the experiment. Implementation of Old Way New Way training in the classroom. Implementation of Old Way New Way training occurred with most teachers in the week immediately following their training. However, not all teachers were teaching classes in that post-training week. Consequently, classroom implementation of Old Way New Way was delayed with some teachers. However, their ability to execute an accurate Old Way New Way intervention with a student was re-tested and, if necessary, refreshed prior to their use of Old Way New Way with students. Selection of student errors. Only one error was selected per student. To be selected, the error had to meet the following criteria: the error had to be significant to the skilled performance in that trade area, i.e. the flaw in performance had to be one that adversely affected student progress in acquiring competence; 30

12 RAPID SKILL DEVELOPMENT the error had to be persistent, i.e. learned and habitual (automatic) rather than a transient mistake ; it had to be resistant to correction by conventional means; it had to be common or typical of students in that skill area; it had to be a discrete component of overall performance, i.e. a behaviour or action that was easy to detect and observe and easy to count; the student had to be demonstrably able to reproduce the new (correct) way when shown this; if the student could not copy the new way after one demonstration by the teacher, then the behaviour was too complicated for this experiment and was not selected; the error had to be something that was easily distinguished from the correct action, e.g. it was easy to tell if the way the student gripped a saw was right or wrong. The following errors that were selected covered eight skill types: incorrect clipper blade use in pet grooming; incorrect grip when hammering nails; unsafe way of handling glass onto a cutting table; incorrect use of cutter when cutting glass; incorrect microscope use; incorrect pincurling technique in hairdressing; incorrect procedure used in soil ph testing; incorrect technique used in soldering. Examples of these student errors are described in more detail later in this article. All the skills that students were required to perform during preand post-tests were part of their normal lesson activities. Use of Old Way New Way scripts for teachers and observers. Separate written instructions were given to teachers and observers on the correct procedure for application of the Old Way New Way methodology. These instructions or scripts outlined the method s procedural steps, and helped to ensure uniformity in how different teachers applied Old Way New Way with students. The script also enabled observers to check on how accurately teachers implemented Old Way New Way. Although scripts were tailored to the specific error a teacher was correcting, an example of a script can be found in Lyndon (2003). Classroom observation. Observers were trained to observe and record target behaviours unobtrusively during normal lesson periods. To control for the possible interactive effects of observation, all students in each experimental group and in the control group were told by their teacher that the observer was present in class to observe the teacher s teaching methods, as part of an assignment that the observer had to complete for 31

13 Paul Baxter et al his or her own studies. Due to resource limitations, only one observer was used for each classroom observation session. The use of trained observers provided an additional control during the experiment because it enabled teachers to concentrate on their teaching without also having to observe and record the effects of the experimental treatments. Teachers kept a diary in which they recorded intervention notes, personal observations and any noteworthy occurrences. The length of observation periods was usually one full lesson, to reduce sampling error, although only that proportion of the lesson time that involved actual performance in the target skill was included in any given observation period. Each student was associated with a particular learning skill. Students were observed for between 30 and 90 minutes, during which observers recorded the number of performance attempts of the skill, and whether or not the skill had been correctly executed. These data were collected prior to any correction methods being employed (baseline) and at three post-correction sessions spaced at approximately weekly intervals. Because there was more opportunity to attempt some skills relative to others, the results were expressed as percentages correct out of all attempts at the skill, rather than as the number of correct attempts. Control of extraneous variables. The study was designed to minimise the influence of other variables by frequency matching on skill types so that similar skills were represented in each experimental group, by randomising the order in which teachers used the two correction methods (Old Way New Way and conventional correction) in their classes, and by having each student observed by only one observer through baseline and all subsequent post-test observation sessions. To facilitate the likelihood that target behaviours would occur more often and thereby improve sampling of target behaviours, the teacher set up lesson activities requiring repeated performance of the target behaviour. For example, students using a microscope had to set up and examine at least five different specimens. Pre- and post-test sessions imposed similar task demands on students, were of similar duration and provided the same opportunity to perform errors. Apart from the experimental group, other sources of variation identified as potentially having an effect on mean percentage correct rate were: gender of student; skill type; teacher; length of session; number of attempts. 32

14 RAPID SKILL DEVELOPMENT The latter two reflected opportunity to learn. Old Way New Way treatment. The Old Way New Way methodology required that the student was informed of the various steps in the Old Way New Way procedure prior to its commencement. This was necessary because Old Way New Way attempts to empower the student with a metacognitive strategy (a mediation) for self-correcting errors. The Old Way New Way treatment therefore consisted of four steps, namely: An explanation of the steps involved. The active behaviour change component, referred to as Old Way New Way. This process consists of four components, namely reactivation of the error memory, labelling of the person s own way, successive discrimination of the old and new ways, and generalisation or practice of the new way. A brief demonstration of a one-step procedure for dealing with student errors that teachers detect after completion of the initial Old Way New Way learning trial (in cases where 100% eradication of the error was not achieved after one Old Way New Way trial). Instructions for students on how to self-correct when students selfdetect an error. In this experiment, step 1 was abbreviated for practical considerations and consisted of a brief explanation. The four steps took, on average, 10 minutes to complete and are described in greater detail in Lyndon (1989, 2000, 2003) and in Hanin et al (2002). Conventional treatment. Teachers spent approximately 10 minutes with each student in the conventional group, correcting errors using the teacher s usual method of error correction that, for all teachers, followed a re-teaching approach. Re-teaching is the most frequently used remedial approach in the face of persistent learning failure (Lyndon, 2000). Control group. Students in the control group did not receive any error correction from the teacher during the pre- and post-tests. However, control group students spent the same amount of time under observation as the experimental groups. At the completion of the experiment, controls received catch-up instruction and all completed their studies successfully. Statistical analysis. The analytical approach of choice for the comparison of the effect of correction method over time was repeated measures analysis of variance modelling. In order to minimise the impact of missing data, a generalised estimating equations approach was used in these models to permit the inclusion of partial data records. An independent working correlation matrix structure was defined for these analyses, 33

15 Paul Baxter et al which were implemented using the SUDAAN statistical package (Shah et al, 1997). This permitted the simultaneous consideration of all potential sources of variation in percentage correct scores. Prior to modelling, the assumptions of normality of distribution of percentage correct scores and homogeneity of the variances in these scores over time were qualitatively considered through frequency distributions and summary measures of location (mean, median), variance, skewness and kurtosis, and more formally, through use of nonparametric Kolmogorov Smirnov tests for normality. There was practically no variation in baseline percentage correct scores, since the median percentage correct score was 0% for nearly all students, ranging from 0 to 39%. Baseline similarity of the three randomised groups was thus established using non-parametric Kruskall Wallis analysis of variance (ANOVA) and chi squared tests of association. Statistical significance was set at the conventional type I error rate of 5% (two-tailed). Results A total of 92 observations was available for 32 students, who contributed between 2 and 4 time points of data to the analysis. Baseline Similarity of Experimental Groups By design, eight skill groups were approximately equally represented in each of the three experimental groups. The Old Way New Way group included one clipper blade use, one cutting glass, one hammering nails, one handling glass, two microscope use, three pincurling, two soil ph testing and one soldering. The conventional error correction group included one less pincurling student and cutting glass was not represented in the control group. Students allocated to the three experimental groups (11 Old Way New Way, 11 conventional correction and 10 control) were similar with respect to gender (χ 2 = 0.62, df = 1, p = 0.431), median observation time (Kruskall Wallis ANOVA χ 2 = 0.226, p = 0.893), median number of attempts (Kruskall Wallis ANOVA χ 2 = 4.070, p = 0.131) and median percentage correct scores (Kruskall Wallis ANOVA χ 2 = 2.128, p = 0.345). The average percentage of correct attempts was nil within all three treatment groups and ranged no higher than 39% correct (Table I). 34

16 Experimental group % Male Median number attempts (min, max) Old Way New Way (n = 11) Conventional (n = 11) Control (n = 10) Median observation time (min, max) RAPID SKILL DEVELOPMENT Median per cent correct (min, max) 64 5 (3, 23) 55 (30, 90) 0 (0, 31) 90 5 (3, 19) 50 (30, 60) 0 (0, 0) 60 6 (4, 46) 60 (30, 60) 0 (0, 39) Table I. Pre-test (baseline) similarity of the three experimental groups. Effectiveness of Old Way New Way Overall and relative to pre-test levels, the average percentage of correct attempts improved at post-testing (time main effect, F 3,25 = 10.6, p < 0.001) from an average pre-test level of 2.8% (standard error 4.4), to 29.9% (5.9) at post-test 1, 45.1% (6.8) at post-test 2 and 47.2% (6.5) at post-test 3. However, error correction rates were significantly more improved for Old Way New Way compared to conventional correction (group by time interaction effect: F 6,20 = 32.6, p < 0.001; Table II and Figure 1). Old Way New Way (n = 11) Conventional (n = 11) Control (n = 10) Baseline % correct 2.8 (2.7) 0.0 (0.0) 3.9 (3.8) Post-test (10.8) 18.3 (10.6) 10.5 (5.6) Post-test (5.4) 24.0 (17.7) 11.8 (7.2) Post-test (5.2) 35.9 (14.2) 4.0 (3.6) Table II. Model-based estimated least squares means (and standard errors) of baseline percentage correct scores at pre- and each of the three post-test periods by experimental group. The significant post-test 1 improvement over baseline was maintained over the three post-test periods (model-based contrast considering time main effect restricted to the three post time points: F 2,26 = 0.77, p = 0.472), and this effect was not influenced by any differences in number of attempts (F 1,17 = 2.57, p = 0.119) or differences in observation times (F 1,17 = 0.12, p = 0.730). There were insufficient data to permit stable model estimates of the influence of skill type or teacher on the association between Old Way New Way, and percentage correct scores. An analysis to address partially the latter restricted the data to consideration of the first difference from baseline, wherein the treatment effectiveness of Old Way New Way over 35

17 Paul Baxter et al conventional and control treatments was not teacher-dependent (F 8,11 = 1.4, p = 0.313). Figure 1. Mean percentage correct scores at pre-intervention and by each of the three interventions over time. Incidence of Self-detection and Self-correction of Errors by Students Because the students in the experimental group achieved 100% or almost 100% accuracy in skilled performance after one correction session, few students were observed self-detecting and self-correcting errors. On three occasions teachers reported that they had observed students engaging in self-detection and self-correction. In one instance, a student was observed by his teacher to have reverted to an old way 2 weeks after the Old Way New Way session, but was then seen to immediately self-correct and was heard saying to himself, Almost got it wrong. In another instance, an Old Way New Way student was observed performing the skill incorrectly, but was then observed to pause in his action, rearrange the tool in his hand and then start using the tool correctly. Another student actually expressed to his teacher his own surprise that he had not reverted back to his old incorrect way of performing the skill. Student Reactions As previously stated, Old Way New Way was used without explaining to students the rationale for the method. Teachers merely followed the script. Consequently, although students were successful in rapidly correcting their skills, their initial reactions were mixed. Some reactions were positive as indicated by the following teachers notes: 36

18 RAPID SKILL DEVELOPMENT The student was keen to try a new method that would help. The student was very keen to learn anything. She took on Old Way New Way and participated willingly. The student was accepting, quite quickly, of the new way because he found that the new way required less physical exertion. The student reacted positively. The student was quite cooperative. Other student reactions ranged from active resistance to confusion, vague interest or no response, as shown by these teachers comments: The student was a little resistant. The student responded: Oh, not this again. Do I really have to do it again? After the fourth discrimination, the student became agitated about having to continually repeat the old way, then the new way so many times. At the teacher s request for the student to verbalise each of the steps for correct performance of the new way, the student responded: This feels like Play School in a rather defensive tone. Sighs, comments, eyes rolled, to signify that Old Way New Way was perceived as a bit of a rigmarole, or a bit over the top. The student was obviously frustrated and stated: I m not doing this one more time. Teachers gave various reasons to try to explain this observed student resistance: Many students just want to jump through the hoops and exit the course. They have an attitude of anything is good enough so I can get through the exam and onto the next module. The student was very negative. She felt that she didn t need correction, and felt exposed to the rest of the class for having to be continually corrected. In cases of obvious resistance to the method, teachers found that they had to encourage students to persist with Old Way New Way, as illustrated in these comments: Once the students became convinced to give it a try, they became more cooperative. 37

19 Paul Baxter et al Implementing Old Way New Way required persistence, and cajoling the student to finish the session. Other students in the experiment expressed confusion about the process or their role in the process. As one observer noted: The student was obliging throughout but was unsure of what the teacher wanted. The student thought that every time the teacher asked him what the difference between the old way and the new way was, that he was required to think up something new every time. This was confusing to the student. In terms of the Old Way New Way process, many students appeared confused about the need to do their old way after being shown the new way. The following observations illustrate the teachers attention to detail in explaining a skill to two different students. The students reactions to the teachers requests indicate the lack of understanding of Old Way New Way. The first dialogue is between a teacher (T) and a student (S) in a glass handling activity: T: Can I call the way you used to handle glass the old way? S: [nods affirmative]. T: I want you to look at the difference between the old way you were doing it and this way, which I will call the new way. Now, instead of bending over the bench, with the glass on a fortyfive degree angle, you must come vertically to the bench putting your whole body behind the glass, and bending your legs. I want you to try your old way for me. S: My old way? This next dialogue is between a different teacher and student in a glass cutting exercise: T: Run a cut for me. S: (Performs task as required). T: Now, what you are doing is putting a lot of pressure onto the cutter, and producing an uneven cut in the glass. Now, I ll show you the way I want you to do it. Now line up the cutter, grip gently rather than choke the cutter, follow through the cut with one smooth cut, and don t finish at the end of the glass. Follow through. Now try to do a cut your old way. S: My old way? Teachers reflected on these incidents, as follows: The student was keen to participate but confused about having to continually do the old way. The student had trouble doing accurate old ways, and this was the confusing part. 38

20 The student appeared apprehensive. He did not like me encouraging him to hold the hammer the wrong way. RAPID SKILL DEVELOPMENT Student Understanding of Concepts Involved in Performance of the Skill Observations of students using Old Way New Way to correct errors provided evidence to suggest that students understanding of concepts involved in performance of the skill improved. As previously stated, under Old Way New Way, many students had become more aware of aspects of their own performance, and could identify when they performed a skill correctly or incorrectly. The following examples describe the actual process of change occurring through the application of Old Way New Way and shows students becoming more able to distinguish the correct from the incorrect performance of the skill: After the third Old Way New Way discrimination (Step 2 in the process), the animal science student had to consciously think about the sequences in her old way and had difficulty reproducing the old way. Initially, the hairdressing student could not articulate the difference between her old way and the new way, but noted that the old way looked terrible. By the third Old Way New Way discrimination, the student was able to articulate the differences quite clearly. At the end of the session, the student was able to point out the differences very quickly, she could easily articulate the steps, and could perform the skill competently. She could also identify sloppy work in her own and others performance and correct it or offer advice. Initially, the glazing student could not articulate the difference between the old way and the new way, but after two more Old Way New Way discriminations, he could clearly articulate the differences between his old performance and the new way: Well, in my old way, my elbow is sticking out. In the new way, I m not pushing so hard on the glass. I am running the cutter gently, I am not gripping the cutter so hard. There was also evidence to suggest that Old Way New Way improved the precision of skilled performance. As noted by one teacher, Once the old way and the new way were contrasted, the student could easily identify the skill error. The new way, once mastered, was markedly neater and the student paid more attention to detail of skill execution. Another teacher commented to a fellow teacher that the Old Way New Way student s skill execution had greatly improved. He was observed to say, Alan s cutting has improved out of sight. It s excellent now. 39