ADMINISTRATOR CHARACTERISTICS AND THEIR INFLUENCE ON LEARNING THROUGH SIMULATION/GAMING

Transcription

1 Referencia: García-Carbonell, A. & Rising, B. Administrator Characteristics and their influence on Learning through Simulation and Gaming. Sidney: University of technology (UTS) ISSN: ADMINISTRATOR CHARACTERISTICS AND THEIR INFLUENCE ON LEARNING THROUGH SIMULATION/GAMING Amparo García Carbonell agarciac@upvnet.upv.es Universidad Politécnica de Valencia Beverly Rising brising@chs.upcomillas.es Universidad Pontificia Comillas This paper reviews the characteristics of the administrator or facilitator which are often mentioned as influencing the effectiveness of the simulation/game for learning. Characteristics such as age, gender, personality, teaching style, knowledge and skill are discussed and the results of a study of 11 university professors with 461 students are presented. Key words: Simulation/gaming; educational effectiveness; teacher variables; research The complexity of learning is recognized by almost all educators, with extensive lists of factors that supposedly influence student success. Language learning adds even more factors to these lists, as does the use of simulation/gaming. Nevertheless, the factor which we will call administrator seems to lead most people s lists. Lucas, Postma and Thompson (1974) demonstrated empirically that there was a significant difference in the cognitive performance of students, depending on the teacher using the methodology. Likewise, Kulik, Kulik and Cohen (1980:539), in their meta-analysis of computer-assisted university teaching-learning experiments, found that the only variable that could forecast results (after eliminating variables such as the type of experiment, the subject matter, the level, the type of publication in which the experiment appeared, etc.) was the use of a design that controlled the influence of the professor. Bredemeier and Greenblat (1981:157) cite administrative variables in first place in their list of factors which affect the educational effectiveness of simulations and games. As they say, it is generally accepted that the teacher who administers the simulation or game can significantly influence the learning, the change in attitude or the enjoyment of the students. In the study we conducted, our results support this statement, at least with reference to learning content. Briefly, our experiment consisted of a study of the vocabulary learnt by 461 universitylevel students using four different business games1 administered by 11 different professors (all native speakers), using pre-test and post-test scores based on a range of 0 to 10. Our data are as follows: 1

2 Professor # Number of Mean difference (postpre) Standard deviation subjects ,727 1, ,815 1, ,556 1, ,591 2, ,709 2, ,373 2, ,000 1, ,427 2, ,316 1, ,346 3, ,100,568 Total 461 It is easy to see that there definitely are differences (statistically in the ANOVA, F=7,175 and p<.001) in the learning produced by the different groups. The cause of these differences is much more difficult to ascertain. Bredemeier and Greenblat had proposed three sub-categories in the study of administrator variables, which are: 1. The style or personality of the teacher. 2. The attitude of the teacher toward the simulation 3. The teacher s knowledge and skill as a facilitator in the simulation. Administrator style or personality There are two studies that we know of that try to analize the relationship between the style of the administrator and the use or effectiveness of a simulation (Schreier and Basuray, 1981; Golden and Smith, 1989) Both use Bostrum s classification, The Inventory of Instructor Styles (1979), in which instructors are divided into four types: 1) the behaviorists who think that new behavior can be molded with well-designed exercises. The basis of this approach is that the teacher chooses the final behavior desired and projects a program of activities which systematically reinforces these aims. The process centers on the product, and the student can often control the speed of his/her learning. The second group is the structuralist who thinks of the administrator as the programmer of the student s mind. The premise here is that an organized content administered in small doses leads the student to learning. The process centers on the teacher and emphasizes planning and evaluation. The functionalists are the third group in Bostrum s classification. For them, people learn better when they do what they want to do. Motivation is very important and the process centers on solving problems and completing tasks, with the student as the center of the activity. 1 The business games used have been described in previous articles published by the authors. They are simple (now almost out-dated technically) computer-assisted and computer-controlled simulations, but the games themselves are not the subject of this paper. 2

3 The last group in this classification is the humanist who thinks that learning is based on self-controlled discovery. People teach themselves if others don t prevent them from doing so. The individual tries to reach a better understanding of him/herself and others. The focus is on interpersonal relations and there is no specific content. Schreier and Basuray (1981) studied the possible relationship between Bostrum s styles and the use of simulation and gaming as a teaching method. They analyzed the results obtained with 113 professors and found that, in principle, there was no realtionship between the professor s dominant style and his/her choice of simulation and gaming as a teaching method. Golden and Smith (1989) returned to the teaching style issue, looking at the fit between students learning style and teaching style. Their hypothesis was that the students whose style fit that of their teachers would learn more than the students who were mismatched. They used Bostrum s styles for teachers and Kolb s (1984) for the learners: assimilators, convergers, divergers or accomodators, according to where they fell on the axis of reflective observationactive experimentation and the axis of concrete experience-abstract conceptualization. According to Golden and Smith, the assimilator should fit with the behaviorist teacher, the converger with the structuralist, the accomodator with the functionalist and the diverger with the humanist. They studied 112 university business administration students and found that they could not explain the difference in learning on the basis of a fit between teacher and learner styles. Given the lack of concrete results in this area, we decided to study one factor of teaching style or personality: interactivity. For this purpose we classified the 13 teachers who participated in our experiment on the basis of a questionnaire which they had filled in before starting the simulations (the questionnaire itself is found in Appendix 1). Questions 7, 8, 9, 10, 11, 12, 15, 16, 17 and 18 were used, inverting the number of the answers in questions 9, 10, 11, 12, 15, 17, and 18, so that the highest number would correspond to the most interactive teacher. The range on this scale of interactivity was from 28 to 40 (44 was the highest number possible); the average was and the standard deviation was We decided that teachers with 32 points or less could be classified as not very interactive (0) and teachers with 39 points or more as very interactive (2). From 33 to 38 would be the somewhat interactive (1) group. Interestingly, the only two teachers classified as not very interactive did not participate with experimental groups (they taught control groups and had not volunteered to use the simulation), so we were left with 4 teachers in category 2 (very interactive) and 7 in category 1 (somewhat interactive). On calculating the Student t for these two groups, we found a significant difference in the post-pre change (t=5.493 and p<.001) with a moderate effect size (d=.513) in the 447 individuals studied. This effect was diluted, however, when the statistics were applied to the classes (26) as such (t=.723 and p=.477). Nevertheless, what was really surprising was that the students with better learning performance had the teachers who were less interactive on this scale. There are several possible explanations for these results. The most plausible, from our point of view, is that the students with the most interactive teachers were accustomed to this type of activity and although they enjoyed it (r=.137 and p<.001), found the simulation pertinent 3

4 to the subject (r=.133 and p<.001) and took notes (r=.110 and p<.001), the method was not innovative enough to rival the reaction in the students who were less used to this type of activity. It might also be possible that the more interactive teachers were emphasizing aspects of the simulation which were not reflected in the vocabulary tests or that there might also be a difference if teachers from the 0 category of interactivity had participated, instead of only categories 1 and 2. From a didactic point of view, these results could be interpreted as favorable in that it is not necessary to have an extremely interactive teacher to be able to take advantage of the learning potential of the simulation. Two other factors related to the administrator s style or personality can be mentioned here. They are the teacher s age and gender. In our experiment there were 4 teachers in the age range, 7 in the 40 to 50 age range and 2 who were over 50. An ANOVA showed no statistically significant difference between the three groups as far as individual student performance or class performance were concerned. In a study of correlations between the teacher s age and other variables, there were several interesting observations to be made: Students with teachers over 50 enjoyed the simulation more than those with teachers between 30 and 39 (r=.66 and p<.005), but there was no difference with regard to the students with teachers between 40 and 50 years of age. Students with older teachers (40-50 and over 50) found the simulations more suitable to the subject matter than students with younger teachers (r=.72 and.62 respectively). The age of the teachers had no effect on the amount of debriefing. Teachers in the age range were more interactive than either of the other ranges. (r=.531) In any case, it appears that the age of the administrator is not a decisive factor in learning from simulation. In a similar study of teacher gender and the effect on learning from simulation, it was found that at an individual level, the female teachers had a slight but significant lead over the male teachers (an average improvement on the post test of 2.78 compared to 2.15 with t=2.946 and p=.003), but at a class level there was no significant difference (t=.598 and p=.563). The fact that only three of the eleven teachers in the study were male could have influenced the results. As with age, it appears that teacher gender is not decisive in influencing the learning with simulation. The administrator s role in the simulation Much has been written on this point. Bredemeier and Greenblat (1981) talk of the difference between facilitation of learning and intervention in the learning process. A decade before, Fletcher (1971) had pointed out the change in the role of the administrator, since the 4

5 simulation/gaming process leads to self-evaluation and changes the administrator s role from that of judge to that of assistant. In 1977 Schreier classified the possible roles in eight categories: the administrator, the facilitator, the president, the presence, the consultant, the evaluator, the observer and the participant. It is clear in Schreier s article that he feels the correct roles are those of facilitator or observer, and that the effectiveness of the simulation is in danger if the personality or methodology of the teacher does not allow him/her to assume these roles. It is important to underline the importance of this role change in simulation/gaming methodology. As St. Germain and Laveault (1997:325) said, The role of the teacher is to help the students build their own learning strategy and understanding of the subject. Decision-making is internal to the student... Students are in charge of their own goals and source of motivation. A teacher who fears for his/her authority will have problems in the simulation. In many cases the simulation itself helps the teacher to assume the correct role. As there is no one correct answer and decisions grow out of previous ones, the teacher cannot intervene with correct answers because they don t exist. Duke and Greenblat (1981) give good advice on how to facilitate a simulation. In our study we found a negative correlation (r=-.129 and p<.001) between the help offered by the teacher and the students performance, i.e. the students learnt more with less teacher intervention. This would corroborate the role of facilitator as the most beneficial. The attitude of the administrator to the simulation Besides Bredemeier and Greenblat (1981), Wolfe (1985) and Hsu (1989) mention the attitude of the administrator as one of the factors that affect the learning from a simulation experience. User profiles have been put forward and some negative speculation (Wolfe, 1985) on the passive role sought by some users with regard to their classes has been mentioned, but the general enthusiasm reflected in the reports of many users over the years (Badgett, 1978; Dordick, 1997; Kalan, 1997) indicate that the majority use simulations to improve their students learning. Gosenpud and Washburn (1996) suggest in this regard that the intellectual behavior of the student is related to the teacher s expectations (Rosenthal effect). In this study we have considered the attitude toward the simulation as an aspect of the teachers interactivity: those teachers who are more willing to participate in interactive activities would also be more likely to have a favorable attitude toward simulation. All of the teachers who volunteered to work with simulation were classified in categories 1 or 2 on our scale of interactivity. Neither of the teachers in the 0 category participated. This is, of course, not statistically significant, but may indicate a tendency along these lines. Administrator knowledge and skill This aspect, mentioned by Bredemeier and Greenblat (1981), appears logical; more knowledge and skill should lead to more learning. Wolfe (1985), however, complains that several universities use teaching assistants, with little knowledge of the subject and little skill in 5

6 simulation facilitation, to handle simulation use, depending for success on the robustness of the simulation itself. Various studies (Baker, Jensen and Kolb, 1997; Lederman, 1992; McMahon and Coote, 1990) underline the importance of the briefing phase on the simulation and, especially, the debriefing phase. A teacher with little knowledge of the simulation and gaming methodology would not be able to facilitate either of these important phases. In our study we have looked at two aspects of this question: the confidence the teacher expressed in a pre-simulation questionnaire concerning the subject matter (in our case, English for specific purposes, i.e. English for the professional world with economic or technical vocabulary and concepts) and concerning the use of the computer in classroom work, and the experience the teacher had with the use of simulation. With a cursory look at the following graph, it is easy to see that two of the teachers (#5 and #6) used the simulations with many more students than the others, in addition to publishing articles in the field. It was felt that the performance of their students could be compared to the performance of the other teachers students to see if there was any difference. Número de alumnos experimentales por profesor Prof. 14 (10) Prof. 10 (26) Prof. 9 (20) Prof. 8 (11) Prof. 7 (13) Prof. 1 (44) Prof. 2 (20) Prof. 3 (19) Prof. 4 (23) Prof. 6 (108) Prof. 5 (167) No significant difference on this point (t=1.175 and p=.257) was found. On the second point, the confidence expressed with regard to subject matter and computer use, we classified the teachers according to their answers to the pre-simulation questionnaire. Of the 20 points possible, the minimum was 10 and the maximum was 17, with an average of and a standard deviation of We used Pearson s correlation to see if there was any relationship between this factor and student performance, finding a relation of r=.140 (p=.003) between this factor and the gain score in individual subjects, indicating that the more confident teachers achieved better results. 6

7 To sum up, we should first give a word of warning concerning our results. The eleven teachers who participated in this study all had more than nine years of teaching experience and volunteered to carry out the simulation; this in itself may indicate a specific group who could be differentiated from other teachers. Nevertheless, we can make some claims from the data we have, at least as to tendencies: it appears that knowledge and skill are more important than age or gender and that the role of facilitator is more effective than that of interventionist. More studies on these and other administrative variables are necessary. References Badgett, T.F., Brenenstuhl, D.C. & Marshall, W.J. (1978). An Analysis of Performance in Simulation Games Compared to Performance on Structured Course Criteria; A Case Study. En Brenenstuhl, D.C. & Certo S.C.(eds.). Exploring Experiential Learning: Simulations and Experiential Exercises. Tempe, Arizona: Bureau of Business and Economic Research, College of Business Administration, Arizona State University Baker, Ann C., Jensen, Patricia J., & Kolb, David A. (1997). In Conversation: Transforming Experience into Learning. Simulation & Gaming, 28(1) Bredemeier, M.E. & Greenblat, C.S. (1981). The Educational Effectiveness of Simulation Games: A Synthesis of Findings. Simulation & Games, 12(3) Dordick, Michael (1997). A Computerized Simulation for ESL Composition Students: Building a House. Simulation & Gaming, 28(1) Duke, R.D. & Greenblat, C. (1981). Running Games: A Guide for Game Operators. En Greenblat, Cathy Stein & Duke, Richard D. (eds.). Principles and Practices of Gaming & Simulation. London: Sage Publications. Fletcher, J.L. (1971). The Effectiveness of Simulation Games as Learning Environments: A Proposed Program of Research. Simulation & Games, December Golden, P.A. & Smith, J.R. (1989). Simulation Performance Revisited: The Fit Between Instructor Style and Learning Style. En Pray T. & Wingender J. (eds.). Developments in Business Simulation & Experiential Exercises. Stillwater, Oklahoma: College of Business Administration, Oklahoma State University Gosenpud, J. & Washbush, J.B. (1996). Total Enterprise Simulation Performance as a Function of Myers-Briggs Personality Type Simulation & Gaming, 27(2) Hsu, E. (1989). Role-Event Gaming, Simulation in Management Education: A Conceptual Framework and Review. Simulation & Games, 20(4). London: Sage Publications Kaplan, Marsha A. (1997). Learning to Converse in a Foreign Language: The RECEPTION GAME. Simulation & Gaming, 28(1) Kulik, J.A., Kulik, C.C. & Cohen, P. A. (1980). Effectiveness of Computer-based College Teaching: A Meta-analysis of Findings. Review of Educational Research, 50(4) Lederman, L.C. (1992). Debriefing: Toward a Systematic Assessment of Theory and Practice. Simulation & Gaming, 23(2) Lucas, L.A., Postma, C. & Thompson, J.C. (1974). A Comparative Study of Cognitive Retention Using Simulation-Gaming as Opposed to Lecture-Discussion Techniques. Paper Presented at the American Educational Research Association Annual Meeting, Chicago. McMahon, L., & Coote, A. (1990). Debriefing simulation games: Personal reflections. En Crookall et al. (eds.). Simulation & Gaming in Education and Training. Oxford: Pergamon Press Schreier, J.W. and Basuray, T. (1981). Teaching Styles in Simulation: Experiential Learning Versus "Traditional " Teaching Styles. En Biggs, W.D. & Fritzsche, D.J. (eds.). Development in Business Simulation & Experiential Learning. Normal, IL: College of Business, Illinois State University St. Germain, Michel & Laveault, Dany (1997). Factors of Success of Simulation and Games: A Systematic Approach to the Evaluation of an Organization s Impact on the User. Simulation & Gaming, 28(1) Wolfe, J. (1985). The Teaching Effectiveness of Games in Collegiate Business Courses: A Update. Simulation & Games, 16(3)

8 Appendix 1 SURVEY OF TEACHERS 1. Years of teaching experience a. less than 1 year d. 7-9 years b. 1-3 years e. more than 9 years c. 4-6 years 2. Age a. under 30 c b d. over Sex a. female b. male 4. Familiarity with microcomputers a. I own one b. I use one at work often c. I use one at work occasionally d. I have never used one 5. Use of microcomputer in teaching a. for exercises and exams b. in-class exercises c. simulations d. none 6. If you have never used a microcomputer for in-class work, why? a. lack of resources b. lack of time c. lack of interest d. lack of belief in usefulness e. lack of knowledge about computers f. lack of flexibility in syllabus 7. Do you use group/pair work? a. never b. once in a while (1 in 10 classes) c. occasionally (1 in 5 classes) d. often (1 in 2 classes) e. in almost every class 8. Explanations of grammar/vocabulary take up a. more than 70% of class time b % of class time c % of class time d. less than 30% of class time 9. Students go to lab or I use video or cassette or OHP a. every class b. often c. occasionally 8

9 d. once in a while e. never 10. I use project work a. every year with all groups b. every year with some groups c. sometimes but not regularly d. never 11. I use role-play a. often b. sometimes c. never 12. I use songs and games a. often b. sometimes c. never 13. If I use songs and games, it a. depends on the group b. as a light relief or prize c. for learning purposes 14. Correction of exercises in my classes is a. usually by the teacher b. by peers in groups c. by peers under the teacher s supervision MARK YOUR AGREEMENT OR DISAGREEMENT WITH THE FOLLOWING SENTENCES ACCORDING TO THIS SCALE: A B C D E agree agree neutral disagree disagree strongly moderately moderately strongly 15. Students learn more by doing than by hearing. A B C D E 16. The teacher should be in control at all times. A B C D E 17. Teachers can learn about their own subject from students. A B C D E 18. Different students require different teaching methods. A B C D E 19. ESP is no different than general English teaching A B C D E 20. A teacher needn t be an expert in business to teach business English. A B C D E 9