Evaluating teachers guides Alan Cunningsworth and Paul Kusel

Evaluating teachers guides Alan Cunningsworth and Paul Kusel Teachers guides (TGs) are an important part of a materials package, especially for teachers who are less experienced, or whose English is not strong. The availability and effectiveness of a TG can contribute greatly to achieving a good standard of teaching, through the provision of information about the language, guidance on teaching procedures, and a rationale for the course. This article considers the functions of TGs, and suggests why evaluation of TGs is a worthwhile activity. It then goes on to examine the evaluation process, with a consideration of how teachers can become actively involved in it, and proposes a set of model criteria for global appraisal and more detailed evaluation of TGs. Introduction Our current interest in the evaluation of teachers guides (TGs) was kindled by a short seminar we conducted with a group of teacher-trainers in southern Africa who were working with a course called Bridge to English in which the TGs are of central importance (Molteno Project, 1987). The detailed resources, guidance, suggestions, and instructions which they provide are crucial to the successful use of the materials, and teachers using the course must understand the TGs and relate to them effectively if their use of the course material is to be productive. The process and the outcome of developing evaluation criteria for TGs were illuminating, and prompted us to consider further the role of TGs and how their content might be analysed and evaluated. A neglected topic This topic is a neglected area in the literature. Hywel Coleman s paper (Coleman, 1985) provides some perceptive insights into what is involved in the process of TG evaluation, supported by some illuminating examples. His major criteria seek out the assumptions on which the TG is based: assumptions about shared attitudes towards language, language learning, and methodology - and assumptions about teachers attitudes towards educational issues and their ability and willingness to handle ambiguity, open-endedness, and incompleteness in the TG. He also considers how well the guidance contained in the TG is organized and expressed. Other writers on the subject of materials evaluation, for example Cunningsworth (1984), Harmer (1983), and Sheldon (1988), either make only brief reference to TGs or ignore them completely. This is perhaps understandable where TGs form only a small part of the total course package or are viewed as being of relatively minor importance, but is less acceptable where courses are being evaluated in which TGs occupy a central position and may well be of greater importance than the student s book. This is 128 ELT Journal Volume 45/2 April 1991 Oxford University Press 1991

typically the case where English is being taught in a non-english-speaking environment and where teachers, for whatever reason, are heavily dependent on the TG for methodological guidance, for linguistic information, or for insights into cultural issues. The functions of Before considering the merits of a planned approach to evaluating TGs, teachers guides we should discuss their role, their functions, and the manner of their use. It is curious that TGs seldom provide explicit statements concerning their function or use, but where these are provided, they help the teacher to form a clearer perception of the role and potential of the TG, allowing it to be employed more effectively. The broad variation found amongst teachers of English across the world is understandably reflected in a similar diversity in the objectives of TGs. For example, the functions of a TG designed for use by teachers in the developing world will differ considerably from those of a TG for teachers who have ample training, rich resources, and small classes. We suggest that each TG will have some, or perhaps functions: all, of the following 1 To provide a statement of the general purpose of the teaching material and describe the linguistic and/or methodological rationale; 2 to encourage the development of teaching skills generally, going beyond the specific skills needed to utilize the class material; 3 to assist the teacher in understanding the structure of the course material and the contribution of each lesson or unit to the overall course; 4 to provide guidance in the practical use of the material; 5 to provide linguistic and cultural information necessary for the effective use of the material in class. The evaluation process enables these functions to be profiled and the profile obtained to be compared with the perceived needs of the users. The closeness of the match between what the TG provides and what the users need is probably the ultimate benchmark for evaluation. Just how much individual teachers can benefit from a particular TG depends largely on the closeness of this match. TGs are important and deserve careful construction; however, it is not unusual for TGs to be written in some haste, as a postscript to the class materials, perhaps under pressure from publishers who want to get the material to the market. The problem is, though, that if a TG is deficient, through omission, inaccuracy, or whatever other reason, the quality of teaching is likely to suffer. A TG carefully attuned to its readership, on the other hand, can enhance teaching quality at remarkably low cost. Evaluation: the We make a distinction between global appraisal, which focuses on the process and the underlying approach of TGs and the broad assumptions they make, and criteria detailed evaluation, which deals with the way in which the objectives, Evaluating teachers guides 129

content, teaching procedures, and so on are spelt out for each unit of the course book. Global appraisal is outlined first in this article because the questions it raises underlie the whole evaluation process. However, the answers to some of the questions asked under this heading may not emerge until towards the end of the process, when the detailed evaluation is being made. Although we identify a number of separate criteria for evaluation, both globally and in detail, these criteria are by no means unconnected and separate. They should not be seen as pigeon holes, convenient though that would be were it feasible, but rather as different strands in a whole: interdependent and each contributing to a common goal. Global appraisal The main aim of TGs must relate to the effective use of the course book and supporting material to which they relate. But TGs may have an additional and rather wider role, which is to elucidate the general principles on which the material is based. TGs which attempt to do this are to some extent going beyond the strictly practical purpose of advising the teacher on how to handle a particular unit of the course book or how to conduct a particular activity within the unit, and are undertaking a more fundamental task of relating the course book, and the attitudes and values which it embodies, to the teacher s understanding of the principles which underlie language teaching. Anyone writing a TG along these broader lines has to make difficult assumptions about the readership of the guide. Typically, writers based in Britain or the USA assume that they are writing for a relatively homogeneous body of teachers who are probably native speakers of either British or American English, and who are trained to a greater or lesser extent in the contemporary orthodoxies of TEFL in the western world. They may assume, for example, an understanding of the concept of communicative language teaching and communicative methodology, or a short summary may be given to set out the ground rules of what is understood by these often-used but rather broad labels. In the TG to Developing Strategies (Abbs and Freebairn, 1980b), a third-level course designed primarily for multilingual classes of adults, we find reference to the principles of a communicative course with a brief but useful summary of functional categories and their relationship to grammatical form; of appropriateness; and of communicative methodology. This last concept is outlined in two paragraphs concentrating on the importance of purposeful communication and the encouragement of active involvement on the part of the learners. Similarly, in the TG to Headway Intermediate (Soars and Soars, 1986), also designed for adult learners, there is early reference to what is generally considered to be a communicative methodology, followed by a list of seven characteristics which exemplify communicative methodology and which focus on the role of the learner, and to a lesser extent the teacher, in the learning process. 130 Alan Cunningsworth and Paul Kusel

However, the majority of teachers of English in the world are not native speakers, nor do many have the benefit of a good standard of training and well-resourced teaching conditions. As Coleman (1985) reminds us, the difficulties which the non-native speaker teacher of English has to struggle with are frequently underestimated by the writers of ELT materials and by writers on ELT. We shall attempt to take this wider constituency into account and to view TGs in relation to what is probably their more crucial role, not to provide guidance to experienced and welltrained native speakers, but to provide essential support for teachers whose own knowledge of English is shaky and incomplete, who perhaps only have a basic understanding of some of the rudimentaries of language teaching and who are in need of confidence-building measures. Consequently, a major principle of our approach to TG evaluation is that a TG should meet the needs of its users as fully as possible and should be eminently flexible. If it is addressed to a wide range of users, then it is the needs of the least well trained, the least knowledgeable and the least confident that should be met as a priority. An important In making a global appraisal of the way the TG handles the main paraquestion meters which go into the design and make-up of a course, one very general but important question is: How explicit is the TG in expressing the views of the writer(s)? Are they clearly stated or are the values and assumptions implicit, leaving the user to tease them out? The answers to this question have important implications, because a clear statement of the underlying approach makes it easier to identify and uncover the implicit, unwritten assumptions contained in any TG, and thus provides a firmer foundation for using the more specific guidance provided. Some relevant Going on to examine what the underlying approach is, with its assumpcriteria tions about the capabilities and needs of the teachers using the TG, the following criteria emerge: - What assumptions are made about the users knowledge and experience of language teaching? - What assumptions are made about their knowledge of English? - What assumptions are made about their awareness of the cultural context portrayed in the course material? - How confident are the teachers assumed to be? Building on the answers to the above, we can ask these questions: - Does the TG provide enough detailed information on language and methods? - Is the advice given on teaching procedures explicit enough? - Is there enough cultural information to enable teachers to interpret appropriately the situations represented in the teaching material? Evaluating teachers guides 131

- How prescriptive is the guidance provided? (Bearing in mind that a moderately prescriptive approach may be preferable in certain circumstances.) Information about A global appraisal will take account of the TG s views of the nature of language and language and of the nature of the language-learning process, leaving conlanguage learning siderations of how individual items are taught to the detailed evaluation. The following criteria are relevant here: - Which aspects of language are covered in the TG? - In what terms is the language content formulated, and in how much detail? - What does the TG say about the language-learning process? - Is there reference to different learning styles and strategies, and are there suggestions for using and developing them? - Is the role of the teacher considered, possibly with reference to changing roles according to the nature of the learning activity? Some TGs make specific reference to the way in which their courses select and organize the language content and to whether they emphasize a structural view of language or a functional approach, or indeed a synthesis of the two. Similarly, some TGs refer to the way in which they view the learning process and the implications of this for the management of classroom activities and the organization of lessons. Some TGs are now paying attention to the development of independent learning strategies so that learners can continue to learn effectively outside the classroom. Vocabulary development strategies are a case in point and the Headway Intermediate Teacher s Book states that great attention is given to the... development of vocabulary learning skills for students to continue learning lexis on their own (Soars and Soars, 1986). Developing A further global criterion concerns the extent to which the TG develops teachers teachers general awareness and understanding of language teaching awareness of theory and principle: theory - D oes the TG assume a teacher-development rule, by providing a rationale for the information and guidance it provides? - Does it help users to gain more understanding of the language-teaching principles involved, in addition to helping them to develop their practical teaching skills? 132 Alan Cunningsworth and Paul Kusel

It is doubtful whether a TG should be expected to fulfil the function of a methods manual, or a teacher-training course, yet in situations where teachers have access to neither and the TG is the only means of support and development, then any developmental content provided may have a disproportionately beneficial result. A good example of this is to be found in the introduction to the teacher s guide to English Every Day Book 1 (Ministry of Education, Colombo 1985), a year-seven school course currently in use in Sri Lanka. The six pages of the introduction provide a clear account of the general methodological approach adopted and elucidate the perceived nature of language learning and its relation to classroom activity and dynamic. For instance, the suitability of an activity-based approach to language learning, rather than a lecture-based approach, is discussed in an accessible and persuasive manner. Unfortunately, this level of support is not always available in those course books where it is perhaps most needed. For example, the TG for Early English 1 (Levasan, 1983), a primary beginners course for Malaysia, consists largely of instructions to the teacher, apart from a six-line paragraph emphasizing the importance of active learning. Detailed Detailed evaluation concerns the way the TG deals with different aspects evaluation of the course and with how each unit, etc. should be taught. For the sake of brevity, we have chosen to focus on those aspects which seem to us to be most important. Objectives and Assuming that the general aim of the course has been made clear in the content TG, with reference to level, age of learner, areas covered, and so on, the first specific criterion concerns the specification of objectives for each unit and the way in which they are formulated. Secondly, we move to the actual content of each unit, and the way in which it is organized. In the case of both content and objectives, the categories used for description will vary according to the view of language form and use taken by the writer(s) of the TG and may include some or all of the following: - How are the objectives and content set out in the TG? Possible categories include these: grammar items vocabulary items pronunciation functional units situational features topics covered skills practised (reading, writing, speaking, listening) - What information is given about the language items to be taught, and what guidance is given about handling them? Are there any helpful notes about possible problems? A good example of a straightforward but sufficiently detailed explanation of a grammar item is the Grammar Note referring to the use of the present perfect in Bridge Plus One Teacher s Manual: Evaluating teachers guides 133

The Perfect Tense of a verb means an action which was done at a time before writing or speaking the sentence in which it stands, and the results of the action are still effective (present) at the time of writing or speaking. (Molteno Project, 1987) Headway Intermediate Teacher s Book gives a similar grammatical explanation and adds, for the support of the teacher who may become disspirited at the difficulty encountered in attempting to teach the present perfect, this use presents problems, as the rule is intangible... Do not expect rapid mastery. Students can get very frustrated in their attempts to grapple with this area. (Soars and Soars, 1986) Teachers using materials that concentrate on specific aspects of language may require a more extended theoretical description to provide them with adequate background knowledge. An interesting example of such a description is to be found in the TG for Intonation in Context (Bradford, 1988), which provides pronunciation material for more advanced learners. In the introduction, the teacher is reminded of the problems posed by attempting to teach intonation; this is followed by a summary of the theory upon which the material is based and by a glossary of terms. Some understanding of this theoretical background is essential for the teacher who wishes to make effective use of the material. Cultural loading Cultural awareness can present problems and the potential difficulties are not always foreseen by TG writers, as this anecdote illustrates. A teacher from Jordan once asked us whereabouts in Britain Unigate was. Somewhat perplexed, we asked where he had come across the name and we were referred to Developing Strategies Unit 3, where there is a text containing the sentence The hero is Jim Charvill, a milkman from Unigate (Abbs and Freebairn, 1980a). Although the visuals in the Student s Book clearly show a milkman on his round, there is no clear picture of the name Unigate. Nor is there any mention of it in the Teacher s Book. Consequently, there is no way that a teacher unfamiliar with Britain could work out or discover what Unigate meant. Certainly no dictionary would contain the name. (For readers who are themselves confused by this, we should add that Unigate is the name of a large dairy company in Britain.) We therefore need to ask: - Does the TG adequately predict difficulties in understanding the cultural setting or background? - Does it deal with these difficulties by providing sufficient information and explanation? Procedural Teachers will expect a TG to provide explicit suggestions for the planning guidance and conduct of lessons. They will seek guidance in what to do and how to do it. Lessons require prior planning and preparation of materials; they involve conducting class activities in the sequence planned. Teachers 134 Alan Cunningsworth and Paul Kusel

need to respond to the particular events and developments within the lesson and to reflect on and note the strengths and weaknesses of the lesson once it is over. The extent to which each TG assists the teacher to do all these can be assessed by asking the following questions: - What is the extent and nature of the guidance in selecting and sequencing units, planning them into a scheme of work, and thereby integrating them into the overall learning programme? - In what manner and in what detail are procedures suggested for the planning, preparation, and conduct of lessons? It may turn out that the teacher is absolved of much responsibility here. Some TGs provide a detailed lesson plan for every unit of the material, in some cases even going as far as explicit description of every activity, including the language to be used by the teacher. Further support may be provided by spelling out just what materials and aids are required. Other TGs simply provide a small number of specimen lesson plans that cover every type of activity in the material. In this case, it is left to the teacher to construct analogous plans for the remaining units. Finally, certain TGs provide no lesson plans as such, but merely outline ways of conducting the various types of activity, for example, drill, reading practice, and role play. A splendid example of a successful combination of detailed lesson plans and more general guidance is to be found in the TG for Enjoy Learning English (Mills et al., 1986), a course for secondary schools in Frenchspeaking Africa. The first half of the TG consists of short, clear sections on general techniques and procedures such as How to use the pictures, How to use the grammar summaries, and How to teach pronunciation. There are twenty-seven such sections, indicating the breadth of practical guidance provided. The second half of the book opens with information about the structure of the course and the interrelationship of the units, and provides guidance concerning the timetabling of the course over one school year. This is followed by detailed lesson notes for the first unit and much briefer notes for each of the remaining units. This book is made easy to use by the clear indexing and layout of the material, and seems to us to demonstrate an excellent balance in the guidance it provides for its intended readership. Advice about the Guidance on less tangible aspects of the lesson can also be evaluated: unpredictable - D oes the TG assist the teacher in dealing with the unpredictable, for example in selecting optional routes through the lesson, or in handling activities that do not go according to plan? To advise on the unpredictable is arguably beyond the scope of TGs. However, some guidance on alternatives is feasible and is often provided. An illustration of how a page layout can be used to clarify alternatives may be found in the Teacher s Book for New Dimensions 1 (Lonergan and Gordon, 1986), an intensive course for adults. A left-hand column describes the standard lesson activities and a right-hand column is Evaluating teachers guides 13.5

reserved for notes about how these activities may be adapted or extended in particular ways. Admissions that activities may fail and need rescuing are understandably harder to find in TGs. We did find a few in Lonergan and Gordon (ibid.) such as this recommendation concerning an exercise in which students were asked to write paragraphs on well-known writers: There may not be much interest in these writers if they are unknown to the students... Elicit information in note form on to the blackboard about a writer or famous person in their own countries. Include some dates. Make sure the person is still alive. Ask the students to write a paragraph about this person. Advice of this kind can avert disaster in future TGs. and we might hope to see more of it Correction and An intrinsic part of language teaching concerns evaluating and respondtesting ing to the language that students produce. Correction and testing, examined separately here, are essentially related activities with different goals. Correction aims to supply feedback to the student and could be assessed by the following: - Is the teacher advised when and how to correct students language? - Does the TG contain suggestions for ways students might respond to correction? Correction requires knowledge of the correct or appropriate answers on the part of the teacher. Assistance with this can be assessed by asking: - To what extent does the TG provide keys of answers to tasks set? Keys are particularly helpful if they provide alternative answers and, in the case of open-ended tasks, if they provide specimen answers. Testing is a broad activity whose purpose is to provide information about the stage of learning reached by the student. To evaluate this we can ask: - Is there adequate guidance in the checking of learning both informally, through practice activities, and more formally, through revision units and achievement tests? Motivation Learner motivation is a major factor in success in language learning and we should consider the following: - Does the TG make a positive contribution to heightening and sustaining learner motivation? It can do this by providing the teacher with advice on including sufficient variety of activity in lessons, using topics of real interest to the learners, encouraging learners to discuss personal concerns and interests, and making use of games and other fun activities. Advice of this kind may be found throughout a TG. However, a section devoted to motivating learners will underline its importance and provide ready access to general guidance on the matter. An illustration of good 136 Alan Cunningsworth and Paul Kusel

practical advice of this kind is provided by Mills et al. (1986) in a section entitled Help your students to learn : Find out about your students interests. Who is in the football team? Who works in the market? Who likes records? Talk about these things in English. Above all encourage the students - never call anyone stupid, never write nul on a report. Help them to feel they are making progress. Presentation and Method and ease of use of a TG could be assessed thus: use - Does the TG suggest how it can best be used? - To what extent do the organization and layout of the contents make it easy to use? The TG for English Every Day (Ministry of Education, Colombo, 1985) illustrates the inconsistency in quality found in certain TGs. In the preface. the authors appear to be responding to the first criterion above: This guide forms an indispensible part of English Every Day, Book One, and the year 7 course cannot be followed without it. The notes are set out by activity types to save repetition of general points through every unit. The general introductions to each section are therefore very important. All the suggestions for activities within the units are designed to make your work more varied and enjoyable and create a more effective learning environment for your pupils. Though some close reading and extra preparation may be necessary at first, it is hoped you and your pupils will find the results rewarding. This promising start to the book, however, heralds a disappointment and much difficulty. Use of the book for lesson preparation requires that the teacher hunt amongst the contents of ten different sections with headings such as Word Study, Grammar in Action, and Role Play in order to find specific notes and guidance relating to one particular lesson. Many teachers will be daunted by such a time-consuming task, which is not made any easier by poor layout. TGs not in English Teachers guides for UK-produced courses are appearing in some countries, notably Italy, France, and Spain, written in the languages of these countries. In such cases, the benefits of comprehensibility and ease of use may be accompanied by disadvantages such as the loss of experience in pedagogic metalanguage in English. We therefore need to ask: - What language is the TG written in? If it is the native language of the users, are there persuasive reasons for this? If it is in English, is the style direct and comprehensible to non-native speakers? Lesson evaluation None of the TGs we have seen appear to address themselves to the evaluation of lessons taught. One suggestion made at our seminar was that where lesson plans were provided for every unit, an empty space might be left after each plan so that teachers could note down their own evaluative Evaluating teachers guides 137

remarks. Such remarks would be of greatest use here as they would automatically be at hand when the lesson was next taught. We therefore include the following criterion: - Is the teacher encouraged to evaluate each lesson, and if so are there any suggestions how this might be done? Practical uses of The information and insights gained from evaluating TGs have valuable TG evaluation practical benefits. We will illustrate some of these by drawing on our Teachers can experience with the trainers referred to at the beginning of this article. It establish their own seemed important to us that the participants should begin by establishing criteria their own evaluation criteria so that: a. they could develop their own particular perceptions of each criterion; and b. they could have the opportunity to include criteria that we missed or considered unimportant. A workshop phase followed, in which small groups evaluated the TGs using the criteria that had been agreed. The findings of the groups were then communicated and compared in a plenary feedback session. What was to be gained by such a thorough appraisal of the TGs? The participants felt that a clear understanding of the tenets of the Bridge to English TG and its relative strengths and weaknesses would assist their teacher-training work. Once they had been identified, strengths could be built upon, and weaknesses could be compensated for. For example, in the TG being evaluated, objectives were stated for each unit. On closer examination, it became clear that the objectives, although undifferentiated in the TG, varied widely in nature. Objectives were expressed in terms of grammar items, functions, situations, skills, and, in some cases, as aspects of social behaviour such as going to the market. An awareness on the part of the trainers of this potential source of confusion enabled them to decide how to handle the topic of objectives in their TG-related training sessions, so as to show a clear categorization of different types of objective and to demonstrate the relationship between objectives and lesson content. Improved effective We also came to feel strongly that TG evaluation could be of great use to use of TGs other groups. For instance, teachers on in-service courses would benefit from evaluating a TG familiar to them; they would appreciate just what the guide had to offer and make more effective use of it. Furthermore, because evaluation by users reveals practical deficiencies, the results of evaluation are prime feedback for publishers when course material becomes due for revision. Evaluation of TGs can also be useful to those responsible for the selection of materials, and to authors when they commence work on a new TG. Evaluating TGs is an activity of potential value to all who are concerned with teaching materials, from those who produce them to those who use them day by day in the classroom. 138 Alan Cunningsworth and Paul Kusel

Conclusion The criteria that we have suggested here are by no means definitive or comprehensive, but are intended to act as an illustrative framework. We hold that evaluation of this kind is a dynamic process which consists of establishing criteria, applying these criteria to specific TGs, and acting on the outcome. There are benefits to be obtained at each stage of the process in terms of greater insights and understanding on the part of those involved, in addition to the achievement of a worthwhile end-product - a fuller appraisal of teachers guides. Received April 1990 References Abbs, B. and I. Freebairn. 1980a. Developing Strategies Students Book. Harlow: Longman. Abbs, B. and I. Freebairn. 1980b. Developing Strategies Teacher s Book. Harlow: Longman. Alderson, J. C. (ed.). 1985. Lancaster Practical Papers in English Language Education, Vol. 6: Evaluation. Oxford: Pergamon Press. Bradford, B. 1988. Intonation in Context. Cambridge: Cambridge University Press. Coleman, H. 1985. Evaluating teachers guides: do teachers guides guide teachers? in Alderson (ed.) 1985. Cunningsworth, A. 1984. Evaluating and Selecting EFL Teaching Material. London: Heinemann. Harmer, J. 1983. The Practice of English Language Teaching. Harlow: Longman. Levasan, M. 1983. Early English 1. Teacher s Guide. Kuala Lumpur: Longman Malaysia. Lonergan, J. and K. Gordon. 1986. New Dimensions 1 Teacher s Book. London: Macmillan. Mills, D. et al. 1986. Enjoy Learning English Teacher s Book 2. Paris: Armand Colin-Longman. Ministry of Education, Colombo. 1985. English Ever? Day. Book I, Year 7. Teacher s Guide. Molteno Project. 1987. Bridge Plus One Teacher s Manual. Pretoria: De Jager-Haum. Sheldon, L. E. 1988. Evaluating ELT textbooks and materials. ELT Journal 42/4: 237-246. Soars, J. and E. Soars. 1986. Headway Intermediate Teacher s Book. Oxford: Oxford University Press. The authors Both authors are lecturers in the Language Studies Department at Christ Church College of Higher Education, Canterbury. Paul Kusel has taught ESL and EFL from primary to tertiary levels, and has contributed to teachertraining seminars in many countries. His particular interests lie in pragmatics and in computer assisted language learning. Alan Cunningsworth regularly contributes to journals in the ELT field. and has written a widelyused book on evaluating ELT teaching materials. His main interests are in the areas of teacher development and training. vocabulary teaching, and the acquisition of conversational skills. Evaluating teachers guides 139